2023-11-25T04:10:40.272 INFO:root:teuthology version: 0.0.1.dev205+g2442239 2023-11-25T04:10:40.272 DEBUG:teuthology.run:Teuthology command: teuthology --description powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-low-osd-mem-target powercycle/default supported-distros/centos_latest tasks/cfuse_workunit_suites_fsx thrashosds-health} --owner scheduled_yuriw@teuthology --name yuriw-2023-11-24_21:20:59-powercycle-main-distro-default-smithi --verbose --archive /home/teuthworker/archive/yuriw-2023-11-24_21:20:59-powercycle-main-distro-default-smithi/7466144 -- /home/teuthworker/archive/yuriw-2023-11-24_21:20:59-powercycle-main-distro-default-smithi/7466144/orig.config.yaml 2023-11-25T04:10:40.305 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-25T04:10:40.390 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-11-24_21:20:59-powercycle-main-distro-default-smithi/7466144 branch: main description: powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-low-osd-mem-target powercycle/default supported-distros/centos_latest tasks/cfuse_workunit_suites_fsx thrashosds-health} email: yweinste@redhat.com first_in_suite: false job_id: '7466144' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: yuriw-2023-11-24_21:20:59-powercycle-main-distro-default-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 9.stream overrides: admin_socket: branch: main ceph: conf: mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd memory target: 1610612736 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(MDS_TRIM\) - \(MDS_SLOW_REQUEST\) - MDS_SLOW_METADATA_IO - Behind on trimming - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from sha1: ec928368edcc5fc71884a56fd9fd821b148f4e30 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: ec928368edcc5fc71884a56fd9fd821b148f4e30 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: main sha1: ec928368edcc5fc71884a56fd9fd821b148f4e30 owner: scheduled_yuriw@teuthology priority: 75 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mon.b - mon.c - mgr.x - mgr.y - mds.a - client.0 - - osd.0 - - osd.1 - - osd.2 seed: 3774 sha1: ec928368edcc5fc71884a56fd9fd821b148f4e30 sleep_before_teardown: 0 subset: 111/12000 suite: powercycle suite_branch: main suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_ec928368edcc5fc71884a56fd9fd821b148f4e30/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: ec928368edcc5fc71884a56fd9fd821b148f4e30 targets: smithi067.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNyhVVF/XtgfyRRDww3bmZ7EPginiqC6uqiypg/ZyPafcAIeLt7VaE94CXqC6yGNd5D2c5UnO4VwxGQYoizloRo= smithi076.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCC/Z4U69AnhkPFrQh+/iMSSZP8/tiy64EQsC7bzJt6p7iGC8jRA87LBnDjvkLlrBj8b8RNN7AYBEuytF8IFbws= smithi092.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIfYQcptwzwhxujcHjAVShwsFCWARoaYbV62s9iIILQuBy0lgscMzeefkNhuHZOia3Xr4VIL6eduVJA3y+mnv9A= smithi195.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLDl41IEbYPQ77UvNCnICkwMfSo5Jnv8VURPY8nB7lwu6U41Fbg5IMmek6b8dvVfg1GWiwOdYIFT+w6cMVtbB3Q= tasks: - install: extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: null - thrashosds: chance_down: 1.0 powercycle: true timeout: 600 - ceph-fuse: null - workunit: clients: all: - suites/fsx.sh timeout: 6h teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 2442239d2653456406c25ae0c71b689c8f2657b6 timestamp: 2023-11-24_21:20:59 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.232157 2023-11-25T04:10:40.390 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_ec928368edcc5fc71884a56fd9fd821b148f4e30/qa; will attempt to use it 2023-11-25T04:10:40.391 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_ec928368edcc5fc71884a56fd9fd821b148f4e30/qa/tasks 2023-11-25T04:10:40.391 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-11-25T04:10:40.393 INFO:teuthology.task.internal:Checking packages... 2023-11-25T04:10:40.416 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'ec928368edcc5fc71884a56fd9fd821b148f4e30' 2023-11-25T04:10:40.416 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-11-25T04:10:40.416 INFO:teuthology.packaging:ref: None 2023-11-25T04:10:40.416 INFO:teuthology.packaging:tag: None 2023-11-25T04:10:40.417 INFO:teuthology.packaging:branch: main 2023-11-25T04:10:40.417 INFO:teuthology.packaging:sha1: ec928368edcc5fc71884a56fd9fd821b148f4e30 2023-11-25T04:10:40.417 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=main 2023-11-25T04:10:40.763 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-7506.gec928368 2023-11-25T04:10:40.763 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-11-25T04:10:40.771 INFO:teuthology.task.internal:no buildpackages task found 2023-11-25T04:10:40.771 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-11-25T04:10:40.784 INFO:teuthology.task.internal:Saving configuration 2023-11-25T04:10:40.796 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-11-25T04:10:40.807 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-25T04:10:40.833 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi067.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-24_21:20:59-powercycle-main-distro-default-smithi/7466144', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-25 04:05:21.922153', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNyhVVF/XtgfyRRDww3bmZ7EPginiqC6uqiypg/ZyPafcAIeLt7VaE94CXqC6yGNd5D2c5UnO4VwxGQYoizloRo='} 2023-11-25T04:10:40.853 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi076.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-24_21:20:59-powercycle-main-distro-default-smithi/7466144', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-25 04:05:21.918425', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCC/Z4U69AnhkPFrQh+/iMSSZP8/tiy64EQsC7bzJt6p7iGC8jRA87LBnDjvkLlrBj8b8RNN7AYBEuytF8IFbws='} 2023-11-25T04:10:40.878 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi092.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-24_21:20:59-powercycle-main-distro-default-smithi/7466144', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-25 04:05:21.920373', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIfYQcptwzwhxujcHjAVShwsFCWARoaYbV62s9iIILQuBy0lgscMzeefkNhuHZOia3Xr4VIL6eduVJA3y+mnv9A='} 2023-11-25T04:10:40.901 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi195.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-24_21:20:59-powercycle-main-distro-default-smithi/7466144', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-25 04:05:21.923666', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLDl41IEbYPQ77UvNCnICkwMfSo5Jnv8VURPY8nB7lwu6U41Fbg5IMmek6b8dvVfg1GWiwOdYIFT+w6cMVtbB3Q='} 2023-11-25T04:10:40.901 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-11-25T04:10:40.911 INFO:teuthology.task.internal:roles: ubuntu@smithi067.front.sepia.ceph.com - ['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'mgr.y', 'mds.a', 'client.0'] 2023-11-25T04:10:40.911 INFO:teuthology.task.internal:roles: ubuntu@smithi076.front.sepia.ceph.com - ['osd.0'] 2023-11-25T04:10:40.911 INFO:teuthology.task.internal:roles: ubuntu@smithi092.front.sepia.ceph.com - ['osd.1'] 2023-11-25T04:10:40.911 INFO:teuthology.task.internal:roles: ubuntu@smithi195.front.sepia.ceph.com - ['osd.2'] 2023-11-25T04:10:40.911 INFO:teuthology.run_tasks:Running task console_log... 2023-11-25T04:10:41.051 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fbcaf88ef70>, signals=[15]) 2023-11-25T04:10:41.052 INFO:teuthology.run_tasks:Running task internal.connect... 2023-11-25T04:10:41.060 INFO:teuthology.task.internal:Opening connections... 2023-11-25T04:10:41.060 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi067.front.sepia.ceph.com 2023-11-25T04:10:41.063 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-25T04:10:41.162 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi076.front.sepia.ceph.com 2023-11-25T04:10:41.162 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-25T04:10:41.246 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi092.front.sepia.ceph.com 2023-11-25T04:10:41.247 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-25T04:10:41.331 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi195.front.sepia.ceph.com 2023-11-25T04:10:41.331 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-25T04:10:41.417 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-11-25T04:10:41.422 DEBUG:teuthology.orchestra.run.smithi067:> uname -m 2023-11-25T04:10:41.441 INFO:teuthology.orchestra.run.smithi067.stdout:x86_64 2023-11-25T04:10:41.442 DEBUG:teuthology.orchestra.run.smithi067:> cat /etc/os-release 2023-11-25T04:10:41.498 INFO:teuthology.orchestra.run.smithi067.stdout:NAME="CentOS Stream" 2023-11-25T04:10:41.498 INFO:teuthology.orchestra.run.smithi067.stdout:VERSION="9" 2023-11-25T04:10:41.498 INFO:teuthology.orchestra.run.smithi067.stdout:ID="centos" 2023-11-25T04:10:41.499 INFO:teuthology.orchestra.run.smithi067.stdout:ID_LIKE="rhel fedora" 2023-11-25T04:10:41.499 INFO:teuthology.orchestra.run.smithi067.stdout:VERSION_ID="9" 2023-11-25T04:10:41.499 INFO:teuthology.orchestra.run.smithi067.stdout:PLATFORM_ID="platform:el9" 2023-11-25T04:10:41.499 INFO:teuthology.orchestra.run.smithi067.stdout:PRETTY_NAME="CentOS Stream 9" 2023-11-25T04:10:41.499 INFO:teuthology.orchestra.run.smithi067.stdout:ANSI_COLOR="0;31" 2023-11-25T04:10:41.499 INFO:teuthology.orchestra.run.smithi067.stdout:LOGO="fedora-logo-icon" 2023-11-25T04:10:41.499 INFO:teuthology.orchestra.run.smithi067.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-11-25T04:10:41.499 INFO:teuthology.orchestra.run.smithi067.stdout:HOME_URL="https://centos.org/" 2023-11-25T04:10:41.499 INFO:teuthology.orchestra.run.smithi067.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-25T04:10:41.499 INFO:teuthology.orchestra.run.smithi067.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-11-25T04:10:41.499 INFO:teuthology.orchestra.run.smithi067.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-25T04:10:41.500 INFO:teuthology.lock.ops:Updating smithi067.front.sepia.ceph.com on lock server 2023-11-25T04:10:41.518 DEBUG:teuthology.orchestra.run.smithi076:> uname -m 2023-11-25T04:10:41.536 INFO:teuthology.orchestra.run.smithi076.stdout:x86_64 2023-11-25T04:10:41.536 DEBUG:teuthology.orchestra.run.smithi076:> cat /etc/os-release 2023-11-25T04:10:41.592 INFO:teuthology.orchestra.run.smithi076.stdout:NAME="CentOS Stream" 2023-11-25T04:10:41.593 INFO:teuthology.orchestra.run.smithi076.stdout:VERSION="9" 2023-11-25T04:10:41.593 INFO:teuthology.orchestra.run.smithi076.stdout:ID="centos" 2023-11-25T04:10:41.593 INFO:teuthology.orchestra.run.smithi076.stdout:ID_LIKE="rhel fedora" 2023-11-25T04:10:41.593 INFO:teuthology.orchestra.run.smithi076.stdout:VERSION_ID="9" 2023-11-25T04:10:41.593 INFO:teuthology.orchestra.run.smithi076.stdout:PLATFORM_ID="platform:el9" 2023-11-25T04:10:41.593 INFO:teuthology.orchestra.run.smithi076.stdout:PRETTY_NAME="CentOS Stream 9" 2023-11-25T04:10:41.593 INFO:teuthology.orchestra.run.smithi076.stdout:ANSI_COLOR="0;31" 2023-11-25T04:10:41.593 INFO:teuthology.orchestra.run.smithi076.stdout:LOGO="fedora-logo-icon" 2023-11-25T04:10:41.593 INFO:teuthology.orchestra.run.smithi076.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-11-25T04:10:41.593 INFO:teuthology.orchestra.run.smithi076.stdout:HOME_URL="https://centos.org/" 2023-11-25T04:10:41.594 INFO:teuthology.orchestra.run.smithi076.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-25T04:10:41.594 INFO:teuthology.orchestra.run.smithi076.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-11-25T04:10:41.594 INFO:teuthology.orchestra.run.smithi076.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-25T04:10:41.594 INFO:teuthology.lock.ops:Updating smithi076.front.sepia.ceph.com on lock server 2023-11-25T04:10:41.635 DEBUG:teuthology.orchestra.run.smithi092:> uname -m 2023-11-25T04:10:41.652 INFO:teuthology.orchestra.run.smithi092.stdout:x86_64 2023-11-25T04:10:41.652 DEBUG:teuthology.orchestra.run.smithi092:> cat /etc/os-release 2023-11-25T04:10:41.709 INFO:teuthology.orchestra.run.smithi092.stdout:NAME="CentOS Stream" 2023-11-25T04:10:41.709 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION="9" 2023-11-25T04:10:41.709 INFO:teuthology.orchestra.run.smithi092.stdout:ID="centos" 2023-11-25T04:10:41.709 INFO:teuthology.orchestra.run.smithi092.stdout:ID_LIKE="rhel fedora" 2023-11-25T04:10:41.709 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION_ID="9" 2023-11-25T04:10:41.709 INFO:teuthology.orchestra.run.smithi092.stdout:PLATFORM_ID="platform:el9" 2023-11-25T04:10:41.709 INFO:teuthology.orchestra.run.smithi092.stdout:PRETTY_NAME="CentOS Stream 9" 2023-11-25T04:10:41.710 INFO:teuthology.orchestra.run.smithi092.stdout:ANSI_COLOR="0;31" 2023-11-25T04:10:41.710 INFO:teuthology.orchestra.run.smithi092.stdout:LOGO="fedora-logo-icon" 2023-11-25T04:10:41.710 INFO:teuthology.orchestra.run.smithi092.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-11-25T04:10:41.710 INFO:teuthology.orchestra.run.smithi092.stdout:HOME_URL="https://centos.org/" 2023-11-25T04:10:41.710 INFO:teuthology.orchestra.run.smithi092.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-25T04:10:41.710 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-11-25T04:10:41.710 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-25T04:10:41.710 INFO:teuthology.lock.ops:Updating smithi092.front.sepia.ceph.com on lock server 2023-11-25T04:10:41.735 DEBUG:teuthology.orchestra.run.smithi195:> uname -m 2023-11-25T04:10:41.752 INFO:teuthology.orchestra.run.smithi195.stdout:x86_64 2023-11-25T04:10:41.753 DEBUG:teuthology.orchestra.run.smithi195:> cat /etc/os-release 2023-11-25T04:10:41.810 INFO:teuthology.orchestra.run.smithi195.stdout:NAME="CentOS Stream" 2023-11-25T04:10:41.810 INFO:teuthology.orchestra.run.smithi195.stdout:VERSION="9" 2023-11-25T04:10:41.811 INFO:teuthology.orchestra.run.smithi195.stdout:ID="centos" 2023-11-25T04:10:41.811 INFO:teuthology.orchestra.run.smithi195.stdout:ID_LIKE="rhel fedora" 2023-11-25T04:10:41.811 INFO:teuthology.orchestra.run.smithi195.stdout:VERSION_ID="9" 2023-11-25T04:10:41.811 INFO:teuthology.orchestra.run.smithi195.stdout:PLATFORM_ID="platform:el9" 2023-11-25T04:10:41.811 INFO:teuthology.orchestra.run.smithi195.stdout:PRETTY_NAME="CentOS Stream 9" 2023-11-25T04:10:41.811 INFO:teuthology.orchestra.run.smithi195.stdout:ANSI_COLOR="0;31" 2023-11-25T04:10:41.811 INFO:teuthology.orchestra.run.smithi195.stdout:LOGO="fedora-logo-icon" 2023-11-25T04:10:41.811 INFO:teuthology.orchestra.run.smithi195.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-11-25T04:10:41.811 INFO:teuthology.orchestra.run.smithi195.stdout:HOME_URL="https://centos.org/" 2023-11-25T04:10:41.811 INFO:teuthology.orchestra.run.smithi195.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-25T04:10:41.812 INFO:teuthology.orchestra.run.smithi195.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-11-25T04:10:41.812 INFO:teuthology.orchestra.run.smithi195.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-25T04:10:41.812 INFO:teuthology.lock.ops:Updating smithi195.front.sepia.ceph.com on lock server 2023-11-25T04:10:41.836 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-11-25T04:10:41.846 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-11-25T04:10:41.853 INFO:teuthology.task.internal:Checking for old test directory... 2023-11-25T04:10:41.854 DEBUG:teuthology.orchestra.run.smithi067:> test '!' -e /home/ubuntu/cephtest 2023-11-25T04:10:41.856 DEBUG:teuthology.orchestra.run.smithi076:> test '!' -e /home/ubuntu/cephtest 2023-11-25T04:10:41.859 DEBUG:teuthology.orchestra.run.smithi092:> test '!' -e /home/ubuntu/cephtest 2023-11-25T04:10:41.861 DEBUG:teuthology.orchestra.run.smithi195:> test '!' -e /home/ubuntu/cephtest 2023-11-25T04:10:41.878 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-11-25T04:10:41.889 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-11-25T04:10:41.889 DEBUG:teuthology.orchestra.run.smithi067:> test -z $(ls -A /var/lib/ceph) 2023-11-25T04:10:41.914 DEBUG:teuthology.orchestra.run.smithi076:> test -z $(ls -A /var/lib/ceph) 2023-11-25T04:10:41.917 DEBUG:teuthology.orchestra.run.smithi092:> test -z $(ls -A /var/lib/ceph) 2023-11-25T04:10:41.919 DEBUG:teuthology.orchestra.run.smithi195:> test -z $(ls -A /var/lib/ceph) 2023-11-25T04:10:42.022 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-11-25T04:10:42.122 INFO:teuthology.run_tasks:Running task kernel... 2023-11-25T04:10:42.139 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-11-25T04:10:42.139 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'osd.0': {'kdb': True, 'sha1': 'distro'}, 'osd.1': {'kdb': True, 'sha1': 'distro'}, 'osd.2': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-11-25T04:10:42.139 DEBUG:teuthology.orchestra.run.smithi067:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-25T04:10:42.140 DEBUG:teuthology.orchestra.run.smithi076:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-25T04:10:42.140 DEBUG:teuthology.orchestra.run.smithi092:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-25T04:10:42.140 DEBUG:teuthology.orchestra.run.smithi195:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-25T04:10:42.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-25T04:10:42.157 DEBUG:teuthology.orchestra.run.smithi092:> uname -r 2023-11-25T04:10:42.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-25T04:10:42.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-25T04:10:42.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-25T04:10:42.161 DEBUG:teuthology.orchestra.run.smithi067:> uname -r 2023-11-25T04:10:42.162 DEBUG:teuthology.orchestra.run.smithi195:> uname -r 2023-11-25T04:10:42.162 DEBUG:teuthology.orchestra.run.smithi076:> uname -r 2023-11-25T04:10:42.214 INFO:teuthology.orchestra.run.smithi092.stdout:5.14.0-361.el9.x86_64 2023-11-25T04:10:42.214 INFO:teuthology.task.kernel:Running kernel on smithi092: 5.14.0-361.el9.x86_64 2023-11-25T04:10:42.214 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum install -y kernel 2023-11-25T04:10:42.217 INFO:teuthology.orchestra.run.smithi076.stdout:5.14.0-361.el9.x86_64 2023-11-25T04:10:42.217 INFO:teuthology.task.kernel:Running kernel on smithi076: 5.14.0-361.el9.x86_64 2023-11-25T04:10:42.218 DEBUG:teuthology.orchestra.run.smithi076:> sudo yum install -y kernel 2023-11-25T04:10:42.219 INFO:teuthology.orchestra.run.smithi195.stdout:5.14.0-361.el9.x86_64 2023-11-25T04:10:42.219 INFO:teuthology.task.kernel:Running kernel on smithi195: 5.14.0-361.el9.x86_64 2023-11-25T04:10:42.219 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum install -y kernel 2023-11-25T04:10:42.221 INFO:teuthology.orchestra.run.smithi067.stdout:5.14.0-361.el9.x86_64 2023-11-25T04:10:42.221 INFO:teuthology.task.kernel:Running kernel on smithi067: 5.14.0-361.el9.x86_64 2023-11-25T04:10:42.222 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum install -y kernel 2023-11-25T04:10:44.308 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS-9 - AppStream 13 MB/s | 6.4 MB 00:00 2023-11-25T04:10:44.350 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS-9 - AppStream 12 MB/s | 6.4 MB 00:00 2023-11-25T04:10:44.354 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS-9 - AppStream 13 MB/s | 6.4 MB 00:00 2023-11-25T04:10:44.448 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS-9 - AppStream 12 MB/s | 6.4 MB 00:00 2023-11-25T04:10:45.617 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS-9 - BaseOS 7.4 MB/s | 2.0 MB 00:00 2023-11-25T04:10:45.704 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS-9 - BaseOS 6.7 MB/s | 2.0 MB 00:00 2023-11-25T04:10:45.733 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS-9 - BaseOS 6.9 MB/s | 2.0 MB 00:00 2023-11-25T04:10:46.007 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS-9 - BaseOS 4.0 MB/s | 2.0 MB 00:00 2023-11-25T04:10:47.047 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 7.9 MB 00:00 2023-11-25T04:10:47.153 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS Stream 9 - BaseOS 7.0 MB/s | 7.9 MB 00:01 2023-11-25T04:10:47.816 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - BaseOS 4.6 MB/s | 7.9 MB 00:01 2023-11-25T04:10:49.124 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - BaseOS 2.4 MB/s | 7.9 MB 00:03 2023-11-25T04:10:49.734 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - AppStream 24 MB/s | 18 MB 00:00 2023-11-25T04:10:49.887 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS Stream 9 - AppStream 11 MB/s | 18 MB 00:01 2023-11-25T04:10:51.076 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - AppStream 20 MB/s | 18 MB 00:00 2023-11-25T04:10:53.738 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS Stream 9 - CRB 14 MB/s | 5.8 MB 00:00 2023-11-25T04:10:53.884 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - CRB 9.5 MB/s | 5.8 MB 00:00 2023-11-25T04:10:55.022 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS Stream 9 - Extras packages 89 kB/s | 15 kB 00:00 2023-11-25T04:10:55.277 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - Extras packages 64 kB/s | 15 kB 00:00 2023-11-25T04:10:56.102 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - CRB 4.0 MB/s | 5.8 MB 00:01 2023-11-25T04:10:56.928 INFO:teuthology.orchestra.run.smithi067.stdout:Extra Packages for Enterprise Linux 13 MB/s | 20 MB 00:01 2023-11-25T04:10:57.936 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - Extras packages 21 kB/s | 15 kB 00:00 2023-11-25T04:10:58.118 INFO:teuthology.orchestra.run.smithi195.stdout:Extra Packages for Enterprise Linux 6.6 MB/s | 20 MB 00:02 2023-11-25T04:10:58.124 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - AppStream 1.8 MB/s | 18 MB 00:09 2023-11-25T04:11:00.288 INFO:teuthology.orchestra.run.smithi076.stdout:Extra Packages for Enterprise Linux 8.9 MB/s | 20 MB 00:02 2023-11-25T04:11:01.911 INFO:teuthology.orchestra.run.smithi067.stdout:lab-extras 40 kB/s | 1.7 kB 00:00 2023-11-25T04:11:02.765 INFO:teuthology.orchestra.run.smithi195.stdout:lab-extras 38 kB/s | 1.7 kB 00:00 2023-11-25T04:11:02.913 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - CRB 4.4 MB/s | 5.8 MB 00:01 2023-11-25T04:11:04.360 INFO:teuthology.orchestra.run.smithi067.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2023-11-25T04:11:04.360 INFO:teuthology.orchestra.run.smithi067.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2023-11-25T04:11:04.448 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2023-11-25T04:11:04.457 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2023-11-25T04:11:04.457 INFO:teuthology.orchestra.run.smithi067.stdout: Package Architecture Version Repository Size 2023-11-25T04:11:04.457 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2023-11-25T04:11:04.457 INFO:teuthology.orchestra.run.smithi067.stdout:Installing: 2023-11-25T04:11:04.457 INFO:teuthology.orchestra.run.smithi067.stdout: kernel x86_64 5.14.0-386.el9 baseos 5.5 M 2023-11-25T04:11:04.458 INFO:teuthology.orchestra.run.smithi067.stdout:Installing dependencies: 2023-11-25T04:11:04.458 INFO:teuthology.orchestra.run.smithi067.stdout: kernel-core x86_64 5.14.0-386.el9 baseos 20 M 2023-11-25T04:11:04.458 INFO:teuthology.orchestra.run.smithi067.stdout: kernel-modules x86_64 5.14.0-386.el9 baseos 38 M 2023-11-25T04:11:04.458 INFO:teuthology.orchestra.run.smithi067.stdout: kernel-modules-core x86_64 5.14.0-386.el9 baseos 32 M 2023-11-25T04:11:04.458 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:11:04.458 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction Summary 2023-11-25T04:11:04.458 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2023-11-25T04:11:04.458 INFO:teuthology.orchestra.run.smithi067.stdout:Install 4 Packages 2023-11-25T04:11:04.458 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:11:04.459 INFO:teuthology.orchestra.run.smithi067.stdout:Total download size: 95 M 2023-11-25T04:11:04.459 INFO:teuthology.orchestra.run.smithi067.stdout:Installed size: 121 M 2023-11-25T04:11:04.459 INFO:teuthology.orchestra.run.smithi067.stdout:Downloading Packages: 2023-11-25T04:11:04.581 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - Extras packages 27 kB/s | 15 kB 00:00 2023-11-25T04:11:05.056 INFO:teuthology.orchestra.run.smithi195.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2023-11-25T04:11:05.056 INFO:teuthology.orchestra.run.smithi195.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2023-11-25T04:11:05.113 INFO:teuthology.orchestra.run.smithi076.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2023-11-25T04:11:05.151 INFO:teuthology.orchestra.run.smithi195.stdout:Dependencies resolved. 2023-11-25T04:11:05.167 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2023-11-25T04:11:05.167 INFO:teuthology.orchestra.run.smithi195.stdout: Package Architecture Version Repository Size 2023-11-25T04:11:05.167 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2023-11-25T04:11:05.168 INFO:teuthology.orchestra.run.smithi195.stdout:Installing: 2023-11-25T04:11:05.168 INFO:teuthology.orchestra.run.smithi195.stdout: kernel x86_64 5.14.0-386.el9 baseos 5.5 M 2023-11-25T04:11:05.168 INFO:teuthology.orchestra.run.smithi195.stdout:Installing dependencies: 2023-11-25T04:11:05.168 INFO:teuthology.orchestra.run.smithi195.stdout: kernel-core x86_64 5.14.0-386.el9 baseos 20 M 2023-11-25T04:11:05.168 INFO:teuthology.orchestra.run.smithi195.stdout: kernel-modules x86_64 5.14.0-386.el9 baseos 38 M 2023-11-25T04:11:05.168 INFO:teuthology.orchestra.run.smithi195.stdout: kernel-modules-core x86_64 5.14.0-386.el9 baseos 32 M 2023-11-25T04:11:05.168 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-25T04:11:05.168 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction Summary 2023-11-25T04:11:05.168 INFO:teuthology.orchestra.run.smithi195.stdout:================================================================================ 2023-11-25T04:11:05.168 INFO:teuthology.orchestra.run.smithi195.stdout:Install 4 Packages 2023-11-25T04:11:05.168 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-25T04:11:05.169 INFO:teuthology.orchestra.run.smithi195.stdout:Total download size: 95 M 2023-11-25T04:11:05.169 INFO:teuthology.orchestra.run.smithi195.stdout:Installed size: 121 M 2023-11-25T04:11:05.169 INFO:teuthology.orchestra.run.smithi195.stdout:Downloading Packages: 2023-11-25T04:11:06.143 INFO:teuthology.orchestra.run.smithi092.stdout:Extra Packages for Enterprise Linux 14 MB/s | 20 MB 00:01 2023-11-25T04:11:06.208 INFO:teuthology.orchestra.run.smithi195.stdout:(1/4): kernel-5.14.0-386.el9.x86_64.rpm 5.5 MB/s | 5.5 MB 00:00 2023-11-25T04:11:06.992 INFO:teuthology.orchestra.run.smithi195.stdout:(2/4): kernel-core-5.14.0-386.el9.x86_64.rpm 11 MB/s | 20 MB 00:01 2023-11-25T04:11:07.429 INFO:teuthology.orchestra.run.smithi067.stdout:(1/4): kernel-5.14.0-386.el9.x86_64.rpm 1.9 MB/s | 5.5 MB 00:02 2023-11-25T04:11:07.436 INFO:teuthology.orchestra.run.smithi076.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2023-11-25T04:11:07.436 INFO:teuthology.orchestra.run.smithi076.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2023-11-25T04:11:07.529 INFO:teuthology.orchestra.run.smithi076.stdout:Dependencies resolved. 2023-11-25T04:11:07.535 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2023-11-25T04:11:07.535 INFO:teuthology.orchestra.run.smithi076.stdout: Package Architecture Version Repository Size 2023-11-25T04:11:07.535 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2023-11-25T04:11:07.536 INFO:teuthology.orchestra.run.smithi076.stdout:Installing: 2023-11-25T04:11:07.536 INFO:teuthology.orchestra.run.smithi076.stdout: kernel x86_64 5.14.0-386.el9 baseos 5.5 M 2023-11-25T04:11:07.536 INFO:teuthology.orchestra.run.smithi076.stdout:Installing dependencies: 2023-11-25T04:11:07.536 INFO:teuthology.orchestra.run.smithi076.stdout: kernel-core x86_64 5.14.0-386.el9 baseos 20 M 2023-11-25T04:11:07.536 INFO:teuthology.orchestra.run.smithi076.stdout: kernel-modules x86_64 5.14.0-386.el9 baseos 38 M 2023-11-25T04:11:07.536 INFO:teuthology.orchestra.run.smithi076.stdout: kernel-modules-core x86_64 5.14.0-386.el9 baseos 32 M 2023-11-25T04:11:07.536 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-25T04:11:07.536 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction Summary 2023-11-25T04:11:07.536 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2023-11-25T04:11:07.536 INFO:teuthology.orchestra.run.smithi076.stdout:Install 4 Packages 2023-11-25T04:11:07.536 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-25T04:11:07.536 INFO:teuthology.orchestra.run.smithi076.stdout:Total download size: 95 M 2023-11-25T04:11:07.537 INFO:teuthology.orchestra.run.smithi076.stdout:Installed size: 121 M 2023-11-25T04:11:07.537 INFO:teuthology.orchestra.run.smithi076.stdout:Downloading Packages: 2023-11-25T04:11:07.766 INFO:teuthology.orchestra.run.smithi195.stdout:(3/4): kernel-modules-core-5.14.0-386.el9.x86_6 21 MB/s | 32 MB 00:01 2023-11-25T04:11:08.081 INFO:teuthology.orchestra.run.smithi076.stdout:(1/4): kernel-5.14.0-386.el9.x86_64.rpm 12 MB/s | 5.5 MB 00:00 2023-11-25T04:11:08.100 INFO:teuthology.orchestra.run.smithi195.stdout:(4/4): kernel-modules-5.14.0-386.el9.x86_64.rpm 13 MB/s | 38 MB 00:02 2023-11-25T04:11:08.100 INFO:teuthology.orchestra.run.smithi195.stdout:-------------------------------------------------------------------------------- 2023-11-25T04:11:08.100 INFO:teuthology.orchestra.run.smithi195.stdout:Total 33 MB/s | 95 MB 00:02 2023-11-25T04:11:08.957 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction check 2023-11-25T04:11:09.198 INFO:teuthology.orchestra.run.smithi076.stdout:(2/4): kernel-core-5.14.0-386.el9.x86_64.rpm 13 MB/s | 20 MB 00:01 2023-11-25T04:11:09.304 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction check succeeded. 2023-11-25T04:11:09.304 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction test 2023-11-25T04:11:09.615 INFO:teuthology.orchestra.run.smithi076.stdout:(3/4): kernel-modules-core-5.14.0-386.el9.x86_6 21 MB/s | 32 MB 00:01 2023-11-25T04:11:09.946 INFO:teuthology.orchestra.run.smithi067.stdout:(2/4): kernel-modules-5.14.0-386.el9.x86_64.rpm 7.0 MB/s | 38 MB 00:05 2023-11-25T04:11:10.282 INFO:teuthology.orchestra.run.smithi076.stdout:(4/4): kernel-modules-5.14.0-386.el9.x86_64.rpm 14 MB/s | 38 MB 00:02 2023-11-25T04:11:10.282 INFO:teuthology.orchestra.run.smithi076.stdout:-------------------------------------------------------------------------------- 2023-11-25T04:11:10.282 INFO:teuthology.orchestra.run.smithi076.stdout:Total 35 MB/s | 95 MB 00:02 2023-11-25T04:11:10.604 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction test succeeded. 2023-11-25T04:11:10.604 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction 2023-11-25T04:11:10.871 INFO:teuthology.orchestra.run.smithi067.stdout:(3/4): kernel-core-5.14.0-386.el9.x86_64.rpm 3.2 MB/s | 20 MB 00:06 2023-11-25T04:11:10.932 INFO:teuthology.orchestra.run.smithi092.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2023-11-25T04:11:11.136 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction check 2023-11-25T04:11:11.238 INFO:teuthology.orchestra.run.smithi067.stdout:(4/4): kernel-modules-core-5.14.0-386.el9.x86_6 8.5 MB/s | 32 MB 00:03 2023-11-25T04:11:11.239 INFO:teuthology.orchestra.run.smithi067.stdout:-------------------------------------------------------------------------------- 2023-11-25T04:11:11.239 INFO:teuthology.orchestra.run.smithi067.stdout:Total 14 MB/s | 95 MB 00:06 2023-11-25T04:11:11.521 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction check succeeded. 2023-11-25T04:11:11.522 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction test 2023-11-25T04:11:12.121 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction check 2023-11-25T04:11:12.531 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction check succeeded. 2023-11-25T04:11:12.531 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction test 2023-11-25T04:11:12.816 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction test succeeded. 2023-11-25T04:11:12.816 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction 2023-11-25T04:11:13.223 INFO:teuthology.orchestra.run.smithi195.stdout: Preparing : 1/1 2023-11-25T04:11:13.263 INFO:teuthology.orchestra.run.smithi092.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2023-11-25T04:11:13.263 INFO:teuthology.orchestra.run.smithi092.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2023-11-25T04:11:13.355 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2023-11-25T04:11:13.359 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2023-11-25T04:11:13.359 INFO:teuthology.orchestra.run.smithi092.stdout: Package Architecture Version Repository Size 2023-11-25T04:11:13.359 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2023-11-25T04:11:13.359 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2023-11-25T04:11:13.359 INFO:teuthology.orchestra.run.smithi092.stdout: kernel x86_64 5.14.0-386.el9 baseos 5.5 M 2023-11-25T04:11:13.359 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2023-11-25T04:11:13.359 INFO:teuthology.orchestra.run.smithi092.stdout: kernel-core x86_64 5.14.0-386.el9 baseos 20 M 2023-11-25T04:11:13.359 INFO:teuthology.orchestra.run.smithi092.stdout: kernel-modules x86_64 5.14.0-386.el9 baseos 38 M 2023-11-25T04:11:13.359 INFO:teuthology.orchestra.run.smithi092.stdout: kernel-modules-core x86_64 5.14.0-386.el9 baseos 32 M 2023-11-25T04:11:13.359 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-25T04:11:13.360 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2023-11-25T04:11:13.360 INFO:teuthology.orchestra.run.smithi092.stdout:================================================================================ 2023-11-25T04:11:13.360 INFO:teuthology.orchestra.run.smithi092.stdout:Install 4 Packages 2023-11-25T04:11:13.360 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-25T04:11:13.360 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 95 M 2023-11-25T04:11:13.360 INFO:teuthology.orchestra.run.smithi092.stdout:Installed size: 121 M 2023-11-25T04:11:13.361 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2023-11-25T04:11:13.863 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : kernel-modules-core-5.14.0-386.el9.x86_64 1/4 2023-11-25T04:11:13.874 INFO:teuthology.orchestra.run.smithi092.stdout:(1/4): kernel-5.14.0-386.el9.x86_64.rpm 14 MB/s | 5.5 MB 00:00 2023-11-25T04:11:13.880 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction test succeeded. 2023-11-25T04:11:13.881 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction 2023-11-25T04:11:13.992 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : kernel-core-5.14.0-386.el9.x86_64 2/4 2023-11-25T04:11:14.558 INFO:teuthology.orchestra.run.smithi092.stdout:(2/4): kernel-core-5.14.0-386.el9.x86_64.rpm 19 MB/s | 20 MB 00:01 2023-11-25T04:11:15.083 INFO:teuthology.orchestra.run.smithi092.stdout:(3/4): kernel-modules-5.14.0-386.el9.x86_64.rpm 24 MB/s | 38 MB 00:01 2023-11-25T04:11:15.456 INFO:teuthology.orchestra.run.smithi076.stdout: Preparing : 1/1 2023-11-25T04:11:15.475 INFO:teuthology.orchestra.run.smithi092.stdout:(4/4): kernel-modules-core-5.14.0-386.el9.x86_6 20 MB/s | 32 MB 00:01 2023-11-25T04:11:15.475 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2023-11-25T04:11:15.475 INFO:teuthology.orchestra.run.smithi092.stdout:Total 45 MB/s | 95 MB 00:02 2023-11-25T04:11:16.083 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : kernel-modules-core-5.14.0-386.el9.x86_64 1/4 2023-11-25T04:11:16.212 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : kernel-core-5.14.0-386.el9.x86_64 2/4 2023-11-25T04:11:16.325 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2023-11-25T04:11:16.644 INFO:teuthology.orchestra.run.smithi067.stdout: Preparing : 1/1 2023-11-25T04:11:16.683 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2023-11-25T04:11:16.684 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2023-11-25T04:11:16.744 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: kernel-core-5.14.0-386.el9.x86_64 2/4 2023-11-25T04:11:16.919 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : kernel-modules-5.14.0-386.el9.x86_64 3/4 2023-11-25T04:11:17.301 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : kernel-modules-core-5.14.0-386.el9.x86_64 1/4 2023-11-25T04:11:17.430 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : kernel-core-5.14.0-386.el9.x86_64 2/4 2023-11-25T04:11:18.027 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2023-11-25T04:11:18.028 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2023-11-25T04:11:18.978 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: kernel-core-5.14.0-386.el9.x86_64 2/4 2023-11-25T04:11:19.144 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : kernel-modules-5.14.0-386.el9.x86_64 3/4 2023-11-25T04:11:20.320 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: kernel-core-5.14.0-386.el9.x86_64 2/4 2023-11-25T04:11:20.489 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : kernel-modules-5.14.0-386.el9.x86_64 3/4 2023-11-25T04:11:20.969 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2023-11-25T04:11:21.508 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: kernel-modules-5.14.0-386.el9.x86_64 3/4 2023-11-25T04:11:21.610 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : kernel-modules-core-5.14.0-386.el9.x86_64 1/4 2023-11-25T04:11:21.636 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : kernel-5.14.0-386.el9.x86_64 4/4 2023-11-25T04:11:21.741 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : kernel-core-5.14.0-386.el9.x86_64 2/4 2023-11-25T04:11:23.773 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: kernel-modules-5.14.0-386.el9.x86_64 3/4 2023-11-25T04:11:23.885 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : kernel-5.14.0-386.el9.x86_64 4/4 2023-11-25T04:11:24.482 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: kernel-core-5.14.0-386.el9.x86_64 2/4 2023-11-25T04:11:24.677 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : kernel-modules-5.14.0-386.el9.x86_64 3/4 2023-11-25T04:11:25.445 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: kernel-modules-5.14.0-386.el9.x86_64 3/4 2023-11-25T04:11:25.564 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : kernel-5.14.0-386.el9.x86_64 4/4 2023-11-25T04:11:26.224 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: kernel-modules-core-5.14.0-386.el9.x86_64 4/4 2023-11-25T04:11:28.626 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: kernel-modules-core-5.14.0-386.el9.x86_64 4/4 2023-11-25T04:11:29.291 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: kernel-modules-5.14.0-386.el9.x86_64 3/4 2023-11-25T04:11:29.542 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : kernel-5.14.0-386.el9.x86_64 4/4 2023-11-25T04:11:30.542 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: kernel-modules-core-5.14.0-386.el9.x86_64 4/4 2023-11-25T04:11:34.135 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: kernel-modules-core-5.14.0-386.el9.x86_64 4/4 2023-11-25T04:11:54.905 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: kernel-core-5.14.0-386.el9.x86_64 4/4 2023-11-25T04:11:54.949 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: kernel-modules-5.14.0-386.el9.x86_64 4/4 2023-11-25T04:11:55.426 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: kernel-5.14.0-386.el9.x86_64 4/4 2023-11-25T04:11:55.427 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : kernel-5.14.0-386.el9.x86_64 1/4 2023-11-25T04:11:55.427 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : kernel-core-5.14.0-386.el9.x86_64 2/4 2023-11-25T04:11:55.427 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : kernel-modules-5.14.0-386.el9.x86_64 3/4 2023-11-25T04:11:55.830 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : kernel-modules-core-5.14.0-386.el9.x86_64 4/4 2023-11-25T04:11:55.831 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-25T04:11:55.831 INFO:teuthology.orchestra.run.smithi195.stdout:Installed: 2023-11-25T04:11:55.831 INFO:teuthology.orchestra.run.smithi195.stdout: kernel-5.14.0-386.el9.x86_64 2023-11-25T04:11:55.831 INFO:teuthology.orchestra.run.smithi195.stdout: kernel-core-5.14.0-386.el9.x86_64 2023-11-25T04:11:55.831 INFO:teuthology.orchestra.run.smithi195.stdout: kernel-modules-5.14.0-386.el9.x86_64 2023-11-25T04:11:55.831 INFO:teuthology.orchestra.run.smithi195.stdout: kernel-modules-core-5.14.0-386.el9.x86_64 2023-11-25T04:11:55.831 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-25T04:11:55.831 INFO:teuthology.orchestra.run.smithi195.stdout:Complete! 2023-11-25T04:11:56.297 DEBUG:teuthology.orchestra.run.smithi195:> rpm -q kernel | sort -rV | head -n 1 2023-11-25T04:11:56.437 INFO:teuthology.orchestra.run.smithi195.stdout:kernel-5.14.0-386.el9.x86_64 2023-11-25T04:11:56.437 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-386.el9.x86_64 2023-11-25T04:11:56.437 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-386.el9.x86_64 2023-11-25T04:11:56.438 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-11-25T04:11:56.438 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-11-25T04:11:56.438 INFO:teuthology.task.kernel:Installing distro kernel on osd.2... 2023-11-25T04:11:56.438 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi195.front.sepia.ceph.com, path=None, version=distro) 2023-11-25T04:11:56.438 DEBUG:teuthology.orchestra.run.smithi195:> rpm -q kernel | sort -rV | head -n 1 2023-11-25T04:11:56.565 INFO:teuthology.orchestra.run.smithi195.stdout:kernel-5.14.0-386.el9.x86_64 2023-11-25T04:11:56.568 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-386.el9.x86_64 2023-11-25T04:11:56.570 DEBUG:teuthology.orchestra.run.smithi195:> sudo rpm -qi grub2-tools 2023-11-25T04:11:56.609 INFO:teuthology.orchestra.run.smithi195.stdout:Name : grub2-tools 2023-11-25T04:11:56.609 INFO:teuthology.orchestra.run.smithi195.stdout:Epoch : 1 2023-11-25T04:11:56.609 INFO:teuthology.orchestra.run.smithi195.stdout:Version : 2.06 2023-11-25T04:11:56.609 INFO:teuthology.orchestra.run.smithi195.stdout:Release : 68.el9 2023-11-25T04:11:56.610 INFO:teuthology.orchestra.run.smithi195.stdout:Architecture: x86_64 2023-11-25T04:11:56.610 INFO:teuthology.orchestra.run.smithi195.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2023-11-25T04:11:56.610 INFO:teuthology.orchestra.run.smithi195.stdout:Group : Unspecified 2023-11-25T04:11:56.610 INFO:teuthology.orchestra.run.smithi195.stdout:Size : 8264254 2023-11-25T04:11:56.610 INFO:teuthology.orchestra.run.smithi195.stdout:License : GPLv3+ 2023-11-25T04:11:56.610 INFO:teuthology.orchestra.run.smithi195.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2023-11-25T04:11:56.610 INFO:teuthology.orchestra.run.smithi195.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2023-11-25T04:11:56.610 INFO:teuthology.orchestra.run.smithi195.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2023-11-25T04:11:56.610 INFO:teuthology.orchestra.run.smithi195.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-11-25T04:11:56.610 INFO:teuthology.orchestra.run.smithi195.stdout:Packager : builder@centos.org 2023-11-25T04:11:56.610 INFO:teuthology.orchestra.run.smithi195.stdout:Vendor : CentOS 2023-11-25T04:11:56.610 INFO:teuthology.orchestra.run.smithi195.stdout:URL : http://www.gnu.org/software/grub/ 2023-11-25T04:11:56.611 INFO:teuthology.orchestra.run.smithi195.stdout:Summary : Support tools for GRUB. 2023-11-25T04:11:56.611 INFO:teuthology.orchestra.run.smithi195.stdout:Description : 2023-11-25T04:11:56.611 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-25T04:11:56.611 INFO:teuthology.orchestra.run.smithi195.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-11-25T04:11:56.611 INFO:teuthology.orchestra.run.smithi195.stdout:customizable bootloader with modular architecture. It supports a rich 2023-11-25T04:11:56.611 INFO:teuthology.orchestra.run.smithi195.stdout:variety of kernel formats, file systems, computer architectures and 2023-11-25T04:11:56.611 INFO:teuthology.orchestra.run.smithi195.stdout:hardware devices. 2023-11-25T04:11:56.611 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-25T04:11:56.611 INFO:teuthology.orchestra.run.smithi195.stdout:This subpackage provides tools for support of all platforms. 2023-11-25T04:11:56.612 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-11-25T04:11:56.612 INFO:teuthology.task.kernel:Updating grub on smithi195 to boot 5.14.0-386.el9.x86_64 2023-11-25T04:11:56.612 DEBUG:teuthology.orchestra.run.smithi195:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-11-25T04:11:57.246 INFO:teuthology.orchestra.run.smithi195.stderr:Generating grub configuration file ... 2023-11-25T04:11:57.523 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: kernel-core-5.14.0-386.el9.x86_64 4/4 2023-11-25T04:11:57.568 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: kernel-modules-5.14.0-386.el9.x86_64 4/4 2023-11-25T04:11:58.020 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: kernel-5.14.0-386.el9.x86_64 4/4 2023-11-25T04:11:58.020 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : kernel-5.14.0-386.el9.x86_64 1/4 2023-11-25T04:11:58.020 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : kernel-core-5.14.0-386.el9.x86_64 2/4 2023-11-25T04:11:58.020 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : kernel-modules-5.14.0-386.el9.x86_64 3/4 2023-11-25T04:11:58.398 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : kernel-modules-core-5.14.0-386.el9.x86_64 4/4 2023-11-25T04:11:58.398 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-25T04:11:58.399 INFO:teuthology.orchestra.run.smithi076.stdout:Installed: 2023-11-25T04:11:58.399 INFO:teuthology.orchestra.run.smithi076.stdout: kernel-5.14.0-386.el9.x86_64 2023-11-25T04:11:58.399 INFO:teuthology.orchestra.run.smithi076.stdout: kernel-core-5.14.0-386.el9.x86_64 2023-11-25T04:11:58.399 INFO:teuthology.orchestra.run.smithi076.stdout: kernel-modules-5.14.0-386.el9.x86_64 2023-11-25T04:11:58.399 INFO:teuthology.orchestra.run.smithi076.stdout: kernel-modules-core-5.14.0-386.el9.x86_64 2023-11-25T04:11:58.399 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-25T04:11:58.399 INFO:teuthology.orchestra.run.smithi076.stdout:Complete! 2023-11-25T04:11:58.631 INFO:teuthology.orchestra.run.smithi195.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-11-25T04:11:58.651 INFO:teuthology.orchestra.run.smithi195.stderr:done 2023-11-25T04:11:58.653 DEBUG:teuthology.orchestra.run.smithi195:> mktemp 2023-11-25T04:11:58.668 INFO:teuthology.orchestra.run.smithi195.stdout:/tmp/tmp.KzbUJ6CdZW 2023-11-25T04:11:58.668 DEBUG:teuthology.orchestra.run.smithi195:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.KzbUJ6CdZW 2023-11-25T04:11:58.748 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod 0666 /tmp/tmp.KzbUJ6CdZW 2023-11-25T04:11:58.897 DEBUG:teuthology.orchestra.remote:smithi195:/tmp/tmp.KzbUJ6CdZW is 6KB 2023-11-25T04:11:58.913 DEBUG:teuthology.orchestra.run.smithi076:> rpm -q kernel | sort -rV | head -n 1 2023-11-25T04:11:58.945 DEBUG:teuthology.orchestra.run.smithi195:> rm -fr /tmp/tmp.KzbUJ6CdZW 2023-11-25T04:11:58.959 DEBUG:teuthology.orchestra.run.smithi195:> sudo /bin/ls /boot/loader/entries || true 2023-11-25T04:11:59.024 INFO:teuthology.orchestra.run.smithi195.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2023-11-25T04:11:59.024 INFO:teuthology.orchestra.run.smithi195.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2023-11-25T04:11:59.024 INFO:teuthology.orchestra.run.smithi195.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2023-11-25T04:11:59.024 INFO:teuthology.orchestra.run.smithi195.stdout:15949e3c88704300882482eddfd7baae-5.14.0-386.el9.x86_64.conf 2023-11-25T04:11:59.025 DEBUG:teuthology.orchestra.run.smithi195:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-386.el9.x86_64 2023-11-25T04:11:59.067 INFO:teuthology.orchestra.run.smithi076.stdout:kernel-5.14.0-386.el9.x86_64 2023-11-25T04:11:59.069 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-386.el9.x86_64 2023-11-25T04:11:59.069 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-386.el9.x86_64 2023-11-25T04:11:59.069 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-11-25T04:11:59.069 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-11-25T04:11:59.069 INFO:teuthology.task.kernel:Installing distro kernel on osd.0... 2023-11-25T04:11:59.069 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi076.front.sepia.ceph.com, path=None, version=distro) 2023-11-25T04:11:59.069 DEBUG:teuthology.orchestra.run.smithi076:> rpm -q kernel | sort -rV | head -n 1 2023-11-25T04:11:59.152 DEBUG:teuthology.orchestra.run.smithi195:> sudo shutdown -r now 2023-11-25T04:11:59.186 INFO:teuthology.orchestra.run.smithi076.stdout:kernel-5.14.0-386.el9.x86_64 2023-11-25T04:11:59.186 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-386.el9.x86_64 2023-11-25T04:11:59.186 DEBUG:teuthology.orchestra.run.smithi076:> sudo rpm -qi grub2-tools 2023-11-25T04:11:59.227 INFO:teuthology.orchestra.run.smithi076.stdout:Name : grub2-tools 2023-11-25T04:11:59.227 INFO:teuthology.orchestra.run.smithi076.stdout:Epoch : 1 2023-11-25T04:11:59.227 INFO:teuthology.orchestra.run.smithi076.stdout:Version : 2.06 2023-11-25T04:11:59.227 INFO:teuthology.orchestra.run.smithi076.stdout:Release : 68.el9 2023-11-25T04:11:59.228 INFO:teuthology.orchestra.run.smithi076.stdout:Architecture: x86_64 2023-11-25T04:11:59.228 INFO:teuthology.orchestra.run.smithi076.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2023-11-25T04:11:59.228 INFO:teuthology.orchestra.run.smithi076.stdout:Group : Unspecified 2023-11-25T04:11:59.228 INFO:teuthology.orchestra.run.smithi076.stdout:Size : 8264254 2023-11-25T04:11:59.228 INFO:teuthology.orchestra.run.smithi076.stdout:License : GPLv3+ 2023-11-25T04:11:59.228 INFO:teuthology.orchestra.run.smithi076.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2023-11-25T04:11:59.228 INFO:teuthology.orchestra.run.smithi076.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2023-11-25T04:11:59.228 INFO:teuthology.orchestra.run.smithi076.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2023-11-25T04:11:59.228 INFO:teuthology.orchestra.run.smithi076.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-11-25T04:11:59.228 INFO:teuthology.orchestra.run.smithi076.stdout:Packager : builder@centos.org 2023-11-25T04:11:59.228 INFO:teuthology.orchestra.run.smithi076.stdout:Vendor : CentOS 2023-11-25T04:11:59.229 INFO:teuthology.orchestra.run.smithi076.stdout:URL : http://www.gnu.org/software/grub/ 2023-11-25T04:11:59.229 INFO:teuthology.orchestra.run.smithi076.stdout:Summary : Support tools for GRUB. 2023-11-25T04:11:59.229 INFO:teuthology.orchestra.run.smithi076.stdout:Description : 2023-11-25T04:11:59.229 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-25T04:11:59.229 INFO:teuthology.orchestra.run.smithi076.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-11-25T04:11:59.229 INFO:teuthology.orchestra.run.smithi076.stdout:customizable bootloader with modular architecture. It supports a rich 2023-11-25T04:11:59.229 INFO:teuthology.orchestra.run.smithi076.stdout:variety of kernel formats, file systems, computer architectures and 2023-11-25T04:11:59.229 INFO:teuthology.orchestra.run.smithi076.stdout:hardware devices. 2023-11-25T04:11:59.229 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-25T04:11:59.229 INFO:teuthology.orchestra.run.smithi076.stdout:This subpackage provides tools for support of all platforms. 2023-11-25T04:11:59.230 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-11-25T04:11:59.231 INFO:teuthology.task.kernel:Updating grub on smithi076 to boot 5.14.0-386.el9.x86_64 2023-11-25T04:11:59.231 DEBUG:teuthology.orchestra.run.smithi076:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-11-25T04:11:59.887 INFO:teuthology.orchestra.run.smithi076.stderr:Generating grub configuration file ... 2023-11-25T04:12:01.162 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: kernel-core-5.14.0-386.el9.x86_64 4/4 2023-11-25T04:12:01.208 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: kernel-modules-5.14.0-386.el9.x86_64 4/4 2023-11-25T04:12:01.280 INFO:teuthology.orchestra.run.smithi076.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-11-25T04:12:01.297 INFO:teuthology.orchestra.run.smithi076.stderr:done 2023-11-25T04:12:01.300 DEBUG:teuthology.orchestra.run.smithi076:> mktemp 2023-11-25T04:12:01.316 INFO:teuthology.orchestra.run.smithi076.stdout:/tmp/tmp.z2IcA11TYr 2023-11-25T04:12:01.317 DEBUG:teuthology.orchestra.run.smithi076:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.z2IcA11TYr 2023-11-25T04:12:01.386 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 0666 /tmp/tmp.z2IcA11TYr 2023-11-25T04:12:01.521 DEBUG:teuthology.orchestra.remote:smithi076:/tmp/tmp.z2IcA11TYr is 6KB 2023-11-25T04:12:01.570 DEBUG:teuthology.orchestra.run.smithi076:> rm -fr /tmp/tmp.z2IcA11TYr 2023-11-25T04:12:01.584 DEBUG:teuthology.orchestra.run.smithi076:> sudo /bin/ls /boot/loader/entries || true 2023-11-25T04:12:01.618 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: kernel-5.14.0-386.el9.x86_64 4/4 2023-11-25T04:12:01.619 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : kernel-5.14.0-386.el9.x86_64 1/4 2023-11-25T04:12:01.619 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : kernel-core-5.14.0-386.el9.x86_64 2/4 2023-11-25T04:12:01.619 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : kernel-modules-5.14.0-386.el9.x86_64 3/4 2023-11-25T04:12:01.649 INFO:teuthology.orchestra.run.smithi076.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2023-11-25T04:12:01.649 INFO:teuthology.orchestra.run.smithi076.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2023-11-25T04:12:01.649 INFO:teuthology.orchestra.run.smithi076.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2023-11-25T04:12:01.649 INFO:teuthology.orchestra.run.smithi076.stdout:15949e3c88704300882482eddfd7baae-5.14.0-386.el9.x86_64.conf 2023-11-25T04:12:01.650 DEBUG:teuthology.orchestra.run.smithi076:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-386.el9.x86_64 2023-11-25T04:12:01.808 DEBUG:teuthology.orchestra.run.smithi076:> sudo shutdown -r now 2023-11-25T04:12:02.009 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : kernel-modules-core-5.14.0-386.el9.x86_64 4/4 2023-11-25T04:12:02.009 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:12:02.009 INFO:teuthology.orchestra.run.smithi067.stdout:Installed: 2023-11-25T04:12:02.009 INFO:teuthology.orchestra.run.smithi067.stdout: kernel-5.14.0-386.el9.x86_64 2023-11-25T04:12:02.009 INFO:teuthology.orchestra.run.smithi067.stdout: kernel-core-5.14.0-386.el9.x86_64 2023-11-25T04:12:02.009 INFO:teuthology.orchestra.run.smithi067.stdout: kernel-modules-5.14.0-386.el9.x86_64 2023-11-25T04:12:02.009 INFO:teuthology.orchestra.run.smithi067.stdout: kernel-modules-core-5.14.0-386.el9.x86_64 2023-11-25T04:12:02.010 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:12:02.010 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2023-11-25T04:12:02.519 DEBUG:teuthology.orchestra.run.smithi067:> rpm -q kernel | sort -rV | head -n 1 2023-11-25T04:12:02.664 INFO:teuthology.orchestra.run.smithi067.stdout:kernel-5.14.0-386.el9.x86_64 2023-11-25T04:12:02.665 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-386.el9.x86_64 2023-11-25T04:12:02.665 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-386.el9.x86_64 2023-11-25T04:12:02.665 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-11-25T04:12:02.665 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-11-25T04:12:02.665 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2023-11-25T04:12:02.665 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi067.front.sepia.ceph.com, path=None, version=distro) 2023-11-25T04:12:02.665 DEBUG:teuthology.orchestra.run.smithi067:> rpm -q kernel | sort -rV | head -n 1 2023-11-25T04:12:02.787 INFO:teuthology.orchestra.run.smithi067.stdout:kernel-5.14.0-386.el9.x86_64 2023-11-25T04:12:02.788 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-386.el9.x86_64 2023-11-25T04:12:02.788 DEBUG:teuthology.orchestra.run.smithi067:> sudo rpm -qi grub2-tools 2023-11-25T04:12:02.829 INFO:teuthology.orchestra.run.smithi067.stdout:Name : grub2-tools 2023-11-25T04:12:02.830 INFO:teuthology.orchestra.run.smithi067.stdout:Epoch : 1 2023-11-25T04:12:02.830 INFO:teuthology.orchestra.run.smithi067.stdout:Version : 2.06 2023-11-25T04:12:02.830 INFO:teuthology.orchestra.run.smithi067.stdout:Release : 68.el9 2023-11-25T04:12:02.831 INFO:teuthology.orchestra.run.smithi067.stdout:Architecture: x86_64 2023-11-25T04:12:02.831 INFO:teuthology.orchestra.run.smithi067.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2023-11-25T04:12:02.831 INFO:teuthology.orchestra.run.smithi067.stdout:Group : Unspecified 2023-11-25T04:12:02.831 INFO:teuthology.orchestra.run.smithi067.stdout:Size : 8264254 2023-11-25T04:12:02.831 INFO:teuthology.orchestra.run.smithi067.stdout:License : GPLv3+ 2023-11-25T04:12:02.832 INFO:teuthology.orchestra.run.smithi067.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2023-11-25T04:12:02.832 INFO:teuthology.orchestra.run.smithi067.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2023-11-25T04:12:02.832 INFO:teuthology.orchestra.run.smithi067.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2023-11-25T04:12:02.832 INFO:teuthology.orchestra.run.smithi067.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-11-25T04:12:02.832 INFO:teuthology.orchestra.run.smithi067.stdout:Packager : builder@centos.org 2023-11-25T04:12:02.833 INFO:teuthology.orchestra.run.smithi067.stdout:Vendor : CentOS 2023-11-25T04:12:02.833 INFO:teuthology.orchestra.run.smithi067.stdout:URL : http://www.gnu.org/software/grub/ 2023-11-25T04:12:02.833 INFO:teuthology.orchestra.run.smithi067.stdout:Summary : Support tools for GRUB. 2023-11-25T04:12:02.833 INFO:teuthology.orchestra.run.smithi067.stdout:Description : 2023-11-25T04:12:02.833 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:12:02.833 INFO:teuthology.orchestra.run.smithi067.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-11-25T04:12:02.833 INFO:teuthology.orchestra.run.smithi067.stdout:customizable bootloader with modular architecture. It supports a rich 2023-11-25T04:12:02.833 INFO:teuthology.orchestra.run.smithi067.stdout:variety of kernel formats, file systems, computer architectures and 2023-11-25T04:12:02.833 INFO:teuthology.orchestra.run.smithi067.stdout:hardware devices. 2023-11-25T04:12:02.833 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:12:02.834 INFO:teuthology.orchestra.run.smithi067.stdout:This subpackage provides tools for support of all platforms. 2023-11-25T04:12:02.834 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-11-25T04:12:02.835 INFO:teuthology.task.kernel:Updating grub on smithi067 to boot 5.14.0-386.el9.x86_64 2023-11-25T04:12:02.835 DEBUG:teuthology.orchestra.run.smithi067:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-11-25T04:12:03.182 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: kernel-core-5.14.0-386.el9.x86_64 4/4 2023-11-25T04:12:03.226 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: kernel-modules-5.14.0-386.el9.x86_64 4/4 2023-11-25T04:12:03.417 INFO:teuthology.orchestra.run.smithi067.stderr:Generating grub configuration file ... 2023-11-25T04:12:03.617 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: kernel-5.14.0-386.el9.x86_64 4/4 2023-11-25T04:12:03.617 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : kernel-5.14.0-386.el9.x86_64 1/4 2023-11-25T04:12:03.617 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : kernel-core-5.14.0-386.el9.x86_64 2/4 2023-11-25T04:12:03.617 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : kernel-modules-5.14.0-386.el9.x86_64 3/4 2023-11-25T04:12:04.046 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : kernel-modules-core-5.14.0-386.el9.x86_64 4/4 2023-11-25T04:12:04.046 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-25T04:12:04.046 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2023-11-25T04:12:04.047 INFO:teuthology.orchestra.run.smithi092.stdout: kernel-5.14.0-386.el9.x86_64 2023-11-25T04:12:04.047 INFO:teuthology.orchestra.run.smithi092.stdout: kernel-core-5.14.0-386.el9.x86_64 2023-11-25T04:12:04.047 INFO:teuthology.orchestra.run.smithi092.stdout: kernel-modules-5.14.0-386.el9.x86_64 2023-11-25T04:12:04.047 INFO:teuthology.orchestra.run.smithi092.stdout: kernel-modules-core-5.14.0-386.el9.x86_64 2023-11-25T04:12:04.047 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-25T04:12:04.047 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2023-11-25T04:12:04.526 DEBUG:teuthology.orchestra.run.smithi092:> rpm -q kernel | sort -rV | head -n 1 2023-11-25T04:12:04.660 INFO:teuthology.orchestra.run.smithi092.stdout:kernel-5.14.0-386.el9.x86_64 2023-11-25T04:12:04.660 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-386.el9.x86_64 2023-11-25T04:12:04.660 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-386.el9.x86_64 2023-11-25T04:12:04.660 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-11-25T04:12:04.660 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-11-25T04:12:04.660 INFO:teuthology.task.kernel:Installing distro kernel on osd.1... 2023-11-25T04:12:04.661 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi092.front.sepia.ceph.com, path=None, version=distro) 2023-11-25T04:12:04.661 DEBUG:teuthology.orchestra.run.smithi092:> rpm -q kernel | sort -rV | head -n 1 2023-11-25T04:12:04.777 INFO:teuthology.orchestra.run.smithi092.stdout:kernel-5.14.0-386.el9.x86_64 2023-11-25T04:12:04.778 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-386.el9.x86_64 2023-11-25T04:12:04.778 DEBUG:teuthology.orchestra.run.smithi092:> sudo rpm -qi grub2-tools 2023-11-25T04:12:04.812 INFO:teuthology.orchestra.run.smithi092.stdout:Name : grub2-tools 2023-11-25T04:12:04.812 INFO:teuthology.orchestra.run.smithi092.stdout:Epoch : 1 2023-11-25T04:12:04.812 INFO:teuthology.orchestra.run.smithi092.stdout:Version : 2.06 2023-11-25T04:12:04.813 INFO:teuthology.orchestra.run.smithi092.stdout:Release : 68.el9 2023-11-25T04:12:04.813 INFO:teuthology.orchestra.run.smithi092.stdout:Architecture: x86_64 2023-11-25T04:12:04.813 INFO:teuthology.orchestra.run.smithi092.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2023-11-25T04:12:04.813 INFO:teuthology.orchestra.run.smithi092.stdout:Group : Unspecified 2023-11-25T04:12:04.813 INFO:teuthology.orchestra.run.smithi092.stdout:Size : 8264254 2023-11-25T04:12:04.813 INFO:teuthology.orchestra.run.smithi092.stdout:License : GPLv3+ 2023-11-25T04:12:04.814 INFO:teuthology.orchestra.run.smithi092.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2023-11-25T04:12:04.814 INFO:teuthology.orchestra.run.smithi092.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2023-11-25T04:12:04.814 INFO:teuthology.orchestra.run.smithi092.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2023-11-25T04:12:04.814 INFO:teuthology.orchestra.run.smithi092.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-11-25T04:12:04.814 INFO:teuthology.orchestra.run.smithi092.stdout:Packager : builder@centos.org 2023-11-25T04:12:04.814 INFO:teuthology.orchestra.run.smithi092.stdout:Vendor : CentOS 2023-11-25T04:12:04.814 INFO:teuthology.orchestra.run.smithi092.stdout:URL : http://www.gnu.org/software/grub/ 2023-11-25T04:12:04.815 INFO:teuthology.orchestra.run.smithi092.stdout:Summary : Support tools for GRUB. 2023-11-25T04:12:04.815 INFO:teuthology.orchestra.run.smithi092.stdout:Description : 2023-11-25T04:12:04.815 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-25T04:12:04.815 INFO:teuthology.orchestra.run.smithi092.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-11-25T04:12:04.815 INFO:teuthology.orchestra.run.smithi092.stdout:customizable bootloader with modular architecture. It supports a rich 2023-11-25T04:12:04.815 INFO:teuthology.orchestra.run.smithi092.stdout:variety of kernel formats, file systems, computer architectures and 2023-11-25T04:12:04.816 INFO:teuthology.orchestra.run.smithi092.stdout:hardware devices. 2023-11-25T04:12:04.816 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-25T04:12:04.816 INFO:teuthology.orchestra.run.smithi092.stdout:This subpackage provides tools for support of all platforms. 2023-11-25T04:12:04.817 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-11-25T04:12:04.818 INFO:teuthology.task.kernel:Updating grub on smithi092 to boot 5.14.0-386.el9.x86_64 2023-11-25T04:12:04.818 DEBUG:teuthology.orchestra.run.smithi092:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-11-25T04:12:04.841 INFO:teuthology.orchestra.run.smithi067.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-11-25T04:12:04.864 INFO:teuthology.orchestra.run.smithi067.stderr:done 2023-11-25T04:12:04.866 DEBUG:teuthology.orchestra.run.smithi067:> mktemp 2023-11-25T04:12:04.882 INFO:teuthology.orchestra.run.smithi067.stdout:/tmp/tmp.fKj3kqYRly 2023-11-25T04:12:04.882 DEBUG:teuthology.orchestra.run.smithi067:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.fKj3kqYRly 2023-11-25T04:12:04.960 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 0666 /tmp/tmp.fKj3kqYRly 2023-11-25T04:12:05.115 DEBUG:teuthology.orchestra.remote:smithi067:/tmp/tmp.fKj3kqYRly is 6KB 2023-11-25T04:12:05.164 DEBUG:teuthology.orchestra.run.smithi067:> rm -fr /tmp/tmp.fKj3kqYRly 2023-11-25T04:12:05.185 DEBUG:teuthology.orchestra.run.smithi067:> sudo /bin/ls /boot/loader/entries || true 2023-11-25T04:12:05.254 INFO:teuthology.orchestra.run.smithi067.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2023-11-25T04:12:05.254 INFO:teuthology.orchestra.run.smithi067.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2023-11-25T04:12:05.254 INFO:teuthology.orchestra.run.smithi067.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2023-11-25T04:12:05.254 INFO:teuthology.orchestra.run.smithi067.stdout:15949e3c88704300882482eddfd7baae-5.14.0-386.el9.x86_64.conf 2023-11-25T04:12:05.255 DEBUG:teuthology.orchestra.run.smithi067:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-386.el9.x86_64 2023-11-25T04:12:05.389 DEBUG:teuthology.orchestra.run.smithi067:> sudo shutdown -r now 2023-11-25T04:12:05.410 INFO:teuthology.orchestra.run.smithi092.stderr:Generating grub configuration file ... 2023-11-25T04:12:06.773 INFO:teuthology.orchestra.run.smithi092.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-11-25T04:12:06.812 INFO:teuthology.orchestra.run.smithi092.stderr:done 2023-11-25T04:12:06.814 DEBUG:teuthology.orchestra.run.smithi092:> mktemp 2023-11-25T04:12:06.830 INFO:teuthology.orchestra.run.smithi092.stdout:/tmp/tmp.fLzyZABdTU 2023-11-25T04:12:06.830 DEBUG:teuthology.orchestra.run.smithi092:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.fLzyZABdTU 2023-11-25T04:12:06.898 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0666 /tmp/tmp.fLzyZABdTU 2023-11-25T04:12:07.058 DEBUG:teuthology.orchestra.remote:smithi092:/tmp/tmp.fLzyZABdTU is 6KB 2023-11-25T04:12:07.077 DEBUG:teuthology.orchestra.run.smithi092:> rm -fr /tmp/tmp.fLzyZABdTU 2023-11-25T04:12:07.092 DEBUG:teuthology.orchestra.run.smithi092:> sudo /bin/ls /boot/loader/entries || true 2023-11-25T04:12:07.156 INFO:teuthology.orchestra.run.smithi092.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2023-11-25T04:12:07.156 INFO:teuthology.orchestra.run.smithi092.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2023-11-25T04:12:07.157 INFO:teuthology.orchestra.run.smithi092.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2023-11-25T04:12:07.157 INFO:teuthology.orchestra.run.smithi092.stdout:15949e3c88704300882482eddfd7baae-5.14.0-386.el9.x86_64.conf 2023-11-25T04:12:07.158 DEBUG:teuthology.orchestra.run.smithi092:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-386.el9.x86_64 2023-11-25T04:12:07.305 DEBUG:teuthology.orchestra.run.smithi092:> sudo shutdown -r now 2023-11-25T04:12:29.174 INFO:teuthology.task.kernel:Checking client osd.2 for new kernel version... 2023-11-25T04:12:29.174 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi195.front.sepia.ceph.com' 2023-11-25T04:12:29.175 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-25T04:12:31.814 INFO:teuthology.task.kernel:Checking client osd.0 for new kernel version... 2023-11-25T04:12:31.814 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi076.front.sepia.ceph.com' 2023-11-25T04:12:31.815 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-25T04:12:35.394 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-11-25T04:12:35.394 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com' 2023-11-25T04:12:35.395 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-25T04:12:37.310 INFO:teuthology.task.kernel:Checking client osd.1 for new kernel version... 2023-11-25T04:12:37.310 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2023-11-25T04:12:37.311 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-25T04:12:47.586 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.195 2023-11-25T04:12:50.150 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.76 2023-11-25T04:12:53.734 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.67 2023-11-25T04:12:55.782 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.92 2023-11-25T04:12:56.636 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi195.front.sepia.ceph.com' 2023-11-25T04:12:56.640 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-25T04:12:59.154 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi076.front.sepia.ceph.com' 2023-11-25T04:12:59.154 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-25T04:13:02.210 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.76 2023-11-25T04:13:02.735 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com' 2023-11-25T04:13:02.736 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-25T04:13:04.785 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2023-11-25T04:13:04.786 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-25T04:13:05.798 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.67 2023-11-25T04:13:07.842 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.92 2023-11-25T04:13:14.216 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi076.front.sepia.ceph.com' 2023-11-25T04:13:14.217 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-25T04:13:17.802 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com' 2023-11-25T04:13:17.802 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-25T04:13:19.845 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2023-11-25T04:13:19.846 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-25T04:13:36.226 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.67 2023-11-25T04:13:51.242 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com' 2023-11-25T04:13:51.243 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-25T04:13:51.640 DEBUG:teuthology.orchestra.run.smithi092:> true 2023-11-25T04:13:51.642 DEBUG:teuthology.orchestra.run.smithi067:> true 2023-11-25T04:13:52.026 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi092.front.sepia.ceph.com' 2023-11-25T04:13:52.026 INFO:teuthology.task.kernel:Checking kernel version of osd.1, want "5.14.0-386.el9.x86_64"... 2023-11-25T04:13:52.026 DEBUG:teuthology.orchestra.run.smithi092:> uname -r 2023-11-25T04:13:52.040 INFO:teuthology.orchestra.run.smithi092.stdout:5.14.0-386.el9.x86_64 2023-11-25T04:13:52.040 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-386.el9.x86_64 vs 5.14.0-386.el9.x86_64 2023-11-25T04:13:52.040 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-11-25T04:13:52.040 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-11-25T04:13:52.078 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi067.front.sepia.ceph.com' 2023-11-25T04:13:52.078 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-386.el9.x86_64"... 2023-11-25T04:13:52.078 DEBUG:teuthology.orchestra.run.smithi067:> uname -r 2023-11-25T04:13:52.094 INFO:teuthology.orchestra.run.smithi067.stdout:5.14.0-386.el9.x86_64 2023-11-25T04:13:52.094 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-386.el9.x86_64 vs 5.14.0-386.el9.x86_64 2023-11-25T04:13:52.094 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-11-25T04:13:52.094 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-11-25T04:13:53.042 INFO:teuthology.task.kernel:Enabling kdb on osd.1... 2023-11-25T04:13:53.043 DEBUG:teuthology.orchestra.run.smithi092:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-11-25T04:13:53.095 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-11-25T04:13:53.096 DEBUG:teuthology.orchestra.run.smithi067:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-11-25T04:13:53.150 INFO:teuthology.orchestra.run.smithi067.stdout:ttyS1 2023-11-25T04:13:53.180 DEBUG:teuthology.parallel:result is None 2023-11-25T04:13:53.212 INFO:teuthology.orchestra.run.smithi092.stdout:ttyS1 2023-11-25T04:13:53.268 DEBUG:teuthology.parallel:result is None 2023-11-25T04:13:56.642 DEBUG:teuthology.orchestra.remote:timed out 2023-11-25T04:14:08.643 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi195.front.sepia.ceph.com' 2023-11-25T04:14:08.644 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-25T04:14:09.048 DEBUG:teuthology.orchestra.run.smithi195:> true 2023-11-25T04:14:09.414 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi195.front.sepia.ceph.com' 2023-11-25T04:14:09.414 INFO:teuthology.task.kernel:Checking kernel version of osd.2, want "5.14.0-386.el9.x86_64"... 2023-11-25T04:14:09.414 DEBUG:teuthology.orchestra.run.smithi195:> uname -r 2023-11-25T04:14:09.430 INFO:teuthology.orchestra.run.smithi195.stdout:5.14.0-386.el9.x86_64 2023-11-25T04:14:09.430 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-386.el9.x86_64 vs 5.14.0-386.el9.x86_64 2023-11-25T04:14:09.430 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-11-25T04:14:09.430 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-11-25T04:14:10.431 INFO:teuthology.task.kernel:Enabling kdb on osd.2... 2023-11-25T04:14:10.431 DEBUG:teuthology.orchestra.run.smithi195:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-11-25T04:14:10.774 INFO:teuthology.orchestra.run.smithi195.stdout:ttyS1 2023-11-25T04:14:13.988 DEBUG:teuthology.parallel:result is None 2023-11-25T04:14:14.219 DEBUG:teuthology.orchestra.remote:timed out 2023-11-25T04:14:29.219 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi076.front.sepia.ceph.com' 2023-11-25T04:14:29.220 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-25T04:14:29.503 DEBUG:teuthology.orchestra.run.smithi076:> true 2023-11-25T04:14:29.841 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi076.front.sepia.ceph.com' 2023-11-25T04:14:29.841 INFO:teuthology.task.kernel:Checking kernel version of osd.0, want "5.14.0-386.el9.x86_64"... 2023-11-25T04:14:29.842 DEBUG:teuthology.orchestra.run.smithi076:> uname -r 2023-11-25T04:14:29.857 INFO:teuthology.orchestra.run.smithi076.stdout:5.14.0-386.el9.x86_64 2023-11-25T04:14:29.857 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-386.el9.x86_64 vs 5.14.0-386.el9.x86_64 2023-11-25T04:14:29.857 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-11-25T04:14:29.857 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-11-25T04:14:30.858 INFO:teuthology.task.kernel:Enabling kdb on osd.0... 2023-11-25T04:14:30.858 DEBUG:teuthology.orchestra.run.smithi076:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-11-25T04:14:30.936 INFO:teuthology.orchestra.run.smithi076.stdout:ttyS1 2023-11-25T04:14:30.980 DEBUG:teuthology.parallel:result is None 2023-11-25T04:14:30.980 INFO:teuthology.run_tasks:Running task internal.base... 2023-11-25T04:14:30.987 INFO:teuthology.task.internal:Creating test directory... 2023-11-25T04:14:30.987 DEBUG:teuthology.orchestra.run.smithi067:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-25T04:14:30.990 DEBUG:teuthology.orchestra.run.smithi076:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-25T04:14:30.993 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-25T04:14:30.996 DEBUG:teuthology.orchestra.run.smithi195:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-25T04:14:31.014 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-11-25T04:14:31.050 INFO:teuthology.run_tasks:Running task internal.archive... 2023-11-25T04:14:31.060 INFO:teuthology.task.internal:Creating archive directory... 2023-11-25T04:14:31.060 DEBUG:teuthology.orchestra.run.smithi067:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-25T04:14:31.062 DEBUG:teuthology.orchestra.run.smithi076:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-25T04:14:31.065 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-25T04:14:31.067 DEBUG:teuthology.orchestra.run.smithi195:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-25T04:14:31.099 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-11-25T04:14:31.106 INFO:teuthology.task.internal:Enabling coredump saving... 2023-11-25T04:14:31.106 DEBUG:teuthology.orchestra.run.smithi067:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-25T04:14:31.127 DEBUG:teuthology.orchestra.run.smithi076:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-25T04:14:31.129 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-25T04:14:31.132 DEBUG:teuthology.orchestra.run.smithi195:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-25T04:14:31.163 INFO:teuthology.orchestra.run.smithi076.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-25T04:14:31.168 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-25T04:14:31.172 INFO:teuthology.orchestra.run.smithi195.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-25T04:14:31.173 INFO:teuthology.orchestra.run.smithi067.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-25T04:14:31.177 INFO:teuthology.orchestra.run.smithi076.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-25T04:14:31.183 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-25T04:14:31.186 INFO:teuthology.orchestra.run.smithi195.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-25T04:14:31.190 INFO:teuthology.orchestra.run.smithi067.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-25T04:14:31.192 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-11-25T04:14:31.199 INFO:teuthology.task.internal:Configuring sudo... 2023-11-25T04:14:31.199 DEBUG:teuthology.orchestra.run.smithi067:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-25T04:14:31.202 DEBUG:teuthology.orchestra.run.smithi076:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-25T04:14:31.221 DEBUG:teuthology.orchestra.run.smithi092:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-25T04:14:31.227 DEBUG:teuthology.orchestra.run.smithi195:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-25T04:14:31.259 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-11-25T04:14:31.268 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-11-25T04:14:31.269 DEBUG:teuthology.orchestra.run.smithi067:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-25T04:14:31.276 DEBUG:teuthology.orchestra.run.smithi076:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-25T04:14:31.290 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-25T04:14:31.298 DEBUG:teuthology.orchestra.run.smithi195:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-25T04:14:31.317 DEBUG:teuthology.orchestra.run.smithi067:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-25T04:14:31.377 DEBUG:teuthology.orchestra.run.smithi067:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-25T04:14:31.426 DEBUG:teuthology.orchestra.run.smithi067:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-25T04:14:31.507 DEBUG:teuthology.orchestra.run.smithi067:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-25T04:14:31.555 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-11-25T04:14:31.556 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-25T04:14:31.627 DEBUG:teuthology.orchestra.run.smithi076:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-25T04:14:31.670 DEBUG:teuthology.orchestra.run.smithi076:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-25T04:14:31.724 DEBUG:teuthology.orchestra.run.smithi076:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-25T04:14:31.808 DEBUG:teuthology.orchestra.run.smithi076:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-25T04:14:31.851 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-25T04:14:31.851 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-25T04:14:31.922 DEBUG:teuthology.orchestra.run.smithi092:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-25T04:14:31.966 DEBUG:teuthology.orchestra.run.smithi092:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-25T04:14:32.030 DEBUG:teuthology.orchestra.run.smithi092:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-25T04:14:32.075 DEBUG:teuthology.orchestra.run.smithi092:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-25T04:14:32.120 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-25T04:14:32.120 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-25T04:14:32.191 DEBUG:teuthology.orchestra.run.smithi195:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-25T04:14:32.233 DEBUG:teuthology.orchestra.run.smithi195:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-25T04:14:32.293 DEBUG:teuthology.orchestra.run.smithi195:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-25T04:14:32.377 DEBUG:teuthology.orchestra.run.smithi195:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-25T04:14:32.423 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-25T04:14:32.423 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-25T04:14:32.491 DEBUG:teuthology.orchestra.run.smithi067:> sudo service rsyslog restart 2023-11-25T04:14:32.494 DEBUG:teuthology.orchestra.run.smithi076:> sudo service rsyslog restart 2023-11-25T04:14:32.496 DEBUG:teuthology.orchestra.run.smithi092:> sudo service rsyslog restart 2023-11-25T04:14:32.498 DEBUG:teuthology.orchestra.run.smithi195:> sudo service rsyslog restart 2023-11-25T04:14:32.549 INFO:teuthology.orchestra.run.smithi067.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-25T04:14:32.549 INFO:teuthology.orchestra.run.smithi092.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-25T04:14:32.554 INFO:teuthology.orchestra.run.smithi076.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-25T04:14:32.570 INFO:teuthology.orchestra.run.smithi195.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-25T04:14:32.970 INFO:teuthology.run_tasks:Running task internal.timer... 2023-11-25T04:14:32.978 INFO:teuthology.task.internal:Starting timer... 2023-11-25T04:14:32.978 INFO:teuthology.run_tasks:Running task pcp... 2023-11-25T04:14:32.988 INFO:teuthology.run_tasks:Running task selinux... 2023-11-25T04:14:32.998 DEBUG:teuthology.orchestra.run.smithi067:> sudo service auditd rotate 2023-11-25T04:14:33.057 INFO:teuthology.orchestra.run.smithi067.stdout:Rotating logs: 2023-11-25T04:14:33.059 DEBUG:teuthology.orchestra.run.smithi076:> sudo service auditd rotate 2023-11-25T04:14:33.117 INFO:teuthology.orchestra.run.smithi076.stdout:Rotating logs: 2023-11-25T04:14:33.119 DEBUG:teuthology.orchestra.run.smithi092:> sudo service auditd rotate 2023-11-25T04:14:33.172 INFO:teuthology.orchestra.run.smithi092.stdout:Rotating logs: 2023-11-25T04:14:33.174 DEBUG:teuthology.orchestra.run.smithi195:> sudo service auditd rotate 2023-11-25T04:14:33.228 INFO:teuthology.orchestra.run.smithi195.stdout:Rotating logs: 2023-11-25T04:14:33.230 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-11-25T04:14:33.230 DEBUG:teuthology.orchestra.run.smithi067:> /usr/sbin/getenforce 2023-11-25T04:14:33.253 INFO:teuthology.orchestra.run.smithi067.stdout:Permissive 2023-11-25T04:14:33.253 DEBUG:teuthology.orchestra.run.smithi076:> /usr/sbin/getenforce 2023-11-25T04:14:33.281 INFO:teuthology.orchestra.run.smithi076.stdout:Permissive 2023-11-25T04:14:33.282 DEBUG:teuthology.orchestra.run.smithi092:> /usr/sbin/getenforce 2023-11-25T04:14:33.306 INFO:teuthology.orchestra.run.smithi092.stdout:Permissive 2023-11-25T04:14:33.306 DEBUG:teuthology.orchestra.run.smithi195:> /usr/sbin/getenforce 2023-11-25T04:14:33.331 INFO:teuthology.orchestra.run.smithi195.stdout:Permissive 2023-11-25T04:14:33.332 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi067.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi076.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi092.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi195.front.sepia.ceph.com': 'permissive'} 2023-11-25T04:14:33.332 DEBUG:teuthology.orchestra.run.smithi067:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-25T04:14:33.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-25T04:14:33.366 DEBUG:teuthology.orchestra.run.smithi076:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-25T04:14:33.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-25T04:14:33.396 DEBUG:teuthology.orchestra.run.smithi092:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-25T04:14:33.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-25T04:14:33.426 DEBUG:teuthology.orchestra.run.smithi195:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-25T04:14:33.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-25T04:14:33.455 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-11-25T04:14:33.455 DEBUG:teuthology.orchestra.run.smithi067:> sudo /usr/sbin/setenforce permissive 2023-11-25T04:14:33.497 DEBUG:teuthology.orchestra.run.smithi076:> sudo /usr/sbin/setenforce permissive 2023-11-25T04:14:33.540 DEBUG:teuthology.orchestra.run.smithi092:> sudo /usr/sbin/setenforce permissive 2023-11-25T04:14:33.591 DEBUG:teuthology.orchestra.run.smithi195:> sudo /usr/sbin/setenforce permissive 2023-11-25T04:14:33.625 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-11-25T04:14:33.635 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-11-25T04:14:33.639 INFO:teuthology.repo_utils:Fetching main from origin 2023-11-25T04:14:33.766 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-11-25T04:14:33.789 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-11-25T04:14:33.790 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi067.front.sepia.ceph.com,smithi076.front.sepia.ceph.com,smithi092.front.sepia.ceph.com,smithi195.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-11-25T04:20:23.734 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi067.front.sepia.ceph.com'), Remote(name='ubuntu@smithi076.front.sepia.ceph.com'), Remote(name='ubuntu@smithi092.front.sepia.ceph.com'), Remote(name='ubuntu@smithi195.front.sepia.ceph.com')] 2023-11-25T04:20:23.735 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com' 2023-11-25T04:20:23.736 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-25T04:20:23.836 DEBUG:teuthology.orchestra.run.smithi067:> true 2023-11-25T04:20:23.905 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi067.front.sepia.ceph.com' 2023-11-25T04:20:23.905 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi076.front.sepia.ceph.com' 2023-11-25T04:20:23.906 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-25T04:20:23.990 DEBUG:teuthology.orchestra.run.smithi076:> true 2023-11-25T04:20:24.057 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi076.front.sepia.ceph.com' 2023-11-25T04:20:24.058 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2023-11-25T04:20:24.058 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-25T04:20:24.141 DEBUG:teuthology.orchestra.run.smithi092:> true 2023-11-25T04:20:24.209 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi092.front.sepia.ceph.com' 2023-11-25T04:20:24.210 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi195.front.sepia.ceph.com' 2023-11-25T04:20:24.210 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-25T04:20:24.294 DEBUG:teuthology.orchestra.run.smithi195:> true 2023-11-25T04:20:24.362 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi195.front.sepia.ceph.com' 2023-11-25T04:20:24.364 INFO:teuthology.run_tasks:Running task clock... 2023-11-25T04:20:24.378 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-11-25T04:20:24.378 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-25T04:20:24.379 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-25T04:20:24.381 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-25T04:20:24.381 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-25T04:20:24.383 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-25T04:20:24.383 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-25T04:20:24.386 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-25T04:20:24.386 DEBUG:teuthology.orchestra.run.smithi195:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-25T04:20:24.414 INFO:teuthology.orchestra.run.smithi076.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-25T04:20:24.416 INFO:teuthology.orchestra.run.smithi067.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-25T04:20:24.417 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-25T04:20:24.431 INFO:teuthology.orchestra.run.smithi076.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-25T04:20:24.435 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-25T04:20:24.435 INFO:teuthology.orchestra.run.smithi195.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-25T04:20:24.437 INFO:teuthology.orchestra.run.smithi067.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-25T04:20:24.453 INFO:teuthology.orchestra.run.smithi195.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-25T04:20:24.461 INFO:teuthology.orchestra.run.smithi076.stderr:sudo: ntpd: command not found 2023-11-25T04:20:24.466 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: ntpd: command not found 2023-11-25T04:20:24.474 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: ntpd: command not found 2023-11-25T04:20:24.475 INFO:teuthology.orchestra.run.smithi076.stdout:506 Cannot talk to daemon 2023-11-25T04:20:24.480 INFO:teuthology.orchestra.run.smithi092.stdout:506 Cannot talk to daemon 2023-11-25T04:20:24.482 INFO:teuthology.orchestra.run.smithi195.stderr:sudo: ntpd: command not found 2023-11-25T04:20:24.491 INFO:teuthology.orchestra.run.smithi067.stdout:506 Cannot talk to daemon 2023-11-25T04:20:24.492 INFO:teuthology.orchestra.run.smithi076.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-25T04:20:24.496 INFO:teuthology.orchestra.run.smithi195.stdout:506 Cannot talk to daemon 2023-11-25T04:20:24.497 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-25T04:20:24.508 INFO:teuthology.orchestra.run.smithi076.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-25T04:20:24.512 INFO:teuthology.orchestra.run.smithi067.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-25T04:20:24.513 INFO:teuthology.orchestra.run.smithi195.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-25T04:20:24.513 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-25T04:20:24.529 INFO:teuthology.orchestra.run.smithi195.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-25T04:20:24.532 INFO:teuthology.orchestra.run.smithi067.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-25T04:20:24.551 INFO:teuthology.orchestra.run.smithi076.stderr:bash: line 1: ntpq: command not found 2023-11-25T04:20:24.553 INFO:teuthology.orchestra.run.smithi076.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-25T04:20:24.553 INFO:teuthology.orchestra.run.smithi076.stdout:=============================================================================== 2023-11-25T04:20:24.553 INFO:teuthology.orchestra.run.smithi076.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-25T04:20:24.553 INFO:teuthology.orchestra.run.smithi076.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-25T04:20:24.554 INFO:teuthology.orchestra.run.smithi076.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-25T04:20:24.554 INFO:teuthology.orchestra.run.smithi076.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-25T04:20:24.556 INFO:teuthology.orchestra.run.smithi092.stderr:bash: line 1: ntpq: command not found 2023-11-25T04:20:24.559 INFO:teuthology.orchestra.run.smithi092.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-25T04:20:24.559 INFO:teuthology.orchestra.run.smithi092.stdout:=============================================================================== 2023-11-25T04:20:24.559 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-25T04:20:24.559 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-25T04:20:24.559 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-25T04:20:24.559 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-25T04:20:24.573 INFO:teuthology.orchestra.run.smithi195.stderr:bash: line 1: ntpq: command not found 2023-11-25T04:20:24.576 INFO:teuthology.orchestra.run.smithi195.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-25T04:20:24.576 INFO:teuthology.orchestra.run.smithi195.stdout:=============================================================================== 2023-11-25T04:20:24.576 INFO:teuthology.orchestra.run.smithi195.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-25T04:20:24.576 INFO:teuthology.orchestra.run.smithi195.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-25T04:20:24.576 INFO:teuthology.orchestra.run.smithi195.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-25T04:20:24.576 INFO:teuthology.orchestra.run.smithi195.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-25T04:20:24.579 INFO:teuthology.orchestra.run.smithi067.stderr:bash: line 1: ntpq: command not found 2023-11-25T04:20:24.582 INFO:teuthology.orchestra.run.smithi067.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-25T04:20:24.582 INFO:teuthology.orchestra.run.smithi067.stdout:=============================================================================== 2023-11-25T04:20:24.583 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-25T04:20:24.583 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-25T04:20:24.583 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-25T04:20:24.583 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-25T04:20:24.584 INFO:teuthology.run_tasks:Running task install... 2023-11-25T04:20:24.593 DEBUG:teuthology.task.install:project ceph 2023-11-25T04:20:24.593 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'ec928368edcc5fc71884a56fd9fd821b148f4e30'}} 2023-11-25T04:20:24.593 DEBUG:teuthology.task.install:config {'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': 'ec928368edcc5fc71884a56fd9fd821b148f4e30'} 2023-11-25T04:20:24.593 INFO:teuthology.task.install:Using flavor: default 2023-11-25T04:20:24.600 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-11-25T04:20:24.600 INFO:teuthology.task.install:extra packages: [] 2023-11-25T04:20:24.600 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'ec928368edcc5fc71884a56fd9fd821b148f4e30', 'tag': None, 'wait_for_package': False} 2023-11-25T04:20:24.601 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ec928368edcc5fc71884a56fd9fd821b148f4e30 2023-11-25T04:20:24.602 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'ec928368edcc5fc71884a56fd9fd821b148f4e30', 'tag': None, 'wait_for_package': False} 2023-11-25T04:20:24.603 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ec928368edcc5fc71884a56fd9fd821b148f4e30 2023-11-25T04:20:24.604 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'ec928368edcc5fc71884a56fd9fd821b148f4e30', 'tag': None, 'wait_for_package': False} 2023-11-25T04:20:24.604 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ec928368edcc5fc71884a56fd9fd821b148f4e30 2023-11-25T04:20:24.605 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'ec928368edcc5fc71884a56fd9fd821b148f4e30', 'tag': None, 'wait_for_package': False} 2023-11-25T04:20:24.605 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ec928368edcc5fc71884a56fd9fd821b148f4e30 2023-11-25T04:20:24.770 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/main/ec928368edcc5fc71884a56fd9fd821b148f4e30/centos/9/flavors/default/ 2023-11-25T04:20:24.770 INFO:teuthology.task.install.rpm:Package version is 18.0.0-7506.gec928368 2023-11-25T04:20:24.785 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/main/ec928368edcc5fc71884a56fd9fd821b148f4e30/centos/9/flavors/default/ 2023-11-25T04:20:24.786 INFO:teuthology.task.install.rpm:Package version is 18.0.0-7506.gec928368 2023-11-25T04:20:24.787 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/main/ec928368edcc5fc71884a56fd9fd821b148f4e30/centos/9/flavors/default/ 2023-11-25T04:20:24.787 INFO:teuthology.task.install.rpm:Package version is 18.0.0-7506.gec928368 2023-11-25T04:20:24.849 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/main/ec928368edcc5fc71884a56fd9fd821b148f4e30/centos/9/flavors/default/ 2023-11-25T04:20:24.849 INFO:teuthology.task.install.rpm:Package version is 18.0.0-7506.gec928368 2023-11-25T04:20:24.898 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/main/ec928368edcc5fc71884a56fd9fd821b148f4e30/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/main/ec928368edcc5fc71884a56fd9fd821b148f4e30/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/main/ec928368edcc5fc71884a56fd9fd821b148f4e30/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-25T04:20:24.898 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-25T04:20:24.898 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-25T04:20:24.926 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, bison, flex, elfutils-libelf-devel, openssl-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2023-11-25T04:20:24.926 DEBUG:teuthology.orchestra.run.smithi076:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/ec928368edcc5fc71884a56fd9fd821b148f4e30/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-25T04:20:24.953 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/main/ec928368edcc5fc71884a56fd9fd821b148f4e30/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/main/ec928368edcc5fc71884a56fd9fd821b148f4e30/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/main/ec928368edcc5fc71884a56fd9fd821b148f4e30/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-25T04:20:24.954 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-11-25T04:20:24.954 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-25T04:20:24.958 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/main/ec928368edcc5fc71884a56fd9fd821b148f4e30/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/main/ec928368edcc5fc71884a56fd9fd821b148f4e30/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/main/ec928368edcc5fc71884a56fd9fd821b148f4e30/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-25T04:20:24.958 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-25T04:20:24.958 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-25T04:20:24.986 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, bison, flex, elfutils-libelf-devel, openssl-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2023-11-25T04:20:24.987 DEBUG:teuthology.orchestra.run.smithi195:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/ec928368edcc5fc71884a56fd9fd821b148f4e30/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-25T04:20:24.990 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, bison, flex, elfutils-libelf-devel, openssl-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2023-11-25T04:20:24.990 DEBUG:teuthology.orchestra.run.smithi067:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/ec928368edcc5fc71884a56fd9fd821b148f4e30/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-25T04:20:25.004 DEBUG:teuthology.orchestra.run.smithi076:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-25T04:20:25.017 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/main/ec928368edcc5fc71884a56fd9fd821b148f4e30/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/main/ec928368edcc5fc71884a56fd9fd821b148f4e30/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/main/ec928368edcc5fc71884a56fd9fd821b148f4e30/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-25T04:20:25.017 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-25T04:20:25.017 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-25T04:20:25.047 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, bison, flex, elfutils-libelf-devel, openssl-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2023-11-25T04:20:25.048 DEBUG:teuthology.orchestra.run.smithi092:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/ec928368edcc5fc71884a56fd9fd821b148f4e30/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-25T04:20:25.054 DEBUG:teuthology.orchestra.run.smithi195:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-25T04:20:25.061 DEBUG:teuthology.orchestra.run.smithi067:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-25T04:20:25.083 DEBUG:teuthology.orchestra.run.smithi076:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-25T04:20:25.116 DEBUG:teuthology.orchestra.run.smithi092:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-25T04:20:25.134 DEBUG:teuthology.orchestra.run.smithi195:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-25T04:20:25.148 DEBUG:teuthology.orchestra.run.smithi067:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-25T04:20:25.149 INFO:teuthology.orchestra.run.smithi076.stdout:check_obsoletes = 1 2023-11-25T04:20:25.151 DEBUG:teuthology.orchestra.run.smithi076:> sudo yum clean all 2023-11-25T04:20:25.177 INFO:teuthology.orchestra.run.smithi067.stdout:check_obsoletes = 1 2023-11-25T04:20:25.179 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum clean all 2023-11-25T04:20:25.198 DEBUG:teuthology.orchestra.run.smithi092:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-25T04:20:25.201 INFO:teuthology.orchestra.run.smithi195.stdout:check_obsoletes = 1 2023-11-25T04:20:25.202 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum clean all 2023-11-25T04:20:25.266 INFO:teuthology.orchestra.run.smithi092.stdout:check_obsoletes = 1 2023-11-25T04:20:25.268 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum clean all 2023-11-25T04:20:25.413 INFO:teuthology.orchestra.run.smithi076.stdout:73 files removed 2023-11-25T04:20:25.433 INFO:teuthology.orchestra.run.smithi067.stdout:73 files removed 2023-11-25T04:20:25.440 DEBUG:teuthology.orchestra.run.smithi076:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2023-11-25T04:20:25.441 INFO:teuthology.orchestra.run.smithi195.stdout:73 files removed 2023-11-25T04:20:25.462 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2023-11-25T04:20:25.468 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2023-11-25T04:20:25.522 INFO:teuthology.orchestra.run.smithi092.stdout:73 files removed 2023-11-25T04:20:25.549 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2023-11-25T04:20:26.117 INFO:teuthology.orchestra.run.smithi195.stdout:ceph packages for x86_64 237 kB/s | 78 kB 00:00 2023-11-25T04:20:26.159 INFO:teuthology.orchestra.run.smithi076.stdout:ceph packages for x86_64 195 kB/s | 78 kB 00:00 2023-11-25T04:20:26.206 INFO:teuthology.orchestra.run.smithi067.stdout:ceph packages for x86_64 192 kB/s | 78 kB 00:00 2023-11-25T04:20:26.274 INFO:teuthology.orchestra.run.smithi092.stdout:ceph packages for x86_64 191 kB/s | 78 kB 00:00 2023-11-25T04:20:26.534 INFO:teuthology.orchestra.run.smithi195.stdout:ceph noarch packages 26 kB/s | 9.5 kB 00:00 2023-11-25T04:20:26.601 INFO:teuthology.orchestra.run.smithi067.stdout:ceph noarch packages 28 kB/s | 9.5 kB 00:00 2023-11-25T04:20:26.610 INFO:teuthology.orchestra.run.smithi076.stdout:ceph noarch packages 24 kB/s | 9.5 kB 00:00 2023-11-25T04:20:26.692 INFO:teuthology.orchestra.run.smithi092.stdout:ceph noarch packages 26 kB/s | 9.5 kB 00:00 2023-11-25T04:20:26.918 INFO:teuthology.orchestra.run.smithi195.stdout:ceph source packages 5.4 kB/s | 1.9 kB 00:00 2023-11-25T04:20:26.933 INFO:teuthology.orchestra.run.smithi067.stdout:ceph source packages 6.4 kB/s | 1.9 kB 00:00 2023-11-25T04:20:26.994 INFO:teuthology.orchestra.run.smithi076.stdout:ceph source packages 5.4 kB/s | 1.9 kB 00:00 2023-11-25T04:20:27.042 INFO:teuthology.orchestra.run.smithi092.stdout:ceph source packages 6.0 kB/s | 1.9 kB 00:00 2023-11-25T04:20:27.379 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS-9 - AppStream 15 MB/s | 6.4 MB 00:00 2023-11-25T04:20:27.393 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS-9 - AppStream 15 MB/s | 6.4 MB 00:00 2023-11-25T04:20:27.490 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS-9 - AppStream 14 MB/s | 6.4 MB 00:00 2023-11-25T04:20:27.528 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS-9 - AppStream 14 MB/s | 6.4 MB 00:00 2023-11-25T04:20:28.737 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS-9 - BaseOS 6.5 MB/s | 2.0 MB 00:00 2023-11-25T04:20:28.777 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS-9 - BaseOS 7.4 MB/s | 2.0 MB 00:00 2023-11-25T04:20:28.972 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS-9 - BaseOS 4.6 MB/s | 2.0 MB 00:00 2023-11-25T04:20:29.028 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS-9 - BaseOS 4.5 MB/s | 2.0 MB 00:00 2023-11-25T04:20:30.087 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - BaseOS 9.9 MB/s | 7.9 MB 00:00 2023-11-25T04:20:30.111 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 7.9 MB 00:00 2023-11-25T04:20:30.678 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS Stream 9 - BaseOS 4.9 MB/s | 7.9 MB 00:01 2023-11-25T04:20:31.879 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - AppStream 29 MB/s | 18 MB 00:00 2023-11-25T04:20:32.010 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - BaseOS 2.7 MB/s | 7.9 MB 00:02 2023-11-25T04:20:32.938 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - AppStream 10 MB/s | 18 MB 00:01 2023-11-25T04:20:34.819 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - AppStream 11 MB/s | 18 MB 00:01 2023-11-25T04:20:35.629 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - CRB 15 MB/s | 5.8 MB 00:00 2023-11-25T04:20:35.903 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS Stream 9 - AppStream 4.4 MB/s | 18 MB 00:04 2023-11-25T04:20:36.980 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - CRB 9.2 MB/s | 5.8 MB 00:00 2023-11-25T04:20:37.188 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - Extras packages 34 kB/s | 15 kB 00:00 2023-11-25T04:20:39.117 INFO:teuthology.orchestra.run.smithi092.stdout:Extra Packages for Enterprise Linux 11 MB/s | 20 MB 00:01 2023-11-25T04:20:39.389 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - Extras packages 14 kB/s | 15 kB 00:01 2023-11-25T04:20:39.713 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS Stream 9 - CRB 14 MB/s | 5.8 MB 00:00 2023-11-25T04:20:40.041 INFO:teuthology.orchestra.run.smithi076.stdout:Extra Packages for Enterprise Linux 39 MB/s | 20 MB 00:00 2023-11-25T04:20:41.947 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS Stream 9 - Extras packages 13 kB/s | 15 kB 00:01 2023-11-25T04:20:42.645 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - CRB 1.4 MB/s | 5.8 MB 00:04 2023-11-25T04:20:43.293 INFO:teuthology.orchestra.run.smithi195.stdout:Extra Packages for Enterprise Linux 16 MB/s | 20 MB 00:01 2023-11-25T04:20:43.923 INFO:teuthology.orchestra.run.smithi092.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2023-11-25T04:20:44.154 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - Extras packages 48 kB/s | 15 kB 00:00 2023-11-25T04:20:44.766 INFO:teuthology.orchestra.run.smithi076.stdout:lab-extras 36 kB/s | 1.7 kB 00:00 2023-11-25T04:20:44.856 INFO:teuthology.orchestra.run.smithi067.stdout:Extra Packages for Enterprise Linux 36 MB/s | 20 MB 00:00 2023-11-25T04:20:46.583 INFO:teuthology.orchestra.run.smithi092.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2023-11-25T04:20:46.584 INFO:teuthology.orchestra.run.smithi092.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2023-11-25T04:20:46.658 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2023-11-25T04:20:46.678 INFO:teuthology.orchestra.run.smithi092.stdout:============================================================================================ 2023-11-25T04:20:46.678 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2023-11-25T04:20:46.678 INFO:teuthology.orchestra.run.smithi092.stdout:============================================================================================ 2023-11-25T04:20:46.678 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2023-11-25T04:20:46.678 INFO:teuthology.orchestra.run.smithi092.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2023-11-25T04:20:46.678 INFO:teuthology.orchestra.run.smithi092.stdout: ceph x86_64 2:18.0.0-7506.gec928368.el9 ceph 6.6 k 2023-11-25T04:20:46.678 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-base x86_64 2:18.0.0-7506.gec928368.el9 ceph 5.2 M 2023-11-25T04:20:46.679 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-fuse x86_64 2:18.0.0-7506.gec928368.el9 ceph 859 k 2023-11-25T04:20:46.679 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-7506.gec928368.el9 ceph 141 k 2023-11-25T04:20:46.679 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr x86_64 2:18.0.0-7506.gec928368.el9 ceph 1.5 M 2023-11-25T04:20:46.679 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-cephadm noarch 2:18.0.0-7506.gec928368.el9 ceph-noarch 133 k 2023-11-25T04:20:46.679 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-dashboard noarch 2:18.0.0-7506.gec928368.el9 ceph-noarch 1.8 M 2023-11-25T04:20:46.679 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-7506.gec928368.el9 ceph-noarch 7.4 M 2023-11-25T04:20:46.679 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-rook noarch 2:18.0.0-7506.gec928368.el9 ceph-noarch 50 k 2023-11-25T04:20:46.679 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-radosgw x86_64 2:18.0.0-7506.gec928368.el9 ceph 8.7 M 2023-11-25T04:20:46.679 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-test x86_64 2:18.0.0-7506.gec928368.el9 ceph 41 M 2023-11-25T04:20:46.679 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-volume noarch 2:18.0.0-7506.gec928368.el9 ceph-noarch 262 k 2023-11-25T04:20:46.679 INFO:teuthology.orchestra.run.smithi092.stdout: cephadm noarch 2:18.0.0-7506.gec928368.el9 ceph-noarch 776 k 2023-11-25T04:20:46.679 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-libelf-devel x86_64 0.190-1.el9 CentOS-AppStream 21 k 2023-11-25T04:20:46.680 INFO:teuthology.orchestra.run.smithi092.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2023-11-25T04:20:46.680 INFO:teuthology.orchestra.run.smithi092.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2023-11-25T04:20:46.680 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs-devel x86_64 2:18.0.0-7506.gec928368.el9 ceph 31 k 2023-11-25T04:20:46.680 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs2 x86_64 2:18.0.0-7506.gec928368.el9 ceph 719 k 2023-11-25T04:20:46.680 INFO:teuthology.orchestra.run.smithi092.stdout: librados-devel x86_64 2:18.0.0-7506.gec928368.el9 ceph 126 k 2023-11-25T04:20:46.680 INFO:teuthology.orchestra.run.smithi092.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2023-11-25T04:20:46.680 INFO:teuthology.orchestra.run.smithi092.stdout: libuuid-devel x86_64 2.37.4-15.el9 CentOS-AppStream 24 k 2023-11-25T04:20:46.680 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-devel x86_64 1:3.0.7-25.el9 appstream 4.1 M 2023-11-25T04:20:46.680 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cephfs x86_64 2:18.0.0-7506.gec928368.el9 ceph 162 k 2023-11-25T04:20:46.680 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rados x86_64 2:18.0.0-7506.gec928368.el9 ceph 320 k 2023-11-25T04:20:46.680 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rbd x86_64 2:18.0.0-7506.gec928368.el9 ceph 298 k 2023-11-25T04:20:46.680 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rgw x86_64 2:18.0.0-7506.gec928368.el9 ceph 99 k 2023-11-25T04:20:46.681 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-fuse x86_64 2:18.0.0-7506.gec928368.el9 ceph 85 k 2023-11-25T04:20:46.681 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-mirror x86_64 2:18.0.0-7506.gec928368.el9 ceph 3.0 M 2023-11-25T04:20:46.681 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-nbd x86_64 2:18.0.0-7506.gec928368.el9 ceph 170 k 2023-11-25T04:20:46.681 INFO:teuthology.orchestra.run.smithi092.stdout: xfsprogs-devel x86_64 5.19.0-4.el9 CentOS-AppStream 85 k 2023-11-25T04:20:46.681 INFO:teuthology.orchestra.run.smithi092.stdout:Upgrading: 2023-11-25T04:20:46.681 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-debuginfod-client x86_64 0.190-1.el9 CentOS-BaseOS 36 k 2023-11-25T04:20:46.681 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-libelf x86_64 0.190-1.el9 CentOS-BaseOS 192 k 2023-11-25T04:20:46.681 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-libs x86_64 0.190-1.el9 CentOS-BaseOS 255 k 2023-11-25T04:20:46.681 INFO:teuthology.orchestra.run.smithi092.stdout: librados2 x86_64 2:18.0.0-7506.gec928368.el9 ceph 3.3 M 2023-11-25T04:20:46.681 INFO:teuthology.orchestra.run.smithi092.stdout: librbd1 x86_64 2:18.0.0-7506.gec928368.el9 ceph 3.0 M 2023-11-25T04:20:46.681 INFO:teuthology.orchestra.run.smithi092.stdout: openssl x86_64 1:3.0.7-25.el9 baseos 1.2 M 2023-11-25T04:20:46.681 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-libs x86_64 1:3.0.7-25.el9 baseos 2.2 M 2023-11-25T04:20:46.682 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2023-11-25T04:20:46.682 INFO:teuthology.orchestra.run.smithi092.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2023-11-25T04:20:46.682 INFO:teuthology.orchestra.run.smithi092.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-11-25T04:20:46.682 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-common x86_64 2:18.0.0-7506.gec928368.el9 ceph 19 M 2023-11-25T04:20:46.682 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-grafana-dashboards noarch 2:18.0.0-7506.gec928368.el9 ceph-noarch 28 k 2023-11-25T04:20:46.682 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mds x86_64 2:18.0.0-7506.gec928368.el9 ceph 2.1 M 2023-11-25T04:20:46.682 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-modules-core noarch 2:18.0.0-7506.gec928368.el9 ceph-noarch 243 k 2023-11-25T04:20:46.682 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mon x86_64 2:18.0.0-7506.gec928368.el9 ceph 4.4 M 2023-11-25T04:20:46.682 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-osd x86_64 2:18.0.0-7506.gec928368.el9 ceph 18 M 2023-11-25T04:20:46.682 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-prometheus-alerts noarch 2:18.0.0-7506.gec928368.el9 ceph-noarch 15 k 2023-11-25T04:20:46.682 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-selinux x86_64 2:18.0.0-7506.gec928368.el9 ceph 24 k 2023-11-25T04:20:46.683 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-11-25T04:20:46.683 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-11-25T04:20:46.683 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-11-25T04:20:46.683 INFO:teuthology.orchestra.run.smithi092.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-11-25T04:20:46.683 INFO:teuthology.orchestra.run.smithi092.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-11-25T04:20:46.683 INFO:teuthology.orchestra.run.smithi092.stdout: jq x86_64 1.6-16.el9 CentOS-AppStream 186 k 2023-11-25T04:20:46.683 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-11-25T04:20:46.683 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-11-25T04:20:46.683 INFO:teuthology.orchestra.run.smithi092.stdout: libcephsqlite x86_64 2:18.0.0-7506.gec928368.el9 ceph 163 k 2023-11-25T04:20:46.683 INFO:teuthology.orchestra.run.smithi092.stdout: libgfortran x86_64 11.4.1-2.3.el9 CentOS-BaseOS 803 k 2023-11-25T04:20:46.683 INFO:teuthology.orchestra.run.smithi092.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-11-25T04:20:46.683 INFO:teuthology.orchestra.run.smithi092.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-11-25T04:20:46.683 INFO:teuthology.orchestra.run.smithi092.stdout: libquadmath x86_64 11.4.1-2.3.el9 CentOS-BaseOS 195 k 2023-11-25T04:20:46.684 INFO:teuthology.orchestra.run.smithi092.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2023-11-25T04:20:46.684 INFO:teuthology.orchestra.run.smithi092.stdout: libradosstriper1 x86_64 2:18.0.0-7506.gec928368.el9 ceph 476 k 2023-11-25T04:20:46.684 INFO:teuthology.orchestra.run.smithi092.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-11-25T04:20:46.684 INFO:teuthology.orchestra.run.smithi092.stdout: librgw2 x86_64 2:18.0.0-7506.gec928368.el9 ceph 4.8 M 2023-11-25T04:20:46.684 INFO:teuthology.orchestra.run.smithi092.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-11-25T04:20:46.684 INFO:teuthology.orchestra.run.smithi092.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2023-11-25T04:20:46.684 INFO:teuthology.orchestra.run.smithi092.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-11-25T04:20:46.684 INFO:teuthology.orchestra.run.smithi092.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2023-11-25T04:20:46.684 INFO:teuthology.orchestra.run.smithi092.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2023-11-25T04:20:46.684 INFO:teuthology.orchestra.run.smithi092.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2023-11-25T04:20:46.684 INFO:teuthology.orchestra.run.smithi092.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-11-25T04:20:46.685 INFO:teuthology.orchestra.run.smithi092.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-11-25T04:20:46.685 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-11-25T04:20:46.685 INFO:teuthology.orchestra.run.smithi092.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-11-25T04:20:46.685 INFO:teuthology.orchestra.run.smithi092.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-11-25T04:20:46.685 INFO:teuthology.orchestra.run.smithi092.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-11-25T04:20:46.685 INFO:teuthology.orchestra.run.smithi092.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-11-25T04:20:46.685 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-11-25T04:20:46.685 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-argparse x86_64 2:18.0.0-7506.gec928368.el9 ceph 45 k 2023-11-25T04:20:46.685 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-common x86_64 2:18.0.0-7506.gec928368.el9 ceph 129 k 2023-11-25T04:20:46.685 INFO:teuthology.orchestra.run.smithi092.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-11-25T04:20:46.685 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-11-25T04:20:46.685 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-11-25T04:20:46.686 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-11-25T04:20:46.686 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2023-11-25T04:20:46.686 INFO:teuthology.orchestra.run.smithi092.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2023-11-25T04:20:46.686 INFO:teuthology.orchestra.run.smithi092.stdout: python3-google-auth noarch 1:2.23.4-1.el9 epel 197 k 2023-11-25T04:20:46.686 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-11-25T04:20:46.686 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-11-25T04:20:46.686 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-11-25T04:20:46.686 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-11-25T04:20:46.686 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-11-25T04:20:46.686 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-11-25T04:20:46.686 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-11-25T04:20:46.686 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-11-25T04:20:46.686 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-11-25T04:20:46.687 INFO:teuthology.orchestra.run.smithi092.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-11-25T04:20:46.687 INFO:teuthology.orchestra.run.smithi092.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-11-25T04:20:46.687 INFO:teuthology.orchestra.run.smithi092.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-11-25T04:20:46.687 INFO:teuthology.orchestra.run.smithi092.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-11-25T04:20:46.687 INFO:teuthology.orchestra.run.smithi092.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-11-25T04:20:46.687 INFO:teuthology.orchestra.run.smithi092.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-11-25T04:20:46.687 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-11-25T04:20:46.687 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-11-25T04:20:46.687 INFO:teuthology.orchestra.run.smithi092.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-11-25T04:20:46.687 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-11-25T04:20:46.687 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-11-25T04:20:46.687 INFO:teuthology.orchestra.run.smithi092.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-11-25T04:20:46.688 INFO:teuthology.orchestra.run.smithi092.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-11-25T04:20:46.688 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-11-25T04:20:46.688 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-11-25T04:20:46.688 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-11-25T04:20:46.688 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-11-25T04:20:46.688 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2023-11-25T04:20:46.688 INFO:teuthology.orchestra.run.smithi092.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-11-25T04:20:46.688 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2023-11-25T04:20:46.688 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-11-25T04:20:46.688 INFO:teuthology.orchestra.run.smithi092.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-11-25T04:20:46.688 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-11-25T04:20:46.688 INFO:teuthology.orchestra.run.smithi092.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-11-25T04:20:46.689 INFO:teuthology.orchestra.run.smithi092.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-11-25T04:20:46.689 INFO:teuthology.orchestra.run.smithi092.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-11-25T04:20:46.689 INFO:teuthology.orchestra.run.smithi092.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-11-25T04:20:46.689 INFO:teuthology.orchestra.run.smithi092.stdout: python3-urllib3 noarch 1.26.5-4.el9 CentOS-BaseOS 187 k 2023-11-25T04:20:46.689 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-11-25T04:20:46.689 INFO:teuthology.orchestra.run.smithi092.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-11-25T04:20:46.689 INFO:teuthology.orchestra.run.smithi092.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-11-25T04:20:46.689 INFO:teuthology.orchestra.run.smithi092.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-11-25T04:20:46.689 INFO:teuthology.orchestra.run.smithi092.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2023-11-25T04:20:46.689 INFO:teuthology.orchestra.run.smithi092.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-11-25T04:20:46.689 INFO:teuthology.orchestra.run.smithi092.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-11-25T04:20:46.690 INFO:teuthology.orchestra.run.smithi092.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-11-25T04:20:46.690 INFO:teuthology.orchestra.run.smithi092.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2023-11-25T04:20:46.690 INFO:teuthology.orchestra.run.smithi092.stdout:Installing weak dependencies: 2023-11-25T04:20:46.690 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-11-25T04:20:46.690 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-25T04:20:46.690 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2023-11-25T04:20:46.690 INFO:teuthology.orchestra.run.smithi092.stdout:============================================================================================ 2023-11-25T04:20:46.690 INFO:teuthology.orchestra.run.smithi092.stdout:Install 128 Packages 2023-11-25T04:20:46.690 INFO:teuthology.orchestra.run.smithi092.stdout:Upgrade 7 Packages 2023-11-25T04:20:46.690 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-25T04:20:46.691 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 191 M 2023-11-25T04:20:46.691 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2023-11-25T04:20:47.420 INFO:teuthology.orchestra.run.smithi076.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2023-11-25T04:20:47.421 INFO:teuthology.orchestra.run.smithi076.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2023-11-25T04:20:47.501 INFO:teuthology.orchestra.run.smithi076.stdout:Dependencies resolved. 2023-11-25T04:20:47.519 INFO:teuthology.orchestra.run.smithi076.stdout:============================================================================================ 2023-11-25T04:20:47.519 INFO:teuthology.orchestra.run.smithi076.stdout: Package Arch Version Repository Size 2023-11-25T04:20:47.519 INFO:teuthology.orchestra.run.smithi076.stdout:============================================================================================ 2023-11-25T04:20:47.519 INFO:teuthology.orchestra.run.smithi076.stdout:Installing: 2023-11-25T04:20:47.519 INFO:teuthology.orchestra.run.smithi076.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2023-11-25T04:20:47.519 INFO:teuthology.orchestra.run.smithi076.stdout: ceph x86_64 2:18.0.0-7506.gec928368.el9 ceph 6.6 k 2023-11-25T04:20:47.519 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-base x86_64 2:18.0.0-7506.gec928368.el9 ceph 5.2 M 2023-11-25T04:20:47.519 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-fuse x86_64 2:18.0.0-7506.gec928368.el9 ceph 859 k 2023-11-25T04:20:47.520 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-7506.gec928368.el9 ceph 141 k 2023-11-25T04:20:47.520 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr x86_64 2:18.0.0-7506.gec928368.el9 ceph 1.5 M 2023-11-25T04:20:47.520 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-cephadm noarch 2:18.0.0-7506.gec928368.el9 ceph-noarch 133 k 2023-11-25T04:20:47.520 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-dashboard noarch 2:18.0.0-7506.gec928368.el9 ceph-noarch 1.8 M 2023-11-25T04:20:47.520 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-7506.gec928368.el9 ceph-noarch 7.4 M 2023-11-25T04:20:47.520 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-rook noarch 2:18.0.0-7506.gec928368.el9 ceph-noarch 50 k 2023-11-25T04:20:47.520 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-radosgw x86_64 2:18.0.0-7506.gec928368.el9 ceph 8.7 M 2023-11-25T04:20:47.520 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-test x86_64 2:18.0.0-7506.gec928368.el9 ceph 41 M 2023-11-25T04:20:47.520 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-volume noarch 2:18.0.0-7506.gec928368.el9 ceph-noarch 262 k 2023-11-25T04:20:47.520 INFO:teuthology.orchestra.run.smithi076.stdout: cephadm noarch 2:18.0.0-7506.gec928368.el9 ceph-noarch 776 k 2023-11-25T04:20:47.520 INFO:teuthology.orchestra.run.smithi076.stdout: elfutils-libelf-devel x86_64 0.190-1.el9 CentOS-AppStream 21 k 2023-11-25T04:20:47.520 INFO:teuthology.orchestra.run.smithi076.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2023-11-25T04:20:47.520 INFO:teuthology.orchestra.run.smithi076.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2023-11-25T04:20:47.521 INFO:teuthology.orchestra.run.smithi076.stdout: libcephfs-devel x86_64 2:18.0.0-7506.gec928368.el9 ceph 31 k 2023-11-25T04:20:47.521 INFO:teuthology.orchestra.run.smithi076.stdout: libcephfs2 x86_64 2:18.0.0-7506.gec928368.el9 ceph 719 k 2023-11-25T04:20:47.521 INFO:teuthology.orchestra.run.smithi076.stdout: librados-devel x86_64 2:18.0.0-7506.gec928368.el9 ceph 126 k 2023-11-25T04:20:47.521 INFO:teuthology.orchestra.run.smithi076.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2023-11-25T04:20:47.521 INFO:teuthology.orchestra.run.smithi076.stdout: libuuid-devel x86_64 2.37.4-15.el9 CentOS-AppStream 24 k 2023-11-25T04:20:47.521 INFO:teuthology.orchestra.run.smithi076.stdout: openssl-devel x86_64 1:3.0.7-25.el9 appstream 4.1 M 2023-11-25T04:20:47.521 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cephfs x86_64 2:18.0.0-7506.gec928368.el9 ceph 162 k 2023-11-25T04:20:47.521 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rados x86_64 2:18.0.0-7506.gec928368.el9 ceph 320 k 2023-11-25T04:20:47.521 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rbd x86_64 2:18.0.0-7506.gec928368.el9 ceph 298 k 2023-11-25T04:20:47.521 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rgw x86_64 2:18.0.0-7506.gec928368.el9 ceph 99 k 2023-11-25T04:20:47.521 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-fuse x86_64 2:18.0.0-7506.gec928368.el9 ceph 85 k 2023-11-25T04:20:47.521 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-mirror x86_64 2:18.0.0-7506.gec928368.el9 ceph 3.0 M 2023-11-25T04:20:47.521 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-nbd x86_64 2:18.0.0-7506.gec928368.el9 ceph 170 k 2023-11-25T04:20:47.521 INFO:teuthology.orchestra.run.smithi076.stdout: xfsprogs-devel x86_64 5.19.0-4.el9 CentOS-AppStream 85 k 2023-11-25T04:20:47.522 INFO:teuthology.orchestra.run.smithi076.stdout:Upgrading: 2023-11-25T04:20:47.522 INFO:teuthology.orchestra.run.smithi076.stdout: elfutils-debuginfod-client x86_64 0.190-1.el9 CentOS-BaseOS 36 k 2023-11-25T04:20:47.522 INFO:teuthology.orchestra.run.smithi076.stdout: elfutils-libelf x86_64 0.190-1.el9 CentOS-BaseOS 192 k 2023-11-25T04:20:47.522 INFO:teuthology.orchestra.run.smithi076.stdout: elfutils-libs x86_64 0.190-1.el9 CentOS-BaseOS 255 k 2023-11-25T04:20:47.522 INFO:teuthology.orchestra.run.smithi076.stdout: librados2 x86_64 2:18.0.0-7506.gec928368.el9 ceph 3.3 M 2023-11-25T04:20:47.522 INFO:teuthology.orchestra.run.smithi076.stdout: librbd1 x86_64 2:18.0.0-7506.gec928368.el9 ceph 3.0 M 2023-11-25T04:20:47.522 INFO:teuthology.orchestra.run.smithi076.stdout: openssl x86_64 1:3.0.7-25.el9 baseos 1.2 M 2023-11-25T04:20:47.522 INFO:teuthology.orchestra.run.smithi076.stdout: openssl-libs x86_64 1:3.0.7-25.el9 baseos 2.2 M 2023-11-25T04:20:47.522 INFO:teuthology.orchestra.run.smithi076.stdout:Installing dependencies: 2023-11-25T04:20:47.522 INFO:teuthology.orchestra.run.smithi076.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2023-11-25T04:20:47.522 INFO:teuthology.orchestra.run.smithi076.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-11-25T04:20:47.522 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-common x86_64 2:18.0.0-7506.gec928368.el9 ceph 19 M 2023-11-25T04:20:47.523 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-grafana-dashboards noarch 2:18.0.0-7506.gec928368.el9 ceph-noarch 28 k 2023-11-25T04:20:47.523 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mds x86_64 2:18.0.0-7506.gec928368.el9 ceph 2.1 M 2023-11-25T04:20:47.523 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-modules-core noarch 2:18.0.0-7506.gec928368.el9 ceph-noarch 243 k 2023-11-25T04:20:47.523 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mon x86_64 2:18.0.0-7506.gec928368.el9 ceph 4.4 M 2023-11-25T04:20:47.523 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-osd x86_64 2:18.0.0-7506.gec928368.el9 ceph 18 M 2023-11-25T04:20:47.523 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-prometheus-alerts noarch 2:18.0.0-7506.gec928368.el9 ceph-noarch 15 k 2023-11-25T04:20:47.523 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-selinux x86_64 2:18.0.0-7506.gec928368.el9 ceph 24 k 2023-11-25T04:20:47.523 INFO:teuthology.orchestra.run.smithi076.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-11-25T04:20:47.523 INFO:teuthology.orchestra.run.smithi076.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-11-25T04:20:47.523 INFO:teuthology.orchestra.run.smithi076.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-11-25T04:20:47.523 INFO:teuthology.orchestra.run.smithi076.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-11-25T04:20:47.523 INFO:teuthology.orchestra.run.smithi076.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-11-25T04:20:47.523 INFO:teuthology.orchestra.run.smithi076.stdout: jq x86_64 1.6-16.el9 CentOS-AppStream 186 k 2023-11-25T04:20:47.524 INFO:teuthology.orchestra.run.smithi076.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-11-25T04:20:47.524 INFO:teuthology.orchestra.run.smithi076.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-11-25T04:20:47.524 INFO:teuthology.orchestra.run.smithi076.stdout: libcephsqlite x86_64 2:18.0.0-7506.gec928368.el9 ceph 163 k 2023-11-25T04:20:47.524 INFO:teuthology.orchestra.run.smithi076.stdout: libgfortran x86_64 11.4.1-2.3.el9 CentOS-BaseOS 803 k 2023-11-25T04:20:47.524 INFO:teuthology.orchestra.run.smithi076.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-11-25T04:20:47.524 INFO:teuthology.orchestra.run.smithi076.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-11-25T04:20:47.524 INFO:teuthology.orchestra.run.smithi076.stdout: libquadmath x86_64 11.4.1-2.3.el9 CentOS-BaseOS 195 k 2023-11-25T04:20:47.524 INFO:teuthology.orchestra.run.smithi076.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2023-11-25T04:20:47.524 INFO:teuthology.orchestra.run.smithi076.stdout: libradosstriper1 x86_64 2:18.0.0-7506.gec928368.el9 ceph 476 k 2023-11-25T04:20:47.524 INFO:teuthology.orchestra.run.smithi076.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-11-25T04:20:47.524 INFO:teuthology.orchestra.run.smithi076.stdout: librgw2 x86_64 2:18.0.0-7506.gec928368.el9 ceph 4.8 M 2023-11-25T04:20:47.524 INFO:teuthology.orchestra.run.smithi076.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-11-25T04:20:47.524 INFO:teuthology.orchestra.run.smithi076.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2023-11-25T04:20:47.524 INFO:teuthology.orchestra.run.smithi076.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-11-25T04:20:47.525 INFO:teuthology.orchestra.run.smithi076.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2023-11-25T04:20:47.525 INFO:teuthology.orchestra.run.smithi076.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2023-11-25T04:20:47.525 INFO:teuthology.orchestra.run.smithi076.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2023-11-25T04:20:47.525 INFO:teuthology.orchestra.run.smithi076.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-11-25T04:20:47.525 INFO:teuthology.orchestra.run.smithi076.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-11-25T04:20:47.525 INFO:teuthology.orchestra.run.smithi076.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-11-25T04:20:47.525 INFO:teuthology.orchestra.run.smithi076.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-11-25T04:20:47.525 INFO:teuthology.orchestra.run.smithi076.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-11-25T04:20:47.525 INFO:teuthology.orchestra.run.smithi076.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-11-25T04:20:47.525 INFO:teuthology.orchestra.run.smithi076.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-11-25T04:20:47.525 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-11-25T04:20:47.525 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ceph-argparse x86_64 2:18.0.0-7506.gec928368.el9 ceph 45 k 2023-11-25T04:20:47.525 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ceph-common x86_64 2:18.0.0-7506.gec928368.el9 ceph 129 k 2023-11-25T04:20:47.526 INFO:teuthology.orchestra.run.smithi076.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-11-25T04:20:47.526 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-11-25T04:20:47.526 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-11-25T04:20:47.526 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-11-25T04:20:47.526 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2023-11-25T04:20:47.526 INFO:teuthology.orchestra.run.smithi076.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2023-11-25T04:20:47.526 INFO:teuthology.orchestra.run.smithi076.stdout: python3-google-auth noarch 1:2.23.4-1.el9 epel 197 k 2023-11-25T04:20:47.526 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-11-25T04:20:47.526 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-11-25T04:20:47.526 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-11-25T04:20:47.526 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-11-25T04:20:47.526 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-11-25T04:20:47.526 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-11-25T04:20:47.527 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-11-25T04:20:47.527 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-11-25T04:20:47.527 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-11-25T04:20:47.527 INFO:teuthology.orchestra.run.smithi076.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-11-25T04:20:47.527 INFO:teuthology.orchestra.run.smithi076.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-11-25T04:20:47.527 INFO:teuthology.orchestra.run.smithi076.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-11-25T04:20:47.527 INFO:teuthology.orchestra.run.smithi076.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-11-25T04:20:47.527 INFO:teuthology.orchestra.run.smithi076.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-11-25T04:20:47.527 INFO:teuthology.orchestra.run.smithi076.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-11-25T04:20:47.527 INFO:teuthology.orchestra.run.smithi076.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-11-25T04:20:47.527 INFO:teuthology.orchestra.run.smithi076.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-11-25T04:20:47.527 INFO:teuthology.orchestra.run.smithi076.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-11-25T04:20:47.527 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-11-25T04:20:47.528 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-11-25T04:20:47.528 INFO:teuthology.orchestra.run.smithi076.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-11-25T04:20:47.528 INFO:teuthology.orchestra.run.smithi076.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-11-25T04:20:47.528 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-11-25T04:20:47.528 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-11-25T04:20:47.528 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-11-25T04:20:47.528 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-11-25T04:20:47.528 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2023-11-25T04:20:47.528 INFO:teuthology.orchestra.run.smithi076.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-11-25T04:20:47.528 INFO:teuthology.orchestra.run.smithi076.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2023-11-25T04:20:47.528 INFO:teuthology.orchestra.run.smithi076.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-11-25T04:20:47.528 INFO:teuthology.orchestra.run.smithi076.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-11-25T04:20:47.528 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-11-25T04:20:47.528 INFO:teuthology.orchestra.run.smithi076.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-11-25T04:20:47.529 INFO:teuthology.orchestra.run.smithi076.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-11-25T04:20:47.529 INFO:teuthology.orchestra.run.smithi076.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-11-25T04:20:47.529 INFO:teuthology.orchestra.run.smithi076.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-11-25T04:20:47.529 INFO:teuthology.orchestra.run.smithi076.stdout: python3-urllib3 noarch 1.26.5-4.el9 CentOS-BaseOS 187 k 2023-11-25T04:20:47.529 INFO:teuthology.orchestra.run.smithi076.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-11-25T04:20:47.529 INFO:teuthology.orchestra.run.smithi076.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-11-25T04:20:47.529 INFO:teuthology.orchestra.run.smithi076.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-11-25T04:20:47.529 INFO:teuthology.orchestra.run.smithi076.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-11-25T04:20:47.529 INFO:teuthology.orchestra.run.smithi076.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2023-11-25T04:20:47.529 INFO:teuthology.orchestra.run.smithi076.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-11-25T04:20:47.529 INFO:teuthology.orchestra.run.smithi076.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-11-25T04:20:47.529 INFO:teuthology.orchestra.run.smithi076.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-11-25T04:20:47.529 INFO:teuthology.orchestra.run.smithi076.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2023-11-25T04:20:47.529 INFO:teuthology.orchestra.run.smithi076.stdout:Installing weak dependencies: 2023-11-25T04:20:47.530 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-11-25T04:20:47.530 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-25T04:20:47.530 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction Summary 2023-11-25T04:20:47.530 INFO:teuthology.orchestra.run.smithi076.stdout:============================================================================================ 2023-11-25T04:20:47.530 INFO:teuthology.orchestra.run.smithi076.stdout:Install 128 Packages 2023-11-25T04:20:47.530 INFO:teuthology.orchestra.run.smithi076.stdout:Upgrade 7 Packages 2023-11-25T04:20:47.530 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-25T04:20:47.531 INFO:teuthology.orchestra.run.smithi076.stdout:Total download size: 191 M 2023-11-25T04:20:47.531 INFO:teuthology.orchestra.run.smithi076.stdout:Downloading Packages: 2023-11-25T04:20:47.724 INFO:teuthology.orchestra.run.smithi092.stdout:(1/135): ceph-18.0.0-7506.gec928368.el9.x86_64. 27 kB/s | 6.6 kB 00:00 2023-11-25T04:20:47.948 INFO:teuthology.orchestra.run.smithi195.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2023-11-25T04:20:48.141 INFO:teuthology.orchestra.run.smithi092.stdout:(2/135): ceph-base-18.0.0-7506.gec928368.el9.x8 7.9 MB/s | 5.2 MB 00:00 2023-11-25T04:20:48.199 INFO:teuthology.orchestra.run.smithi092.stdout:(3/135): ceph-immutable-object-cache-18.0.0-750 2.4 MB/s | 141 kB 00:00 2023-11-25T04:20:48.384 INFO:teuthology.orchestra.run.smithi076.stdout:(1/135): ceph-18.0.0-7506.gec928368.el9.x86_64. 23 kB/s | 6.6 kB 00:00 2023-11-25T04:20:48.433 INFO:teuthology.orchestra.run.smithi092.stdout:(4/135): ceph-mds-18.0.0-7506.gec928368.el9.x86 8.9 MB/s | 2.1 MB 00:00 2023-11-25T04:20:48.675 INFO:teuthology.orchestra.run.smithi092.stdout:(5/135): ceph-mgr-18.0.0-7506.gec928368.el9.x86 6.1 MB/s | 1.5 MB 00:00 2023-11-25T04:20:48.742 INFO:teuthology.orchestra.run.smithi092.stdout:(6/135): ceph-fuse-18.0.0-7506.gec928368.el9.x8 845 kB/s | 859 kB 00:01 2023-11-25T04:20:48.950 INFO:teuthology.orchestra.run.smithi092.stdout:(7/135): ceph-common-18.0.0-7506.gec928368.el9. 13 MB/s | 19 MB 00:01 2023-11-25T04:20:48.959 INFO:teuthology.orchestra.run.smithi076.stdout:(2/135): ceph-fuse-18.0.0-7506.gec928368.el9.x8 1.5 MB/s | 859 kB 00:00 2023-11-25T04:20:49.084 INFO:teuthology.orchestra.run.smithi076.stdout:(3/135): ceph-immutable-object-cache-18.0.0-750 1.1 MB/s | 141 kB 00:00 2023-11-25T04:20:49.550 INFO:teuthology.orchestra.run.smithi092.stdout:(8/135): ceph-radosgw-18.0.0-7506.gec928368.el9 14 MB/s | 8.7 MB 00:00 2023-11-25T04:20:49.592 INFO:teuthology.orchestra.run.smithi092.stdout:(9/135): ceph-selinux-18.0.0-7506.gec928368.el9 580 kB/s | 24 kB 00:00 2023-11-25T04:20:49.651 INFO:teuthology.orchestra.run.smithi076.stdout:(4/135): ceph-base-18.0.0-7506.gec928368.el9.x8 3.4 MB/s | 5.2 MB 00:01 2023-11-25T04:20:49.835 INFO:teuthology.orchestra.run.smithi076.stdout:(5/135): ceph-mds-18.0.0-7506.gec928368.el9.x86 2.8 MB/s | 2.1 MB 00:00 2023-11-25T04:20:49.842 INFO:teuthology.orchestra.run.smithi092.stdout:(10/135): ceph-mon-18.0.0-7506.gec928368.el9.x8 3.8 MB/s | 4.4 MB 00:01 2023-11-25T04:20:49.898 INFO:teuthology.orchestra.run.smithi067.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2023-11-25T04:20:49.918 INFO:teuthology.orchestra.run.smithi092.stdout:(11/135): libcephfs-devel-18.0.0-7506.gec928368 416 kB/s | 31 kB 00:00 2023-11-25T04:20:49.960 INFO:teuthology.orchestra.run.smithi076.stdout:(6/135): ceph-mgr-18.0.0-7506.gec928368.el9.x86 4.7 MB/s | 1.5 MB 00:00 2023-11-25T04:20:50.151 INFO:teuthology.orchestra.run.smithi092.stdout:(12/135): libcephfs2-18.0.0-7506.gec928368.el9. 3.0 MB/s | 719 kB 00:00 2023-11-25T04:20:50.252 INFO:teuthology.orchestra.run.smithi092.stdout:(13/135): libcephsqlite-18.0.0-7506.gec928368.e 1.6 MB/s | 163 kB 00:00 2023-11-25T04:20:50.369 INFO:teuthology.orchestra.run.smithi092.stdout:(14/135): librados-devel-18.0.0-7506.gec928368. 1.1 MB/s | 126 kB 00:00 2023-11-25T04:20:50.544 INFO:teuthology.orchestra.run.smithi092.stdout:(15/135): libradosstriper1-18.0.0-7506.gec92836 2.7 MB/s | 476 kB 00:00 2023-11-25T04:20:50.603 INFO:teuthology.orchestra.run.smithi195.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2023-11-25T04:20:50.604 INFO:teuthology.orchestra.run.smithi195.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2023-11-25T04:20:50.683 INFO:teuthology.orchestra.run.smithi195.stdout:Dependencies resolved. 2023-11-25T04:20:50.705 INFO:teuthology.orchestra.run.smithi195.stdout:============================================================================================ 2023-11-25T04:20:50.705 INFO:teuthology.orchestra.run.smithi195.stdout: Package Arch Version Repository Size 2023-11-25T04:20:50.705 INFO:teuthology.orchestra.run.smithi195.stdout:============================================================================================ 2023-11-25T04:20:50.705 INFO:teuthology.orchestra.run.smithi195.stdout:Installing: 2023-11-25T04:20:50.705 INFO:teuthology.orchestra.run.smithi195.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2023-11-25T04:20:50.705 INFO:teuthology.orchestra.run.smithi195.stdout: ceph x86_64 2:18.0.0-7506.gec928368.el9 ceph 6.6 k 2023-11-25T04:20:50.706 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-base x86_64 2:18.0.0-7506.gec928368.el9 ceph 5.2 M 2023-11-25T04:20:50.706 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-fuse x86_64 2:18.0.0-7506.gec928368.el9 ceph 859 k 2023-11-25T04:20:50.706 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-7506.gec928368.el9 ceph 141 k 2023-11-25T04:20:50.706 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr x86_64 2:18.0.0-7506.gec928368.el9 ceph 1.5 M 2023-11-25T04:20:50.706 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-cephadm noarch 2:18.0.0-7506.gec928368.el9 ceph-noarch 133 k 2023-11-25T04:20:50.706 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-dashboard noarch 2:18.0.0-7506.gec928368.el9 ceph-noarch 1.8 M 2023-11-25T04:20:50.706 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-7506.gec928368.el9 ceph-noarch 7.4 M 2023-11-25T04:20:50.706 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-rook noarch 2:18.0.0-7506.gec928368.el9 ceph-noarch 50 k 2023-11-25T04:20:50.706 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-radosgw x86_64 2:18.0.0-7506.gec928368.el9 ceph 8.7 M 2023-11-25T04:20:50.706 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-test x86_64 2:18.0.0-7506.gec928368.el9 ceph 41 M 2023-11-25T04:20:50.706 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-volume noarch 2:18.0.0-7506.gec928368.el9 ceph-noarch 262 k 2023-11-25T04:20:50.707 INFO:teuthology.orchestra.run.smithi195.stdout: cephadm noarch 2:18.0.0-7506.gec928368.el9 ceph-noarch 776 k 2023-11-25T04:20:50.707 INFO:teuthology.orchestra.run.smithi195.stdout: elfutils-libelf-devel x86_64 0.190-1.el9 CentOS-AppStream 21 k 2023-11-25T04:20:50.707 INFO:teuthology.orchestra.run.smithi195.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2023-11-25T04:20:50.707 INFO:teuthology.orchestra.run.smithi195.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2023-11-25T04:20:50.707 INFO:teuthology.orchestra.run.smithi195.stdout: libcephfs-devel x86_64 2:18.0.0-7506.gec928368.el9 ceph 31 k 2023-11-25T04:20:50.707 INFO:teuthology.orchestra.run.smithi195.stdout: libcephfs2 x86_64 2:18.0.0-7506.gec928368.el9 ceph 719 k 2023-11-25T04:20:50.707 INFO:teuthology.orchestra.run.smithi195.stdout: librados-devel x86_64 2:18.0.0-7506.gec928368.el9 ceph 126 k 2023-11-25T04:20:50.707 INFO:teuthology.orchestra.run.smithi195.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2023-11-25T04:20:50.707 INFO:teuthology.orchestra.run.smithi195.stdout: libuuid-devel x86_64 2.37.4-15.el9 CentOS-AppStream 24 k 2023-11-25T04:20:50.707 INFO:teuthology.orchestra.run.smithi195.stdout: openssl-devel x86_64 1:3.0.7-25.el9 appstream 4.1 M 2023-11-25T04:20:50.707 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cephfs x86_64 2:18.0.0-7506.gec928368.el9 ceph 162 k 2023-11-25T04:20:50.707 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rados x86_64 2:18.0.0-7506.gec928368.el9 ceph 320 k 2023-11-25T04:20:50.708 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rbd x86_64 2:18.0.0-7506.gec928368.el9 ceph 298 k 2023-11-25T04:20:50.708 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rgw x86_64 2:18.0.0-7506.gec928368.el9 ceph 99 k 2023-11-25T04:20:50.708 INFO:teuthology.orchestra.run.smithi195.stdout: rbd-fuse x86_64 2:18.0.0-7506.gec928368.el9 ceph 85 k 2023-11-25T04:20:50.708 INFO:teuthology.orchestra.run.smithi195.stdout: rbd-mirror x86_64 2:18.0.0-7506.gec928368.el9 ceph 3.0 M 2023-11-25T04:20:50.708 INFO:teuthology.orchestra.run.smithi195.stdout: rbd-nbd x86_64 2:18.0.0-7506.gec928368.el9 ceph 170 k 2023-11-25T04:20:50.708 INFO:teuthology.orchestra.run.smithi195.stdout: xfsprogs-devel x86_64 5.19.0-4.el9 CentOS-AppStream 85 k 2023-11-25T04:20:50.708 INFO:teuthology.orchestra.run.smithi195.stdout:Upgrading: 2023-11-25T04:20:50.708 INFO:teuthology.orchestra.run.smithi195.stdout: elfutils-debuginfod-client x86_64 0.190-1.el9 CentOS-BaseOS 36 k 2023-11-25T04:20:50.708 INFO:teuthology.orchestra.run.smithi195.stdout: elfutils-libelf x86_64 0.190-1.el9 CentOS-BaseOS 192 k 2023-11-25T04:20:50.708 INFO:teuthology.orchestra.run.smithi195.stdout: elfutils-libs x86_64 0.190-1.el9 CentOS-BaseOS 255 k 2023-11-25T04:20:50.709 INFO:teuthology.orchestra.run.smithi195.stdout: librados2 x86_64 2:18.0.0-7506.gec928368.el9 ceph 3.3 M 2023-11-25T04:20:50.709 INFO:teuthology.orchestra.run.smithi195.stdout: librbd1 x86_64 2:18.0.0-7506.gec928368.el9 ceph 3.0 M 2023-11-25T04:20:50.709 INFO:teuthology.orchestra.run.smithi195.stdout: openssl x86_64 1:3.0.7-25.el9 baseos 1.2 M 2023-11-25T04:20:50.709 INFO:teuthology.orchestra.run.smithi195.stdout: openssl-libs x86_64 1:3.0.7-25.el9 baseos 2.2 M 2023-11-25T04:20:50.709 INFO:teuthology.orchestra.run.smithi195.stdout:Installing dependencies: 2023-11-25T04:20:50.709 INFO:teuthology.orchestra.run.smithi195.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2023-11-25T04:20:50.709 INFO:teuthology.orchestra.run.smithi195.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-11-25T04:20:50.709 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-common x86_64 2:18.0.0-7506.gec928368.el9 ceph 19 M 2023-11-25T04:20:50.709 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-grafana-dashboards noarch 2:18.0.0-7506.gec928368.el9 ceph-noarch 28 k 2023-11-25T04:20:50.709 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mds x86_64 2:18.0.0-7506.gec928368.el9 ceph 2.1 M 2023-11-25T04:20:50.709 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-modules-core noarch 2:18.0.0-7506.gec928368.el9 ceph-noarch 243 k 2023-11-25T04:20:50.709 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mon x86_64 2:18.0.0-7506.gec928368.el9 ceph 4.4 M 2023-11-25T04:20:50.710 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-osd x86_64 2:18.0.0-7506.gec928368.el9 ceph 18 M 2023-11-25T04:20:50.710 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-prometheus-alerts noarch 2:18.0.0-7506.gec928368.el9 ceph-noarch 15 k 2023-11-25T04:20:50.710 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-selinux x86_64 2:18.0.0-7506.gec928368.el9 ceph 24 k 2023-11-25T04:20:50.710 INFO:teuthology.orchestra.run.smithi195.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-11-25T04:20:50.710 INFO:teuthology.orchestra.run.smithi195.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-11-25T04:20:50.710 INFO:teuthology.orchestra.run.smithi195.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-11-25T04:20:50.710 INFO:teuthology.orchestra.run.smithi195.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-11-25T04:20:50.710 INFO:teuthology.orchestra.run.smithi195.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-11-25T04:20:50.710 INFO:teuthology.orchestra.run.smithi195.stdout: jq x86_64 1.6-16.el9 CentOS-AppStream 186 k 2023-11-25T04:20:50.710 INFO:teuthology.orchestra.run.smithi195.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-11-25T04:20:50.710 INFO:teuthology.orchestra.run.smithi195.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-11-25T04:20:50.710 INFO:teuthology.orchestra.run.smithi195.stdout: libcephsqlite x86_64 2:18.0.0-7506.gec928368.el9 ceph 163 k 2023-11-25T04:20:50.711 INFO:teuthology.orchestra.run.smithi195.stdout: libgfortran x86_64 11.4.1-2.3.el9 CentOS-BaseOS 803 k 2023-11-25T04:20:50.711 INFO:teuthology.orchestra.run.smithi195.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-11-25T04:20:50.711 INFO:teuthology.orchestra.run.smithi195.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-11-25T04:20:50.711 INFO:teuthology.orchestra.run.smithi195.stdout: libquadmath x86_64 11.4.1-2.3.el9 CentOS-BaseOS 195 k 2023-11-25T04:20:50.711 INFO:teuthology.orchestra.run.smithi195.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2023-11-25T04:20:50.711 INFO:teuthology.orchestra.run.smithi195.stdout: libradosstriper1 x86_64 2:18.0.0-7506.gec928368.el9 ceph 476 k 2023-11-25T04:20:50.711 INFO:teuthology.orchestra.run.smithi195.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-11-25T04:20:50.711 INFO:teuthology.orchestra.run.smithi195.stdout: librgw2 x86_64 2:18.0.0-7506.gec928368.el9 ceph 4.8 M 2023-11-25T04:20:50.711 INFO:teuthology.orchestra.run.smithi195.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-11-25T04:20:50.711 INFO:teuthology.orchestra.run.smithi195.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2023-11-25T04:20:50.711 INFO:teuthology.orchestra.run.smithi195.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-11-25T04:20:50.711 INFO:teuthology.orchestra.run.smithi195.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2023-11-25T04:20:50.711 INFO:teuthology.orchestra.run.smithi195.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2023-11-25T04:20:50.712 INFO:teuthology.orchestra.run.smithi195.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2023-11-25T04:20:50.712 INFO:teuthology.orchestra.run.smithi195.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-11-25T04:20:50.712 INFO:teuthology.orchestra.run.smithi195.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-11-25T04:20:50.712 INFO:teuthology.orchestra.run.smithi195.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-11-25T04:20:50.712 INFO:teuthology.orchestra.run.smithi195.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-11-25T04:20:50.712 INFO:teuthology.orchestra.run.smithi195.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-11-25T04:20:50.712 INFO:teuthology.orchestra.run.smithi195.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-11-25T04:20:50.712 INFO:teuthology.orchestra.run.smithi195.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-11-25T04:20:50.712 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-11-25T04:20:50.712 INFO:teuthology.orchestra.run.smithi195.stdout: python3-ceph-argparse x86_64 2:18.0.0-7506.gec928368.el9 ceph 45 k 2023-11-25T04:20:50.712 INFO:teuthology.orchestra.run.smithi195.stdout: python3-ceph-common x86_64 2:18.0.0-7506.gec928368.el9 ceph 129 k 2023-11-25T04:20:50.712 INFO:teuthology.orchestra.run.smithi195.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-11-25T04:20:50.712 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-11-25T04:20:50.713 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-11-25T04:20:50.713 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-11-25T04:20:50.713 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2023-11-25T04:20:50.713 INFO:teuthology.orchestra.run.smithi195.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2023-11-25T04:20:50.713 INFO:teuthology.orchestra.run.smithi195.stdout: python3-google-auth noarch 1:2.23.4-1.el9 epel 197 k 2023-11-25T04:20:50.713 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-11-25T04:20:50.713 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-11-25T04:20:50.713 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-11-25T04:20:50.713 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-11-25T04:20:50.713 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-11-25T04:20:50.713 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-11-25T04:20:50.714 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-11-25T04:20:50.714 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-11-25T04:20:50.714 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-11-25T04:20:50.714 INFO:teuthology.orchestra.run.smithi195.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-11-25T04:20:50.714 INFO:teuthology.orchestra.run.smithi195.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-11-25T04:20:50.714 INFO:teuthology.orchestra.run.smithi195.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-11-25T04:20:50.714 INFO:teuthology.orchestra.run.smithi195.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-11-25T04:20:50.714 INFO:teuthology.orchestra.run.smithi195.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-11-25T04:20:50.714 INFO:teuthology.orchestra.run.smithi195.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-11-25T04:20:50.714 INFO:teuthology.orchestra.run.smithi195.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-11-25T04:20:50.714 INFO:teuthology.orchestra.run.smithi195.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-11-25T04:20:50.714 INFO:teuthology.orchestra.run.smithi195.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-11-25T04:20:50.714 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-11-25T04:20:50.715 INFO:teuthology.orchestra.run.smithi195.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-11-25T04:20:50.715 INFO:teuthology.orchestra.run.smithi195.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-11-25T04:20:50.715 INFO:teuthology.orchestra.run.smithi195.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-11-25T04:20:50.715 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-11-25T04:20:50.715 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-11-25T04:20:50.715 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-11-25T04:20:50.715 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-11-25T04:20:50.715 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2023-11-25T04:20:50.715 INFO:teuthology.orchestra.run.smithi195.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-11-25T04:20:50.715 INFO:teuthology.orchestra.run.smithi195.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2023-11-25T04:20:50.715 INFO:teuthology.orchestra.run.smithi195.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-11-25T04:20:50.715 INFO:teuthology.orchestra.run.smithi195.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-11-25T04:20:50.715 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-11-25T04:20:50.716 INFO:teuthology.orchestra.run.smithi195.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-11-25T04:20:50.716 INFO:teuthology.orchestra.run.smithi195.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-11-25T04:20:50.716 INFO:teuthology.orchestra.run.smithi195.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-11-25T04:20:50.716 INFO:teuthology.orchestra.run.smithi195.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-11-25T04:20:50.716 INFO:teuthology.orchestra.run.smithi195.stdout: python3-urllib3 noarch 1.26.5-4.el9 CentOS-BaseOS 187 k 2023-11-25T04:20:50.716 INFO:teuthology.orchestra.run.smithi195.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-11-25T04:20:50.716 INFO:teuthology.orchestra.run.smithi195.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-11-25T04:20:50.716 INFO:teuthology.orchestra.run.smithi195.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-11-25T04:20:50.716 INFO:teuthology.orchestra.run.smithi195.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-11-25T04:20:50.716 INFO:teuthology.orchestra.run.smithi195.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2023-11-25T04:20:50.716 INFO:teuthology.orchestra.run.smithi195.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-11-25T04:20:50.716 INFO:teuthology.orchestra.run.smithi195.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-11-25T04:20:50.717 INFO:teuthology.orchestra.run.smithi195.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-11-25T04:20:50.717 INFO:teuthology.orchestra.run.smithi195.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2023-11-25T04:20:50.717 INFO:teuthology.orchestra.run.smithi195.stdout:Installing weak dependencies: 2023-11-25T04:20:50.717 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-11-25T04:20:50.717 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-25T04:20:50.717 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction Summary 2023-11-25T04:20:50.717 INFO:teuthology.orchestra.run.smithi195.stdout:============================================================================================ 2023-11-25T04:20:50.717 INFO:teuthology.orchestra.run.smithi195.stdout:Install 128 Packages 2023-11-25T04:20:50.717 INFO:teuthology.orchestra.run.smithi195.stdout:Upgrade 7 Packages 2023-11-25T04:20:50.717 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-25T04:20:50.718 INFO:teuthology.orchestra.run.smithi195.stdout:Total download size: 191 M 2023-11-25T04:20:50.718 INFO:teuthology.orchestra.run.smithi195.stdout:Downloading Packages: 2023-11-25T04:20:51.202 INFO:teuthology.orchestra.run.smithi076.stdout:(7/135): ceph-mon-18.0.0-7506.gec928368.el9.x86 3.2 MB/s | 4.4 MB 00:01 2023-11-25T04:20:51.525 INFO:teuthology.orchestra.run.smithi195.stdout:(1/135): ceph-18.0.0-7506.gec928368.el9.x86_64. 15 kB/s | 6.6 kB 00:00 2023-11-25T04:20:51.544 INFO:teuthology.orchestra.run.smithi092.stdout:(16/135): librgw2-18.0.0-7506.gec928368.el9.x86 4.8 MB/s | 4.8 MB 00:01 2023-11-25T04:20:51.627 INFO:teuthology.orchestra.run.smithi092.stdout:(17/135): python3-ceph-argparse-18.0.0-7506.gec 544 kB/s | 45 kB 00:00 2023-11-25T04:20:51.728 INFO:teuthology.orchestra.run.smithi092.stdout:(18/135): python3-ceph-common-18.0.0-7506.gec92 1.3 MB/s | 129 kB 00:00 2023-11-25T04:20:51.828 INFO:teuthology.orchestra.run.smithi092.stdout:(19/135): python3-cephfs-18.0.0-7506.gec928368. 1.6 MB/s | 162 kB 00:00 2023-11-25T04:20:51.939 INFO:teuthology.orchestra.run.smithi092.stdout:(20/135): python3-rados-18.0.0-7506.gec928368.e 2.9 MB/s | 320 kB 00:00 2023-11-25T04:20:52.037 INFO:teuthology.orchestra.run.smithi092.stdout:(21/135): python3-rbd-18.0.0-7506.gec928368.el9 2.9 MB/s | 298 kB 00:00 2023-11-25T04:20:52.120 INFO:teuthology.orchestra.run.smithi092.stdout:(22/135): python3-rgw-18.0.0-7506.gec928368.el9 1.2 MB/s | 99 kB 00:00 2023-11-25T04:20:52.204 INFO:teuthology.orchestra.run.smithi092.stdout:(23/135): rbd-fuse-18.0.0-7506.gec928368.el9.x8 1.0 MB/s | 85 kB 00:00 2023-11-25T04:20:52.667 INFO:teuthology.orchestra.run.smithi067.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2023-11-25T04:20:52.667 INFO:teuthology.orchestra.run.smithi067.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2023-11-25T04:20:52.749 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2023-11-25T04:20:52.764 INFO:teuthology.orchestra.run.smithi067.stdout:============================================================================================ 2023-11-25T04:20:52.764 INFO:teuthology.orchestra.run.smithi067.stdout: Package Arch Version Repository Size 2023-11-25T04:20:52.764 INFO:teuthology.orchestra.run.smithi067.stdout:============================================================================================ 2023-11-25T04:20:52.765 INFO:teuthology.orchestra.run.smithi067.stdout:Installing: 2023-11-25T04:20:52.765 INFO:teuthology.orchestra.run.smithi067.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2023-11-25T04:20:52.765 INFO:teuthology.orchestra.run.smithi067.stdout: ceph x86_64 2:18.0.0-7506.gec928368.el9 ceph 6.6 k 2023-11-25T04:20:52.765 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-base x86_64 2:18.0.0-7506.gec928368.el9 ceph 5.2 M 2023-11-25T04:20:52.765 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-fuse x86_64 2:18.0.0-7506.gec928368.el9 ceph 859 k 2023-11-25T04:20:52.765 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-7506.gec928368.el9 ceph 141 k 2023-11-25T04:20:52.765 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr x86_64 2:18.0.0-7506.gec928368.el9 ceph 1.5 M 2023-11-25T04:20:52.765 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-cephadm noarch 2:18.0.0-7506.gec928368.el9 ceph-noarch 133 k 2023-11-25T04:20:52.765 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-dashboard noarch 2:18.0.0-7506.gec928368.el9 ceph-noarch 1.8 M 2023-11-25T04:20:52.765 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-7506.gec928368.el9 ceph-noarch 7.4 M 2023-11-25T04:20:52.765 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-rook noarch 2:18.0.0-7506.gec928368.el9 ceph-noarch 50 k 2023-11-25T04:20:52.765 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-radosgw x86_64 2:18.0.0-7506.gec928368.el9 ceph 8.7 M 2023-11-25T04:20:52.765 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-test x86_64 2:18.0.0-7506.gec928368.el9 ceph 41 M 2023-11-25T04:20:52.766 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-volume noarch 2:18.0.0-7506.gec928368.el9 ceph-noarch 262 k 2023-11-25T04:20:52.766 INFO:teuthology.orchestra.run.smithi067.stdout: cephadm noarch 2:18.0.0-7506.gec928368.el9 ceph-noarch 776 k 2023-11-25T04:20:52.766 INFO:teuthology.orchestra.run.smithi067.stdout: elfutils-libelf-devel x86_64 0.190-1.el9 CentOS-AppStream 21 k 2023-11-25T04:20:52.766 INFO:teuthology.orchestra.run.smithi067.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2023-11-25T04:20:52.766 INFO:teuthology.orchestra.run.smithi067.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2023-11-25T04:20:52.766 INFO:teuthology.orchestra.run.smithi067.stdout: libcephfs-devel x86_64 2:18.0.0-7506.gec928368.el9 ceph 31 k 2023-11-25T04:20:52.766 INFO:teuthology.orchestra.run.smithi067.stdout: libcephfs2 x86_64 2:18.0.0-7506.gec928368.el9 ceph 719 k 2023-11-25T04:20:52.766 INFO:teuthology.orchestra.run.smithi067.stdout: librados-devel x86_64 2:18.0.0-7506.gec928368.el9 ceph 126 k 2023-11-25T04:20:52.766 INFO:teuthology.orchestra.run.smithi067.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2023-11-25T04:20:52.766 INFO:teuthology.orchestra.run.smithi067.stdout: libuuid-devel x86_64 2.37.4-15.el9 CentOS-AppStream 24 k 2023-11-25T04:20:52.766 INFO:teuthology.orchestra.run.smithi067.stdout: openssl-devel x86_64 1:3.0.7-25.el9 appstream 4.1 M 2023-11-25T04:20:52.766 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cephfs x86_64 2:18.0.0-7506.gec928368.el9 ceph 162 k 2023-11-25T04:20:52.766 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rados x86_64 2:18.0.0-7506.gec928368.el9 ceph 320 k 2023-11-25T04:20:52.767 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rbd x86_64 2:18.0.0-7506.gec928368.el9 ceph 298 k 2023-11-25T04:20:52.767 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rgw x86_64 2:18.0.0-7506.gec928368.el9 ceph 99 k 2023-11-25T04:20:52.767 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-fuse x86_64 2:18.0.0-7506.gec928368.el9 ceph 85 k 2023-11-25T04:20:52.767 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-mirror x86_64 2:18.0.0-7506.gec928368.el9 ceph 3.0 M 2023-11-25T04:20:52.767 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-nbd x86_64 2:18.0.0-7506.gec928368.el9 ceph 170 k 2023-11-25T04:20:52.767 INFO:teuthology.orchestra.run.smithi067.stdout: xfsprogs-devel x86_64 5.19.0-4.el9 CentOS-AppStream 85 k 2023-11-25T04:20:52.767 INFO:teuthology.orchestra.run.smithi067.stdout:Upgrading: 2023-11-25T04:20:52.767 INFO:teuthology.orchestra.run.smithi067.stdout: elfutils-debuginfod-client x86_64 0.190-1.el9 CentOS-BaseOS 36 k 2023-11-25T04:20:52.767 INFO:teuthology.orchestra.run.smithi067.stdout: elfutils-libelf x86_64 0.190-1.el9 CentOS-BaseOS 192 k 2023-11-25T04:20:52.767 INFO:teuthology.orchestra.run.smithi067.stdout: elfutils-libs x86_64 0.190-1.el9 CentOS-BaseOS 255 k 2023-11-25T04:20:52.767 INFO:teuthology.orchestra.run.smithi067.stdout: librados2 x86_64 2:18.0.0-7506.gec928368.el9 ceph 3.3 M 2023-11-25T04:20:52.767 INFO:teuthology.orchestra.run.smithi067.stdout: librbd1 x86_64 2:18.0.0-7506.gec928368.el9 ceph 3.0 M 2023-11-25T04:20:52.767 INFO:teuthology.orchestra.run.smithi067.stdout: openssl x86_64 1:3.0.7-25.el9 baseos 1.2 M 2023-11-25T04:20:52.767 INFO:teuthology.orchestra.run.smithi067.stdout: openssl-libs x86_64 1:3.0.7-25.el9 baseos 2.2 M 2023-11-25T04:20:52.768 INFO:teuthology.orchestra.run.smithi067.stdout:Installing dependencies: 2023-11-25T04:20:52.768 INFO:teuthology.orchestra.run.smithi067.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2023-11-25T04:20:52.768 INFO:teuthology.orchestra.run.smithi067.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-11-25T04:20:52.768 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-common x86_64 2:18.0.0-7506.gec928368.el9 ceph 19 M 2023-11-25T04:20:52.768 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-grafana-dashboards noarch 2:18.0.0-7506.gec928368.el9 ceph-noarch 28 k 2023-11-25T04:20:52.768 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mds x86_64 2:18.0.0-7506.gec928368.el9 ceph 2.1 M 2023-11-25T04:20:52.768 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-modules-core noarch 2:18.0.0-7506.gec928368.el9 ceph-noarch 243 k 2023-11-25T04:20:52.768 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mon x86_64 2:18.0.0-7506.gec928368.el9 ceph 4.4 M 2023-11-25T04:20:52.768 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-osd x86_64 2:18.0.0-7506.gec928368.el9 ceph 18 M 2023-11-25T04:20:52.768 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-prometheus-alerts noarch 2:18.0.0-7506.gec928368.el9 ceph-noarch 15 k 2023-11-25T04:20:52.768 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-selinux x86_64 2:18.0.0-7506.gec928368.el9 ceph 24 k 2023-11-25T04:20:52.768 INFO:teuthology.orchestra.run.smithi067.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-11-25T04:20:52.768 INFO:teuthology.orchestra.run.smithi067.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-11-25T04:20:52.769 INFO:teuthology.orchestra.run.smithi067.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-11-25T04:20:52.769 INFO:teuthology.orchestra.run.smithi067.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-11-25T04:20:52.769 INFO:teuthology.orchestra.run.smithi067.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-11-25T04:20:52.769 INFO:teuthology.orchestra.run.smithi067.stdout: jq x86_64 1.6-16.el9 CentOS-AppStream 186 k 2023-11-25T04:20:52.769 INFO:teuthology.orchestra.run.smithi067.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-11-25T04:20:52.769 INFO:teuthology.orchestra.run.smithi067.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-11-25T04:20:52.769 INFO:teuthology.orchestra.run.smithi067.stdout: libcephsqlite x86_64 2:18.0.0-7506.gec928368.el9 ceph 163 k 2023-11-25T04:20:52.769 INFO:teuthology.orchestra.run.smithi067.stdout: libgfortran x86_64 11.4.1-2.3.el9 CentOS-BaseOS 803 k 2023-11-25T04:20:52.769 INFO:teuthology.orchestra.run.smithi067.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-11-25T04:20:52.769 INFO:teuthology.orchestra.run.smithi067.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-11-25T04:20:52.769 INFO:teuthology.orchestra.run.smithi067.stdout: libquadmath x86_64 11.4.1-2.3.el9 CentOS-BaseOS 195 k 2023-11-25T04:20:52.769 INFO:teuthology.orchestra.run.smithi067.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2023-11-25T04:20:52.769 INFO:teuthology.orchestra.run.smithi067.stdout: libradosstriper1 x86_64 2:18.0.0-7506.gec928368.el9 ceph 476 k 2023-11-25T04:20:52.769 INFO:teuthology.orchestra.run.smithi067.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-11-25T04:20:52.770 INFO:teuthology.orchestra.run.smithi067.stdout: librgw2 x86_64 2:18.0.0-7506.gec928368.el9 ceph 4.8 M 2023-11-25T04:20:52.770 INFO:teuthology.orchestra.run.smithi067.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-11-25T04:20:52.770 INFO:teuthology.orchestra.run.smithi067.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2023-11-25T04:20:52.770 INFO:teuthology.orchestra.run.smithi067.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-11-25T04:20:52.770 INFO:teuthology.orchestra.run.smithi067.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2023-11-25T04:20:52.770 INFO:teuthology.orchestra.run.smithi067.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2023-11-25T04:20:52.770 INFO:teuthology.orchestra.run.smithi067.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2023-11-25T04:20:52.770 INFO:teuthology.orchestra.run.smithi067.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-11-25T04:20:52.770 INFO:teuthology.orchestra.run.smithi067.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-11-25T04:20:52.770 INFO:teuthology.orchestra.run.smithi067.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-11-25T04:20:52.770 INFO:teuthology.orchestra.run.smithi067.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-11-25T04:20:52.770 INFO:teuthology.orchestra.run.smithi067.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-11-25T04:20:52.770 INFO:teuthology.orchestra.run.smithi067.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-11-25T04:20:52.771 INFO:teuthology.orchestra.run.smithi067.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-11-25T04:20:52.771 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-11-25T04:20:52.771 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ceph-argparse x86_64 2:18.0.0-7506.gec928368.el9 ceph 45 k 2023-11-25T04:20:52.771 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ceph-common x86_64 2:18.0.0-7506.gec928368.el9 ceph 129 k 2023-11-25T04:20:52.771 INFO:teuthology.orchestra.run.smithi067.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-11-25T04:20:52.771 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-11-25T04:20:52.771 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-11-25T04:20:52.771 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-11-25T04:20:52.771 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2023-11-25T04:20:52.771 INFO:teuthology.orchestra.run.smithi067.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2023-11-25T04:20:52.771 INFO:teuthology.orchestra.run.smithi067.stdout: python3-google-auth noarch 1:2.23.4-1.el9 epel 197 k 2023-11-25T04:20:52.771 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-11-25T04:20:52.771 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-11-25T04:20:52.772 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-11-25T04:20:52.772 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-11-25T04:20:52.772 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-11-25T04:20:52.772 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-11-25T04:20:52.772 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-11-25T04:20:52.772 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-11-25T04:20:52.772 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-11-25T04:20:52.772 INFO:teuthology.orchestra.run.smithi067.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-11-25T04:20:52.772 INFO:teuthology.orchestra.run.smithi067.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-11-25T04:20:52.772 INFO:teuthology.orchestra.run.smithi067.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-11-25T04:20:52.772 INFO:teuthology.orchestra.run.smithi067.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-11-25T04:20:52.772 INFO:teuthology.orchestra.run.smithi067.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-11-25T04:20:52.772 INFO:teuthology.orchestra.run.smithi067.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-11-25T04:20:52.772 INFO:teuthology.orchestra.run.smithi067.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-11-25T04:20:52.773 INFO:teuthology.orchestra.run.smithi067.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-11-25T04:20:52.773 INFO:teuthology.orchestra.run.smithi067.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-11-25T04:20:52.773 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-11-25T04:20:52.773 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-11-25T04:20:52.773 INFO:teuthology.orchestra.run.smithi067.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-11-25T04:20:52.773 INFO:teuthology.orchestra.run.smithi067.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-11-25T04:20:52.773 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-11-25T04:20:52.773 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-11-25T04:20:52.773 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-11-25T04:20:52.773 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-11-25T04:20:52.773 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2023-11-25T04:20:52.773 INFO:teuthology.orchestra.run.smithi067.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-11-25T04:20:52.773 INFO:teuthology.orchestra.run.smithi067.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2023-11-25T04:20:52.774 INFO:teuthology.orchestra.run.smithi067.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-11-25T04:20:52.774 INFO:teuthology.orchestra.run.smithi067.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-11-25T04:20:52.774 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-11-25T04:20:52.774 INFO:teuthology.orchestra.run.smithi067.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-11-25T04:20:52.774 INFO:teuthology.orchestra.run.smithi067.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-11-25T04:20:52.774 INFO:teuthology.orchestra.run.smithi067.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-11-25T04:20:52.774 INFO:teuthology.orchestra.run.smithi067.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-11-25T04:20:52.774 INFO:teuthology.orchestra.run.smithi067.stdout: python3-urllib3 noarch 1.26.5-4.el9 CentOS-BaseOS 187 k 2023-11-25T04:20:52.774 INFO:teuthology.orchestra.run.smithi067.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-11-25T04:20:52.774 INFO:teuthology.orchestra.run.smithi067.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-11-25T04:20:52.774 INFO:teuthology.orchestra.run.smithi067.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-11-25T04:20:52.774 INFO:teuthology.orchestra.run.smithi067.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-11-25T04:20:52.774 INFO:teuthology.orchestra.run.smithi067.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2023-11-25T04:20:52.774 INFO:teuthology.orchestra.run.smithi067.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-11-25T04:20:52.775 INFO:teuthology.orchestra.run.smithi067.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-11-25T04:20:52.775 INFO:teuthology.orchestra.run.smithi067.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-11-25T04:20:52.775 INFO:teuthology.orchestra.run.smithi067.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2023-11-25T04:20:52.775 INFO:teuthology.orchestra.run.smithi067.stdout:Installing weak dependencies: 2023-11-25T04:20:52.775 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-11-25T04:20:52.775 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:20:52.775 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction Summary 2023-11-25T04:20:52.775 INFO:teuthology.orchestra.run.smithi067.stdout:============================================================================================ 2023-11-25T04:20:52.775 INFO:teuthology.orchestra.run.smithi067.stdout:Install 128 Packages 2023-11-25T04:20:52.775 INFO:teuthology.orchestra.run.smithi067.stdout:Upgrade 7 Packages 2023-11-25T04:20:52.775 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:20:52.776 INFO:teuthology.orchestra.run.smithi092.stdout:(24/135): rbd-mirror-18.0.0-7506.gec928368.el9. 5.3 MB/s | 3.0 MB 00:00 2023-11-25T04:20:52.776 INFO:teuthology.orchestra.run.smithi067.stdout:Total download size: 191 M 2023-11-25T04:20:52.776 INFO:teuthology.orchestra.run.smithi067.stdout:Downloading Packages: 2023-11-25T04:20:52.863 INFO:teuthology.orchestra.run.smithi092.stdout:(25/135): rbd-nbd-18.0.0-7506.gec928368.el9.x86 1.8 MB/s | 170 kB 00:00 2023-11-25T04:20:52.900 INFO:teuthology.orchestra.run.smithi195.stdout:(2/135): ceph-fuse-18.0.0-7506.gec928368.el9.x8 625 kB/s | 859 kB 00:01 2023-11-25T04:20:52.964 INFO:teuthology.orchestra.run.smithi092.stdout:(26/135): ceph-grafana-dashboards-18.0.0-7506.g 281 kB/s | 28 kB 00:00 2023-11-25T04:20:53.047 INFO:teuthology.orchestra.run.smithi092.stdout:(27/135): ceph-mgr-cephadm-18.0.0-7506.gec92836 1.6 MB/s | 133 kB 00:00 2023-11-25T04:20:53.175 INFO:teuthology.orchestra.run.smithi195.stdout:(3/135): ceph-immutable-object-cache-18.0.0-750 513 kB/s | 141 kB 00:00 2023-11-25T04:20:53.407 INFO:teuthology.orchestra.run.smithi067.stdout:(1/135): ceph-18.0.0-7506.gec928368.el9.x86_64. 19 kB/s | 6.6 kB 00:00 2023-11-25T04:20:53.439 INFO:teuthology.orchestra.run.smithi092.stdout:(28/135): ceph-mgr-dashboard-18.0.0-7506.gec928 4.5 MB/s | 1.8 MB 00:00 2023-11-25T04:20:53.743 INFO:teuthology.orchestra.run.smithi076.stdout:(8/135): ceph-osd-18.0.0-7506.gec928368.el9.x86 4.7 MB/s | 18 MB 00:03 2023-11-25T04:20:53.844 INFO:teuthology.orchestra.run.smithi076.stdout:(9/135): ceph-selinux-18.0.0-7506.gec928368.el9 241 kB/s | 24 kB 00:00 2023-11-25T04:20:54.050 INFO:teuthology.orchestra.run.smithi195.stdout:(4/135): ceph-base-18.0.0-7506.gec928368.el9.x8 1.8 MB/s | 5.2 MB 00:02 2023-11-25T04:20:54.124 INFO:teuthology.orchestra.run.smithi067.stdout:(2/135): ceph-fuse-18.0.0-7506.gec928368.el9.x8 1.2 MB/s | 859 kB 00:00 2023-11-25T04:20:54.266 INFO:teuthology.orchestra.run.smithi067.stdout:(3/135): ceph-immutable-object-cache-18.0.0-750 997 kB/s | 141 kB 00:00 2023-11-25T04:20:54.297 INFO:teuthology.orchestra.run.smithi092.stdout:(29/135): ceph-test-18.0.0-7506.gec928368.el9.x 8.8 MB/s | 41 MB 00:04 2023-11-25T04:20:54.625 INFO:teuthology.orchestra.run.smithi195.stdout:(5/135): ceph-mgr-18.0.0-7506.gec928368.el9.x86 2.5 MB/s | 1.5 MB 00:00 2023-11-25T04:20:54.698 INFO:teuthology.orchestra.run.smithi092.stdout:(30/135): ceph-mgr-modules-core-18.0.0-7506.gec 607 kB/s | 243 kB 00:00 2023-11-25T04:20:54.789 INFO:teuthology.orchestra.run.smithi092.stdout:(31/135): ceph-mgr-rook-18.0.0-7506.gec928368.e 548 kB/s | 50 kB 00:00 2023-11-25T04:20:54.861 INFO:teuthology.orchestra.run.smithi076.stdout:(10/135): ceph-radosgw-18.0.0-7506.gec928368.el 2.4 MB/s | 8.7 MB 00:03 2023-11-25T04:20:54.907 INFO:teuthology.orchestra.run.smithi092.stdout:(32/135): ceph-mgr-diskprediction-local-18.0.0- 5.0 MB/s | 7.4 MB 00:01 2023-11-25T04:20:54.923 INFO:teuthology.orchestra.run.smithi092.stdout:(33/135): ceph-prometheus-alerts-18.0.0-7506.ge 113 kB/s | 15 kB 00:00 2023-11-25T04:20:54.935 INFO:teuthology.orchestra.run.smithi076.stdout:(11/135): libcephfs-devel-18.0.0-7506.gec928368 417 kB/s | 31 kB 00:00 2023-11-25T04:20:54.999 INFO:teuthology.orchestra.run.smithi092.stdout:(34/135): ceph-volume-18.0.0-7506.gec928368.el9 2.8 MB/s | 262 kB 00:00 2023-11-25T04:20:55.119 INFO:teuthology.orchestra.run.smithi076.stdout:(12/135): ceph-common-18.0.0-7506.gec928368.el9 2.7 MB/s | 19 MB 00:07 2023-11-25T04:20:55.132 INFO:teuthology.orchestra.run.smithi092.stdout:(35/135): automake-1.16.2-8.el9.noarch.rpm 4.9 MB/s | 662 kB 00:00 2023-11-25T04:20:55.183 INFO:teuthology.orchestra.run.smithi092.stdout:(36/135): bison-3.7.4-5.el9.x86_64.rpm 18 MB/s | 920 kB 00:00 2023-11-25T04:20:55.216 INFO:teuthology.orchestra.run.smithi092.stdout:(37/135): boost-program-options-1.75.0-8.el9.x8 3.1 MB/s | 106 kB 00:00 2023-11-25T04:20:55.228 INFO:teuthology.orchestra.run.smithi076.stdout:(13/135): libcephsqlite-18.0.0-7506.gec928368.e 1.5 MB/s | 163 kB 00:00 2023-11-25T04:20:55.241 INFO:teuthology.orchestra.run.smithi092.stdout:(38/135): cephadm-18.0.0-7506.gec928368.el9.noa 2.4 MB/s | 776 kB 00:00 2023-11-25T04:20:55.258 INFO:teuthology.orchestra.run.smithi092.stdout:(39/135): elfutils-libelf-devel-0.190-1.el9.x86 514 kB/s | 21 kB 00:00 2023-11-25T04:20:55.284 INFO:teuthology.orchestra.run.smithi092.stdout:(40/135): flex-2.6.4-9.el9.x86_64.rpm 7.2 MB/s | 307 kB 00:00 2023-11-25T04:20:55.300 INFO:teuthology.orchestra.run.smithi092.stdout:(41/135): flexiblas-3.0.4-8.el9.x86_64.rpm 766 kB/s | 32 kB 00:00 2023-11-25T04:20:55.324 INFO:teuthology.orchestra.run.smithi067.stdout:(4/135): ceph-mds-18.0.0-7506.gec928368.el9.x86 2.0 MB/s | 2.1 MB 00:01 2023-11-25T04:20:55.345 INFO:teuthology.orchestra.run.smithi076.stdout:(14/135): libcephfs2-18.0.0-7506.gec928368.el9. 1.7 MB/s | 719 kB 00:00 2023-11-25T04:20:55.370 INFO:teuthology.orchestra.run.smithi076.stdout:(15/135): librados-devel-18.0.0-7506.gec928368. 892 kB/s | 126 kB 00:00 2023-11-25T04:20:55.395 INFO:teuthology.orchestra.run.smithi092.stdout:(42/135): flexiblas-netlib-3.0.4-8.el9.x86_64.r 27 MB/s | 3.0 MB 00:00 2023-11-25T04:20:55.420 INFO:teuthology.orchestra.run.smithi092.stdout:(43/135): flexiblas-openblas-openmp-3.0.4-8.el9 145 kB/s | 17 kB 00:00 2023-11-25T04:20:55.445 INFO:teuthology.orchestra.run.smithi092.stdout:(44/135): jq-1.6-16.el9.x86_64.rpm 3.6 MB/s | 186 kB 00:00 2023-11-25T04:20:55.470 INFO:teuthology.orchestra.run.smithi092.stdout:(45/135): libaio-devel-0.3.111-13.el9.x86_64.rp 201 kB/s | 10 kB 00:00 2023-11-25T04:20:55.496 INFO:teuthology.orchestra.run.smithi092.stdout:(46/135): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 159 kB 00:00 2023-11-25T04:20:55.522 INFO:teuthology.orchestra.run.smithi092.stdout:(47/135): librabbitmq-0.11.0-7.el9.x86_64.rpm 874 kB/s | 44 kB 00:00 2023-11-25T04:20:55.533 INFO:teuthology.orchestra.run.smithi195.stdout:(6/135): ceph-mds-18.0.0-7506.gec928368.el9.x86 905 kB/s | 2.1 MB 00:02 2023-11-25T04:20:55.547 INFO:teuthology.orchestra.run.smithi092.stdout:(48/135): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 661 kB 00:00 2023-11-25T04:20:55.558 INFO:teuthology.orchestra.run.smithi067.stdout:(5/135): ceph-base-18.0.0-7506.gec928368.el9.x8 2.1 MB/s | 5.2 MB 00:02 2023-11-25T04:20:55.572 INFO:teuthology.orchestra.run.smithi092.stdout:(49/135): libtool-2.4.6-45.el9.x86_64.rpm 11 MB/s | 578 kB 00:00 2023-11-25T04:20:55.595 INFO:teuthology.orchestra.run.smithi076.stdout:(16/135): libradosstriper1-18.0.0-7506.gec92836 1.9 MB/s | 476 kB 00:00 2023-11-25T04:20:55.597 INFO:teuthology.orchestra.run.smithi092.stdout:(50/135): libuuid-devel-2.37.4-15.el9.x86_64.rp 497 kB/s | 24 kB 00:00 2023-11-25T04:20:55.623 INFO:teuthology.orchestra.run.smithi092.stdout:(51/135): libzstd-devel-1.5.1-2.el9.x86_64.rpm 925 kB/s | 46 kB 00:00 2023-11-25T04:20:55.648 INFO:teuthology.orchestra.run.smithi092.stdout:(52/135): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.5 MB/s | 282 kB 00:00 2023-11-25T04:20:55.673 INFO:teuthology.orchestra.run.smithi092.stdout:(53/135): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2023-11-25T04:20:55.696 INFO:teuthology.orchestra.run.smithi076.stdout:(17/135): python3-ceph-argparse-18.0.0-7506.gec 451 kB/s | 45 kB 00:00 2023-11-25T04:20:55.699 INFO:teuthology.orchestra.run.smithi092.stdout:(54/135): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.2 MB/s | 217 kB 00:00 2023-11-25T04:20:55.716 INFO:teuthology.orchestra.run.smithi092.stdout:(55/135): openblas-0.3.21-2.el9.x86_64.rpm 814 kB/s | 34 kB 00:00 2023-11-25T04:20:55.796 INFO:teuthology.orchestra.run.smithi076.stdout:(18/135): python3-ceph-common-18.0.0-7506.gec92 1.3 MB/s | 129 kB 00:00 2023-11-25T04:20:55.833 INFO:teuthology.orchestra.run.smithi092.stdout:(56/135): openblas-openmp-0.3.21-2.el9.x86_64.r 35 MB/s | 4.7 MB 00:00 2023-11-25T04:20:55.858 INFO:teuthology.orchestra.run.smithi067.stdout:(6/135): ceph-mgr-18.0.0-7506.gec928368.el9.x86 2.7 MB/s | 1.5 MB 00:00 2023-11-25T04:20:55.938 INFO:teuthology.orchestra.run.smithi076.stdout:(19/135): python3-cephfs-18.0.0-7506.gec928368. 1.1 MB/s | 162 kB 00:00 2023-11-25T04:20:55.941 INFO:teuthology.orchestra.run.smithi092.stdout:(57/135): python3-babel-2.9.1-2.el9.noarch.rpm 26 MB/s | 5.8 MB 00:00 2023-11-25T04:20:55.966 INFO:teuthology.orchestra.run.smithi092.stdout:(58/135): python3-jinja2-2.11.3-4.el9.noarch.rp 1.7 MB/s | 228 kB 00:00 2023-11-25T04:20:55.983 INFO:teuthology.orchestra.run.smithi092.stdout:(59/135): python3-jsonpatch-1.21-16.el9.noarch. 590 kB/s | 24 kB 00:00 2023-11-25T04:20:56.000 INFO:teuthology.orchestra.run.smithi092.stdout:(60/135): python3-jsonpointer-2.0-4.el9.noarch. 538 kB/s | 18 kB 00:00 2023-11-25T04:20:56.026 INFO:teuthology.orchestra.run.smithi092.stdout:(61/135): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 144 kB 00:00 2023-11-25T04:20:56.042 INFO:teuthology.orchestra.run.smithi195.stdout:(7/135): ceph-common-18.0.0-7506.gec928368.el9. 3.8 MB/s | 19 MB 00:04 2023-11-25T04:20:56.043 INFO:teuthology.orchestra.run.smithi092.stdout:(62/135): python3-markupsafe-1.1.1-12.el9.x86_6 755 kB/s | 32 kB 00:00 2023-11-25T04:20:56.092 INFO:teuthology.orchestra.run.smithi076.stdout:(20/135): python3-rados-18.0.0-7506.gec928368.e 2.0 MB/s | 320 kB 00:00 2023-11-25T04:20:56.167 INFO:teuthology.orchestra.run.smithi195.stdout:(8/135): ceph-mon-18.0.0-7506.gec928368.el9.x86 2.8 MB/s | 4.4 MB 00:01 2023-11-25T04:20:56.168 INFO:teuthology.orchestra.run.smithi092.stdout:(63/135): python3-numpy-1.20.1-5.el9.x86_64.rpm 34 MB/s | 4.8 MB 00:00 2023-11-25T04:20:56.193 INFO:teuthology.orchestra.run.smithi092.stdout:(64/135): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.4 MB/s | 217 kB 00:00 2023-11-25T04:20:56.218 INFO:teuthology.orchestra.run.smithi092.stdout:(65/135): python3-oauthlib-3.1.1-5.el9.noarch.r 3.4 MB/s | 174 kB 00:00 2023-11-25T04:20:56.235 INFO:teuthology.orchestra.run.smithi092.stdout:(66/135): python3-prettytable-0.7.2-27.el9.noar 988 kB/s | 41 kB 00:00 2023-11-25T04:20:56.242 INFO:teuthology.orchestra.run.smithi076.stdout:(21/135): python3-rbd-18.0.0-7506.gec928368.el9 1.9 MB/s | 298 kB 00:00 2023-11-25T04:20:56.259 INFO:teuthology.orchestra.run.smithi195.stdout:(9/135): ceph-selinux-18.0.0-7506.gec928368.el9 263 kB/s | 24 kB 00:00 2023-11-25T04:20:56.261 INFO:teuthology.orchestra.run.smithi092.stdout:(67/135): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 132 kB 00:00 2023-11-25T04:20:56.286 INFO:teuthology.orchestra.run.smithi092.stdout:(68/135): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2023-11-25T04:20:56.303 INFO:teuthology.orchestra.run.smithi092.stdout:(69/135): python3-pytz-2021.1-5.el9.noarch.rpm 1.1 MB/s | 47 kB 00:00 2023-11-25T04:20:56.320 INFO:teuthology.orchestra.run.smithi092.stdout:(70/135): python3-requests-oauthlib-1.3.0-12.el 1.2 MB/s | 43 kB 00:00 2023-11-25T04:20:56.334 INFO:teuthology.orchestra.run.smithi076.stdout:(22/135): python3-rgw-18.0.0-7506.gec928368.el9 1.1 MB/s | 99 kB 00:00 2023-11-25T04:20:56.427 INFO:teuthology.orchestra.run.smithi076.stdout:(23/135): rbd-fuse-18.0.0-7506.gec928368.el9.x8 927 kB/s | 85 kB 00:00 2023-11-25T04:20:56.529 INFO:teuthology.orchestra.run.smithi092.stdout:(71/135): python3-toml-0.10.2-6.el9.noarch.rpm 186 kB/s | 39 kB 00:00 2023-11-25T04:20:56.746 INFO:teuthology.orchestra.run.smithi092.stdout:(72/135): python3-scipy-1.6.2-8.el9.x86_64.rpm 35 MB/s | 16 MB 00:00 2023-11-25T04:20:56.771 INFO:teuthology.orchestra.run.smithi092.stdout:(73/135): socat-1.7.4.1-5.el9.x86_64.rpm 1.2 MB/s | 300 kB 00:00 2023-11-25T04:20:56.796 INFO:teuthology.orchestra.run.smithi092.stdout:(74/135): xfsprogs-devel-5.19.0-4.el9.x86_64.rp 1.7 MB/s | 85 kB 00:00 2023-11-25T04:20:56.821 INFO:teuthology.orchestra.run.smithi092.stdout:(75/135): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2023-11-25T04:20:56.825 INFO:teuthology.orchestra.run.smithi067.stdout:(7/135): ceph-mon-18.0.0-7506.gec928368.el9.x86 3.5 MB/s | 4.4 MB 00:01 2023-11-25T04:20:56.847 INFO:teuthology.orchestra.run.smithi092.stdout:(76/135): zlib-devel-1.2.11-40.el9.x86_64.rpm 876 kB/s | 44 kB 00:00 2023-11-25T04:20:56.872 INFO:teuthology.orchestra.run.smithi092.stdout:(77/135): fuse-2.9.9-15.el9.x86_64.rpm 1.5 MB/s | 79 kB 00:00 2023-11-25T04:20:56.897 INFO:teuthology.orchestra.run.smithi092.stdout:(78/135): libgfortran-11.4.1-2.3.el9.x86_64.rpm 16 MB/s | 803 kB 00:00 2023-11-25T04:20:56.923 INFO:teuthology.orchestra.run.smithi092.stdout:(79/135): libquadmath-11.4.1-2.3.el9.x86_64.rpm 3.9 MB/s | 195 kB 00:00 2023-11-25T04:20:56.948 INFO:teuthology.orchestra.run.smithi092.stdout:(80/135): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2023-11-25T04:20:56.982 INFO:teuthology.orchestra.run.smithi092.stdout:(81/135): python3-cryptography-36.0.1-4.el9.x86 20 MB/s | 1.1 MB 00:00 2023-11-25T04:20:57.015 INFO:teuthology.orchestra.run.smithi092.stdout:(82/135): python3-ply-3.11-14.el9.noarch.rpm 1.5 MB/s | 103 kB 00:00 2023-11-25T04:20:57.041 INFO:teuthology.orchestra.run.smithi092.stdout:(83/135): python3-pycparser-2.20-6.el9.noarch.r 2.1 MB/s | 124 kB 00:00 2023-11-25T04:20:57.066 INFO:teuthology.orchestra.run.smithi092.stdout:(84/135): python3-requests-2.25.1-7.el9.noarch. 2.2 MB/s | 113 kB 00:00 2023-11-25T04:20:57.092 INFO:teuthology.orchestra.run.smithi092.stdout:(85/135): python3-urllib3-1.26.5-4.el9.noarch.r 3.7 MB/s | 187 kB 00:00 2023-11-25T04:20:57.217 INFO:teuthology.orchestra.run.smithi092.stdout:(86/135): python3-devel-3.9.17-2.el9.x86_64.rpm 1.9 MB/s | 245 kB 00:00 2023-11-25T04:20:57.219 INFO:teuthology.orchestra.run.smithi076.stdout:(24/135): librgw2-18.0.0-7506.gec928368.el9.x86 2.6 MB/s | 4.8 MB 00:01 2023-11-25T04:20:57.292 INFO:teuthology.orchestra.run.smithi092.stdout:(87/135): lua-devel-5.4.4-4.el9.x86_64.rpm 297 kB/s | 22 kB 00:00 2023-11-25T04:20:57.336 INFO:teuthology.orchestra.run.smithi076.stdout:(25/135): rbd-nbd-18.0.0-7506.gec928368.el9.x86 1.4 MB/s | 170 kB 00:00 2023-11-25T04:20:57.359 INFO:teuthology.orchestra.run.smithi092.stdout:(88/135): openssl-devel-3.0.7-25.el9.x86_64.rpm 14 MB/s | 4.1 MB 00:00 2023-11-25T04:20:57.453 INFO:teuthology.orchestra.run.smithi076.stdout:(26/135): ceph-grafana-dashboards-18.0.0-7506.g 240 kB/s | 28 kB 00:00 2023-11-25T04:20:57.545 INFO:teuthology.orchestra.run.smithi076.stdout:(27/135): ceph-mgr-cephadm-18.0.0-7506.gec92836 1.4 MB/s | 133 kB 00:00 2023-11-25T04:20:57.625 INFO:teuthology.orchestra.run.smithi067.stdout:(8/135): ceph-common-18.0.0-7506.gec928368.el9. 4.2 MB/s | 19 MB 00:04 2023-11-25T04:20:57.709 INFO:teuthology.orchestra.run.smithi067.stdout:(9/135): ceph-selinux-18.0.0-7506.gec928368.el9 289 kB/s | 24 kB 00:00 2023-11-25T04:20:58.037 INFO:teuthology.orchestra.run.smithi076.stdout:(28/135): rbd-mirror-18.0.0-7506.gec928368.el9. 1.9 MB/s | 3.0 MB 00:01 2023-11-25T04:20:58.218 INFO:teuthology.orchestra.run.smithi092.stdout:(89/135): gperftools-libs-2.9.1-2.el9.x86_64.rp 334 kB/s | 309 kB 00:00 2023-11-25T04:20:58.245 INFO:teuthology.orchestra.run.smithi076.stdout:(29/135): ceph-mgr-dashboard-18.0.0-7506.gec928 2.5 MB/s | 1.8 MB 00:00 2023-11-25T04:20:58.334 INFO:teuthology.orchestra.run.smithi092.stdout:(90/135): libarrow-doc-9.0.0-9.el9.noarch.rpm 209 kB/s | 24 kB 00:00 2023-11-25T04:20:58.370 INFO:teuthology.orchestra.run.smithi076.stdout:(30/135): ceph-mgr-modules-core-18.0.0-7506.gec 1.9 MB/s | 243 kB 00:00 2023-11-25T04:20:58.452 INFO:teuthology.orchestra.run.smithi092.stdout:(91/135): liboath-2.6.7-2.el9.x86_64.rpm 416 kB/s | 49 kB 00:00 2023-11-25T04:20:58.487 INFO:teuthology.orchestra.run.smithi076.stdout:(31/135): ceph-mgr-rook-18.0.0-7506.gec928368.e 430 kB/s | 50 kB 00:00 2023-11-25T04:20:58.604 INFO:teuthology.orchestra.run.smithi076.stdout:(32/135): ceph-prometheus-alerts-18.0.0-7506.ge 130 kB/s | 15 kB 00:00 2023-11-25T04:20:58.660 INFO:teuthology.orchestra.run.smithi092.stdout:(92/135): libunwind-1.6.2-1.el9.x86_64.rpm 323 kB/s | 67 kB 00:00 2023-11-25T04:20:58.805 INFO:teuthology.orchestra.run.smithi076.stdout:(33/135): ceph-volume-18.0.0-7506.gec928368.el9 1.3 MB/s | 262 kB 00:00 2023-11-25T04:20:58.960 INFO:teuthology.orchestra.run.smithi092.stdout:(93/135): luarocks-3.9.2-1.el9.noarch.rpm 504 kB/s | 151 kB 00:00 2023-11-25T04:20:59.059 INFO:teuthology.orchestra.run.smithi195.stdout:(10/135): ceph-radosgw-18.0.0-7506.gec928368.el 2.9 MB/s | 8.7 MB 00:03 2023-11-25T04:20:59.113 INFO:teuthology.orchestra.run.smithi076.stdout:(34/135): cephadm-18.0.0-7506.gec928368.el9.noa 2.5 MB/s | 776 kB 00:00 2023-11-25T04:20:59.134 INFO:teuthology.orchestra.run.smithi195.stdout:(11/135): libcephfs-devel-18.0.0-7506.gec928368 417 kB/s | 31 kB 00:00 2023-11-25T04:20:59.142 INFO:teuthology.orchestra.run.smithi067.stdout:(10/135): ceph-osd-18.0.0-7506.gec928368.el9.x8 5.4 MB/s | 18 MB 00:03 2023-11-25T04:20:59.218 INFO:teuthology.orchestra.run.smithi067.stdout:(11/135): libcephfs-devel-18.0.0-7506.gec928368 417 kB/s | 31 kB 00:00 2023-11-25T04:20:59.255 INFO:teuthology.orchestra.run.smithi076.stdout:(35/135): automake-1.16.2-8.el9.noarch.rpm 4.6 MB/s | 662 kB 00:00 2023-11-25T04:20:59.305 INFO:teuthology.orchestra.run.smithi076.stdout:(36/135): bison-3.7.4-5.el9.x86_64.rpm 18 MB/s | 920 kB 00:00 2023-11-25T04:20:59.347 INFO:teuthology.orchestra.run.smithi076.stdout:(37/135): boost-program-options-1.75.0-8.el9.x8 2.5 MB/s | 106 kB 00:00 2023-11-25T04:20:59.373 INFO:teuthology.orchestra.run.smithi076.stdout:(38/135): elfutils-libelf-devel-0.190-1.el9.x86 861 kB/s | 21 kB 00:00 2023-11-25T04:20:59.376 INFO:teuthology.orchestra.run.smithi195.stdout:(12/135): libcephfs2-18.0.0-7506.gec928368.el9. 2.9 MB/s | 719 kB 00:00 2023-11-25T04:20:59.414 INFO:teuthology.orchestra.run.smithi076.stdout:(39/135): flex-2.6.4-9.el9.x86_64.rpm 7.2 MB/s | 307 kB 00:00 2023-11-25T04:20:59.434 INFO:teuthology.orchestra.run.smithi067.stdout:(12/135): libcephfs2-18.0.0-7506.gec928368.el9. 3.2 MB/s | 719 kB 00:00 2023-11-25T04:20:59.439 INFO:teuthology.orchestra.run.smithi076.stdout:(40/135): flexiblas-3.0.4-8.el9.x86_64.rpm 1.3 MB/s | 32 kB 00:00 2023-11-25T04:20:59.477 INFO:teuthology.orchestra.run.smithi195.stdout:(13/135): libcephsqlite-18.0.0-7506.gec928368.e 1.6 MB/s | 163 kB 00:00 2023-11-25T04:20:59.510 INFO:teuthology.orchestra.run.smithi067.stdout:(13/135): libcephsqlite-18.0.0-7506.gec928368.e 2.1 MB/s | 163 kB 00:00 2023-11-25T04:20:59.531 INFO:teuthology.orchestra.run.smithi076.stdout:(41/135): flexiblas-netlib-3.0.4-8.el9.x86_64.r 33 MB/s | 3.0 MB 00:00 2023-11-25T04:20:59.557 INFO:teuthology.orchestra.run.smithi076.stdout:(42/135): flexiblas-openblas-openmp-3.0.4-8.el9 685 kB/s | 17 kB 00:00 2023-11-25T04:20:59.569 INFO:teuthology.orchestra.run.smithi195.stdout:(14/135): librados-devel-18.0.0-7506.gec928368. 1.4 MB/s | 126 kB 00:00 2023-11-25T04:20:59.576 INFO:teuthology.orchestra.run.smithi067.stdout:(14/135): librados-devel-18.0.0-7506.gec928368. 1.9 MB/s | 126 kB 00:00 2023-11-25T04:20:59.590 INFO:teuthology.orchestra.run.smithi076.stdout:(43/135): jq-1.6-16.el9.x86_64.rpm 5.4 MB/s | 186 kB 00:00 2023-11-25T04:20:59.615 INFO:teuthology.orchestra.run.smithi076.stdout:(44/135): libaio-devel-0.3.111-13.el9.x86_64.rp 406 kB/s | 10 kB 00:00 2023-11-25T04:20:59.657 INFO:teuthology.orchestra.run.smithi076.stdout:(45/135): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.7 MB/s | 159 kB 00:00 2023-11-25T04:20:59.677 INFO:teuthology.orchestra.run.smithi067.stdout:(15/135): ceph-radosgw-18.0.0-7506.gec928368.el 3.0 MB/s | 8.7 MB 00:02 2023-11-25T04:20:59.683 INFO:teuthology.orchestra.run.smithi076.stdout:(46/135): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.7 MB/s | 44 kB 00:00 2023-11-25T04:20:59.711 INFO:teuthology.orchestra.run.smithi195.stdout:(15/135): libradosstriper1-18.0.0-7506.gec92836 3.3 MB/s | 476 kB 00:00 2023-11-25T04:20:59.725 INFO:teuthology.orchestra.run.smithi076.stdout:(47/135): librdkafka-1.6.1-102.el9.x86_64.rpm 16 MB/s | 661 kB 00:00 2023-11-25T04:20:59.735 INFO:teuthology.orchestra.run.smithi067.stdout:(16/135): libradosstriper1-18.0.0-7506.gec92836 2.9 MB/s | 476 kB 00:00 2023-11-25T04:20:59.766 INFO:teuthology.orchestra.run.smithi076.stdout:(48/135): libtool-2.4.6-45.el9.x86_64.rpm 14 MB/s | 578 kB 00:00 2023-11-25T04:20:59.792 INFO:teuthology.orchestra.run.smithi076.stdout:(49/135): libuuid-devel-2.37.4-15.el9.x86_64.rp 975 kB/s | 24 kB 00:00 2023-11-25T04:20:59.817 INFO:teuthology.orchestra.run.smithi076.stdout:(50/135): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.8 MB/s | 46 kB 00:00 2023-11-25T04:20:59.819 INFO:teuthology.orchestra.run.smithi067.stdout:(17/135): python3-ceph-argparse-18.0.0-7506.gec 541 kB/s | 45 kB 00:00 2023-11-25T04:20:59.851 INFO:teuthology.orchestra.run.smithi076.stdout:(51/135): lttng-ust-2.12.0-6.el9.x86_64.rpm 8.3 MB/s | 282 kB 00:00 2023-11-25T04:20:59.893 INFO:teuthology.orchestra.run.smithi076.stdout:(52/135): lua-5.4.4-4.el9.x86_64.rpm 4.4 MB/s | 187 kB 00:00 2023-11-25T04:20:59.910 INFO:teuthology.orchestra.run.smithi067.stdout:(18/135): python3-ceph-common-18.0.0-7506.gec92 1.4 MB/s | 129 kB 00:00 2023-11-25T04:20:59.935 INFO:teuthology.orchestra.run.smithi076.stdout:(53/135): oniguruma-6.9.6-1.el9.5.x86_64.rpm 5.1 MB/s | 217 kB 00:00 2023-11-25T04:20:59.968 INFO:teuthology.orchestra.run.smithi076.stdout:(54/135): openblas-0.3.21-2.el9.x86_64.rpm 1.0 MB/s | 34 kB 00:00 2023-11-25T04:20:59.994 INFO:teuthology.orchestra.run.smithi067.stdout:(19/135): python3-cephfs-18.0.0-7506.gec928368. 1.9 MB/s | 162 kB 00:00 2023-11-25T04:21:00.078 INFO:teuthology.orchestra.run.smithi067.stdout:(20/135): python3-rados-18.0.0-7506.gec928368.e 3.8 MB/s | 320 kB 00:00 2023-11-25T04:21:00.094 INFO:teuthology.orchestra.run.smithi076.stdout:(55/135): openblas-openmp-0.3.21-2.el9.x86_64.r 38 MB/s | 4.7 MB 00:00 2023-11-25T04:21:00.162 INFO:teuthology.orchestra.run.smithi067.stdout:(21/135): python3-rbd-18.0.0-7506.gec928368.el9 3.5 MB/s | 298 kB 00:00 2023-11-25T04:21:00.236 INFO:teuthology.orchestra.run.smithi076.stdout:(56/135): python3-babel-2.9.1-2.el9.noarch.rpm 41 MB/s | 5.8 MB 00:00 2023-11-25T04:21:00.245 INFO:teuthology.orchestra.run.smithi067.stdout:(22/135): python3-rgw-18.0.0-7506.gec928368.el9 1.2 MB/s | 99 kB 00:00 2023-11-25T04:21:00.278 INFO:teuthology.orchestra.run.smithi076.stdout:(57/135): python3-jinja2-2.11.3-4.el9.noarch.rp 5.4 MB/s | 228 kB 00:00 2023-11-25T04:21:00.311 INFO:teuthology.orchestra.run.smithi076.stdout:(58/135): python3-jsonpatch-1.21-16.el9.noarch. 730 kB/s | 24 kB 00:00 2023-11-25T04:21:00.320 INFO:teuthology.orchestra.run.smithi067.stdout:(23/135): rbd-fuse-18.0.0-7506.gec928368.el9.x8 1.1 MB/s | 85 kB 00:00 2023-11-25T04:21:00.330 INFO:teuthology.orchestra.run.smithi092.stdout:(94/135): parquet-libs-9.0.0-9.el9.x86_64.rpm 611 kB/s | 837 kB 00:01 2023-11-25T04:21:00.345 INFO:teuthology.orchestra.run.smithi076.stdout:(59/135): python3-jsonpointer-2.0-4.el9.noarch. 544 kB/s | 18 kB 00:00 2023-11-25T04:21:00.378 INFO:teuthology.orchestra.run.smithi076.stdout:(60/135): python3-mako-1.1.4-6.el9.noarch.rpm 4.2 MB/s | 144 kB 00:00 2023-11-25T04:21:00.412 INFO:teuthology.orchestra.run.smithi076.stdout:(61/135): python3-markupsafe-1.1.1-12.el9.x86_6 945 kB/s | 32 kB 00:00 2023-11-25T04:21:00.538 INFO:teuthology.orchestra.run.smithi076.stdout:(62/135): python3-numpy-1.20.1-5.el9.x86_64.rpm 38 MB/s | 4.8 MB 00:00 2023-11-25T04:21:00.579 INFO:teuthology.orchestra.run.smithi076.stdout:(63/135): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.1 MB/s | 217 kB 00:00 2023-11-25T04:21:00.613 INFO:teuthology.orchestra.run.smithi076.stdout:(64/135): python3-oauthlib-3.1.1-5.el9.noarch.r 5.2 MB/s | 174 kB 00:00 2023-11-25T04:21:00.646 INFO:teuthology.orchestra.run.smithi076.stdout:(65/135): python3-prettytable-0.7.2-27.el9.noar 1.2 MB/s | 41 kB 00:00 2023-11-25T04:21:00.672 INFO:teuthology.orchestra.run.smithi076.stdout:(66/135): python3-pyasn1-0.4.8-6.el9.noarch.rpm 5.1 MB/s | 132 kB 00:00 2023-11-25T04:21:00.706 INFO:teuthology.orchestra.run.smithi076.stdout:(67/135): python3-pyasn1-modules-0.4.8-6.el9.no 6.2 MB/s | 211 kB 00:00 2023-11-25T04:21:00.822 INFO:teuthology.orchestra.run.smithi076.stdout:(68/135): python3-pytz-2021.1-5.el9.noarch.rpm 405 kB/s | 47 kB 00:00 2023-11-25T04:21:00.856 INFO:teuthology.orchestra.run.smithi076.stdout:(69/135): python3-requests-oauthlib-1.3.0-12.el 1.3 MB/s | 43 kB 00:00 2023-11-25T04:21:01.013 INFO:teuthology.orchestra.run.smithi067.stdout:(24/135): rbd-mirror-18.0.0-7506.gec928368.el9. 4.4 MB/s | 3.0 MB 00:00 2023-11-25T04:21:01.138 INFO:teuthology.orchestra.run.smithi067.stdout:(25/135): librgw2-18.0.0-7506.gec928368.el9.x86 3.3 MB/s | 4.8 MB 00:01 2023-11-25T04:21:01.165 INFO:teuthology.orchestra.run.smithi067.stdout:(26/135): rbd-nbd-18.0.0-7506.gec928368.el9.x86 1.1 MB/s | 170 kB 00:00 2023-11-25T04:21:01.188 INFO:teuthology.orchestra.run.smithi067.stdout:(27/135): ceph-grafana-dashboards-18.0.0-7506.g 559 kB/s | 28 kB 00:00 2023-11-25T04:21:01.205 INFO:teuthology.orchestra.run.smithi092.stdout:(95/135): python3-asyncssh-2.13.2-3.el9.noarch. 626 kB/s | 548 kB 00:00 2023-11-25T04:21:01.217 INFO:teuthology.orchestra.run.smithi076.stdout:(70/135): python3-scipy-1.6.2-8.el9.x86_64.rpm 43 MB/s | 16 MB 00:00 2023-11-25T04:21:01.222 INFO:teuthology.orchestra.run.smithi067.stdout:(28/135): ceph-mgr-cephadm-18.0.0-7506.gec92836 2.2 MB/s | 133 kB 00:00 2023-11-25T04:21:01.261 INFO:teuthology.orchestra.run.smithi195.stdout:(16/135): librgw2-18.0.0-7506.gec928368.el9.x86 3.1 MB/s | 4.8 MB 00:01 2023-11-25T04:21:01.309 INFO:teuthology.orchestra.run.smithi076.stdout:(71/135): ceph-mgr-diskprediction-local-18.0.0- 2.3 MB/s | 7.4 MB 00:03 2023-11-25T04:21:01.325 INFO:teuthology.orchestra.run.smithi076.stdout:(72/135): python3-toml-0.10.2-6.el9.noarch.rpm 356 kB/s | 39 kB 00:00 2023-11-25T04:21:01.330 INFO:teuthology.orchestra.run.smithi092.stdout:(96/135): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 348 kB/s | 43 kB 00:00 2023-11-25T04:21:01.351 INFO:teuthology.orchestra.run.smithi076.stdout:(73/135): socat-1.7.4.1-5.el9.x86_64.rpm 7.0 MB/s | 300 kB 00:00 2023-11-25T04:21:01.369 INFO:teuthology.orchestra.run.smithi195.stdout:(17/135): python3-ceph-argparse-18.0.0-7506.gec 418 kB/s | 45 kB 00:00 2023-11-25T04:21:01.376 INFO:teuthology.orchestra.run.smithi076.stdout:(74/135): xfsprogs-devel-5.19.0-4.el9.x86_64.rp 1.7 MB/s | 85 kB 00:00 2023-11-25T04:21:01.393 INFO:teuthology.orchestra.run.smithi076.stdout:(75/135): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 63 kB 00:00 2023-11-25T04:21:01.410 INFO:teuthology.orchestra.run.smithi076.stdout:(76/135): zlib-devel-1.2.11-40.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2023-11-25T04:21:01.435 INFO:teuthology.orchestra.run.smithi076.stdout:(77/135): fuse-2.9.9-15.el9.x86_64.rpm 1.8 MB/s | 79 kB 00:00 2023-11-25T04:21:01.447 INFO:teuthology.orchestra.run.smithi092.stdout:(97/135): python3-cachetools-4.2.4-1.el9.noarch 276 kB/s | 32 kB 00:00 2023-11-25T04:21:01.469 INFO:teuthology.orchestra.run.smithi076.stdout:(78/135): libgfortran-11.4.1-2.3.el9.x86_64.rpm 13 MB/s | 803 kB 00:00 2023-11-25T04:21:01.470 INFO:teuthology.orchestra.run.smithi195.stdout:(18/135): python3-ceph-common-18.0.0-7506.gec92 1.3 MB/s | 129 kB 00:00 2023-11-25T04:21:01.494 INFO:teuthology.orchestra.run.smithi076.stdout:(79/135): libquadmath-11.4.1-2.3.el9.x86_64.rpm 3.3 MB/s | 195 kB 00:00 2023-11-25T04:21:01.520 INFO:teuthology.orchestra.run.smithi076.stdout:(80/135): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2023-11-25T04:21:01.553 INFO:teuthology.orchestra.run.smithi076.stdout:(81/135): python3-cryptography-36.0.1-4.el9.x86 20 MB/s | 1.1 MB 00:00 2023-11-25T04:21:01.564 INFO:teuthology.orchestra.run.smithi092.stdout:(98/135): python3-certifi-2021.10.8-2.el9.noarc 129 kB/s | 15 kB 00:00 2023-11-25T04:21:01.570 INFO:teuthology.orchestra.run.smithi195.stdout:(19/135): python3-cephfs-18.0.0-7506.gec928368. 1.6 MB/s | 162 kB 00:00 2023-11-25T04:21:01.578 INFO:teuthology.orchestra.run.smithi076.stdout:(82/135): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 103 kB 00:00 2023-11-25T04:21:01.604 INFO:teuthology.orchestra.run.smithi076.stdout:(83/135): python3-pycparser-2.20-6.el9.noarch.r 2.4 MB/s | 124 kB 00:00 2023-11-25T04:21:01.629 INFO:teuthology.orchestra.run.smithi076.stdout:(84/135): python3-requests-2.25.1-7.el9.noarch. 2.2 MB/s | 113 kB 00:00 2023-11-25T04:21:01.639 INFO:teuthology.orchestra.run.smithi067.stdout:(29/135): ceph-mgr-dashboard-18.0.0-7506.gec928 3.9 MB/s | 1.8 MB 00:00 2023-11-25T04:21:01.654 INFO:teuthology.orchestra.run.smithi076.stdout:(85/135): python3-urllib3-1.26.5-4.el9.noarch.r 3.6 MB/s | 187 kB 00:00 2023-11-25T04:21:01.745 INFO:teuthology.orchestra.run.smithi195.stdout:(20/135): python3-rados-18.0.0-7506.gec928368.e 1.8 MB/s | 320 kB 00:00 2023-11-25T04:21:01.781 INFO:teuthology.orchestra.run.smithi067.stdout:(30/135): ceph-mgr-modules-core-18.0.0-7506.gec 1.7 MB/s | 243 kB 00:00 2023-11-25T04:21:01.856 INFO:teuthology.orchestra.run.smithi092.stdout:(99/135): python3-cheroot-8.6.0-4.el9.noarch.rp 589 kB/s | 172 kB 00:00 2023-11-25T04:21:01.873 INFO:teuthology.orchestra.run.smithi067.stdout:(31/135): ceph-mgr-rook-18.0.0-7506.gec928368.e 548 kB/s | 50 kB 00:00 2023-11-25T04:21:01.912 INFO:teuthology.orchestra.run.smithi195.stdout:(21/135): python3-rbd-18.0.0-7506.gec928368.el9 1.7 MB/s | 298 kB 00:00 2023-11-25T04:21:01.965 INFO:teuthology.orchestra.run.smithi067.stdout:(32/135): ceph-prometheus-alerts-18.0.0-7506.ge 164 kB/s | 15 kB 00:00 2023-11-25T04:21:02.012 INFO:teuthology.orchestra.run.smithi195.stdout:(22/135): python3-rgw-18.0.0-7506.gec928368.el9 997 kB/s | 99 kB 00:00 2023-11-25T04:21:02.115 INFO:teuthology.orchestra.run.smithi067.stdout:(33/135): ceph-volume-18.0.0-7506.gec928368.el9 1.7 MB/s | 262 kB 00:00 2023-11-25T04:21:02.129 INFO:teuthology.orchestra.run.smithi195.stdout:(23/135): rbd-fuse-18.0.0-7506.gec928368.el9.x8 730 kB/s | 85 kB 00:00 2023-11-25T04:21:02.416 INFO:teuthology.orchestra.run.smithi067.stdout:(34/135): cephadm-18.0.0-7506.gec928368.el9.noa 2.5 MB/s | 776 kB 00:00 2023-11-25T04:21:02.440 INFO:teuthology.orchestra.run.smithi092.stdout:(100/135): python3-cherrypy-18.6.1-2.el9.noarch 615 kB/s | 358 kB 00:00 2023-11-25T04:21:02.479 INFO:teuthology.orchestra.run.smithi076.stdout:(86/135): python3-devel-3.9.17-2.el9.x86_64.rpm 297 kB/s | 245 kB 00:00 2023-11-25T04:21:02.591 INFO:teuthology.orchestra.run.smithi067.stdout:(35/135): ceph-mgr-diskprediction-local-18.0.0- 5.4 MB/s | 7.4 MB 00:01 2023-11-25T04:21:02.616 INFO:teuthology.orchestra.run.smithi067.stdout:(36/135): automake-1.16.2-8.el9.noarch.rpm 3.2 MB/s | 662 kB 00:00 2023-11-25T04:21:02.662 INFO:teuthology.orchestra.run.smithi067.stdout:(37/135): bison-3.7.4-5.el9.x86_64.rpm 13 MB/s | 920 kB 00:00 2023-11-25T04:21:02.688 INFO:teuthology.orchestra.run.smithi067.stdout:(38/135): boost-program-options-1.75.0-8.el9.x8 1.5 MB/s | 106 kB 00:00 2023-11-25T04:21:02.713 INFO:teuthology.orchestra.run.smithi067.stdout:(39/135): elfutils-libelf-devel-0.190-1.el9.x86 429 kB/s | 21 kB 00:00 2023-11-25T04:21:02.738 INFO:teuthology.orchestra.run.smithi067.stdout:(40/135): flex-2.6.4-9.el9.x86_64.rpm 6.0 MB/s | 307 kB 00:00 2023-11-25T04:21:02.764 INFO:teuthology.orchestra.run.smithi067.stdout:(41/135): flexiblas-3.0.4-8.el9.x86_64.rpm 634 kB/s | 32 kB 00:00 2023-11-25T04:21:02.773 INFO:teuthology.orchestra.run.smithi092.stdout:(101/135): python3-google-auth-2.23.4-1.el9.noa 592 kB/s | 197 kB 00:00 2023-11-25T04:21:02.779 INFO:teuthology.orchestra.run.smithi076.stdout:(87/135): lua-devel-5.4.4-4.el9.x86_64.rpm 74 kB/s | 22 kB 00:00 2023-11-25T04:21:02.839 INFO:teuthology.orchestra.run.smithi067.stdout:(42/135): flexiblas-netlib-3.0.4-8.el9.x86_64.r 30 MB/s | 3.0 MB 00:00 2023-11-25T04:21:02.865 INFO:teuthology.orchestra.run.smithi067.stdout:(43/135): flexiblas-openblas-openmp-3.0.4-8.el9 176 kB/s | 17 kB 00:00 2023-11-25T04:21:02.882 INFO:teuthology.orchestra.run.smithi092.stdout:(102/135): python3-jaraco-8.2.1-3.el9.noarch.rp 99 kB/s | 11 kB 00:00 2023-11-25T04:21:02.890 INFO:teuthology.orchestra.run.smithi067.stdout:(44/135): jq-1.6-16.el9.x86_64.rpm 3.7 MB/s | 186 kB 00:00 2023-11-25T04:21:02.915 INFO:teuthology.orchestra.run.smithi067.stdout:(45/135): libaio-devel-0.3.111-13.el9.x86_64.rp 202 kB/s | 10 kB 00:00 2023-11-25T04:21:02.940 INFO:teuthology.orchestra.run.smithi067.stdout:(46/135): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 159 kB 00:00 2023-11-25T04:21:02.966 INFO:teuthology.orchestra.run.smithi067.stdout:(47/135): librabbitmq-0.11.0-7.el9.x86_64.rpm 887 kB/s | 44 kB 00:00 2023-11-25T04:21:02.991 INFO:teuthology.orchestra.run.smithi092.stdout:(103/135): python3-jaraco-classes-3.2.1-5.el9.n 163 kB/s | 18 kB 00:00 2023-11-25T04:21:02.999 INFO:teuthology.orchestra.run.smithi067.stdout:(48/135): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 661 kB 00:00 2023-11-25T04:21:03.024 INFO:teuthology.orchestra.run.smithi067.stdout:(49/135): libtool-2.4.6-45.el9.x86_64.rpm 9.9 MB/s | 578 kB 00:00 2023-11-25T04:21:03.029 INFO:teuthology.orchestra.run.smithi195.stdout:(24/135): rbd-mirror-18.0.0-7506.gec928368.el9. 3.3 MB/s | 3.0 MB 00:00 2023-11-25T04:21:03.050 INFO:teuthology.orchestra.run.smithi067.stdout:(50/135): libuuid-devel-2.37.4-15.el9.x86_64.rp 500 kB/s | 24 kB 00:00 2023-11-25T04:21:03.075 INFO:teuthology.orchestra.run.smithi067.stdout:(51/135): libzstd-devel-1.5.1-2.el9.x86_64.rpm 926 kB/s | 46 kB 00:00 2023-11-25T04:21:03.099 INFO:teuthology.orchestra.run.smithi092.stdout:(104/135): python3-jaraco-collections-3.0.0-8.e 214 kB/s | 23 kB 00:00 2023-11-25T04:21:03.100 INFO:teuthology.orchestra.run.smithi067.stdout:(52/135): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.5 MB/s | 282 kB 00:00 2023-11-25T04:21:03.105 INFO:teuthology.orchestra.run.smithi195.stdout:(25/135): rbd-nbd-18.0.0-7506.gec928368.el9.x86 2.2 MB/s | 170 kB 00:00 2023-11-25T04:21:03.126 INFO:teuthology.orchestra.run.smithi067.stdout:(53/135): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2023-11-25T04:21:03.151 INFO:teuthology.orchestra.run.smithi067.stdout:(54/135): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.3 MB/s | 217 kB 00:00 2023-11-25T04:21:03.176 INFO:teuthology.orchestra.run.smithi067.stdout:(55/135): openblas-0.3.21-2.el9.x86_64.rpm 684 kB/s | 34 kB 00:00 2023-11-25T04:21:03.180 INFO:teuthology.orchestra.run.smithi195.stdout:(26/135): ceph-grafana-dashboards-18.0.0-7506.g 373 kB/s | 28 kB 00:00 2023-11-25T04:21:03.216 INFO:teuthology.orchestra.run.smithi092.stdout:(105/135): python3-jaraco-functools-3.5.0-2.el9 167 kB/s | 19 kB 00:00 2023-11-25T04:21:03.264 INFO:teuthology.orchestra.run.smithi195.stdout:(27/135): ceph-mgr-cephadm-18.0.0-7506.gec92836 1.6 MB/s | 133 kB 00:00 2023-11-25T04:21:03.277 INFO:teuthology.orchestra.run.smithi067.stdout:(56/135): openblas-openmp-0.3.21-2.el9.x86_64.r 38 MB/s | 4.7 MB 00:00 2023-11-25T04:21:03.325 INFO:teuthology.orchestra.run.smithi092.stdout:(106/135): python3-jaraco-text-3.2.0-6.el9.noar 181 kB/s | 20 kB 00:00 2023-11-25T04:21:03.386 INFO:teuthology.orchestra.run.smithi067.stdout:(57/135): python3-babel-2.9.1-2.el9.noarch.rpm 28 MB/s | 5.8 MB 00:00 2023-11-25T04:21:03.405 INFO:teuthology.orchestra.run.smithi076.stdout:(88/135): ceph-test-18.0.0-7506.gec928368.el9.x 4.3 MB/s | 41 MB 00:09 2023-11-25T04:21:03.411 INFO:teuthology.orchestra.run.smithi067.stdout:(58/135): python3-jinja2-2.11.3-4.el9.noarch.rp 1.7 MB/s | 228 kB 00:00 2023-11-25T04:21:03.434 INFO:teuthology.orchestra.run.smithi092.stdout:(107/135): python3-jwt+crypto-2.4.0-1.el9.noarc 84 kB/s | 9.0 kB 00:00 2023-11-25T04:21:03.453 INFO:teuthology.orchestra.run.smithi067.stdout:(59/135): python3-jsonpatch-1.21-16.el9.noarch. 366 kB/s | 24 kB 00:00 2023-11-25T04:21:03.478 INFO:teuthology.orchestra.run.smithi067.stdout:(60/135): python3-jsonpointer-2.0-4.el9.noarch. 270 kB/s | 18 kB 00:00 2023-11-25T04:21:03.489 INFO:teuthology.orchestra.run.smithi076.stdout:(89/135): openssl-devel-3.0.7-25.el9.x86_64.rpm 2.2 MB/s | 4.1 MB 00:01 2023-11-25T04:21:03.503 INFO:teuthology.orchestra.run.smithi067.stdout:(61/135): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 144 kB 00:00 2023-11-25T04:21:03.528 INFO:teuthology.orchestra.run.smithi067.stdout:(62/135): python3-markupsafe-1.1.1-12.el9.x86_6 635 kB/s | 32 kB 00:00 2023-11-25T04:21:03.551 INFO:teuthology.orchestra.run.smithi092.stdout:(108/135): python3-jwt-2.4.0-1.el9.noarch.rpm 350 kB/s | 41 kB 00:00 2023-11-25T04:21:03.662 INFO:teuthology.orchestra.run.smithi067.stdout:(63/135): python3-numpy-1.20.1-5.el9.x86_64.rpm 30 MB/s | 4.8 MB 00:00 2023-11-25T04:21:03.672 INFO:teuthology.orchestra.run.smithi195.stdout:(28/135): ceph-mgr-dashboard-18.0.0-7506.gec928 4.3 MB/s | 1.8 MB 00:00 2023-11-25T04:21:03.688 INFO:teuthology.orchestra.run.smithi067.stdout:(64/135): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.4 MB/s | 217 kB 00:00 2023-11-25T04:21:03.713 INFO:teuthology.orchestra.run.smithi067.stdout:(65/135): python3-oauthlib-3.1.1-5.el9.noarch.r 3.4 MB/s | 174 kB 00:00 2023-11-25T04:21:03.730 INFO:teuthology.orchestra.run.smithi067.stdout:(66/135): python3-prettytable-0.7.2-27.el9.noar 993 kB/s | 41 kB 00:00 2023-11-25T04:21:03.747 INFO:teuthology.orchestra.run.smithi076.stdout:(90/135): gperftools-libs-2.9.1-2.el9.x86_64.rp 319 kB/s | 309 kB 00:00 2023-11-25T04:21:03.755 INFO:teuthology.orchestra.run.smithi067.stdout:(67/135): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 132 kB 00:00 2023-11-25T04:21:03.780 INFO:teuthology.orchestra.run.smithi067.stdout:(68/135): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2023-11-25T04:21:03.797 INFO:teuthology.orchestra.run.smithi067.stdout:(69/135): python3-pytz-2021.1-5.el9.noarch.rpm 1.1 MB/s | 47 kB 00:00 2023-11-25T04:21:03.814 INFO:teuthology.orchestra.run.smithi067.stdout:(70/135): python3-requests-oauthlib-1.3.0-12.el 1.3 MB/s | 43 kB 00:00 2023-11-25T04:21:03.826 INFO:teuthology.orchestra.run.smithi092.stdout:(109/135): ceph-osd-18.0.0-7506.gec928368.el9.x 1.2 MB/s | 18 MB 00:15 2023-11-25T04:21:03.848 INFO:teuthology.orchestra.run.smithi076.stdout:(91/135): libarrow-doc-9.0.0-9.el9.noarch.rpm 73 kB/s | 24 kB 00:00 2023-11-25T04:21:03.890 INFO:teuthology.orchestra.run.smithi076.stdout:(92/135): liboath-2.6.7-2.el9.x86_64.rpm 343 kB/s | 49 kB 00:00 2023-11-25T04:21:03.901 INFO:teuthology.orchestra.run.smithi092.stdout:(110/135): libarrow-9.0.0-9.el9.x86_64.rpm 691 kB/s | 4.4 MB 00:06 2023-11-25T04:21:03.981 INFO:teuthology.orchestra.run.smithi067.stdout:(71/135): python3-toml-0.10.2-6.el9.noarch.rpm 234 kB/s | 39 kB 00:00 2023-11-25T04:21:04.044 INFO:teuthology.orchestra.run.smithi092.stdout:(111/135): python3-more-itertools-8.12.0-2.el9. 557 kB/s | 79 kB 00:00 2023-11-25T04:21:04.065 INFO:teuthology.orchestra.run.smithi076.stdout:(93/135): libunwind-1.6.2-1.el9.x86_64.rpm 310 kB/s | 67 kB 00:00 2023-11-25T04:21:04.148 INFO:teuthology.orchestra.run.smithi076.stdout:(94/135): luarocks-3.9.2-1.el9.noarch.rpm 584 kB/s | 151 kB 00:00 2023-11-25T04:21:04.160 INFO:teuthology.orchestra.run.smithi092.stdout:(112/135): python3-natsort-7.1.1-5.el9.noarch.r 495 kB/s | 58 kB 00:00 2023-11-25T04:21:04.269 INFO:teuthology.orchestra.run.smithi092.stdout:(113/135): python3-logutils-0.3.5-21.el9.noarch 105 kB/s | 46 kB 00:00 2023-11-25T04:21:04.281 INFO:teuthology.orchestra.run.smithi195.stdout:(29/135): ceph-osd-18.0.0-7506.gec928368.el9.x8 2.0 MB/s | 18 MB 00:08 2023-11-25T04:21:04.290 INFO:teuthology.orchestra.run.smithi076.stdout:(95/135): libarrow-9.0.0-9.el9.x86_64.rpm 5.0 MB/s | 4.4 MB 00:00 2023-11-25T04:21:04.365 INFO:teuthology.orchestra.run.smithi195.stdout:(30/135): ceph-mgr-modules-core-18.0.0-7506.gec 2.9 MB/s | 243 kB 00:00 2023-11-25T04:21:04.365 INFO:teuthology.orchestra.run.smithi067.stdout:(72/135): ceph-test-18.0.0-7506.gec928368.el9.x 6.2 MB/s | 41 MB 00:06 2023-11-25T04:21:04.374 INFO:teuthology.orchestra.run.smithi076.stdout:(96/135): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 521 kB/s | 43 kB 00:00 2023-11-25T04:21:04.386 INFO:teuthology.orchestra.run.smithi092.stdout:(114/135): python3-portend-3.1.0-2.el9.noarch.r 141 kB/s | 16 kB 00:00 2023-11-25T04:21:04.399 INFO:teuthology.orchestra.run.smithi076.stdout:(97/135): parquet-libs-9.0.0-9.el9.x86_64.rpm 2.4 MB/s | 837 kB 00:00 2023-11-25T04:21:04.440 INFO:teuthology.orchestra.run.smithi195.stdout:(31/135): ceph-mgr-rook-18.0.0-7506.gec928368.e 671 kB/s | 50 kB 00:00 2023-11-25T04:21:04.449 INFO:teuthology.orchestra.run.smithi076.stdout:(98/135): python3-cachetools-4.2.4-1.el9.noarch 428 kB/s | 32 kB 00:00 2023-11-25T04:21:04.475 INFO:teuthology.orchestra.run.smithi076.stdout:(99/135): python3-certifi-2021.10.8-2.el9.noarc 199 kB/s | 15 kB 00:00 2023-11-25T04:21:04.498 INFO:teuthology.orchestra.run.smithi195.stdout:(32/135): ceph-prometheus-alerts-18.0.0-7506.ge 259 kB/s | 15 kB 00:00 2023-11-25T04:21:04.525 INFO:teuthology.orchestra.run.smithi076.stdout:(100/135): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2023-11-25T04:21:04.532 INFO:teuthology.orchestra.run.smithi067.stdout:(73/135): python3-scipy-1.6.2-8.el9.x86_64.rpm 21 MB/s | 16 MB 00:00 2023-11-25T04:21:04.557 INFO:teuthology.orchestra.run.smithi067.stdout:(74/135): socat-1.7.4.1-5.el9.x86_64.rpm 523 kB/s | 300 kB 00:00 2023-11-25T04:21:04.561 INFO:teuthology.orchestra.run.smithi092.stdout:(115/135): python3-pecan-1.4.2-3.el9.noarch.rpm 680 kB/s | 272 kB 00:00 2023-11-25T04:21:04.567 INFO:teuthology.orchestra.run.smithi076.stdout:(101/135): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2023-11-25T04:21:04.573 INFO:teuthology.orchestra.run.smithi195.stdout:(33/135): ceph-volume-18.0.0-7506.gec928368.el9 3.4 MB/s | 262 kB 00:00 2023-11-25T04:21:04.582 INFO:teuthology.orchestra.run.smithi067.stdout:(75/135): xfsprogs-devel-5.19.0-4.el9.x86_64.rp 398 kB/s | 85 kB 00:00 2023-11-25T04:21:04.600 INFO:teuthology.orchestra.run.smithi076.stdout:(102/135): python3-google-auth-2.23.4-1.el9.noa 2.6 MB/s | 197 kB 00:00 2023-11-25T04:21:04.607 INFO:teuthology.orchestra.run.smithi067.stdout:(76/135): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2023-11-25T04:21:04.624 INFO:teuthology.orchestra.run.smithi067.stdout:(77/135): zlib-devel-1.2.11-40.el9.x86_64.rpm 665 kB/s | 44 kB 00:00 2023-11-25T04:21:04.649 INFO:teuthology.orchestra.run.smithi067.stdout:(78/135): fuse-2.9.9-15.el9.x86_64.rpm 1.2 MB/s | 79 kB 00:00 2023-11-25T04:21:04.661 INFO:teuthology.orchestra.run.smithi076.stdout:(103/135): python3-jaraco-8.2.1-3.el9.noarch.rp 114 kB/s | 11 kB 00:00 2023-11-25T04:21:04.678 INFO:teuthology.orchestra.run.smithi092.stdout:(116/135): python3-repoze-lru-0.7-16.el9.noarch 263 kB/s | 31 kB 00:00 2023-11-25T04:21:04.683 INFO:teuthology.orchestra.run.smithi067.stdout:(79/135): libquadmath-11.4.1-2.3.el9.x86_64.rpm 3.3 MB/s | 195 kB 00:00 2023-11-25T04:21:04.684 INFO:teuthology.orchestra.run.smithi076.stdout:(104/135): python3-jaraco-classes-3.2.1-5.el9.n 211 kB/s | 18 kB 00:00 2023-11-25T04:21:04.703 INFO:teuthology.orchestra.run.smithi092.stdout:(117/135): python3-pyOpenSSL-21.0.0-1.el9.noarc 284 kB/s | 90 kB 00:00 2023-11-25T04:21:04.717 INFO:teuthology.orchestra.run.smithi067.stdout:(80/135): libgfortran-11.4.1-2.3.el9.x86_64.rpm 8.5 MB/s | 803 kB 00:00 2023-11-25T04:21:04.724 INFO:teuthology.orchestra.run.smithi195.stdout:(34/135): cephadm-18.0.0-7506.gec928368.el9.noa 5.1 MB/s | 776 kB 00:00 2023-11-25T04:21:04.735 INFO:teuthology.orchestra.run.smithi076.stdout:(105/135): python3-jaraco-collections-3.0.0-8.e 316 kB/s | 23 kB 00:00 2023-11-25T04:21:04.742 INFO:teuthology.orchestra.run.smithi067.stdout:(81/135): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.6 MB/s | 241 kB 00:00 2023-11-25T04:21:04.760 INFO:teuthology.orchestra.run.smithi076.stdout:(106/135): python3-jaraco-functools-3.5.0-2.el9 258 kB/s | 19 kB 00:00 2023-11-25T04:21:04.775 INFO:teuthology.orchestra.run.smithi067.stdout:(82/135): python3-cryptography-36.0.1-4.el9.x86 13 MB/s | 1.1 MB 00:00 2023-11-25T04:21:04.800 INFO:teuthology.orchestra.run.smithi067.stdout:(83/135): python3-pycparser-2.20-6.el9.noarch.r 2.2 MB/s | 124 kB 00:00 2023-11-25T04:21:04.802 INFO:teuthology.orchestra.run.smithi076.stdout:(107/135): python3-asyncssh-2.13.2-3.el9.noarch 839 kB/s | 548 kB 00:00 2023-11-25T04:21:04.817 INFO:teuthology.orchestra.run.smithi067.stdout:(84/135): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 103 kB 00:00 2023-11-25T04:21:04.827 INFO:teuthology.orchestra.run.smithi076.stdout:(108/135): python3-jaraco-text-3.2.0-6.el9.noar 213 kB/s | 20 kB 00:00 2023-11-25T04:21:04.843 INFO:teuthology.orchestra.run.smithi067.stdout:(85/135): python3-requests-2.25.1-7.el9.noarch. 1.6 MB/s | 113 kB 00:00 2023-11-25T04:21:04.852 INFO:teuthology.orchestra.run.smithi076.stdout:(109/135): python3-jwt+crypto-2.4.0-1.el9.noarc 98 kB/s | 9.0 kB 00:00 2023-11-25T04:21:04.857 INFO:teuthology.orchestra.run.smithi195.stdout:(35/135): automake-1.16.2-8.el9.noarch.rpm 4.9 MB/s | 662 kB 00:00 2023-11-25T04:21:04.868 INFO:teuthology.orchestra.run.smithi067.stdout:(86/135): python3-urllib3-1.26.5-4.el9.noarch.r 2.7 MB/s | 187 kB 00:00 2023-11-25T04:21:04.908 INFO:teuthology.orchestra.run.smithi195.stdout:(36/135): bison-3.7.4-5.el9.x86_64.rpm 18 MB/s | 920 kB 00:00 2023-11-25T04:21:04.912 INFO:teuthology.orchestra.run.smithi092.stdout:(118/135): python3-rsa-4.9-2.el9.noarch.rpm 284 kB/s | 59 kB 00:00 2023-11-25T04:21:04.919 INFO:teuthology.orchestra.run.smithi076.stdout:(110/135): python3-jwt-2.4.0-1.el9.noarch.rpm 349 kB/s | 41 kB 00:00 2023-11-25T04:21:04.941 INFO:teuthology.orchestra.run.smithi195.stdout:(37/135): boost-program-options-1.75.0-8.el9.x8 3.1 MB/s | 106 kB 00:00 2023-11-25T04:21:04.952 INFO:teuthology.orchestra.run.smithi067.stdout:(87/135): lua-devel-5.4.4-4.el9.x86_64.rpm 268 kB/s | 22 kB 00:00 2023-11-25T04:21:04.953 INFO:teuthology.orchestra.run.smithi076.stdout:(111/135): python3-kubernetes-26.1.0-2.el9.noar 8.2 MB/s | 1.0 MB 00:00 2023-11-25T04:21:04.970 INFO:teuthology.orchestra.run.smithi076.stdout:(112/135): python3-logutils-0.3.5-21.el9.noarch 395 kB/s | 46 kB 00:00 2023-11-25T04:21:04.975 INFO:teuthology.orchestra.run.smithi195.stdout:(38/135): elfutils-libelf-devel-0.190-1.el9.x86 647 kB/s | 21 kB 00:00 2023-11-25T04:21:04.978 INFO:teuthology.orchestra.run.smithi092.stdout:(119/135): python3-kubernetes-26.1.0-2.el9.noar 732 kB/s | 1.0 MB 00:01 2023-11-25T04:21:04.996 INFO:teuthology.orchestra.run.smithi092.stdout:(120/135): python3-routes-2.5.1-5.el9.noarch.rp 594 kB/s | 188 kB 00:00 2023-11-25T04:21:05.029 INFO:teuthology.orchestra.run.smithi092.stdout:(121/135): python3-tempora-5.0.0-2.el9.noarch.r 305 kB/s | 36 kB 00:00 2023-11-25T04:21:05.037 INFO:teuthology.orchestra.run.smithi076.stdout:(113/135): python3-natsort-7.1.1-5.el9.noarch.r 866 kB/s | 58 kB 00:00 2023-11-25T04:21:05.062 INFO:teuthology.orchestra.run.smithi076.stdout:(114/135): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2023-11-25T04:21:05.073 INFO:teuthology.orchestra.run.smithi195.stdout:(39/135): ceph-mgr-diskprediction-local-18.0.0- 5.3 MB/s | 7.4 MB 00:01 2023-11-25T04:21:05.088 INFO:teuthology.orchestra.run.smithi076.stdout:(115/135): python3-more-itertools-8.12.0-2.el9. 470 kB/s | 79 kB 00:00 2023-11-25T04:21:05.096 INFO:teuthology.orchestra.run.smithi092.stdout:(122/135): python3-typing-extensions-4.4.0-2.el 435 kB/s | 51 kB 00:00 2023-11-25T04:21:05.098 INFO:teuthology.orchestra.run.smithi195.stdout:(40/135): flex-2.6.4-9.el9.x86_64.rpm 2.5 MB/s | 307 kB 00:00 2023-11-25T04:21:05.105 INFO:teuthology.orchestra.run.smithi076.stdout:(116/135): python3-portend-3.1.0-2.el9.noarch.r 244 kB/s | 16 kB 00:00 2023-11-25T04:21:05.123 INFO:teuthology.orchestra.run.smithi195.stdout:(41/135): flexiblas-3.0.4-8.el9.x86_64.rpm 638 kB/s | 32 kB 00:00 2023-11-25T04:21:05.154 INFO:teuthology.orchestra.run.smithi076.stdout:(117/135): python3-repoze-lru-0.7-16.el9.noarch 461 kB/s | 31 kB 00:00 2023-11-25T04:21:05.189 INFO:teuthology.orchestra.run.smithi076.stdout:(118/135): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2023-11-25T04:21:05.199 INFO:teuthology.orchestra.run.smithi195.stdout:(42/135): flexiblas-netlib-3.0.4-8.el9.x86_64.r 30 MB/s | 3.0 MB 00:00 2023-11-25T04:21:05.213 INFO:teuthology.orchestra.run.smithi076.stdout:(119/135): python3-pyOpenSSL-21.0.0-1.el9.noarc 595 kB/s | 90 kB 00:00 2023-11-25T04:21:05.224 INFO:teuthology.orchestra.run.smithi195.stdout:(43/135): flexiblas-openblas-openmp-3.0.4-8.el9 176 kB/s | 17 kB 00:00 2023-11-25T04:21:05.239 INFO:teuthology.orchestra.run.smithi076.stdout:(120/135): python3-rsa-4.9-2.el9.noarch.rpm 702 kB/s | 59 kB 00:00 2023-11-25T04:21:05.249 INFO:teuthology.orchestra.run.smithi195.stdout:(44/135): jq-1.6-16.el9.x86_64.rpm 3.6 MB/s | 186 kB 00:00 2023-11-25T04:21:05.254 INFO:teuthology.orchestra.run.smithi092.stdout:(123/135): python3-websocket-client-1.2.3-2.el9 398 kB/s | 90 kB 00:00 2023-11-25T04:21:05.264 INFO:teuthology.orchestra.run.smithi076.stdout:(121/135): python3-tempora-5.0.0-2.el9.noarch.r 472 kB/s | 36 kB 00:00 2023-11-25T04:21:05.275 INFO:teuthology.orchestra.run.smithi195.stdout:(45/135): libaio-devel-0.3.111-13.el9.x86_64.rp 202 kB/s | 10 kB 00:00 2023-11-25T04:21:05.277 INFO:teuthology.orchestra.run.smithi067.stdout:(88/135): gperftools-libs-2.9.1-2.el9.x86_64.rp 1.1 MB/s | 309 kB 00:00 2023-11-25T04:21:05.300 INFO:teuthology.orchestra.run.smithi195.stdout:(46/135): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 159 kB 00:00 2023-11-25T04:21:05.305 INFO:teuthology.orchestra.run.smithi092.stdout:(124/135): python3-webob-1.8.7-6.el9.noarch.rpm 743 kB/s | 230 kB 00:00 2023-11-25T04:21:05.325 INFO:teuthology.orchestra.run.smithi195.stdout:(47/135): librabbitmq-0.11.0-7.el9.x86_64.rpm 883 kB/s | 44 kB 00:00 2023-11-25T04:21:05.333 INFO:teuthology.orchestra.run.smithi076.stdout:(122/135): python3-webob-1.8.7-6.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2023-11-25T04:21:05.350 INFO:teuthology.orchestra.run.smithi195.stdout:(48/135): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 661 kB 00:00 2023-11-25T04:21:05.356 INFO:teuthology.orchestra.run.smithi076.stdout:(123/135): python3-typing-extensions-4.4.0-2.el 357 kB/s | 51 kB 00:00 2023-11-25T04:21:05.363 INFO:teuthology.orchestra.run.smithi092.stdout:(125/135): python3-zc-lockfile-2.0-10.el9.noarc 184 kB/s | 20 kB 00:00 2023-11-25T04:21:05.382 INFO:teuthology.orchestra.run.smithi076.stdout:(124/135): python3-websocket-client-1.2.3-2.el9 766 kB/s | 90 kB 00:00 2023-11-25T04:21:05.384 INFO:teuthology.orchestra.run.smithi195.stdout:(49/135): libtool-2.4.6-45.el9.x86_64.rpm 9.8 MB/s | 578 kB 00:00 2023-11-25T04:21:05.410 INFO:teuthology.orchestra.run.smithi195.stdout:(50/135): libuuid-devel-2.37.4-15.el9.x86_64.rp 424 kB/s | 24 kB 00:00 2023-11-25T04:21:05.432 INFO:teuthology.orchestra.run.smithi076.stdout:(125/135): python3-werkzeug-2.0.3-3.el9.1.noarc 4.2 MB/s | 427 kB 00:00 2023-11-25T04:21:05.435 INFO:teuthology.orchestra.run.smithi195.stdout:(51/135): libzstd-devel-1.5.1-2.el9.x86_64.rpm 927 kB/s | 46 kB 00:00 2023-11-25T04:21:05.457 INFO:teuthology.orchestra.run.smithi076.stdout:(126/135): python3-zc-lockfile-2.0-10.el9.noarc 198 kB/s | 20 kB 00:00 2023-11-25T04:21:05.460 INFO:teuthology.orchestra.run.smithi195.stdout:(52/135): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.5 MB/s | 282 kB 00:00 2023-11-25T04:21:05.482 INFO:teuthology.orchestra.run.smithi076.stdout:(127/135): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2023-11-25T04:21:05.485 INFO:teuthology.orchestra.run.smithi195.stdout:(53/135): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2023-11-25T04:21:05.502 INFO:teuthology.orchestra.run.smithi067.stdout:(89/135): python3-devel-3.9.17-2.el9.x86_64.rpm 372 kB/s | 245 kB 00:00 2023-11-25T04:21:05.511 INFO:teuthology.orchestra.run.smithi195.stdout:(54/135): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.3 MB/s | 217 kB 00:00 2023-11-25T04:21:05.536 INFO:teuthology.orchestra.run.smithi195.stdout:(55/135): openblas-0.3.21-2.el9.x86_64.rpm 688 kB/s | 34 kB 00:00 2023-11-25T04:21:05.614 INFO:teuthology.orchestra.run.smithi092.stdout:(126/135): re2-20211101-3.el9.x86_64.rpm 621 kB/s | 192 kB 00:00 2023-11-25T04:21:05.641 INFO:teuthology.orchestra.run.smithi076.stdout:(128/135): thrift-0.15.0-2.el9.x86_64.rpm 7.6 MB/s | 1.6 MB 00:00 2023-11-25T04:21:05.675 INFO:teuthology.orchestra.run.smithi076.stdout:(129/135): elfutils-debuginfod-client-0.190-1.e 1.1 MB/s | 36 kB 00:00 2023-11-25T04:21:05.714 INFO:teuthology.orchestra.run.smithi092.stdout:(127/135): python3-werkzeug-2.0.3-3.el9.1.noarc 692 kB/s | 427 kB 00:00 2023-11-25T04:21:05.717 INFO:teuthology.orchestra.run.smithi076.stdout:(130/135): elfutils-libelf-0.190-1.el9.x86_64.r 4.5 MB/s | 192 kB 00:00 2023-11-25T04:21:05.759 INFO:teuthology.orchestra.run.smithi076.stdout:(131/135): elfutils-libs-0.190-1.el9.x86_64.rpm 6.0 MB/s | 255 kB 00:00 2023-11-25T04:21:05.871 INFO:teuthology.orchestra.run.smithi067.stdout:(90/135): openssl-devel-3.0.7-25.el9.x86_64.rpm 4.0 MB/s | 4.1 MB 00:01 2023-11-25T04:21:05.921 INFO:teuthology.orchestra.run.smithi195.stdout:(56/135): ceph-test-18.0.0-7506.gec928368.el9.x 4.3 MB/s | 41 MB 00:09 2023-11-25T04:21:05.987 INFO:teuthology.orchestra.run.smithi195.stdout:(57/135): openblas-openmp-0.3.21-2.el9.x86_64.r 10 MB/s | 4.7 MB 00:00 2023-11-25T04:21:06.067 INFO:teuthology.orchestra.run.smithi076.stdout:(132/135): librbd1-18.0.0-7506.gec928368.el9.x8 5.2 MB/s | 3.0 MB 00:00 2023-11-25T04:21:06.087 INFO:teuthology.orchestra.run.smithi195.stdout:(58/135): python3-babel-2.9.1-2.el9.noarch.rpm 11 MB/s | 5.8 MB 00:00 2023-11-25T04:21:06.109 INFO:teuthology.orchestra.run.smithi076.stdout:(133/135): openssl-3.0.7-25.el9.x86_64.rpm 3.5 MB/s | 1.2 MB 00:00 2023-11-25T04:21:06.112 INFO:teuthology.orchestra.run.smithi195.stdout:(59/135): python3-jinja2-2.11.3-4.el9.noarch.rp 1.2 MB/s | 228 kB 00:00 2023-11-25T04:21:06.129 INFO:teuthology.orchestra.run.smithi195.stdout:(60/135): python3-jsonpatch-1.21-16.el9.noarch. 175 kB/s | 24 kB 00:00 2023-11-25T04:21:06.154 INFO:teuthology.orchestra.run.smithi195.stdout:(61/135): python3-jsonpointer-2.0-4.el9.noarch. 273 kB/s | 18 kB 00:00 2023-11-25T04:21:06.155 INFO:teuthology.orchestra.run.smithi067.stdout:(91/135): libarrow-9.0.0-9.el9.x86_64.rpm 5.0 MB/s | 4.4 MB 00:00 2023-11-25T04:21:06.173 INFO:teuthology.orchestra.run.smithi092.stdout:(128/135): librbd1-18.0.0-7506.gec928368.el9.x8 6.6 MB/s | 3.0 MB 00:00 2023-11-25T04:21:06.180 INFO:teuthology.orchestra.run.smithi195.stdout:(62/135): python3-markupsafe-1.1.1-12.el9.x86_6 633 kB/s | 32 kB 00:00 2023-11-25T04:21:06.180 INFO:teuthology.orchestra.run.smithi067.stdout:(92/135): libarrow-doc-9.0.0-9.el9.noarch.rpm 36 kB/s | 24 kB 00:00 2023-11-25T04:21:06.205 INFO:teuthology.orchestra.run.smithi067.stdout:(93/135): liboath-2.6.7-2.el9.x86_64.rpm 146 kB/s | 49 kB 00:00 2023-11-25T04:21:06.206 INFO:teuthology.orchestra.run.smithi092.stdout:(129/135): elfutils-debuginfod-client-0.190-1.e 1.1 MB/s | 36 kB 00:00 2023-11-25T04:21:06.213 INFO:teuthology.orchestra.run.smithi195.stdout:(63/135): python3-mako-1.1.4-6.el9.noarch.rpm 1.7 MB/s | 144 kB 00:00 2023-11-25T04:21:06.230 INFO:teuthology.orchestra.run.smithi067.stdout:(94/135): libunwind-1.6.2-1.el9.x86_64.rpm 899 kB/s | 67 kB 00:00 2023-11-25T04:21:06.251 INFO:teuthology.orchestra.run.smithi076.stdout:(134/135): librados2-18.0.0-7506.gec928368.el9. 4.2 MB/s | 3.3 MB 00:00 2023-11-25T04:21:06.265 INFO:teuthology.orchestra.run.smithi092.stdout:(130/135): elfutils-libelf-0.190-1.el9.x86_64.r 3.2 MB/s | 192 kB 00:00 2023-11-25T04:21:06.272 INFO:teuthology.orchestra.run.smithi067.stdout:(95/135): luarocks-3.9.2-1.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2023-11-25T04:21:06.324 INFO:teuthology.orchestra.run.smithi092.stdout:(131/135): librados2-18.0.0-7506.gec928368.el9. 4.7 MB/s | 3.3 MB 00:00 2023-11-25T04:21:06.347 INFO:teuthology.orchestra.run.smithi067.stdout:(96/135): parquet-libs-9.0.0-9.el9.x86_64.rpm 5.8 MB/s | 837 kB 00:00 2023-11-25T04:21:06.347 INFO:teuthology.orchestra.run.smithi195.stdout:(64/135): python3-numpy-1.20.1-5.el9.x86_64.rpm 25 MB/s | 4.8 MB 00:00 2023-11-25T04:21:06.349 INFO:teuthology.orchestra.run.smithi092.stdout:(132/135): elfutils-libs-0.190-1.el9.x86_64.rpm 3.0 MB/s | 255 kB 00:00 2023-11-25T04:21:06.372 INFO:teuthology.orchestra.run.smithi195.stdout:(65/135): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.1 MB/s | 217 kB 00:00 2023-11-25T04:21:06.397 INFO:teuthology.orchestra.run.smithi195.stdout:(66/135): python3-oauthlib-3.1.1-5.el9.noarch.r 968 kB/s | 174 kB 00:00 2023-11-25T04:21:06.397 INFO:teuthology.orchestra.run.smithi067.stdout:(97/135): python3-asyncssh-2.13.2-3.el9.noarch. 3.2 MB/s | 548 kB 00:00 2023-11-25T04:21:06.414 INFO:teuthology.orchestra.run.smithi195.stdout:(67/135): python3-prettytable-0.7.2-27.el9.noar 624 kB/s | 41 kB 00:00 2023-11-25T04:21:06.414 INFO:teuthology.orchestra.run.smithi067.stdout:(98/135): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 307 kB/s | 43 kB 00:00 2023-11-25T04:21:06.426 INFO:teuthology.orchestra.run.smithi076.stdout:(135/135): openssl-libs-3.0.7-25.el9.x86_64.rpm 6.0 MB/s | 2.2 MB 00:00 2023-11-25T04:21:06.431 INFO:teuthology.orchestra.run.smithi067.stdout:(99/135): python3-cachetools-4.2.4-1.el9.noarch 387 kB/s | 32 kB 00:00 2023-11-25T04:21:06.433 INFO:teuthology.orchestra.run.smithi076.stdout:-------------------------------------------------------------------------------- 2023-11-25T04:21:06.433 INFO:teuthology.orchestra.run.smithi076.stdout:Total 10 MB/s | 191 MB 00:18 2023-11-25T04:21:06.439 INFO:teuthology.orchestra.run.smithi195.stdout:(68/135): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 132 kB 00:00 2023-11-25T04:21:06.448 INFO:teuthology.orchestra.run.smithi067.stdout:(100/135): python3-certifi-2021.10.8-2.el9.noar 298 kB/s | 15 kB 00:00 2023-11-25T04:21:06.465 INFO:teuthology.orchestra.run.smithi195.stdout:(69/135): python3-pyasn1-modules-0.4.8-6.el9.no 3.1 MB/s | 211 kB 00:00 2023-11-25T04:21:06.482 INFO:teuthology.orchestra.run.smithi067.stdout:(101/135): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2023-11-25T04:21:06.490 INFO:teuthology.orchestra.run.smithi195.stdout:(70/135): python3-pytz-2021.1-5.el9.noarch.rpm 633 kB/s | 47 kB 00:00 2023-11-25T04:21:06.515 INFO:teuthology.orchestra.run.smithi195.stdout:(71/135): python3-requests-oauthlib-1.3.0-12.el 569 kB/s | 43 kB 00:00 2023-11-25T04:21:06.516 INFO:teuthology.orchestra.run.smithi067.stdout:(102/135): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2023-11-25T04:21:06.537 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction check 2023-11-25T04:21:06.540 INFO:teuthology.orchestra.run.smithi067.stdout:(103/135): python3-google-auth-2.23.4-1.el9.noa 2.1 MB/s | 197 kB 00:00 2023-11-25T04:21:06.549 INFO:teuthology.orchestra.run.smithi195.stdout:(72/135): python3-toml-0.10.2-6.el9.noarch.rpm 663 kB/s | 39 kB 00:00 2023-11-25T04:21:06.566 INFO:teuthology.orchestra.run.smithi067.stdout:(104/135): python3-jaraco-8.2.1-3.el9.noarch.rp 129 kB/s | 11 kB 00:00 2023-11-25T04:21:06.591 INFO:teuthology.orchestra.run.smithi067.stdout:(105/135): python3-jaraco-classes-3.2.1-5.el9.n 236 kB/s | 18 kB 00:00 2023-11-25T04:21:06.625 INFO:teuthology.orchestra.run.smithi067.stdout:(106/135): python3-jaraco-collections-3.0.0-8.e 397 kB/s | 23 kB 00:00 2023-11-25T04:21:06.650 INFO:teuthology.orchestra.run.smithi067.stdout:(107/135): python3-jaraco-functools-3.5.0-2.el9 233 kB/s | 19 kB 00:00 2023-11-25T04:21:06.675 INFO:teuthology.orchestra.run.smithi067.stdout:(108/135): python3-jaraco-text-3.2.0-6.el9.noar 235 kB/s | 20 kB 00:00 2023-11-25T04:21:06.708 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction check succeeded. 2023-11-25T04:21:06.708 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction test 2023-11-25T04:21:06.708 INFO:teuthology.orchestra.run.smithi067.stdout:(109/135): python3-jwt+crypto-2.4.0-1.el9.noarc 155 kB/s | 9.0 kB 00:00 2023-11-25T04:21:06.716 INFO:teuthology.orchestra.run.smithi195.stdout:(73/135): socat-1.7.4.1-5.el9.x86_64.rpm 1.5 MB/s | 300 kB 00:00 2023-11-25T04:21:06.734 INFO:teuthology.orchestra.run.smithi067.stdout:(110/135): python3-jwt-2.4.0-1.el9.noarch.rpm 490 kB/s | 41 kB 00:00 2023-11-25T04:21:06.826 INFO:teuthology.orchestra.run.smithi067.stdout:(111/135): python3-kubernetes-26.1.0-2.el9.noar 6.8 MB/s | 1.0 MB 00:00 2023-11-25T04:21:06.858 INFO:teuthology.orchestra.run.smithi067.stdout:(112/135): python3-logutils-0.3.5-21.el9.noarch 400 kB/s | 46 kB 00:00 2023-11-25T04:21:06.876 INFO:teuthology.orchestra.run.smithi067.stdout:(113/135): python3-more-itertools-8.12.0-2.el9. 560 kB/s | 79 kB 00:00 2023-11-25T04:21:06.901 INFO:teuthology.orchestra.run.smithi067.stdout:(114/135): python3-natsort-7.1.1-5.el9.noarch.r 770 kB/s | 58 kB 00:00 2023-11-25T04:21:06.941 INFO:teuthology.orchestra.run.smithi195.stdout:(74/135): python3-scipy-1.6.2-8.el9.x86_64.rpm 35 MB/s | 16 MB 00:00 2023-11-25T04:21:06.945 INFO:teuthology.orchestra.run.smithi067.stdout:(115/135): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2023-11-25T04:21:06.966 INFO:teuthology.orchestra.run.smithi195.stdout:(75/135): xfsprogs-devel-5.19.0-4.el9.x86_64.rp 206 kB/s | 85 kB 00:00 2023-11-25T04:21:06.968 INFO:teuthology.orchestra.run.smithi067.stdout:(116/135): python3-portend-3.1.0-2.el9.noarch.r 179 kB/s | 16 kB 00:00 2023-11-25T04:21:06.985 INFO:teuthology.orchestra.run.smithi067.stdout:(117/135): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.1 MB/s | 90 kB 00:00 2023-11-25T04:21:06.991 INFO:teuthology.orchestra.run.smithi092.stdout:(133/135): openssl-3.0.7-25.el9.x86_64.rpm 1.8 MB/s | 1.2 MB 00:00 2023-11-25T04:21:06.991 INFO:teuthology.orchestra.run.smithi195.stdout:(76/135): xmlstarlet-1.6.1-20.el9.x86_64.rpm 229 kB/s | 63 kB 00:00 2023-11-25T04:21:07.017 INFO:teuthology.orchestra.run.smithi195.stdout:(77/135): zlib-devel-1.2.11-40.el9.x86_64.rpm 590 kB/s | 44 kB 00:00 2023-11-25T04:21:07.019 INFO:teuthology.orchestra.run.smithi067.stdout:(118/135): python3-repoze-lru-0.7-16.el9.noarch 617 kB/s | 31 kB 00:00 2023-11-25T04:21:07.036 INFO:teuthology.orchestra.run.smithi067.stdout:(119/135): python3-routes-2.5.1-5.el9.noarch.rp 2.8 MB/s | 188 kB 00:00 2023-11-25T04:21:07.067 INFO:teuthology.orchestra.run.smithi195.stdout:(78/135): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 79 kB 00:00 2023-11-25T04:21:07.070 INFO:teuthology.orchestra.run.smithi067.stdout:(120/135): python3-rsa-4.9-2.el9.noarch.rpm 702 kB/s | 59 kB 00:00 2023-11-25T04:21:07.092 INFO:teuthology.orchestra.run.smithi195.stdout:(79/135): libquadmath-11.4.1-2.3.el9.x86_64.rpm 2.5 MB/s | 195 kB 00:00 2023-11-25T04:21:07.103 INFO:teuthology.orchestra.run.smithi067.stdout:(121/135): python3-tempora-5.0.0-2.el9.noarch.r 428 kB/s | 36 kB 00:00 2023-11-25T04:21:07.129 INFO:teuthology.orchestra.run.smithi067.stdout:(122/135): python3-typing-extensions-4.4.0-2.el 876 kB/s | 51 kB 00:00 2023-11-25T04:21:07.134 INFO:teuthology.orchestra.run.smithi195.stdout:(80/135): libgfortran-11.4.1-2.3.el9.x86_64.rpm 5.5 MB/s | 803 kB 00:00 2023-11-25T04:21:07.154 INFO:teuthology.orchestra.run.smithi067.stdout:(123/135): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2023-11-25T04:21:07.159 INFO:teuthology.orchestra.run.smithi195.stdout:(81/135): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.6 MB/s | 241 kB 00:00 2023-11-25T04:21:07.187 INFO:teuthology.orchestra.run.smithi067.stdout:(124/135): python3-websocket-client-1.2.3-2.el9 1.1 MB/s | 90 kB 00:00 2023-11-25T04:21:07.201 INFO:teuthology.orchestra.run.smithi195.stdout:(82/135): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.1 MB 00:00 2023-11-25T04:21:07.212 INFO:teuthology.orchestra.run.smithi067.stdout:(125/135): python3-werkzeug-2.0.3-3.el9.1.noarc 7.2 MB/s | 427 kB 00:00 2023-11-25T04:21:07.226 INFO:teuthology.orchestra.run.smithi195.stdout:(83/135): python3-pycparser-2.20-6.el9.noarch.r 1.9 MB/s | 124 kB 00:00 2023-11-25T04:21:07.238 INFO:teuthology.orchestra.run.smithi067.stdout:(126/135): python3-zc-lockfile-2.0-10.el9.noarc 239 kB/s | 20 kB 00:00 2023-11-25T04:21:07.252 INFO:teuthology.orchestra.run.smithi195.stdout:(84/135): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 103 kB 00:00 2023-11-25T04:21:07.263 INFO:teuthology.orchestra.run.smithi067.stdout:(127/135): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2023-11-25T04:21:07.277 INFO:teuthology.orchestra.run.smithi195.stdout:(85/135): python3-requests-2.25.1-7.el9.noarch. 1.5 MB/s | 113 kB 00:00 2023-11-25T04:21:07.283 INFO:teuthology.orchestra.run.smithi092.stdout:(134/135): openssl-libs-3.0.7-25.el9.x86_64.rpm 2.3 MB/s | 2.2 MB 00:00 2023-11-25T04:21:07.302 INFO:teuthology.orchestra.run.smithi195.stdout:(86/135): python3-urllib3-1.26.5-4.el9.noarch.r 2.4 MB/s | 187 kB 00:00 2023-11-25T04:21:07.372 INFO:teuthology.orchestra.run.smithi067.stdout:(128/135): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2023-11-25T04:21:07.405 INFO:teuthology.orchestra.run.smithi067.stdout:(129/135): elfutils-debuginfod-client-0.190-1.e 1.1 MB/s | 36 kB 00:00 2023-11-25T04:21:07.439 INFO:teuthology.orchestra.run.smithi067.stdout:(130/135): elfutils-libelf-0.190-1.el9.x86_64.r 5.7 MB/s | 192 kB 00:00 2023-11-25T04:21:07.461 INFO:teuthology.orchestra.run.smithi195.stdout:(87/135): lua-devel-5.4.4-4.el9.x86_64.rpm 141 kB/s | 22 kB 00:00 2023-11-25T04:21:07.481 INFO:teuthology.orchestra.run.smithi067.stdout:(131/135): elfutils-libs-0.190-1.el9.x86_64.rpm 6.0 MB/s | 255 kB 00:00 2023-11-25T04:21:07.511 INFO:teuthology.orchestra.run.smithi195.stdout:(88/135): python3-devel-3.9.17-2.el9.x86_64.rpm 1.0 MB/s | 245 kB 00:00 2023-11-25T04:21:07.639 INFO:teuthology.orchestra.run.smithi067.stdout:(132/135): librados2-18.0.0-7506.gec928368.el9. 8.3 MB/s | 3.3 MB 00:00 2023-11-25T04:21:07.773 INFO:teuthology.orchestra.run.smithi067.stdout:(133/135): librbd1-18.0.0-7506.gec928368.el9.x8 5.9 MB/s | 3.0 MB 00:00 2023-11-25T04:21:07.836 INFO:teuthology.orchestra.run.smithi195.stdout:(89/135): gperftools-libs-2.9.1-2.el9.x86_64.rp 823 kB/s | 309 kB 00:00 2023-11-25T04:21:07.873 INFO:teuthology.orchestra.run.smithi067.stdout:(134/135): openssl-3.0.7-25.el9.x86_64.rpm 3.1 MB/s | 1.2 MB 00:00 2023-11-25T04:21:07.883 INFO:teuthology.orchestra.run.smithi092.stdout:(135/135): thrift-0.15.0-2.el9.x86_64.rpm 643 kB/s | 1.6 MB 00:02 2023-11-25T04:21:07.887 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2023-11-25T04:21:07.887 INFO:teuthology.orchestra.run.smithi092.stdout:Total 9.0 MB/s | 191 MB 00:21 2023-11-25T04:21:07.903 INFO:teuthology.orchestra.run.smithi195.stdout:(90/135): libarrow-doc-9.0.0-9.el9.noarch.rpm 367 kB/s | 24 kB 00:00 2023-11-25T04:21:07.970 INFO:teuthology.orchestra.run.smithi195.stdout:(91/135): liboath-2.6.7-2.el9.x86_64.rpm 729 kB/s | 49 kB 00:00 2023-11-25T04:21:07.990 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2023-11-25T04:21:08.029 INFO:teuthology.orchestra.run.smithi195.stdout:(92/135): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2023-11-25T04:21:08.098 INFO:teuthology.orchestra.run.smithi195.stdout:(93/135): luarocks-3.9.2-1.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2023-11-25T04:21:08.158 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2023-11-25T04:21:08.159 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2023-11-25T04:21:08.215 INFO:teuthology.orchestra.run.smithi067.stdout:(135/135): openssl-libs-3.0.7-25.el9.x86_64.rpm 3.7 MB/s | 2.2 MB 00:00 2023-11-25T04:21:08.222 INFO:teuthology.orchestra.run.smithi067.stdout:-------------------------------------------------------------------------------- 2023-11-25T04:21:08.222 INFO:teuthology.orchestra.run.smithi067.stdout:Total 12 MB/s | 191 MB 00:15 2023-11-25T04:21:08.231 INFO:teuthology.orchestra.run.smithi195.stdout:(94/135): parquet-libs-9.0.0-9.el9.x86_64.rpm 6.1 MB/s | 837 kB 00:00 2023-11-25T04:21:08.329 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction check 2023-11-25T04:21:08.400 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction test succeeded. 2023-11-25T04:21:08.400 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction 2023-11-25T04:21:08.423 INFO:teuthology.orchestra.run.smithi195.stdout:(95/135): libarrow-9.0.0-9.el9.x86_64.rpm 5.0 MB/s | 4.4 MB 00:00 2023-11-25T04:21:08.457 INFO:teuthology.orchestra.run.smithi195.stdout:(96/135): python3-asyncssh-2.13.2-3.el9.noarch. 2.4 MB/s | 548 kB 00:00 2023-11-25T04:21:08.482 INFO:teuthology.orchestra.run.smithi195.stdout:(97/135): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 756 kB/s | 43 kB 00:00 2023-11-25T04:21:08.514 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction check succeeded. 2023-11-25T04:21:08.514 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction test 2023-11-25T04:21:08.515 INFO:teuthology.orchestra.run.smithi195.stdout:(98/135): python3-cachetools-4.2.4-1.el9.noarch 550 kB/s | 32 kB 00:00 2023-11-25T04:21:08.541 INFO:teuthology.orchestra.run.smithi195.stdout:(99/135): python3-certifi-2021.10.8-2.el9.noarc 256 kB/s | 15 kB 00:00 2023-11-25T04:21:08.583 INFO:teuthology.orchestra.run.smithi195.stdout:(100/135): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2023-11-25T04:21:08.616 INFO:teuthology.orchestra.run.smithi195.stdout:(101/135): python3-cherrypy-18.6.1-2.el9.noarch 4.7 MB/s | 358 kB 00:00 2023-11-25T04:21:08.675 INFO:teuthology.orchestra.run.smithi195.stdout:(102/135): openssl-devel-3.0.7-25.el9.x86_64.rp 3.0 MB/s | 4.1 MB 00:01 2023-11-25T04:21:08.701 INFO:teuthology.orchestra.run.smithi195.stdout:(103/135): python3-google-auth-2.23.4-1.el9.noa 1.6 MB/s | 197 kB 00:00 2023-11-25T04:21:08.726 INFO:teuthology.orchestra.run.smithi195.stdout:(104/135): python3-jaraco-8.2.1-3.el9.noarch.rp 98 kB/s | 11 kB 00:00 2023-11-25T04:21:08.751 INFO:teuthology.orchestra.run.smithi195.stdout:(105/135): python3-jaraco-classes-3.2.1-5.el9.n 236 kB/s | 18 kB 00:00 2023-11-25T04:21:08.793 INFO:teuthology.orchestra.run.smithi195.stdout:(106/135): python3-jaraco-collections-3.0.0-8.e 347 kB/s | 23 kB 00:00 2023-11-25T04:21:08.818 INFO:teuthology.orchestra.run.smithi195.stdout:(107/135): python3-jaraco-functools-3.5.0-2.el9 211 kB/s | 19 kB 00:00 2023-11-25T04:21:08.843 INFO:teuthology.orchestra.run.smithi195.stdout:(108/135): python3-jaraco-text-3.2.0-6.el9.noar 213 kB/s | 20 kB 00:00 2023-11-25T04:21:08.869 INFO:teuthology.orchestra.run.smithi195.stdout:(109/135): python3-jwt+crypto-2.4.0-1.el9.noarc 120 kB/s | 9.0 kB 00:00 2023-11-25T04:21:08.910 INFO:teuthology.orchestra.run.smithi195.stdout:(110/135): python3-jwt-2.4.0-1.el9.noarch.rpm 613 kB/s | 41 kB 00:00 2023-11-25T04:21:08.944 INFO:teuthology.orchestra.run.smithi195.stdout:(111/135): python3-kubernetes-26.1.0-2.el9.noar 10 MB/s | 1.0 MB 00:00 2023-11-25T04:21:08.969 INFO:teuthology.orchestra.run.smithi195.stdout:(112/135): python3-logutils-0.3.5-21.el9.noarch 461 kB/s | 46 kB 00:00 2023-11-25T04:21:08.995 INFO:teuthology.orchestra.run.smithi195.stdout:(113/135): python3-more-itertools-8.12.0-2.el9. 960 kB/s | 79 kB 00:00 2023-11-25T04:21:09.020 INFO:teuthology.orchestra.run.smithi195.stdout:(114/135): python3-natsort-7.1.1-5.el9.noarch.r 767 kB/s | 58 kB 00:00 2023-11-25T04:21:09.045 INFO:teuthology.orchestra.run.smithi195.stdout:(115/135): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2023-11-25T04:21:09.070 INFO:teuthology.orchestra.run.smithi195.stdout:(116/135): python3-portend-3.1.0-2.el9.noarch.r 218 kB/s | 16 kB 00:00 2023-11-25T04:21:09.095 INFO:teuthology.orchestra.run.smithi195.stdout:(117/135): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2023-11-25T04:21:09.121 INFO:teuthology.orchestra.run.smithi195.stdout:(118/135): python3-repoze-lru-0.7-16.el9.noarch 407 kB/s | 31 kB 00:00 2023-11-25T04:21:09.146 INFO:teuthology.orchestra.run.smithi195.stdout:(119/135): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2023-11-25T04:21:09.172 INFO:teuthology.orchestra.run.smithi195.stdout:(120/135): python3-rsa-4.9-2.el9.noarch.rpm 782 kB/s | 59 kB 00:00 2023-11-25T04:21:09.197 INFO:teuthology.orchestra.run.smithi195.stdout:(121/135): python3-tempora-5.0.0-2.el9.noarch.r 476 kB/s | 36 kB 00:00 2023-11-25T04:21:09.222 INFO:teuthology.orchestra.run.smithi195.stdout:(122/135): python3-typing-extensions-4.4.0-2.el 676 kB/s | 51 kB 00:00 2023-11-25T04:21:09.247 INFO:teuthology.orchestra.run.smithi195.stdout:(123/135): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2023-11-25T04:21:09.273 INFO:teuthology.orchestra.run.smithi195.stdout:(124/135): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2023-11-25T04:21:09.298 INFO:teuthology.orchestra.run.smithi195.stdout:(125/135): python3-werkzeug-2.0.3-3.el9.1.noarc 5.5 MB/s | 427 kB 00:00 2023-11-25T04:21:09.323 INFO:teuthology.orchestra.run.smithi195.stdout:(126/135): python3-zc-lockfile-2.0-10.el9.noarc 265 kB/s | 20 kB 00:00 2023-11-25T04:21:09.348 INFO:teuthology.orchestra.run.smithi195.stdout:(127/135): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2023-11-25T04:21:09.399 INFO:teuthology.orchestra.run.smithi195.stdout:(128/135): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2023-11-25T04:21:09.432 INFO:teuthology.orchestra.run.smithi195.stdout:(129/135): elfutils-debuginfod-client-0.190-1.e 1.1 MB/s | 36 kB 00:00 2023-11-25T04:21:09.474 INFO:teuthology.orchestra.run.smithi195.stdout:(130/135): elfutils-libelf-0.190-1.el9.x86_64.r 4.5 MB/s | 192 kB 00:00 2023-11-25T04:21:09.516 INFO:teuthology.orchestra.run.smithi195.stdout:(131/135): elfutils-libs-0.190-1.el9.x86_64.rpm 6.0 MB/s | 255 kB 00:00 2023-11-25T04:21:09.775 INFO:teuthology.orchestra.run.smithi195.stdout:(132/135): librbd1-18.0.0-7506.gec928368.el9.x8 7.1 MB/s | 3.0 MB 00:00 2023-11-25T04:21:09.942 INFO:teuthology.orchestra.run.smithi195.stdout:(133/135): librados2-18.0.0-7506.gec928368.el9. 5.4 MB/s | 3.3 MB 00:00 2023-11-25T04:21:10.052 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2023-11-25T04:21:10.052 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2023-11-25T04:21:10.292 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction test succeeded. 2023-11-25T04:21:10.292 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction 2023-11-25T04:21:10.342 INFO:teuthology.orchestra.run.smithi195.stdout:(134/135): openssl-3.0.7-25.el9.x86_64.rpm 1.5 MB/s | 1.2 MB 00:00 2023-11-25T04:21:10.475 INFO:teuthology.orchestra.run.smithi195.stdout:(135/135): openssl-libs-3.0.7-25.el9.x86_64.rpm 3.1 MB/s | 2.2 MB 00:00 2023-11-25T04:21:10.482 INFO:teuthology.orchestra.run.smithi195.stdout:-------------------------------------------------------------------------------- 2023-11-25T04:21:10.482 INFO:teuthology.orchestra.run.smithi195.stdout:Total 9.7 MB/s | 191 MB 00:19 2023-11-25T04:21:10.591 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction check 2023-11-25T04:21:10.762 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction check succeeded. 2023-11-25T04:21:10.762 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction test 2023-11-25T04:21:12.477 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction test succeeded. 2023-11-25T04:21:12.478 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction 2023-11-25T04:21:13.779 INFO:teuthology.orchestra.run.smithi076.stdout: Preparing : 1/1 2023-11-25T04:21:13.907 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/142 2023-11-25T04:21:13.939 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : thrift-0.15.0-2.el9.x86_64 2/142 2023-11-25T04:21:13.977 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/142 2023-11-25T04:21:14.014 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 4/142 2023-11-25T04:21:14.039 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/142 2023-11-25T04:21:14.108 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/142 2023-11-25T04:21:14.426 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/142 2023-11-25T04:21:14.474 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : librados2-2:18.0.0-7506.gec928368.el9.x86_64 8/142 2023-11-25T04:21:14.593 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: librados2-2:18.0.0-7506.gec928368.el9.x86_64 8/142 2023-11-25T04:21:14.625 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libcephfs2-2:18.0.0-7506.gec928368.el9.x86_64 9/142 2023-11-25T04:21:14.717 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: libcephfs2-2:18.0.0-7506.gec928368.el9.x86_64 9/142 2023-11-25T04:21:14.761 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-rados-2:18.0.0-7506.gec928368.el9.x86_64 10/142 2023-11-25T04:21:14.850 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : elfutils-libelf-0.190-1.el9.x86_64 11/142 2023-11-25T04:21:14.877 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libcephsqlite-2:18.0.0-7506.gec928368.el9.x86_64 12/142 2023-11-25T04:21:14.978 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: libcephsqlite-2:18.0.0-7506.gec928368.el9.x86_64 12/142 2023-11-25T04:21:15.024 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libradosstriper1-2:18.0.0-7506.gec928368.el9.x86 13/142 2023-11-25T04:21:15.153 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: libradosstriper1-2:18.0.0-7506.gec928368.el9.x86 13/142 2023-11-25T04:21:15.190 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 14/142 2023-11-25T04:21:15.266 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : liboath-2.6.7-2.el9.x86_64 15/142 2023-11-25T04:21:15.312 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-urllib3-1.26.5-4.el9.noarch 16/142 2023-11-25T04:21:15.349 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 17/142 2023-11-25T04:21:15.387 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libquadmath-11.4.1-2.3.el9.x86_64 18/142 2023-11-25T04:21:15.427 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libgfortran-11.4.1-2.3.el9.x86_64 19/142 2023-11-25T04:21:15.491 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/142 2023-11-25T04:21:15.536 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/142 2023-11-25T04:21:15.554 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2023-11-25T04:21:15.569 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/142 2023-11-25T04:21:15.683 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/142 2023-11-25T04:21:15.712 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : thrift-0.15.0-2.el9.x86_64 2/142 2023-11-25T04:21:15.750 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/142 2023-11-25T04:21:15.753 INFO:teuthology.orchestra.run.smithi067.stdout: Preparing : 1/1 2023-11-25T04:21:15.773 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : lua-5.4.4-4.el9.x86_64 23/142 2023-11-25T04:21:15.786 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 4/142 2023-11-25T04:21:15.812 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/142 2023-11-25T04:21:15.861 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/142 2023-11-25T04:21:15.862 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/142 2023-11-25T04:21:15.894 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : thrift-0.15.0-2.el9.x86_64 2/142 2023-11-25T04:21:15.942 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/142 2023-11-25T04:21:15.957 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 24/142 2023-11-25T04:21:15.978 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 4/142 2023-11-25T04:21:15.993 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-ceph-common-2:18.0.0-7506.gec928368.el9. 25/142 2023-11-25T04:21:16.003 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/142 2023-11-25T04:21:16.037 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-ceph-argparse-2:18.0.0-7506.gec928368.el 26/142 2023-11-25T04:21:16.058 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/142 2023-11-25T04:21:16.066 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cephfs-2:18.0.0-7506.gec928368.el9.x86_6 27/142 2023-11-25T04:21:16.135 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 28/142 2023-11-25T04:21:16.189 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/142 2023-11-25T04:21:16.211 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : luarocks-3.9.2-1.el9.noarch 29/142 2023-11-25T04:21:16.246 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : librados2-2:18.0.0-7506.gec928368.el9.x86_64 8/142 2023-11-25T04:21:16.343 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 30/142 2023-11-25T04:21:16.374 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librados2-2:18.0.0-7506.gec928368.el9.x86_64 8/142 2023-11-25T04:21:16.390 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 31/142 2023-11-25T04:21:16.398 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephfs2-2:18.0.0-7506.gec928368.el9.x86_64 9/142 2023-11-25T04:21:16.405 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/142 2023-11-25T04:21:16.454 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : librados2-2:18.0.0-7506.gec928368.el9.x86_64 8/142 2023-11-25T04:21:16.480 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libcephfs2-2:18.0.0-7506.gec928368.el9.x86_64 9/142 2023-11-25T04:21:16.528 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rados-2:18.0.0-7506.gec928368.el9.x86_64 10/142 2023-11-25T04:21:16.567 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librados2-2:18.0.0-7506.gec928368.el9.x86_64 8/142 2023-11-25T04:21:16.596 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : elfutils-libelf-0.190-1.el9.x86_64 11/142 2023-11-25T04:21:16.597 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libcephfs2-2:18.0.0-7506.gec928368.el9.x86_64 9/142 2023-11-25T04:21:16.616 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephsqlite-2:18.0.0-7506.gec928368.el9.x86_64 12/142 2023-11-25T04:21:16.677 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: libcephfs2-2:18.0.0-7506.gec928368.el9.x86_64 9/142 2023-11-25T04:21:16.701 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libcephsqlite-2:18.0.0-7506.gec928368.el9.x86_64 12/142 2023-11-25T04:21:16.726 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libradosstriper1-2:18.0.0-7506.gec928368.el9.x86 13/142 2023-11-25T04:21:16.727 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-rados-2:18.0.0-7506.gec928368.el9.x86_64 10/142 2023-11-25T04:21:16.788 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : elfutils-libelf-0.190-1.el9.x86_64 11/142 2023-11-25T04:21:16.815 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libcephsqlite-2:18.0.0-7506.gec928368.el9.x86_64 12/142 2023-11-25T04:21:16.831 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libradosstriper1-2:18.0.0-7506.gec928368.el9.x86 13/142 2023-11-25T04:21:16.871 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 14/142 2023-11-25T04:21:16.915 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-rsa-4.9-2.el9.noarch 32/142 2023-11-25T04:21:16.918 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: libcephsqlite-2:18.0.0-7506.gec928368.el9.x86_64 12/142 2023-11-25T04:21:16.938 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : liboath-2.6.7-2.el9.x86_64 15/142 2023-11-25T04:21:16.941 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libradosstriper1-2:18.0.0-7506.gec928368.el9.x86 13/142 2023-11-25T04:21:16.985 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-urllib3-1.26.5-4.el9.noarch 16/142 2023-11-25T04:21:16.993 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 33/142 2023-11-25T04:21:17.023 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 17/142 2023-11-25T04:21:17.040 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 34/142 2023-11-25T04:21:17.060 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libquadmath-11.4.1-2.3.el9.x86_64 18/142 2023-11-25T04:21:17.070 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: libradosstriper1-2:18.0.0-7506.gec928368.el9.x86 13/142 2023-11-25T04:21:17.092 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : elfutils-libs-0.190-1.el9.x86_64 35/142 2023-11-25T04:21:17.104 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 14/142 2023-11-25T04:21:17.108 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libgfortran-11.4.1-2.3.el9.x86_64 19/142 2023-11-25T04:21:17.129 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : librados-devel-2:18.0.0-7506.gec928368.el9.x86_6 36/142 2023-11-25T04:21:17.171 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/142 2023-11-25T04:21:17.172 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 37/142 2023-11-25T04:21:17.175 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : liboath-2.6.7-2.el9.x86_64 15/142 2023-11-25T04:21:17.208 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/142 2023-11-25T04:21:17.228 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-urllib3-1.26.5-4.el9.noarch 16/142 2023-11-25T04:21:17.241 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/142 2023-11-25T04:21:17.269 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : lua-5.4.4-4.el9.x86_64 23/142 2023-11-25T04:21:17.290 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 17/142 2023-11-25T04:21:17.312 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 38/142 2023-11-25T04:21:17.326 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libquadmath-11.4.1-2.3.el9.x86_64 18/142 2023-11-25T04:21:17.337 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 24/142 2023-11-25T04:21:17.355 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 39/142 2023-11-25T04:21:17.367 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libgfortran-11.4.1-2.3.el9.x86_64 19/142 2023-11-25T04:21:17.374 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ceph-common-2:18.0.0-7506.gec928368.el9. 25/142 2023-11-25T04:21:17.418 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ceph-argparse-2:18.0.0-7506.gec928368.el 26/142 2023-11-25T04:21:17.442 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/142 2023-11-25T04:21:17.446 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cephfs-2:18.0.0-7506.gec928368.el9.x86_6 27/142 2023-11-25T04:21:17.462 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : re2-1:20211101-3.el9.x86_64 40/142 2023-11-25T04:21:17.482 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/142 2023-11-25T04:21:17.500 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 41/142 2023-11-25T04:21:17.516 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/142 2023-11-25T04:21:17.522 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 28/142 2023-11-25T04:21:17.543 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : lua-5.4.4-4.el9.x86_64 23/142 2023-11-25T04:21:17.554 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 42/142 2023-11-25T04:21:17.600 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : luarocks-3.9.2-1.el9.noarch 29/142 2023-11-25T04:21:17.615 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 24/142 2023-11-25T04:21:17.619 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 43/142 2023-11-25T04:21:17.648 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-ceph-common-2:18.0.0-7506.gec928368.el9. 25/142 2023-11-25T04:21:17.651 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 44/142 2023-11-25T04:21:17.687 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 45/142 2023-11-25T04:21:17.692 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-ceph-argparse-2:18.0.0-7506.gec928368.el 26/142 2023-11-25T04:21:17.720 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cephfs-2:18.0.0-7506.gec928368.el9.x86_6 27/142 2023-11-25T04:21:17.730 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 30/142 2023-11-25T04:21:17.733 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 46/142 2023-11-25T04:21:17.781 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 47/142 2023-11-25T04:21:17.799 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 28/142 2023-11-25T04:21:17.806 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 31/142 2023-11-25T04:21:17.830 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 48/142 2023-11-25T04:21:17.846 INFO:teuthology.orchestra.run.smithi195.stdout: Preparing : 1/1 2023-11-25T04:21:17.878 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : luarocks-3.9.2-1.el9.noarch 29/142 2023-11-25T04:21:17.962 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 49/142 2023-11-25T04:21:17.980 INFO:teuthology.orchestra.run.smithi195.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/142 2023-11-25T04:21:18.007 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 50/142 2023-11-25T04:21:18.016 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 30/142 2023-11-25T04:21:18.035 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : thrift-0.15.0-2.el9.x86_64 2/142 2023-11-25T04:21:18.048 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 51/142 2023-11-25T04:21:18.073 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 31/142 2023-11-25T04:21:18.140 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 52/142 2023-11-25T04:21:18.174 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-google-auth-1:2.23.4-1.el9.noarch 53/142 2023-11-25T04:21:18.214 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 54/142 2023-11-25T04:21:18.239 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 55/142 2023-11-25T04:21:18.290 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/142 2023-11-25T04:21:18.314 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rsa-4.9-2.el9.noarch 32/142 2023-11-25T04:21:18.331 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 4/142 2023-11-25T04:21:18.333 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 56/142 2023-11-25T04:21:18.355 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/142 2023-11-25T04:21:18.383 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 57/142 2023-11-25T04:21:18.393 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 33/142 2023-11-25T04:21:18.409 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/142 2023-11-25T04:21:18.432 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-ply-3.11-14.el9.noarch 58/142 2023-11-25T04:21:18.439 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 34/142 2023-11-25T04:21:18.483 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : elfutils-libs-0.190-1.el9.x86_64 35/142 2023-11-25T04:21:18.506 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 59/142 2023-11-25T04:21:18.521 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librados-devel-2:18.0.0-7506.gec928368.el9.x86_6 36/142 2023-11-25T04:21:18.554 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 37/142 2023-11-25T04:21:18.631 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-rsa-4.9-2.el9.noarch 32/142 2023-11-25T04:21:18.691 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 38/142 2023-11-25T04:21:18.691 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 60/142 2023-11-25T04:21:18.710 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 33/142 2023-11-25T04:21:18.729 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 39/142 2023-11-25T04:21:18.738 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 61/142 2023-11-25T04:21:18.738 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/142 2023-11-25T04:21:18.756 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 34/142 2023-11-25T04:21:18.798 INFO:teuthology.orchestra.run.smithi195.stdout: Upgrading : librados2-2:18.0.0-7506.gec928368.el9.x86_64 8/142 2023-11-25T04:21:18.801 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : elfutils-libs-0.190-1.el9.x86_64 35/142 2023-11-25T04:21:18.835 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : re2-1:20211101-3.el9.x86_64 40/142 2023-11-25T04:21:18.835 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : librados-devel-2:18.0.0-7506.gec928368.el9.x86_6 36/142 2023-11-25T04:21:18.864 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 62/142 2023-11-25T04:21:18.878 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 37/142 2023-11-25T04:21:18.882 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 41/142 2023-11-25T04:21:18.915 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 63/142 2023-11-25T04:21:18.917 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: librados2-2:18.0.0-7506.gec928368.el9.x86_64 8/142 2023-11-25T04:21:18.936 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 42/142 2023-11-25T04:21:18.941 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libcephfs2-2:18.0.0-7506.gec928368.el9.x86_64 9/142 2023-11-25T04:21:18.945 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 64/142 2023-11-25T04:21:18.978 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 65/142 2023-11-25T04:21:19.000 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 43/142 2023-11-25T04:21:19.011 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 66/142 2023-11-25T04:21:19.022 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: libcephfs2-2:18.0.0-7506.gec928368.el9.x86_64 9/142 2023-11-25T04:21:19.028 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 38/142 2023-11-25T04:21:19.041 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 44/142 2023-11-25T04:21:19.042 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : fuse-2.9.9-15.el9.x86_64 67/142 2023-11-25T04:21:19.068 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-rados-2:18.0.0-7506.gec928368.el9.x86_64 10/142 2023-11-25T04:21:19.069 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 39/142 2023-11-25T04:21:19.069 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 68/142 2023-11-25T04:21:19.077 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 45/142 2023-11-25T04:21:19.107 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 69/142 2023-11-25T04:21:19.124 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 46/142 2023-11-25T04:21:19.131 INFO:teuthology.orchestra.run.smithi195.stdout: Upgrading : elfutils-libelf-0.190-1.el9.x86_64 11/142 2023-11-25T04:21:19.137 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 70/142 2023-11-25T04:21:19.160 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libcephsqlite-2:18.0.0-7506.gec928368.el9.x86_64 12/142 2023-11-25T04:21:19.169 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 47/142 2023-11-25T04:21:19.179 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : re2-1:20211101-3.el9.x86_64 40/142 2023-11-25T04:21:19.211 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 48/142 2023-11-25T04:21:19.212 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 71/142 2023-11-25T04:21:19.213 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 41/142 2023-11-25T04:21:19.247 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 72/142 2023-11-25T04:21:19.261 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: libcephsqlite-2:18.0.0-7506.gec928368.el9.x86_64 12/142 2023-11-25T04:21:19.269 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 42/142 2023-11-25T04:21:19.280 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/142 2023-11-25T04:21:19.286 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libradosstriper1-2:18.0.0-7506.gec928368.el9.x86 13/142 2023-11-25T04:21:19.322 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/142 2023-11-25T04:21:19.341 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 49/142 2023-11-25T04:21:19.344 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 43/142 2023-11-25T04:21:19.348 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/142 2023-11-25T04:21:19.380 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 50/142 2023-11-25T04:21:19.381 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 44/142 2023-11-25T04:21:19.417 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 45/142 2023-11-25T04:21:19.421 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 51/142 2023-11-25T04:21:19.427 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: libradosstriper1-2:18.0.0-7506.gec928368.el9.x86 13/142 2023-11-25T04:21:19.436 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/142 2023-11-25T04:21:19.465 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 46/142 2023-11-25T04:21:19.473 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 14/142 2023-11-25T04:21:19.484 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 77/142 2023-11-25T04:21:19.509 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 47/142 2023-11-25T04:21:19.512 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 52/142 2023-11-25T04:21:19.541 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : liboath-2.6.7-2.el9.x86_64 15/142 2023-11-25T04:21:19.551 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 48/142 2023-11-25T04:21:19.556 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-google-auth-1:2.23.4-1.el9.noarch 53/142 2023-11-25T04:21:19.585 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 78/142 2023-11-25T04:21:19.595 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-urllib3-1.26.5-4.el9.noarch 16/142 2023-11-25T04:21:19.597 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 54/142 2023-11-25T04:21:19.629 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 55/142 2023-11-25T04:21:19.632 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 17/142 2023-11-25T04:21:19.634 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/142 2023-11-25T04:21:19.670 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libquadmath-11.4.1-2.3.el9.x86_64 18/142 2023-11-25T04:21:19.692 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 49/142 2023-11-25T04:21:19.710 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libgfortran-11.4.1-2.3.el9.x86_64 19/142 2023-11-25T04:21:19.720 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 56/142 2023-11-25T04:21:19.736 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 50/142 2023-11-25T04:21:19.764 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 57/142 2023-11-25T04:21:19.774 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/142 2023-11-25T04:21:19.778 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 51/142 2023-11-25T04:21:19.809 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/142 2023-11-25T04:21:19.813 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ply-3.11-14.el9.noarch 58/142 2023-11-25T04:21:19.843 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/142 2023-11-25T04:21:19.871 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : lua-5.4.4-4.el9.x86_64 23/142 2023-11-25T04:21:19.874 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 52/142 2023-11-25T04:21:19.876 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 59/142 2023-11-25T04:21:19.920 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-google-auth-1:2.23.4-1.el9.noarch 53/142 2023-11-25T04:21:19.940 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 24/142 2023-11-25T04:21:19.960 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 54/142 2023-11-25T04:21:19.976 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-ceph-common-2:18.0.0-7506.gec928368.el9. 25/142 2023-11-25T04:21:19.992 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 55/142 2023-11-25T04:21:20.019 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-ceph-argparse-2:18.0.0-7506.gec928368.el 26/142 2023-11-25T04:21:20.048 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-cephfs-2:18.0.0-7506.gec928368.el9.x86_6 27/142 2023-11-25T04:21:20.060 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 60/142 2023-11-25T04:21:20.091 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 56/142 2023-11-25T04:21:20.102 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 61/142 2023-11-25T04:21:20.116 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 28/142 2023-11-25T04:21:20.137 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 57/142 2023-11-25T04:21:20.195 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-ply-3.11-14.el9.noarch 58/142 2023-11-25T04:21:20.202 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : luarocks-3.9.2-1.el9.noarch 29/142 2023-11-25T04:21:20.227 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 62/142 2023-11-25T04:21:20.260 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 59/142 2023-11-25T04:21:20.271 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 63/142 2023-11-25T04:21:20.301 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 64/142 2023-11-25T04:21:20.334 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 65/142 2023-11-25T04:21:20.338 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 30/142 2023-11-25T04:21:20.358 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/142 2023-11-25T04:21:20.366 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 66/142 2023-11-25T04:21:20.392 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 31/142 2023-11-25T04:21:20.397 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : fuse-2.9.9-15.el9.x86_64 67/142 2023-11-25T04:21:20.415 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/142 2023-11-25T04:21:20.425 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 68/142 2023-11-25T04:21:20.451 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/142 2023-11-25T04:21:20.461 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 60/142 2023-11-25T04:21:20.462 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 69/142 2023-11-25T04:21:20.484 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/142 2023-11-25T04:21:20.492 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 70/142 2023-11-25T04:21:20.516 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 61/142 2023-11-25T04:21:20.569 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 71/142 2023-11-25T04:21:20.603 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 72/142 2023-11-25T04:21:20.636 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/142 2023-11-25T04:21:20.654 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : openblas-0.3.21-2.el9.x86_64 84/142 2023-11-25T04:21:20.660 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 62/142 2023-11-25T04:21:20.677 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/142 2023-11-25T04:21:20.682 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 85/142 2023-11-25T04:21:20.703 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 63/142 2023-11-25T04:21:20.712 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/142 2023-11-25T04:21:20.731 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 64/142 2023-11-25T04:21:20.760 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 86/142 2023-11-25T04:21:20.769 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 65/142 2023-11-25T04:21:20.802 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/142 2023-11-25T04:21:20.805 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 66/142 2023-11-25T04:21:20.835 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : fuse-2.9.9-15.el9.x86_64 67/142 2023-11-25T04:21:20.840 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 77/142 2023-11-25T04:21:20.863 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 68/142 2023-11-25T04:21:20.901 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 69/142 2023-11-25T04:21:20.906 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-rsa-4.9-2.el9.noarch 32/142 2023-11-25T04:21:20.930 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 70/142 2023-11-25T04:21:20.949 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 78/142 2023-11-25T04:21:20.978 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 33/142 2023-11-25T04:21:21.006 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/142 2023-11-25T04:21:21.010 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 71/142 2023-11-25T04:21:21.028 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 34/142 2023-11-25T04:21:21.041 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 72/142 2023-11-25T04:21:21.075 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/142 2023-11-25T04:21:21.076 INFO:teuthology.orchestra.run.smithi195.stdout: Upgrading : elfutils-libs-0.190-1.el9.x86_64 35/142 2023-11-25T04:21:21.113 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : librados-devel-2:18.0.0-7506.gec928368.el9.x86_6 36/142 2023-11-25T04:21:21.116 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/142 2023-11-25T04:21:21.142 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/142 2023-11-25T04:21:21.155 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 37/142 2023-11-25T04:21:21.225 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/142 2023-11-25T04:21:21.261 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 77/142 2023-11-25T04:21:21.288 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 38/142 2023-11-25T04:21:21.331 INFO:teuthology.orchestra.run.smithi195.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 39/142 2023-11-25T04:21:21.367 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 78/142 2023-11-25T04:21:21.421 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/142 2023-11-25T04:21:21.427 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 87/142 2023-11-25T04:21:21.438 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : re2-1:20211101-3.el9.x86_64 40/142 2023-11-25T04:21:21.476 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 41/142 2023-11-25T04:21:21.529 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 88/142 2023-11-25T04:21:21.530 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 42/142 2023-11-25T04:21:21.612 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 43/142 2023-11-25T04:21:21.643 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 44/142 2023-11-25T04:21:21.679 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 45/142 2023-11-25T04:21:21.714 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/142 2023-11-25T04:21:21.726 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 46/142 2023-11-25T04:21:21.762 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/142 2023-11-25T04:21:21.771 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 47/142 2023-11-25T04:21:21.796 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/142 2023-11-25T04:21:21.813 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 48/142 2023-11-25T04:21:21.828 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/142 2023-11-25T04:21:21.946 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 49/142 2023-11-25T04:21:21.991 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 50/142 2023-11-25T04:21:22.006 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openblas-0.3.21-2.el9.x86_64 84/142 2023-11-25T04:21:22.032 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 51/142 2023-11-25T04:21:22.035 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 85/142 2023-11-25T04:21:22.113 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 86/142 2023-11-25T04:21:22.124 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 52/142 2023-11-25T04:21:22.166 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-google-auth-1:2.23.4-1.el9.noarch 53/142 2023-11-25T04:21:22.182 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/142 2023-11-25T04:21:22.207 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 54/142 2023-11-25T04:21:22.233 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/142 2023-11-25T04:21:22.239 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 55/142 2023-11-25T04:21:22.259 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/142 2023-11-25T04:21:22.291 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/142 2023-11-25T04:21:22.332 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 56/142 2023-11-25T04:21:22.375 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 57/142 2023-11-25T04:21:22.448 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-ply-3.11-14.el9.noarch 58/142 2023-11-25T04:21:22.469 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : openblas-0.3.21-2.el9.x86_64 84/142 2023-11-25T04:21:22.513 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 59/142 2023-11-25T04:21:22.648 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 85/142 2023-11-25T04:21:22.703 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 60/142 2023-11-25T04:21:22.737 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 86/142 2023-11-25T04:21:22.746 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 61/142 2023-11-25T04:21:22.761 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 89/142 2023-11-25T04:21:22.769 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 87/142 2023-11-25T04:21:22.841 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 90/142 2023-11-25T04:21:22.874 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 91/142 2023-11-25T04:21:22.882 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 62/142 2023-11-25T04:21:22.898 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 88/142 2023-11-25T04:21:22.903 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : jq-1.6-16.el9.x86_64 92/142 2023-11-25T04:21:22.932 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 63/142 2023-11-25T04:21:22.932 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 93/142 2023-11-25T04:21:22.961 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 64/142 2023-11-25T04:21:22.965 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 94/142 2023-11-25T04:21:22.995 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 65/142 2023-11-25T04:21:23.027 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 66/142 2023-11-25T04:21:23.058 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : fuse-2.9.9-15.el9.x86_64 67/142 2023-11-25T04:21:23.085 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 68/142 2023-11-25T04:21:23.123 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 69/142 2023-11-25T04:21:23.153 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 70/142 2023-11-25T04:21:23.229 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 71/142 2023-11-25T04:21:23.263 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 72/142 2023-11-25T04:21:23.266 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 95/142 2023-11-25T04:21:23.295 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : librbd1-2:18.0.0-7506.gec928368.el9.x86_64 96/142 2023-11-25T04:21:23.305 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/142 2023-11-25T04:21:23.346 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/142 2023-11-25T04:21:23.381 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/142 2023-11-25T04:21:23.390 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: librbd1-2:18.0.0-7506.gec928368.el9.x86_64 96/142 2023-11-25T04:21:23.424 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-rbd-2:18.0.0-7506.gec928368.el9.x86_64 97/142 2023-11-25T04:21:23.425 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 87/142 2023-11-25T04:21:23.460 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 98/142 2023-11-25T04:21:23.469 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/142 2023-11-25T04:21:23.509 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 77/142 2023-11-25T04:21:23.549 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 88/142 2023-11-25T04:21:23.610 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 78/142 2023-11-25T04:21:23.660 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/142 2023-11-25T04:21:23.892 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 99/142 2023-11-25T04:21:23.923 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : librgw2-2:18.0.0-7506.gec928368.el9.x86_64 100/142 2023-11-25T04:21:23.994 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: librgw2-2:18.0.0-7506.gec928368.el9.x86_64 100/142 2023-11-25T04:21:24.016 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-rgw-2:18.0.0-7506.gec928368.el9.x86_64 101/142 2023-11-25T04:21:24.163 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 89/142 2023-11-25T04:21:24.243 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 90/142 2023-11-25T04:21:24.275 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 91/142 2023-11-25T04:21:24.306 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : jq-1.6-16.el9.x86_64 92/142 2023-11-25T04:21:24.334 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 93/142 2023-11-25T04:21:24.367 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 94/142 2023-11-25T04:21:24.399 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/142 2023-11-25T04:21:24.449 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/142 2023-11-25T04:21:24.483 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/142 2023-11-25T04:21:24.516 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/142 2023-11-25T04:21:24.679 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 95/142 2023-11-25T04:21:24.700 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : openblas-0.3.21-2.el9.x86_64 84/142 2023-11-25T04:21:24.713 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : librbd1-2:18.0.0-7506.gec928368.el9.x86_64 96/142 2023-11-25T04:21:24.739 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 85/142 2023-11-25T04:21:24.796 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librbd1-2:18.0.0-7506.gec928368.el9.x86_64 96/142 2023-11-25T04:21:24.819 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 86/142 2023-11-25T04:21:24.826 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rbd-2:18.0.0-7506.gec928368.el9.x86_64 97/142 2023-11-25T04:21:24.857 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 89/142 2023-11-25T04:21:24.863 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 98/142 2023-11-25T04:21:24.939 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 90/142 2023-11-25T04:21:24.975 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 91/142 2023-11-25T04:21:25.000 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : jq-1.6-16.el9.x86_64 92/142 2023-11-25T04:21:25.029 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 93/142 2023-11-25T04:21:25.063 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 94/142 2023-11-25T04:21:25.289 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 99/142 2023-11-25T04:21:25.317 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librgw2-2:18.0.0-7506.gec928368.el9.x86_64 100/142 2023-11-25T04:21:25.381 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 95/142 2023-11-25T04:21:25.387 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librgw2-2:18.0.0-7506.gec928368.el9.x86_64 100/142 2023-11-25T04:21:25.410 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rgw-2:18.0.0-7506.gec928368.el9.x86_64 101/142 2023-11-25T04:21:25.417 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : librbd1-2:18.0.0-7506.gec928368.el9.x86_64 96/142 2023-11-25T04:21:25.454 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 87/142 2023-11-25T04:21:25.491 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librbd1-2:18.0.0-7506.gec928368.el9.x86_64 96/142 2023-11-25T04:21:25.522 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-rbd-2:18.0.0-7506.gec928368.el9.x86_64 97/142 2023-11-25T04:21:25.558 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 98/142 2023-11-25T04:21:25.579 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 88/142 2023-11-25T04:21:26.011 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 99/142 2023-11-25T04:21:26.036 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : librgw2-2:18.0.0-7506.gec928368.el9.x86_64 100/142 2023-11-25T04:21:26.110 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librgw2-2:18.0.0-7506.gec928368.el9.x86_64 100/142 2023-11-25T04:21:26.125 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-common-2:18.0.0-7506.gec928368.el9.x86_64 102/142 2023-11-25T04:21:26.138 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-rgw-2:18.0.0-7506.gec928368.el9.x86_64 101/142 2023-11-25T04:21:26.157 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-common-2:18.0.0-7506.gec928368.el9.x86_64 102/142 2023-11-25T04:21:26.693 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-common-2:18.0.0-7506.gec928368.el9.x86_64 102/142 2023-11-25T04:21:26.726 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-base-2:18.0.0-7506.gec928368.el9.x86_64 103/142 2023-11-25T04:21:26.807 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-base-2:18.0.0-7506.gec928368.el9.x86_64 103/142 2023-11-25T04:21:26.807 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-11-25T04:21:26.807 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-11-25T04:21:26.807 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-25T04:21:26.825 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 89/142 2023-11-25T04:21:26.844 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-selinux-2:18.0.0-7506.gec928368.el9.x86_64 104/142 2023-11-25T04:21:26.907 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 90/142 2023-11-25T04:21:26.939 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 91/142 2023-11-25T04:21:26.969 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : jq-1.6-16.el9.x86_64 92/142 2023-11-25T04:21:26.998 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 93/142 2023-11-25T04:21:27.031 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 94/142 2023-11-25T04:21:27.332 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 95/142 2023-11-25T04:21:27.378 INFO:teuthology.orchestra.run.smithi195.stdout: Upgrading : librbd1-2:18.0.0-7506.gec928368.el9.x86_64 96/142 2023-11-25T04:21:27.468 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: librbd1-2:18.0.0-7506.gec928368.el9.x86_64 96/142 2023-11-25T04:21:27.499 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-rbd-2:18.0.0-7506.gec928368.el9.x86_64 97/142 2023-11-25T04:21:27.535 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 98/142 2023-11-25T04:21:27.548 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-common-2:18.0.0-7506.gec928368.el9.x86_64 102/142 2023-11-25T04:21:27.581 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-common-2:18.0.0-7506.gec928368.el9.x86_64 102/142 2023-11-25T04:21:27.964 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 99/142 2023-11-25T04:21:27.989 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : librgw2-2:18.0.0-7506.gec928368.el9.x86_64 100/142 2023-11-25T04:21:28.044 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: librgw2-2:18.0.0-7506.gec928368.el9.x86_64 100/142 2023-11-25T04:21:28.074 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-rgw-2:18.0.0-7506.gec928368.el9.x86_64 101/142 2023-11-25T04:21:28.136 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-common-2:18.0.0-7506.gec928368.el9.x86_64 102/142 2023-11-25T04:21:28.183 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-base-2:18.0.0-7506.gec928368.el9.x86_64 103/142 2023-11-25T04:21:28.260 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-base-2:18.0.0-7506.gec928368.el9.x86_64 103/142 2023-11-25T04:21:28.261 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-11-25T04:21:28.261 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-11-25T04:21:28.261 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-25T04:21:28.293 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-selinux-2:18.0.0-7506.gec928368.el9.x86_64 104/142 2023-11-25T04:21:28.328 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-common-2:18.0.0-7506.gec928368.el9.x86_64 102/142 2023-11-25T04:21:28.367 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-common-2:18.0.0-7506.gec928368.el9.x86_64 102/142 2023-11-25T04:21:28.935 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-common-2:18.0.0-7506.gec928368.el9.x86_64 102/142 2023-11-25T04:21:28.970 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-base-2:18.0.0-7506.gec928368.el9.x86_64 103/142 2023-11-25T04:21:29.077 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-base-2:18.0.0-7506.gec928368.el9.x86_64 103/142 2023-11-25T04:21:29.077 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-11-25T04:21:29.077 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-11-25T04:21:29.077 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:21:29.122 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-selinux-2:18.0.0-7506.gec928368.el9.x86_64 104/142 2023-11-25T04:21:30.177 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-common-2:18.0.0-7506.gec928368.el9.x86_64 102/142 2023-11-25T04:21:30.212 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-common-2:18.0.0-7506.gec928368.el9.x86_64 102/142 2023-11-25T04:21:30.773 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-common-2:18.0.0-7506.gec928368.el9.x86_64 102/142 2023-11-25T04:21:30.808 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-base-2:18.0.0-7506.gec928368.el9.x86_64 103/142 2023-11-25T04:21:30.877 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-base-2:18.0.0-7506.gec928368.el9.x86_64 103/142 2023-11-25T04:21:30.877 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-11-25T04:21:30.877 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-11-25T04:21:30.877 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-25T04:21:30.908 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-selinux-2:18.0.0-7506.gec928368.el9.x86_64 104/142 2023-11-25T04:21:38.404 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-selinux-2:18.0.0-7506.gec928368.el9.x86_64 104/142 2023-11-25T04:21:38.404 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /sys 2023-11-25T04:21:38.404 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /proc 2023-11-25T04:21:38.404 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /mnt 2023-11-25T04:21:38.404 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /var/tmp 2023-11-25T04:21:38.405 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /home 2023-11-25T04:21:38.405 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /root 2023-11-25T04:21:38.405 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /tmp 2023-11-25T04:21:38.405 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-25T04:21:38.563 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-volume-2:18.0.0-7506.gec928368.el9.noarch 105/142 2023-11-25T04:21:38.578 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-volume-2:18.0.0-7506.gec928368.el9.noarch 105/142 2023-11-25T04:21:38.578 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-25T04:21:38.578 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-11-25T04:21:38.579 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-25T04:21:39.661 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-selinux-2:18.0.0-7506.gec928368.el9.x86_64 104/142 2023-11-25T04:21:39.661 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /sys 2023-11-25T04:21:39.661 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /proc 2023-11-25T04:21:39.662 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /mnt 2023-11-25T04:21:39.662 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /var/tmp 2023-11-25T04:21:39.662 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /home 2023-11-25T04:21:39.662 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /root 2023-11-25T04:21:39.662 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /tmp 2023-11-25T04:21:39.662 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-25T04:21:39.835 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-volume-2:18.0.0-7506.gec928368.el9.noarch 105/142 2023-11-25T04:21:39.850 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-volume-2:18.0.0-7506.gec928368.el9.noarch 105/142 2023-11-25T04:21:39.851 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-25T04:21:39.851 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-11-25T04:21:39.851 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-25T04:21:40.142 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-osd-2:18.0.0-7506.gec928368.el9.x86_64 106/142 2023-11-25T04:21:40.167 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-osd-2:18.0.0-7506.gec928368.el9.x86_64 106/142 2023-11-25T04:21:40.167 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-25T04:21:40.167 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-11-25T04:21:40.167 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-11-25T04:21:40.167 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-11-25T04:21:40.167 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-25T04:21:40.367 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mds-2:18.0.0-7506.gec928368.el9.x86_64 107/142 2023-11-25T04:21:40.390 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mds-2:18.0.0-7506.gec928368.el9.x86_64 107/142 2023-11-25T04:21:40.390 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-25T04:21:40.390 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-11-25T04:21:40.390 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-11-25T04:21:40.390 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-11-25T04:21:40.390 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-25T04:21:40.826 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mon-2:18.0.0-7506.gec928368.el9.x86_64 108/142 2023-11-25T04:21:45.124 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-selinux-2:18.0.0-7506.gec928368.el9.x86_64 104/142 2023-11-25T04:21:45.124 INFO:teuthology.orchestra.run.smithi195.stdout:skipping the directory /sys 2023-11-25T04:21:45.124 INFO:teuthology.orchestra.run.smithi195.stdout:skipping the directory /proc 2023-11-25T04:21:45.124 INFO:teuthology.orchestra.run.smithi195.stdout:skipping the directory /mnt 2023-11-25T04:21:45.124 INFO:teuthology.orchestra.run.smithi195.stdout:skipping the directory /var/tmp 2023-11-25T04:21:45.148 INFO:teuthology.orchestra.run.smithi195.stdout:skipping the directory /home 2023-11-25T04:21:45.149 INFO:teuthology.orchestra.run.smithi195.stdout:skipping the directory /root 2023-11-25T04:21:45.149 INFO:teuthology.orchestra.run.smithi195.stdout:skipping the directory /tmp 2023-11-25T04:21:45.149 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-25T04:21:45.149 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-volume-2:18.0.0-7506.gec928368.el9.noarch 105/142 2023-11-25T04:21:45.149 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-volume-2:18.0.0-7506.gec928368.el9.noarch 105/142 2023-11-25T04:21:45.149 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-25T04:21:45.149 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-11-25T04:21:45.149 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-25T04:21:45.149 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-osd-2:18.0.0-7506.gec928368.el9.x86_64 106/142 2023-11-25T04:21:45.149 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-osd-2:18.0.0-7506.gec928368.el9.x86_64 106/142 2023-11-25T04:21:45.149 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-25T04:21:45.149 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-11-25T04:21:45.150 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-11-25T04:21:45.150 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-11-25T04:21:45.150 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-25T04:21:45.150 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mds-2:18.0.0-7506.gec928368.el9.x86_64 107/142 2023-11-25T04:21:45.150 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-mds-2:18.0.0-7506.gec928368.el9.x86_64 107/142 2023-11-25T04:21:45.150 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-25T04:21:45.150 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-11-25T04:21:45.150 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-11-25T04:21:45.150 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-11-25T04:21:45.150 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-25T04:21:45.151 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mon-2:18.0.0-7506.gec928368.el9.x86_64 108/142 2023-11-25T04:21:45.151 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-mon-2:18.0.0-7506.gec928368.el9.x86_64 108/142 2023-11-25T04:21:45.151 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-25T04:21:45.151 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-11-25T04:21:45.151 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-11-25T04:21:45.151 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-11-25T04:21:45.151 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-25T04:21:45.151 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : automake-1.16.2-8.el9.noarch 109/142 2023-11-25T04:21:45.153 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-selinux-2:18.0.0-7506.gec928368.el9.x86_64 104/142 2023-11-25T04:21:45.154 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /sys 2023-11-25T04:21:45.154 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /proc 2023-11-25T04:21:45.154 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /mnt 2023-11-25T04:21:45.154 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /var/tmp 2023-11-25T04:21:45.154 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /home 2023-11-25T04:21:45.154 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /root 2023-11-25T04:21:45.154 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /tmp 2023-11-25T04:21:45.154 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:21:45.154 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-volume-2:18.0.0-7506.gec928368.el9.noarch 105/142 2023-11-25T04:21:45.154 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-volume-2:18.0.0-7506.gec928368.el9.noarch 105/142 2023-11-25T04:21:45.154 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-25T04:21:45.155 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-11-25T04:21:45.155 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:21:45.155 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-osd-2:18.0.0-7506.gec928368.el9.x86_64 106/142 2023-11-25T04:21:45.155 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-osd-2:18.0.0-7506.gec928368.el9.x86_64 106/142 2023-11-25T04:21:45.155 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-25T04:21:45.155 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-11-25T04:21:45.155 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-11-25T04:21:45.155 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-11-25T04:21:45.155 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:21:45.155 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mds-2:18.0.0-7506.gec928368.el9.x86_64 107/142 2023-11-25T04:21:45.155 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mds-2:18.0.0-7506.gec928368.el9.x86_64 107/142 2023-11-25T04:21:45.155 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-25T04:21:45.156 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-11-25T04:21:45.156 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-11-25T04:21:45.156 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-11-25T04:21:45.156 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:21:45.156 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mon-2:18.0.0-7506.gec928368.el9.x86_64 108/142 2023-11-25T04:21:45.156 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mon-2:18.0.0-7506.gec928368.el9.x86_64 108/142 2023-11-25T04:21:45.156 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-25T04:21:45.156 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-11-25T04:21:45.156 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-11-25T04:21:45.156 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-11-25T04:21:45.156 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:21:45.156 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : automake-1.16.2-8.el9.noarch 109/142 2023-11-25T04:21:45.156 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: cephadm-2:18.0.0-7506.gec928368.el9.noarch 110/142 2023-11-25T04:21:45.157 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : cephadm-2:18.0.0-7506.gec928368.el9.noarch 110/142 2023-11-25T04:21:45.157 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-7506.gec928368.e 111/142 2023-11-25T04:21:45.157 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-7506.gec928368. 112/142 2023-11-25T04:21:45.157 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-7506.gec928368.el9.noa 113/142 2023-11-25T04:21:45.157 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-7506.gec928368.el9.noa 113/142 2023-11-25T04:21:45.157 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-7506.gec928368.el9.n 114/142 2023-11-25T04:21:45.158 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-osd-2:18.0.0-7506.gec928368.el9.x86_64 106/142 2023-11-25T04:21:45.158 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-osd-2:18.0.0-7506.gec928368.el9.x86_64 106/142 2023-11-25T04:21:45.158 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-25T04:21:45.158 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-11-25T04:21:45.159 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-11-25T04:21:45.159 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-11-25T04:21:45.159 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-25T04:21:45.184 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mds-2:18.0.0-7506.gec928368.el9.x86_64 107/142 2023-11-25T04:21:45.184 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mds-2:18.0.0-7506.gec928368.el9.x86_64 107/142 2023-11-25T04:21:45.185 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-25T04:21:45.185 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-11-25T04:21:45.185 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-11-25T04:21:45.185 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-11-25T04:21:45.186 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-25T04:21:45.186 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mon-2:18.0.0-7506.gec928368.el9.x86_64 108/142 2023-11-25T04:21:45.186 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mon-2:18.0.0-7506.gec928368.el9.x86_64 108/142 2023-11-25T04:21:45.186 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-25T04:21:45.186 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-11-25T04:21:45.187 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-11-25T04:21:45.187 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-11-25T04:21:45.187 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-25T04:21:45.187 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : automake-1.16.2-8.el9.noarch 109/142 2023-11-25T04:21:45.188 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: cephadm-2:18.0.0-7506.gec928368.el9.noarch 110/142 2023-11-25T04:21:45.188 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephadm-2:18.0.0-7506.gec928368.el9.noarch 110/142 2023-11-25T04:21:45.188 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-7506.gec928368.e 111/142 2023-11-25T04:21:45.188 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-7506.gec928368. 112/142 2023-11-25T04:21:45.188 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-7506.gec928368.el9.noa 113/142 2023-11-25T04:21:45.189 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-7506.gec928368.el9.noa 113/142 2023-11-25T04:21:45.189 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-7506.gec928368.el9.n 114/142 2023-11-25T04:21:45.189 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-7506.gec928368.el9.n 114/142 2023-11-25T04:21:45.189 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-7506.gec9 115/142 2023-11-25T04:21:45.189 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-7506.gec9 115/142 2023-11-25T04:21:45.189 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-7506.gec928368.el 116/142 2023-11-25T04:21:45.190 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-2:18.0.0-7506.gec928368.el9.x86_64 117/142 2023-11-25T04:21:45.190 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-2:18.0.0-7506.gec928368.el9.x86_64 117/142 2023-11-25T04:21:45.190 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-25T04:21:45.190 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-11-25T04:21:45.190 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-11-25T04:21:45.191 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-11-25T04:21:45.191 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-25T04:21:45.191 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-rook-2:18.0.0-7506.gec928368.el9.noarch 118/142 2023-11-25T04:21:45.191 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-7506.gec928368.el9.noarch 118/142 2023-11-25T04:21:45.192 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-2:18.0.0-7506.gec928368.el9.x86_64 119/142 2023-11-25T04:21:45.192 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mon-2:18.0.0-7506.gec928368.el9.x86_64 108/142 2023-11-25T04:21:45.193 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-25T04:21:45.193 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-11-25T04:21:45.193 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-11-25T04:21:45.193 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-11-25T04:21:45.193 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-25T04:21:45.193 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : automake-1.16.2-8.el9.noarch 109/142 2023-11-25T04:21:45.193 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: cephadm-2:18.0.0-7506.gec928368.el9.noarch 110/142 2023-11-25T04:21:45.193 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : cephadm-2:18.0.0-7506.gec928368.el9.noarch 110/142 2023-11-25T04:21:45.194 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-7506.gec928368.e 111/142 2023-11-25T04:21:45.194 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-7506.gec928368. 112/142 2023-11-25T04:21:45.194 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-7506.gec928368.el9.noa 113/142 2023-11-25T04:21:45.194 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-7506.gec928368.el9.noa 113/142 2023-11-25T04:21:45.194 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-7506.gec928368.el9.n 114/142 2023-11-25T04:21:45.194 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-7506.gec928368.el9.n 114/142 2023-11-25T04:21:45.194 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-7506.gec9 115/142 2023-11-25T04:21:45.194 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-7506.gec9 115/142 2023-11-25T04:21:45.195 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-7506.gec928368.el 116/142 2023-11-25T04:21:45.195 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-2:18.0.0-7506.gec928368.el9.x86_64 117/142 2023-11-25T04:21:45.195 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mgr-2:18.0.0-7506.gec928368.el9.x86_64 117/142 2023-11-25T04:21:45.195 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-25T04:21:45.195 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-11-25T04:21:45.195 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-11-25T04:21:45.196 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-11-25T04:21:45.196 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-25T04:21:45.196 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-rook-2:18.0.0-7506.gec928368.el9.noarch 118/142 2023-11-25T04:21:45.196 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-7506.gec928368.el9.noarch 118/142 2023-11-25T04:21:45.196 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-2:18.0.0-7506.gec928368.el9.x86_64 119/142 2023-11-25T04:21:45.196 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libtool-2.4.6-45.el9.x86_64 120/142 2023-11-25T04:21:45.197 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-radosgw-2:18.0.0-7506.gec928368.el9.x86_64 121/142 2023-11-25T04:21:45.197 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-7506.gec928368.el9.x86_64 121/142 2023-11-25T04:21:45.197 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-25T04:21:45.197 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-11-25T04:21:45.197 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-11-25T04:21:45.197 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-11-25T04:21:45.197 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-25T04:21:45.198 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-7506.gec928 122/142 2023-11-25T04:21:45.198 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-7506.gec928 122/142 2023-11-25T04:21:45.198 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-25T04:21:45.198 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-11-25T04:21:45.198 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-25T04:21:45.198 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : rbd-mirror-2:18.0.0-7506.gec928368.el9.x86_64 123/142 2023-11-25T04:21:45.198 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: rbd-mirror-2:18.0.0-7506.gec928368.el9.x86_64 123/142 2023-11-25T04:21:45.198 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-25T04:21:45.199 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-11-25T04:21:45.199 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-11-25T04:21:45.199 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-11-25T04:21:45.199 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-25T04:21:45.275 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libtool-2.4.6-45.el9.x86_64 120/142 2023-11-25T04:21:45.310 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-radosgw-2:18.0.0-7506.gec928368.el9.x86_64 121/142 2023-11-25T04:21:45.333 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-7506.gec928368.el9.x86_64 121/142 2023-11-25T04:21:45.333 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-25T04:21:45.333 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-11-25T04:21:45.333 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-11-25T04:21:45.333 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-11-25T04:21:45.333 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-25T04:21:45.369 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-7506.gec928 122/142 2023-11-25T04:21:45.381 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: cephadm-2:18.0.0-7506.gec928368.el9.noarch 110/142 2023-11-25T04:21:45.391 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-7506.gec928 122/142 2023-11-25T04:21:45.391 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-25T04:21:45.391 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-11-25T04:21:45.391 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-25T04:21:45.407 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : cephadm-2:18.0.0-7506.gec928368.el9.noarch 110/142 2023-11-25T04:21:45.437 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-7506.gec928368.e 111/142 2023-11-25T04:21:45.497 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-7506.gec928368. 112/142 2023-11-25T04:21:45.522 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-7506.gec928368.el9.noa 113/142 2023-11-25T04:21:45.686 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-mirror-2:18.0.0-7506.gec928368.el9.x86_64 123/142 2023-11-25T04:21:45.708 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: rbd-mirror-2:18.0.0-7506.gec928368.el9.x86_64 123/142 2023-11-25T04:21:45.708 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-25T04:21:45.708 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-11-25T04:21:45.709 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-11-25T04:21:45.709 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-11-25T04:21:45.709 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-25T04:21:45.716 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-7506.gec928368.el9.n 114/142 2023-11-25T04:21:45.742 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-7506.gec9 115/142 2023-11-25T04:21:45.746 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-7506.gec928368.el9.noa 113/142 2023-11-25T04:21:45.773 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-7506.gec928368.el9.n 114/142 2023-11-25T04:21:45.847 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-7506.gec9 115/142 2023-11-25T04:21:46.018 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-7506.gec928368.el 116/142 2023-11-25T04:21:46.044 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-2:18.0.0-7506.gec928368.el9.x86_64 117/142 2023-11-25T04:21:46.068 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mgr-2:18.0.0-7506.gec928368.el9.x86_64 117/142 2023-11-25T04:21:46.068 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-25T04:21:46.069 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-11-25T04:21:46.069 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-11-25T04:21:46.069 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-11-25T04:21:46.069 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:21:46.111 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-rook-2:18.0.0-7506.gec928368.el9.noarch 118/142 2023-11-25T04:21:46.122 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-7506.gec928368.el9.noarch 118/142 2023-11-25T04:21:46.180 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-2:18.0.0-7506.gec928368.el9.x86_64 119/142 2023-11-25T04:21:46.732 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-7506.gec928368.el9.n 114/142 2023-11-25T04:21:46.765 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-7506.gec9 115/142 2023-11-25T04:21:46.864 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-7506.gec9 115/142 2023-11-25T04:21:47.007 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libtool-2.4.6-45.el9.x86_64 120/142 2023-11-25T04:21:47.029 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-7506.gec928368.el 116/142 2023-11-25T04:21:47.039 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-radosgw-2:18.0.0-7506.gec928368.el9.x86_64 121/142 2023-11-25T04:21:47.062 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-7506.gec928368.el9.x86_64 121/142 2023-11-25T04:21:47.063 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-25T04:21:47.063 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-11-25T04:21:47.063 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-11-25T04:21:47.063 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-11-25T04:21:47.063 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:21:47.066 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mgr-2:18.0.0-7506.gec928368.el9.x86_64 117/142 2023-11-25T04:21:47.090 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-mgr-2:18.0.0-7506.gec928368.el9.x86_64 117/142 2023-11-25T04:21:47.090 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-25T04:21:47.090 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-11-25T04:21:47.090 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-11-25T04:21:47.090 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-11-25T04:21:47.090 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-25T04:21:47.106 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-7506.gec928 122/142 2023-11-25T04:21:47.129 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-7506.gec928 122/142 2023-11-25T04:21:47.130 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-25T04:21:47.130 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-11-25T04:21:47.130 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:21:47.134 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mgr-rook-2:18.0.0-7506.gec928368.el9.noarch 118/142 2023-11-25T04:21:47.145 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-7506.gec928368.el9.noarch 118/142 2023-11-25T04:21:47.212 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-2:18.0.0-7506.gec928368.el9.x86_64 119/142 2023-11-25T04:21:47.440 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : rbd-mirror-2:18.0.0-7506.gec928368.el9.x86_64 123/142 2023-11-25T04:21:47.464 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: rbd-mirror-2:18.0.0-7506.gec928368.el9.x86_64 123/142 2023-11-25T04:21:47.464 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-25T04:21:47.464 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-11-25T04:21:47.464 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-11-25T04:21:47.465 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-11-25T04:21:47.465 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:21:47.989 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libtool-2.4.6-45.el9.x86_64 120/142 2023-11-25T04:21:48.019 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-radosgw-2:18.0.0-7506.gec928368.el9.x86_64 121/142 2023-11-25T04:21:48.042 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-7506.gec928368.el9.x86_64 121/142 2023-11-25T04:21:48.043 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-25T04:21:48.043 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-11-25T04:21:48.043 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-11-25T04:21:48.043 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-11-25T04:21:48.043 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-25T04:21:48.087 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-7506.gec928 122/142 2023-11-25T04:21:48.109 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-7506.gec928 122/142 2023-11-25T04:21:48.109 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-25T04:21:48.109 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-11-25T04:21:48.109 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-25T04:21:48.185 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-test-2:18.0.0-7506.gec928368.el9.x86_64 124/142 2023-11-25T04:21:48.227 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : rbd-fuse-2:18.0.0-7506.gec928368.el9.x86_64 125/142 2023-11-25T04:21:48.267 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : rbd-nbd-2:18.0.0-7506.gec928368.el9.x86_64 126/142 2023-11-25T04:21:48.309 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : xfsprogs-devel-5.19.0-4.el9.x86_64 127/142 2023-11-25T04:21:48.401 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : elfutils-libelf-devel-0.190-1.el9.x86_64 128/142 2023-11-25T04:21:48.404 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : rbd-mirror-2:18.0.0-7506.gec928368.el9.x86_64 123/142 2023-11-25T04:21:48.427 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: rbd-mirror-2:18.0.0-7506.gec928368.el9.x86_64 123/142 2023-11-25T04:21:48.427 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-25T04:21:48.427 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-11-25T04:21:48.427 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-11-25T04:21:48.427 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-11-25T04:21:48.427 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-25T04:21:48.444 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-fuse-2:18.0.0-7506.gec928368.el9.x86_64 129/142 2023-11-25T04:21:48.476 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libcephfs-devel-2:18.0.0-7506.gec928368.el9.x86_ 130/142 2023-11-25T04:21:49.303 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : elfutils-debuginfod-client-0.190-1.el9.x86_64 131/142 2023-11-25T04:21:49.422 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 132/142 2023-11-25T04:21:49.468 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 133/142 2023-11-25T04:21:49.522 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-test-2:18.0.0-7506.gec928368.el9.x86_64 124/142 2023-11-25T04:21:49.561 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : flex-2.6.4-9.el9.x86_64 134/142 2023-11-25T04:21:49.563 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-fuse-2:18.0.0-7506.gec928368.el9.x86_64 125/142 2023-11-25T04:21:49.592 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : bison-3.7.4-5.el9.x86_64 135/142 2023-11-25T04:21:49.604 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-nbd-2:18.0.0-7506.gec928368.el9.x86_64 126/142 2023-11-25T04:21:49.617 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 136/142 2023-11-25T04:21:49.636 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : xfsprogs-devel-5.19.0-4.el9.x86_64 127/142 2023-11-25T04:21:49.673 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 137/142 2023-11-25T04:21:49.673 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 138/142 2023-11-25T04:21:49.730 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : elfutils-libelf-devel-0.190-1.el9.x86_64 128/142 2023-11-25T04:21:49.764 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-fuse-2:18.0.0-7506.gec928368.el9.x86_64 129/142 2023-11-25T04:21:49.764 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 138/142 2023-11-25T04:21:49.794 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 139/142 2023-11-25T04:21:49.794 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 140/142 2023-11-25T04:21:49.796 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephfs-devel-2:18.0.0-7506.gec928368.el9.x86_ 130/142 2023-11-25T04:21:49.866 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 140/142 2023-11-25T04:21:49.896 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 141/142 2023-11-25T04:21:49.953 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 142/142 2023-11-25T04:21:50.639 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : elfutils-debuginfod-client-0.190-1.el9.x86_64 131/142 2023-11-25T04:21:50.731 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 132/142 2023-11-25T04:21:50.772 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 133/142 2023-11-25T04:21:50.871 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flex-2.6.4-9.el9.x86_64 134/142 2023-11-25T04:21:50.909 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : bison-3.7.4-5.el9.x86_64 135/142 2023-11-25T04:21:50.934 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 136/142 2023-11-25T04:21:50.961 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 137/142 2023-11-25T04:21:50.961 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 138/142 2023-11-25T04:21:51.048 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 138/142 2023-11-25T04:21:51.078 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 139/142 2023-11-25T04:21:51.078 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 140/142 2023-11-25T04:21:51.164 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 140/142 2023-11-25T04:21:51.196 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 141/142 2023-11-25T04:21:51.261 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 142/142 2023-11-25T04:21:51.510 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-test-2:18.0.0-7506.gec928368.el9.x86_64 124/142 2023-11-25T04:21:51.551 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : rbd-fuse-2:18.0.0-7506.gec928368.el9.x86_64 125/142 2023-11-25T04:21:51.593 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : rbd-nbd-2:18.0.0-7506.gec928368.el9.x86_64 126/142 2023-11-25T04:21:51.624 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : xfsprogs-devel-5.19.0-4.el9.x86_64 127/142 2023-11-25T04:21:51.723 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : elfutils-libelf-devel-0.190-1.el9.x86_64 128/142 2023-11-25T04:21:51.751 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-fuse-2:18.0.0-7506.gec928368.el9.x86_64 129/142 2023-11-25T04:21:51.784 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libcephfs-devel-2:18.0.0-7506.gec928368.el9.x86_ 130/142 2023-11-25T04:21:52.215 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-test-2:18.0.0-7506.gec928368.el9.x86_64 124/142 2023-11-25T04:21:52.256 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : rbd-fuse-2:18.0.0-7506.gec928368.el9.x86_64 125/142 2023-11-25T04:21:52.297 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : rbd-nbd-2:18.0.0-7506.gec928368.el9.x86_64 126/142 2023-11-25T04:21:52.330 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : xfsprogs-devel-5.19.0-4.el9.x86_64 127/142 2023-11-25T04:21:52.423 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : elfutils-libelf-devel-0.190-1.el9.x86_64 128/142 2023-11-25T04:21:52.466 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-fuse-2:18.0.0-7506.gec928368.el9.x86_64 129/142 2023-11-25T04:21:52.498 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libcephfs-devel-2:18.0.0-7506.gec928368.el9.x86_ 130/142 2023-11-25T04:21:52.672 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : elfutils-debuginfod-client-0.190-1.el9.x86_64 131/142 2023-11-25T04:21:52.769 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 132/142 2023-11-25T04:21:52.811 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 133/142 2023-11-25T04:21:52.906 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : flex-2.6.4-9.el9.x86_64 134/142 2023-11-25T04:21:52.938 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : bison-3.7.4-5.el9.x86_64 135/142 2023-11-25T04:21:52.963 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 136/142 2023-11-25T04:21:52.989 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 137/142 2023-11-25T04:21:52.990 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 138/142 2023-11-25T04:21:53.069 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 138/142 2023-11-25T04:21:53.098 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 139/142 2023-11-25T04:21:53.099 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 140/142 2023-11-25T04:21:53.176 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 140/142 2023-11-25T04:21:53.200 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 141/142 2023-11-25T04:21:53.268 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 142/142 2023-11-25T04:21:53.319 INFO:teuthology.orchestra.run.smithi195.stdout: Upgrading : elfutils-debuginfod-client-0.190-1.el9.x86_64 131/142 2023-11-25T04:21:53.400 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 132/142 2023-11-25T04:21:53.442 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 133/142 2023-11-25T04:21:53.539 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : flex-2.6.4-9.el9.x86_64 134/142 2023-11-25T04:21:53.570 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : bison-3.7.4-5.el9.x86_64 135/142 2023-11-25T04:21:53.595 INFO:teuthology.orchestra.run.smithi195.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 136/142 2023-11-25T04:21:53.622 INFO:teuthology.orchestra.run.smithi195.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 137/142 2023-11-25T04:21:53.622 INFO:teuthology.orchestra.run.smithi195.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 138/142 2023-11-25T04:21:53.709 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 138/142 2023-11-25T04:21:53.739 INFO:teuthology.orchestra.run.smithi195.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 139/142 2023-11-25T04:21:53.739 INFO:teuthology.orchestra.run.smithi195.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 140/142 2023-11-25T04:21:53.825 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 140/142 2023-11-25T04:21:53.865 INFO:teuthology.orchestra.run.smithi195.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 141/142 2023-11-25T04:21:53.932 INFO:teuthology.orchestra.run.smithi195.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 142/142 2023-11-25T04:21:54.165 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 142/142 2023-11-25T04:21:54.165 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-2:18.0.0-7506.gec928368.el9.x86_64 1/142 2023-11-25T04:21:54.165 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-base-2:18.0.0-7506.gec928368.el9.x86_64 2/142 2023-11-25T04:21:54.165 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-common-2:18.0.0-7506.gec928368.el9.x86_64 3/142 2023-11-25T04:21:54.165 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-fuse-2:18.0.0-7506.gec928368.el9.x86_64 4/142 2023-11-25T04:21:54.165 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-7506.gec928 5/142 2023-11-25T04:21:54.165 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mds-2:18.0.0-7506.gec928368.el9.x86_64 6/142 2023-11-25T04:21:54.166 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-2:18.0.0-7506.gec928368.el9.x86_64 7/142 2023-11-25T04:21:54.166 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mon-2:18.0.0-7506.gec928368.el9.x86_64 8/142 2023-11-25T04:21:54.166 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-osd-2:18.0.0-7506.gec928368.el9.x86_64 9/142 2023-11-25T04:21:54.166 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-radosgw-2:18.0.0-7506.gec928368.el9.x86_64 10/142 2023-11-25T04:21:54.166 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-selinux-2:18.0.0-7506.gec928368.el9.x86_64 11/142 2023-11-25T04:21:54.166 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-test-2:18.0.0-7506.gec928368.el9.x86_64 12/142 2023-11-25T04:21:54.166 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libcephfs-devel-2:18.0.0-7506.gec928368.el9.x86_ 13/142 2023-11-25T04:21:54.166 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libcephfs2-2:18.0.0-7506.gec928368.el9.x86_64 14/142 2023-11-25T04:21:54.166 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libcephsqlite-2:18.0.0-7506.gec928368.el9.x86_64 15/142 2023-11-25T04:21:54.166 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librados-devel-2:18.0.0-7506.gec928368.el9.x86_6 16/142 2023-11-25T04:21:54.166 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libradosstriper1-2:18.0.0-7506.gec928368.el9.x86 17/142 2023-11-25T04:21:54.166 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librgw2-2:18.0.0-7506.gec928368.el9.x86_64 18/142 2023-11-25T04:21:54.166 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-ceph-argparse-2:18.0.0-7506.gec928368.el 19/142 2023-11-25T04:21:54.167 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-ceph-common-2:18.0.0-7506.gec928368.el9. 20/142 2023-11-25T04:21:54.167 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cephfs-2:18.0.0-7506.gec928368.el9.x86_6 21/142 2023-11-25T04:21:54.167 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-rados-2:18.0.0-7506.gec928368.el9.x86_64 22/142 2023-11-25T04:21:54.167 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-rbd-2:18.0.0-7506.gec928368.el9.x86_64 23/142 2023-11-25T04:21:54.167 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-rgw-2:18.0.0-7506.gec928368.el9.x86_64 24/142 2023-11-25T04:21:54.167 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : rbd-fuse-2:18.0.0-7506.gec928368.el9.x86_64 25/142 2023-11-25T04:21:54.167 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : rbd-mirror-2:18.0.0-7506.gec928368.el9.x86_64 26/142 2023-11-25T04:21:54.167 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : rbd-nbd-2:18.0.0-7506.gec928368.el9.x86_64 27/142 2023-11-25T04:21:54.181 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-7506.gec928368. 28/142 2023-11-25T04:21:54.181 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-7506.gec928368.el9.noa 29/142 2023-11-25T04:21:54.181 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-7506.gec928368.el9.n 30/142 2023-11-25T04:21:54.181 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-7506.gec9 31/142 2023-11-25T04:21:54.181 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-7506.gec928368.el 32/142 2023-11-25T04:21:54.182 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-rook-2:18.0.0-7506.gec928368.el9.noarch 33/142 2023-11-25T04:21:54.182 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-7506.gec928368.e 34/142 2023-11-25T04:21:54.182 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-volume-2:18.0.0-7506.gec928368.el9.noarch 35/142 2023-11-25T04:21:54.182 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : cephadm-2:18.0.0-7506.gec928368.el9.noarch 36/142 2023-11-25T04:21:54.182 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : automake-1.16.2-8.el9.noarch 37/142 2023-11-25T04:21:54.182 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : bison-3.7.4-5.el9.x86_64 38/142 2023-11-25T04:21:54.182 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 39/142 2023-11-25T04:21:54.182 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : elfutils-libelf-devel-0.190-1.el9.x86_64 40/142 2023-11-25T04:21:54.182 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : flex-2.6.4-9.el9.x86_64 41/142 2023-11-25T04:21:54.182 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 42/142 2023-11-25T04:21:54.183 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 43/142 2023-11-25T04:21:54.183 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 44/142 2023-11-25T04:21:54.183 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : jq-1.6-16.el9.x86_64 45/142 2023-11-25T04:21:54.183 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 46/142 2023-11-25T04:21:54.183 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 47/142 2023-11-25T04:21:54.183 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 48/142 2023-11-25T04:21:54.183 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 49/142 2023-11-25T04:21:54.183 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 50/142 2023-11-25T04:21:54.183 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 51/142 2023-11-25T04:21:54.184 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/142 2023-11-25T04:21:54.184 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/142 2023-11-25T04:21:54.184 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/142 2023-11-25T04:21:54.184 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/142 2023-11-25T04:21:54.184 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/142 2023-11-25T04:21:54.184 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/142 2023-11-25T04:21:54.184 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 58/142 2023-11-25T04:21:54.184 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 59/142 2023-11-25T04:21:54.184 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 60/142 2023-11-25T04:21:54.184 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 61/142 2023-11-25T04:21:54.184 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 62/142 2023-11-25T04:21:54.185 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 63/142 2023-11-25T04:21:54.185 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 64/142 2023-11-25T04:21:54.185 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 65/142 2023-11-25T04:21:54.185 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 66/142 2023-11-25T04:21:54.185 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 67/142 2023-11-25T04:21:54.185 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 68/142 2023-11-25T04:21:54.185 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 69/142 2023-11-25T04:21:54.185 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 70/142 2023-11-25T04:21:54.185 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 71/142 2023-11-25T04:21:54.185 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 72/142 2023-11-25T04:21:54.185 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 73/142 2023-11-25T04:21:54.185 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 74/142 2023-11-25T04:21:54.186 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : xfsprogs-devel-5.19.0-4.el9.x86_64 75/142 2023-11-25T04:21:54.186 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 76/142 2023-11-25T04:21:54.186 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 77/142 2023-11-25T04:21:54.186 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 78/142 2023-11-25T04:21:54.186 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libgfortran-11.4.1-2.3.el9.x86_64 79/142 2023-11-25T04:21:54.186 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libquadmath-11.4.1-2.3.el9.x86_64 80/142 2023-11-25T04:21:54.186 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 81/142 2023-11-25T04:21:54.186 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 82/142 2023-11-25T04:21:54.186 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-ply-3.11-14.el9.noarch 83/142 2023-11-25T04:21:54.186 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 84/142 2023-11-25T04:21:54.186 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 85/142 2023-11-25T04:21:54.187 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-urllib3-1.26.5-4.el9.noarch 86/142 2023-11-25T04:21:54.187 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 87/142 2023-11-25T04:21:54.187 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 88/142 2023-11-25T04:21:54.187 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 89/142 2023-11-25T04:21:54.187 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 90/142 2023-11-25T04:21:54.187 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 91/142 2023-11-25T04:21:54.187 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 92/142 2023-11-25T04:21:54.187 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 93/142 2023-11-25T04:21:54.187 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 94/142 2023-11-25T04:21:54.187 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 95/142 2023-11-25T04:21:54.187 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 96/142 2023-11-25T04:21:54.187 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 97/142 2023-11-25T04:21:54.188 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 98/142 2023-11-25T04:21:54.188 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 99/142 2023-11-25T04:21:54.188 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 100/142 2023-11-25T04:21:54.188 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 101/142 2023-11-25T04:21:54.188 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 102/142 2023-11-25T04:21:54.188 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-google-auth-1:2.23.4-1.el9.noarch 103/142 2023-11-25T04:21:54.188 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 104/142 2023-11-25T04:21:54.188 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 105/142 2023-11-25T04:21:54.188 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 106/142 2023-11-25T04:21:54.188 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 107/142 2023-11-25T04:21:54.189 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 108/142 2023-11-25T04:21:54.189 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 109/142 2023-11-25T04:21:54.189 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 110/142 2023-11-25T04:21:54.189 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 111/142 2023-11-25T04:21:54.189 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 112/142 2023-11-25T04:21:54.189 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 113/142 2023-11-25T04:21:54.189 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 114/142 2023-11-25T04:21:54.189 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 115/142 2023-11-25T04:21:54.189 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 116/142 2023-11-25T04:21:54.189 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/142 2023-11-25T04:21:54.189 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 118/142 2023-11-25T04:21:54.189 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 119/142 2023-11-25T04:21:54.189 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 120/142 2023-11-25T04:21:54.190 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 121/142 2023-11-25T04:21:54.190 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 122/142 2023-11-25T04:21:54.190 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 123/142 2023-11-25T04:21:54.190 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 124/142 2023-11-25T04:21:54.190 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 125/142 2023-11-25T04:21:54.190 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 126/142 2023-11-25T04:21:54.190 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : re2-1:20211101-3.el9.x86_64 127/142 2023-11-25T04:21:54.190 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 128/142 2023-11-25T04:21:54.190 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librados2-2:18.0.0-7506.gec928368.el9.x86_64 129/142 2023-11-25T04:21:54.190 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 130/142 2023-11-25T04:21:54.190 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librbd1-2:18.0.0-7506.gec928368.el9.x86_64 131/142 2023-11-25T04:21:54.190 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 132/142 2023-11-25T04:21:54.191 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : elfutils-debuginfod-client-0.190-1.el9.x86_64 133/142 2023-11-25T04:21:54.191 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 134/142 2023-11-25T04:21:54.191 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : elfutils-libelf-0.190-1.el9.x86_64 135/142 2023-11-25T04:21:54.191 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 136/142 2023-11-25T04:21:54.191 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : elfutils-libs-0.190-1.el9.x86_64 137/142 2023-11-25T04:21:54.191 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 138/142 2023-11-25T04:21:54.191 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 139/142 2023-11-25T04:21:54.191 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 140/142 2023-11-25T04:21:54.191 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 141/142 2023-11-25T04:21:55.432 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 142/142 2023-11-25T04:21:55.432 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-2:18.0.0-7506.gec928368.el9.x86_64 1/142 2023-11-25T04:21:55.432 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-base-2:18.0.0-7506.gec928368.el9.x86_64 2/142 2023-11-25T04:21:55.433 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-common-2:18.0.0-7506.gec928368.el9.x86_64 3/142 2023-11-25T04:21:55.433 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-fuse-2:18.0.0-7506.gec928368.el9.x86_64 4/142 2023-11-25T04:21:55.433 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-7506.gec928 5/142 2023-11-25T04:21:55.433 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mds-2:18.0.0-7506.gec928368.el9.x86_64 6/142 2023-11-25T04:21:55.433 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-2:18.0.0-7506.gec928368.el9.x86_64 7/142 2023-11-25T04:21:55.433 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mon-2:18.0.0-7506.gec928368.el9.x86_64 8/142 2023-11-25T04:21:55.433 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-osd-2:18.0.0-7506.gec928368.el9.x86_64 9/142 2023-11-25T04:21:55.433 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-radosgw-2:18.0.0-7506.gec928368.el9.x86_64 10/142 2023-11-25T04:21:55.433 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-selinux-2:18.0.0-7506.gec928368.el9.x86_64 11/142 2023-11-25T04:21:55.433 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-test-2:18.0.0-7506.gec928368.el9.x86_64 12/142 2023-11-25T04:21:55.433 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephfs-devel-2:18.0.0-7506.gec928368.el9.x86_ 13/142 2023-11-25T04:21:55.434 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephfs2-2:18.0.0-7506.gec928368.el9.x86_64 14/142 2023-11-25T04:21:55.434 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephsqlite-2:18.0.0-7506.gec928368.el9.x86_64 15/142 2023-11-25T04:21:55.434 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados-devel-2:18.0.0-7506.gec928368.el9.x86_6 16/142 2023-11-25T04:21:55.434 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libradosstriper1-2:18.0.0-7506.gec928368.el9.x86 17/142 2023-11-25T04:21:55.434 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librgw2-2:18.0.0-7506.gec928368.el9.x86_64 18/142 2023-11-25T04:21:55.434 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ceph-argparse-2:18.0.0-7506.gec928368.el 19/142 2023-11-25T04:21:55.434 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ceph-common-2:18.0.0-7506.gec928368.el9. 20/142 2023-11-25T04:21:55.434 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cephfs-2:18.0.0-7506.gec928368.el9.x86_6 21/142 2023-11-25T04:21:55.434 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rados-2:18.0.0-7506.gec928368.el9.x86_64 22/142 2023-11-25T04:21:55.434 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rbd-2:18.0.0-7506.gec928368.el9.x86_64 23/142 2023-11-25T04:21:55.434 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rgw-2:18.0.0-7506.gec928368.el9.x86_64 24/142 2023-11-25T04:21:55.434 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-fuse-2:18.0.0-7506.gec928368.el9.x86_64 25/142 2023-11-25T04:21:55.435 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-mirror-2:18.0.0-7506.gec928368.el9.x86_64 26/142 2023-11-25T04:21:55.435 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-nbd-2:18.0.0-7506.gec928368.el9.x86_64 27/142 2023-11-25T04:21:55.444 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-7506.gec928368. 28/142 2023-11-25T04:21:55.445 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-7506.gec928368.el9.noa 29/142 2023-11-25T04:21:55.445 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-7506.gec928368.el9.n 30/142 2023-11-25T04:21:55.445 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-7506.gec9 31/142 2023-11-25T04:21:55.445 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-7506.gec928368.el 32/142 2023-11-25T04:21:55.445 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-rook-2:18.0.0-7506.gec928368.el9.noarch 33/142 2023-11-25T04:21:55.445 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-7506.gec928368.e 34/142 2023-11-25T04:21:55.445 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-volume-2:18.0.0-7506.gec928368.el9.noarch 35/142 2023-11-25T04:21:55.445 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephadm-2:18.0.0-7506.gec928368.el9.noarch 36/142 2023-11-25T04:21:55.445 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : automake-1.16.2-8.el9.noarch 37/142 2023-11-25T04:21:55.445 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : bison-3.7.4-5.el9.x86_64 38/142 2023-11-25T04:21:55.445 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 39/142 2023-11-25T04:21:55.446 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : elfutils-libelf-devel-0.190-1.el9.x86_64 40/142 2023-11-25T04:21:55.446 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flex-2.6.4-9.el9.x86_64 41/142 2023-11-25T04:21:55.446 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 42/142 2023-11-25T04:21:55.446 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 43/142 2023-11-25T04:21:55.446 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 44/142 2023-11-25T04:21:55.446 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : jq-1.6-16.el9.x86_64 45/142 2023-11-25T04:21:55.446 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 46/142 2023-11-25T04:21:55.446 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 47/142 2023-11-25T04:21:55.446 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 48/142 2023-11-25T04:21:55.446 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 49/142 2023-11-25T04:21:55.446 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 50/142 2023-11-25T04:21:55.447 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 51/142 2023-11-25T04:21:55.447 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/142 2023-11-25T04:21:55.447 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/142 2023-11-25T04:21:55.447 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/142 2023-11-25T04:21:55.447 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/142 2023-11-25T04:21:55.447 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/142 2023-11-25T04:21:55.447 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/142 2023-11-25T04:21:55.447 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 58/142 2023-11-25T04:21:55.447 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 59/142 2023-11-25T04:21:55.447 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 60/142 2023-11-25T04:21:55.447 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 61/142 2023-11-25T04:21:55.448 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 62/142 2023-11-25T04:21:55.448 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 63/142 2023-11-25T04:21:55.448 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 64/142 2023-11-25T04:21:55.448 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 65/142 2023-11-25T04:21:55.448 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 66/142 2023-11-25T04:21:55.448 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 67/142 2023-11-25T04:21:55.448 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 68/142 2023-11-25T04:21:55.448 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 69/142 2023-11-25T04:21:55.448 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 70/142 2023-11-25T04:21:55.448 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 71/142 2023-11-25T04:21:55.448 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 72/142 2023-11-25T04:21:55.448 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 73/142 2023-11-25T04:21:55.449 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 74/142 2023-11-25T04:21:55.449 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : xfsprogs-devel-5.19.0-4.el9.x86_64 75/142 2023-11-25T04:21:55.449 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 76/142 2023-11-25T04:21:55.449 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 77/142 2023-11-25T04:21:55.449 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 78/142 2023-11-25T04:21:55.449 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libgfortran-11.4.1-2.3.el9.x86_64 79/142 2023-11-25T04:21:55.449 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libquadmath-11.4.1-2.3.el9.x86_64 80/142 2023-11-25T04:21:55.449 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 81/142 2023-11-25T04:21:55.449 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 82/142 2023-11-25T04:21:55.449 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ply-3.11-14.el9.noarch 83/142 2023-11-25T04:21:55.449 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 84/142 2023-11-25T04:21:55.450 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 85/142 2023-11-25T04:21:55.450 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-urllib3-1.26.5-4.el9.noarch 86/142 2023-11-25T04:21:55.450 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 87/142 2023-11-25T04:21:55.450 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 88/142 2023-11-25T04:21:55.450 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 89/142 2023-11-25T04:21:55.450 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 90/142 2023-11-25T04:21:55.450 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 91/142 2023-11-25T04:21:55.450 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 92/142 2023-11-25T04:21:55.450 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 93/142 2023-11-25T04:21:55.450 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 94/142 2023-11-25T04:21:55.450 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 95/142 2023-11-25T04:21:55.450 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 96/142 2023-11-25T04:21:55.451 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 97/142 2023-11-25T04:21:55.451 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 98/142 2023-11-25T04:21:55.451 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 99/142 2023-11-25T04:21:55.451 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 100/142 2023-11-25T04:21:55.451 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 101/142 2023-11-25T04:21:55.451 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 102/142 2023-11-25T04:21:55.451 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-google-auth-1:2.23.4-1.el9.noarch 103/142 2023-11-25T04:21:55.451 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 104/142 2023-11-25T04:21:55.451 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 105/142 2023-11-25T04:21:55.451 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 106/142 2023-11-25T04:21:55.451 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 107/142 2023-11-25T04:21:55.451 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 108/142 2023-11-25T04:21:55.451 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 109/142 2023-11-25T04:21:55.452 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 110/142 2023-11-25T04:21:55.452 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 111/142 2023-11-25T04:21:55.452 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 112/142 2023-11-25T04:21:55.452 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 113/142 2023-11-25T04:21:55.452 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 114/142 2023-11-25T04:21:55.452 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 115/142 2023-11-25T04:21:55.452 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 116/142 2023-11-25T04:21:55.452 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/142 2023-11-25T04:21:55.452 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 118/142 2023-11-25T04:21:55.452 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 119/142 2023-11-25T04:21:55.452 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 120/142 2023-11-25T04:21:55.452 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 121/142 2023-11-25T04:21:55.452 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 122/142 2023-11-25T04:21:55.453 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 123/142 2023-11-25T04:21:55.453 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 124/142 2023-11-25T04:21:55.453 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 125/142 2023-11-25T04:21:55.453 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 126/142 2023-11-25T04:21:55.453 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : re2-1:20211101-3.el9.x86_64 127/142 2023-11-25T04:21:55.453 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 128/142 2023-11-25T04:21:55.453 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados2-2:18.0.0-7506.gec928368.el9.x86_64 129/142 2023-11-25T04:21:55.453 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 130/142 2023-11-25T04:21:55.453 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librbd1-2:18.0.0-7506.gec928368.el9.x86_64 131/142 2023-11-25T04:21:55.453 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 132/142 2023-11-25T04:21:55.453 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : elfutils-debuginfod-client-0.190-1.el9.x86_64 133/142 2023-11-25T04:21:55.453 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 134/142 2023-11-25T04:21:55.454 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : elfutils-libelf-0.190-1.el9.x86_64 135/142 2023-11-25T04:21:55.454 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 136/142 2023-11-25T04:21:55.454 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : elfutils-libs-0.190-1.el9.x86_64 137/142 2023-11-25T04:21:55.454 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 138/142 2023-11-25T04:21:55.454 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 139/142 2023-11-25T04:21:55.454 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 140/142 2023-11-25T04:21:55.454 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 141/142 2023-11-25T04:21:56.538 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 142/142 2023-11-25T04:21:56.539 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-25T04:21:56.539 INFO:teuthology.orchestra.run.smithi076.stdout:Upgraded: 2023-11-25T04:21:56.539 INFO:teuthology.orchestra.run.smithi076.stdout: elfutils-debuginfod-client-0.190-1.el9.x86_64 2023-11-25T04:21:56.539 INFO:teuthology.orchestra.run.smithi076.stdout: elfutils-libelf-0.190-1.el9.x86_64 2023-11-25T04:21:56.539 INFO:teuthology.orchestra.run.smithi076.stdout: elfutils-libs-0.190-1.el9.x86_64 2023-11-25T04:21:56.539 INFO:teuthology.orchestra.run.smithi076.stdout: librados2-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:56.539 INFO:teuthology.orchestra.run.smithi076.stdout: librbd1-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:56.539 INFO:teuthology.orchestra.run.smithi076.stdout: openssl-1:3.0.7-25.el9.x86_64 2023-11-25T04:21:56.539 INFO:teuthology.orchestra.run.smithi076.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2023-11-25T04:21:56.539 INFO:teuthology.orchestra.run.smithi076.stdout:Installed: 2023-11-25T04:21:56.539 INFO:teuthology.orchestra.run.smithi076.stdout: automake-1.16.2-8.el9.noarch 2023-11-25T04:21:56.539 INFO:teuthology.orchestra.run.smithi076.stdout: bison-3.7.4-5.el9.x86_64 2023-11-25T04:21:56.540 INFO:teuthology.orchestra.run.smithi076.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-11-25T04:21:56.540 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:56.540 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-base-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:56.540 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-common-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:56.540 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-fuse-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:56.540 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-grafana-dashboards-2:18.0.0-7506.gec928368.el9.noarch 2023-11-25T04:21:56.540 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-immutable-object-cache-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:56.540 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mds-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:56.540 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:56.540 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-cephadm-2:18.0.0-7506.gec928368.el9.noarch 2023-11-25T04:21:56.540 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-dashboard-2:18.0.0-7506.gec928368.el9.noarch 2023-11-25T04:21:56.540 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-diskprediction-local-2:18.0.0-7506.gec928368.el9.noarch 2023-11-25T04:21:56.540 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-modules-core-2:18.0.0-7506.gec928368.el9.noarch 2023-11-25T04:21:56.541 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-rook-2:18.0.0-7506.gec928368.el9.noarch 2023-11-25T04:21:56.541 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mon-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:56.541 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-osd-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:56.541 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-prometheus-alerts-2:18.0.0-7506.gec928368.el9.noarch 2023-11-25T04:21:56.541 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-radosgw-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:56.541 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-selinux-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:56.541 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-test-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:56.541 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-volume-2:18.0.0-7506.gec928368.el9.noarch 2023-11-25T04:21:56.541 INFO:teuthology.orchestra.run.smithi076.stdout: cephadm-2:18.0.0-7506.gec928368.el9.noarch 2023-11-25T04:21:56.541 INFO:teuthology.orchestra.run.smithi076.stdout: elfutils-libelf-devel-0.190-1.el9.x86_64 2023-11-25T04:21:56.541 INFO:teuthology.orchestra.run.smithi076.stdout: flex-2.6.4-9.el9.x86_64 2023-11-25T04:21:56.541 INFO:teuthology.orchestra.run.smithi076.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-11-25T04:21:56.542 INFO:teuthology.orchestra.run.smithi076.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-11-25T04:21:56.542 INFO:teuthology.orchestra.run.smithi076.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-11-25T04:21:56.542 INFO:teuthology.orchestra.run.smithi076.stdout: fuse-2.9.9-15.el9.x86_64 2023-11-25T04:21:56.542 INFO:teuthology.orchestra.run.smithi076.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-11-25T04:21:56.542 INFO:teuthology.orchestra.run.smithi076.stdout: jq-1.6-16.el9.x86_64 2023-11-25T04:21:56.542 INFO:teuthology.orchestra.run.smithi076.stdout: libaio-devel-0.3.111-13.el9.x86_64 2023-11-25T04:21:56.542 INFO:teuthology.orchestra.run.smithi076.stdout: libarrow-9.0.0-9.el9.x86_64 2023-11-25T04:21:56.542 INFO:teuthology.orchestra.run.smithi076.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-11-25T04:21:56.542 INFO:teuthology.orchestra.run.smithi076.stdout: libcephfs-devel-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:56.542 INFO:teuthology.orchestra.run.smithi076.stdout: libcephfs2-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:56.546 INFO:teuthology.orchestra.run.smithi076.stdout: libcephsqlite-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:56.546 INFO:teuthology.orchestra.run.smithi076.stdout: libgfortran-11.4.1-2.3.el9.x86_64 2023-11-25T04:21:56.546 INFO:teuthology.orchestra.run.smithi076.stdout: liboath-2.6.7-2.el9.x86_64 2023-11-25T04:21:56.546 INFO:teuthology.orchestra.run.smithi076.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-11-25T04:21:56.546 INFO:teuthology.orchestra.run.smithi076.stdout: libquadmath-11.4.1-2.3.el9.x86_64 2023-11-25T04:21:56.547 INFO:teuthology.orchestra.run.smithi076.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-11-25T04:21:56.547 INFO:teuthology.orchestra.run.smithi076.stdout: librados-devel-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:56.547 INFO:teuthology.orchestra.run.smithi076.stdout: libradosstriper1-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:56.547 INFO:teuthology.orchestra.run.smithi076.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-11-25T04:21:56.547 INFO:teuthology.orchestra.run.smithi076.stdout: librgw2-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:56.547 INFO:teuthology.orchestra.run.smithi076.stdout: libtool-2.4.6-45.el9.x86_64 2023-11-25T04:21:56.547 INFO:teuthology.orchestra.run.smithi076.stdout: libunwind-1.6.2-1.el9.x86_64 2023-11-25T04:21:56.547 INFO:teuthology.orchestra.run.smithi076.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2023-11-25T04:21:56.547 INFO:teuthology.orchestra.run.smithi076.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2023-11-25T04:21:56.547 INFO:teuthology.orchestra.run.smithi076.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-11-25T04:21:56.547 INFO:teuthology.orchestra.run.smithi076.stdout: lua-5.4.4-4.el9.x86_64 2023-11-25T04:21:56.547 INFO:teuthology.orchestra.run.smithi076.stdout: lua-devel-5.4.4-4.el9.x86_64 2023-11-25T04:21:56.547 INFO:teuthology.orchestra.run.smithi076.stdout: luarocks-3.9.2-1.el9.noarch 2023-11-25T04:21:56.548 INFO:teuthology.orchestra.run.smithi076.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-11-25T04:21:56.548 INFO:teuthology.orchestra.run.smithi076.stdout: openblas-0.3.21-2.el9.x86_64 2023-11-25T04:21:56.548 INFO:teuthology.orchestra.run.smithi076.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-11-25T04:21:56.548 INFO:teuthology.orchestra.run.smithi076.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2023-11-25T04:21:56.548 INFO:teuthology.orchestra.run.smithi076.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-11-25T04:21:56.548 INFO:teuthology.orchestra.run.smithi076.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-11-25T04:21:56.548 INFO:teuthology.orchestra.run.smithi076.stdout: python3-babel-2.9.1-2.el9.noarch 2023-11-25T04:21:56.548 INFO:teuthology.orchestra.run.smithi076.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-11-25T04:21:56.548 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-11-25T04:21:56.548 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ceph-argparse-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:56.548 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ceph-common-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:56.548 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cephfs-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:56.548 INFO:teuthology.orchestra.run.smithi076.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-11-25T04:21:56.549 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-11-25T04:21:56.549 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-11-25T04:21:56.549 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-11-25T04:21:56.549 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-11-25T04:21:56.549 INFO:teuthology.orchestra.run.smithi076.stdout: python3-devel-3.9.17-2.el9.x86_64 2023-11-25T04:21:56.549 INFO:teuthology.orchestra.run.smithi076.stdout: python3-google-auth-1:2.23.4-1.el9.noarch 2023-11-25T04:21:56.549 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-11-25T04:21:56.549 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-11-25T04:21:56.549 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-11-25T04:21:56.549 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-11-25T04:21:56.549 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-11-25T04:21:56.549 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-11-25T04:21:56.549 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-11-25T04:21:56.550 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-11-25T04:21:56.550 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-11-25T04:21:56.550 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-11-25T04:21:56.550 INFO:teuthology.orchestra.run.smithi076.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-11-25T04:21:56.550 INFO:teuthology.orchestra.run.smithi076.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-11-25T04:21:56.550 INFO:teuthology.orchestra.run.smithi076.stdout: python3-mako-1.1.4-6.el9.noarch 2023-11-25T04:21:56.550 INFO:teuthology.orchestra.run.smithi076.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-11-25T04:21:56.550 INFO:teuthology.orchestra.run.smithi076.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-11-25T04:21:56.550 INFO:teuthology.orchestra.run.smithi076.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-11-25T04:21:56.550 INFO:teuthology.orchestra.run.smithi076.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-11-25T04:21:56.550 INFO:teuthology.orchestra.run.smithi076.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-11-25T04:21:56.550 INFO:teuthology.orchestra.run.smithi076.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-11-25T04:21:56.550 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-11-25T04:21:56.551 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ply-3.11-14.el9.noarch 2023-11-25T04:21:56.551 INFO:teuthology.orchestra.run.smithi076.stdout: python3-portend-3.1.0-2.el9.noarch 2023-11-25T04:21:56.551 INFO:teuthology.orchestra.run.smithi076.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-11-25T04:21:56.551 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-11-25T04:21:56.551 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-11-25T04:21:56.551 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-11-25T04:21:56.551 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pycparser-2.20-6.el9.noarch 2023-11-25T04:21:56.551 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pytz-2021.1-5.el9.noarch 2023-11-25T04:21:56.551 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rados-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:56.551 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rbd-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:56.551 INFO:teuthology.orchestra.run.smithi076.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-11-25T04:21:56.551 INFO:teuthology.orchestra.run.smithi076.stdout: python3-requests-2.25.1-7.el9.noarch 2023-11-25T04:21:56.551 INFO:teuthology.orchestra.run.smithi076.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-11-25T04:21:56.551 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rgw-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:56.552 INFO:teuthology.orchestra.run.smithi076.stdout: python3-routes-2.5.1-5.el9.noarch 2023-11-25T04:21:56.552 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rsa-4.9-2.el9.noarch 2023-11-25T04:21:56.552 INFO:teuthology.orchestra.run.smithi076.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-11-25T04:21:56.552 INFO:teuthology.orchestra.run.smithi076.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-11-25T04:21:56.552 INFO:teuthology.orchestra.run.smithi076.stdout: python3-toml-0.10.2-6.el9.noarch 2023-11-25T04:21:56.552 INFO:teuthology.orchestra.run.smithi076.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-11-25T04:21:56.552 INFO:teuthology.orchestra.run.smithi076.stdout: python3-urllib3-1.26.5-4.el9.noarch 2023-11-25T04:21:56.552 INFO:teuthology.orchestra.run.smithi076.stdout: python3-webob-1.8.7-6.el9.noarch 2023-11-25T04:21:56.552 INFO:teuthology.orchestra.run.smithi076.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-11-25T04:21:56.552 INFO:teuthology.orchestra.run.smithi076.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-11-25T04:21:56.552 INFO:teuthology.orchestra.run.smithi076.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-11-25T04:21:56.552 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-fuse-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:56.552 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-mirror-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:56.552 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-nbd-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:56.553 INFO:teuthology.orchestra.run.smithi076.stdout: re2-1:20211101-3.el9.x86_64 2023-11-25T04:21:56.553 INFO:teuthology.orchestra.run.smithi076.stdout: socat-1.7.4.1-5.el9.x86_64 2023-11-25T04:21:56.553 INFO:teuthology.orchestra.run.smithi076.stdout: thrift-0.15.0-2.el9.x86_64 2023-11-25T04:21:56.553 INFO:teuthology.orchestra.run.smithi076.stdout: xfsprogs-devel-5.19.0-4.el9.x86_64 2023-11-25T04:21:56.553 INFO:teuthology.orchestra.run.smithi076.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-11-25T04:21:56.553 INFO:teuthology.orchestra.run.smithi076.stdout: zlib-devel-1.2.11-40.el9.x86_64 2023-11-25T04:21:56.553 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-25T04:21:56.554 INFO:teuthology.orchestra.run.smithi076.stdout:Complete! 2023-11-25T04:21:56.966 DEBUG:teuthology.parallel:result is None 2023-11-25T04:21:57.208 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 142/142 2023-11-25T04:21:57.208 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-2:18.0.0-7506.gec928368.el9.x86_64 1/142 2023-11-25T04:21:57.208 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-base-2:18.0.0-7506.gec928368.el9.x86_64 2/142 2023-11-25T04:21:57.208 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-common-2:18.0.0-7506.gec928368.el9.x86_64 3/142 2023-11-25T04:21:57.208 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-fuse-2:18.0.0-7506.gec928368.el9.x86_64 4/142 2023-11-25T04:21:57.208 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-7506.gec928 5/142 2023-11-25T04:21:57.208 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mds-2:18.0.0-7506.gec928368.el9.x86_64 6/142 2023-11-25T04:21:57.209 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-2:18.0.0-7506.gec928368.el9.x86_64 7/142 2023-11-25T04:21:57.209 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mon-2:18.0.0-7506.gec928368.el9.x86_64 8/142 2023-11-25T04:21:57.209 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-osd-2:18.0.0-7506.gec928368.el9.x86_64 9/142 2023-11-25T04:21:57.209 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-radosgw-2:18.0.0-7506.gec928368.el9.x86_64 10/142 2023-11-25T04:21:57.209 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-selinux-2:18.0.0-7506.gec928368.el9.x86_64 11/142 2023-11-25T04:21:57.209 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-test-2:18.0.0-7506.gec928368.el9.x86_64 12/142 2023-11-25T04:21:57.209 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libcephfs-devel-2:18.0.0-7506.gec928368.el9.x86_ 13/142 2023-11-25T04:21:57.209 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libcephfs2-2:18.0.0-7506.gec928368.el9.x86_64 14/142 2023-11-25T04:21:57.209 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libcephsqlite-2:18.0.0-7506.gec928368.el9.x86_64 15/142 2023-11-25T04:21:57.209 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librados-devel-2:18.0.0-7506.gec928368.el9.x86_6 16/142 2023-11-25T04:21:57.209 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libradosstriper1-2:18.0.0-7506.gec928368.el9.x86 17/142 2023-11-25T04:21:57.209 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librgw2-2:18.0.0-7506.gec928368.el9.x86_64 18/142 2023-11-25T04:21:57.209 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-ceph-argparse-2:18.0.0-7506.gec928368.el 19/142 2023-11-25T04:21:57.210 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-ceph-common-2:18.0.0-7506.gec928368.el9. 20/142 2023-11-25T04:21:57.210 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cephfs-2:18.0.0-7506.gec928368.el9.x86_6 21/142 2023-11-25T04:21:57.210 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-rados-2:18.0.0-7506.gec928368.el9.x86_64 22/142 2023-11-25T04:21:57.210 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-rbd-2:18.0.0-7506.gec928368.el9.x86_64 23/142 2023-11-25T04:21:57.210 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-rgw-2:18.0.0-7506.gec928368.el9.x86_64 24/142 2023-11-25T04:21:57.210 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : rbd-fuse-2:18.0.0-7506.gec928368.el9.x86_64 25/142 2023-11-25T04:21:57.210 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : rbd-mirror-2:18.0.0-7506.gec928368.el9.x86_64 26/142 2023-11-25T04:21:57.210 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : rbd-nbd-2:18.0.0-7506.gec928368.el9.x86_64 27/142 2023-11-25T04:21:57.210 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-7506.gec928368. 28/142 2023-11-25T04:21:57.210 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-7506.gec928368.el9.noa 29/142 2023-11-25T04:21:57.210 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-7506.gec928368.el9.n 30/142 2023-11-25T04:21:57.210 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-7506.gec9 31/142 2023-11-25T04:21:57.220 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-7506.gec928368.el 32/142 2023-11-25T04:21:57.220 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-rook-2:18.0.0-7506.gec928368.el9.noarch 33/142 2023-11-25T04:21:57.220 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-7506.gec928368.e 34/142 2023-11-25T04:21:57.220 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-volume-2:18.0.0-7506.gec928368.el9.noarch 35/142 2023-11-25T04:21:57.220 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : cephadm-2:18.0.0-7506.gec928368.el9.noarch 36/142 2023-11-25T04:21:57.220 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : automake-1.16.2-8.el9.noarch 37/142 2023-11-25T04:21:57.220 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : bison-3.7.4-5.el9.x86_64 38/142 2023-11-25T04:21:57.220 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 39/142 2023-11-25T04:21:57.220 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : elfutils-libelf-devel-0.190-1.el9.x86_64 40/142 2023-11-25T04:21:57.220 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : flex-2.6.4-9.el9.x86_64 41/142 2023-11-25T04:21:57.220 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 42/142 2023-11-25T04:21:57.220 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 43/142 2023-11-25T04:21:57.221 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 44/142 2023-11-25T04:21:57.221 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : jq-1.6-16.el9.x86_64 45/142 2023-11-25T04:21:57.221 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 46/142 2023-11-25T04:21:57.221 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 47/142 2023-11-25T04:21:57.221 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 48/142 2023-11-25T04:21:57.221 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 49/142 2023-11-25T04:21:57.221 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 50/142 2023-11-25T04:21:57.221 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 51/142 2023-11-25T04:21:57.221 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/142 2023-11-25T04:21:57.221 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/142 2023-11-25T04:21:57.221 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/142 2023-11-25T04:21:57.221 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/142 2023-11-25T04:21:57.221 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/142 2023-11-25T04:21:57.222 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/142 2023-11-25T04:21:57.222 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 58/142 2023-11-25T04:21:57.222 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 59/142 2023-11-25T04:21:57.222 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 60/142 2023-11-25T04:21:57.222 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 61/142 2023-11-25T04:21:57.222 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 62/142 2023-11-25T04:21:57.222 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 63/142 2023-11-25T04:21:57.222 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 64/142 2023-11-25T04:21:57.222 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 65/142 2023-11-25T04:21:57.222 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 66/142 2023-11-25T04:21:57.222 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 67/142 2023-11-25T04:21:57.222 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 68/142 2023-11-25T04:21:57.222 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 69/142 2023-11-25T04:21:57.222 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 70/142 2023-11-25T04:21:57.223 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 71/142 2023-11-25T04:21:57.223 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 72/142 2023-11-25T04:21:57.223 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 73/142 2023-11-25T04:21:57.223 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 74/142 2023-11-25T04:21:57.223 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : xfsprogs-devel-5.19.0-4.el9.x86_64 75/142 2023-11-25T04:21:57.223 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 76/142 2023-11-25T04:21:57.223 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 77/142 2023-11-25T04:21:57.223 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 78/142 2023-11-25T04:21:57.223 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libgfortran-11.4.1-2.3.el9.x86_64 79/142 2023-11-25T04:21:57.223 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libquadmath-11.4.1-2.3.el9.x86_64 80/142 2023-11-25T04:21:57.223 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 81/142 2023-11-25T04:21:57.223 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 82/142 2023-11-25T04:21:57.223 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-ply-3.11-14.el9.noarch 83/142 2023-11-25T04:21:57.224 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 84/142 2023-11-25T04:21:57.224 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 85/142 2023-11-25T04:21:57.224 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-urllib3-1.26.5-4.el9.noarch 86/142 2023-11-25T04:21:57.224 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 87/142 2023-11-25T04:21:57.224 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 88/142 2023-11-25T04:21:57.224 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 89/142 2023-11-25T04:21:57.224 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 90/142 2023-11-25T04:21:57.224 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 91/142 2023-11-25T04:21:57.224 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 92/142 2023-11-25T04:21:57.224 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 93/142 2023-11-25T04:21:57.224 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 94/142 2023-11-25T04:21:57.224 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 95/142 2023-11-25T04:21:57.224 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 96/142 2023-11-25T04:21:57.225 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 97/142 2023-11-25T04:21:57.225 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 98/142 2023-11-25T04:21:57.225 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 99/142 2023-11-25T04:21:57.225 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 100/142 2023-11-25T04:21:57.225 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 101/142 2023-11-25T04:21:57.225 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 102/142 2023-11-25T04:21:57.225 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-google-auth-1:2.23.4-1.el9.noarch 103/142 2023-11-25T04:21:57.225 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 104/142 2023-11-25T04:21:57.225 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 105/142 2023-11-25T04:21:57.225 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 106/142 2023-11-25T04:21:57.225 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 107/142 2023-11-25T04:21:57.225 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 108/142 2023-11-25T04:21:57.226 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 109/142 2023-11-25T04:21:57.226 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 110/142 2023-11-25T04:21:57.226 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 111/142 2023-11-25T04:21:57.226 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 112/142 2023-11-25T04:21:57.226 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 113/142 2023-11-25T04:21:57.226 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 114/142 2023-11-25T04:21:57.226 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 115/142 2023-11-25T04:21:57.226 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 116/142 2023-11-25T04:21:57.226 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/142 2023-11-25T04:21:57.226 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 118/142 2023-11-25T04:21:57.226 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 119/142 2023-11-25T04:21:57.226 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 120/142 2023-11-25T04:21:57.226 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 121/142 2023-11-25T04:21:57.227 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 122/142 2023-11-25T04:21:57.227 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 123/142 2023-11-25T04:21:57.227 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 124/142 2023-11-25T04:21:57.227 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 125/142 2023-11-25T04:21:57.227 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 126/142 2023-11-25T04:21:57.227 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : re2-1:20211101-3.el9.x86_64 127/142 2023-11-25T04:21:57.227 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 128/142 2023-11-25T04:21:57.227 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librados2-2:18.0.0-7506.gec928368.el9.x86_64 129/142 2023-11-25T04:21:57.227 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 130/142 2023-11-25T04:21:57.227 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librbd1-2:18.0.0-7506.gec928368.el9.x86_64 131/142 2023-11-25T04:21:57.227 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 132/142 2023-11-25T04:21:57.227 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : elfutils-debuginfod-client-0.190-1.el9.x86_64 133/142 2023-11-25T04:21:57.227 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 134/142 2023-11-25T04:21:57.228 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : elfutils-libelf-0.190-1.el9.x86_64 135/142 2023-11-25T04:21:57.228 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 136/142 2023-11-25T04:21:57.228 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : elfutils-libs-0.190-1.el9.x86_64 137/142 2023-11-25T04:21:57.228 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 138/142 2023-11-25T04:21:57.228 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 139/142 2023-11-25T04:21:57.228 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 140/142 2023-11-25T04:21:57.228 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 141/142 2023-11-25T04:21:57.999 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 142/142 2023-11-25T04:21:57.999 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-25T04:21:57.999 INFO:teuthology.orchestra.run.smithi092.stdout:Upgraded: 2023-11-25T04:21:58.000 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-debuginfod-client-0.190-1.el9.x86_64 2023-11-25T04:21:58.000 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-libelf-0.190-1.el9.x86_64 2023-11-25T04:21:58.000 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-libs-0.190-1.el9.x86_64 2023-11-25T04:21:58.000 INFO:teuthology.orchestra.run.smithi092.stdout: librados2-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:58.000 INFO:teuthology.orchestra.run.smithi092.stdout: librbd1-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:58.000 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-1:3.0.7-25.el9.x86_64 2023-11-25T04:21:58.001 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2023-11-25T04:21:58.001 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2023-11-25T04:21:58.001 INFO:teuthology.orchestra.run.smithi092.stdout: automake-1.16.2-8.el9.noarch 2023-11-25T04:21:58.001 INFO:teuthology.orchestra.run.smithi092.stdout: bison-3.7.4-5.el9.x86_64 2023-11-25T04:21:58.001 INFO:teuthology.orchestra.run.smithi092.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-11-25T04:21:58.002 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:58.002 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-base-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:58.002 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-common-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:58.002 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-fuse-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:58.002 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-grafana-dashboards-2:18.0.0-7506.gec928368.el9.noarch 2023-11-25T04:21:58.002 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-immutable-object-cache-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:58.003 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mds-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:58.003 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:58.003 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-cephadm-2:18.0.0-7506.gec928368.el9.noarch 2023-11-25T04:21:58.003 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-dashboard-2:18.0.0-7506.gec928368.el9.noarch 2023-11-25T04:21:58.003 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-diskprediction-local-2:18.0.0-7506.gec928368.el9.noarch 2023-11-25T04:21:58.003 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-modules-core-2:18.0.0-7506.gec928368.el9.noarch 2023-11-25T04:21:58.004 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-rook-2:18.0.0-7506.gec928368.el9.noarch 2023-11-25T04:21:58.004 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mon-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:58.004 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-osd-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:58.004 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-prometheus-alerts-2:18.0.0-7506.gec928368.el9.noarch 2023-11-25T04:21:58.004 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-radosgw-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:58.005 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-selinux-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:58.005 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-test-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:58.005 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-volume-2:18.0.0-7506.gec928368.el9.noarch 2023-11-25T04:21:58.005 INFO:teuthology.orchestra.run.smithi092.stdout: cephadm-2:18.0.0-7506.gec928368.el9.noarch 2023-11-25T04:21:58.005 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-libelf-devel-0.190-1.el9.x86_64 2023-11-25T04:21:58.005 INFO:teuthology.orchestra.run.smithi092.stdout: flex-2.6.4-9.el9.x86_64 2023-11-25T04:21:58.006 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-11-25T04:21:58.006 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-11-25T04:21:58.006 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-11-25T04:21:58.006 INFO:teuthology.orchestra.run.smithi092.stdout: fuse-2.9.9-15.el9.x86_64 2023-11-25T04:21:58.006 INFO:teuthology.orchestra.run.smithi092.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-11-25T04:21:58.006 INFO:teuthology.orchestra.run.smithi092.stdout: jq-1.6-16.el9.x86_64 2023-11-25T04:21:58.007 INFO:teuthology.orchestra.run.smithi092.stdout: libaio-devel-0.3.111-13.el9.x86_64 2023-11-25T04:21:58.007 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow-9.0.0-9.el9.x86_64 2023-11-25T04:21:58.007 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-11-25T04:21:58.007 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs-devel-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:58.007 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs2-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:58.008 INFO:teuthology.orchestra.run.smithi092.stdout: libcephsqlite-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:58.008 INFO:teuthology.orchestra.run.smithi092.stdout: libgfortran-11.4.1-2.3.el9.x86_64 2023-11-25T04:21:58.008 INFO:teuthology.orchestra.run.smithi092.stdout: liboath-2.6.7-2.el9.x86_64 2023-11-25T04:21:58.008 INFO:teuthology.orchestra.run.smithi092.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-11-25T04:21:58.008 INFO:teuthology.orchestra.run.smithi092.stdout: libquadmath-11.4.1-2.3.el9.x86_64 2023-11-25T04:21:58.008 INFO:teuthology.orchestra.run.smithi092.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-11-25T04:21:58.009 INFO:teuthology.orchestra.run.smithi092.stdout: librados-devel-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:58.009 INFO:teuthology.orchestra.run.smithi092.stdout: libradosstriper1-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:58.009 INFO:teuthology.orchestra.run.smithi092.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-11-25T04:21:58.009 INFO:teuthology.orchestra.run.smithi092.stdout: librgw2-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:58.009 INFO:teuthology.orchestra.run.smithi092.stdout: libtool-2.4.6-45.el9.x86_64 2023-11-25T04:21:58.009 INFO:teuthology.orchestra.run.smithi092.stdout: libunwind-1.6.2-1.el9.x86_64 2023-11-25T04:21:58.010 INFO:teuthology.orchestra.run.smithi092.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2023-11-25T04:21:58.010 INFO:teuthology.orchestra.run.smithi092.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2023-11-25T04:21:58.010 INFO:teuthology.orchestra.run.smithi092.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-11-25T04:21:58.010 INFO:teuthology.orchestra.run.smithi092.stdout: lua-5.4.4-4.el9.x86_64 2023-11-25T04:21:58.010 INFO:teuthology.orchestra.run.smithi092.stdout: lua-devel-5.4.4-4.el9.x86_64 2023-11-25T04:21:58.010 INFO:teuthology.orchestra.run.smithi092.stdout: luarocks-3.9.2-1.el9.noarch 2023-11-25T04:21:58.011 INFO:teuthology.orchestra.run.smithi092.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-11-25T04:21:58.011 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-0.3.21-2.el9.x86_64 2023-11-25T04:21:58.011 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-11-25T04:21:58.011 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2023-11-25T04:21:58.011 INFO:teuthology.orchestra.run.smithi092.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-11-25T04:21:58.011 INFO:teuthology.orchestra.run.smithi092.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-11-25T04:21:58.012 INFO:teuthology.orchestra.run.smithi092.stdout: python3-babel-2.9.1-2.el9.noarch 2023-11-25T04:21:58.012 INFO:teuthology.orchestra.run.smithi092.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-11-25T04:21:58.012 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-11-25T04:21:58.012 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-argparse-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:58.012 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-common-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:58.012 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cephfs-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:58.013 INFO:teuthology.orchestra.run.smithi092.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-11-25T04:21:58.013 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-11-25T04:21:58.013 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-11-25T04:21:58.013 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-11-25T04:21:58.013 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-11-25T04:21:58.013 INFO:teuthology.orchestra.run.smithi092.stdout: python3-devel-3.9.17-2.el9.x86_64 2023-11-25T04:21:58.014 INFO:teuthology.orchestra.run.smithi092.stdout: python3-google-auth-1:2.23.4-1.el9.noarch 2023-11-25T04:21:58.014 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-11-25T04:21:58.014 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-11-25T04:21:58.014 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-11-25T04:21:58.014 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-11-25T04:21:58.015 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-11-25T04:21:58.015 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-11-25T04:21:58.015 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-11-25T04:21:58.015 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-11-25T04:21:58.015 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-11-25T04:21:58.015 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-11-25T04:21:58.016 INFO:teuthology.orchestra.run.smithi092.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-11-25T04:21:58.016 INFO:teuthology.orchestra.run.smithi092.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-11-25T04:21:58.016 INFO:teuthology.orchestra.run.smithi092.stdout: python3-mako-1.1.4-6.el9.noarch 2023-11-25T04:21:58.016 INFO:teuthology.orchestra.run.smithi092.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-11-25T04:21:58.016 INFO:teuthology.orchestra.run.smithi092.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-11-25T04:21:58.017 INFO:teuthology.orchestra.run.smithi092.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-11-25T04:21:58.017 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-11-25T04:21:58.017 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-11-25T04:21:58.017 INFO:teuthology.orchestra.run.smithi092.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-11-25T04:21:58.017 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-11-25T04:21:58.017 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ply-3.11-14.el9.noarch 2023-11-25T04:21:58.018 INFO:teuthology.orchestra.run.smithi092.stdout: python3-portend-3.1.0-2.el9.noarch 2023-11-25T04:21:58.018 INFO:teuthology.orchestra.run.smithi092.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-11-25T04:21:58.018 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-11-25T04:21:58.018 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-11-25T04:21:58.018 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-11-25T04:21:58.018 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pycparser-2.20-6.el9.noarch 2023-11-25T04:21:58.019 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pytz-2021.1-5.el9.noarch 2023-11-25T04:21:58.019 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rados-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:58.019 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rbd-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:58.019 INFO:teuthology.orchestra.run.smithi092.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-11-25T04:21:58.019 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-2.25.1-7.el9.noarch 2023-11-25T04:21:58.020 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-11-25T04:21:58.020 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rgw-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:58.020 INFO:teuthology.orchestra.run.smithi092.stdout: python3-routes-2.5.1-5.el9.noarch 2023-11-25T04:21:58.020 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rsa-4.9-2.el9.noarch 2023-11-25T04:21:58.020 INFO:teuthology.orchestra.run.smithi092.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-11-25T04:21:58.020 INFO:teuthology.orchestra.run.smithi092.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-11-25T04:21:58.021 INFO:teuthology.orchestra.run.smithi092.stdout: python3-toml-0.10.2-6.el9.noarch 2023-11-25T04:21:58.021 INFO:teuthology.orchestra.run.smithi092.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-11-25T04:21:58.021 INFO:teuthology.orchestra.run.smithi092.stdout: python3-urllib3-1.26.5-4.el9.noarch 2023-11-25T04:21:58.021 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webob-1.8.7-6.el9.noarch 2023-11-25T04:21:58.021 INFO:teuthology.orchestra.run.smithi092.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-11-25T04:21:58.022 INFO:teuthology.orchestra.run.smithi092.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-11-25T04:21:58.022 INFO:teuthology.orchestra.run.smithi092.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-11-25T04:21:58.022 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-fuse-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:58.022 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-mirror-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:58.022 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-nbd-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:58.022 INFO:teuthology.orchestra.run.smithi092.stdout: re2-1:20211101-3.el9.x86_64 2023-11-25T04:21:58.023 INFO:teuthology.orchestra.run.smithi092.stdout: socat-1.7.4.1-5.el9.x86_64 2023-11-25T04:21:58.023 INFO:teuthology.orchestra.run.smithi092.stdout: thrift-0.15.0-2.el9.x86_64 2023-11-25T04:21:58.023 INFO:teuthology.orchestra.run.smithi092.stdout: xfsprogs-devel-5.19.0-4.el9.x86_64 2023-11-25T04:21:58.023 INFO:teuthology.orchestra.run.smithi092.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-11-25T04:21:58.024 INFO:teuthology.orchestra.run.smithi092.stdout: zlib-devel-1.2.11-40.el9.x86_64 2023-11-25T04:21:58.024 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-25T04:21:58.024 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2023-11-25T04:21:58.272 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 142/142 2023-11-25T04:21:58.272 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-2:18.0.0-7506.gec928368.el9.x86_64 1/142 2023-11-25T04:21:58.272 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-base-2:18.0.0-7506.gec928368.el9.x86_64 2/142 2023-11-25T04:21:58.272 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-common-2:18.0.0-7506.gec928368.el9.x86_64 3/142 2023-11-25T04:21:58.273 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-fuse-2:18.0.0-7506.gec928368.el9.x86_64 4/142 2023-11-25T04:21:58.273 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-7506.gec928 5/142 2023-11-25T04:21:58.273 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mds-2:18.0.0-7506.gec928368.el9.x86_64 6/142 2023-11-25T04:21:58.273 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mgr-2:18.0.0-7506.gec928368.el9.x86_64 7/142 2023-11-25T04:21:58.273 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mon-2:18.0.0-7506.gec928368.el9.x86_64 8/142 2023-11-25T04:21:58.273 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-osd-2:18.0.0-7506.gec928368.el9.x86_64 9/142 2023-11-25T04:21:58.273 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-radosgw-2:18.0.0-7506.gec928368.el9.x86_64 10/142 2023-11-25T04:21:58.273 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-selinux-2:18.0.0-7506.gec928368.el9.x86_64 11/142 2023-11-25T04:21:58.273 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-test-2:18.0.0-7506.gec928368.el9.x86_64 12/142 2023-11-25T04:21:58.273 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libcephfs-devel-2:18.0.0-7506.gec928368.el9.x86_ 13/142 2023-11-25T04:21:58.274 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libcephfs2-2:18.0.0-7506.gec928368.el9.x86_64 14/142 2023-11-25T04:21:58.274 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libcephsqlite-2:18.0.0-7506.gec928368.el9.x86_64 15/142 2023-11-25T04:21:58.274 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librados-devel-2:18.0.0-7506.gec928368.el9.x86_6 16/142 2023-11-25T04:21:58.274 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libradosstriper1-2:18.0.0-7506.gec928368.el9.x86 17/142 2023-11-25T04:21:58.274 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librgw2-2:18.0.0-7506.gec928368.el9.x86_64 18/142 2023-11-25T04:21:58.274 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-ceph-argparse-2:18.0.0-7506.gec928368.el 19/142 2023-11-25T04:21:58.274 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-ceph-common-2:18.0.0-7506.gec928368.el9. 20/142 2023-11-25T04:21:58.274 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-cephfs-2:18.0.0-7506.gec928368.el9.x86_6 21/142 2023-11-25T04:21:58.274 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-rados-2:18.0.0-7506.gec928368.el9.x86_64 22/142 2023-11-25T04:21:58.274 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-rbd-2:18.0.0-7506.gec928368.el9.x86_64 23/142 2023-11-25T04:21:58.275 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-rgw-2:18.0.0-7506.gec928368.el9.x86_64 24/142 2023-11-25T04:21:58.275 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : rbd-fuse-2:18.0.0-7506.gec928368.el9.x86_64 25/142 2023-11-25T04:21:58.275 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : rbd-mirror-2:18.0.0-7506.gec928368.el9.x86_64 26/142 2023-11-25T04:21:58.275 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : rbd-nbd-2:18.0.0-7506.gec928368.el9.x86_64 27/142 2023-11-25T04:21:58.275 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-7506.gec928368. 28/142 2023-11-25T04:21:58.275 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-7506.gec928368.el9.noa 29/142 2023-11-25T04:21:58.275 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-7506.gec928368.el9.n 30/142 2023-11-25T04:21:58.275 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-7506.gec9 31/142 2023-11-25T04:21:58.276 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-7506.gec928368.el 32/142 2023-11-25T04:21:58.276 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mgr-rook-2:18.0.0-7506.gec928368.el9.noarch 33/142 2023-11-25T04:21:58.276 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-7506.gec928368.e 34/142 2023-11-25T04:21:58.276 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-volume-2:18.0.0-7506.gec928368.el9.noarch 35/142 2023-11-25T04:21:58.276 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : cephadm-2:18.0.0-7506.gec928368.el9.noarch 36/142 2023-11-25T04:21:58.276 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : automake-1.16.2-8.el9.noarch 37/142 2023-11-25T04:21:58.276 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : bison-3.7.4-5.el9.x86_64 38/142 2023-11-25T04:21:58.276 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 39/142 2023-11-25T04:21:58.277 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : elfutils-libelf-devel-0.190-1.el9.x86_64 40/142 2023-11-25T04:21:58.277 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : flex-2.6.4-9.el9.x86_64 41/142 2023-11-25T04:21:58.277 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 42/142 2023-11-25T04:21:58.277 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 43/142 2023-11-25T04:21:58.277 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 44/142 2023-11-25T04:21:58.277 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : jq-1.6-16.el9.x86_64 45/142 2023-11-25T04:21:58.277 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 46/142 2023-11-25T04:21:58.277 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 47/142 2023-11-25T04:21:58.277 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 48/142 2023-11-25T04:21:58.278 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 49/142 2023-11-25T04:21:58.278 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 50/142 2023-11-25T04:21:58.278 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 51/142 2023-11-25T04:21:58.278 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/142 2023-11-25T04:21:58.278 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/142 2023-11-25T04:21:58.278 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/142 2023-11-25T04:21:58.278 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/142 2023-11-25T04:21:58.278 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/142 2023-11-25T04:21:58.278 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/142 2023-11-25T04:21:58.279 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 58/142 2023-11-25T04:21:58.279 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 59/142 2023-11-25T04:21:58.279 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 60/142 2023-11-25T04:21:58.279 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 61/142 2023-11-25T04:21:58.279 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 62/142 2023-11-25T04:21:58.279 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 63/142 2023-11-25T04:21:58.279 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 64/142 2023-11-25T04:21:58.279 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 65/142 2023-11-25T04:21:58.279 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 66/142 2023-11-25T04:21:58.279 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 67/142 2023-11-25T04:21:58.280 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 68/142 2023-11-25T04:21:58.280 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 69/142 2023-11-25T04:21:58.280 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 70/142 2023-11-25T04:21:58.280 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 71/142 2023-11-25T04:21:58.280 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 72/142 2023-11-25T04:21:58.280 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 73/142 2023-11-25T04:21:58.280 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 74/142 2023-11-25T04:21:58.280 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : xfsprogs-devel-5.19.0-4.el9.x86_64 75/142 2023-11-25T04:21:58.281 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 76/142 2023-11-25T04:21:58.281 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 77/142 2023-11-25T04:21:58.281 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 78/142 2023-11-25T04:21:58.281 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libgfortran-11.4.1-2.3.el9.x86_64 79/142 2023-11-25T04:21:58.281 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libquadmath-11.4.1-2.3.el9.x86_64 80/142 2023-11-25T04:21:58.281 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 81/142 2023-11-25T04:21:58.281 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 82/142 2023-11-25T04:21:58.281 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-ply-3.11-14.el9.noarch 83/142 2023-11-25T04:21:58.281 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 84/142 2023-11-25T04:21:58.281 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 85/142 2023-11-25T04:21:58.282 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-urllib3-1.26.5-4.el9.noarch 86/142 2023-11-25T04:21:58.282 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 87/142 2023-11-25T04:21:58.282 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 88/142 2023-11-25T04:21:58.282 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 89/142 2023-11-25T04:21:58.282 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 90/142 2023-11-25T04:21:58.282 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 91/142 2023-11-25T04:21:58.282 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 92/142 2023-11-25T04:21:58.282 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 93/142 2023-11-25T04:21:58.282 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 94/142 2023-11-25T04:21:58.283 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 95/142 2023-11-25T04:21:58.283 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 96/142 2023-11-25T04:21:58.283 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 97/142 2023-11-25T04:21:58.283 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 98/142 2023-11-25T04:21:58.283 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 99/142 2023-11-25T04:21:58.283 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 100/142 2023-11-25T04:21:58.283 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 101/142 2023-11-25T04:21:58.283 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 102/142 2023-11-25T04:21:58.283 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-google-auth-1:2.23.4-1.el9.noarch 103/142 2023-11-25T04:21:58.284 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 104/142 2023-11-25T04:21:58.284 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 105/142 2023-11-25T04:21:58.284 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 106/142 2023-11-25T04:21:58.284 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 107/142 2023-11-25T04:21:58.284 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 108/142 2023-11-25T04:21:58.284 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 109/142 2023-11-25T04:21:58.284 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 110/142 2023-11-25T04:21:58.284 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 111/142 2023-11-25T04:21:58.284 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 112/142 2023-11-25T04:21:58.285 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 113/142 2023-11-25T04:21:58.285 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 114/142 2023-11-25T04:21:58.285 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 115/142 2023-11-25T04:21:58.285 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 116/142 2023-11-25T04:21:58.285 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/142 2023-11-25T04:21:58.285 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 118/142 2023-11-25T04:21:58.285 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 119/142 2023-11-25T04:21:58.285 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 120/142 2023-11-25T04:21:58.285 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 121/142 2023-11-25T04:21:58.285 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 122/142 2023-11-25T04:21:58.285 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 123/142 2023-11-25T04:21:58.286 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 124/142 2023-11-25T04:21:58.286 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 125/142 2023-11-25T04:21:58.286 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 126/142 2023-11-25T04:21:58.286 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : re2-1:20211101-3.el9.x86_64 127/142 2023-11-25T04:21:58.286 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 128/142 2023-11-25T04:21:58.286 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librados2-2:18.0.0-7506.gec928368.el9.x86_64 129/142 2023-11-25T04:21:58.286 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 130/142 2023-11-25T04:21:58.286 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librbd1-2:18.0.0-7506.gec928368.el9.x86_64 131/142 2023-11-25T04:21:58.286 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 132/142 2023-11-25T04:21:58.286 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : elfutils-debuginfod-client-0.190-1.el9.x86_64 133/142 2023-11-25T04:21:58.287 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 134/142 2023-11-25T04:21:58.287 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : elfutils-libelf-0.190-1.el9.x86_64 135/142 2023-11-25T04:21:58.287 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 136/142 2023-11-25T04:21:58.287 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : elfutils-libs-0.190-1.el9.x86_64 137/142 2023-11-25T04:21:58.287 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 138/142 2023-11-25T04:21:58.287 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 139/142 2023-11-25T04:21:58.287 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 140/142 2023-11-25T04:21:58.287 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 141/142 2023-11-25T04:21:58.393 DEBUG:teuthology.parallel:result is None 2023-11-25T04:21:59.531 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 142/142 2023-11-25T04:21:59.531 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:21:59.531 INFO:teuthology.orchestra.run.smithi067.stdout:Upgraded: 2023-11-25T04:21:59.531 INFO:teuthology.orchestra.run.smithi067.stdout: elfutils-debuginfod-client-0.190-1.el9.x86_64 2023-11-25T04:21:59.531 INFO:teuthology.orchestra.run.smithi067.stdout: elfutils-libelf-0.190-1.el9.x86_64 2023-11-25T04:21:59.532 INFO:teuthology.orchestra.run.smithi067.stdout: elfutils-libs-0.190-1.el9.x86_64 2023-11-25T04:21:59.532 INFO:teuthology.orchestra.run.smithi067.stdout: librados2-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:59.532 INFO:teuthology.orchestra.run.smithi067.stdout: librbd1-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:59.532 INFO:teuthology.orchestra.run.smithi067.stdout: openssl-1:3.0.7-25.el9.x86_64 2023-11-25T04:21:59.532 INFO:teuthology.orchestra.run.smithi067.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2023-11-25T04:21:59.532 INFO:teuthology.orchestra.run.smithi067.stdout:Installed: 2023-11-25T04:21:59.533 INFO:teuthology.orchestra.run.smithi067.stdout: automake-1.16.2-8.el9.noarch 2023-11-25T04:21:59.533 INFO:teuthology.orchestra.run.smithi067.stdout: bison-3.7.4-5.el9.x86_64 2023-11-25T04:21:59.533 INFO:teuthology.orchestra.run.smithi067.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-11-25T04:21:59.533 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:59.533 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-base-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:59.533 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-common-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:59.534 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-fuse-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:59.534 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-grafana-dashboards-2:18.0.0-7506.gec928368.el9.noarch 2023-11-25T04:21:59.534 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-immutable-object-cache-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:59.534 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mds-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:59.534 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:59.534 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-cephadm-2:18.0.0-7506.gec928368.el9.noarch 2023-11-25T04:21:59.534 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-dashboard-2:18.0.0-7506.gec928368.el9.noarch 2023-11-25T04:21:59.534 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-diskprediction-local-2:18.0.0-7506.gec928368.el9.noarch 2023-11-25T04:21:59.535 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-modules-core-2:18.0.0-7506.gec928368.el9.noarch 2023-11-25T04:21:59.535 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-rook-2:18.0.0-7506.gec928368.el9.noarch 2023-11-25T04:21:59.535 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mon-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:59.535 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-osd-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:59.535 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-prometheus-alerts-2:18.0.0-7506.gec928368.el9.noarch 2023-11-25T04:21:59.535 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-radosgw-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:59.535 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-selinux-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:59.536 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-test-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:59.536 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-volume-2:18.0.0-7506.gec928368.el9.noarch 2023-11-25T04:21:59.536 INFO:teuthology.orchestra.run.smithi067.stdout: cephadm-2:18.0.0-7506.gec928368.el9.noarch 2023-11-25T04:21:59.536 INFO:teuthology.orchestra.run.smithi067.stdout: elfutils-libelf-devel-0.190-1.el9.x86_64 2023-11-25T04:21:59.536 INFO:teuthology.orchestra.run.smithi067.stdout: flex-2.6.4-9.el9.x86_64 2023-11-25T04:21:59.536 INFO:teuthology.orchestra.run.smithi067.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-11-25T04:21:59.536 INFO:teuthology.orchestra.run.smithi067.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-11-25T04:21:59.536 INFO:teuthology.orchestra.run.smithi067.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-11-25T04:21:59.537 INFO:teuthology.orchestra.run.smithi067.stdout: fuse-2.9.9-15.el9.x86_64 2023-11-25T04:21:59.537 INFO:teuthology.orchestra.run.smithi067.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-11-25T04:21:59.537 INFO:teuthology.orchestra.run.smithi067.stdout: jq-1.6-16.el9.x86_64 2023-11-25T04:21:59.537 INFO:teuthology.orchestra.run.smithi067.stdout: libaio-devel-0.3.111-13.el9.x86_64 2023-11-25T04:21:59.537 INFO:teuthology.orchestra.run.smithi067.stdout: libarrow-9.0.0-9.el9.x86_64 2023-11-25T04:21:59.537 INFO:teuthology.orchestra.run.smithi067.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-11-25T04:21:59.537 INFO:teuthology.orchestra.run.smithi067.stdout: libcephfs-devel-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:59.538 INFO:teuthology.orchestra.run.smithi067.stdout: libcephfs2-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:59.538 INFO:teuthology.orchestra.run.smithi067.stdout: libcephsqlite-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:59.538 INFO:teuthology.orchestra.run.smithi067.stdout: libgfortran-11.4.1-2.3.el9.x86_64 2023-11-25T04:21:59.538 INFO:teuthology.orchestra.run.smithi067.stdout: liboath-2.6.7-2.el9.x86_64 2023-11-25T04:21:59.538 INFO:teuthology.orchestra.run.smithi067.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-11-25T04:21:59.538 INFO:teuthology.orchestra.run.smithi067.stdout: libquadmath-11.4.1-2.3.el9.x86_64 2023-11-25T04:21:59.538 INFO:teuthology.orchestra.run.smithi067.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-11-25T04:21:59.538 INFO:teuthology.orchestra.run.smithi067.stdout: librados-devel-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:59.539 INFO:teuthology.orchestra.run.smithi067.stdout: libradosstriper1-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:59.539 INFO:teuthology.orchestra.run.smithi067.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-11-25T04:21:59.539 INFO:teuthology.orchestra.run.smithi067.stdout: librgw2-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:59.539 INFO:teuthology.orchestra.run.smithi067.stdout: libtool-2.4.6-45.el9.x86_64 2023-11-25T04:21:59.539 INFO:teuthology.orchestra.run.smithi067.stdout: libunwind-1.6.2-1.el9.x86_64 2023-11-25T04:21:59.539 INFO:teuthology.orchestra.run.smithi067.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2023-11-25T04:21:59.539 INFO:teuthology.orchestra.run.smithi067.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2023-11-25T04:21:59.539 INFO:teuthology.orchestra.run.smithi067.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-11-25T04:21:59.539 INFO:teuthology.orchestra.run.smithi067.stdout: lua-5.4.4-4.el9.x86_64 2023-11-25T04:21:59.539 INFO:teuthology.orchestra.run.smithi067.stdout: lua-devel-5.4.4-4.el9.x86_64 2023-11-25T04:21:59.539 INFO:teuthology.orchestra.run.smithi067.stdout: luarocks-3.9.2-1.el9.noarch 2023-11-25T04:21:59.539 INFO:teuthology.orchestra.run.smithi067.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-11-25T04:21:59.539 INFO:teuthology.orchestra.run.smithi067.stdout: openblas-0.3.21-2.el9.x86_64 2023-11-25T04:21:59.540 INFO:teuthology.orchestra.run.smithi067.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-11-25T04:21:59.540 INFO:teuthology.orchestra.run.smithi067.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2023-11-25T04:21:59.540 INFO:teuthology.orchestra.run.smithi067.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-11-25T04:21:59.540 INFO:teuthology.orchestra.run.smithi067.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-11-25T04:21:59.540 INFO:teuthology.orchestra.run.smithi067.stdout: python3-babel-2.9.1-2.el9.noarch 2023-11-25T04:21:59.540 INFO:teuthology.orchestra.run.smithi067.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-11-25T04:21:59.540 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-11-25T04:21:59.540 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ceph-argparse-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:59.540 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ceph-common-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:59.541 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cephfs-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:59.541 INFO:teuthology.orchestra.run.smithi067.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-11-25T04:21:59.541 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-11-25T04:21:59.541 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-11-25T04:21:59.541 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-11-25T04:21:59.541 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-11-25T04:21:59.541 INFO:teuthology.orchestra.run.smithi067.stdout: python3-devel-3.9.17-2.el9.x86_64 2023-11-25T04:21:59.542 INFO:teuthology.orchestra.run.smithi067.stdout: python3-google-auth-1:2.23.4-1.el9.noarch 2023-11-25T04:21:59.542 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-11-25T04:21:59.542 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-11-25T04:21:59.542 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-11-25T04:21:59.542 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-11-25T04:21:59.542 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-11-25T04:21:59.542 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-11-25T04:21:59.543 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-11-25T04:21:59.543 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-11-25T04:21:59.543 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-11-25T04:21:59.543 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-11-25T04:21:59.543 INFO:teuthology.orchestra.run.smithi067.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-11-25T04:21:59.543 INFO:teuthology.orchestra.run.smithi067.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-11-25T04:21:59.543 INFO:teuthology.orchestra.run.smithi067.stdout: python3-mako-1.1.4-6.el9.noarch 2023-11-25T04:21:59.544 INFO:teuthology.orchestra.run.smithi067.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-11-25T04:21:59.544 INFO:teuthology.orchestra.run.smithi067.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-11-25T04:21:59.544 INFO:teuthology.orchestra.run.smithi067.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-11-25T04:21:59.544 INFO:teuthology.orchestra.run.smithi067.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-11-25T04:21:59.544 INFO:teuthology.orchestra.run.smithi067.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-11-25T04:21:59.544 INFO:teuthology.orchestra.run.smithi067.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-11-25T04:21:59.545 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-11-25T04:21:59.545 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ply-3.11-14.el9.noarch 2023-11-25T04:21:59.545 INFO:teuthology.orchestra.run.smithi067.stdout: python3-portend-3.1.0-2.el9.noarch 2023-11-25T04:21:59.545 INFO:teuthology.orchestra.run.smithi067.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-11-25T04:21:59.545 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-11-25T04:21:59.545 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-11-25T04:21:59.545 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-11-25T04:21:59.545 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pycparser-2.20-6.el9.noarch 2023-11-25T04:21:59.545 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pytz-2021.1-5.el9.noarch 2023-11-25T04:21:59.545 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rados-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:59.545 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rbd-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:59.546 INFO:teuthology.orchestra.run.smithi067.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-11-25T04:21:59.546 INFO:teuthology.orchestra.run.smithi067.stdout: python3-requests-2.25.1-7.el9.noarch 2023-11-25T04:21:59.546 INFO:teuthology.orchestra.run.smithi067.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-11-25T04:21:59.546 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rgw-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:59.546 INFO:teuthology.orchestra.run.smithi067.stdout: python3-routes-2.5.1-5.el9.noarch 2023-11-25T04:21:59.546 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rsa-4.9-2.el9.noarch 2023-11-25T04:21:59.546 INFO:teuthology.orchestra.run.smithi067.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-11-25T04:21:59.546 INFO:teuthology.orchestra.run.smithi067.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-11-25T04:21:59.546 INFO:teuthology.orchestra.run.smithi067.stdout: python3-toml-0.10.2-6.el9.noarch 2023-11-25T04:21:59.546 INFO:teuthology.orchestra.run.smithi067.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-11-25T04:21:59.547 INFO:teuthology.orchestra.run.smithi067.stdout: python3-urllib3-1.26.5-4.el9.noarch 2023-11-25T04:21:59.547 INFO:teuthology.orchestra.run.smithi067.stdout: python3-webob-1.8.7-6.el9.noarch 2023-11-25T04:21:59.547 INFO:teuthology.orchestra.run.smithi067.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-11-25T04:21:59.547 INFO:teuthology.orchestra.run.smithi067.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-11-25T04:21:59.547 INFO:teuthology.orchestra.run.smithi067.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-11-25T04:21:59.547 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-fuse-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:59.547 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-mirror-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:59.547 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-nbd-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:21:59.548 INFO:teuthology.orchestra.run.smithi067.stdout: re2-1:20211101-3.el9.x86_64 2023-11-25T04:21:59.548 INFO:teuthology.orchestra.run.smithi067.stdout: socat-1.7.4.1-5.el9.x86_64 2023-11-25T04:21:59.548 INFO:teuthology.orchestra.run.smithi067.stdout: thrift-0.15.0-2.el9.x86_64 2023-11-25T04:21:59.548 INFO:teuthology.orchestra.run.smithi067.stdout: xfsprogs-devel-5.19.0-4.el9.x86_64 2023-11-25T04:21:59.548 INFO:teuthology.orchestra.run.smithi067.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-11-25T04:21:59.549 INFO:teuthology.orchestra.run.smithi067.stdout: zlib-devel-1.2.11-40.el9.x86_64 2023-11-25T04:21:59.549 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:21:59.549 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2023-11-25T04:21:59.946 DEBUG:teuthology.parallel:result is None 2023-11-25T04:22:00.733 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 142/142 2023-11-25T04:22:00.734 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-25T04:22:00.734 INFO:teuthology.orchestra.run.smithi195.stdout:Upgraded: 2023-11-25T04:22:00.734 INFO:teuthology.orchestra.run.smithi195.stdout: elfutils-debuginfod-client-0.190-1.el9.x86_64 2023-11-25T04:22:00.734 INFO:teuthology.orchestra.run.smithi195.stdout: elfutils-libelf-0.190-1.el9.x86_64 2023-11-25T04:22:00.734 INFO:teuthology.orchestra.run.smithi195.stdout: elfutils-libs-0.190-1.el9.x86_64 2023-11-25T04:22:00.734 INFO:teuthology.orchestra.run.smithi195.stdout: librados2-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:22:00.734 INFO:teuthology.orchestra.run.smithi195.stdout: librbd1-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:22:00.734 INFO:teuthology.orchestra.run.smithi195.stdout: openssl-1:3.0.7-25.el9.x86_64 2023-11-25T04:22:00.735 INFO:teuthology.orchestra.run.smithi195.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2023-11-25T04:22:00.735 INFO:teuthology.orchestra.run.smithi195.stdout:Installed: 2023-11-25T04:22:00.735 INFO:teuthology.orchestra.run.smithi195.stdout: automake-1.16.2-8.el9.noarch 2023-11-25T04:22:00.735 INFO:teuthology.orchestra.run.smithi195.stdout: bison-3.7.4-5.el9.x86_64 2023-11-25T04:22:00.735 INFO:teuthology.orchestra.run.smithi195.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-11-25T04:22:00.735 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:22:00.735 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-base-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:22:00.735 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-common-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:22:00.735 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-fuse-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:22:00.735 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-grafana-dashboards-2:18.0.0-7506.gec928368.el9.noarch 2023-11-25T04:22:00.735 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-immutable-object-cache-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:22:00.736 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mds-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:22:00.736 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:22:00.736 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-cephadm-2:18.0.0-7506.gec928368.el9.noarch 2023-11-25T04:22:00.736 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-dashboard-2:18.0.0-7506.gec928368.el9.noarch 2023-11-25T04:22:00.736 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-diskprediction-local-2:18.0.0-7506.gec928368.el9.noarch 2023-11-25T04:22:00.736 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-modules-core-2:18.0.0-7506.gec928368.el9.noarch 2023-11-25T04:22:00.736 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-rook-2:18.0.0-7506.gec928368.el9.noarch 2023-11-25T04:22:00.736 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mon-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:22:00.736 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-osd-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:22:00.736 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-prometheus-alerts-2:18.0.0-7506.gec928368.el9.noarch 2023-11-25T04:22:00.736 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-radosgw-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:22:00.736 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-selinux-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:22:00.737 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-test-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:22:00.737 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-volume-2:18.0.0-7506.gec928368.el9.noarch 2023-11-25T04:22:00.737 INFO:teuthology.orchestra.run.smithi195.stdout: cephadm-2:18.0.0-7506.gec928368.el9.noarch 2023-11-25T04:22:00.737 INFO:teuthology.orchestra.run.smithi195.stdout: elfutils-libelf-devel-0.190-1.el9.x86_64 2023-11-25T04:22:00.737 INFO:teuthology.orchestra.run.smithi195.stdout: flex-2.6.4-9.el9.x86_64 2023-11-25T04:22:00.737 INFO:teuthology.orchestra.run.smithi195.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-11-25T04:22:00.737 INFO:teuthology.orchestra.run.smithi195.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-11-25T04:22:00.737 INFO:teuthology.orchestra.run.smithi195.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-11-25T04:22:00.737 INFO:teuthology.orchestra.run.smithi195.stdout: fuse-2.9.9-15.el9.x86_64 2023-11-25T04:22:00.737 INFO:teuthology.orchestra.run.smithi195.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-11-25T04:22:00.737 INFO:teuthology.orchestra.run.smithi195.stdout: jq-1.6-16.el9.x86_64 2023-11-25T04:22:00.738 INFO:teuthology.orchestra.run.smithi195.stdout: libaio-devel-0.3.111-13.el9.x86_64 2023-11-25T04:22:00.738 INFO:teuthology.orchestra.run.smithi195.stdout: libarrow-9.0.0-9.el9.x86_64 2023-11-25T04:22:00.738 INFO:teuthology.orchestra.run.smithi195.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-11-25T04:22:00.738 INFO:teuthology.orchestra.run.smithi195.stdout: libcephfs-devel-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:22:00.738 INFO:teuthology.orchestra.run.smithi195.stdout: libcephfs2-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:22:00.738 INFO:teuthology.orchestra.run.smithi195.stdout: libcephsqlite-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:22:00.738 INFO:teuthology.orchestra.run.smithi195.stdout: libgfortran-11.4.1-2.3.el9.x86_64 2023-11-25T04:22:00.738 INFO:teuthology.orchestra.run.smithi195.stdout: liboath-2.6.7-2.el9.x86_64 2023-11-25T04:22:00.738 INFO:teuthology.orchestra.run.smithi195.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-11-25T04:22:00.738 INFO:teuthology.orchestra.run.smithi195.stdout: libquadmath-11.4.1-2.3.el9.x86_64 2023-11-25T04:22:00.738 INFO:teuthology.orchestra.run.smithi195.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-11-25T04:22:00.739 INFO:teuthology.orchestra.run.smithi195.stdout: librados-devel-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:22:00.739 INFO:teuthology.orchestra.run.smithi195.stdout: libradosstriper1-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:22:00.739 INFO:teuthology.orchestra.run.smithi195.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-11-25T04:22:00.739 INFO:teuthology.orchestra.run.smithi195.stdout: librgw2-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:22:00.739 INFO:teuthology.orchestra.run.smithi195.stdout: libtool-2.4.6-45.el9.x86_64 2023-11-25T04:22:00.739 INFO:teuthology.orchestra.run.smithi195.stdout: libunwind-1.6.2-1.el9.x86_64 2023-11-25T04:22:00.739 INFO:teuthology.orchestra.run.smithi195.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2023-11-25T04:22:00.739 INFO:teuthology.orchestra.run.smithi195.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2023-11-25T04:22:00.739 INFO:teuthology.orchestra.run.smithi195.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-11-25T04:22:00.739 INFO:teuthology.orchestra.run.smithi195.stdout: lua-5.4.4-4.el9.x86_64 2023-11-25T04:22:00.739 INFO:teuthology.orchestra.run.smithi195.stdout: lua-devel-5.4.4-4.el9.x86_64 2023-11-25T04:22:00.739 INFO:teuthology.orchestra.run.smithi195.stdout: luarocks-3.9.2-1.el9.noarch 2023-11-25T04:22:00.740 INFO:teuthology.orchestra.run.smithi195.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-11-25T04:22:00.740 INFO:teuthology.orchestra.run.smithi195.stdout: openblas-0.3.21-2.el9.x86_64 2023-11-25T04:22:00.740 INFO:teuthology.orchestra.run.smithi195.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-11-25T04:22:00.740 INFO:teuthology.orchestra.run.smithi195.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2023-11-25T04:22:00.740 INFO:teuthology.orchestra.run.smithi195.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-11-25T04:22:00.740 INFO:teuthology.orchestra.run.smithi195.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-11-25T04:22:00.740 INFO:teuthology.orchestra.run.smithi195.stdout: python3-babel-2.9.1-2.el9.noarch 2023-11-25T04:22:00.740 INFO:teuthology.orchestra.run.smithi195.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-11-25T04:22:00.740 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-11-25T04:22:00.740 INFO:teuthology.orchestra.run.smithi195.stdout: python3-ceph-argparse-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:22:00.740 INFO:teuthology.orchestra.run.smithi195.stdout: python3-ceph-common-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:22:00.740 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cephfs-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:22:00.740 INFO:teuthology.orchestra.run.smithi195.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-11-25T04:22:00.741 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-11-25T04:22:00.741 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-11-25T04:22:00.741 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-11-25T04:22:00.741 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-11-25T04:22:00.741 INFO:teuthology.orchestra.run.smithi195.stdout: python3-devel-3.9.17-2.el9.x86_64 2023-11-25T04:22:00.741 INFO:teuthology.orchestra.run.smithi195.stdout: python3-google-auth-1:2.23.4-1.el9.noarch 2023-11-25T04:22:00.741 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-11-25T04:22:00.741 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-11-25T04:22:00.741 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-11-25T04:22:00.741 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-11-25T04:22:00.741 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-11-25T04:22:00.741 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-11-25T04:22:00.742 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-11-25T04:22:00.742 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-11-25T04:22:00.742 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-11-25T04:22:00.742 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-11-25T04:22:00.742 INFO:teuthology.orchestra.run.smithi195.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-11-25T04:22:00.742 INFO:teuthology.orchestra.run.smithi195.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-11-25T04:22:00.742 INFO:teuthology.orchestra.run.smithi195.stdout: python3-mako-1.1.4-6.el9.noarch 2023-11-25T04:22:00.742 INFO:teuthology.orchestra.run.smithi195.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-11-25T04:22:00.742 INFO:teuthology.orchestra.run.smithi195.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-11-25T04:22:00.742 INFO:teuthology.orchestra.run.smithi195.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-11-25T04:22:00.742 INFO:teuthology.orchestra.run.smithi195.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-11-25T04:22:00.743 INFO:teuthology.orchestra.run.smithi195.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-11-25T04:22:00.743 INFO:teuthology.orchestra.run.smithi195.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-11-25T04:22:00.743 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-11-25T04:22:00.743 INFO:teuthology.orchestra.run.smithi195.stdout: python3-ply-3.11-14.el9.noarch 2023-11-25T04:22:00.743 INFO:teuthology.orchestra.run.smithi195.stdout: python3-portend-3.1.0-2.el9.noarch 2023-11-25T04:22:00.743 INFO:teuthology.orchestra.run.smithi195.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-11-25T04:22:00.743 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-11-25T04:22:00.743 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-11-25T04:22:00.743 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-11-25T04:22:00.743 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pycparser-2.20-6.el9.noarch 2023-11-25T04:22:00.743 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pytz-2021.1-5.el9.noarch 2023-11-25T04:22:00.743 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rados-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:22:00.744 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rbd-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:22:00.744 INFO:teuthology.orchestra.run.smithi195.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-11-25T04:22:00.744 INFO:teuthology.orchestra.run.smithi195.stdout: python3-requests-2.25.1-7.el9.noarch 2023-11-25T04:22:00.744 INFO:teuthology.orchestra.run.smithi195.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-11-25T04:22:00.744 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rgw-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:22:00.744 INFO:teuthology.orchestra.run.smithi195.stdout: python3-routes-2.5.1-5.el9.noarch 2023-11-25T04:22:00.744 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rsa-4.9-2.el9.noarch 2023-11-25T04:22:00.744 INFO:teuthology.orchestra.run.smithi195.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-11-25T04:22:00.744 INFO:teuthology.orchestra.run.smithi195.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-11-25T04:22:00.744 INFO:teuthology.orchestra.run.smithi195.stdout: python3-toml-0.10.2-6.el9.noarch 2023-11-25T04:22:00.744 INFO:teuthology.orchestra.run.smithi195.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-11-25T04:22:00.744 INFO:teuthology.orchestra.run.smithi195.stdout: python3-urllib3-1.26.5-4.el9.noarch 2023-11-25T04:22:00.745 INFO:teuthology.orchestra.run.smithi195.stdout: python3-webob-1.8.7-6.el9.noarch 2023-11-25T04:22:00.745 INFO:teuthology.orchestra.run.smithi195.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-11-25T04:22:00.745 INFO:teuthology.orchestra.run.smithi195.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-11-25T04:22:00.745 INFO:teuthology.orchestra.run.smithi195.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-11-25T04:22:00.745 INFO:teuthology.orchestra.run.smithi195.stdout: rbd-fuse-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:22:00.745 INFO:teuthology.orchestra.run.smithi195.stdout: rbd-mirror-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:22:00.745 INFO:teuthology.orchestra.run.smithi195.stdout: rbd-nbd-2:18.0.0-7506.gec928368.el9.x86_64 2023-11-25T04:22:00.745 INFO:teuthology.orchestra.run.smithi195.stdout: re2-1:20211101-3.el9.x86_64 2023-11-25T04:22:00.745 INFO:teuthology.orchestra.run.smithi195.stdout: socat-1.7.4.1-5.el9.x86_64 2023-11-25T04:22:00.745 INFO:teuthology.orchestra.run.smithi195.stdout: thrift-0.15.0-2.el9.x86_64 2023-11-25T04:22:00.745 INFO:teuthology.orchestra.run.smithi195.stdout: xfsprogs-devel-5.19.0-4.el9.x86_64 2023-11-25T04:22:00.745 INFO:teuthology.orchestra.run.smithi195.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-11-25T04:22:00.746 INFO:teuthology.orchestra.run.smithi195.stdout: zlib-devel-1.2.11-40.el9.x86_64 2023-11-25T04:22:00.746 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-25T04:22:00.746 INFO:teuthology.orchestra.run.smithi195.stdout:Complete! 2023-11-25T04:22:01.092 DEBUG:teuthology.parallel:result is None 2023-11-25T04:22:01.093 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ec928368edcc5fc71884a56fd9fd821b148f4e30 2023-11-25T04:22:01.310 DEBUG:teuthology.orchestra.run.smithi067:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-25T04:22:01.337 INFO:teuthology.orchestra.run.smithi067.stdout:18.0.0-7506.gec928368.el9 2023-11-25T04:22:01.338 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-7506.gec928368.el9 2023-11-25T04:22:01.338 INFO:teuthology.task.install:The correct ceph version 18.0.0-7506.gec928368 is installed. 2023-11-25T04:22:01.338 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ec928368edcc5fc71884a56fd9fd821b148f4e30 2023-11-25T04:22:01.555 DEBUG:teuthology.orchestra.run.smithi076:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-25T04:22:01.582 INFO:teuthology.orchestra.run.smithi076.stdout:18.0.0-7506.gec928368.el9 2023-11-25T04:22:01.582 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-7506.gec928368.el9 2023-11-25T04:22:01.582 INFO:teuthology.task.install:The correct ceph version 18.0.0-7506.gec928368 is installed. 2023-11-25T04:22:01.583 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ec928368edcc5fc71884a56fd9fd821b148f4e30 2023-11-25T04:22:01.802 DEBUG:teuthology.orchestra.run.smithi092:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-25T04:22:01.828 INFO:teuthology.orchestra.run.smithi092.stdout:18.0.0-7506.gec928368.el9 2023-11-25T04:22:01.828 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-7506.gec928368.el9 2023-11-25T04:22:01.828 INFO:teuthology.task.install:The correct ceph version 18.0.0-7506.gec928368 is installed. 2023-11-25T04:22:01.829 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ec928368edcc5fc71884a56fd9fd821b148f4e30 2023-11-25T04:22:01.966 DEBUG:teuthology.orchestra.run.smithi195:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-25T04:22:01.989 INFO:teuthology.orchestra.run.smithi195.stdout:18.0.0-7506.gec928368.el9 2023-11-25T04:22:01.990 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-7506.gec928368.el9 2023-11-25T04:22:01.990 INFO:teuthology.task.install:The correct ceph version 18.0.0-7506.gec928368 is installed. 2023-11-25T04:22:01.990 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-11-25T04:22:01.990 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-11-25T04:22:01.990 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-25T04:22:02.030 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-25T04:22:02.031 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-25T04:22:02.064 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-25T04:22:02.065 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-25T04:22:02.099 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-25T04:22:02.099 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-25T04:22:02.132 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-11-25T04:22:02.132 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-11-25T04:22:02.133 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/usr/bin/daemon-helper 2023-11-25T04:22:02.169 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-25T04:22:02.245 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-25T04:22:02.246 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/usr/bin/daemon-helper 2023-11-25T04:22:02.275 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-25T04:22:02.345 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-25T04:22:02.345 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/daemon-helper 2023-11-25T04:22:02.376 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-25T04:22:02.447 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-25T04:22:02.447 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/usr/bin/daemon-helper 2023-11-25T04:22:02.478 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-25T04:22:02.550 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-11-25T04:22:02.550 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-11-25T04:22:02.550 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-25T04:22:02.586 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-25T04:22:02.660 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-25T04:22:02.660 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-25T04:22:02.691 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-25T04:22:02.762 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-25T04:22:02.762 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-25T04:22:02.793 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-25T04:22:02.864 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-25T04:22:02.864 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-25T04:22:02.895 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-25T04:22:02.965 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-11-25T04:22:02.966 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-11-25T04:22:02.966 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/usr/bin/stdin-killer 2023-11-25T04:22:03.002 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-25T04:22:03.076 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-25T04:22:03.077 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/usr/bin/stdin-killer 2023-11-25T04:22:03.107 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-25T04:22:03.177 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-25T04:22:03.178 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/stdin-killer 2023-11-25T04:22:03.208 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-25T04:22:03.278 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-25T04:22:03.278 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/usr/bin/stdin-killer 2023-11-25T04:22:03.309 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-25T04:22:03.379 INFO:teuthology.run_tasks:Running task ceph... 2023-11-25T04:22:03.450 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-11-25T04:22:03.451 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 777 /var/log/ceph 2023-11-25T04:22:03.453 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 777 /var/log/ceph 2023-11-25T04:22:03.457 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 777 /var/log/ceph 2023-11-25T04:22:03.460 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod 777 /var/log/ceph 2023-11-25T04:22:03.489 INFO:tasks.ceph:Disabling ceph logrotate... 2023-11-25T04:22:03.489 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-25T04:22:03.529 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-25T04:22:03.531 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-25T04:22:03.533 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-25T04:22:03.562 INFO:tasks.ceph:Creating extra log directories... 2023-11-25T04:22:03.563 DEBUG:teuthology.orchestra.run.smithi067:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-25T04:22:03.604 DEBUG:teuthology.orchestra.run.smithi076:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-25T04:22:03.606 DEBUG:teuthology.orchestra.run.smithi092:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-25T04:22:03.609 DEBUG:teuthology.orchestra.run.smithi195:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-25T04:22:03.647 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-11-25T04:22:03.647 INFO:tasks.ceph:config {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(MDS_TRIM\\)', '\\(MDS_SLOW_REQUEST\\)', 'MDS_SLOW_METADATA_IO', 'Behind on trimming', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-11-25T04:22:03.648 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2023-11-24_21:20:59-powercycle-main-distro-default-smithi/7466144', 'branch': 'main', 'description': 'powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-low-osd-mem-target powercycle/default supported-distros/centos_latest tasks/cfuse_workunit_suites_fsx thrashosds-health}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'job_id': '7466144', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'yuriw-2023-11-24_21:20:59-powercycle-main-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'main'}, 'ceph': {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(MDS_TRIM\\)', '\\(MDS_SLOW_REQUEST\\)', 'MDS_SLOW_METADATA_IO', 'Behind on trimming', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from'], 'sha1': 'ec928368edcc5fc71884a56fd9fd821b148f4e30'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'ec928368edcc5fc71884a56fd9fd821b148f4e30'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'main', 'sha1': 'ec928368edcc5fc71884a56fd9fd821b148f4e30'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 75, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'mgr.y', 'mds.a', 'client.0'], ['osd.0'], ['osd.1'], ['osd.2']], 'seed': 3774, 'sha1': 'ec928368edcc5fc71884a56fd9fd821b148f4e30', 'sleep_before_teardown': 0, 'subset': '111/12000', 'suite': 'powercycle', 'suite_branch': 'main', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_ec928368edcc5fc71884a56fd9fd821b148f4e30/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': 'ec928368edcc5fc71884a56fd9fd821b148f4e30', 'targets': {'smithi067.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNyhVVF/XtgfyRRDww3bmZ7EPginiqC6uqiypg/ZyPafcAIeLt7VaE94CXqC6yGNd5D2c5UnO4VwxGQYoizloRo=', 'smithi076.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCC/Z4U69AnhkPFrQh+/iMSSZP8/tiy64EQsC7bzJt6p7iGC8jRA87LBnDjvkLlrBj8b8RNN7AYBEuytF8IFbws=', 'smithi092.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIfYQcptwzwhxujcHjAVShwsFCWARoaYbV62s9iIILQuBy0lgscMzeefkNhuHZOia3Xr4VIL6eduVJA3y+mnv9A=', 'smithi195.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLDl41IEbYPQ77UvNCnICkwMfSo5Jnv8VURPY8nB7lwu6U41Fbg5IMmek6b8dvVfg1GWiwOdYIFT+w6cMVtbB3Q='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': 'ec928368edcc5fc71884a56fd9fd821b148f4e30'}}, {'ceph': None}, {'thrashosds': {'chance_down': 1.0, 'powercycle': True, 'timeout': 600}}, {'ceph-fuse': None}, {'workunit': {'clients': {'all': ['suites/fsx.sh']}, 'timeout': '6h'}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '2442239d2653456406c25ae0c71b689c8f2657b6', 'timestamp': '2023-11-24_21:20:59', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.232157'} 2023-11-25T04:22:03.648 DEBUG:teuthology.orchestra.run.smithi067:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-25T04:22:03.678 DEBUG:teuthology.orchestra.run.smithi076:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-25T04:22:03.681 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-25T04:22:03.683 DEBUG:teuthology.orchestra.run.smithi195:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-25T04:22:03.698 DEBUG:teuthology.orchestra.run.smithi067:> sudo install -d -m0777 -- /var/run/ceph 2023-11-25T04:22:03.735 DEBUG:teuthology.orchestra.run.smithi076:> sudo install -d -m0777 -- /var/run/ceph 2023-11-25T04:22:03.737 DEBUG:teuthology.orchestra.run.smithi092:> sudo install -d -m0777 -- /var/run/ceph 2023-11-25T04:22:03.740 DEBUG:teuthology.orchestra.run.smithi195:> sudo install -d -m0777 -- /var/run/ceph 2023-11-25T04:22:03.767 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-25T04:22:03.767 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/scratch_devs of=/dev/stdout 2023-11-25T04:22:03.820 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-25T04:22:03.820 DEBUG:teuthology.orchestra.run.smithi076:> stat /dev/vg_nvme/lv_1 2023-11-25T04:22:03.875 INFO:teuthology.orchestra.run.smithi076.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-25T04:22:03.875 INFO:teuthology.orchestra.run.smithi076.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-25T04:22:03.876 INFO:teuthology.orchestra.run.smithi076.stdout:Device: 5h/5d Inode: 664 Links: 1 2023-11-25T04:22:03.876 INFO:teuthology.orchestra.run.smithi076.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-25T04:22:03.876 INFO:teuthology.orchestra.run.smithi076.stdout:Context: system_u:object_r:device_t:s0 2023-11-25T04:22:03.876 INFO:teuthology.orchestra.run.smithi076.stdout:Access: 2023-11-25 04:21:50.296093526 +0000 2023-11-25T04:22:03.876 INFO:teuthology.orchestra.run.smithi076.stdout:Modify: 2023-11-25 04:19:38.624375612 +0000 2023-11-25T04:22:03.876 INFO:teuthology.orchestra.run.smithi076.stdout:Change: 2023-11-25 04:19:38.624375612 +0000 2023-11-25T04:22:03.876 INFO:teuthology.orchestra.run.smithi076.stdout: Birth: 2023-11-25 04:19:38.624375612 +0000 2023-11-25T04:22:03.877 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-25T04:22:03.942 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records in 2023-11-25T04:22:03.942 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records out 2023-11-25T04:22:03.942 INFO:teuthology.orchestra.run.smithi076.stderr:512 bytes copied, 0.000208179 s, 2.5 MB/s 2023-11-25T04:22:03.943 DEBUG:teuthology.orchestra.run.smithi076:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-25T04:22:04.000 DEBUG:teuthology.orchestra.run.smithi076:> stat /dev/vg_nvme/lv_2 2023-11-25T04:22:04.055 INFO:teuthology.orchestra.run.smithi076.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-25T04:22:04.055 INFO:teuthology.orchestra.run.smithi076.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-25T04:22:04.055 INFO:teuthology.orchestra.run.smithi076.stdout:Device: 5h/5d Inode: 683 Links: 1 2023-11-25T04:22:04.055 INFO:teuthology.orchestra.run.smithi076.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-25T04:22:04.055 INFO:teuthology.orchestra.run.smithi076.stdout:Context: system_u:object_r:device_t:s0 2023-11-25T04:22:04.055 INFO:teuthology.orchestra.run.smithi076.stdout:Access: 2023-11-25 04:21:50.296093526 +0000 2023-11-25T04:22:04.055 INFO:teuthology.orchestra.run.smithi076.stdout:Modify: 2023-11-25 04:19:39.133359274 +0000 2023-11-25T04:22:04.055 INFO:teuthology.orchestra.run.smithi076.stdout:Change: 2023-11-25 04:19:39.133359274 +0000 2023-11-25T04:22:04.055 INFO:teuthology.orchestra.run.smithi076.stdout: Birth: 2023-11-25 04:19:39.133359274 +0000 2023-11-25T04:22:04.056 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-25T04:22:04.120 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records in 2023-11-25T04:22:04.120 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records out 2023-11-25T04:22:04.120 INFO:teuthology.orchestra.run.smithi076.stderr:512 bytes copied, 0.00019246 s, 2.7 MB/s 2023-11-25T04:22:04.121 DEBUG:teuthology.orchestra.run.smithi076:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-25T04:22:04.177 DEBUG:teuthology.orchestra.run.smithi076:> stat /dev/vg_nvme/lv_3 2023-11-25T04:22:04.232 INFO:teuthology.orchestra.run.smithi076.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-25T04:22:04.233 INFO:teuthology.orchestra.run.smithi076.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-25T04:22:04.233 INFO:teuthology.orchestra.run.smithi076.stdout:Device: 5h/5d Inode: 702 Links: 1 2023-11-25T04:22:04.233 INFO:teuthology.orchestra.run.smithi076.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-25T04:22:04.233 INFO:teuthology.orchestra.run.smithi076.stdout:Context: system_u:object_r:device_t:s0 2023-11-25T04:22:04.233 INFO:teuthology.orchestra.run.smithi076.stdout:Access: 2023-11-25 04:21:50.296093526 +0000 2023-11-25T04:22:04.233 INFO:teuthology.orchestra.run.smithi076.stdout:Modify: 2023-11-25 04:19:39.618343708 +0000 2023-11-25T04:22:04.234 INFO:teuthology.orchestra.run.smithi076.stdout:Change: 2023-11-25 04:19:39.618343708 +0000 2023-11-25T04:22:04.234 INFO:teuthology.orchestra.run.smithi076.stdout: Birth: 2023-11-25 04:19:39.618343708 +0000 2023-11-25T04:22:04.234 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-25T04:22:04.298 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records in 2023-11-25T04:22:04.298 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records out 2023-11-25T04:22:04.298 INFO:teuthology.orchestra.run.smithi076.stderr:512 bytes copied, 0.000207674 s, 2.5 MB/s 2023-11-25T04:22:04.299 DEBUG:teuthology.orchestra.run.smithi076:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-25T04:22:04.355 DEBUG:teuthology.orchestra.run.smithi076:> stat /dev/vg_nvme/lv_4 2023-11-25T04:22:04.410 INFO:teuthology.orchestra.run.smithi076.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-25T04:22:04.411 INFO:teuthology.orchestra.run.smithi076.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-25T04:22:04.411 INFO:teuthology.orchestra.run.smithi076.stdout:Device: 5h/5d Inode: 721 Links: 1 2023-11-25T04:22:04.411 INFO:teuthology.orchestra.run.smithi076.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-25T04:22:04.411 INFO:teuthology.orchestra.run.smithi076.stdout:Context: system_u:object_r:device_t:s0 2023-11-25T04:22:04.411 INFO:teuthology.orchestra.run.smithi076.stdout:Access: 2023-11-25 04:21:50.296093526 +0000 2023-11-25T04:22:04.411 INFO:teuthology.orchestra.run.smithi076.stdout:Modify: 2023-11-25 04:19:40.110327918 +0000 2023-11-25T04:22:04.411 INFO:teuthology.orchestra.run.smithi076.stdout:Change: 2023-11-25 04:19:40.110327918 +0000 2023-11-25T04:22:04.411 INFO:teuthology.orchestra.run.smithi076.stdout: Birth: 2023-11-25 04:19:40.110327918 +0000 2023-11-25T04:22:04.412 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-25T04:22:04.475 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records in 2023-11-25T04:22:04.476 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records out 2023-11-25T04:22:04.476 INFO:teuthology.orchestra.run.smithi076.stderr:512 bytes copied, 0.000250624 s, 2.0 MB/s 2023-11-25T04:22:04.477 DEBUG:teuthology.orchestra.run.smithi076:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-25T04:22:04.533 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1'} 2023-11-25T04:22:04.533 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-25T04:22:04.533 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/scratch_devs of=/dev/stdout 2023-11-25T04:22:04.551 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-25T04:22:04.551 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_1 2023-11-25T04:22:04.608 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-25T04:22:04.609 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-25T04:22:04.609 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 663 Links: 1 2023-11-25T04:22:04.609 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-25T04:22:04.609 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2023-11-25T04:22:04.609 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2023-11-25 04:21:51.620709162 +0000 2023-11-25T04:22:04.609 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2023-11-25 04:19:38.641591560 +0000 2023-11-25T04:22:04.609 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2023-11-25 04:19:38.641591560 +0000 2023-11-25T04:22:04.610 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2023-11-25 04:19:38.641591560 +0000 2023-11-25T04:22:04.610 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-25T04:22:04.676 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2023-11-25T04:22:04.676 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2023-11-25T04:22:04.677 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000178362 s, 2.9 MB/s 2023-11-25T04:22:04.678 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-25T04:22:04.736 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_2 2023-11-25T04:22:04.792 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-25T04:22:04.792 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-25T04:22:04.793 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 682 Links: 1 2023-11-25T04:22:04.793 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-25T04:22:04.793 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2023-11-25T04:22:04.794 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2023-11-25 04:21:51.620709162 +0000 2023-11-25T04:22:04.794 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2023-11-25 04:19:39.142576955 +0000 2023-11-25T04:22:04.794 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2023-11-25 04:19:39.142576955 +0000 2023-11-25T04:22:04.794 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2023-11-25 04:19:39.142576955 +0000 2023-11-25T04:22:04.794 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-25T04:22:04.859 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2023-11-25T04:22:04.860 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2023-11-25T04:22:04.860 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000205052 s, 2.5 MB/s 2023-11-25T04:22:04.860 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-25T04:22:04.917 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_3 2023-11-25T04:22:04.973 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-25T04:22:04.973 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-25T04:22:04.973 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 701 Links: 1 2023-11-25T04:22:04.973 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-25T04:22:04.974 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2023-11-25T04:22:04.974 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2023-11-25 04:21:51.621709133 +0000 2023-11-25T04:22:04.974 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2023-11-25 04:19:39.638562495 +0000 2023-11-25T04:22:04.974 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2023-11-25 04:19:39.638562495 +0000 2023-11-25T04:22:04.974 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2023-11-25 04:19:39.638562495 +0000 2023-11-25T04:22:04.974 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-25T04:22:05.039 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2023-11-25T04:22:05.039 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2023-11-25T04:22:05.039 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000218047 s, 2.3 MB/s 2023-11-25T04:22:05.041 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-25T04:22:05.098 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_4 2023-11-25T04:22:05.153 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-25T04:22:05.154 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-25T04:22:05.154 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 721 Links: 1 2023-11-25T04:22:05.154 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-25T04:22:05.154 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2023-11-25T04:22:05.154 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2023-11-25 04:21:51.621709133 +0000 2023-11-25T04:22:05.154 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2023-11-25 04:19:40.132548094 +0000 2023-11-25T04:22:05.154 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2023-11-25 04:19:40.132548094 +0000 2023-11-25T04:22:05.154 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2023-11-25 04:19:40.132548094 +0000 2023-11-25T04:22:05.155 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-25T04:22:05.219 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2023-11-25T04:22:05.219 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2023-11-25T04:22:05.219 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000210016 s, 2.4 MB/s 2023-11-25T04:22:05.220 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-25T04:22:05.276 INFO:tasks.ceph:osd dev map: {'osd.1': '/dev/vg_nvme/lv_1'} 2023-11-25T04:22:05.276 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-25T04:22:05.276 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/scratch_devs of=/dev/stdout 2023-11-25T04:22:05.295 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-25T04:22:05.295 DEBUG:teuthology.orchestra.run.smithi195:> stat /dev/vg_nvme/lv_1 2023-11-25T04:22:05.353 INFO:teuthology.orchestra.run.smithi195.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-25T04:22:05.353 INFO:teuthology.orchestra.run.smithi195.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-25T04:22:05.353 INFO:teuthology.orchestra.run.smithi195.stdout:Device: 5h/5d Inode: 662 Links: 1 2023-11-25T04:22:05.353 INFO:teuthology.orchestra.run.smithi195.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-25T04:22:05.353 INFO:teuthology.orchestra.run.smithi195.stdout:Context: system_u:object_r:device_t:s0 2023-11-25T04:22:05.353 INFO:teuthology.orchestra.run.smithi195.stdout:Access: 2023-11-25 04:21:54.264982789 +0000 2023-11-25T04:22:05.353 INFO:teuthology.orchestra.run.smithi195.stdout:Modify: 2023-11-25 04:19:39.477619677 +0000 2023-11-25T04:22:05.353 INFO:teuthology.orchestra.run.smithi195.stdout:Change: 2023-11-25 04:19:39.477619677 +0000 2023-11-25T04:22:05.354 INFO:teuthology.orchestra.run.smithi195.stdout: Birth: 2023-11-25 04:19:39.477619677 +0000 2023-11-25T04:22:05.354 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-25T04:22:05.423 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records in 2023-11-25T04:22:05.424 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records out 2023-11-25T04:22:05.424 INFO:teuthology.orchestra.run.smithi195.stderr:512 bytes copied, 9.7689e-05 s, 5.2 MB/s 2023-11-25T04:22:05.425 DEBUG:teuthology.orchestra.run.smithi195:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-25T04:22:05.484 DEBUG:teuthology.orchestra.run.smithi195:> stat /dev/vg_nvme/lv_2 2023-11-25T04:22:05.542 INFO:teuthology.orchestra.run.smithi195.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-25T04:22:05.542 INFO:teuthology.orchestra.run.smithi195.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-25T04:22:05.542 INFO:teuthology.orchestra.run.smithi195.stdout:Device: 5h/5d Inode: 683 Links: 1 2023-11-25T04:22:05.542 INFO:teuthology.orchestra.run.smithi195.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-25T04:22:05.542 INFO:teuthology.orchestra.run.smithi195.stdout:Context: system_u:object_r:device_t:s0 2023-11-25T04:22:05.543 INFO:teuthology.orchestra.run.smithi195.stdout:Access: 2023-11-25 04:21:54.264982789 +0000 2023-11-25T04:22:05.543 INFO:teuthology.orchestra.run.smithi195.stdout:Modify: 2023-11-25 04:19:39.962610338 +0000 2023-11-25T04:22:05.543 INFO:teuthology.orchestra.run.smithi195.stdout:Change: 2023-11-25 04:19:39.962610338 +0000 2023-11-25T04:22:05.543 INFO:teuthology.orchestra.run.smithi195.stdout: Birth: 2023-11-25 04:19:39.962610338 +0000 2023-11-25T04:22:05.543 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-25T04:22:05.611 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records in 2023-11-25T04:22:05.611 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records out 2023-11-25T04:22:05.611 INFO:teuthology.orchestra.run.smithi195.stderr:512 bytes copied, 0.000113144 s, 4.5 MB/s 2023-11-25T04:22:05.612 DEBUG:teuthology.orchestra.run.smithi195:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-25T04:22:05.669 DEBUG:teuthology.orchestra.run.smithi195:> stat /dev/vg_nvme/lv_3 2023-11-25T04:22:05.726 INFO:teuthology.orchestra.run.smithi195.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-25T04:22:05.726 INFO:teuthology.orchestra.run.smithi195.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-25T04:22:05.726 INFO:teuthology.orchestra.run.smithi195.stdout:Device: 5h/5d Inode: 702 Links: 1 2023-11-25T04:22:05.726 INFO:teuthology.orchestra.run.smithi195.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-25T04:22:05.727 INFO:teuthology.orchestra.run.smithi195.stdout:Context: system_u:object_r:device_t:s0 2023-11-25T04:22:05.727 INFO:teuthology.orchestra.run.smithi195.stdout:Access: 2023-11-25 04:21:54.265982770 +0000 2023-11-25T04:22:05.727 INFO:teuthology.orchestra.run.smithi195.stdout:Modify: 2023-11-25 04:19:40.436601210 +0000 2023-11-25T04:22:05.727 INFO:teuthology.orchestra.run.smithi195.stdout:Change: 2023-11-25 04:19:40.436601210 +0000 2023-11-25T04:22:05.727 INFO:teuthology.orchestra.run.smithi195.stdout: Birth: 2023-11-25 04:19:40.436601210 +0000 2023-11-25T04:22:05.727 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-25T04:22:05.794 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records in 2023-11-25T04:22:05.794 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records out 2023-11-25T04:22:05.794 INFO:teuthology.orchestra.run.smithi195.stderr:512 bytes copied, 0.000110976 s, 4.6 MB/s 2023-11-25T04:22:05.795 DEBUG:teuthology.orchestra.run.smithi195:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-25T04:22:05.852 DEBUG:teuthology.orchestra.run.smithi195:> stat /dev/vg_nvme/lv_4 2023-11-25T04:22:05.907 INFO:teuthology.orchestra.run.smithi195.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-25T04:22:05.907 INFO:teuthology.orchestra.run.smithi195.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-25T04:22:05.907 INFO:teuthology.orchestra.run.smithi195.stdout:Device: 5h/5d Inode: 720 Links: 1 2023-11-25T04:22:05.907 INFO:teuthology.orchestra.run.smithi195.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-25T04:22:05.907 INFO:teuthology.orchestra.run.smithi195.stdout:Context: system_u:object_r:device_t:s0 2023-11-25T04:22:05.907 INFO:teuthology.orchestra.run.smithi195.stdout:Access: 2023-11-25 04:21:54.265982770 +0000 2023-11-25T04:22:05.907 INFO:teuthology.orchestra.run.smithi195.stdout:Modify: 2023-11-25 04:19:40.912592044 +0000 2023-11-25T04:22:05.907 INFO:teuthology.orchestra.run.smithi195.stdout:Change: 2023-11-25 04:19:40.912592044 +0000 2023-11-25T04:22:05.907 INFO:teuthology.orchestra.run.smithi195.stdout: Birth: 2023-11-25 04:19:40.912592044 +0000 2023-11-25T04:22:05.908 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-25T04:22:05.972 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records in 2023-11-25T04:22:05.972 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records out 2023-11-25T04:22:05.972 INFO:teuthology.orchestra.run.smithi195.stderr:512 bytes copied, 8.3885e-05 s, 6.1 MB/s 2023-11-25T04:22:05.974 DEBUG:teuthology.orchestra.run.smithi195:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-25T04:22:06.029 INFO:tasks.ceph:osd dev map: {'osd.2': '/dev/vg_nvme/lv_1'} 2023-11-25T04:22:06.030 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi076.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1'}, Remote(name='ubuntu@smithi092.front.sepia.ceph.com'): {'osd.1': '/dev/vg_nvme/lv_1'}, Remote(name='ubuntu@smithi195.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1'}} 2023-11-25T04:22:06.030 INFO:tasks.ceph:Generating config... 2023-11-25T04:22:06.031 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-11-25T04:22:06.031 INFO:tasks.ceph:[mgr] debug ms = 1 2023-11-25T04:22:06.031 INFO:tasks.ceph:[mon] debug mon = 20 2023-11-25T04:22:06.031 INFO:tasks.ceph:[mon] debug ms = 1 2023-11-25T04:22:06.032 INFO:tasks.ceph:[mon] debug paxos = 20 2023-11-25T04:22:06.032 INFO:tasks.ceph:[osd] bdev async discard = True 2023-11-25T04:22:06.032 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-11-25T04:22:06.032 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-11-25T04:22:06.032 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-11-25T04:22:06.032 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-11-25T04:22:06.032 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-11-25T04:22:06.032 INFO:tasks.ceph:[osd] debug ms = 1 2023-11-25T04:22:06.033 INFO:tasks.ceph:[osd] debug osd = 20 2023-11-25T04:22:06.033 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-11-25T04:22:06.034 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-11-25T04:22:06.034 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-11-25T04:22:06.034 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-11-25T04:22:06.034 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2023-11-25T04:22:06.034 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-11-25T04:22:06.034 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2023-11-25T04:22:06.035 INFO:tasks.ceph:[osd] osd memory target = 1610612736 2023-11-25T04:22:06.035 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-11-25T04:22:06.035 INFO:tasks.ceph:Setting up mon.a... 2023-11-25T04:22:06.035 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-11-25T04:22:06.116 INFO:teuthology.orchestra.run.smithi067.stdout:creating /etc/ceph/ceph.keyring 2023-11-25T04:22:06.117 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-11-25T04:22:06.169 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-25T04:22:06.200 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.67'), ('mon.b', '[v2:172.21.15.67:3301,v1:172.21.15.67:6790]'), ('mon.c', '[v2:172.21.15.67:3302,v1:172.21.15.67:6791]')] 2023-11-25T04:22:06.200 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.67,[v2:172.21.15.67:3301,v1:172.21.15.67:6790],[v2:172.21.15.67:3302,v1:172.21.15.67:6791]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}} 2023-11-25T04:22:06.201 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-11-25T04:22:06.201 DEBUG:teuthology.orchestra.run.smithi067:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-11-25T04:22:06.255 DEBUG:teuthology.orchestra.run.smithi067:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.67 --addv b '[v2:172.21.15.67:3301,v1:172.21.15.67:6790]' --addv c '[v2:172.21.15.67:3302,v1:172.21.15.67:6791]' --print /home/ubuntu/cephtest/ceph.monmap 2023-11-25T04:22:06.343 INFO:teuthology.orchestra.run.smithi067.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-11-25T04:22:06.343 INFO:teuthology.orchestra.run.smithi067.stdout:monmaptool: generated fsid a677f511-1143-4e76-8da6-5ddfadaff077 2023-11-25T04:22:06.343 INFO:teuthology.orchestra.run.smithi067.stdout:setting min_mon_release = pacific 2023-11-25T04:22:06.343 INFO:teuthology.orchestra.run.smithi067.stdout:epoch 0 2023-11-25T04:22:06.343 INFO:teuthology.orchestra.run.smithi067.stdout:fsid a677f511-1143-4e76-8da6-5ddfadaff077 2023-11-25T04:22:06.343 INFO:teuthology.orchestra.run.smithi067.stdout:last_changed 2023-11-25T04:22:06.342409+0000 2023-11-25T04:22:06.343 INFO:teuthology.orchestra.run.smithi067.stdout:created 2023-11-25T04:22:06.342409+0000 2023-11-25T04:22:06.343 INFO:teuthology.orchestra.run.smithi067.stdout:min_mon_release 16 (pacific) 2023-11-25T04:22:06.344 INFO:teuthology.orchestra.run.smithi067.stdout:election_strategy: 1 2023-11-25T04:22:06.344 INFO:teuthology.orchestra.run.smithi067.stdout:0: [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] mon.a 2023-11-25T04:22:06.344 INFO:teuthology.orchestra.run.smithi067.stdout:1: [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] mon.b 2023-11-25T04:22:06.344 INFO:teuthology.orchestra.run.smithi067.stdout:2: [v2:172.21.15.67:3302/0,v1:172.21.15.67:6791/0] mon.c 2023-11-25T04:22:06.344 INFO:teuthology.orchestra.run.smithi067.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-11-25T04:22:06.345 DEBUG:teuthology.orchestra.run.smithi067:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-11-25T04:22:06.360 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID a677f511-1143-4e76-8da6-5ddfadaff077... 2023-11-25T04:22:06.361 DEBUG:teuthology.orchestra.run.smithi067:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-25T04:22:06.403 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-25T04:22:06.405 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-25T04:22:06.407 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-25T04:22:06.459 INFO:teuthology.orchestra.run.smithi076.stdout:[global] 2023-11-25T04:22:06.460 INFO:teuthology.orchestra.run.smithi076.stdout: chdir = "" 2023-11-25T04:22:06.460 INFO:teuthology.orchestra.run.smithi076.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-25T04:22:06.460 INFO:teuthology.orchestra.run.smithi076.stdout: auth supported = cephx 2023-11-25T04:22:06.460 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-25T04:22:06.460 INFO:teuthology.orchestra.run.smithi076.stdout: filestore xattr use omap = true 2023-11-25T04:22:06.460 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-25T04:22:06.460 INFO:teuthology.orchestra.run.smithi076.stdout: mon clock drift allowed = 1.000 2023-11-25T04:22:06.460 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-25T04:22:06.460 INFO:teuthology.orchestra.run.smithi076.stdout: osd crush chooseleaf type = 0 2023-11-25T04:22:06.460 INFO:teuthology.orchestra.run.smithi076.stdout: auth debug = true 2023-11-25T04:22:06.460 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-25T04:22:06.461 INFO:teuthology.orchestra.run.smithi076.stdout: ms die on old message = true 2023-11-25T04:22:06.461 INFO:teuthology.orchestra.run.smithi076.stdout: ms die on bug = true 2023-11-25T04:22:06.461 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-25T04:22:06.461 INFO:teuthology.orchestra.run.smithi076.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-25T04:22:06.461 INFO:teuthology.orchestra.run.smithi076.stdout: mon pg warn max object skew = 0 2023-11-25T04:22:06.461 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-25T04:22:06.461 INFO:teuthology.orchestra.run.smithi076.stdout: # disable pg_autoscaler by default for new pools 2023-11-25T04:22:06.461 INFO:teuthology.orchestra.run.smithi076.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-25T04:22:06.461 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-25T04:22:06.461 INFO:teuthology.orchestra.run.smithi076.stdout: osd pool default size = 2 2023-11-25T04:22:06.461 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-25T04:22:06.461 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd allow primary affinity = true 2023-11-25T04:22:06.462 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd allow pg remap = true 2023-11-25T04:22:06.462 INFO:teuthology.orchestra.run.smithi076.stdout: mon warn on legacy crush tunables = false 2023-11-25T04:22:06.462 INFO:teuthology.orchestra.run.smithi076.stdout: mon warn on crush straw calc version zero = false 2023-11-25T04:22:06.462 INFO:teuthology.orchestra.run.smithi076.stdout: mon warn on no sortbitwise = false 2023-11-25T04:22:06.462 INFO:teuthology.orchestra.run.smithi076.stdout: mon warn on osd down out interval zero = false 2023-11-25T04:22:06.462 INFO:teuthology.orchestra.run.smithi076.stdout: mon warn on too few osds = false 2023-11-25T04:22:06.462 INFO:teuthology.orchestra.run.smithi076.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-25T04:22:06.462 INFO:teuthology.orchestra.run.smithi076.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-25T04:22:06.462 INFO:teuthology.orchestra.run.smithi076.stdout: mon_allow_pool_size_one = true 2023-11-25T04:22:06.462 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-25T04:22:06.462 INFO:teuthology.orchestra.run.smithi076.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-25T04:22:06.462 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-25T04:22:06.463 INFO:teuthology.orchestra.run.smithi076.stdout: osd default data pool replay window = 5 2023-11-25T04:22:06.463 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-25T04:22:06.463 INFO:teuthology.orchestra.run.smithi076.stdout: mon allow pool delete = true 2023-11-25T04:22:06.463 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-25T04:22:06.463 INFO:teuthology.orchestra.run.smithi076.stdout: mon cluster log file level = debug 2023-11-25T04:22:06.463 INFO:teuthology.orchestra.run.smithi076.stdout: debug asserts on shutdown = true 2023-11-25T04:22:06.463 INFO:teuthology.orchestra.run.smithi076.stdout: mon health detail to clog = false 2023-11-25T04:22:06.463 INFO:teuthology.orchestra.run.smithi076.stdout: mon host = "172.21.15.67,[v2:172.21.15.67:3301,v1:172.21.15.67:6790],[v2:172.21.15.67:3302,v1:172.21.15.67:6791]" 2023-11-25T04:22:06.463 INFO:teuthology.orchestra.run.smithi076.stdout: fsid = a677f511-1143-4e76-8da6-5ddfadaff077 2023-11-25T04:22:06.463 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-25T04:22:06.463 INFO:teuthology.orchestra.run.smithi076.stdout:[osd] 2023-11-25T04:22:06.463 INFO:teuthology.orchestra.run.smithi076.stdout: osd journal size = 100 2023-11-25T04:22:06.463 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-25T04:22:06.464 INFO:teuthology.orchestra.run.smithi076.stdout: osd scrub load threshold = 5.0 2023-11-25T04:22:06.464 INFO:teuthology.orchestra.run.smithi076.stdout: osd scrub max interval = 600 2023-11-25T04:22:06.464 INFO:teuthology.orchestra.run.smithi076.stdout: osd mclock profile = high_recovery_ops 2023-11-25T04:22:06.464 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-25T04:22:06.464 INFO:teuthology.orchestra.run.smithi076.stdout: osd recover clone overlap = true 2023-11-25T04:22:06.464 INFO:teuthology.orchestra.run.smithi076.stdout: osd recovery max chunk = 1048576 2023-11-25T04:22:06.464 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-25T04:22:06.464 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug shutdown = true 2023-11-25T04:22:06.464 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug op order = true 2023-11-25T04:22:06.464 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug verify stray on activate = true 2023-11-25T04:22:06.464 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug trim objects = true 2023-11-25T04:22:06.464 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-25T04:22:06.465 INFO:teuthology.orchestra.run.smithi076.stdout: osd open classes on start = true 2023-11-25T04:22:06.465 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug pg log writeout = true 2023-11-25T04:22:06.465 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-25T04:22:06.465 INFO:teuthology.orchestra.run.smithi076.stdout: osd deep scrub update digest min age = 30 2023-11-25T04:22:06.465 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-25T04:22:06.465 INFO:teuthology.orchestra.run.smithi076.stdout: osd map max advance = 10 2023-11-25T04:22:06.465 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-25T04:22:06.465 INFO:teuthology.orchestra.run.smithi076.stdout: journal zero on create = true 2023-11-25T04:22:06.465 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-25T04:22:06.465 INFO:teuthology.orchestra.run.smithi076.stdout: filestore ondisk finisher threads = 3 2023-11-25T04:22:06.465 INFO:teuthology.orchestra.run.smithi076.stdout: filestore apply finisher threads = 3 2023-11-25T04:22:06.465 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-25T04:22:06.466 INFO:teuthology.orchestra.run.smithi076.stdout: bdev debug aio = true 2023-11-25T04:22:06.466 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug misdirected ops = true 2023-11-25T04:22:06.466 INFO:teuthology.orchestra.run.smithi076.stdout: bdev async discard = True 2023-11-25T04:22:06.466 INFO:teuthology.orchestra.run.smithi076.stdout: bdev enable discard = True 2023-11-25T04:22:06.466 INFO:teuthology.orchestra.run.smithi076.stdout: bluestore block size = 96636764160 2023-11-25T04:22:06.466 INFO:teuthology.orchestra.run.smithi076.stdout: bluestore fsck on mount = True 2023-11-25T04:22:06.466 INFO:teuthology.orchestra.run.smithi076.stdout: debug bluefs = 1/20 2023-11-25T04:22:06.466 INFO:teuthology.orchestra.run.smithi076.stdout: debug bluestore = 1/20 2023-11-25T04:22:06.466 INFO:teuthology.orchestra.run.smithi076.stdout: debug ms = 1 2023-11-25T04:22:06.466 INFO:teuthology.orchestra.run.smithi076.stdout: debug osd = 20 2023-11-25T04:22:06.466 INFO:teuthology.orchestra.run.smithi076.stdout: debug rocksdb = 4/10 2023-11-25T04:22:06.466 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd backfillfull_ratio = 0.85 2023-11-25T04:22:06.466 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd full ratio = 0.9 2023-11-25T04:22:06.467 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd nearfull ratio = 0.8 2023-11-25T04:22:06.467 INFO:teuthology.orchestra.run.smithi076.stdout: osd blocked scrub grace period = 3600 2023-11-25T04:22:06.467 INFO:teuthology.orchestra.run.smithi076.stdout: osd failsafe full ratio = 0.95 2023-11-25T04:22:06.467 INFO:teuthology.orchestra.run.smithi076.stdout: osd max markdown count = 1000 2023-11-25T04:22:06.467 INFO:teuthology.orchestra.run.smithi076.stdout: osd memory target = 1610612736 2023-11-25T04:22:06.467 INFO:teuthology.orchestra.run.smithi076.stdout: osd objectstore = bluestore 2023-11-25T04:22:06.467 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-25T04:22:06.467 INFO:teuthology.orchestra.run.smithi076.stdout:[mgr] 2023-11-25T04:22:06.467 INFO:teuthology.orchestra.run.smithi076.stdout: debug ms = 1 2023-11-25T04:22:06.467 INFO:teuthology.orchestra.run.smithi076.stdout: debug mgr = 20 2023-11-25T04:22:06.467 INFO:teuthology.orchestra.run.smithi076.stdout: debug mon = 20 2023-11-25T04:22:06.467 INFO:teuthology.orchestra.run.smithi076.stdout: debug auth = 20 2023-11-25T04:22:06.468 INFO:teuthology.orchestra.run.smithi076.stdout: mon reweight min pgs per osd = 4 2023-11-25T04:22:06.468 INFO:teuthology.orchestra.run.smithi076.stdout: mon reweight min bytes per osd = 10 2023-11-25T04:22:06.468 INFO:teuthology.orchestra.run.smithi076.stdout: mgr/telemetry/nag = false 2023-11-25T04:22:06.468 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-25T04:22:06.468 INFO:teuthology.orchestra.run.smithi076.stdout:[mon] 2023-11-25T04:22:06.468 INFO:teuthology.orchestra.run.smithi076.stdout: debug ms = 1 2023-11-25T04:22:06.468 INFO:teuthology.orchestra.run.smithi076.stdout: debug mon = 20 2023-11-25T04:22:06.468 INFO:teuthology.orchestra.run.smithi076.stdout: debug paxos = 20 2023-11-25T04:22:06.468 INFO:teuthology.orchestra.run.smithi076.stdout: debug auth = 20 2023-11-25T04:22:06.468 INFO:teuthology.orchestra.run.smithi076.stdout: mon data avail warn = 5 2023-11-25T04:22:06.468 INFO:teuthology.orchestra.run.smithi076.stdout: mon mgr mkfs grace = 240 2023-11-25T04:22:06.468 INFO:teuthology.orchestra.run.smithi076.stdout: mon reweight min pgs per osd = 4 2023-11-25T04:22:06.469 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd reporter subtree level = osd 2023-11-25T04:22:06.469 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd prime pg temp = true 2023-11-25T04:22:06.469 INFO:teuthology.orchestra.run.smithi076.stdout: mon reweight min bytes per osd = 10 2023-11-25T04:22:06.469 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-25T04:22:06.469 INFO:teuthology.orchestra.run.smithi076.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-25T04:22:06.469 INFO:teuthology.orchestra.run.smithi076.stdout: auth mon ticket ttl = 660 # 11m 2023-11-25T04:22:06.469 INFO:teuthology.orchestra.run.smithi076.stdout: auth service ticket ttl = 240 # 4m 2023-11-25T04:22:06.469 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-25T04:22:06.469 INFO:teuthology.orchestra.run.smithi076.stdout: # don't complain about insecure global_id in the test suite 2023-11-25T04:22:06.469 INFO:teuthology.orchestra.run.smithi076.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-25T04:22:06.469 INFO:teuthology.orchestra.run.smithi076.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-25T04:22:06.469 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-25T04:22:06.470 INFO:teuthology.orchestra.run.smithi076.stdout: # 1m isn't quite enough 2023-11-25T04:22:06.470 INFO:teuthology.orchestra.run.smithi076.stdout: mon_down_mkfs_grace = 2m 2023-11-25T04:22:06.470 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-25T04:22:06.470 INFO:teuthology.orchestra.run.smithi076.stdout: mon_warn_on_filestore_osds = false 2023-11-25T04:22:06.470 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-25T04:22:06.470 INFO:teuthology.orchestra.run.smithi076.stdout:[client] 2023-11-25T04:22:06.470 INFO:teuthology.orchestra.run.smithi076.stdout: rgw cache enabled = true 2023-11-25T04:22:06.470 INFO:teuthology.orchestra.run.smithi076.stdout: rgw enable ops log = true 2023-11-25T04:22:06.470 INFO:teuthology.orchestra.run.smithi076.stdout: rgw enable usage log = true 2023-11-25T04:22:06.470 INFO:teuthology.orchestra.run.smithi076.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-25T04:22:06.470 INFO:teuthology.orchestra.run.smithi076.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-25T04:22:06.470 INFO:teuthology.orchestra.run.smithi076.stdout:[mon.a] 2023-11-25T04:22:06.471 INFO:teuthology.orchestra.run.smithi076.stdout:[mon.b] 2023-11-25T04:22:06.471 INFO:teuthology.orchestra.run.smithi076.stdout:[mon.c] 2023-11-25T04:22:06.471 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.a] 2023-11-25T04:22:06.471 INFO:teuthology.orchestra.run.smithi092.stdout:[global] 2023-11-25T04:22:06.471 INFO:teuthology.orchestra.run.smithi092.stdout: chdir = "" 2023-11-25T04:22:06.472 INFO:teuthology.orchestra.run.smithi092.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-25T04:22:06.472 INFO:teuthology.orchestra.run.smithi092.stdout: auth supported = cephx 2023-11-25T04:22:06.472 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-25T04:22:06.472 INFO:teuthology.orchestra.run.smithi092.stdout: filestore xattr use omap = true 2023-11-25T04:22:06.472 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-25T04:22:06.472 INFO:teuthology.orchestra.run.smithi092.stdout: mon clock drift allowed = 1.000 2023-11-25T04:22:06.472 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-25T04:22:06.472 INFO:teuthology.orchestra.run.smithi092.stdout: osd crush chooseleaf type = 0 2023-11-25T04:22:06.472 INFO:teuthology.orchestra.run.smithi092.stdout: auth debug = true 2023-11-25T04:22:06.472 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-25T04:22:06.473 INFO:teuthology.orchestra.run.smithi092.stdout: ms die on old message = true 2023-11-25T04:22:06.473 INFO:teuthology.orchestra.run.smithi092.stdout: ms die on bug = true 2023-11-25T04:22:06.473 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-25T04:22:06.473 INFO:teuthology.orchestra.run.smithi092.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-25T04:22:06.473 INFO:teuthology.orchestra.run.smithi092.stdout: mon pg warn max object skew = 0 2023-11-25T04:22:06.473 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-25T04:22:06.473 INFO:teuthology.orchestra.run.smithi092.stdout: # disable pg_autoscaler by default for new pools 2023-11-25T04:22:06.473 INFO:teuthology.orchestra.run.smithi092.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-25T04:22:06.473 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-25T04:22:06.473 INFO:teuthology.orchestra.run.smithi092.stdout: osd pool default size = 2 2023-11-25T04:22:06.473 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-25T04:22:06.473 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd allow primary affinity = true 2023-11-25T04:22:06.474 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd allow pg remap = true 2023-11-25T04:22:06.474 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on legacy crush tunables = false 2023-11-25T04:22:06.474 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on crush straw calc version zero = false 2023-11-25T04:22:06.474 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on no sortbitwise = false 2023-11-25T04:22:06.474 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on osd down out interval zero = false 2023-11-25T04:22:06.474 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on too few osds = false 2023-11-25T04:22:06.474 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-25T04:22:06.474 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-25T04:22:06.474 INFO:teuthology.orchestra.run.smithi092.stdout: mon_allow_pool_size_one = true 2023-11-25T04:22:06.474 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-25T04:22:06.474 INFO:teuthology.orchestra.run.smithi092.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-25T04:22:06.474 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-25T04:22:06.474 INFO:teuthology.orchestra.run.smithi092.stdout: osd default data pool replay window = 5 2023-11-25T04:22:06.475 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-25T04:22:06.475 INFO:teuthology.orchestra.run.smithi092.stdout: mon allow pool delete = true 2023-11-25T04:22:06.475 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-25T04:22:06.475 INFO:teuthology.orchestra.run.smithi092.stdout: mon cluster log file level = debug 2023-11-25T04:22:06.475 INFO:teuthology.orchestra.run.smithi092.stdout: debug asserts on shutdown = true 2023-11-25T04:22:06.475 INFO:teuthology.orchestra.run.smithi092.stdout: mon health detail to clog = false 2023-11-25T04:22:06.475 INFO:teuthology.orchestra.run.smithi092.stdout: mon host = "172.21.15.67,[v2:172.21.15.67:3301,v1:172.21.15.67:6790],[v2:172.21.15.67:3302,v1:172.21.15.67:6791]" 2023-11-25T04:22:06.475 INFO:teuthology.orchestra.run.smithi092.stdout: fsid = a677f511-1143-4e76-8da6-5ddfadaff077 2023-11-25T04:22:06.475 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-25T04:22:06.475 INFO:teuthology.orchestra.run.smithi092.stdout:[osd] 2023-11-25T04:22:06.475 INFO:teuthology.orchestra.run.smithi092.stdout: osd journal size = 100 2023-11-25T04:22:06.476 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-25T04:22:06.476 INFO:teuthology.orchestra.run.smithi092.stdout: osd scrub load threshold = 5.0 2023-11-25T04:22:06.476 INFO:teuthology.orchestra.run.smithi092.stdout: osd scrub max interval = 600 2023-11-25T04:22:06.476 INFO:teuthology.orchestra.run.smithi092.stdout: osd mclock profile = high_recovery_ops 2023-11-25T04:22:06.476 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-25T04:22:06.476 INFO:teuthology.orchestra.run.smithi092.stdout: osd recover clone overlap = true 2023-11-25T04:22:06.476 INFO:teuthology.orchestra.run.smithi092.stdout: osd recovery max chunk = 1048576 2023-11-25T04:22:06.476 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-25T04:22:06.476 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug shutdown = true 2023-11-25T04:22:06.476 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug op order = true 2023-11-25T04:22:06.476 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug verify stray on activate = true 2023-11-25T04:22:06.476 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug trim objects = true 2023-11-25T04:22:06.477 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-25T04:22:06.477 INFO:teuthology.orchestra.run.smithi092.stdout: osd open classes on start = true 2023-11-25T04:22:06.477 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug pg log writeout = true 2023-11-25T04:22:06.477 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-25T04:22:06.477 INFO:teuthology.orchestra.run.smithi092.stdout: osd deep scrub update digest min age = 30 2023-11-25T04:22:06.477 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-25T04:22:06.477 INFO:teuthology.orchestra.run.smithi092.stdout: osd map max advance = 10 2023-11-25T04:22:06.477 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-25T04:22:06.477 INFO:teuthology.orchestra.run.smithi092.stdout: journal zero on create = true 2023-11-25T04:22:06.477 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-25T04:22:06.477 INFO:teuthology.orchestra.run.smithi092.stdout: filestore ondisk finisher threads = 3 2023-11-25T04:22:06.477 INFO:teuthology.orchestra.run.smithi092.stdout: filestore apply finisher threads = 3 2023-11-25T04:22:06.478 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-25T04:22:06.478 INFO:teuthology.orchestra.run.smithi092.stdout: bdev debug aio = true 2023-11-25T04:22:06.478 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug misdirected ops = true 2023-11-25T04:22:06.478 INFO:teuthology.orchestra.run.smithi092.stdout: bdev async discard = True 2023-11-25T04:22:06.478 INFO:teuthology.orchestra.run.smithi092.stdout: bdev enable discard = True 2023-11-25T04:22:06.478 INFO:teuthology.orchestra.run.smithi092.stdout: bluestore block size = 96636764160 2023-11-25T04:22:06.478 INFO:teuthology.orchestra.run.smithi092.stdout: bluestore fsck on mount = True 2023-11-25T04:22:06.478 INFO:teuthology.orchestra.run.smithi092.stdout: debug bluefs = 1/20 2023-11-25T04:22:06.478 INFO:teuthology.orchestra.run.smithi092.stdout: debug bluestore = 1/20 2023-11-25T04:22:06.478 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2023-11-25T04:22:06.478 INFO:teuthology.orchestra.run.smithi092.stdout: debug osd = 20 2023-11-25T04:22:06.479 INFO:teuthology.orchestra.run.smithi092.stdout: debug rocksdb = 4/10 2023-11-25T04:22:06.479 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd backfillfull_ratio = 0.85 2023-11-25T04:22:06.479 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd full ratio = 0.9 2023-11-25T04:22:06.479 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd nearfull ratio = 0.8 2023-11-25T04:22:06.479 INFO:teuthology.orchestra.run.smithi092.stdout: osd blocked scrub grace period = 3600 2023-11-25T04:22:06.479 INFO:teuthology.orchestra.run.smithi092.stdout: osd failsafe full ratio = 0.95 2023-11-25T04:22:06.479 INFO:teuthology.orchestra.run.smithi092.stdout: osd max markdown count = 1000 2023-11-25T04:22:06.479 INFO:teuthology.orchestra.run.smithi092.stdout: osd memory target = 1610612736 2023-11-25T04:22:06.479 INFO:teuthology.orchestra.run.smithi092.stdout: osd objectstore = bluestore 2023-11-25T04:22:06.479 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-25T04:22:06.480 INFO:teuthology.orchestra.run.smithi092.stdout:[mgr] 2023-11-25T04:22:06.480 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2023-11-25T04:22:06.480 INFO:teuthology.orchestra.run.smithi092.stdout: debug mgr = 20 2023-11-25T04:22:06.481 INFO:teuthology.orchestra.run.smithi092.stdout: debug mon = 20 2023-11-25T04:22:06.481 INFO:teuthology.orchestra.run.smithi092.stdout: debug auth = 20 2023-11-25T04:22:06.481 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min pgs per osd = 4 2023-11-25T04:22:06.481 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min bytes per osd = 10 2023-11-25T04:22:06.481 INFO:teuthology.orchestra.run.smithi092.stdout: mgr/telemetry/nag = false 2023-11-25T04:22:06.481 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-25T04:22:06.481 INFO:teuthology.orchestra.run.smithi092.stdout:[mon] 2023-11-25T04:22:06.481 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2023-11-25T04:22:06.481 INFO:teuthology.orchestra.run.smithi092.stdout: debug mon = 20 2023-11-25T04:22:06.481 INFO:teuthology.orchestra.run.smithi092.stdout: debug paxos = 20 2023-11-25T04:22:06.482 INFO:teuthology.orchestra.run.smithi092.stdout: debug auth = 20 2023-11-25T04:22:06.482 INFO:teuthology.orchestra.run.smithi092.stdout: mon data avail warn = 5 2023-11-25T04:22:06.482 INFO:teuthology.orchestra.run.smithi092.stdout: mon mgr mkfs grace = 240 2023-11-25T04:22:06.482 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min pgs per osd = 4 2023-11-25T04:22:06.482 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd reporter subtree level = osd 2023-11-25T04:22:06.482 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd prime pg temp = true 2023-11-25T04:22:06.482 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min bytes per osd = 10 2023-11-25T04:22:06.482 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-25T04:22:06.482 INFO:teuthology.orchestra.run.smithi092.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-25T04:22:06.482 INFO:teuthology.orchestra.run.smithi092.stdout: auth mon ticket ttl = 660 # 11m 2023-11-25T04:22:06.482 INFO:teuthology.orchestra.run.smithi092.stdout: auth service ticket ttl = 240 # 4m 2023-11-25T04:22:06.483 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-25T04:22:06.483 INFO:teuthology.orchestra.run.smithi092.stdout: # don't complain about insecure global_id in the test suite 2023-11-25T04:22:06.483 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-25T04:22:06.483 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-25T04:22:06.483 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-25T04:22:06.483 INFO:teuthology.orchestra.run.smithi092.stdout: # 1m isn't quite enough 2023-11-25T04:22:06.483 INFO:teuthology.orchestra.run.smithi092.stdout: mon_down_mkfs_grace = 2m 2023-11-25T04:22:06.483 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-25T04:22:06.483 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_filestore_osds = false 2023-11-25T04:22:06.483 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-11-25T04:22:06.483 INFO:teuthology.orchestra.run.smithi092.stdout:[client] 2023-11-25T04:22:06.483 INFO:teuthology.orchestra.run.smithi092.stdout: rgw cache enabled = true 2023-11-25T04:22:06.483 INFO:teuthology.orchestra.run.smithi092.stdout: rgw enable ops log = true 2023-11-25T04:22:06.484 INFO:teuthology.orchestra.run.smithi092.stdout: rgw enable usage log = true 2023-11-25T04:22:06.484 INFO:teuthology.orchestra.run.smithi092.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-25T04:22:06.484 INFO:teuthology.orchestra.run.smithi092.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-25T04:22:06.484 INFO:teuthology.orchestra.run.smithi092.stdout:[mon.a] 2023-11-25T04:22:06.484 INFO:teuthology.orchestra.run.smithi092.stdout:[mon.b] 2023-11-25T04:22:06.484 INFO:teuthology.orchestra.run.smithi092.stdout:[mon.c] 2023-11-25T04:22:06.484 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.a] 2023-11-25T04:22:06.484 INFO:teuthology.orchestra.run.smithi067.stdout:[global] 2023-11-25T04:22:06.484 INFO:teuthology.orchestra.run.smithi067.stdout: chdir = "" 2023-11-25T04:22:06.484 INFO:teuthology.orchestra.run.smithi067.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-25T04:22:06.484 INFO:teuthology.orchestra.run.smithi067.stdout: auth supported = cephx 2023-11-25T04:22:06.484 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:22:06.485 INFO:teuthology.orchestra.run.smithi067.stdout: filestore xattr use omap = true 2023-11-25T04:22:06.485 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:22:06.485 INFO:teuthology.orchestra.run.smithi067.stdout: mon clock drift allowed = 1.000 2023-11-25T04:22:06.485 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:22:06.485 INFO:teuthology.orchestra.run.smithi067.stdout: osd crush chooseleaf type = 0 2023-11-25T04:22:06.485 INFO:teuthology.orchestra.run.smithi067.stdout: auth debug = true 2023-11-25T04:22:06.486 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:22:06.486 INFO:teuthology.orchestra.run.smithi067.stdout: ms die on old message = true 2023-11-25T04:22:06.486 INFO:teuthology.orchestra.run.smithi067.stdout: ms die on bug = true 2023-11-25T04:22:06.486 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:22:06.486 INFO:teuthology.orchestra.run.smithi067.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-25T04:22:06.486 INFO:teuthology.orchestra.run.smithi067.stdout: mon pg warn max object skew = 0 2023-11-25T04:22:06.486 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:22:06.486 INFO:teuthology.orchestra.run.smithi067.stdout: # disable pg_autoscaler by default for new pools 2023-11-25T04:22:06.486 INFO:teuthology.orchestra.run.smithi067.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-25T04:22:06.487 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:22:06.487 INFO:teuthology.orchestra.run.smithi067.stdout: osd pool default size = 2 2023-11-25T04:22:06.487 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:22:06.487 INFO:teuthology.orchestra.run.smithi067.stdout: mon osd allow primary affinity = true 2023-11-25T04:22:06.487 INFO:teuthology.orchestra.run.smithi067.stdout: mon osd allow pg remap = true 2023-11-25T04:22:06.487 INFO:teuthology.orchestra.run.smithi067.stdout: mon warn on legacy crush tunables = false 2023-11-25T04:22:06.487 INFO:teuthology.orchestra.run.smithi067.stdout: mon warn on crush straw calc version zero = false 2023-11-25T04:22:06.487 INFO:teuthology.orchestra.run.smithi067.stdout: mon warn on no sortbitwise = false 2023-11-25T04:22:06.487 INFO:teuthology.orchestra.run.smithi067.stdout: mon warn on osd down out interval zero = false 2023-11-25T04:22:06.487 INFO:teuthology.orchestra.run.smithi067.stdout: mon warn on too few osds = false 2023-11-25T04:22:06.487 INFO:teuthology.orchestra.run.smithi067.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-25T04:22:06.487 INFO:teuthology.orchestra.run.smithi067.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-25T04:22:06.487 INFO:teuthology.orchestra.run.smithi067.stdout: mon_allow_pool_size_one = true 2023-11-25T04:22:06.488 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:22:06.488 INFO:teuthology.orchestra.run.smithi067.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-25T04:22:06.488 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:22:06.488 INFO:teuthology.orchestra.run.smithi067.stdout: osd default data pool replay window = 5 2023-11-25T04:22:06.488 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:22:06.488 INFO:teuthology.orchestra.run.smithi067.stdout: mon allow pool delete = true 2023-11-25T04:22:06.488 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:22:06.488 INFO:teuthology.orchestra.run.smithi067.stdout: mon cluster log file level = debug 2023-11-25T04:22:06.488 INFO:teuthology.orchestra.run.smithi067.stdout: debug asserts on shutdown = true 2023-11-25T04:22:06.488 INFO:teuthology.orchestra.run.smithi067.stdout: mon health detail to clog = false 2023-11-25T04:22:06.488 INFO:teuthology.orchestra.run.smithi067.stdout: mon host = "172.21.15.67,[v2:172.21.15.67:3301,v1:172.21.15.67:6790],[v2:172.21.15.67:3302,v1:172.21.15.67:6791]" 2023-11-25T04:22:06.488 INFO:teuthology.orchestra.run.smithi067.stdout: fsid = a677f511-1143-4e76-8da6-5ddfadaff077 2023-11-25T04:22:06.488 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:22:06.489 INFO:teuthology.orchestra.run.smithi067.stdout:[osd] 2023-11-25T04:22:06.489 INFO:teuthology.orchestra.run.smithi067.stdout: osd journal size = 100 2023-11-25T04:22:06.489 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:22:06.489 INFO:teuthology.orchestra.run.smithi067.stdout: osd scrub load threshold = 5.0 2023-11-25T04:22:06.489 INFO:teuthology.orchestra.run.smithi067.stdout: osd scrub max interval = 600 2023-11-25T04:22:06.489 INFO:teuthology.orchestra.run.smithi067.stdout: osd mclock profile = high_recovery_ops 2023-11-25T04:22:06.489 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:22:06.489 INFO:teuthology.orchestra.run.smithi067.stdout: osd recover clone overlap = true 2023-11-25T04:22:06.489 INFO:teuthology.orchestra.run.smithi067.stdout: osd recovery max chunk = 1048576 2023-11-25T04:22:06.489 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:22:06.489 INFO:teuthology.orchestra.run.smithi067.stdout: osd debug shutdown = true 2023-11-25T04:22:06.489 INFO:teuthology.orchestra.run.smithi067.stdout: osd debug op order = true 2023-11-25T04:22:06.489 INFO:teuthology.orchestra.run.smithi067.stdout: osd debug verify stray on activate = true 2023-11-25T04:22:06.490 INFO:teuthology.orchestra.run.smithi067.stdout: osd debug trim objects = true 2023-11-25T04:22:06.490 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:22:06.490 INFO:teuthology.orchestra.run.smithi067.stdout: osd open classes on start = true 2023-11-25T04:22:06.490 INFO:teuthology.orchestra.run.smithi067.stdout: osd debug pg log writeout = true 2023-11-25T04:22:06.490 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:22:06.490 INFO:teuthology.orchestra.run.smithi067.stdout: osd deep scrub update digest min age = 30 2023-11-25T04:22:06.490 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:22:06.490 INFO:teuthology.orchestra.run.smithi067.stdout: osd map max advance = 10 2023-11-25T04:22:06.490 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:22:06.490 INFO:teuthology.orchestra.run.smithi067.stdout: journal zero on create = true 2023-11-25T04:22:06.490 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:22:06.490 INFO:teuthology.orchestra.run.smithi067.stdout: filestore ondisk finisher threads = 3 2023-11-25T04:22:06.490 INFO:teuthology.orchestra.run.smithi067.stdout: filestore apply finisher threads = 3 2023-11-25T04:22:06.491 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:22:06.491 INFO:teuthology.orchestra.run.smithi067.stdout: bdev debug aio = true 2023-11-25T04:22:06.491 INFO:teuthology.orchestra.run.smithi067.stdout: osd debug misdirected ops = true 2023-11-25T04:22:06.491 INFO:teuthology.orchestra.run.smithi067.stdout: bdev async discard = True 2023-11-25T04:22:06.491 INFO:teuthology.orchestra.run.smithi067.stdout: bdev enable discard = True 2023-11-25T04:22:06.491 INFO:teuthology.orchestra.run.smithi067.stdout: bluestore block size = 96636764160 2023-11-25T04:22:06.491 INFO:teuthology.orchestra.run.smithi067.stdout: bluestore fsck on mount = True 2023-11-25T04:22:06.491 INFO:teuthology.orchestra.run.smithi067.stdout: debug bluefs = 1/20 2023-11-25T04:22:06.491 INFO:teuthology.orchestra.run.smithi067.stdout: debug bluestore = 1/20 2023-11-25T04:22:06.491 INFO:teuthology.orchestra.run.smithi067.stdout: debug ms = 1 2023-11-25T04:22:06.491 INFO:teuthology.orchestra.run.smithi067.stdout: debug osd = 20 2023-11-25T04:22:06.491 INFO:teuthology.orchestra.run.smithi067.stdout: debug rocksdb = 4/10 2023-11-25T04:22:06.491 INFO:teuthology.orchestra.run.smithi067.stdout: mon osd backfillfull_ratio = 0.85 2023-11-25T04:22:06.491 INFO:teuthology.orchestra.run.smithi067.stdout: mon osd full ratio = 0.9 2023-11-25T04:22:06.492 INFO:teuthology.orchestra.run.smithi067.stdout: mon osd nearfull ratio = 0.8 2023-11-25T04:22:06.492 INFO:teuthology.orchestra.run.smithi067.stdout: osd blocked scrub grace period = 3600 2023-11-25T04:22:06.492 INFO:teuthology.orchestra.run.smithi067.stdout: osd failsafe full ratio = 0.95 2023-11-25T04:22:06.492 INFO:teuthology.orchestra.run.smithi067.stdout: osd max markdown count = 1000 2023-11-25T04:22:06.492 INFO:teuthology.orchestra.run.smithi067.stdout: osd memory target = 1610612736 2023-11-25T04:22:06.492 INFO:teuthology.orchestra.run.smithi067.stdout: osd objectstore = bluestore 2023-11-25T04:22:06.492 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:22:06.492 INFO:teuthology.orchestra.run.smithi067.stdout:[mgr] 2023-11-25T04:22:06.492 INFO:teuthology.orchestra.run.smithi067.stdout: debug ms = 1 2023-11-25T04:22:06.492 INFO:teuthology.orchestra.run.smithi067.stdout: debug mgr = 20 2023-11-25T04:22:06.492 INFO:teuthology.orchestra.run.smithi067.stdout: debug mon = 20 2023-11-25T04:22:06.492 INFO:teuthology.orchestra.run.smithi067.stdout: debug auth = 20 2023-11-25T04:22:06.492 INFO:teuthology.orchestra.run.smithi067.stdout: mon reweight min pgs per osd = 4 2023-11-25T04:22:06.493 INFO:teuthology.orchestra.run.smithi067.stdout: mon reweight min bytes per osd = 10 2023-11-25T04:22:06.493 INFO:teuthology.orchestra.run.smithi067.stdout: mgr/telemetry/nag = false 2023-11-25T04:22:06.493 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:22:06.493 INFO:teuthology.orchestra.run.smithi067.stdout:[mon] 2023-11-25T04:22:06.493 INFO:teuthology.orchestra.run.smithi067.stdout: debug ms = 1 2023-11-25T04:22:06.493 INFO:teuthology.orchestra.run.smithi067.stdout: debug mon = 20 2023-11-25T04:22:06.493 INFO:teuthology.orchestra.run.smithi067.stdout: debug paxos = 20 2023-11-25T04:22:06.494 INFO:teuthology.orchestra.run.smithi067.stdout: debug auth = 20 2023-11-25T04:22:06.494 INFO:teuthology.orchestra.run.smithi067.stdout: mon data avail warn = 5 2023-11-25T04:22:06.495 INFO:teuthology.orchestra.run.smithi067.stdout: mon mgr mkfs grace = 240 2023-11-25T04:22:06.495 INFO:teuthology.orchestra.run.smithi067.stdout: mon reweight min pgs per osd = 4 2023-11-25T04:22:06.495 INFO:teuthology.orchestra.run.smithi067.stdout: mon osd reporter subtree level = osd 2023-11-25T04:22:06.495 INFO:teuthology.orchestra.run.smithi067.stdout: mon osd prime pg temp = true 2023-11-25T04:22:06.495 INFO:teuthology.orchestra.run.smithi067.stdout: mon reweight min bytes per osd = 10 2023-11-25T04:22:06.495 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:22:06.495 INFO:teuthology.orchestra.run.smithi067.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-25T04:22:06.495 INFO:teuthology.orchestra.run.smithi067.stdout: auth mon ticket ttl = 660 # 11m 2023-11-25T04:22:06.495 INFO:teuthology.orchestra.run.smithi067.stdout: auth service ticket ttl = 240 # 4m 2023-11-25T04:22:06.495 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:22:06.495 INFO:teuthology.orchestra.run.smithi067.stdout: # don't complain about insecure global_id in the test suite 2023-11-25T04:22:06.495 INFO:teuthology.orchestra.run.smithi067.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-25T04:22:06.495 INFO:teuthology.orchestra.run.smithi067.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-25T04:22:06.496 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:22:06.496 INFO:teuthology.orchestra.run.smithi067.stdout: # 1m isn't quite enough 2023-11-25T04:22:06.496 INFO:teuthology.orchestra.run.smithi067.stdout: mon_down_mkfs_grace = 2m 2023-11-25T04:22:06.496 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:22:06.496 INFO:teuthology.orchestra.run.smithi067.stdout: mon_warn_on_filestore_osds = false 2023-11-25T04:22:06.496 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:22:06.496 INFO:teuthology.orchestra.run.smithi067.stdout:[client] 2023-11-25T04:22:06.496 INFO:teuthology.orchestra.run.smithi067.stdout: rgw cache enabled = true 2023-11-25T04:22:06.496 INFO:teuthology.orchestra.run.smithi067.stdout: rgw enable ops log = true 2023-11-25T04:22:06.496 INFO:teuthology.orchestra.run.smithi067.stdout: rgw enable usage log = true 2023-11-25T04:22:06.496 INFO:teuthology.orchestra.run.smithi067.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-25T04:22:06.497 INFO:teuthology.orchestra.run.smithi067.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-25T04:22:06.497 INFO:teuthology.orchestra.run.smithi067.stdout:[mon.a] 2023-11-25T04:22:06.497 INFO:teuthology.orchestra.run.smithi067.stdout:[mon.b] 2023-11-25T04:22:06.497 INFO:teuthology.orchestra.run.smithi067.stdout:[mon.c] 2023-11-25T04:22:06.497 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.a] 2023-11-25T04:22:06.497 INFO:teuthology.orchestra.run.smithi195.stdout:[global] 2023-11-25T04:22:06.497 INFO:teuthology.orchestra.run.smithi195.stdout: chdir = "" 2023-11-25T04:22:06.497 INFO:teuthology.orchestra.run.smithi195.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-25T04:22:06.497 INFO:teuthology.orchestra.run.smithi195.stdout: auth supported = cephx 2023-11-25T04:22:06.497 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-25T04:22:06.498 INFO:teuthology.orchestra.run.smithi195.stdout: filestore xattr use omap = true 2023-11-25T04:22:06.498 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-25T04:22:06.498 INFO:teuthology.orchestra.run.smithi195.stdout: mon clock drift allowed = 1.000 2023-11-25T04:22:06.498 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-25T04:22:06.498 INFO:teuthology.orchestra.run.smithi195.stdout: osd crush chooseleaf type = 0 2023-11-25T04:22:06.498 INFO:teuthology.orchestra.run.smithi195.stdout: auth debug = true 2023-11-25T04:22:06.498 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-25T04:22:06.498 INFO:teuthology.orchestra.run.smithi195.stdout: ms die on old message = true 2023-11-25T04:22:06.498 INFO:teuthology.orchestra.run.smithi195.stdout: ms die on bug = true 2023-11-25T04:22:06.498 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-25T04:22:06.498 INFO:teuthology.orchestra.run.smithi195.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-25T04:22:06.498 INFO:teuthology.orchestra.run.smithi195.stdout: mon pg warn max object skew = 0 2023-11-25T04:22:06.499 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-25T04:22:06.499 INFO:teuthology.orchestra.run.smithi195.stdout: # disable pg_autoscaler by default for new pools 2023-11-25T04:22:06.499 INFO:teuthology.orchestra.run.smithi195.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-25T04:22:06.499 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-25T04:22:06.499 INFO:teuthology.orchestra.run.smithi195.stdout: osd pool default size = 2 2023-11-25T04:22:06.499 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-25T04:22:06.499 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd allow primary affinity = true 2023-11-25T04:22:06.499 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd allow pg remap = true 2023-11-25T04:22:06.499 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on legacy crush tunables = false 2023-11-25T04:22:06.499 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on crush straw calc version zero = false 2023-11-25T04:22:06.499 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on no sortbitwise = false 2023-11-25T04:22:06.499 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on osd down out interval zero = false 2023-11-25T04:22:06.499 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on too few osds = false 2023-11-25T04:22:06.499 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-25T04:22:06.500 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-25T04:22:06.500 INFO:teuthology.orchestra.run.smithi195.stdout: mon_allow_pool_size_one = true 2023-11-25T04:22:06.500 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-25T04:22:06.500 INFO:teuthology.orchestra.run.smithi195.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-25T04:22:06.500 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-25T04:22:06.500 INFO:teuthology.orchestra.run.smithi195.stdout: osd default data pool replay window = 5 2023-11-25T04:22:06.500 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-25T04:22:06.500 INFO:teuthology.orchestra.run.smithi195.stdout: mon allow pool delete = true 2023-11-25T04:22:06.500 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-25T04:22:06.500 INFO:teuthology.orchestra.run.smithi195.stdout: mon cluster log file level = debug 2023-11-25T04:22:06.500 INFO:teuthology.orchestra.run.smithi195.stdout: debug asserts on shutdown = true 2023-11-25T04:22:06.500 INFO:teuthology.orchestra.run.smithi195.stdout: mon health detail to clog = false 2023-11-25T04:22:06.501 INFO:teuthology.orchestra.run.smithi195.stdout: mon host = "172.21.15.67,[v2:172.21.15.67:3301,v1:172.21.15.67:6790],[v2:172.21.15.67:3302,v1:172.21.15.67:6791]" 2023-11-25T04:22:06.501 INFO:teuthology.orchestra.run.smithi195.stdout: fsid = a677f511-1143-4e76-8da6-5ddfadaff077 2023-11-25T04:22:06.501 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-25T04:22:06.501 INFO:teuthology.orchestra.run.smithi195.stdout:[osd] 2023-11-25T04:22:06.501 INFO:teuthology.orchestra.run.smithi195.stdout: osd journal size = 100 2023-11-25T04:22:06.501 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-25T04:22:06.501 INFO:teuthology.orchestra.run.smithi195.stdout: osd scrub load threshold = 5.0 2023-11-25T04:22:06.501 INFO:teuthology.orchestra.run.smithi195.stdout: osd scrub max interval = 600 2023-11-25T04:22:06.501 INFO:teuthology.orchestra.run.smithi195.stdout: osd mclock profile = high_recovery_ops 2023-11-25T04:22:06.501 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-25T04:22:06.501 INFO:teuthology.orchestra.run.smithi195.stdout: osd recover clone overlap = true 2023-11-25T04:22:06.501 INFO:teuthology.orchestra.run.smithi195.stdout: osd recovery max chunk = 1048576 2023-11-25T04:22:06.501 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-25T04:22:06.502 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug shutdown = true 2023-11-25T04:22:06.502 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug op order = true 2023-11-25T04:22:06.502 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug verify stray on activate = true 2023-11-25T04:22:06.502 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug trim objects = true 2023-11-25T04:22:06.502 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-25T04:22:06.502 INFO:teuthology.orchestra.run.smithi195.stdout: osd open classes on start = true 2023-11-25T04:22:06.502 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug pg log writeout = true 2023-11-25T04:22:06.502 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-25T04:22:06.502 INFO:teuthology.orchestra.run.smithi195.stdout: osd deep scrub update digest min age = 30 2023-11-25T04:22:06.502 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-25T04:22:06.502 INFO:teuthology.orchestra.run.smithi195.stdout: osd map max advance = 10 2023-11-25T04:22:06.502 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-25T04:22:06.502 INFO:teuthology.orchestra.run.smithi195.stdout: journal zero on create = true 2023-11-25T04:22:06.503 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-25T04:22:06.503 INFO:teuthology.orchestra.run.smithi195.stdout: filestore ondisk finisher threads = 3 2023-11-25T04:22:06.503 INFO:teuthology.orchestra.run.smithi195.stdout: filestore apply finisher threads = 3 2023-11-25T04:22:06.503 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-25T04:22:06.503 INFO:teuthology.orchestra.run.smithi195.stdout: bdev debug aio = true 2023-11-25T04:22:06.503 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug misdirected ops = true 2023-11-25T04:22:06.503 INFO:teuthology.orchestra.run.smithi195.stdout: bdev async discard = True 2023-11-25T04:22:06.503 INFO:teuthology.orchestra.run.smithi195.stdout: bdev enable discard = True 2023-11-25T04:22:06.503 INFO:teuthology.orchestra.run.smithi195.stdout: bluestore block size = 96636764160 2023-11-25T04:22:06.503 INFO:teuthology.orchestra.run.smithi195.stdout: bluestore fsck on mount = True 2023-11-25T04:22:06.503 INFO:teuthology.orchestra.run.smithi195.stdout: debug bluefs = 1/20 2023-11-25T04:22:06.503 INFO:teuthology.orchestra.run.smithi195.stdout: debug bluestore = 1/20 2023-11-25T04:22:06.503 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2023-11-25T04:22:06.504 INFO:teuthology.orchestra.run.smithi195.stdout: debug osd = 20 2023-11-25T04:22:06.504 INFO:teuthology.orchestra.run.smithi195.stdout: debug rocksdb = 4/10 2023-11-25T04:22:06.504 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd backfillfull_ratio = 0.85 2023-11-25T04:22:06.504 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd full ratio = 0.9 2023-11-25T04:22:06.504 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd nearfull ratio = 0.8 2023-11-25T04:22:06.504 INFO:teuthology.orchestra.run.smithi195.stdout: osd blocked scrub grace period = 3600 2023-11-25T04:22:06.504 INFO:teuthology.orchestra.run.smithi195.stdout: osd failsafe full ratio = 0.95 2023-11-25T04:22:06.504 INFO:teuthology.orchestra.run.smithi195.stdout: osd max markdown count = 1000 2023-11-25T04:22:06.504 INFO:teuthology.orchestra.run.smithi195.stdout: osd memory target = 1610612736 2023-11-25T04:22:06.504 INFO:teuthology.orchestra.run.smithi195.stdout: osd objectstore = bluestore 2023-11-25T04:22:06.504 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-25T04:22:06.504 INFO:teuthology.orchestra.run.smithi195.stdout:[mgr] 2023-11-25T04:22:06.504 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2023-11-25T04:22:06.505 INFO:teuthology.orchestra.run.smithi195.stdout: debug mgr = 20 2023-11-25T04:22:06.505 INFO:teuthology.orchestra.run.smithi195.stdout: debug mon = 20 2023-11-25T04:22:06.505 INFO:teuthology.orchestra.run.smithi195.stdout: debug auth = 20 2023-11-25T04:22:06.505 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min pgs per osd = 4 2023-11-25T04:22:06.505 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min bytes per osd = 10 2023-11-25T04:22:06.505 INFO:teuthology.orchestra.run.smithi195.stdout: mgr/telemetry/nag = false 2023-11-25T04:22:06.505 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-25T04:22:06.505 INFO:teuthology.orchestra.run.smithi195.stdout:[mon] 2023-11-25T04:22:06.505 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2023-11-25T04:22:06.505 INFO:teuthology.orchestra.run.smithi195.stdout: debug mon = 20 2023-11-25T04:22:06.505 INFO:teuthology.orchestra.run.smithi195.stdout: debug paxos = 20 2023-11-25T04:22:06.505 INFO:teuthology.orchestra.run.smithi195.stdout: debug auth = 20 2023-11-25T04:22:06.505 INFO:teuthology.orchestra.run.smithi195.stdout: mon data avail warn = 5 2023-11-25T04:22:06.506 INFO:teuthology.orchestra.run.smithi195.stdout: mon mgr mkfs grace = 240 2023-11-25T04:22:06.506 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min pgs per osd = 4 2023-11-25T04:22:06.506 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd reporter subtree level = osd 2023-11-25T04:22:06.506 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd prime pg temp = true 2023-11-25T04:22:06.506 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min bytes per osd = 10 2023-11-25T04:22:06.506 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-25T04:22:06.506 INFO:teuthology.orchestra.run.smithi195.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-25T04:22:06.506 INFO:teuthology.orchestra.run.smithi195.stdout: auth mon ticket ttl = 660 # 11m 2023-11-25T04:22:06.506 INFO:teuthology.orchestra.run.smithi195.stdout: auth service ticket ttl = 240 # 4m 2023-11-25T04:22:06.506 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-25T04:22:06.506 INFO:teuthology.orchestra.run.smithi195.stdout: # don't complain about insecure global_id in the test suite 2023-11-25T04:22:06.506 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-25T04:22:06.506 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-25T04:22:06.507 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-25T04:22:06.507 INFO:teuthology.orchestra.run.smithi195.stdout: # 1m isn't quite enough 2023-11-25T04:22:06.507 INFO:teuthology.orchestra.run.smithi195.stdout: mon_down_mkfs_grace = 2m 2023-11-25T04:22:06.507 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-25T04:22:06.507 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_filestore_osds = false 2023-11-25T04:22:06.507 INFO:teuthology.orchestra.run.smithi195.stdout: 2023-11-25T04:22:06.507 INFO:teuthology.orchestra.run.smithi195.stdout:[client] 2023-11-25T04:22:06.507 INFO:teuthology.orchestra.run.smithi195.stdout: rgw cache enabled = true 2023-11-25T04:22:06.507 INFO:teuthology.orchestra.run.smithi195.stdout: rgw enable ops log = true 2023-11-25T04:22:06.507 INFO:teuthology.orchestra.run.smithi195.stdout: rgw enable usage log = true 2023-11-25T04:22:06.507 INFO:teuthology.orchestra.run.smithi195.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-25T04:22:06.507 INFO:teuthology.orchestra.run.smithi195.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-25T04:22:06.507 INFO:teuthology.orchestra.run.smithi195.stdout:[mon.a] 2023-11-25T04:22:06.508 INFO:teuthology.orchestra.run.smithi195.stdout:[mon.b] 2023-11-25T04:22:06.508 INFO:teuthology.orchestra.run.smithi195.stdout:[mon.c] 2023-11-25T04:22:06.508 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.a] 2023-11-25T04:22:06.510 INFO:tasks.ceph:Creating admin key on mon.a... 2023-11-25T04:22:06.510 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-11-25T04:22:06.611 INFO:tasks.ceph:Copying monmap to all nodes... 2023-11-25T04:22:06.611 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-11-25T04:22:06.611 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-11-25T04:22:06.626 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-11-25T04:22:06.626 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-11-25T04:22:06.681 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi067.front.sepia.ceph.com 2023-11-25T04:22:06.682 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-11-25T04:22:06.682 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-25T04:22:06.682 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-25T04:22:06.768 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-11-25T04:22:06.769 DEBUG:teuthology.orchestra.run.smithi067:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-25T04:22:06.824 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi076.front.sepia.ceph.com 2023-11-25T04:22:06.824 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-25T04:22:06.824 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-25T04:22:06.824 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-25T04:22:06.869 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-25T04:22:06.869 DEBUG:teuthology.orchestra.run.smithi076:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-25T04:22:06.926 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi092.front.sepia.ceph.com 2023-11-25T04:22:06.926 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-25T04:22:06.927 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-25T04:22:06.927 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-25T04:22:06.970 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-25T04:22:06.971 DEBUG:teuthology.orchestra.run.smithi092:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-25T04:22:07.026 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi195.front.sepia.ceph.com 2023-11-25T04:22:07.026 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-25T04:22:07.027 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-25T04:22:07.027 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-25T04:22:07.072 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-25T04:22:07.072 DEBUG:teuthology.orchestra.run.smithi195:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-25T04:22:07.128 INFO:tasks.ceph:Setting up mon nodes... 2023-11-25T04:22:07.128 INFO:tasks.ceph:Setting up mgr nodes... 2023-11-25T04:22:07.129 DEBUG:teuthology.orchestra.run.smithi067:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-11-25T04:22:07.194 INFO:teuthology.orchestra.run.smithi067.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-11-25T04:22:07.196 DEBUG:teuthology.orchestra.run.smithi067:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-11-25T04:22:07.259 INFO:teuthology.orchestra.run.smithi067.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-11-25T04:22:07.262 INFO:tasks.ceph:Setting up mds nodes... 2023-11-25T04:22:07.262 DEBUG:teuthology.orchestra.run.smithi067:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-11-25T04:22:07.324 INFO:teuthology.orchestra.run.smithi067.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-11-25T04:22:07.327 DEBUG:teuthology.orchestra.run.smithi067:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-11-25T04:22:07.357 INFO:tasks.ceph_client:Setting up client nodes... 2023-11-25T04:22:07.357 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-11-25T04:22:07.445 INFO:teuthology.orchestra.run.smithi067.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-11-25T04:22:07.464 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-11-25T04:22:07.464 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi076.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1'}, Remote(name='ubuntu@smithi092.front.sepia.ceph.com'): {'osd.1': '/dev/vg_nvme/lv_1'}, Remote(name='ubuntu@smithi195.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1'}} 2023-11-25T04:22:07.464 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-11-25T04:22:07.496 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1'} 2023-11-25T04:22:07.496 INFO:tasks.ceph:role: osd.0 2023-11-25T04:22:07.496 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi076.front.sepia.ceph.com 2023-11-25T04:22:07.496 DEBUG:teuthology.orchestra.run.smithi076:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-25T04:22:07.572 INFO:teuthology.orchestra.run.smithi076.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-25T04:22:07.572 INFO:teuthology.orchestra.run.smithi076.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-25T04:22:07.572 INFO:teuthology.orchestra.run.smithi076.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-25T04:22:07.572 INFO:teuthology.orchestra.run.smithi076.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-25T04:22:07.572 INFO:teuthology.orchestra.run.smithi076.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-25T04:22:07.572 INFO:teuthology.orchestra.run.smithi076.stdout: = sunit=0 swidth=0 blks 2023-11-25T04:22:07.572 INFO:teuthology.orchestra.run.smithi076.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-25T04:22:07.572 INFO:teuthology.orchestra.run.smithi076.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-25T04:22:07.572 INFO:teuthology.orchestra.run.smithi076.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-25T04:22:07.573 INFO:teuthology.orchestra.run.smithi076.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-25T04:22:08.074 INFO:teuthology.orchestra.run.smithi076.stdout:Discarding blocks...Done. 2023-11-25T04:22:08.074 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi076.front.sepia.ceph.com -o noatime 2023-11-25T04:22:08.074 DEBUG:teuthology.orchestra.run.smithi076:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-11-25T04:22:08.200 DEBUG:teuthology.orchestra.run.smithi076:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-11-25T04:22:08.236 DEBUG:teuthology.orchestra.run.smithi076:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-25T04:22:08.343 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-25T04:22:08.341+0000 7f921d71e500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-11-25T04:22:08.348 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-25T04:22:08.347+0000 7f921d71e500 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-11-25T04:22:08.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-25T04:22:08.347+0000 7f921d71e500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-25T04:22:08.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-25T04:22:08.347+0000 7f921d71e500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-25T04:22:08.349 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-25T04:22:08.347+0000 7f921d71e500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-25T04:22:08.350 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-25T04:22:08.348+0000 7f921d71e500 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-11-25T04:22:11.193 DEBUG:teuthology.orchestra.run.smithi076:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-11-25T04:22:11.225 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-11-25T04:22:11.256 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1'} 2023-11-25T04:22:11.256 INFO:tasks.ceph:role: osd.1 2023-11-25T04:22:11.257 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi092.front.sepia.ceph.com 2023-11-25T04:22:11.257 DEBUG:teuthology.orchestra.run.smithi092:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-25T04:22:11.333 INFO:teuthology.orchestra.run.smithi092.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-25T04:22:11.333 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-25T04:22:11.333 INFO:teuthology.orchestra.run.smithi092.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-25T04:22:11.333 INFO:teuthology.orchestra.run.smithi092.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-25T04:22:11.334 INFO:teuthology.orchestra.run.smithi092.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-25T04:22:11.334 INFO:teuthology.orchestra.run.smithi092.stdout: = sunit=0 swidth=0 blks 2023-11-25T04:22:11.334 INFO:teuthology.orchestra.run.smithi092.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-25T04:22:11.334 INFO:teuthology.orchestra.run.smithi092.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-25T04:22:11.334 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-25T04:22:11.334 INFO:teuthology.orchestra.run.smithi092.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-25T04:22:11.835 INFO:teuthology.orchestra.run.smithi092.stdout:Discarding blocks...Done. 2023-11-25T04:22:11.837 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi092.front.sepia.ceph.com -o noatime 2023-11-25T04:22:11.837 DEBUG:teuthology.orchestra.run.smithi092:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-1 2023-11-25T04:22:11.966 DEBUG:teuthology.orchestra.run.smithi092:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-11-25T04:22:12.002 DEBUG:teuthology.orchestra.run.smithi092:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-25T04:22:12.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-25T04:22:12.113+0000 7f24f178b500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-11-25T04:22:12.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-25T04:22:12.113+0000 7f24f178b500 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-11-25T04:22:12.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-25T04:22:12.113+0000 7f24f178b500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-25T04:22:12.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-25T04:22:12.113+0000 7f24f178b500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-25T04:22:12.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-25T04:22:12.113+0000 7f24f178b500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-25T04:22:12.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-11-25T04:22:12.118+0000 7f24f178b500 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-11-25T04:22:14.968 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-11-25T04:22:14.999 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-11-25T04:22:15.031 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1'} 2023-11-25T04:22:15.031 INFO:tasks.ceph:role: osd.2 2023-11-25T04:22:15.031 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi195.front.sepia.ceph.com 2023-11-25T04:22:15.031 DEBUG:teuthology.orchestra.run.smithi195:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-25T04:22:15.101 INFO:teuthology.orchestra.run.smithi195.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-25T04:22:15.101 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-25T04:22:15.101 INFO:teuthology.orchestra.run.smithi195.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-25T04:22:15.101 INFO:teuthology.orchestra.run.smithi195.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-25T04:22:15.102 INFO:teuthology.orchestra.run.smithi195.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-25T04:22:15.102 INFO:teuthology.orchestra.run.smithi195.stdout: = sunit=0 swidth=0 blks 2023-11-25T04:22:15.102 INFO:teuthology.orchestra.run.smithi195.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-25T04:22:15.102 INFO:teuthology.orchestra.run.smithi195.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-25T04:22:15.102 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-25T04:22:15.102 INFO:teuthology.orchestra.run.smithi195.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-25T04:22:15.135 INFO:teuthology.orchestra.run.smithi195.stdout:Discarding blocks...Done. 2023-11-25T04:22:15.137 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi195.front.sepia.ceph.com -o noatime 2023-11-25T04:22:15.137 DEBUG:teuthology.orchestra.run.smithi195:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-2 2023-11-25T04:22:15.211 DEBUG:teuthology.orchestra.run.smithi195:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-11-25T04:22:15.284 DEBUG:teuthology.orchestra.run.smithi195:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-25T04:22:15.383 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-25T04:22:15.381+0000 7fd7fe574500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-11-25T04:22:15.384 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-25T04:22:15.381+0000 7fd7fe574500 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-11-25T04:22:15.384 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-25T04:22:15.381+0000 7fd7fe574500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-25T04:22:15.384 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-25T04:22:15.381+0000 7fd7fe574500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-25T04:22:15.384 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-25T04:22:15.381+0000 7fd7fe574500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-25T04:22:15.384 INFO:teuthology.orchestra.run.smithi195.stderr:2023-11-25T04:22:15.382+0000 7fd7fe574500 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-11-25T04:22:18.238 DEBUG:teuthology.orchestra.run.smithi195:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-11-25T04:22:18.270 INFO:tasks.ceph:Reading keys from all nodes... 2023-11-25T04:22:18.271 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-11-25T04:22:18.271 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-11-25T04:22:18.308 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-11-25T04:22:18.308 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-11-25T04:22:18.383 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-11-25T04:22:18.383 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-11-25T04:22:18.458 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-25T04:22:18.458 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-11-25T04:22:18.490 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-25T04:22:18.490 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-11-25T04:22:18.521 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-25T04:22:18.521 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-11-25T04:22:18.553 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-11-25T04:22:18.553 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-11-25T04:22:18.571 INFO:tasks.ceph:Adding keys to all mons... 2023-11-25T04:22:18.571 DEBUG:teuthology.orchestra.run.smithi067:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-25T04:22:18.646 INFO:teuthology.orchestra.run.smithi067.stdout:[mgr.x] 2023-11-25T04:22:18.646 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQBvdmFlkB6GCxAAsB8HRBsgqj1EN1rNzr8g4Q== 2023-11-25T04:22:18.646 INFO:teuthology.orchestra.run.smithi067.stdout:[mgr.y] 2023-11-25T04:22:18.647 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQBvdmFlKkpuDxAAbjqon4ynzN54mWKn0VH2AA== 2023-11-25T04:22:18.647 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.a] 2023-11-25T04:22:18.647 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQBvdmFlxl5NExAAONL5LeT58ppf97Sr151Lwg== 2023-11-25T04:22:18.647 INFO:teuthology.orchestra.run.smithi067.stdout:[osd.0] 2023-11-25T04:22:18.647 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQBwdmFluDFxFBAA6CLdtFunIbGgDuEfPLfMWA== 2023-11-25T04:22:18.647 INFO:teuthology.orchestra.run.smithi067.stdout:[osd.1] 2023-11-25T04:22:18.647 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQB0dmFl7MfKBhAAWZQbpSQ0O8rmPTPMEGRj/A== 2023-11-25T04:22:18.647 INFO:teuthology.orchestra.run.smithi067.stdout:[osd.2] 2023-11-25T04:22:18.647 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQB3dmFly0/OFhAACqTdQBkhMusn/QsFVUApiw== 2023-11-25T04:22:18.647 INFO:teuthology.orchestra.run.smithi067.stdout:[client.0] 2023-11-25T04:22:18.647 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQBvdmFlqK6GGhAA0idNFuoKu5cQXcZsgoi1WQ== 2023-11-25T04:22:18.648 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-25T04:22:18.749 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-25T04:22:18.809 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-25T04:22:18.868 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-25T04:22:18.925 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-25T04:22:18.981 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-25T04:22:19.037 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-25T04:22:19.096 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-11-25T04:22:19.096 DEBUG:teuthology.orchestra.run.smithi067:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-11-25T04:22:19.127 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-25T04:22:19.230 DEBUG:teuthology.orchestra.run.smithi067:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-11-25T04:22:19.261 DEBUG:teuthology.orchestra.run.smithi067:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-11-25T04:22:19.333 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-25T04:22:19.434 DEBUG:teuthology.orchestra.run.smithi067:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-11-25T04:22:19.466 DEBUG:teuthology.orchestra.run.smithi067:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-11-25T04:22:19.536 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-25T04:22:19.641 DEBUG:teuthology.orchestra.run.smithi067:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-11-25T04:22:19.672 DEBUG:teuthology.orchestra.run.smithi067:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-25T04:22:19.734 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-11-25T04:22:19.734 INFO:tasks.ceph.mon.a:Restarting daemon 2023-11-25T04:22:19.734 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-11-25T04:22:19.776 INFO:tasks.ceph.mon.a:Started 2023-11-25T04:22:19.776 INFO:tasks.ceph.mon.b:Restarting daemon 2023-11-25T04:22:19.777 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-11-25T04:22:19.778 INFO:tasks.ceph.mon.b:Started 2023-11-25T04:22:19.778 INFO:tasks.ceph.mon.c:Restarting daemon 2023-11-25T04:22:19.779 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-11-25T04:22:19.780 INFO:tasks.ceph.mon.c:Started 2023-11-25T04:22:19.780 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-11-25T04:22:19.781 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-11-25T04:22:19.781 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-11-25T04:22:19.783 INFO:tasks.ceph.mgr.x:Started 2023-11-25T04:22:19.783 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-11-25T04:22:19.783 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-11-25T04:22:19.785 INFO:tasks.ceph.mgr.y:Started 2023-11-25T04:22:19.785 INFO:tasks.ceph:Setting crush tunables to default 2023-11-25T04:22:19.785 DEBUG:teuthology.orchestra.run.smithi067:> sudo ceph --cluster ceph osd crush tunables default 2023-11-25T04:22:25.259 INFO:teuthology.orchestra.run.smithi067.stderr:adjusted tunables profile to default 2023-11-25T04:22:25.272 INFO:tasks.ceph:check_enable_crimson: False 2023-11-25T04:22:25.272 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-11-25T04:22:25.272 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-25T04:22:25.272 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-11-25T04:22:25.304 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-11-25T04:22:25.304 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-11-25T04:22:25.335 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2023-11-25T04:22:25.335 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-11-25T04:22:25.371 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new bc397d35-53ea-430c-a96a-63a4160e64c3 0 2023-11-25T04:22:25.605 INFO:teuthology.orchestra.run.smithi195.stdout:0 2023-11-25T04:22:25.616 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new d4f1b88a-bbd4-4e58-9519-569489ced7b2 1 2023-11-25T04:22:25.692 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-11-25T04:22:25.690+0000 7ff421413040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-25T04:22:25.693 INFO:tasks.ceph.mgr.x.smithi067.stderr:2023-11-25T04:22:25.691+0000 7fdb2402e040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-25T04:22:25.748 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-11-25T04:22:25.746+0000 7ff421413040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-25T04:22:25.748 INFO:tasks.ceph.mgr.x.smithi067.stderr:2023-11-25T04:22:25.746+0000 7fdb2402e040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-25T04:22:25.799 INFO:teuthology.orchestra.run.smithi195.stdout:1 2023-11-25T04:22:25.810 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new 55c82801-e090-4c10-91e5-5fa9dbfba36d 2 2023-11-25T04:22:25.989 INFO:teuthology.orchestra.run.smithi195.stdout:2 2023-11-25T04:22:26.000 INFO:tasks.ceph.osd.0:Restarting daemon 2023-11-25T04:22:26.001 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-11-25T04:22:26.003 INFO:tasks.ceph.osd.0:Started 2023-11-25T04:22:26.003 INFO:tasks.ceph.osd.1:Restarting daemon 2023-11-25T04:22:26.003 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-11-25T04:22:26.006 INFO:tasks.ceph.osd.1:Started 2023-11-25T04:22:26.006 INFO:tasks.ceph.osd.2:Restarting daemon 2023-11-25T04:22:26.006 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-11-25T04:22:26.008 INFO:tasks.ceph.osd.2:Started 2023-11-25T04:22:26.008 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-25T04:22:26.189 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:22:26.189 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":5,"fsid":"a677f511-1143-4e76-8da6-5ddfadaff077","created":"2023-11-25T04:22:25.140474+0000","modified":"2023-11-25T04:22:25.985372+0000","last_up_change":"0.000000","last_in_change":"2023-11-25T04:22:25.985372+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"bc397d35-53ea-430c-a96a-63a4160e64c3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d4f1b88a-bbd4-4e58-9519-569489ced7b2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"55c82801-e090-4c10-91e5-5fa9dbfba36d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-25T04:22:26.200 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-11-25T04:22:26.200 INFO:tasks.ceph:Waiting for OSDs to come up 2023-11-25T04:22:26.347 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-11-25T04:22:26.345+0000 7ff421413040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-25T04:22:26.347 INFO:tasks.ceph.mgr.x.smithi067.stderr:2023-11-25T04:22:26.345+0000 7fdb2402e040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-25T04:22:26.403 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-11-25T04:22:26.401+0000 7ff421413040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-25T04:22:26.404 INFO:tasks.ceph.mgr.x.smithi067.stderr:2023-11-25T04:22:26.401+0000 7fdb2402e040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-25T04:22:26.451 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-11-25T04:22:26.449+0000 7ff421413040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-25T04:22:26.451 INFO:tasks.ceph.mgr.x.smithi067.stderr:2023-11-25T04:22:26.449+0000 7fdb2402e040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-25T04:22:26.503 DEBUG:teuthology.orchestra.run.smithi067:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-25T04:22:26.508 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-11-25T04:22:26.506+0000 7ff421413040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-25T04:22:26.508 INFO:tasks.ceph.mgr.x.smithi067.stderr:2023-11-25T04:22:26.506+0000 7fdb2402e040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-25T04:22:26.554 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-11-25T04:22:26.552+0000 7ff421413040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-25T04:22:26.555 INFO:tasks.ceph.mgr.x.smithi067.stderr:2023-11-25T04:22:26.553+0000 7fdb2402e040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-25T04:22:26.595 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:22:26.592+0000 7f3a7fe26500 -1 Falling back to public interface 2023-11-25T04:22:26.600 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:22:26.598+0000 7f0d8040e500 -1 Falling back to public interface 2023-11-25T04:22:26.606 INFO:tasks.ceph.osd.0.smithi076.stderr:2023-11-25T04:22:26.604+0000 7f393cafa500 -1 Falling back to public interface 2023-11-25T04:22:26.664 INFO:tasks.ceph.mgr.x.smithi067.stderr:2023-11-25T04:22:26.663+0000 7fdb2402e040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-25T04:22:26.665 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-11-25T04:22:26.663+0000 7ff421413040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-25T04:22:26.666 INFO:teuthology.misc.health.smithi067.stdout: 2023-11-25T04:22:26.666 INFO:teuthology.misc.health.smithi067.stdout:{"epoch":5,"fsid":"a677f511-1143-4e76-8da6-5ddfadaff077","created":"2023-11-25T04:22:25.140474+0000","modified":"2023-11-25T04:22:25.985372+0000","last_up_change":"0.000000","last_in_change":"2023-11-25T04:22:25.985372+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"bc397d35-53ea-430c-a96a-63a4160e64c3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d4f1b88a-bbd4-4e58-9519-569489ced7b2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"55c82801-e090-4c10-91e5-5fa9dbfba36d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-25T04:22:26.677 DEBUG:teuthology.misc:0 of 3 OSDs are up 2023-11-25T04:22:26.740 INFO:tasks.ceph.mgr.x.smithi067.stderr:2023-11-25T04:22:26.738+0000 7fdb2402e040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-25T04:22:26.741 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-11-25T04:22:26.738+0000 7ff421413040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-25T04:22:26.789 INFO:tasks.ceph.mgr.x.smithi067.stderr:2023-11-25T04:22:26.787+0000 7fdb2402e040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-25T04:22:26.789 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-11-25T04:22:26.787+0000 7ff421413040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-25T04:22:26.845 INFO:tasks.ceph.mgr.x.smithi067.stderr:2023-11-25T04:22:26.843+0000 7fdb2402e040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-25T04:22:26.845 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-11-25T04:22:26.843+0000 7ff421413040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-25T04:22:26.895 INFO:tasks.ceph.mgr.x.smithi067.stderr:2023-11-25T04:22:26.893+0000 7fdb2402e040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-25T04:22:26.895 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-11-25T04:22:26.893+0000 7ff421413040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-25T04:22:27.305 INFO:tasks.ceph.mgr.x.smithi067.stderr:2023-11-25T04:22:27.303+0000 7fdb2402e040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-25T04:22:27.305 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-11-25T04:22:27.303+0000 7ff421413040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-25T04:22:27.449 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-11-25T04:22:27.447+0000 7ff421413040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-25T04:22:27.450 INFO:tasks.ceph.mgr.x.smithi067.stderr:2023-11-25T04:22:27.447+0000 7fdb2402e040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-25T04:22:27.573 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-11-25T04:22:27.571+0000 7ff421413040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-25T04:22:27.573 INFO:tasks.ceph.mgr.x.smithi067.stderr:2023-11-25T04:22:27.571+0000 7fdb2402e040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-25T04:22:27.622 INFO:tasks.ceph.mgr.x.smithi067.stderr:2023-11-25T04:22:27.619+0000 7fdb2402e040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-25T04:22:27.622 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-11-25T04:22:27.620+0000 7ff421413040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-25T04:22:27.725 INFO:tasks.ceph.mgr.x.smithi067.stderr:2023-11-25T04:22:27.723+0000 7fdb2402e040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-25T04:22:27.725 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-11-25T04:22:27.723+0000 7ff421413040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-25T04:22:27.826 INFO:tasks.ceph.mgr.x.smithi067.stderr:2023-11-25T04:22:27.824+0000 7fdb2402e040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-25T04:22:27.826 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-11-25T04:22:27.824+0000 7ff421413040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-25T04:22:27.974 INFO:tasks.ceph.mgr.x.smithi067.stderr:2023-11-25T04:22:27.972+0000 7fdb2402e040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-25T04:22:27.975 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-11-25T04:22:27.973+0000 7ff421413040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-25T04:22:28.019 INFO:tasks.ceph.mgr.x.smithi067.stderr:2023-11-25T04:22:28.017+0000 7fdb2402e040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-25T04:22:28.020 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-11-25T04:22:28.018+0000 7ff421413040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-25T04:22:28.170 INFO:tasks.ceph.mgr.x.smithi067.stderr:2023-11-25T04:22:28.167+0000 7fdb2402e040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-25T04:22:28.170 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-11-25T04:22:28.168+0000 7ff421413040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-25T04:22:29.034 INFO:tasks.ceph.mgr.x.smithi067.stderr:2023-11-25T04:22:29.032+0000 7fdb2402e040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-25T04:22:29.034 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-11-25T04:22:29.032+0000 7ff421413040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-25T04:22:29.322 INFO:tasks.ceph.mgr.x.smithi067.stderr:2023-11-25T04:22:29.320+0000 7fdb2402e040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-25T04:22:29.322 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-11-25T04:22:29.320+0000 7ff421413040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-25T04:22:29.376 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-11-25T04:22:29.374+0000 7ff421413040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-25T04:22:29.377 INFO:tasks.ceph.mgr.x.smithi067.stderr:2023-11-25T04:22:29.374+0000 7fdb2402e040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-25T04:22:29.483 INFO:tasks.ceph.mgr.y.smithi067.stderr:2023-11-25T04:22:29.481+0000 7ff421413040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-25T04:22:29.483 INFO:tasks.ceph.mgr.x.smithi067.stderr:2023-11-25T04:22:29.481+0000 7fdb2402e040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-25T04:22:30.021 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:22:30.020+0000 7f0d8040e500 -1 osd.1 0 log_to_monitors true 2023-11-25T04:22:30.026 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:22:30.025+0000 7f3a7fe26500 -1 osd.2 0 log_to_monitors true 2023-11-25T04:22:30.032 INFO:tasks.ceph.osd.0.smithi076.stderr:2023-11-25T04:22:30.030+0000 7f393cafa500 -1 osd.0 0 log_to_monitors true 2023-11-25T04:22:30.501 INFO:tasks.ceph.mgr.x.smithi067.stderr:2023-11-25T04:22:30.498+0000 7fdac4b82640 -1 mgr.server handle_report got status from non-daemon mon.b 2023-11-25T04:22:30.501 INFO:tasks.ceph.mgr.x.smithi067.stderr:2023-11-25T04:22:30.499+0000 7fdac4b82640 -1 mgr.server handle_report got status from non-daemon mon.c 2023-11-25T04:22:32.676 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:22:32.675+0000 7f3a7af45640 -1 osd.2 0 waiting for initial osdmap 2023-11-25T04:22:32.696 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:22:32.694+0000 7f3a7357e640 -1 osd.2 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-25T04:22:32.792 INFO:tasks.ceph.osd.0.smithi076.stderr:2023-11-25T04:22:32.790+0000 7f3938267640 -1 osd.0 0 waiting for initial osdmap 2023-11-25T04:22:32.810 INFO:tasks.ceph.osd.0.smithi076.stderr:2023-11-25T04:22:32.808+0000 7f39308a0640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-25T04:22:32.857 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:22:32.856+0000 7f0d7b597640 -1 osd.1 0 waiting for initial osdmap 2023-11-25T04:22:32.867 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:22:32.866+0000 7f0d73bd0640 -1 osd.1 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-25T04:22:32.981 DEBUG:teuthology.orchestra.run.smithi067:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-25T04:22:33.272 INFO:teuthology.misc.health.smithi067.stdout: 2023-11-25T04:22:33.273 INFO:teuthology.misc.health.smithi067.stdout:{"epoch":7,"fsid":"a677f511-1143-4e76-8da6-5ddfadaff077","created":"2023-11-25T04:22:25.140474+0000","modified":"2023-11-25T04:22:31.883609+0000","last_up_change":"0.000000","last_in_change":"2023-11-25T04:22:25.985372+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"bc397d35-53ea-430c-a96a-63a4160e64c3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d4f1b88a-bbd4-4e58-9519-569489ced7b2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"55c82801-e090-4c10-91e5-5fa9dbfba36d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-25T04:22:33.284 DEBUG:teuthology.misc:0 of 3 OSDs are up 2023-11-25T04:22:39.588 DEBUG:teuthology.orchestra.run.smithi067:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-25T04:22:39.883 INFO:teuthology.misc.health.smithi067.stdout: 2023-11-25T04:22:39.883 INFO:teuthology.misc.health.smithi067.stdout:{"epoch":12,"fsid":"a677f511-1143-4e76-8da6-5ddfadaff077","created":"2023-11-25T04:22:25.140474+0000","modified":"2023-11-25T04:22:37.980112+0000","last_up_change":"2023-11-25T04:22:33.677691+0000","last_in_change":"2023-11-25T04:22:25.985372+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-25T04:22:35.548359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bc397d35-53ea-430c-a96a-63a4160e64c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":181847435},{"type":"v1","addr":"172.21.15.76:6801","nonce":181847435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":181847435},{"type":"v1","addr":"172.21.15.76:6803","nonce":181847435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":181847435},{"type":"v1","addr":"172.21.15.76:6807","nonce":181847435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":181847435},{"type":"v1","addr":"172.21.15.76:6805","nonce":181847435}]},"public_addr":"172.21.15.76:6801/181847435","cluster_addr":"172.21.15.76:6803/181847435","heartbeat_back_addr":"172.21.15.76:6807/181847435","heartbeat_front_addr":"172.21.15.76:6805/181847435","state":["exists","up"]},{"osd":1,"uuid":"d4f1b88a-bbd4-4e58-9519-569489ced7b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6801","nonce":3130474108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6803","nonce":3130474108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6807","nonce":3130474108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6805","nonce":3130474108}]},"public_addr":"172.21.15.92:6801/3130474108","cluster_addr":"172.21.15.92:6803/3130474108","heartbeat_back_addr":"172.21.15.92:6807/3130474108","heartbeat_front_addr":"172.21.15.92:6805/3130474108","state":["exists","up"]},{"osd":2,"uuid":"55c82801-e090-4c10-91e5-5fa9dbfba36d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6801","nonce":4099978417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6803","nonce":4099978417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6807","nonce":4099978417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6805","nonce":4099978417}]},"public_addr":"172.21.15.195:6801/4099978417","cluster_addr":"172.21.15.195:6803/4099978417","heartbeat_back_addr":"172.21.15.195:6807/4099978417","heartbeat_front_addr":"172.21.15.195:6805/4099978417","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-25T04:22:31.039867+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-25T04:22:31.021863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-25T04:22:31.009821+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-25T04:22:39.895 DEBUG:teuthology.misc:3 of 3 OSDs are up 2023-11-25T04:22:39.895 INFO:tasks.ceph:Creating RBD pool 2023-11-25T04:22:39.895 DEBUG:teuthology.orchestra.run.smithi067:> sudo ceph --cluster ceph osd pool create rbd 8 2023-11-25T04:22:40.992 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'rbd' created 2023-11-25T04:22:41.007 DEBUG:teuthology.orchestra.run.smithi067:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-11-25T04:22:41.998 INFO:teuthology.orchestra.run.smithi067.stderr:enabled application 'rbd' on pool 'rbd' 2023-11-25T04:22:42.014 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-11-25T04:22:42.014 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-25T04:22:42.014 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-25T04:22:42.016 INFO:tasks.ceph.mds.a:Started 2023-11-25T04:22:42.017 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-11-25T04:22:42.017 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-25T04:22:42.102 INFO:tasks.ceph.mds.a.smithi067.stdout:starting mds.a at 2023-11-25T04:22:42.320 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:22:42.320 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":14,"fsid":"a677f511-1143-4e76-8da6-5ddfadaff077","created":"2023-11-25T04:22:25.140474+0000","modified":"2023-11-25T04:22:41.991334+0000","last_up_change":"2023-11-25T04:22:33.677691+0000","last_in_change":"2023-11-25T04:22:25.985372+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-25T04:22:35.548359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-25T04:22:40.218527+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bc397d35-53ea-430c-a96a-63a4160e64c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":181847435},{"type":"v1","addr":"172.21.15.76:6801","nonce":181847435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":181847435},{"type":"v1","addr":"172.21.15.76:6803","nonce":181847435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":181847435},{"type":"v1","addr":"172.21.15.76:6807","nonce":181847435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":181847435},{"type":"v1","addr":"172.21.15.76:6805","nonce":181847435}]},"public_addr":"172.21.15.76:6801/181847435","cluster_addr":"172.21.15.76:6803/181847435","heartbeat_back_addr":"172.21.15.76:6807/181847435","heartbeat_front_addr":"172.21.15.76:6805/181847435","state":["exists","up"]},{"osd":1,"uuid":"d4f1b88a-bbd4-4e58-9519-569489ced7b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6801","nonce":3130474108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6803","nonce":3130474108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6807","nonce":3130474108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6805","nonce":3130474108}]},"public_addr":"172.21.15.92:6801/3130474108","cluster_addr":"172.21.15.92:6803/3130474108","heartbeat_back_addr":"172.21.15.92:6807/3130474108","heartbeat_front_addr":"172.21.15.92:6805/3130474108","state":["exists","up"]},{"osd":2,"uuid":"55c82801-e090-4c10-91e5-5fa9dbfba36d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6801","nonce":4099978417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6803","nonce":4099978417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6807","nonce":4099978417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6805","nonce":4099978417}]},"public_addr":"172.21.15.195:6801/4099978417","cluster_addr":"172.21.15.195:6803/4099978417","heartbeat_back_addr":"172.21.15.195:6807/4099978417","heartbeat_front_addr":"172.21.15.195:6805/4099978417","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-25T04:22:31.039867+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-25T04:22:31.021863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-25T04:22:31.009821+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-25T04:22:42.332 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-25T04:22:35.548359+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-25T04:22:40.218527+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '14', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-25T04:22:42.332 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-25T04:22:42.629 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2023-11-25T04:22:42.641 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-25T04:22:42.933 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 8 2023-11-25T04:22:42.946 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-25T04:22:43.276 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:22:43.277 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4233,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6803/2443032509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2443032509},{"type":"v1","addr":"172.21.15.67:6803","nonce":2443032509}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-11-25T04:22:43.277 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 2 2023-11-25T04:22:43.289 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-25T04:22:43.584 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:22:43.584 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":15,"fsid":"a677f511-1143-4e76-8da6-5ddfadaff077","created":"2023-11-25T04:22:25.140474+0000","modified":"2023-11-25T04:22:42.995028+0000","last_up_change":"2023-11-25T04:22:33.677691+0000","last_in_change":"2023-11-25T04:22:25.985372+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-25T04:22:35.548359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-25T04:22:40.218527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bc397d35-53ea-430c-a96a-63a4160e64c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":181847435},{"type":"v1","addr":"172.21.15.76:6801","nonce":181847435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":181847435},{"type":"v1","addr":"172.21.15.76:6803","nonce":181847435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":181847435},{"type":"v1","addr":"172.21.15.76:6807","nonce":181847435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":181847435},{"type":"v1","addr":"172.21.15.76:6805","nonce":181847435}]},"public_addr":"172.21.15.76:6801/181847435","cluster_addr":"172.21.15.76:6803/181847435","heartbeat_back_addr":"172.21.15.76:6807/181847435","heartbeat_front_addr":"172.21.15.76:6805/181847435","state":["exists","up"]},{"osd":1,"uuid":"d4f1b88a-bbd4-4e58-9519-569489ced7b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6801","nonce":3130474108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6803","nonce":3130474108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6807","nonce":3130474108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6805","nonce":3130474108}]},"public_addr":"172.21.15.92:6801/3130474108","cluster_addr":"172.21.15.92:6803/3130474108","heartbeat_back_addr":"172.21.15.92:6807/3130474108","heartbeat_front_addr":"172.21.15.92:6805/3130474108","state":["exists","up"]},{"osd":2,"uuid":"55c82801-e090-4c10-91e5-5fa9dbfba36d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6801","nonce":4099978417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6803","nonce":4099978417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6807","nonce":4099978417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6805","nonce":4099978417}]},"public_addr":"172.21.15.195:6801/4099978417","cluster_addr":"172.21.15.195:6803/4099978417","heartbeat_back_addr":"172.21.15.195:6807/4099978417","heartbeat_front_addr":"172.21.15.195:6805/4099978417","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-25T04:22:31.039867+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-25T04:22:31.021863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-25T04:22:31.009821+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-25T04:22:43.597 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-25T04:22:35.548359+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-25T04:22:40.218527+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-25T04:22:43.597 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-25T04:22:43.890 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2023-11-25T04:22:43.903 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-25T04:22:44.197 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 8 2023-11-25T04:22:44.210 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-25T04:22:44.507 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:22:44.507 INFO:teuthology.orchestra.run.smithi067.stdout:[ 2023-11-25T04:22:44.507 INFO:teuthology.orchestra.run.smithi067.stdout: { 2023-11-25T04:22:44.507 INFO:teuthology.orchestra.run.smithi067.stdout: "poolnum": 1, 2023-11-25T04:22:44.507 INFO:teuthology.orchestra.run.smithi067.stdout: "poolname": ".mgr" 2023-11-25T04:22:44.507 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2023-11-25T04:22:44.508 INFO:teuthology.orchestra.run.smithi067.stdout: { 2023-11-25T04:22:44.508 INFO:teuthology.orchestra.run.smithi067.stdout: "poolnum": 2, 2023-11-25T04:22:44.508 INFO:teuthology.orchestra.run.smithi067.stdout: "poolname": "rbd" 2023-11-25T04:22:44.508 INFO:teuthology.orchestra.run.smithi067.stdout: } 2023-11-25T04:22:44.508 INFO:teuthology.orchestra.run.smithi067.stdout:] 2023-11-25T04:22:44.520 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-25T04:22:44.520 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-25T04:22:45.022 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_metadata' created 2023-11-25T04:22:45.035 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-25T04:22:46.022 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_data' created 2023-11-25T04:22:46.040 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-25T04:22:47.029 INFO:teuthology.orchestra.run.smithi067.stderr:new fs with metadata pool 3 and data pool 4 2023-11-25T04:22:47.048 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-25T04:22:47.352 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:22:47.352 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":18,"fsid":"a677f511-1143-4e76-8da6-5ddfadaff077","created":"2023-11-25T04:22:25.140474+0000","modified":"2023-11-25T04:22:46.384075+0000","last_up_change":"2023-11-25T04:22:33.677691+0000","last_in_change":"2023-11-25T04:22:25.985372+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-25T04:22:35.548359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-25T04:22:40.218527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-25T04:22:44.844906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-25T04:22:45.365536+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bc397d35-53ea-430c-a96a-63a4160e64c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":181847435},{"type":"v1","addr":"172.21.15.76:6801","nonce":181847435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":181847435},{"type":"v1","addr":"172.21.15.76:6803","nonce":181847435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":181847435},{"type":"v1","addr":"172.21.15.76:6807","nonce":181847435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":181847435},{"type":"v1","addr":"172.21.15.76:6805","nonce":181847435}]},"public_addr":"172.21.15.76:6801/181847435","cluster_addr":"172.21.15.76:6803/181847435","heartbeat_back_addr":"172.21.15.76:6807/181847435","heartbeat_front_addr":"172.21.15.76:6805/181847435","state":["exists","up"]},{"osd":1,"uuid":"d4f1b88a-bbd4-4e58-9519-569489ced7b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6801","nonce":3130474108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6803","nonce":3130474108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6807","nonce":3130474108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6805","nonce":3130474108}]},"public_addr":"172.21.15.92:6801/3130474108","cluster_addr":"172.21.15.92:6803/3130474108","heartbeat_back_addr":"172.21.15.92:6807/3130474108","heartbeat_front_addr":"172.21.15.92:6805/3130474108","state":["exists","up"]},{"osd":2,"uuid":"55c82801-e090-4c10-91e5-5fa9dbfba36d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6801","nonce":4099978417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6803","nonce":4099978417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6807","nonce":4099978417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6805","nonce":4099978417}]},"public_addr":"172.21.15.195:6801/4099978417","cluster_addr":"172.21.15.195:6803/4099978417","heartbeat_back_addr":"172.21.15.195:6807/4099978417","heartbeat_front_addr":"172.21.15.195:6805/4099978417","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-25T04:22:31.039867+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-25T04:22:31.021863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-25T04:22:31.009821+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-25T04:22:47.364 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-25T04:22:47.662 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:22:47.662 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":18,"fsid":"a677f511-1143-4e76-8da6-5ddfadaff077","created":"2023-11-25T04:22:25.140474+0000","modified":"2023-11-25T04:22:46.384075+0000","last_up_change":"2023-11-25T04:22:33.677691+0000","last_in_change":"2023-11-25T04:22:25.985372+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-25T04:22:35.548359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-25T04:22:40.218527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-25T04:22:44.844906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-25T04:22:45.365536+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bc397d35-53ea-430c-a96a-63a4160e64c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":181847435},{"type":"v1","addr":"172.21.15.76:6801","nonce":181847435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":181847435},{"type":"v1","addr":"172.21.15.76:6803","nonce":181847435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":181847435},{"type":"v1","addr":"172.21.15.76:6807","nonce":181847435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":181847435},{"type":"v1","addr":"172.21.15.76:6805","nonce":181847435}]},"public_addr":"172.21.15.76:6801/181847435","cluster_addr":"172.21.15.76:6803/181847435","heartbeat_back_addr":"172.21.15.76:6807/181847435","heartbeat_front_addr":"172.21.15.76:6805/181847435","state":["exists","up"]},{"osd":1,"uuid":"d4f1b88a-bbd4-4e58-9519-569489ced7b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6801","nonce":3130474108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6803","nonce":3130474108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6807","nonce":3130474108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6805","nonce":3130474108}]},"public_addr":"172.21.15.92:6801/3130474108","cluster_addr":"172.21.15.92:6803/3130474108","heartbeat_back_addr":"172.21.15.92:6807/3130474108","heartbeat_front_addr":"172.21.15.92:6805/3130474108","state":["exists","up"]},{"osd":2,"uuid":"55c82801-e090-4c10-91e5-5fa9dbfba36d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6801","nonce":4099978417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6803","nonce":4099978417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6807","nonce":4099978417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6805","nonce":4099978417}]},"public_addr":"172.21.15.195:6801/4099978417","cluster_addr":"172.21.15.195:6803/4099978417","heartbeat_back_addr":"172.21.15.195:6807/4099978417","heartbeat_front_addr":"172.21.15.195:6805/4099978417","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-25T04:22:31.039867+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-25T04:22:31.021863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-25T04:22:31.009821+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-25T04:22:47.674 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-25T04:22:48.060 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-25T04:22:48.061 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-25T04:22:49.064 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-25T04:22:49.064 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-25T04:22:49.398 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:22:49.398 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-25T04:22:46.384357+0000","modified":"2023-11-25T04:22:48.396173+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4233},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4233":{"gid":4233,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":3,"addr":"172.21.15.67:6803/2443032509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2443032509},{"type":"v1","addr":"172.21.15.67:6803","nonce":2443032509}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-25T04:22:49.398 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 5 2023-11-25T04:22:49.410 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-25T04:22:49.709 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:22:49.709 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":19,"fsid":"a677f511-1143-4e76-8da6-5ddfadaff077","created":"2023-11-25T04:22:25.140474+0000","modified":"2023-11-25T04:22:48.032728+0000","last_up_change":"2023-11-25T04:22:33.677691+0000","last_in_change":"2023-11-25T04:22:25.985372+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-25T04:22:35.548359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-25T04:22:40.218527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-25T04:22:44.844906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-25T04:22:45.365536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bc397d35-53ea-430c-a96a-63a4160e64c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":181847435},{"type":"v1","addr":"172.21.15.76:6801","nonce":181847435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":181847435},{"type":"v1","addr":"172.21.15.76:6803","nonce":181847435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":181847435},{"type":"v1","addr":"172.21.15.76:6807","nonce":181847435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":181847435},{"type":"v1","addr":"172.21.15.76:6805","nonce":181847435}]},"public_addr":"172.21.15.76:6801/181847435","cluster_addr":"172.21.15.76:6803/181847435","heartbeat_back_addr":"172.21.15.76:6807/181847435","heartbeat_front_addr":"172.21.15.76:6805/181847435","state":["exists","up"]},{"osd":1,"uuid":"d4f1b88a-bbd4-4e58-9519-569489ced7b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6801","nonce":3130474108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6803","nonce":3130474108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6807","nonce":3130474108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6805","nonce":3130474108}]},"public_addr":"172.21.15.92:6801/3130474108","cluster_addr":"172.21.15.92:6803/3130474108","heartbeat_back_addr":"172.21.15.92:6807/3130474108","heartbeat_front_addr":"172.21.15.92:6805/3130474108","state":["exists","up"]},{"osd":2,"uuid":"55c82801-e090-4c10-91e5-5fa9dbfba36d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6801","nonce":4099978417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6803","nonce":4099978417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6807","nonce":4099978417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6805","nonce":4099978417}]},"public_addr":"172.21.15.195:6801/4099978417","cluster_addr":"172.21.15.195:6803/4099978417","heartbeat_back_addr":"172.21.15.195:6807/4099978417","heartbeat_front_addr":"172.21.15.195:6805/4099978417","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-25T04:22:31.039867+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-25T04:22:31.021863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-25T04:22:31.009821+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-25T04:22:49.721 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-25T04:22:49.721 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-25T04:22:50.020 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:22:50.021 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2023-11-25T04:22:50.030 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-11-25T04:22:49.493367+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":247,"num_write_kb":3474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":195,"ondisk_log_size":195,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":99,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":99404,"kb_used_data":1696,"kb_used_omap":4,"kb_used_meta":97659,"kb_avail":283016116,"statfs":{"total":289910292480,"available":289808502784,"internally_reserved":0,"allocated":1736704,"data_stored":1354080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":100003151},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.473744"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048437+0000","last_change":"2023-11-25T04:22:47.039847+0000","last_active":"2023-11-25T04:22:48.048437+0000","last_peered":"2023-11-25T04:22:48.048437+0000","last_clean":"2023-11-25T04:22:48.048437+0000","last_became_active":"2023-11-25T04:22:47.039490+0000","last_became_peered":"2023-11-25T04:22:47.039490+0000","last_unstale":"2023-11-25T04:22:48.048437+0000","last_undegraded":"2023-11-25T04:22:48.048437+0000","last_fullsized":"2023-11-25T04:22:48.048437+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:25:06.132517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048105+0000","last_change":"2023-11-25T04:22:47.040836+0000","last_active":"2023-11-25T04:22:48.048105+0000","last_peered":"2023-11-25T04:22:48.048105+0000","last_clean":"2023-11-25T04:22:48.048105+0000","last_became_active":"2023-11-25T04:22:47.040511+0000","last_became_peered":"2023-11-25T04:22:47.040511+0000","last_unstale":"2023-11-25T04:22:48.048105+0000","last_undegraded":"2023-11-25T04:22:48.048105+0000","last_fullsized":"2023-11-25T04:22:48.048105+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:15:39.511043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048297+0000","last_change":"2023-11-25T04:22:47.043529+0000","last_active":"2023-11-25T04:22:48.048297+0000","last_peered":"2023-11-25T04:22:48.048297+0000","last_clean":"2023-11-25T04:22:48.048297+0000","last_became_active":"2023-11-25T04:22:47.042733+0000","last_became_peered":"2023-11-25T04:22:47.042733+0000","last_unstale":"2023-11-25T04:22:48.048297+0000","last_undegraded":"2023-11-25T04:22:48.048297+0000","last_fullsized":"2023-11-25T04:22:48.048297+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:47:34.764521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047862+0000","last_change":"2023-11-25T04:22:47.040314+0000","last_active":"2023-11-25T04:22:48.047862+0000","last_peered":"2023-11-25T04:22:48.047862+0000","last_clean":"2023-11-25T04:22:48.047862+0000","last_became_active":"2023-11-25T04:22:47.039185+0000","last_became_peered":"2023-11-25T04:22:47.039185+0000","last_unstale":"2023-11-25T04:22:48.047862+0000","last_undegraded":"2023-11-25T04:22:48.047862+0000","last_fullsized":"2023-11-25T04:22:48.047862+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:16:29.085050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047841+0000","last_change":"2023-11-25T04:22:47.039499+0000","last_active":"2023-11-25T04:22:48.047841+0000","last_peered":"2023-11-25T04:22:48.047841+0000","last_clean":"2023-11-25T04:22:48.047841+0000","last_became_active":"2023-11-25T04:22:47.039165+0000","last_became_peered":"2023-11-25T04:22:47.039165+0000","last_unstale":"2023-11-25T04:22:48.047841+0000","last_undegraded":"2023-11-25T04:22:48.047841+0000","last_fullsized":"2023-11-25T04:22:48.047841+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:57:28.855151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047565+0000","last_change":"2023-11-25T04:22:47.039992+0000","last_active":"2023-11-25T04:22:48.047565+0000","last_peered":"2023-11-25T04:22:48.047565+0000","last_clean":"2023-11-25T04:22:48.047565+0000","last_became_active":"2023-11-25T04:22:47.039405+0000","last_became_peered":"2023-11-25T04:22:47.039405+0000","last_unstale":"2023-11-25T04:22:48.047565+0000","last_undegraded":"2023-11-25T04:22:48.047565+0000","last_fullsized":"2023-11-25T04:22:48.047565+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:56:59.689578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047396+0000","last_change":"2023-11-25T04:22:47.037971+0000","last_active":"2023-11-25T04:22:48.047396+0000","last_peered":"2023-11-25T04:22:48.047396+0000","last_clean":"2023-11-25T04:22:48.047396+0000","last_became_active":"2023-11-25T04:22:47.036736+0000","last_became_peered":"2023-11-25T04:22:47.036736+0000","last_unstale":"2023-11-25T04:22:48.047396+0000","last_undegraded":"2023-11-25T04:22:48.047396+0000","last_fullsized":"2023-11-25T04:22:48.047396+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:02:07.088241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047307+0000","last_change":"2023-11-25T04:22:47.041728+0000","last_active":"2023-11-25T04:22:48.047307+0000","last_peered":"2023-11-25T04:22:48.047307+0000","last_clean":"2023-11-25T04:22:48.047307+0000","last_became_active":"2023-11-25T04:22:47.041017+0000","last_became_peered":"2023-11-25T04:22:47.041017+0000","last_unstale":"2023-11-25T04:22:48.047307+0000","last_undegraded":"2023-11-25T04:22:48.047307+0000","last_fullsized":"2023-11-25T04:22:48.047307+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:57:28.855151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048097+0000","last_change":"2023-11-25T04:22:47.038397+0000","last_active":"2023-11-25T04:22:48.048097+0000","last_peered":"2023-11-25T04:22:48.048097+0000","last_clean":"2023-11-25T04:22:48.048097+0000","last_became_active":"2023-11-25T04:22:47.037328+0000","last_became_peered":"2023-11-25T04:22:47.037328+0000","last_unstale":"2023-11-25T04:22:48.048097+0000","last_undegraded":"2023-11-25T04:22:48.048097+0000","last_fullsized":"2023-11-25T04:22:48.048097+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:56:59.689578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048223+0000","last_change":"2023-11-25T04:22:47.040182+0000","last_active":"2023-11-25T04:22:48.048223+0000","last_peered":"2023-11-25T04:22:48.048223+0000","last_clean":"2023-11-25T04:22:48.048223+0000","last_became_active":"2023-11-25T04:22:47.039892+0000","last_became_peered":"2023-11-25T04:22:47.039892+0000","last_unstale":"2023-11-25T04:22:48.048223+0000","last_undegraded":"2023-11-25T04:22:48.048223+0000","last_fullsized":"2023-11-25T04:22:48.048223+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:38:09.176582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048202+0000","last_change":"2023-11-25T04:22:47.039907+0000","last_active":"2023-11-25T04:22:48.048202+0000","last_peered":"2023-11-25T04:22:48.048202+0000","last_clean":"2023-11-25T04:22:48.048202+0000","last_became_active":"2023-11-25T04:22:47.039046+0000","last_became_peered":"2023-11-25T04:22:47.039046+0000","last_unstale":"2023-11-25T04:22:48.048202+0000","last_undegraded":"2023-11-25T04:22:48.048202+0000","last_fullsized":"2023-11-25T04:22:48.048202+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:01:51.418774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047609+0000","last_change":"2023-11-25T04:22:47.037618+0000","last_active":"2023-11-25T04:22:48.047609+0000","last_peered":"2023-11-25T04:22:48.047609+0000","last_clean":"2023-11-25T04:22:48.047609+0000","last_became_active":"2023-11-25T04:22:47.036806+0000","last_became_peered":"2023-11-25T04:22:47.036806+0000","last_unstale":"2023-11-25T04:22:48.047609+0000","last_undegraded":"2023-11-25T04:22:48.047609+0000","last_fullsized":"2023-11-25T04:22:48.047609+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:12:51.990448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047608+0000","last_change":"2023-11-25T04:22:47.040388+0000","last_active":"2023-11-25T04:22:48.047608+0000","last_peered":"2023-11-25T04:22:48.047608+0000","last_clean":"2023-11-25T04:22:48.047608+0000","last_became_active":"2023-11-25T04:22:47.040129+0000","last_became_peered":"2023-11-25T04:22:47.040129+0000","last_unstale":"2023-11-25T04:22:48.047608+0000","last_undegraded":"2023-11-25T04:22:48.047608+0000","last_fullsized":"2023-11-25T04:22:48.047608+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:31:54.857270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047643+0000","last_change":"2023-11-25T04:22:47.038887+0000","last_active":"2023-11-25T04:22:48.047643+0000","last_peered":"2023-11-25T04:22:48.047643+0000","last_clean":"2023-11-25T04:22:48.047643+0000","last_became_active":"2023-11-25T04:22:47.038014+0000","last_became_peered":"2023-11-25T04:22:47.038014+0000","last_unstale":"2023-11-25T04:22:48.047643+0000","last_undegraded":"2023-11-25T04:22:48.047643+0000","last_fullsized":"2023-11-25T04:22:48.047643+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:38:38.839941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047471+0000","last_change":"2023-11-25T04:22:47.035843+0000","last_active":"2023-11-25T04:22:48.047471+0000","last_peered":"2023-11-25T04:22:48.047471+0000","last_clean":"2023-11-25T04:22:48.047471+0000","last_became_active":"2023-11-25T04:22:47.035453+0000","last_became_peered":"2023-11-25T04:22:47.035453+0000","last_unstale":"2023-11-25T04:22:48.047471+0000","last_undegraded":"2023-11-25T04:22:48.047471+0000","last_fullsized":"2023-11-25T04:22:48.047471+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:45:40.065146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047308+0000","last_change":"2023-11-25T04:22:47.041778+0000","last_active":"2023-11-25T04:22:48.047308+0000","last_peered":"2023-11-25T04:22:48.047308+0000","last_clean":"2023-11-25T04:22:48.047308+0000","last_became_active":"2023-11-25T04:22:47.041115+0000","last_became_peered":"2023-11-25T04:22:47.041115+0000","last_unstale":"2023-11-25T04:22:48.047308+0000","last_undegraded":"2023-11-25T04:22:48.047308+0000","last_fullsized":"2023-11-25T04:22:48.047308+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:30:04.568790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048422+0000","last_change":"2023-11-25T04:22:47.039389+0000","last_active":"2023-11-25T04:22:48.048422+0000","last_peered":"2023-11-25T04:22:48.048422+0000","last_clean":"2023-11-25T04:22:48.048422+0000","last_became_active":"2023-11-25T04:22:47.039123+0000","last_became_peered":"2023-11-25T04:22:47.039123+0000","last_unstale":"2023-11-25T04:22:48.048422+0000","last_undegraded":"2023-11-25T04:22:48.048422+0000","last_fullsized":"2023-11-25T04:22:48.048422+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:09:11.570561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048403+0000","last_change":"2023-11-25T04:22:47.036946+0000","last_active":"2023-11-25T04:22:48.048403+0000","last_peered":"2023-11-25T04:22:48.048403+0000","last_clean":"2023-11-25T04:22:48.048403+0000","last_became_active":"2023-11-25T04:22:47.036411+0000","last_became_peered":"2023-11-25T04:22:47.036411+0000","last_unstale":"2023-11-25T04:22:48.048403+0000","last_undegraded":"2023-11-25T04:22:48.048403+0000","last_fullsized":"2023-11-25T04:22:48.048403+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:16:29.085050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047729+0000","last_change":"2023-11-25T04:22:47.038983+0000","last_active":"2023-11-25T04:22:48.047729+0000","last_peered":"2023-11-25T04:22:48.047729+0000","last_clean":"2023-11-25T04:22:48.047729+0000","last_became_active":"2023-11-25T04:22:47.038477+0000","last_became_peered":"2023-11-25T04:22:47.038477+0000","last_unstale":"2023-11-25T04:22:48.047729+0000","last_undegraded":"2023-11-25T04:22:48.047729+0000","last_fullsized":"2023-11-25T04:22:48.047729+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:30:04.568790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047891+0000","last_change":"2023-11-25T04:22:47.038155+0000","last_active":"2023-11-25T04:22:48.047891+0000","last_peered":"2023-11-25T04:22:48.047891+0000","last_clean":"2023-11-25T04:22:48.047891+0000","last_became_active":"2023-11-25T04:22:47.037740+0000","last_became_peered":"2023-11-25T04:22:47.037740+0000","last_unstale":"2023-11-25T04:22:48.047891+0000","last_undegraded":"2023-11-25T04:22:48.047891+0000","last_fullsized":"2023-11-25T04:22:48.047891+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:09:11.570561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048543+0000","last_change":"2023-11-25T04:22:47.040641+0000","last_active":"2023-11-25T04:22:48.048543+0000","last_peered":"2023-11-25T04:22:48.048543+0000","last_clean":"2023-11-25T04:22:48.048543+0000","last_became_active":"2023-11-25T04:22:47.039409+0000","last_became_peered":"2023-11-25T04:22:47.039409+0000","last_unstale":"2023-11-25T04:22:48.048543+0000","last_undegraded":"2023-11-25T04:22:48.048543+0000","last_fullsized":"2023-11-25T04:22:48.048543+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:23:15.914817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047384+0000","last_change":"2023-11-25T04:22:47.043456+0000","last_active":"2023-11-25T04:22:48.047384+0000","last_peered":"2023-11-25T04:22:48.047384+0000","last_clean":"2023-11-25T04:22:48.047384+0000","last_became_active":"2023-11-25T04:22:47.043098+0000","last_became_peered":"2023-11-25T04:22:47.043098+0000","last_unstale":"2023-11-25T04:22:48.047384+0000","last_undegraded":"2023-11-25T04:22:48.047384+0000","last_fullsized":"2023-11-25T04:22:48.047384+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:47:57.307757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047103+0000","last_change":"2023-11-25T04:22:47.035260+0000","last_active":"2023-11-25T04:22:48.047103+0000","last_peered":"2023-11-25T04:22:48.047103+0000","last_clean":"2023-11-25T04:22:48.047103+0000","last_became_active":"2023-11-25T04:22:47.034702+0000","last_became_peered":"2023-11-25T04:22:47.034702+0000","last_unstale":"2023-11-25T04:22:48.047103+0000","last_undegraded":"2023-11-25T04:22:48.047103+0000","last_fullsized":"2023-11-25T04:22:48.047103+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:49:02.500022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047259+0000","last_change":"2023-11-25T04:22:47.036805+0000","last_active":"2023-11-25T04:22:48.047259+0000","last_peered":"2023-11-25T04:22:48.047259+0000","last_clean":"2023-11-25T04:22:48.047259+0000","last_became_active":"2023-11-25T04:22:47.036135+0000","last_became_peered":"2023-11-25T04:22:47.036135+0000","last_unstale":"2023-11-25T04:22:48.047259+0000","last_undegraded":"2023-11-25T04:22:48.047259+0000","last_fullsized":"2023-11-25T04:22:48.047259+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:25:21.730992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048430+0000","last_change":"2023-11-25T04:22:47.036611+0000","last_active":"2023-11-25T04:22:48.048430+0000","last_peered":"2023-11-25T04:22:48.048430+0000","last_clean":"2023-11-25T04:22:48.048430+0000","last_became_active":"2023-11-25T04:22:47.035762+0000","last_became_peered":"2023-11-25T04:22:47.035762+0000","last_unstale":"2023-11-25T04:22:48.048430+0000","last_undegraded":"2023-11-25T04:22:48.048430+0000","last_fullsized":"2023-11-25T04:22:48.048430+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:09:11.570561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048216+0000","last_change":"2023-11-25T04:22:47.042423+0000","last_active":"2023-11-25T04:22:48.048216+0000","last_peered":"2023-11-25T04:22:48.048216+0000","last_clean":"2023-11-25T04:22:48.048216+0000","last_became_active":"2023-11-25T04:22:47.042022+0000","last_became_peered":"2023-11-25T04:22:47.042022+0000","last_unstale":"2023-11-25T04:22:48.048216+0000","last_undegraded":"2023-11-25T04:22:48.048216+0000","last_fullsized":"2023-11-25T04:22:48.048216+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:38:09.176582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048337+0000","last_change":"2023-11-25T04:22:47.044040+0000","last_active":"2023-11-25T04:22:48.048337+0000","last_peered":"2023-11-25T04:22:48.048337+0000","last_clean":"2023-11-25T04:22:48.048337+0000","last_became_active":"2023-11-25T04:22:47.043329+0000","last_became_peered":"2023-11-25T04:22:47.043329+0000","last_unstale":"2023-11-25T04:22:48.048337+0000","last_undegraded":"2023-11-25T04:22:48.048337+0000","last_fullsized":"2023-11-25T04:22:48.048337+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:10:59.482775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047954+0000","last_change":"2023-11-25T04:22:47.036485+0000","last_active":"2023-11-25T04:22:48.047954+0000","last_peered":"2023-11-25T04:22:48.047954+0000","last_clean":"2023-11-25T04:22:48.047954+0000","last_became_active":"2023-11-25T04:22:47.035799+0000","last_became_peered":"2023-11-25T04:22:47.035799+0000","last_unstale":"2023-11-25T04:22:48.047954+0000","last_undegraded":"2023-11-25T04:22:48.047954+0000","last_fullsized":"2023-11-25T04:22:48.047954+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:45:40.065146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047835+0000","last_change":"2023-11-25T04:22:47.037668+0000","last_active":"2023-11-25T04:22:48.047835+0000","last_peered":"2023-11-25T04:22:48.047835+0000","last_clean":"2023-11-25T04:22:48.047835+0000","last_became_active":"2023-11-25T04:22:47.036823+0000","last_became_peered":"2023-11-25T04:22:47.036823+0000","last_unstale":"2023-11-25T04:22:48.047835+0000","last_undegraded":"2023-11-25T04:22:48.047835+0000","last_fullsized":"2023-11-25T04:22:48.047835+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:02:07.088241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047758+0000","last_change":"2023-11-25T04:22:47.038117+0000","last_active":"2023-11-25T04:22:48.047758+0000","last_peered":"2023-11-25T04:22:48.047758+0000","last_clean":"2023-11-25T04:22:48.047758+0000","last_became_active":"2023-11-25T04:22:47.037751+0000","last_became_peered":"2023-11-25T04:22:47.037751+0000","last_unstale":"2023-11-25T04:22:48.047758+0000","last_undegraded":"2023-11-25T04:22:48.047758+0000","last_fullsized":"2023-11-25T04:22:48.047758+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:30:04.568790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047184+0000","last_change":"2023-11-25T04:22:47.037786+0000","last_active":"2023-11-25T04:22:48.047184+0000","last_peered":"2023-11-25T04:22:48.047184+0000","last_clean":"2023-11-25T04:22:48.047184+0000","last_became_active":"2023-11-25T04:22:47.037206+0000","last_became_peered":"2023-11-25T04:22:47.037206+0000","last_unstale":"2023-11-25T04:22:48.047184+0000","last_undegraded":"2023-11-25T04:22:48.047184+0000","last_fullsized":"2023-11-25T04:22:48.047184+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:11:40.532140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047505+0000","last_change":"2023-11-25T04:22:47.041325+0000","last_active":"2023-11-25T04:22:48.047505+0000","last_peered":"2023-11-25T04:22:48.047505+0000","last_clean":"2023-11-25T04:22:48.047505+0000","last_became_active":"2023-11-25T04:22:47.040086+0000","last_became_peered":"2023-11-25T04:22:47.040086+0000","last_unstale":"2023-11-25T04:22:48.047505+0000","last_undegraded":"2023-11-25T04:22:48.047505+0000","last_fullsized":"2023-11-25T04:22:48.047505+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:15:39.511043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047822+0000","last_change":"2023-11-25T04:22:47.036689+0000","last_active":"2023-11-25T04:22:48.047822+0000","last_peered":"2023-11-25T04:22:48.047822+0000","last_clean":"2023-11-25T04:22:48.047822+0000","last_became_active":"2023-11-25T04:22:47.036107+0000","last_became_peered":"2023-11-25T04:22:47.036107+0000","last_unstale":"2023-11-25T04:22:48.047822+0000","last_undegraded":"2023-11-25T04:22:48.047822+0000","last_fullsized":"2023-11-25T04:22:48.047822+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:02:07.088241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"18'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048313+0000","last_change":"2023-11-25T04:22:46.040844+0000","last_active":"2023-11-25T04:22:48.048313+0000","last_peered":"2023-11-25T04:22:48.048313+0000","last_clean":"2023-11-25T04:22:48.048313+0000","last_became_active":"2023-11-25T04:22:46.040395+0000","last_became_peered":"2023-11-25T04:22:46.040395+0000","last_unstale":"2023-11-25T04:22:48.048313+0000","last_undegraded":"2023-11-25T04:22:48.048313+0000","last_fullsized":"2023-11-25T04:22:48.048313+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:11:39.522677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048200+0000","last_change":"2023-11-25T04:22:47.037888+0000","last_active":"2023-11-25T04:22:48.048200+0000","last_peered":"2023-11-25T04:22:48.048200+0000","last_clean":"2023-11-25T04:22:48.048200+0000","last_became_active":"2023-11-25T04:22:47.037158+0000","last_became_peered":"2023-11-25T04:22:47.037158+0000","last_unstale":"2023-11-25T04:22:48.048200+0000","last_undegraded":"2023-11-25T04:22:48.048200+0000","last_fullsized":"2023-11-25T04:22:48.048200+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:56:59.689578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047391+0000","last_change":"2023-11-25T04:22:46.036054+0000","last_active":"2023-11-25T04:22:48.047391+0000","last_peered":"2023-11-25T04:22:48.047391+0000","last_clean":"2023-11-25T04:22:48.047391+0000","last_became_active":"2023-11-25T04:22:46.035620+0000","last_became_peered":"2023-11-25T04:22:46.035620+0000","last_unstale":"2023-11-25T04:22:48.047391+0000","last_undegraded":"2023-11-25T04:22:48.047391+0000","last_fullsized":"2023-11-25T04:22:48.047391+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:19:48.371980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048391+0000","last_change":"2023-11-25T04:22:47.045025+0000","last_active":"2023-11-25T04:22:48.048391+0000","last_peered":"2023-11-25T04:22:48.048391+0000","last_clean":"2023-11-25T04:22:48.048391+0000","last_became_active":"2023-11-25T04:22:47.044697+0000","last_became_peered":"2023-11-25T04:22:47.044697+0000","last_unstale":"2023-11-25T04:22:48.048391+0000","last_undegraded":"2023-11-25T04:22:48.048391+0000","last_fullsized":"2023-11-25T04:22:48.048391+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:16:29.085050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047517+0000","last_change":"2023-11-25T04:22:46.037169+0000","last_active":"2023-11-25T04:22:48.047517+0000","last_peered":"2023-11-25T04:22:48.047517+0000","last_clean":"2023-11-25T04:22:48.047517+0000","last_became_active":"2023-11-25T04:22:46.036948+0000","last_became_peered":"2023-11-25T04:22:46.036948+0000","last_unstale":"2023-11-25T04:22:48.047517+0000","last_undegraded":"2023-11-25T04:22:48.047517+0000","last_fullsized":"2023-11-25T04:22:48.047517+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:45:14.670326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047715+0000","last_change":"2023-11-25T04:22:47.044860+0000","last_active":"2023-11-25T04:22:48.047715+0000","last_peered":"2023-11-25T04:22:48.047715+0000","last_clean":"2023-11-25T04:22:48.047715+0000","last_became_active":"2023-11-25T04:22:47.044551+0000","last_became_peered":"2023-11-25T04:22:47.044551+0000","last_unstale":"2023-11-25T04:22:48.047715+0000","last_undegraded":"2023-11-25T04:22:48.047715+0000","last_fullsized":"2023-11-25T04:22:48.047715+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:09:55.357642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.10","version":"18'8","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047593+0000","last_change":"2023-11-25T04:22:46.040356+0000","last_active":"2023-11-25T04:22:48.047593+0000","last_peered":"2023-11-25T04:22:48.047593+0000","last_clean":"2023-11-25T04:22:48.047593+0000","last_became_active":"2023-11-25T04:22:46.039802+0000","last_became_peered":"2023-11-25T04:22:46.039802+0000","last_unstale":"2023-11-25T04:22:48.047593+0000","last_undegraded":"2023-11-25T04:22:48.047593+0000","last_fullsized":"2023-11-25T04:22:48.047593+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:09:54.348179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"18'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048372+0000","last_change":"2023-11-25T04:22:46.040082+0000","last_active":"2023-11-25T04:22:48.048372+0000","last_peered":"2023-11-25T04:22:48.048372+0000","last_clean":"2023-11-25T04:22:48.048372+0000","last_became_active":"2023-11-25T04:22:46.039763+0000","last_became_peered":"2023-11-25T04:22:46.039763+0000","last_unstale":"2023-11-25T04:22:48.048372+0000","last_undegraded":"2023-11-25T04:22:48.048372+0000","last_fullsized":"2023-11-25T04:22:48.048372+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:10:20.178136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047265+0000","last_change":"2023-11-25T04:22:47.043644+0000","last_active":"2023-11-25T04:22:48.047265+0000","last_peered":"2023-11-25T04:22:48.047265+0000","last_clean":"2023-11-25T04:22:48.047265+0000","last_became_active":"2023-11-25T04:22:47.043221+0000","last_became_peered":"2023-11-25T04:22:47.043221+0000","last_unstale":"2023-11-25T04:22:48.047265+0000","last_undegraded":"2023-11-25T04:22:48.047265+0000","last_fullsized":"2023-11-25T04:22:48.047265+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:41:16.672268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"18'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048462+0000","last_change":"2023-11-25T04:22:46.035806+0000","last_active":"2023-11-25T04:22:48.048462+0000","last_peered":"2023-11-25T04:22:48.048462+0000","last_clean":"2023-11-25T04:22:48.048462+0000","last_became_active":"2023-11-25T04:22:46.035418+0000","last_became_peered":"2023-11-25T04:22:46.035418+0000","last_unstale":"2023-11-25T04:22:48.048462+0000","last_undegraded":"2023-11-25T04:22:48.048462+0000","last_fullsized":"2023-11-25T04:22:48.048462+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:31:53.068767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047508+0000","last_change":"2023-11-25T04:22:47.039685+0000","last_active":"2023-11-25T04:22:48.047508+0000","last_peered":"2023-11-25T04:22:48.047508+0000","last_clean":"2023-11-25T04:22:48.047508+0000","last_became_active":"2023-11-25T04:22:47.039296+0000","last_became_peered":"2023-11-25T04:22:47.039296+0000","last_unstale":"2023-11-25T04:22:48.047508+0000","last_undegraded":"2023-11-25T04:22:48.047508+0000","last_fullsized":"2023-11-25T04:22:48.047508+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:11:40.532140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"18'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048658+0000","last_change":"2023-11-25T04:22:46.039953+0000","last_active":"2023-11-25T04:22:48.048658+0000","last_peered":"2023-11-25T04:22:48.048658+0000","last_clean":"2023-11-25T04:22:48.048658+0000","last_became_active":"2023-11-25T04:22:46.039660+0000","last_became_peered":"2023-11-25T04:22:46.039660+0000","last_unstale":"2023-11-25T04:22:48.048658+0000","last_undegraded":"2023-11-25T04:22:48.048658+0000","last_fullsized":"2023-11-25T04:22:48.048658+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:59:44.482613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047887+0000","last_change":"2023-11-25T04:22:47.037912+0000","last_active":"2023-11-25T04:22:48.047887+0000","last_peered":"2023-11-25T04:22:48.047887+0000","last_clean":"2023-11-25T04:22:48.047887+0000","last_became_active":"2023-11-25T04:22:47.036552+0000","last_became_peered":"2023-11-25T04:22:47.036552+0000","last_unstale":"2023-11-25T04:22:48.047887+0000","last_undegraded":"2023-11-25T04:22:48.047887+0000","last_fullsized":"2023-11-25T04:22:48.047887+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:38:09.176582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048295+0000","last_change":"2023-11-25T04:22:46.039176+0000","last_active":"2023-11-25T04:22:48.048295+0000","last_peered":"2023-11-25T04:22:48.048295+0000","last_clean":"2023-11-25T04:22:48.048295+0000","last_became_active":"2023-11-25T04:22:46.038842+0000","last_became_peered":"2023-11-25T04:22:46.038842+0000","last_unstale":"2023-11-25T04:22:48.048295+0000","last_undegraded":"2023-11-25T04:22:48.048295+0000","last_fullsized":"2023-11-25T04:22:48.048295+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:47:56.298293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047733+0000","last_change":"2023-11-25T04:22:47.042155+0000","last_active":"2023-11-25T04:22:48.047733+0000","last_peered":"2023-11-25T04:22:48.047733+0000","last_clean":"2023-11-25T04:22:48.047733+0000","last_became_active":"2023-11-25T04:22:47.041433+0000","last_became_peered":"2023-11-25T04:22:47.041433+0000","last_unstale":"2023-11-25T04:22:48.047733+0000","last_undegraded":"2023-11-25T04:22:48.047733+0000","last_fullsized":"2023-11-25T04:22:48.047733+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:57:28.855151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"18'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048201+0000","last_change":"2023-11-25T04:22:46.037827+0000","last_active":"2023-11-25T04:22:48.048201+0000","last_peered":"2023-11-25T04:22:48.048201+0000","last_clean":"2023-11-25T04:22:48.048201+0000","last_became_active":"2023-11-25T04:22:46.036515+0000","last_became_peered":"2023-11-25T04:22:46.036515+0000","last_unstale":"2023-11-25T04:22:48.048201+0000","last_undegraded":"2023-11-25T04:22:48.048201+0000","last_fullsized":"2023-11-25T04:22:48.048201+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:10:20.178136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048154+0000","last_change":"2023-11-25T04:22:47.038856+0000","last_active":"2023-11-25T04:22:48.048154+0000","last_peered":"2023-11-25T04:22:48.048154+0000","last_clean":"2023-11-25T04:22:48.048154+0000","last_became_active":"2023-11-25T04:22:47.038190+0000","last_became_peered":"2023-11-25T04:22:47.038190+0000","last_unstale":"2023-11-25T04:22:48.048154+0000","last_undegraded":"2023-11-25T04:22:48.048154+0000","last_fullsized":"2023-11-25T04:22:48.048154+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:29:37.268624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047789+0000","last_change":"2023-11-25T04:22:46.038473+0000","last_active":"2023-11-25T04:22:48.047789+0000","last_peered":"2023-11-25T04:22:48.047789+0000","last_clean":"2023-11-25T04:22:48.047789+0000","last_became_active":"2023-11-25T04:22:46.038159+0000","last_became_peered":"2023-11-25T04:22:46.038159+0000","last_unstale":"2023-11-25T04:22:48.047789+0000","last_undegraded":"2023-11-25T04:22:48.047789+0000","last_fullsized":"2023-11-25T04:22:48.047789+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:31:53.847807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048568+0000","last_change":"2023-11-25T04:22:47.036222+0000","last_active":"2023-11-25T04:22:48.048568+0000","last_peered":"2023-11-25T04:22:48.048568+0000","last_clean":"2023-11-25T04:22:48.048568+0000","last_became_active":"2023-11-25T04:22:47.035871+0000","last_became_peered":"2023-11-25T04:22:47.035871+0000","last_unstale":"2023-11-25T04:22:48.048568+0000","last_undegraded":"2023-11-25T04:22:48.048568+0000","last_fullsized":"2023-11-25T04:22:48.048568+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:13:35.199538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047681+0000","last_change":"2023-11-25T04:22:46.035907+0000","last_active":"2023-11-25T04:22:48.047681+0000","last_peered":"2023-11-25T04:22:48.047681+0000","last_clean":"2023-11-25T04:22:48.047681+0000","last_became_active":"2023-11-25T04:22:46.035612+0000","last_became_peered":"2023-11-25T04:22:46.035612+0000","last_unstale":"2023-11-25T04:22:48.047681+0000","last_undegraded":"2023-11-25T04:22:48.047681+0000","last_fullsized":"2023-11-25T04:22:48.047681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:00:35.249875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048441+0000","last_change":"2023-11-25T04:22:47.038725+0000","last_active":"2023-11-25T04:22:48.048441+0000","last_peered":"2023-11-25T04:22:48.048441+0000","last_clean":"2023-11-25T04:22:48.048441+0000","last_became_active":"2023-11-25T04:22:47.038124+0000","last_became_peered":"2023-11-25T04:22:47.038124+0000","last_unstale":"2023-11-25T04:22:48.048441+0000","last_undegraded":"2023-11-25T04:22:48.048441+0000","last_fullsized":"2023-11-25T04:22:48.048441+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:00:26.187878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047437+0000","last_change":"2023-11-25T04:22:46.034509+0000","last_active":"2023-11-25T04:22:48.047437+0000","last_peered":"2023-11-25T04:22:48.047437+0000","last_clean":"2023-11-25T04:22:48.047437+0000","last_became_active":"2023-11-25T04:22:46.034158+0000","last_became_peered":"2023-11-25T04:22:46.034158+0000","last_unstale":"2023-11-25T04:22:48.047437+0000","last_undegraded":"2023-11-25T04:22:48.047437+0000","last_fullsized":"2023-11-25T04:22:48.047437+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:26:25.338117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048005+0000","last_change":"2023-11-25T04:22:47.038800+0000","last_active":"2023-11-25T04:22:48.048005+0000","last_peered":"2023-11-25T04:22:48.048005+0000","last_clean":"2023-11-25T04:22:48.048005+0000","last_became_active":"2023-11-25T04:22:47.038081+0000","last_became_peered":"2023-11-25T04:22:47.038081+0000","last_unstale":"2023-11-25T04:22:48.048005+0000","last_undegraded":"2023-11-25T04:22:48.048005+0000","last_fullsized":"2023-11-25T04:22:48.048005+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:45:40.065146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"12'157","reported_seq":214,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-25T04:22:47.033945+0000","last_change":"2023-11-25T04:22:36.931828+0000","last_active":"2023-11-25T04:22:47.033945+0000","last_peered":"2023-11-25T04:22:47.033945+0000","last_clean":"2023-11-25T04:22:47.033945+0000","last_became_active":"2023-11-25T04:22:36.931582+0000","last_became_peered":"2023-11-25T04:22:36.931582+0000","last_unstale":"2023-11-25T04:22:47.033945+0000","last_undegraded":"2023-11-25T04:22:47.033945+0000","last_fullsized":"2023-11-25T04:22:47.033945+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:35.919850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:35.919850+0000","last_clean_scrub_stamp":"2023-11-25T04:22:35.919850+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:59:59.198309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-25T04:22:47.031925+0000","last_change":"2023-11-25T04:22:41.999590+0000","last_active":"2023-11-25T04:22:47.031925+0000","last_peered":"2023-11-25T04:22:47.031925+0000","last_clean":"2023-11-25T04:22:47.031925+0000","last_became_active":"2023-11-25T04:22:41.999314+0000","last_became_peered":"2023-11-25T04:22:41.999314+0000","last_unstale":"2023-11-25T04:22:47.031925+0000","last_undegraded":"2023-11-25T04:22:47.031925+0000","last_fullsized":"2023-11-25T04:22:47.031925+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:20:12.785425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047840+0000","last_change":"2023-11-25T04:22:46.037044+0000","last_active":"2023-11-25T04:22:48.047840+0000","last_peered":"2023-11-25T04:22:48.047840+0000","last_clean":"2023-11-25T04:22:48.047840+0000","last_became_active":"2023-11-25T04:22:46.035916+0000","last_became_peered":"2023-11-25T04:22:46.035916+0000","last_unstale":"2023-11-25T04:22:48.047840+0000","last_undegraded":"2023-11-25T04:22:48.047840+0000","last_fullsized":"2023-11-25T04:22:48.047840+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:38:20.501102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048611+0000","last_change":"2023-11-25T04:22:47.036222+0000","last_active":"2023-11-25T04:22:48.048611+0000","last_peered":"2023-11-25T04:22:48.048611+0000","last_clean":"2023-11-25T04:22:48.048611+0000","last_became_active":"2023-11-25T04:22:47.035484+0000","last_became_peered":"2023-11-25T04:22:47.035484+0000","last_unstale":"2023-11-25T04:22:48.048611+0000","last_undegraded":"2023-11-25T04:22:48.048611+0000","last_fullsized":"2023-11-25T04:22:48.048611+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:18:56.845916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-25T04:22:47.032801+0000","last_change":"2023-11-25T04:22:42.000257+0000","last_active":"2023-11-25T04:22:47.032801+0000","last_peered":"2023-11-25T04:22:47.032801+0000","last_clean":"2023-11-25T04:22:47.032801+0000","last_became_active":"2023-11-25T04:22:41.999969+0000","last_became_peered":"2023-11-25T04:22:41.999969+0000","last_unstale":"2023-11-25T04:22:47.032801+0000","last_undegraded":"2023-11-25T04:22:47.032801+0000","last_fullsized":"2023-11-25T04:22:47.032801+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:21:34.424091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047780+0000","last_change":"2023-11-25T04:22:46.036121+0000","last_active":"2023-11-25T04:22:48.047780+0000","last_peered":"2023-11-25T04:22:48.047780+0000","last_clean":"2023-11-25T04:22:48.047780+0000","last_became_active":"2023-11-25T04:22:46.035654+0000","last_became_peered":"2023-11-25T04:22:46.035654+0000","last_unstale":"2023-11-25T04:22:48.047780+0000","last_undegraded":"2023-11-25T04:22:48.047780+0000","last_fullsized":"2023-11-25T04:22:48.047780+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:27:31.901618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048273+0000","last_change":"2023-11-25T04:22:47.039583+0000","last_active":"2023-11-25T04:22:48.048273+0000","last_peered":"2023-11-25T04:22:48.048273+0000","last_clean":"2023-11-25T04:22:48.048273+0000","last_became_active":"2023-11-25T04:22:47.039197+0000","last_became_peered":"2023-11-25T04:22:47.039197+0000","last_unstale":"2023-11-25T04:22:48.048273+0000","last_undegraded":"2023-11-25T04:22:48.048273+0000","last_fullsized":"2023-11-25T04:22:48.048273+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:37:30.959147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-25T04:22:47.029239+0000","last_change":"2023-11-25T04:22:42.000382+0000","last_active":"2023-11-25T04:22:47.029239+0000","last_peered":"2023-11-25T04:22:47.029239+0000","last_clean":"2023-11-25T04:22:47.029239+0000","last_became_active":"2023-11-25T04:22:42.000041+0000","last_became_peered":"2023-11-25T04:22:42.000041+0000","last_unstale":"2023-11-25T04:22:47.029239+0000","last_undegraded":"2023-11-25T04:22:47.029239+0000","last_fullsized":"2023-11-25T04:22:47.029239+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:08:19.655864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047444+0000","last_change":"2023-11-25T04:22:46.037431+0000","last_active":"2023-11-25T04:22:48.047444+0000","last_peered":"2023-11-25T04:22:48.047444+0000","last_clean":"2023-11-25T04:22:48.047444+0000","last_became_active":"2023-11-25T04:22:46.037141+0000","last_became_peered":"2023-11-25T04:22:46.037141+0000","last_unstale":"2023-11-25T04:22:48.047444+0000","last_undegraded":"2023-11-25T04:22:48.047444+0000","last_fullsized":"2023-11-25T04:22:48.047444+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:40:42.500663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048182+0000","last_change":"2023-11-25T04:22:47.038454+0000","last_active":"2023-11-25T04:22:48.048182+0000","last_peered":"2023-11-25T04:22:48.048182+0000","last_clean":"2023-11-25T04:22:48.048182+0000","last_became_active":"2023-11-25T04:22:47.038046+0000","last_became_peered":"2023-11-25T04:22:47.038046+0000","last_unstale":"2023-11-25T04:22:48.048182+0000","last_undegraded":"2023-11-25T04:22:48.048182+0000","last_fullsized":"2023-11-25T04:22:48.048182+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:38:38.839941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-25T04:22:47.031534+0000","last_change":"2023-11-25T04:22:41.999424+0000","last_active":"2023-11-25T04:22:47.031534+0000","last_peered":"2023-11-25T04:22:47.031534+0000","last_clean":"2023-11-25T04:22:47.031534+0000","last_became_active":"2023-11-25T04:22:41.999179+0000","last_became_peered":"2023-11-25T04:22:41.999179+0000","last_unstale":"2023-11-25T04:22:47.031534+0000","last_undegraded":"2023-11-25T04:22:47.031534+0000","last_fullsized":"2023-11-25T04:22:47.031534+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:12:04.291696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048523+0000","last_change":"2023-11-25T04:22:46.038758+0000","last_active":"2023-11-25T04:22:48.048523+0000","last_peered":"2023-11-25T04:22:48.048523+0000","last_clean":"2023-11-25T04:22:48.048523+0000","last_became_active":"2023-11-25T04:22:46.038470+0000","last_became_peered":"2023-11-25T04:22:46.038470+0000","last_unstale":"2023-11-25T04:22:48.048523+0000","last_undegraded":"2023-11-25T04:22:48.048523+0000","last_fullsized":"2023-11-25T04:22:48.048523+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:49:01.490558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048075+0000","last_change":"2023-11-25T04:22:47.037355+0000","last_active":"2023-11-25T04:22:48.048075+0000","last_peered":"2023-11-25T04:22:48.048075+0000","last_clean":"2023-11-25T04:22:48.048075+0000","last_became_active":"2023-11-25T04:22:47.036924+0000","last_became_peered":"2023-11-25T04:22:47.036924+0000","last_unstale":"2023-11-25T04:22:48.048075+0000","last_undegraded":"2023-11-25T04:22:48.048075+0000","last_fullsized":"2023-11-25T04:22:48.048075+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:15:39.511043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048677+0000","last_change":"2023-11-25T04:22:46.039025+0000","last_active":"2023-11-25T04:22:48.048677+0000","last_peered":"2023-11-25T04:22:48.048677+0000","last_clean":"2023-11-25T04:22:48.048677+0000","last_became_active":"2023-11-25T04:22:46.038762+0000","last_became_peered":"2023-11-25T04:22:46.038762+0000","last_unstale":"2023-11-25T04:22:48.048677+0000","last_undegraded":"2023-11-25T04:22:48.048677+0000","last_fullsized":"2023-11-25T04:22:48.048677+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:44:47.544509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-25T04:22:47.029103+0000","last_change":"2023-11-25T04:22:42.000204+0000","last_active":"2023-11-25T04:22:47.029103+0000","last_peered":"2023-11-25T04:22:47.029103+0000","last_clean":"2023-11-25T04:22:47.029103+0000","last_became_active":"2023-11-25T04:22:41.999915+0000","last_became_peered":"2023-11-25T04:22:41.999915+0000","last_unstale":"2023-11-25T04:22:47.029103+0000","last_undegraded":"2023-11-25T04:22:47.029103+0000","last_fullsized":"2023-11-25T04:22:47.029103+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:07:32.457905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047409+0000","last_change":"2023-11-25T04:22:47.038119+0000","last_active":"2023-11-25T04:22:48.047409+0000","last_peered":"2023-11-25T04:22:48.047409+0000","last_clean":"2023-11-25T04:22:48.047409+0000","last_became_active":"2023-11-25T04:22:47.037732+0000","last_became_peered":"2023-11-25T04:22:47.037732+0000","last_unstale":"2023-11-25T04:22:48.047409+0000","last_undegraded":"2023-11-25T04:22:48.047409+0000","last_fullsized":"2023-11-25T04:22:48.047409+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:49:02.500022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048004+0000","last_change":"2023-11-25T04:22:46.035885+0000","last_active":"2023-11-25T04:22:48.048004+0000","last_peered":"2023-11-25T04:22:48.048004+0000","last_clean":"2023-11-25T04:22:48.048004+0000","last_became_active":"2023-11-25T04:22:46.035405+0000","last_became_peered":"2023-11-25T04:22:46.035405+0000","last_unstale":"2023-11-25T04:22:48.048004+0000","last_undegraded":"2023-11-25T04:22:48.048004+0000","last_fullsized":"2023-11-25T04:22:48.048004+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:53:47.398746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-25T04:22:47.032524+0000","last_change":"2023-11-25T04:22:41.999067+0000","last_active":"2023-11-25T04:22:47.032524+0000","last_peered":"2023-11-25T04:22:47.032524+0000","last_clean":"2023-11-25T04:22:47.032524+0000","last_became_active":"2023-11-25T04:22:41.998798+0000","last_became_peered":"2023-11-25T04:22:41.998798+0000","last_unstale":"2023-11-25T04:22:47.032524+0000","last_undegraded":"2023-11-25T04:22:47.032524+0000","last_fullsized":"2023-11-25T04:22:47.032524+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:00:04.266051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047617+0000","last_change":"2023-11-25T04:22:47.043464+0000","last_active":"2023-11-25T04:22:48.047617+0000","last_peered":"2023-11-25T04:22:48.047617+0000","last_clean":"2023-11-25T04:22:48.047617+0000","last_became_active":"2023-11-25T04:22:47.043050+0000","last_became_peered":"2023-11-25T04:22:47.043050+0000","last_unstale":"2023-11-25T04:22:48.047617+0000","last_undegraded":"2023-11-25T04:22:48.047617+0000","last_fullsized":"2023-11-25T04:22:48.047617+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:59:45.492077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"18'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048417+0000","last_change":"2023-11-25T04:22:46.034606+0000","last_active":"2023-11-25T04:22:48.048417+0000","last_peered":"2023-11-25T04:22:48.048417+0000","last_clean":"2023-11-25T04:22:48.048417+0000","last_became_active":"2023-11-25T04:22:46.034229+0000","last_became_peered":"2023-11-25T04:22:46.034229+0000","last_unstale":"2023-11-25T04:22:48.048417+0000","last_undegraded":"2023-11-25T04:22:48.048417+0000","last_fullsized":"2023-11-25T04:22:48.048417+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:21:42.146264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-25T04:22:47.031216+0000","last_change":"2023-11-25T04:22:42.000258+0000","last_active":"2023-11-25T04:22:47.031216+0000","last_peered":"2023-11-25T04:22:47.031216+0000","last_clean":"2023-11-25T04:22:47.031216+0000","last_became_active":"2023-11-25T04:22:41.999992+0000","last_became_peered":"2023-11-25T04:22:41.999992+0000","last_unstale":"2023-11-25T04:22:47.031216+0000","last_undegraded":"2023-11-25T04:22:47.031216+0000","last_fullsized":"2023-11-25T04:22:47.031216+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:31:13.715992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047510+0000","last_change":"2023-11-25T04:22:47.042860+0000","last_active":"2023-11-25T04:22:48.047510+0000","last_peered":"2023-11-25T04:22:48.047510+0000","last_clean":"2023-11-25T04:22:48.047510+0000","last_became_active":"2023-11-25T04:22:47.042472+0000","last_became_peered":"2023-11-25T04:22:47.042472+0000","last_unstale":"2023-11-25T04:22:48.047510+0000","last_undegraded":"2023-11-25T04:22:48.047510+0000","last_fullsized":"2023-11-25T04:22:48.047510+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:44:48.553972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048418+0000","last_change":"2023-11-25T04:22:46.038417+0000","last_active":"2023-11-25T04:22:48.048418+0000","last_peered":"2023-11-25T04:22:48.048418+0000","last_clean":"2023-11-25T04:22:48.048418+0000","last_became_active":"2023-11-25T04:22:46.037513+0000","last_became_peered":"2023-11-25T04:22:46.037513+0000","last_unstale":"2023-11-25T04:22:48.048418+0000","last_undegraded":"2023-11-25T04:22:48.048418+0000","last_fullsized":"2023-11-25T04:22:48.048418+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:58:14.877413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-25T04:22:47.030950+0000","last_change":"2023-11-25T04:22:42.000217+0000","last_active":"2023-11-25T04:22:47.030950+0000","last_peered":"2023-11-25T04:22:47.030950+0000","last_clean":"2023-11-25T04:22:47.030950+0000","last_became_active":"2023-11-25T04:22:41.999987+0000","last_became_peered":"2023-11-25T04:22:41.999987+0000","last_unstale":"2023-11-25T04:22:47.030950+0000","last_undegraded":"2023-11-25T04:22:47.030950+0000","last_fullsized":"2023-11-25T04:22:47.030950+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:11:51.100459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047385+0000","last_change":"2023-11-25T04:22:47.036099+0000","last_active":"2023-11-25T04:22:48.047385+0000","last_peered":"2023-11-25T04:22:48.047385+0000","last_clean":"2023-11-25T04:22:48.047385+0000","last_became_active":"2023-11-25T04:22:47.035581+0000","last_became_peered":"2023-11-25T04:22:47.035581+0000","last_unstale":"2023-11-25T04:22:48.047385+0000","last_undegraded":"2023-11-25T04:22:48.047385+0000","last_fullsized":"2023-11-25T04:22:48.047385+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:21:43.155728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048171+0000","last_change":"2023-11-25T04:22:47.040343+0000","last_active":"2023-11-25T04:22:48.048171+0000","last_peered":"2023-11-25T04:22:48.048171+0000","last_clean":"2023-11-25T04:22:48.048171+0000","last_became_active":"2023-11-25T04:22:47.039151+0000","last_became_peered":"2023-11-25T04:22:47.039151+0000","last_unstale":"2023-11-25T04:22:48.048171+0000","last_undegraded":"2023-11-25T04:22:48.048171+0000","last_fullsized":"2023-11-25T04:22:48.048171+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:21:15.981901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"18'6","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048260+0000","last_change":"2023-11-25T04:22:46.040005+0000","last_active":"2023-11-25T04:22:48.048260+0000","last_peered":"2023-11-25T04:22:48.048260+0000","last_clean":"2023-11-25T04:22:48.048260+0000","last_became_active":"2023-11-25T04:22:46.039655+0000","last_became_peered":"2023-11-25T04:22:46.039655+0000","last_unstale":"2023-11-25T04:22:48.048260+0000","last_undegraded":"2023-11-25T04:22:48.048260+0000","last_fullsized":"2023-11-25T04:22:48.048260+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:41:15.662804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047237+0000","last_change":"2023-11-25T04:22:47.037181+0000","last_active":"2023-11-25T04:22:48.047237+0000","last_peered":"2023-11-25T04:22:48.047237+0000","last_clean":"2023-11-25T04:22:48.047237+0000","last_became_active":"2023-11-25T04:22:47.036821+0000","last_became_peered":"2023-11-25T04:22:47.036821+0000","last_unstale":"2023-11-25T04:22:48.047237+0000","last_undegraded":"2023-11-25T04:22:48.047237+0000","last_fullsized":"2023-11-25T04:22:48.047237+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:12:51.990448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"18'3","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048375+0000","last_change":"2023-11-25T04:22:46.035816+0000","last_active":"2023-11-25T04:22:48.048375+0000","last_peered":"2023-11-25T04:22:48.048375+0000","last_clean":"2023-11-25T04:22:48.048375+0000","last_became_active":"2023-11-25T04:22:46.035461+0000","last_became_peered":"2023-11-25T04:22:46.035461+0000","last_unstale":"2023-11-25T04:22:48.048375+0000","last_undegraded":"2023-11-25T04:22:48.048375+0000","last_fullsized":"2023-11-25T04:22:48.048375+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:58:26.155063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048370+0000","last_change":"2023-11-25T04:22:46.040458+0000","last_active":"2023-11-25T04:22:48.048370+0000","last_peered":"2023-11-25T04:22:48.048370+0000","last_clean":"2023-11-25T04:22:48.048370+0000","last_became_active":"2023-11-25T04:22:46.040143+0000","last_became_peered":"2023-11-25T04:22:46.040143+0000","last_unstale":"2023-11-25T04:22:48.048370+0000","last_undegraded":"2023-11-25T04:22:48.048370+0000","last_fullsized":"2023-11-25T04:22:48.048370+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:27:31.901618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047387+0000","last_change":"2023-11-25T04:22:47.040346+0000","last_active":"2023-11-25T04:22:48.047387+0000","last_peered":"2023-11-25T04:22:48.047387+0000","last_clean":"2023-11-25T04:22:48.047387+0000","last_became_active":"2023-11-25T04:22:47.040084+0000","last_became_peered":"2023-11-25T04:22:47.040084+0000","last_unstale":"2023-11-25T04:22:48.047387+0000","last_undegraded":"2023-11-25T04:22:48.047387+0000","last_fullsized":"2023-11-25T04:22:48.047387+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:59:45.492077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048308+0000","last_change":"2023-11-25T04:22:46.038721+0000","last_active":"2023-11-25T04:22:48.048308+0000","last_peered":"2023-11-25T04:22:48.048308+0000","last_clean":"2023-11-25T04:22:48.048308+0000","last_became_active":"2023-11-25T04:22:46.037698+0000","last_became_peered":"2023-11-25T04:22:46.037698+0000","last_unstale":"2023-11-25T04:22:48.048308+0000","last_undegraded":"2023-11-25T04:22:48.048308+0000","last_fullsized":"2023-11-25T04:22:48.048308+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:25:20.721529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047199+0000","last_change":"2023-11-25T04:22:47.037790+0000","last_active":"2023-11-25T04:22:48.047199+0000","last_peered":"2023-11-25T04:22:48.047199+0000","last_clean":"2023-11-25T04:22:48.047199+0000","last_became_active":"2023-11-25T04:22:47.037149+0000","last_became_peered":"2023-11-25T04:22:47.037149+0000","last_unstale":"2023-11-25T04:22:48.047199+0000","last_undegraded":"2023-11-25T04:22:48.047199+0000","last_fullsized":"2023-11-25T04:22:48.047199+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:31:54.857270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048584+0000","last_change":"2023-11-25T04:22:47.036079+0000","last_active":"2023-11-25T04:22:48.048584+0000","last_peered":"2023-11-25T04:22:48.048584+0000","last_clean":"2023-11-25T04:22:48.048584+0000","last_became_active":"2023-11-25T04:22:47.035625+0000","last_became_peered":"2023-11-25T04:22:47.035625+0000","last_unstale":"2023-11-25T04:22:48.048584+0000","last_undegraded":"2023-11-25T04:22:48.048584+0000","last_fullsized":"2023-11-25T04:22:48.048584+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:21:15.981901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047749+0000","last_change":"2023-11-25T04:22:46.036085+0000","last_active":"2023-11-25T04:22:48.047749+0000","last_peered":"2023-11-25T04:22:48.047749+0000","last_clean":"2023-11-25T04:22:48.047749+0000","last_became_active":"2023-11-25T04:22:46.035495+0000","last_became_peered":"2023-11-25T04:22:46.035495+0000","last_unstale":"2023-11-25T04:22:48.047749+0000","last_undegraded":"2023-11-25T04:22:48.047749+0000","last_fullsized":"2023-11-25T04:22:48.047749+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:14:56.735365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048251+0000","last_change":"2023-11-25T04:22:47.043064+0000","last_active":"2023-11-25T04:22:48.048251+0000","last_peered":"2023-11-25T04:22:48.048251+0000","last_clean":"2023-11-25T04:22:48.048251+0000","last_became_active":"2023-11-25T04:22:47.042760+0000","last_became_peered":"2023-11-25T04:22:47.042760+0000","last_unstale":"2023-11-25T04:22:48.048251+0000","last_undegraded":"2023-11-25T04:22:48.048251+0000","last_fullsized":"2023-11-25T04:22:48.048251+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:37:30.959147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047514+0000","last_change":"2023-11-25T04:22:46.038298+0000","last_active":"2023-11-25T04:22:48.047514+0000","last_peered":"2023-11-25T04:22:48.047514+0000","last_clean":"2023-11-25T04:22:48.047514+0000","last_became_active":"2023-11-25T04:22:46.036629+0000","last_became_peered":"2023-11-25T04:22:46.036629+0000","last_unstale":"2023-11-25T04:22:48.047514+0000","last_undegraded":"2023-11-25T04:22:48.047514+0000","last_fullsized":"2023-11-25T04:22:48.047514+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:45:14.670326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048209+0000","last_change":"2023-11-25T04:22:47.043156+0000","last_active":"2023-11-25T04:22:48.048209+0000","last_peered":"2023-11-25T04:22:48.048209+0000","last_clean":"2023-11-25T04:22:48.048209+0000","last_became_active":"2023-11-25T04:22:47.042693+0000","last_became_peered":"2023-11-25T04:22:47.042693+0000","last_unstale":"2023-11-25T04:22:48.048209+0000","last_undegraded":"2023-11-25T04:22:48.048209+0000","last_fullsized":"2023-11-25T04:22:48.048209+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:38:38.839941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1a","version":"18'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047866+0000","last_change":"2023-11-25T04:22:46.034340+0000","last_active":"2023-11-25T04:22:48.047866+0000","last_peered":"2023-11-25T04:22:48.047866+0000","last_clean":"2023-11-25T04:22:48.047866+0000","last_became_active":"2023-11-25T04:22:46.033944+0000","last_became_peered":"2023-11-25T04:22:46.033944+0000","last_unstale":"2023-11-25T04:22:48.047866+0000","last_undegraded":"2023-11-25T04:22:48.047866+0000","last_fullsized":"2023-11-25T04:22:48.047866+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:59:30.291004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047694+0000","last_change":"2023-11-25T04:22:47.039547+0000","last_active":"2023-11-25T04:22:48.047694+0000","last_peered":"2023-11-25T04:22:48.047694+0000","last_clean":"2023-11-25T04:22:48.047694+0000","last_became_active":"2023-11-25T04:22:47.039218+0000","last_became_peered":"2023-11-25T04:22:47.039218+0000","last_unstale":"2023-11-25T04:22:48.047694+0000","last_undegraded":"2023-11-25T04:22:48.047694+0000","last_fullsized":"2023-11-25T04:22:48.047694+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:44:48.553972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"18'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047860+0000","last_change":"2023-11-25T04:22:46.035882+0000","last_active":"2023-11-25T04:22:48.047860+0000","last_peered":"2023-11-25T04:22:48.047860+0000","last_clean":"2023-11-25T04:22:48.047860+0000","last_became_active":"2023-11-25T04:22:46.035588+0000","last_became_peered":"2023-11-25T04:22:46.035588+0000","last_unstale":"2023-11-25T04:22:48.047860+0000","last_undegraded":"2023-11-25T04:22:48.047860+0000","last_fullsized":"2023-11-25T04:22:48.047860+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:43:10.702849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047831+0000","last_change":"2023-11-25T04:22:47.041267+0000","last_active":"2023-11-25T04:22:48.047831+0000","last_peered":"2023-11-25T04:22:48.047831+0000","last_clean":"2023-11-25T04:22:48.047831+0000","last_became_active":"2023-11-25T04:22:47.040408+0000","last_became_peered":"2023-11-25T04:22:47.040408+0000","last_unstale":"2023-11-25T04:22:48.047831+0000","last_undegraded":"2023-11-25T04:22:48.047831+0000","last_fullsized":"2023-11-25T04:22:48.047831+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:37:30.959147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"18'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048131+0000","last_change":"2023-11-25T04:22:46.040793+0000","last_active":"2023-11-25T04:22:48.048131+0000","last_peered":"2023-11-25T04:22:48.048131+0000","last_clean":"2023-11-25T04:22:48.048131+0000","last_became_active":"2023-11-25T04:22:46.040493+0000","last_became_peered":"2023-11-25T04:22:46.040493+0000","last_unstale":"2023-11-25T04:22:48.048131+0000","last_undegraded":"2023-11-25T04:22:48.048131+0000","last_fullsized":"2023-11-25T04:22:48.048131+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:58:26.155063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048056+0000","last_change":"2023-11-25T04:22:47.040067+0000","last_active":"2023-11-25T04:22:48.048056+0000","last_peered":"2023-11-25T04:22:48.048056+0000","last_clean":"2023-11-25T04:22:48.048056+0000","last_became_active":"2023-11-25T04:22:47.038425+0000","last_became_peered":"2023-11-25T04:22:47.038425+0000","last_unstale":"2023-11-25T04:22:48.048056+0000","last_undegraded":"2023-11-25T04:22:48.048056+0000","last_fullsized":"2023-11-25T04:22:48.048056+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:25:06.132517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"18'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048297+0000","last_change":"2023-11-25T04:22:46.035872+0000","last_active":"2023-11-25T04:22:48.048297+0000","last_peered":"2023-11-25T04:22:48.048297+0000","last_clean":"2023-11-25T04:22:48.048297+0000","last_became_active":"2023-11-25T04:22:46.035575+0000","last_became_peered":"2023-11-25T04:22:46.035575+0000","last_unstale":"2023-11-25T04:22:48.048297+0000","last_undegraded":"2023-11-25T04:22:48.048297+0000","last_fullsized":"2023-11-25T04:22:48.048297+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:58:14.877413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047725+0000","last_change":"2023-11-25T04:22:47.036764+0000","last_active":"2023-11-25T04:22:48.047725+0000","last_peered":"2023-11-25T04:22:48.047725+0000","last_clean":"2023-11-25T04:22:48.047725+0000","last_became_active":"2023-11-25T04:22:47.036406+0000","last_became_peered":"2023-11-25T04:22:47.036406+0000","last_unstale":"2023-11-25T04:22:48.047725+0000","last_undegraded":"2023-11-25T04:22:48.047725+0000","last_fullsized":"2023-11-25T04:22:48.047725+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:09:55.357642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1e","version":"18'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048335+0000","last_change":"2023-11-25T04:22:46.035659+0000","last_active":"2023-11-25T04:22:48.048335+0000","last_peered":"2023-11-25T04:22:48.048335+0000","last_clean":"2023-11-25T04:22:48.048335+0000","last_became_active":"2023-11-25T04:22:46.035113+0000","last_became_peered":"2023-11-25T04:22:46.035113+0000","last_unstale":"2023-11-25T04:22:48.048335+0000","last_undegraded":"2023-11-25T04:22:48.048335+0000","last_fullsized":"2023-11-25T04:22:48.048335+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:26:25.338117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048474+0000","last_change":"2023-11-25T04:22:47.041554+0000","last_active":"2023-11-25T04:22:48.048474+0000","last_peered":"2023-11-25T04:22:48.048474+0000","last_clean":"2023-11-25T04:22:48.048474+0000","last_became_active":"2023-11-25T04:22:47.040314+0000","last_became_peered":"2023-11-25T04:22:47.040314+0000","last_unstale":"2023-11-25T04:22:48.048474+0000","last_undegraded":"2023-11-25T04:22:48.048474+0000","last_fullsized":"2023-11-25T04:22:48.048474+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:24:56.315251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"18'4","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048204+0000","last_change":"2023-11-25T04:22:46.038702+0000","last_active":"2023-11-25T04:22:48.048204+0000","last_peered":"2023-11-25T04:22:48.048204+0000","last_clean":"2023-11-25T04:22:48.048204+0000","last_became_active":"2023-11-25T04:22:46.037567+0000","last_became_peered":"2023-11-25T04:22:46.037567+0000","last_unstale":"2023-11-25T04:22:48.048204+0000","last_undegraded":"2023-11-25T04:22:48.048204+0000","last_fullsized":"2023-11-25T04:22:48.048204+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:58:26.155063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738374,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35508,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336332,"statfs":{"total":96636764160,"available":96600403968,"internally_reserved":0,"allocated":172032,"data_stored":58325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39500000000000002}]}]},{"osd":1,"up_from":8,"seq":34359738374,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31944,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339896,"statfs":{"total":96636764160,"available":96604053504,"internally_reserved":0,"allocated":782336,"data_stored":649871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738373,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31952,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339888,"statfs":{"total":96636764160,"available":96604045312,"internally_reserved":0,"allocated":782336,"data_stored":645884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-25T04:22:50.035 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-25T04:22:50.373 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:22:50.375 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2023-11-25T04:22:50.386 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-11-25T04:22:49.493367+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":247,"num_write_kb":3474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":195,"ondisk_log_size":195,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":99,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":99404,"kb_used_data":1696,"kb_used_omap":4,"kb_used_meta":97659,"kb_avail":283016116,"statfs":{"total":289910292480,"available":289808502784,"internally_reserved":0,"allocated":1736704,"data_stored":1354080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":100003151},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.473744"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048437+0000","last_change":"2023-11-25T04:22:47.039847+0000","last_active":"2023-11-25T04:22:48.048437+0000","last_peered":"2023-11-25T04:22:48.048437+0000","last_clean":"2023-11-25T04:22:48.048437+0000","last_became_active":"2023-11-25T04:22:47.039490+0000","last_became_peered":"2023-11-25T04:22:47.039490+0000","last_unstale":"2023-11-25T04:22:48.048437+0000","last_undegraded":"2023-11-25T04:22:48.048437+0000","last_fullsized":"2023-11-25T04:22:48.048437+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:25:06.132517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048105+0000","last_change":"2023-11-25T04:22:47.040836+0000","last_active":"2023-11-25T04:22:48.048105+0000","last_peered":"2023-11-25T04:22:48.048105+0000","last_clean":"2023-11-25T04:22:48.048105+0000","last_became_active":"2023-11-25T04:22:47.040511+0000","last_became_peered":"2023-11-25T04:22:47.040511+0000","last_unstale":"2023-11-25T04:22:48.048105+0000","last_undegraded":"2023-11-25T04:22:48.048105+0000","last_fullsized":"2023-11-25T04:22:48.048105+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:15:39.511043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048297+0000","last_change":"2023-11-25T04:22:47.043529+0000","last_active":"2023-11-25T04:22:48.048297+0000","last_peered":"2023-11-25T04:22:48.048297+0000","last_clean":"2023-11-25T04:22:48.048297+0000","last_became_active":"2023-11-25T04:22:47.042733+0000","last_became_peered":"2023-11-25T04:22:47.042733+0000","last_unstale":"2023-11-25T04:22:48.048297+0000","last_undegraded":"2023-11-25T04:22:48.048297+0000","last_fullsized":"2023-11-25T04:22:48.048297+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:47:34.764521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047862+0000","last_change":"2023-11-25T04:22:47.040314+0000","last_active":"2023-11-25T04:22:48.047862+0000","last_peered":"2023-11-25T04:22:48.047862+0000","last_clean":"2023-11-25T04:22:48.047862+0000","last_became_active":"2023-11-25T04:22:47.039185+0000","last_became_peered":"2023-11-25T04:22:47.039185+0000","last_unstale":"2023-11-25T04:22:48.047862+0000","last_undegraded":"2023-11-25T04:22:48.047862+0000","last_fullsized":"2023-11-25T04:22:48.047862+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:16:29.085050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047841+0000","last_change":"2023-11-25T04:22:47.039499+0000","last_active":"2023-11-25T04:22:48.047841+0000","last_peered":"2023-11-25T04:22:48.047841+0000","last_clean":"2023-11-25T04:22:48.047841+0000","last_became_active":"2023-11-25T04:22:47.039165+0000","last_became_peered":"2023-11-25T04:22:47.039165+0000","last_unstale":"2023-11-25T04:22:48.047841+0000","last_undegraded":"2023-11-25T04:22:48.047841+0000","last_fullsized":"2023-11-25T04:22:48.047841+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:57:28.855151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047565+0000","last_change":"2023-11-25T04:22:47.039992+0000","last_active":"2023-11-25T04:22:48.047565+0000","last_peered":"2023-11-25T04:22:48.047565+0000","last_clean":"2023-11-25T04:22:48.047565+0000","last_became_active":"2023-11-25T04:22:47.039405+0000","last_became_peered":"2023-11-25T04:22:47.039405+0000","last_unstale":"2023-11-25T04:22:48.047565+0000","last_undegraded":"2023-11-25T04:22:48.047565+0000","last_fullsized":"2023-11-25T04:22:48.047565+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:56:59.689578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047396+0000","last_change":"2023-11-25T04:22:47.037971+0000","last_active":"2023-11-25T04:22:48.047396+0000","last_peered":"2023-11-25T04:22:48.047396+0000","last_clean":"2023-11-25T04:22:48.047396+0000","last_became_active":"2023-11-25T04:22:47.036736+0000","last_became_peered":"2023-11-25T04:22:47.036736+0000","last_unstale":"2023-11-25T04:22:48.047396+0000","last_undegraded":"2023-11-25T04:22:48.047396+0000","last_fullsized":"2023-11-25T04:22:48.047396+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:02:07.088241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047307+0000","last_change":"2023-11-25T04:22:47.041728+0000","last_active":"2023-11-25T04:22:48.047307+0000","last_peered":"2023-11-25T04:22:48.047307+0000","last_clean":"2023-11-25T04:22:48.047307+0000","last_became_active":"2023-11-25T04:22:47.041017+0000","last_became_peered":"2023-11-25T04:22:47.041017+0000","last_unstale":"2023-11-25T04:22:48.047307+0000","last_undegraded":"2023-11-25T04:22:48.047307+0000","last_fullsized":"2023-11-25T04:22:48.047307+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:57:28.855151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048097+0000","last_change":"2023-11-25T04:22:47.038397+0000","last_active":"2023-11-25T04:22:48.048097+0000","last_peered":"2023-11-25T04:22:48.048097+0000","last_clean":"2023-11-25T04:22:48.048097+0000","last_became_active":"2023-11-25T04:22:47.037328+0000","last_became_peered":"2023-11-25T04:22:47.037328+0000","last_unstale":"2023-11-25T04:22:48.048097+0000","last_undegraded":"2023-11-25T04:22:48.048097+0000","last_fullsized":"2023-11-25T04:22:48.048097+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:56:59.689578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048223+0000","last_change":"2023-11-25T04:22:47.040182+0000","last_active":"2023-11-25T04:22:48.048223+0000","last_peered":"2023-11-25T04:22:48.048223+0000","last_clean":"2023-11-25T04:22:48.048223+0000","last_became_active":"2023-11-25T04:22:47.039892+0000","last_became_peered":"2023-11-25T04:22:47.039892+0000","last_unstale":"2023-11-25T04:22:48.048223+0000","last_undegraded":"2023-11-25T04:22:48.048223+0000","last_fullsized":"2023-11-25T04:22:48.048223+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:38:09.176582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048202+0000","last_change":"2023-11-25T04:22:47.039907+0000","last_active":"2023-11-25T04:22:48.048202+0000","last_peered":"2023-11-25T04:22:48.048202+0000","last_clean":"2023-11-25T04:22:48.048202+0000","last_became_active":"2023-11-25T04:22:47.039046+0000","last_became_peered":"2023-11-25T04:22:47.039046+0000","last_unstale":"2023-11-25T04:22:48.048202+0000","last_undegraded":"2023-11-25T04:22:48.048202+0000","last_fullsized":"2023-11-25T04:22:48.048202+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:01:51.418774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047609+0000","last_change":"2023-11-25T04:22:47.037618+0000","last_active":"2023-11-25T04:22:48.047609+0000","last_peered":"2023-11-25T04:22:48.047609+0000","last_clean":"2023-11-25T04:22:48.047609+0000","last_became_active":"2023-11-25T04:22:47.036806+0000","last_became_peered":"2023-11-25T04:22:47.036806+0000","last_unstale":"2023-11-25T04:22:48.047609+0000","last_undegraded":"2023-11-25T04:22:48.047609+0000","last_fullsized":"2023-11-25T04:22:48.047609+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:12:51.990448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047608+0000","last_change":"2023-11-25T04:22:47.040388+0000","last_active":"2023-11-25T04:22:48.047608+0000","last_peered":"2023-11-25T04:22:48.047608+0000","last_clean":"2023-11-25T04:22:48.047608+0000","last_became_active":"2023-11-25T04:22:47.040129+0000","last_became_peered":"2023-11-25T04:22:47.040129+0000","last_unstale":"2023-11-25T04:22:48.047608+0000","last_undegraded":"2023-11-25T04:22:48.047608+0000","last_fullsized":"2023-11-25T04:22:48.047608+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:31:54.857270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047643+0000","last_change":"2023-11-25T04:22:47.038887+0000","last_active":"2023-11-25T04:22:48.047643+0000","last_peered":"2023-11-25T04:22:48.047643+0000","last_clean":"2023-11-25T04:22:48.047643+0000","last_became_active":"2023-11-25T04:22:47.038014+0000","last_became_peered":"2023-11-25T04:22:47.038014+0000","last_unstale":"2023-11-25T04:22:48.047643+0000","last_undegraded":"2023-11-25T04:22:48.047643+0000","last_fullsized":"2023-11-25T04:22:48.047643+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:38:38.839941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047471+0000","last_change":"2023-11-25T04:22:47.035843+0000","last_active":"2023-11-25T04:22:48.047471+0000","last_peered":"2023-11-25T04:22:48.047471+0000","last_clean":"2023-11-25T04:22:48.047471+0000","last_became_active":"2023-11-25T04:22:47.035453+0000","last_became_peered":"2023-11-25T04:22:47.035453+0000","last_unstale":"2023-11-25T04:22:48.047471+0000","last_undegraded":"2023-11-25T04:22:48.047471+0000","last_fullsized":"2023-11-25T04:22:48.047471+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:45:40.065146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047308+0000","last_change":"2023-11-25T04:22:47.041778+0000","last_active":"2023-11-25T04:22:48.047308+0000","last_peered":"2023-11-25T04:22:48.047308+0000","last_clean":"2023-11-25T04:22:48.047308+0000","last_became_active":"2023-11-25T04:22:47.041115+0000","last_became_peered":"2023-11-25T04:22:47.041115+0000","last_unstale":"2023-11-25T04:22:48.047308+0000","last_undegraded":"2023-11-25T04:22:48.047308+0000","last_fullsized":"2023-11-25T04:22:48.047308+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:30:04.568790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048422+0000","last_change":"2023-11-25T04:22:47.039389+0000","last_active":"2023-11-25T04:22:48.048422+0000","last_peered":"2023-11-25T04:22:48.048422+0000","last_clean":"2023-11-25T04:22:48.048422+0000","last_became_active":"2023-11-25T04:22:47.039123+0000","last_became_peered":"2023-11-25T04:22:47.039123+0000","last_unstale":"2023-11-25T04:22:48.048422+0000","last_undegraded":"2023-11-25T04:22:48.048422+0000","last_fullsized":"2023-11-25T04:22:48.048422+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:09:11.570561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048403+0000","last_change":"2023-11-25T04:22:47.036946+0000","last_active":"2023-11-25T04:22:48.048403+0000","last_peered":"2023-11-25T04:22:48.048403+0000","last_clean":"2023-11-25T04:22:48.048403+0000","last_became_active":"2023-11-25T04:22:47.036411+0000","last_became_peered":"2023-11-25T04:22:47.036411+0000","last_unstale":"2023-11-25T04:22:48.048403+0000","last_undegraded":"2023-11-25T04:22:48.048403+0000","last_fullsized":"2023-11-25T04:22:48.048403+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:16:29.085050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047729+0000","last_change":"2023-11-25T04:22:47.038983+0000","last_active":"2023-11-25T04:22:48.047729+0000","last_peered":"2023-11-25T04:22:48.047729+0000","last_clean":"2023-11-25T04:22:48.047729+0000","last_became_active":"2023-11-25T04:22:47.038477+0000","last_became_peered":"2023-11-25T04:22:47.038477+0000","last_unstale":"2023-11-25T04:22:48.047729+0000","last_undegraded":"2023-11-25T04:22:48.047729+0000","last_fullsized":"2023-11-25T04:22:48.047729+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:30:04.568790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047891+0000","last_change":"2023-11-25T04:22:47.038155+0000","last_active":"2023-11-25T04:22:48.047891+0000","last_peered":"2023-11-25T04:22:48.047891+0000","last_clean":"2023-11-25T04:22:48.047891+0000","last_became_active":"2023-11-25T04:22:47.037740+0000","last_became_peered":"2023-11-25T04:22:47.037740+0000","last_unstale":"2023-11-25T04:22:48.047891+0000","last_undegraded":"2023-11-25T04:22:48.047891+0000","last_fullsized":"2023-11-25T04:22:48.047891+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:09:11.570561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048543+0000","last_change":"2023-11-25T04:22:47.040641+0000","last_active":"2023-11-25T04:22:48.048543+0000","last_peered":"2023-11-25T04:22:48.048543+0000","last_clean":"2023-11-25T04:22:48.048543+0000","last_became_active":"2023-11-25T04:22:47.039409+0000","last_became_peered":"2023-11-25T04:22:47.039409+0000","last_unstale":"2023-11-25T04:22:48.048543+0000","last_undegraded":"2023-11-25T04:22:48.048543+0000","last_fullsized":"2023-11-25T04:22:48.048543+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:23:15.914817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047384+0000","last_change":"2023-11-25T04:22:47.043456+0000","last_active":"2023-11-25T04:22:48.047384+0000","last_peered":"2023-11-25T04:22:48.047384+0000","last_clean":"2023-11-25T04:22:48.047384+0000","last_became_active":"2023-11-25T04:22:47.043098+0000","last_became_peered":"2023-11-25T04:22:47.043098+0000","last_unstale":"2023-11-25T04:22:48.047384+0000","last_undegraded":"2023-11-25T04:22:48.047384+0000","last_fullsized":"2023-11-25T04:22:48.047384+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:47:57.307757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047103+0000","last_change":"2023-11-25T04:22:47.035260+0000","last_active":"2023-11-25T04:22:48.047103+0000","last_peered":"2023-11-25T04:22:48.047103+0000","last_clean":"2023-11-25T04:22:48.047103+0000","last_became_active":"2023-11-25T04:22:47.034702+0000","last_became_peered":"2023-11-25T04:22:47.034702+0000","last_unstale":"2023-11-25T04:22:48.047103+0000","last_undegraded":"2023-11-25T04:22:48.047103+0000","last_fullsized":"2023-11-25T04:22:48.047103+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:49:02.500022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047259+0000","last_change":"2023-11-25T04:22:47.036805+0000","last_active":"2023-11-25T04:22:48.047259+0000","last_peered":"2023-11-25T04:22:48.047259+0000","last_clean":"2023-11-25T04:22:48.047259+0000","last_became_active":"2023-11-25T04:22:47.036135+0000","last_became_peered":"2023-11-25T04:22:47.036135+0000","last_unstale":"2023-11-25T04:22:48.047259+0000","last_undegraded":"2023-11-25T04:22:48.047259+0000","last_fullsized":"2023-11-25T04:22:48.047259+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:25:21.730992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048430+0000","last_change":"2023-11-25T04:22:47.036611+0000","last_active":"2023-11-25T04:22:48.048430+0000","last_peered":"2023-11-25T04:22:48.048430+0000","last_clean":"2023-11-25T04:22:48.048430+0000","last_became_active":"2023-11-25T04:22:47.035762+0000","last_became_peered":"2023-11-25T04:22:47.035762+0000","last_unstale":"2023-11-25T04:22:48.048430+0000","last_undegraded":"2023-11-25T04:22:48.048430+0000","last_fullsized":"2023-11-25T04:22:48.048430+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:09:11.570561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048216+0000","last_change":"2023-11-25T04:22:47.042423+0000","last_active":"2023-11-25T04:22:48.048216+0000","last_peered":"2023-11-25T04:22:48.048216+0000","last_clean":"2023-11-25T04:22:48.048216+0000","last_became_active":"2023-11-25T04:22:47.042022+0000","last_became_peered":"2023-11-25T04:22:47.042022+0000","last_unstale":"2023-11-25T04:22:48.048216+0000","last_undegraded":"2023-11-25T04:22:48.048216+0000","last_fullsized":"2023-11-25T04:22:48.048216+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:38:09.176582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048337+0000","last_change":"2023-11-25T04:22:47.044040+0000","last_active":"2023-11-25T04:22:48.048337+0000","last_peered":"2023-11-25T04:22:48.048337+0000","last_clean":"2023-11-25T04:22:48.048337+0000","last_became_active":"2023-11-25T04:22:47.043329+0000","last_became_peered":"2023-11-25T04:22:47.043329+0000","last_unstale":"2023-11-25T04:22:48.048337+0000","last_undegraded":"2023-11-25T04:22:48.048337+0000","last_fullsized":"2023-11-25T04:22:48.048337+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:10:59.482775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047954+0000","last_change":"2023-11-25T04:22:47.036485+0000","last_active":"2023-11-25T04:22:48.047954+0000","last_peered":"2023-11-25T04:22:48.047954+0000","last_clean":"2023-11-25T04:22:48.047954+0000","last_became_active":"2023-11-25T04:22:47.035799+0000","last_became_peered":"2023-11-25T04:22:47.035799+0000","last_unstale":"2023-11-25T04:22:48.047954+0000","last_undegraded":"2023-11-25T04:22:48.047954+0000","last_fullsized":"2023-11-25T04:22:48.047954+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:45:40.065146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047835+0000","last_change":"2023-11-25T04:22:47.037668+0000","last_active":"2023-11-25T04:22:48.047835+0000","last_peered":"2023-11-25T04:22:48.047835+0000","last_clean":"2023-11-25T04:22:48.047835+0000","last_became_active":"2023-11-25T04:22:47.036823+0000","last_became_peered":"2023-11-25T04:22:47.036823+0000","last_unstale":"2023-11-25T04:22:48.047835+0000","last_undegraded":"2023-11-25T04:22:48.047835+0000","last_fullsized":"2023-11-25T04:22:48.047835+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:02:07.088241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047758+0000","last_change":"2023-11-25T04:22:47.038117+0000","last_active":"2023-11-25T04:22:48.047758+0000","last_peered":"2023-11-25T04:22:48.047758+0000","last_clean":"2023-11-25T04:22:48.047758+0000","last_became_active":"2023-11-25T04:22:47.037751+0000","last_became_peered":"2023-11-25T04:22:47.037751+0000","last_unstale":"2023-11-25T04:22:48.047758+0000","last_undegraded":"2023-11-25T04:22:48.047758+0000","last_fullsized":"2023-11-25T04:22:48.047758+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:30:04.568790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047184+0000","last_change":"2023-11-25T04:22:47.037786+0000","last_active":"2023-11-25T04:22:48.047184+0000","last_peered":"2023-11-25T04:22:48.047184+0000","last_clean":"2023-11-25T04:22:48.047184+0000","last_became_active":"2023-11-25T04:22:47.037206+0000","last_became_peered":"2023-11-25T04:22:47.037206+0000","last_unstale":"2023-11-25T04:22:48.047184+0000","last_undegraded":"2023-11-25T04:22:48.047184+0000","last_fullsized":"2023-11-25T04:22:48.047184+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:11:40.532140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047505+0000","last_change":"2023-11-25T04:22:47.041325+0000","last_active":"2023-11-25T04:22:48.047505+0000","last_peered":"2023-11-25T04:22:48.047505+0000","last_clean":"2023-11-25T04:22:48.047505+0000","last_became_active":"2023-11-25T04:22:47.040086+0000","last_became_peered":"2023-11-25T04:22:47.040086+0000","last_unstale":"2023-11-25T04:22:48.047505+0000","last_undegraded":"2023-11-25T04:22:48.047505+0000","last_fullsized":"2023-11-25T04:22:48.047505+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:15:39.511043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047822+0000","last_change":"2023-11-25T04:22:47.036689+0000","last_active":"2023-11-25T04:22:48.047822+0000","last_peered":"2023-11-25T04:22:48.047822+0000","last_clean":"2023-11-25T04:22:48.047822+0000","last_became_active":"2023-11-25T04:22:47.036107+0000","last_became_peered":"2023-11-25T04:22:47.036107+0000","last_unstale":"2023-11-25T04:22:48.047822+0000","last_undegraded":"2023-11-25T04:22:48.047822+0000","last_fullsized":"2023-11-25T04:22:48.047822+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:02:07.088241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"18'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048313+0000","last_change":"2023-11-25T04:22:46.040844+0000","last_active":"2023-11-25T04:22:48.048313+0000","last_peered":"2023-11-25T04:22:48.048313+0000","last_clean":"2023-11-25T04:22:48.048313+0000","last_became_active":"2023-11-25T04:22:46.040395+0000","last_became_peered":"2023-11-25T04:22:46.040395+0000","last_unstale":"2023-11-25T04:22:48.048313+0000","last_undegraded":"2023-11-25T04:22:48.048313+0000","last_fullsized":"2023-11-25T04:22:48.048313+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:11:39.522677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048200+0000","last_change":"2023-11-25T04:22:47.037888+0000","last_active":"2023-11-25T04:22:48.048200+0000","last_peered":"2023-11-25T04:22:48.048200+0000","last_clean":"2023-11-25T04:22:48.048200+0000","last_became_active":"2023-11-25T04:22:47.037158+0000","last_became_peered":"2023-11-25T04:22:47.037158+0000","last_unstale":"2023-11-25T04:22:48.048200+0000","last_undegraded":"2023-11-25T04:22:48.048200+0000","last_fullsized":"2023-11-25T04:22:48.048200+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:56:59.689578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047391+0000","last_change":"2023-11-25T04:22:46.036054+0000","last_active":"2023-11-25T04:22:48.047391+0000","last_peered":"2023-11-25T04:22:48.047391+0000","last_clean":"2023-11-25T04:22:48.047391+0000","last_became_active":"2023-11-25T04:22:46.035620+0000","last_became_peered":"2023-11-25T04:22:46.035620+0000","last_unstale":"2023-11-25T04:22:48.047391+0000","last_undegraded":"2023-11-25T04:22:48.047391+0000","last_fullsized":"2023-11-25T04:22:48.047391+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:19:48.371980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048391+0000","last_change":"2023-11-25T04:22:47.045025+0000","last_active":"2023-11-25T04:22:48.048391+0000","last_peered":"2023-11-25T04:22:48.048391+0000","last_clean":"2023-11-25T04:22:48.048391+0000","last_became_active":"2023-11-25T04:22:47.044697+0000","last_became_peered":"2023-11-25T04:22:47.044697+0000","last_unstale":"2023-11-25T04:22:48.048391+0000","last_undegraded":"2023-11-25T04:22:48.048391+0000","last_fullsized":"2023-11-25T04:22:48.048391+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:16:29.085050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047517+0000","last_change":"2023-11-25T04:22:46.037169+0000","last_active":"2023-11-25T04:22:48.047517+0000","last_peered":"2023-11-25T04:22:48.047517+0000","last_clean":"2023-11-25T04:22:48.047517+0000","last_became_active":"2023-11-25T04:22:46.036948+0000","last_became_peered":"2023-11-25T04:22:46.036948+0000","last_unstale":"2023-11-25T04:22:48.047517+0000","last_undegraded":"2023-11-25T04:22:48.047517+0000","last_fullsized":"2023-11-25T04:22:48.047517+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:45:14.670326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047715+0000","last_change":"2023-11-25T04:22:47.044860+0000","last_active":"2023-11-25T04:22:48.047715+0000","last_peered":"2023-11-25T04:22:48.047715+0000","last_clean":"2023-11-25T04:22:48.047715+0000","last_became_active":"2023-11-25T04:22:47.044551+0000","last_became_peered":"2023-11-25T04:22:47.044551+0000","last_unstale":"2023-11-25T04:22:48.047715+0000","last_undegraded":"2023-11-25T04:22:48.047715+0000","last_fullsized":"2023-11-25T04:22:48.047715+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:09:55.357642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.10","version":"18'8","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047593+0000","last_change":"2023-11-25T04:22:46.040356+0000","last_active":"2023-11-25T04:22:48.047593+0000","last_peered":"2023-11-25T04:22:48.047593+0000","last_clean":"2023-11-25T04:22:48.047593+0000","last_became_active":"2023-11-25T04:22:46.039802+0000","last_became_peered":"2023-11-25T04:22:46.039802+0000","last_unstale":"2023-11-25T04:22:48.047593+0000","last_undegraded":"2023-11-25T04:22:48.047593+0000","last_fullsized":"2023-11-25T04:22:48.047593+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:09:54.348179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"18'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048372+0000","last_change":"2023-11-25T04:22:46.040082+0000","last_active":"2023-11-25T04:22:48.048372+0000","last_peered":"2023-11-25T04:22:48.048372+0000","last_clean":"2023-11-25T04:22:48.048372+0000","last_became_active":"2023-11-25T04:22:46.039763+0000","last_became_peered":"2023-11-25T04:22:46.039763+0000","last_unstale":"2023-11-25T04:22:48.048372+0000","last_undegraded":"2023-11-25T04:22:48.048372+0000","last_fullsized":"2023-11-25T04:22:48.048372+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:10:20.178136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047265+0000","last_change":"2023-11-25T04:22:47.043644+0000","last_active":"2023-11-25T04:22:48.047265+0000","last_peered":"2023-11-25T04:22:48.047265+0000","last_clean":"2023-11-25T04:22:48.047265+0000","last_became_active":"2023-11-25T04:22:47.043221+0000","last_became_peered":"2023-11-25T04:22:47.043221+0000","last_unstale":"2023-11-25T04:22:48.047265+0000","last_undegraded":"2023-11-25T04:22:48.047265+0000","last_fullsized":"2023-11-25T04:22:48.047265+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:41:16.672268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"18'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048462+0000","last_change":"2023-11-25T04:22:46.035806+0000","last_active":"2023-11-25T04:22:48.048462+0000","last_peered":"2023-11-25T04:22:48.048462+0000","last_clean":"2023-11-25T04:22:48.048462+0000","last_became_active":"2023-11-25T04:22:46.035418+0000","last_became_peered":"2023-11-25T04:22:46.035418+0000","last_unstale":"2023-11-25T04:22:48.048462+0000","last_undegraded":"2023-11-25T04:22:48.048462+0000","last_fullsized":"2023-11-25T04:22:48.048462+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:31:53.068767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047508+0000","last_change":"2023-11-25T04:22:47.039685+0000","last_active":"2023-11-25T04:22:48.047508+0000","last_peered":"2023-11-25T04:22:48.047508+0000","last_clean":"2023-11-25T04:22:48.047508+0000","last_became_active":"2023-11-25T04:22:47.039296+0000","last_became_peered":"2023-11-25T04:22:47.039296+0000","last_unstale":"2023-11-25T04:22:48.047508+0000","last_undegraded":"2023-11-25T04:22:48.047508+0000","last_fullsized":"2023-11-25T04:22:48.047508+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:11:40.532140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"18'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048658+0000","last_change":"2023-11-25T04:22:46.039953+0000","last_active":"2023-11-25T04:22:48.048658+0000","last_peered":"2023-11-25T04:22:48.048658+0000","last_clean":"2023-11-25T04:22:48.048658+0000","last_became_active":"2023-11-25T04:22:46.039660+0000","last_became_peered":"2023-11-25T04:22:46.039660+0000","last_unstale":"2023-11-25T04:22:48.048658+0000","last_undegraded":"2023-11-25T04:22:48.048658+0000","last_fullsized":"2023-11-25T04:22:48.048658+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:59:44.482613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047887+0000","last_change":"2023-11-25T04:22:47.037912+0000","last_active":"2023-11-25T04:22:48.047887+0000","last_peered":"2023-11-25T04:22:48.047887+0000","last_clean":"2023-11-25T04:22:48.047887+0000","last_became_active":"2023-11-25T04:22:47.036552+0000","last_became_peered":"2023-11-25T04:22:47.036552+0000","last_unstale":"2023-11-25T04:22:48.047887+0000","last_undegraded":"2023-11-25T04:22:48.047887+0000","last_fullsized":"2023-11-25T04:22:48.047887+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:38:09.176582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048295+0000","last_change":"2023-11-25T04:22:46.039176+0000","last_active":"2023-11-25T04:22:48.048295+0000","last_peered":"2023-11-25T04:22:48.048295+0000","last_clean":"2023-11-25T04:22:48.048295+0000","last_became_active":"2023-11-25T04:22:46.038842+0000","last_became_peered":"2023-11-25T04:22:46.038842+0000","last_unstale":"2023-11-25T04:22:48.048295+0000","last_undegraded":"2023-11-25T04:22:48.048295+0000","last_fullsized":"2023-11-25T04:22:48.048295+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:47:56.298293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047733+0000","last_change":"2023-11-25T04:22:47.042155+0000","last_active":"2023-11-25T04:22:48.047733+0000","last_peered":"2023-11-25T04:22:48.047733+0000","last_clean":"2023-11-25T04:22:48.047733+0000","last_became_active":"2023-11-25T04:22:47.041433+0000","last_became_peered":"2023-11-25T04:22:47.041433+0000","last_unstale":"2023-11-25T04:22:48.047733+0000","last_undegraded":"2023-11-25T04:22:48.047733+0000","last_fullsized":"2023-11-25T04:22:48.047733+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:57:28.855151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"18'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048201+0000","last_change":"2023-11-25T04:22:46.037827+0000","last_active":"2023-11-25T04:22:48.048201+0000","last_peered":"2023-11-25T04:22:48.048201+0000","last_clean":"2023-11-25T04:22:48.048201+0000","last_became_active":"2023-11-25T04:22:46.036515+0000","last_became_peered":"2023-11-25T04:22:46.036515+0000","last_unstale":"2023-11-25T04:22:48.048201+0000","last_undegraded":"2023-11-25T04:22:48.048201+0000","last_fullsized":"2023-11-25T04:22:48.048201+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:10:20.178136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048154+0000","last_change":"2023-11-25T04:22:47.038856+0000","last_active":"2023-11-25T04:22:48.048154+0000","last_peered":"2023-11-25T04:22:48.048154+0000","last_clean":"2023-11-25T04:22:48.048154+0000","last_became_active":"2023-11-25T04:22:47.038190+0000","last_became_peered":"2023-11-25T04:22:47.038190+0000","last_unstale":"2023-11-25T04:22:48.048154+0000","last_undegraded":"2023-11-25T04:22:48.048154+0000","last_fullsized":"2023-11-25T04:22:48.048154+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:29:37.268624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047789+0000","last_change":"2023-11-25T04:22:46.038473+0000","last_active":"2023-11-25T04:22:48.047789+0000","last_peered":"2023-11-25T04:22:48.047789+0000","last_clean":"2023-11-25T04:22:48.047789+0000","last_became_active":"2023-11-25T04:22:46.038159+0000","last_became_peered":"2023-11-25T04:22:46.038159+0000","last_unstale":"2023-11-25T04:22:48.047789+0000","last_undegraded":"2023-11-25T04:22:48.047789+0000","last_fullsized":"2023-11-25T04:22:48.047789+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:31:53.847807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048568+0000","last_change":"2023-11-25T04:22:47.036222+0000","last_active":"2023-11-25T04:22:48.048568+0000","last_peered":"2023-11-25T04:22:48.048568+0000","last_clean":"2023-11-25T04:22:48.048568+0000","last_became_active":"2023-11-25T04:22:47.035871+0000","last_became_peered":"2023-11-25T04:22:47.035871+0000","last_unstale":"2023-11-25T04:22:48.048568+0000","last_undegraded":"2023-11-25T04:22:48.048568+0000","last_fullsized":"2023-11-25T04:22:48.048568+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:13:35.199538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047681+0000","last_change":"2023-11-25T04:22:46.035907+0000","last_active":"2023-11-25T04:22:48.047681+0000","last_peered":"2023-11-25T04:22:48.047681+0000","last_clean":"2023-11-25T04:22:48.047681+0000","last_became_active":"2023-11-25T04:22:46.035612+0000","last_became_peered":"2023-11-25T04:22:46.035612+0000","last_unstale":"2023-11-25T04:22:48.047681+0000","last_undegraded":"2023-11-25T04:22:48.047681+0000","last_fullsized":"2023-11-25T04:22:48.047681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:00:35.249875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048441+0000","last_change":"2023-11-25T04:22:47.038725+0000","last_active":"2023-11-25T04:22:48.048441+0000","last_peered":"2023-11-25T04:22:48.048441+0000","last_clean":"2023-11-25T04:22:48.048441+0000","last_became_active":"2023-11-25T04:22:47.038124+0000","last_became_peered":"2023-11-25T04:22:47.038124+0000","last_unstale":"2023-11-25T04:22:48.048441+0000","last_undegraded":"2023-11-25T04:22:48.048441+0000","last_fullsized":"2023-11-25T04:22:48.048441+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:00:26.187878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047437+0000","last_change":"2023-11-25T04:22:46.034509+0000","last_active":"2023-11-25T04:22:48.047437+0000","last_peered":"2023-11-25T04:22:48.047437+0000","last_clean":"2023-11-25T04:22:48.047437+0000","last_became_active":"2023-11-25T04:22:46.034158+0000","last_became_peered":"2023-11-25T04:22:46.034158+0000","last_unstale":"2023-11-25T04:22:48.047437+0000","last_undegraded":"2023-11-25T04:22:48.047437+0000","last_fullsized":"2023-11-25T04:22:48.047437+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:26:25.338117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048005+0000","last_change":"2023-11-25T04:22:47.038800+0000","last_active":"2023-11-25T04:22:48.048005+0000","last_peered":"2023-11-25T04:22:48.048005+0000","last_clean":"2023-11-25T04:22:48.048005+0000","last_became_active":"2023-11-25T04:22:47.038081+0000","last_became_peered":"2023-11-25T04:22:47.038081+0000","last_unstale":"2023-11-25T04:22:48.048005+0000","last_undegraded":"2023-11-25T04:22:48.048005+0000","last_fullsized":"2023-11-25T04:22:48.048005+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:45:40.065146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"12'157","reported_seq":214,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-25T04:22:47.033945+0000","last_change":"2023-11-25T04:22:36.931828+0000","last_active":"2023-11-25T04:22:47.033945+0000","last_peered":"2023-11-25T04:22:47.033945+0000","last_clean":"2023-11-25T04:22:47.033945+0000","last_became_active":"2023-11-25T04:22:36.931582+0000","last_became_peered":"2023-11-25T04:22:36.931582+0000","last_unstale":"2023-11-25T04:22:47.033945+0000","last_undegraded":"2023-11-25T04:22:47.033945+0000","last_fullsized":"2023-11-25T04:22:47.033945+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:35.919850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:35.919850+0000","last_clean_scrub_stamp":"2023-11-25T04:22:35.919850+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:59:59.198309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-25T04:22:47.031925+0000","last_change":"2023-11-25T04:22:41.999590+0000","last_active":"2023-11-25T04:22:47.031925+0000","last_peered":"2023-11-25T04:22:47.031925+0000","last_clean":"2023-11-25T04:22:47.031925+0000","last_became_active":"2023-11-25T04:22:41.999314+0000","last_became_peered":"2023-11-25T04:22:41.999314+0000","last_unstale":"2023-11-25T04:22:47.031925+0000","last_undegraded":"2023-11-25T04:22:47.031925+0000","last_fullsized":"2023-11-25T04:22:47.031925+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:20:12.785425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047840+0000","last_change":"2023-11-25T04:22:46.037044+0000","last_active":"2023-11-25T04:22:48.047840+0000","last_peered":"2023-11-25T04:22:48.047840+0000","last_clean":"2023-11-25T04:22:48.047840+0000","last_became_active":"2023-11-25T04:22:46.035916+0000","last_became_peered":"2023-11-25T04:22:46.035916+0000","last_unstale":"2023-11-25T04:22:48.047840+0000","last_undegraded":"2023-11-25T04:22:48.047840+0000","last_fullsized":"2023-11-25T04:22:48.047840+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:38:20.501102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048611+0000","last_change":"2023-11-25T04:22:47.036222+0000","last_active":"2023-11-25T04:22:48.048611+0000","last_peered":"2023-11-25T04:22:48.048611+0000","last_clean":"2023-11-25T04:22:48.048611+0000","last_became_active":"2023-11-25T04:22:47.035484+0000","last_became_peered":"2023-11-25T04:22:47.035484+0000","last_unstale":"2023-11-25T04:22:48.048611+0000","last_undegraded":"2023-11-25T04:22:48.048611+0000","last_fullsized":"2023-11-25T04:22:48.048611+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:18:56.845916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-25T04:22:47.032801+0000","last_change":"2023-11-25T04:22:42.000257+0000","last_active":"2023-11-25T04:22:47.032801+0000","last_peered":"2023-11-25T04:22:47.032801+0000","last_clean":"2023-11-25T04:22:47.032801+0000","last_became_active":"2023-11-25T04:22:41.999969+0000","last_became_peered":"2023-11-25T04:22:41.999969+0000","last_unstale":"2023-11-25T04:22:47.032801+0000","last_undegraded":"2023-11-25T04:22:47.032801+0000","last_fullsized":"2023-11-25T04:22:47.032801+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:21:34.424091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047780+0000","last_change":"2023-11-25T04:22:46.036121+0000","last_active":"2023-11-25T04:22:48.047780+0000","last_peered":"2023-11-25T04:22:48.047780+0000","last_clean":"2023-11-25T04:22:48.047780+0000","last_became_active":"2023-11-25T04:22:46.035654+0000","last_became_peered":"2023-11-25T04:22:46.035654+0000","last_unstale":"2023-11-25T04:22:48.047780+0000","last_undegraded":"2023-11-25T04:22:48.047780+0000","last_fullsized":"2023-11-25T04:22:48.047780+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:27:31.901618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048273+0000","last_change":"2023-11-25T04:22:47.039583+0000","last_active":"2023-11-25T04:22:48.048273+0000","last_peered":"2023-11-25T04:22:48.048273+0000","last_clean":"2023-11-25T04:22:48.048273+0000","last_became_active":"2023-11-25T04:22:47.039197+0000","last_became_peered":"2023-11-25T04:22:47.039197+0000","last_unstale":"2023-11-25T04:22:48.048273+0000","last_undegraded":"2023-11-25T04:22:48.048273+0000","last_fullsized":"2023-11-25T04:22:48.048273+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:37:30.959147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-25T04:22:47.029239+0000","last_change":"2023-11-25T04:22:42.000382+0000","last_active":"2023-11-25T04:22:47.029239+0000","last_peered":"2023-11-25T04:22:47.029239+0000","last_clean":"2023-11-25T04:22:47.029239+0000","last_became_active":"2023-11-25T04:22:42.000041+0000","last_became_peered":"2023-11-25T04:22:42.000041+0000","last_unstale":"2023-11-25T04:22:47.029239+0000","last_undegraded":"2023-11-25T04:22:47.029239+0000","last_fullsized":"2023-11-25T04:22:47.029239+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:08:19.655864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047444+0000","last_change":"2023-11-25T04:22:46.037431+0000","last_active":"2023-11-25T04:22:48.047444+0000","last_peered":"2023-11-25T04:22:48.047444+0000","last_clean":"2023-11-25T04:22:48.047444+0000","last_became_active":"2023-11-25T04:22:46.037141+0000","last_became_peered":"2023-11-25T04:22:46.037141+0000","last_unstale":"2023-11-25T04:22:48.047444+0000","last_undegraded":"2023-11-25T04:22:48.047444+0000","last_fullsized":"2023-11-25T04:22:48.047444+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:40:42.500663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048182+0000","last_change":"2023-11-25T04:22:47.038454+0000","last_active":"2023-11-25T04:22:48.048182+0000","last_peered":"2023-11-25T04:22:48.048182+0000","last_clean":"2023-11-25T04:22:48.048182+0000","last_became_active":"2023-11-25T04:22:47.038046+0000","last_became_peered":"2023-11-25T04:22:47.038046+0000","last_unstale":"2023-11-25T04:22:48.048182+0000","last_undegraded":"2023-11-25T04:22:48.048182+0000","last_fullsized":"2023-11-25T04:22:48.048182+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:38:38.839941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-25T04:22:47.031534+0000","last_change":"2023-11-25T04:22:41.999424+0000","last_active":"2023-11-25T04:22:47.031534+0000","last_peered":"2023-11-25T04:22:47.031534+0000","last_clean":"2023-11-25T04:22:47.031534+0000","last_became_active":"2023-11-25T04:22:41.999179+0000","last_became_peered":"2023-11-25T04:22:41.999179+0000","last_unstale":"2023-11-25T04:22:47.031534+0000","last_undegraded":"2023-11-25T04:22:47.031534+0000","last_fullsized":"2023-11-25T04:22:47.031534+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:12:04.291696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048523+0000","last_change":"2023-11-25T04:22:46.038758+0000","last_active":"2023-11-25T04:22:48.048523+0000","last_peered":"2023-11-25T04:22:48.048523+0000","last_clean":"2023-11-25T04:22:48.048523+0000","last_became_active":"2023-11-25T04:22:46.038470+0000","last_became_peered":"2023-11-25T04:22:46.038470+0000","last_unstale":"2023-11-25T04:22:48.048523+0000","last_undegraded":"2023-11-25T04:22:48.048523+0000","last_fullsized":"2023-11-25T04:22:48.048523+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:49:01.490558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048075+0000","last_change":"2023-11-25T04:22:47.037355+0000","last_active":"2023-11-25T04:22:48.048075+0000","last_peered":"2023-11-25T04:22:48.048075+0000","last_clean":"2023-11-25T04:22:48.048075+0000","last_became_active":"2023-11-25T04:22:47.036924+0000","last_became_peered":"2023-11-25T04:22:47.036924+0000","last_unstale":"2023-11-25T04:22:48.048075+0000","last_undegraded":"2023-11-25T04:22:48.048075+0000","last_fullsized":"2023-11-25T04:22:48.048075+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:15:39.511043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048677+0000","last_change":"2023-11-25T04:22:46.039025+0000","last_active":"2023-11-25T04:22:48.048677+0000","last_peered":"2023-11-25T04:22:48.048677+0000","last_clean":"2023-11-25T04:22:48.048677+0000","last_became_active":"2023-11-25T04:22:46.038762+0000","last_became_peered":"2023-11-25T04:22:46.038762+0000","last_unstale":"2023-11-25T04:22:48.048677+0000","last_undegraded":"2023-11-25T04:22:48.048677+0000","last_fullsized":"2023-11-25T04:22:48.048677+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:44:47.544509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-25T04:22:47.029103+0000","last_change":"2023-11-25T04:22:42.000204+0000","last_active":"2023-11-25T04:22:47.029103+0000","last_peered":"2023-11-25T04:22:47.029103+0000","last_clean":"2023-11-25T04:22:47.029103+0000","last_became_active":"2023-11-25T04:22:41.999915+0000","last_became_peered":"2023-11-25T04:22:41.999915+0000","last_unstale":"2023-11-25T04:22:47.029103+0000","last_undegraded":"2023-11-25T04:22:47.029103+0000","last_fullsized":"2023-11-25T04:22:47.029103+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:07:32.457905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047409+0000","last_change":"2023-11-25T04:22:47.038119+0000","last_active":"2023-11-25T04:22:48.047409+0000","last_peered":"2023-11-25T04:22:48.047409+0000","last_clean":"2023-11-25T04:22:48.047409+0000","last_became_active":"2023-11-25T04:22:47.037732+0000","last_became_peered":"2023-11-25T04:22:47.037732+0000","last_unstale":"2023-11-25T04:22:48.047409+0000","last_undegraded":"2023-11-25T04:22:48.047409+0000","last_fullsized":"2023-11-25T04:22:48.047409+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:49:02.500022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048004+0000","last_change":"2023-11-25T04:22:46.035885+0000","last_active":"2023-11-25T04:22:48.048004+0000","last_peered":"2023-11-25T04:22:48.048004+0000","last_clean":"2023-11-25T04:22:48.048004+0000","last_became_active":"2023-11-25T04:22:46.035405+0000","last_became_peered":"2023-11-25T04:22:46.035405+0000","last_unstale":"2023-11-25T04:22:48.048004+0000","last_undegraded":"2023-11-25T04:22:48.048004+0000","last_fullsized":"2023-11-25T04:22:48.048004+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:53:47.398746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-25T04:22:47.032524+0000","last_change":"2023-11-25T04:22:41.999067+0000","last_active":"2023-11-25T04:22:47.032524+0000","last_peered":"2023-11-25T04:22:47.032524+0000","last_clean":"2023-11-25T04:22:47.032524+0000","last_became_active":"2023-11-25T04:22:41.998798+0000","last_became_peered":"2023-11-25T04:22:41.998798+0000","last_unstale":"2023-11-25T04:22:47.032524+0000","last_undegraded":"2023-11-25T04:22:47.032524+0000","last_fullsized":"2023-11-25T04:22:47.032524+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:00:04.266051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047617+0000","last_change":"2023-11-25T04:22:47.043464+0000","last_active":"2023-11-25T04:22:48.047617+0000","last_peered":"2023-11-25T04:22:48.047617+0000","last_clean":"2023-11-25T04:22:48.047617+0000","last_became_active":"2023-11-25T04:22:47.043050+0000","last_became_peered":"2023-11-25T04:22:47.043050+0000","last_unstale":"2023-11-25T04:22:48.047617+0000","last_undegraded":"2023-11-25T04:22:48.047617+0000","last_fullsized":"2023-11-25T04:22:48.047617+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:59:45.492077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"18'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048417+0000","last_change":"2023-11-25T04:22:46.034606+0000","last_active":"2023-11-25T04:22:48.048417+0000","last_peered":"2023-11-25T04:22:48.048417+0000","last_clean":"2023-11-25T04:22:48.048417+0000","last_became_active":"2023-11-25T04:22:46.034229+0000","last_became_peered":"2023-11-25T04:22:46.034229+0000","last_unstale":"2023-11-25T04:22:48.048417+0000","last_undegraded":"2023-11-25T04:22:48.048417+0000","last_fullsized":"2023-11-25T04:22:48.048417+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:21:42.146264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-25T04:22:47.031216+0000","last_change":"2023-11-25T04:22:42.000258+0000","last_active":"2023-11-25T04:22:47.031216+0000","last_peered":"2023-11-25T04:22:47.031216+0000","last_clean":"2023-11-25T04:22:47.031216+0000","last_became_active":"2023-11-25T04:22:41.999992+0000","last_became_peered":"2023-11-25T04:22:41.999992+0000","last_unstale":"2023-11-25T04:22:47.031216+0000","last_undegraded":"2023-11-25T04:22:47.031216+0000","last_fullsized":"2023-11-25T04:22:47.031216+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:31:13.715992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047510+0000","last_change":"2023-11-25T04:22:47.042860+0000","last_active":"2023-11-25T04:22:48.047510+0000","last_peered":"2023-11-25T04:22:48.047510+0000","last_clean":"2023-11-25T04:22:48.047510+0000","last_became_active":"2023-11-25T04:22:47.042472+0000","last_became_peered":"2023-11-25T04:22:47.042472+0000","last_unstale":"2023-11-25T04:22:48.047510+0000","last_undegraded":"2023-11-25T04:22:48.047510+0000","last_fullsized":"2023-11-25T04:22:48.047510+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:44:48.553972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048418+0000","last_change":"2023-11-25T04:22:46.038417+0000","last_active":"2023-11-25T04:22:48.048418+0000","last_peered":"2023-11-25T04:22:48.048418+0000","last_clean":"2023-11-25T04:22:48.048418+0000","last_became_active":"2023-11-25T04:22:46.037513+0000","last_became_peered":"2023-11-25T04:22:46.037513+0000","last_unstale":"2023-11-25T04:22:48.048418+0000","last_undegraded":"2023-11-25T04:22:48.048418+0000","last_fullsized":"2023-11-25T04:22:48.048418+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:58:14.877413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-25T04:22:47.030950+0000","last_change":"2023-11-25T04:22:42.000217+0000","last_active":"2023-11-25T04:22:47.030950+0000","last_peered":"2023-11-25T04:22:47.030950+0000","last_clean":"2023-11-25T04:22:47.030950+0000","last_became_active":"2023-11-25T04:22:41.999987+0000","last_became_peered":"2023-11-25T04:22:41.999987+0000","last_unstale":"2023-11-25T04:22:47.030950+0000","last_undegraded":"2023-11-25T04:22:47.030950+0000","last_fullsized":"2023-11-25T04:22:47.030950+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:11:51.100459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047385+0000","last_change":"2023-11-25T04:22:47.036099+0000","last_active":"2023-11-25T04:22:48.047385+0000","last_peered":"2023-11-25T04:22:48.047385+0000","last_clean":"2023-11-25T04:22:48.047385+0000","last_became_active":"2023-11-25T04:22:47.035581+0000","last_became_peered":"2023-11-25T04:22:47.035581+0000","last_unstale":"2023-11-25T04:22:48.047385+0000","last_undegraded":"2023-11-25T04:22:48.047385+0000","last_fullsized":"2023-11-25T04:22:48.047385+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:21:43.155728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048171+0000","last_change":"2023-11-25T04:22:47.040343+0000","last_active":"2023-11-25T04:22:48.048171+0000","last_peered":"2023-11-25T04:22:48.048171+0000","last_clean":"2023-11-25T04:22:48.048171+0000","last_became_active":"2023-11-25T04:22:47.039151+0000","last_became_peered":"2023-11-25T04:22:47.039151+0000","last_unstale":"2023-11-25T04:22:48.048171+0000","last_undegraded":"2023-11-25T04:22:48.048171+0000","last_fullsized":"2023-11-25T04:22:48.048171+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:21:15.981901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"18'6","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048260+0000","last_change":"2023-11-25T04:22:46.040005+0000","last_active":"2023-11-25T04:22:48.048260+0000","last_peered":"2023-11-25T04:22:48.048260+0000","last_clean":"2023-11-25T04:22:48.048260+0000","last_became_active":"2023-11-25T04:22:46.039655+0000","last_became_peered":"2023-11-25T04:22:46.039655+0000","last_unstale":"2023-11-25T04:22:48.048260+0000","last_undegraded":"2023-11-25T04:22:48.048260+0000","last_fullsized":"2023-11-25T04:22:48.048260+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:41:15.662804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047237+0000","last_change":"2023-11-25T04:22:47.037181+0000","last_active":"2023-11-25T04:22:48.047237+0000","last_peered":"2023-11-25T04:22:48.047237+0000","last_clean":"2023-11-25T04:22:48.047237+0000","last_became_active":"2023-11-25T04:22:47.036821+0000","last_became_peered":"2023-11-25T04:22:47.036821+0000","last_unstale":"2023-11-25T04:22:48.047237+0000","last_undegraded":"2023-11-25T04:22:48.047237+0000","last_fullsized":"2023-11-25T04:22:48.047237+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:12:51.990448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"18'3","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048375+0000","last_change":"2023-11-25T04:22:46.035816+0000","last_active":"2023-11-25T04:22:48.048375+0000","last_peered":"2023-11-25T04:22:48.048375+0000","last_clean":"2023-11-25T04:22:48.048375+0000","last_became_active":"2023-11-25T04:22:46.035461+0000","last_became_peered":"2023-11-25T04:22:46.035461+0000","last_unstale":"2023-11-25T04:22:48.048375+0000","last_undegraded":"2023-11-25T04:22:48.048375+0000","last_fullsized":"2023-11-25T04:22:48.048375+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:58:26.155063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048370+0000","last_change":"2023-11-25T04:22:46.040458+0000","last_active":"2023-11-25T04:22:48.048370+0000","last_peered":"2023-11-25T04:22:48.048370+0000","last_clean":"2023-11-25T04:22:48.048370+0000","last_became_active":"2023-11-25T04:22:46.040143+0000","last_became_peered":"2023-11-25T04:22:46.040143+0000","last_unstale":"2023-11-25T04:22:48.048370+0000","last_undegraded":"2023-11-25T04:22:48.048370+0000","last_fullsized":"2023-11-25T04:22:48.048370+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:27:31.901618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047387+0000","last_change":"2023-11-25T04:22:47.040346+0000","last_active":"2023-11-25T04:22:48.047387+0000","last_peered":"2023-11-25T04:22:48.047387+0000","last_clean":"2023-11-25T04:22:48.047387+0000","last_became_active":"2023-11-25T04:22:47.040084+0000","last_became_peered":"2023-11-25T04:22:47.040084+0000","last_unstale":"2023-11-25T04:22:48.047387+0000","last_undegraded":"2023-11-25T04:22:48.047387+0000","last_fullsized":"2023-11-25T04:22:48.047387+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:59:45.492077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048308+0000","last_change":"2023-11-25T04:22:46.038721+0000","last_active":"2023-11-25T04:22:48.048308+0000","last_peered":"2023-11-25T04:22:48.048308+0000","last_clean":"2023-11-25T04:22:48.048308+0000","last_became_active":"2023-11-25T04:22:46.037698+0000","last_became_peered":"2023-11-25T04:22:46.037698+0000","last_unstale":"2023-11-25T04:22:48.048308+0000","last_undegraded":"2023-11-25T04:22:48.048308+0000","last_fullsized":"2023-11-25T04:22:48.048308+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:25:20.721529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047199+0000","last_change":"2023-11-25T04:22:47.037790+0000","last_active":"2023-11-25T04:22:48.047199+0000","last_peered":"2023-11-25T04:22:48.047199+0000","last_clean":"2023-11-25T04:22:48.047199+0000","last_became_active":"2023-11-25T04:22:47.037149+0000","last_became_peered":"2023-11-25T04:22:47.037149+0000","last_unstale":"2023-11-25T04:22:48.047199+0000","last_undegraded":"2023-11-25T04:22:48.047199+0000","last_fullsized":"2023-11-25T04:22:48.047199+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:31:54.857270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048584+0000","last_change":"2023-11-25T04:22:47.036079+0000","last_active":"2023-11-25T04:22:48.048584+0000","last_peered":"2023-11-25T04:22:48.048584+0000","last_clean":"2023-11-25T04:22:48.048584+0000","last_became_active":"2023-11-25T04:22:47.035625+0000","last_became_peered":"2023-11-25T04:22:47.035625+0000","last_unstale":"2023-11-25T04:22:48.048584+0000","last_undegraded":"2023-11-25T04:22:48.048584+0000","last_fullsized":"2023-11-25T04:22:48.048584+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:21:15.981901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047749+0000","last_change":"2023-11-25T04:22:46.036085+0000","last_active":"2023-11-25T04:22:48.047749+0000","last_peered":"2023-11-25T04:22:48.047749+0000","last_clean":"2023-11-25T04:22:48.047749+0000","last_became_active":"2023-11-25T04:22:46.035495+0000","last_became_peered":"2023-11-25T04:22:46.035495+0000","last_unstale":"2023-11-25T04:22:48.047749+0000","last_undegraded":"2023-11-25T04:22:48.047749+0000","last_fullsized":"2023-11-25T04:22:48.047749+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:14:56.735365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048251+0000","last_change":"2023-11-25T04:22:47.043064+0000","last_active":"2023-11-25T04:22:48.048251+0000","last_peered":"2023-11-25T04:22:48.048251+0000","last_clean":"2023-11-25T04:22:48.048251+0000","last_became_active":"2023-11-25T04:22:47.042760+0000","last_became_peered":"2023-11-25T04:22:47.042760+0000","last_unstale":"2023-11-25T04:22:48.048251+0000","last_undegraded":"2023-11-25T04:22:48.048251+0000","last_fullsized":"2023-11-25T04:22:48.048251+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:37:30.959147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047514+0000","last_change":"2023-11-25T04:22:46.038298+0000","last_active":"2023-11-25T04:22:48.047514+0000","last_peered":"2023-11-25T04:22:48.047514+0000","last_clean":"2023-11-25T04:22:48.047514+0000","last_became_active":"2023-11-25T04:22:46.036629+0000","last_became_peered":"2023-11-25T04:22:46.036629+0000","last_unstale":"2023-11-25T04:22:48.047514+0000","last_undegraded":"2023-11-25T04:22:48.047514+0000","last_fullsized":"2023-11-25T04:22:48.047514+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:45:14.670326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048209+0000","last_change":"2023-11-25T04:22:47.043156+0000","last_active":"2023-11-25T04:22:48.048209+0000","last_peered":"2023-11-25T04:22:48.048209+0000","last_clean":"2023-11-25T04:22:48.048209+0000","last_became_active":"2023-11-25T04:22:47.042693+0000","last_became_peered":"2023-11-25T04:22:47.042693+0000","last_unstale":"2023-11-25T04:22:48.048209+0000","last_undegraded":"2023-11-25T04:22:48.048209+0000","last_fullsized":"2023-11-25T04:22:48.048209+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:38:38.839941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1a","version":"18'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047866+0000","last_change":"2023-11-25T04:22:46.034340+0000","last_active":"2023-11-25T04:22:48.047866+0000","last_peered":"2023-11-25T04:22:48.047866+0000","last_clean":"2023-11-25T04:22:48.047866+0000","last_became_active":"2023-11-25T04:22:46.033944+0000","last_became_peered":"2023-11-25T04:22:46.033944+0000","last_unstale":"2023-11-25T04:22:48.047866+0000","last_undegraded":"2023-11-25T04:22:48.047866+0000","last_fullsized":"2023-11-25T04:22:48.047866+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:59:30.291004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047694+0000","last_change":"2023-11-25T04:22:47.039547+0000","last_active":"2023-11-25T04:22:48.047694+0000","last_peered":"2023-11-25T04:22:48.047694+0000","last_clean":"2023-11-25T04:22:48.047694+0000","last_became_active":"2023-11-25T04:22:47.039218+0000","last_became_peered":"2023-11-25T04:22:47.039218+0000","last_unstale":"2023-11-25T04:22:48.047694+0000","last_undegraded":"2023-11-25T04:22:48.047694+0000","last_fullsized":"2023-11-25T04:22:48.047694+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:44:48.553972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"18'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047860+0000","last_change":"2023-11-25T04:22:46.035882+0000","last_active":"2023-11-25T04:22:48.047860+0000","last_peered":"2023-11-25T04:22:48.047860+0000","last_clean":"2023-11-25T04:22:48.047860+0000","last_became_active":"2023-11-25T04:22:46.035588+0000","last_became_peered":"2023-11-25T04:22:46.035588+0000","last_unstale":"2023-11-25T04:22:48.047860+0000","last_undegraded":"2023-11-25T04:22:48.047860+0000","last_fullsized":"2023-11-25T04:22:48.047860+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:43:10.702849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047831+0000","last_change":"2023-11-25T04:22:47.041267+0000","last_active":"2023-11-25T04:22:48.047831+0000","last_peered":"2023-11-25T04:22:48.047831+0000","last_clean":"2023-11-25T04:22:48.047831+0000","last_became_active":"2023-11-25T04:22:47.040408+0000","last_became_peered":"2023-11-25T04:22:47.040408+0000","last_unstale":"2023-11-25T04:22:48.047831+0000","last_undegraded":"2023-11-25T04:22:48.047831+0000","last_fullsized":"2023-11-25T04:22:48.047831+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:37:30.959147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"18'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048131+0000","last_change":"2023-11-25T04:22:46.040793+0000","last_active":"2023-11-25T04:22:48.048131+0000","last_peered":"2023-11-25T04:22:48.048131+0000","last_clean":"2023-11-25T04:22:48.048131+0000","last_became_active":"2023-11-25T04:22:46.040493+0000","last_became_peered":"2023-11-25T04:22:46.040493+0000","last_unstale":"2023-11-25T04:22:48.048131+0000","last_undegraded":"2023-11-25T04:22:48.048131+0000","last_fullsized":"2023-11-25T04:22:48.048131+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:58:26.155063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048056+0000","last_change":"2023-11-25T04:22:47.040067+0000","last_active":"2023-11-25T04:22:48.048056+0000","last_peered":"2023-11-25T04:22:48.048056+0000","last_clean":"2023-11-25T04:22:48.048056+0000","last_became_active":"2023-11-25T04:22:47.038425+0000","last_became_peered":"2023-11-25T04:22:47.038425+0000","last_unstale":"2023-11-25T04:22:48.048056+0000","last_undegraded":"2023-11-25T04:22:48.048056+0000","last_fullsized":"2023-11-25T04:22:48.048056+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:25:06.132517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"18'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048297+0000","last_change":"2023-11-25T04:22:46.035872+0000","last_active":"2023-11-25T04:22:48.048297+0000","last_peered":"2023-11-25T04:22:48.048297+0000","last_clean":"2023-11-25T04:22:48.048297+0000","last_became_active":"2023-11-25T04:22:46.035575+0000","last_became_peered":"2023-11-25T04:22:46.035575+0000","last_unstale":"2023-11-25T04:22:48.048297+0000","last_undegraded":"2023-11-25T04:22:48.048297+0000","last_fullsized":"2023-11-25T04:22:48.048297+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:58:14.877413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047725+0000","last_change":"2023-11-25T04:22:47.036764+0000","last_active":"2023-11-25T04:22:48.047725+0000","last_peered":"2023-11-25T04:22:48.047725+0000","last_clean":"2023-11-25T04:22:48.047725+0000","last_became_active":"2023-11-25T04:22:47.036406+0000","last_became_peered":"2023-11-25T04:22:47.036406+0000","last_unstale":"2023-11-25T04:22:48.047725+0000","last_undegraded":"2023-11-25T04:22:48.047725+0000","last_fullsized":"2023-11-25T04:22:48.047725+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:09:55.357642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1e","version":"18'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048335+0000","last_change":"2023-11-25T04:22:46.035659+0000","last_active":"2023-11-25T04:22:48.048335+0000","last_peered":"2023-11-25T04:22:48.048335+0000","last_clean":"2023-11-25T04:22:48.048335+0000","last_became_active":"2023-11-25T04:22:46.035113+0000","last_became_peered":"2023-11-25T04:22:46.035113+0000","last_unstale":"2023-11-25T04:22:48.048335+0000","last_undegraded":"2023-11-25T04:22:48.048335+0000","last_fullsized":"2023-11-25T04:22:48.048335+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:26:25.338117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048474+0000","last_change":"2023-11-25T04:22:47.041554+0000","last_active":"2023-11-25T04:22:48.048474+0000","last_peered":"2023-11-25T04:22:48.048474+0000","last_clean":"2023-11-25T04:22:48.048474+0000","last_became_active":"2023-11-25T04:22:47.040314+0000","last_became_peered":"2023-11-25T04:22:47.040314+0000","last_unstale":"2023-11-25T04:22:48.048474+0000","last_undegraded":"2023-11-25T04:22:48.048474+0000","last_fullsized":"2023-11-25T04:22:48.048474+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:24:56.315251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"18'4","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048204+0000","last_change":"2023-11-25T04:22:46.038702+0000","last_active":"2023-11-25T04:22:48.048204+0000","last_peered":"2023-11-25T04:22:48.048204+0000","last_clean":"2023-11-25T04:22:48.048204+0000","last_became_active":"2023-11-25T04:22:46.037567+0000","last_became_peered":"2023-11-25T04:22:46.037567+0000","last_unstale":"2023-11-25T04:22:48.048204+0000","last_undegraded":"2023-11-25T04:22:48.048204+0000","last_fullsized":"2023-11-25T04:22:48.048204+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:58:26.155063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738374,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35508,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336332,"statfs":{"total":96636764160,"available":96600403968,"internally_reserved":0,"allocated":172032,"data_stored":58325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39500000000000002}]}]},{"osd":1,"up_from":8,"seq":34359738374,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31944,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339896,"statfs":{"total":96636764160,"available":96604053504,"internally_reserved":0,"allocated":782336,"data_stored":649871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738373,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31952,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339888,"statfs":{"total":96636764160,"available":96604045312,"internally_reserved":0,"allocated":782336,"data_stored":645884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-25T04:22:50.391 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-25T04:22:50.391 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-11-25T04:22:50.392 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-11-25T04:22:50.392 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-11-25T04:22:50.392 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-11-25T04:22:50.772 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:22:50.783 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":5,"active_gid":4106,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1552268534},{"type":"v1","addr":"172.21.15.67:6801","nonce":1552268534}]},"active_addr":"172.21.15.67:6801/1552268534","active_change":"2023-11-25T04:22:29.484351+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4110,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.67:0","nonce":1113824879}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.67:0","nonce":3776965466}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.67:0","nonce":3633418397}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.67:0","nonce":2447609085}]}]} 2023-11-25T04:22:50.788 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-11-25T04:22:50.788 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-11-25T04:22:50.788 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-25T04:22:51.083 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:22:51.083 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":19,"fsid":"a677f511-1143-4e76-8da6-5ddfadaff077","created":"2023-11-25T04:22:25.140474+0000","modified":"2023-11-25T04:22:48.032728+0000","last_up_change":"2023-11-25T04:22:33.677691+0000","last_in_change":"2023-11-25T04:22:25.985372+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-25T04:22:35.548359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-25T04:22:40.218527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-25T04:22:44.844906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-25T04:22:45.365536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bc397d35-53ea-430c-a96a-63a4160e64c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":181847435},{"type":"v1","addr":"172.21.15.76:6801","nonce":181847435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":181847435},{"type":"v1","addr":"172.21.15.76:6803","nonce":181847435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":181847435},{"type":"v1","addr":"172.21.15.76:6807","nonce":181847435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":181847435},{"type":"v1","addr":"172.21.15.76:6805","nonce":181847435}]},"public_addr":"172.21.15.76:6801/181847435","cluster_addr":"172.21.15.76:6803/181847435","heartbeat_back_addr":"172.21.15.76:6807/181847435","heartbeat_front_addr":"172.21.15.76:6805/181847435","state":["exists","up"]},{"osd":1,"uuid":"d4f1b88a-bbd4-4e58-9519-569489ced7b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6801","nonce":3130474108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6803","nonce":3130474108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6807","nonce":3130474108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6805","nonce":3130474108}]},"public_addr":"172.21.15.92:6801/3130474108","cluster_addr":"172.21.15.92:6803/3130474108","heartbeat_back_addr":"172.21.15.92:6807/3130474108","heartbeat_front_addr":"172.21.15.92:6805/3130474108","state":["exists","up"]},{"osd":2,"uuid":"55c82801-e090-4c10-91e5-5fa9dbfba36d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6801","nonce":4099978417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6803","nonce":4099978417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6807","nonce":4099978417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6805","nonce":4099978417}]},"public_addr":"172.21.15.195:6801/4099978417","cluster_addr":"172.21.15.195:6803/4099978417","heartbeat_back_addr":"172.21.15.195:6807/4099978417","heartbeat_front_addr":"172.21.15.195:6805/4099978417","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-25T04:22:31.039867+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-25T04:22:31.021863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-25T04:22:31.009821+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-25T04:22:51.095 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-11-25T04:22:51.096 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-25T04:22:51.393 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:22:51.394 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":19,"fsid":"a677f511-1143-4e76-8da6-5ddfadaff077","created":"2023-11-25T04:22:25.140474+0000","modified":"2023-11-25T04:22:48.032728+0000","last_up_change":"2023-11-25T04:22:33.677691+0000","last_in_change":"2023-11-25T04:22:25.985372+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-25T04:22:35.548359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-25T04:22:40.218527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-25T04:22:44.844906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-25T04:22:45.365536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bc397d35-53ea-430c-a96a-63a4160e64c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":181847435},{"type":"v1","addr":"172.21.15.76:6801","nonce":181847435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":181847435},{"type":"v1","addr":"172.21.15.76:6803","nonce":181847435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":181847435},{"type":"v1","addr":"172.21.15.76:6807","nonce":181847435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":181847435},{"type":"v1","addr":"172.21.15.76:6805","nonce":181847435}]},"public_addr":"172.21.15.76:6801/181847435","cluster_addr":"172.21.15.76:6803/181847435","heartbeat_back_addr":"172.21.15.76:6807/181847435","heartbeat_front_addr":"172.21.15.76:6805/181847435","state":["exists","up"]},{"osd":1,"uuid":"d4f1b88a-bbd4-4e58-9519-569489ced7b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6801","nonce":3130474108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6803","nonce":3130474108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6807","nonce":3130474108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6805","nonce":3130474108}]},"public_addr":"172.21.15.92:6801/3130474108","cluster_addr":"172.21.15.92:6803/3130474108","heartbeat_back_addr":"172.21.15.92:6807/3130474108","heartbeat_front_addr":"172.21.15.92:6805/3130474108","state":["exists","up"]},{"osd":2,"uuid":"55c82801-e090-4c10-91e5-5fa9dbfba36d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6801","nonce":4099978417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6803","nonce":4099978417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6807","nonce":4099978417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6805","nonce":4099978417}]},"public_addr":"172.21.15.195:6801/4099978417","cluster_addr":"172.21.15.195:6803/4099978417","heartbeat_back_addr":"172.21.15.195:6807/4099978417","heartbeat_front_addr":"172.21.15.195:6805/4099978417","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-25T04:22:31.039867+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-25T04:22:31.021863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-25T04:22:31.009821+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-25T04:22:51.406 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-11-25T04:22:51.407 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-11-25T04:22:51.407 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-11-25T04:22:51.553 INFO:teuthology.orchestra.run.smithi067.stdout:34359738375 2023-11-25T04:22:51.553 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-25T04:22:51.574 INFO:teuthology.orchestra.run.smithi067.stdout:34359738375 2023-11-25T04:22:51.575 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-25T04:22:51.575 INFO:teuthology.orchestra.run.smithi067.stdout:34359738374 2023-11-25T04:22:51.576 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-25T04:22:51.860 INFO:teuthology.orchestra.run.smithi067.stdout:34359738374 2023-11-25T04:22:51.873 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738375 got 34359738374 for osd.1 2023-11-25T04:22:51.920 INFO:teuthology.orchestra.run.smithi067.stdout:34359738374 2023-11-25T04:22:51.920 INFO:teuthology.orchestra.run.smithi067.stdout:34359738373 2023-11-25T04:22:51.932 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738375 got 34359738374 for osd.2 2023-11-25T04:22:51.934 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738374 got 34359738373 for osd.0 2023-11-25T04:22:52.123 INFO:tasks.ceph.mds.a.smithi067.stderr:2023-11-25T04:22:52.121+0000 7fae25c33640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-25T04:22:52.874 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-25T04:22:52.932 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-25T04:22:52.934 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-25T04:22:53.207 INFO:teuthology.orchestra.run.smithi067.stdout:34359738374 2023-11-25T04:22:53.220 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738375 got 34359738374 for osd.1 2023-11-25T04:22:53.260 INFO:teuthology.orchestra.run.smithi067.stdout:34359738374 2023-11-25T04:22:53.272 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738375 got 34359738374 for osd.2 2023-11-25T04:22:53.273 INFO:teuthology.orchestra.run.smithi067.stdout:34359738373 2023-11-25T04:22:53.286 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738374 got 34359738373 for osd.0 2023-11-25T04:22:54.221 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-25T04:22:54.273 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-25T04:22:54.287 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-25T04:22:54.539 INFO:teuthology.orchestra.run.smithi067.stdout:34359738375 2023-11-25T04:22:54.555 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738375 got 34359738375 for osd.1 2023-11-25T04:22:54.556 DEBUG:teuthology.parallel:result is None 2023-11-25T04:22:54.617 INFO:teuthology.orchestra.run.smithi067.stdout:34359738375 2023-11-25T04:22:54.630 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738375 got 34359738375 for osd.2 2023-11-25T04:22:54.631 DEBUG:teuthology.parallel:result is None 2023-11-25T04:22:54.659 INFO:teuthology.orchestra.run.smithi067.stdout:34359738374 2023-11-25T04:22:54.672 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738374 got 34359738374 for osd.0 2023-11-25T04:22:54.672 DEBUG:teuthology.parallel:result is None 2023-11-25T04:22:54.672 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-25T04:22:54.673 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-25T04:22:55.013 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:22:55.014 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2023-11-25T04:22:55.023 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-11-25T04:22:53.494202+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":247,"num_write_kb":3474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":195,"ondisk_log_size":195,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":192,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":99404,"kb_used_data":1704,"kb_used_omap":4,"kb_used_meta":97659,"kb_avail":283016116,"statfs":{"total":289910292480,"available":289808502784,"internally_reserved":0,"allocated":1744896,"data_stored":1358683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":100003151},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.469702"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048437+0000","last_change":"2023-11-25T04:22:47.039847+0000","last_active":"2023-11-25T04:22:48.048437+0000","last_peered":"2023-11-25T04:22:48.048437+0000","last_clean":"2023-11-25T04:22:48.048437+0000","last_became_active":"2023-11-25T04:22:47.039490+0000","last_became_peered":"2023-11-25T04:22:47.039490+0000","last_unstale":"2023-11-25T04:22:48.048437+0000","last_undegraded":"2023-11-25T04:22:48.048437+0000","last_fullsized":"2023-11-25T04:22:48.048437+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:25:06.132517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048105+0000","last_change":"2023-11-25T04:22:47.040836+0000","last_active":"2023-11-25T04:22:48.048105+0000","last_peered":"2023-11-25T04:22:48.048105+0000","last_clean":"2023-11-25T04:22:48.048105+0000","last_became_active":"2023-11-25T04:22:47.040511+0000","last_became_peered":"2023-11-25T04:22:47.040511+0000","last_unstale":"2023-11-25T04:22:48.048105+0000","last_undegraded":"2023-11-25T04:22:48.048105+0000","last_fullsized":"2023-11-25T04:22:48.048105+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:15:39.511043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048297+0000","last_change":"2023-11-25T04:22:47.043529+0000","last_active":"2023-11-25T04:22:48.048297+0000","last_peered":"2023-11-25T04:22:48.048297+0000","last_clean":"2023-11-25T04:22:48.048297+0000","last_became_active":"2023-11-25T04:22:47.042733+0000","last_became_peered":"2023-11-25T04:22:47.042733+0000","last_unstale":"2023-11-25T04:22:48.048297+0000","last_undegraded":"2023-11-25T04:22:48.048297+0000","last_fullsized":"2023-11-25T04:22:48.048297+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:47:34.764521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047862+0000","last_change":"2023-11-25T04:22:47.040314+0000","last_active":"2023-11-25T04:22:48.047862+0000","last_peered":"2023-11-25T04:22:48.047862+0000","last_clean":"2023-11-25T04:22:48.047862+0000","last_became_active":"2023-11-25T04:22:47.039185+0000","last_became_peered":"2023-11-25T04:22:47.039185+0000","last_unstale":"2023-11-25T04:22:48.047862+0000","last_undegraded":"2023-11-25T04:22:48.047862+0000","last_fullsized":"2023-11-25T04:22:48.047862+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:16:29.085050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047841+0000","last_change":"2023-11-25T04:22:47.039499+0000","last_active":"2023-11-25T04:22:48.047841+0000","last_peered":"2023-11-25T04:22:48.047841+0000","last_clean":"2023-11-25T04:22:48.047841+0000","last_became_active":"2023-11-25T04:22:47.039165+0000","last_became_peered":"2023-11-25T04:22:47.039165+0000","last_unstale":"2023-11-25T04:22:48.047841+0000","last_undegraded":"2023-11-25T04:22:48.047841+0000","last_fullsized":"2023-11-25T04:22:48.047841+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:57:28.855151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047565+0000","last_change":"2023-11-25T04:22:47.039992+0000","last_active":"2023-11-25T04:22:48.047565+0000","last_peered":"2023-11-25T04:22:48.047565+0000","last_clean":"2023-11-25T04:22:48.047565+0000","last_became_active":"2023-11-25T04:22:47.039405+0000","last_became_peered":"2023-11-25T04:22:47.039405+0000","last_unstale":"2023-11-25T04:22:48.047565+0000","last_undegraded":"2023-11-25T04:22:48.047565+0000","last_fullsized":"2023-11-25T04:22:48.047565+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:56:59.689578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047396+0000","last_change":"2023-11-25T04:22:47.037971+0000","last_active":"2023-11-25T04:22:48.047396+0000","last_peered":"2023-11-25T04:22:48.047396+0000","last_clean":"2023-11-25T04:22:48.047396+0000","last_became_active":"2023-11-25T04:22:47.036736+0000","last_became_peered":"2023-11-25T04:22:47.036736+0000","last_unstale":"2023-11-25T04:22:48.047396+0000","last_undegraded":"2023-11-25T04:22:48.047396+0000","last_fullsized":"2023-11-25T04:22:48.047396+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:02:07.088241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047307+0000","last_change":"2023-11-25T04:22:47.041728+0000","last_active":"2023-11-25T04:22:48.047307+0000","last_peered":"2023-11-25T04:22:48.047307+0000","last_clean":"2023-11-25T04:22:48.047307+0000","last_became_active":"2023-11-25T04:22:47.041017+0000","last_became_peered":"2023-11-25T04:22:47.041017+0000","last_unstale":"2023-11-25T04:22:48.047307+0000","last_undegraded":"2023-11-25T04:22:48.047307+0000","last_fullsized":"2023-11-25T04:22:48.047307+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:57:28.855151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048097+0000","last_change":"2023-11-25T04:22:47.038397+0000","last_active":"2023-11-25T04:22:48.048097+0000","last_peered":"2023-11-25T04:22:48.048097+0000","last_clean":"2023-11-25T04:22:48.048097+0000","last_became_active":"2023-11-25T04:22:47.037328+0000","last_became_peered":"2023-11-25T04:22:47.037328+0000","last_unstale":"2023-11-25T04:22:48.048097+0000","last_undegraded":"2023-11-25T04:22:48.048097+0000","last_fullsized":"2023-11-25T04:22:48.048097+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:56:59.689578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048223+0000","last_change":"2023-11-25T04:22:47.040182+0000","last_active":"2023-11-25T04:22:48.048223+0000","last_peered":"2023-11-25T04:22:48.048223+0000","last_clean":"2023-11-25T04:22:48.048223+0000","last_became_active":"2023-11-25T04:22:47.039892+0000","last_became_peered":"2023-11-25T04:22:47.039892+0000","last_unstale":"2023-11-25T04:22:48.048223+0000","last_undegraded":"2023-11-25T04:22:48.048223+0000","last_fullsized":"2023-11-25T04:22:48.048223+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:38:09.176582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048202+0000","last_change":"2023-11-25T04:22:47.039907+0000","last_active":"2023-11-25T04:22:48.048202+0000","last_peered":"2023-11-25T04:22:48.048202+0000","last_clean":"2023-11-25T04:22:48.048202+0000","last_became_active":"2023-11-25T04:22:47.039046+0000","last_became_peered":"2023-11-25T04:22:47.039046+0000","last_unstale":"2023-11-25T04:22:48.048202+0000","last_undegraded":"2023-11-25T04:22:48.048202+0000","last_fullsized":"2023-11-25T04:22:48.048202+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:01:51.418774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047609+0000","last_change":"2023-11-25T04:22:47.037618+0000","last_active":"2023-11-25T04:22:48.047609+0000","last_peered":"2023-11-25T04:22:48.047609+0000","last_clean":"2023-11-25T04:22:48.047609+0000","last_became_active":"2023-11-25T04:22:47.036806+0000","last_became_peered":"2023-11-25T04:22:47.036806+0000","last_unstale":"2023-11-25T04:22:48.047609+0000","last_undegraded":"2023-11-25T04:22:48.047609+0000","last_fullsized":"2023-11-25T04:22:48.047609+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:12:51.990448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047608+0000","last_change":"2023-11-25T04:22:47.040388+0000","last_active":"2023-11-25T04:22:48.047608+0000","last_peered":"2023-11-25T04:22:48.047608+0000","last_clean":"2023-11-25T04:22:48.047608+0000","last_became_active":"2023-11-25T04:22:47.040129+0000","last_became_peered":"2023-11-25T04:22:47.040129+0000","last_unstale":"2023-11-25T04:22:48.047608+0000","last_undegraded":"2023-11-25T04:22:48.047608+0000","last_fullsized":"2023-11-25T04:22:48.047608+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:31:54.857270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047643+0000","last_change":"2023-11-25T04:22:47.038887+0000","last_active":"2023-11-25T04:22:48.047643+0000","last_peered":"2023-11-25T04:22:48.047643+0000","last_clean":"2023-11-25T04:22:48.047643+0000","last_became_active":"2023-11-25T04:22:47.038014+0000","last_became_peered":"2023-11-25T04:22:47.038014+0000","last_unstale":"2023-11-25T04:22:48.047643+0000","last_undegraded":"2023-11-25T04:22:48.047643+0000","last_fullsized":"2023-11-25T04:22:48.047643+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:38:38.839941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047471+0000","last_change":"2023-11-25T04:22:47.035843+0000","last_active":"2023-11-25T04:22:48.047471+0000","last_peered":"2023-11-25T04:22:48.047471+0000","last_clean":"2023-11-25T04:22:48.047471+0000","last_became_active":"2023-11-25T04:22:47.035453+0000","last_became_peered":"2023-11-25T04:22:47.035453+0000","last_unstale":"2023-11-25T04:22:48.047471+0000","last_undegraded":"2023-11-25T04:22:48.047471+0000","last_fullsized":"2023-11-25T04:22:48.047471+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:45:40.065146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047308+0000","last_change":"2023-11-25T04:22:47.041778+0000","last_active":"2023-11-25T04:22:48.047308+0000","last_peered":"2023-11-25T04:22:48.047308+0000","last_clean":"2023-11-25T04:22:48.047308+0000","last_became_active":"2023-11-25T04:22:47.041115+0000","last_became_peered":"2023-11-25T04:22:47.041115+0000","last_unstale":"2023-11-25T04:22:48.047308+0000","last_undegraded":"2023-11-25T04:22:48.047308+0000","last_fullsized":"2023-11-25T04:22:48.047308+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:30:04.568790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048422+0000","last_change":"2023-11-25T04:22:47.039389+0000","last_active":"2023-11-25T04:22:48.048422+0000","last_peered":"2023-11-25T04:22:48.048422+0000","last_clean":"2023-11-25T04:22:48.048422+0000","last_became_active":"2023-11-25T04:22:47.039123+0000","last_became_peered":"2023-11-25T04:22:47.039123+0000","last_unstale":"2023-11-25T04:22:48.048422+0000","last_undegraded":"2023-11-25T04:22:48.048422+0000","last_fullsized":"2023-11-25T04:22:48.048422+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:09:11.570561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048403+0000","last_change":"2023-11-25T04:22:47.036946+0000","last_active":"2023-11-25T04:22:48.048403+0000","last_peered":"2023-11-25T04:22:48.048403+0000","last_clean":"2023-11-25T04:22:48.048403+0000","last_became_active":"2023-11-25T04:22:47.036411+0000","last_became_peered":"2023-11-25T04:22:47.036411+0000","last_unstale":"2023-11-25T04:22:48.048403+0000","last_undegraded":"2023-11-25T04:22:48.048403+0000","last_fullsized":"2023-11-25T04:22:48.048403+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:16:29.085050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047729+0000","last_change":"2023-11-25T04:22:47.038983+0000","last_active":"2023-11-25T04:22:48.047729+0000","last_peered":"2023-11-25T04:22:48.047729+0000","last_clean":"2023-11-25T04:22:48.047729+0000","last_became_active":"2023-11-25T04:22:47.038477+0000","last_became_peered":"2023-11-25T04:22:47.038477+0000","last_unstale":"2023-11-25T04:22:48.047729+0000","last_undegraded":"2023-11-25T04:22:48.047729+0000","last_fullsized":"2023-11-25T04:22:48.047729+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:30:04.568790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047891+0000","last_change":"2023-11-25T04:22:47.038155+0000","last_active":"2023-11-25T04:22:48.047891+0000","last_peered":"2023-11-25T04:22:48.047891+0000","last_clean":"2023-11-25T04:22:48.047891+0000","last_became_active":"2023-11-25T04:22:47.037740+0000","last_became_peered":"2023-11-25T04:22:47.037740+0000","last_unstale":"2023-11-25T04:22:48.047891+0000","last_undegraded":"2023-11-25T04:22:48.047891+0000","last_fullsized":"2023-11-25T04:22:48.047891+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:09:11.570561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048543+0000","last_change":"2023-11-25T04:22:47.040641+0000","last_active":"2023-11-25T04:22:48.048543+0000","last_peered":"2023-11-25T04:22:48.048543+0000","last_clean":"2023-11-25T04:22:48.048543+0000","last_became_active":"2023-11-25T04:22:47.039409+0000","last_became_peered":"2023-11-25T04:22:47.039409+0000","last_unstale":"2023-11-25T04:22:48.048543+0000","last_undegraded":"2023-11-25T04:22:48.048543+0000","last_fullsized":"2023-11-25T04:22:48.048543+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:23:15.914817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047384+0000","last_change":"2023-11-25T04:22:47.043456+0000","last_active":"2023-11-25T04:22:48.047384+0000","last_peered":"2023-11-25T04:22:48.047384+0000","last_clean":"2023-11-25T04:22:48.047384+0000","last_became_active":"2023-11-25T04:22:47.043098+0000","last_became_peered":"2023-11-25T04:22:47.043098+0000","last_unstale":"2023-11-25T04:22:48.047384+0000","last_undegraded":"2023-11-25T04:22:48.047384+0000","last_fullsized":"2023-11-25T04:22:48.047384+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:47:57.307757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047103+0000","last_change":"2023-11-25T04:22:47.035260+0000","last_active":"2023-11-25T04:22:48.047103+0000","last_peered":"2023-11-25T04:22:48.047103+0000","last_clean":"2023-11-25T04:22:48.047103+0000","last_became_active":"2023-11-25T04:22:47.034702+0000","last_became_peered":"2023-11-25T04:22:47.034702+0000","last_unstale":"2023-11-25T04:22:48.047103+0000","last_undegraded":"2023-11-25T04:22:48.047103+0000","last_fullsized":"2023-11-25T04:22:48.047103+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:49:02.500022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047259+0000","last_change":"2023-11-25T04:22:47.036805+0000","last_active":"2023-11-25T04:22:48.047259+0000","last_peered":"2023-11-25T04:22:48.047259+0000","last_clean":"2023-11-25T04:22:48.047259+0000","last_became_active":"2023-11-25T04:22:47.036135+0000","last_became_peered":"2023-11-25T04:22:47.036135+0000","last_unstale":"2023-11-25T04:22:48.047259+0000","last_undegraded":"2023-11-25T04:22:48.047259+0000","last_fullsized":"2023-11-25T04:22:48.047259+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:25:21.730992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048430+0000","last_change":"2023-11-25T04:22:47.036611+0000","last_active":"2023-11-25T04:22:48.048430+0000","last_peered":"2023-11-25T04:22:48.048430+0000","last_clean":"2023-11-25T04:22:48.048430+0000","last_became_active":"2023-11-25T04:22:47.035762+0000","last_became_peered":"2023-11-25T04:22:47.035762+0000","last_unstale":"2023-11-25T04:22:48.048430+0000","last_undegraded":"2023-11-25T04:22:48.048430+0000","last_fullsized":"2023-11-25T04:22:48.048430+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:09:11.570561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048216+0000","last_change":"2023-11-25T04:22:47.042423+0000","last_active":"2023-11-25T04:22:48.048216+0000","last_peered":"2023-11-25T04:22:48.048216+0000","last_clean":"2023-11-25T04:22:48.048216+0000","last_became_active":"2023-11-25T04:22:47.042022+0000","last_became_peered":"2023-11-25T04:22:47.042022+0000","last_unstale":"2023-11-25T04:22:48.048216+0000","last_undegraded":"2023-11-25T04:22:48.048216+0000","last_fullsized":"2023-11-25T04:22:48.048216+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:38:09.176582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048337+0000","last_change":"2023-11-25T04:22:47.044040+0000","last_active":"2023-11-25T04:22:48.048337+0000","last_peered":"2023-11-25T04:22:48.048337+0000","last_clean":"2023-11-25T04:22:48.048337+0000","last_became_active":"2023-11-25T04:22:47.043329+0000","last_became_peered":"2023-11-25T04:22:47.043329+0000","last_unstale":"2023-11-25T04:22:48.048337+0000","last_undegraded":"2023-11-25T04:22:48.048337+0000","last_fullsized":"2023-11-25T04:22:48.048337+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:10:59.482775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047954+0000","last_change":"2023-11-25T04:22:47.036485+0000","last_active":"2023-11-25T04:22:48.047954+0000","last_peered":"2023-11-25T04:22:48.047954+0000","last_clean":"2023-11-25T04:22:48.047954+0000","last_became_active":"2023-11-25T04:22:47.035799+0000","last_became_peered":"2023-11-25T04:22:47.035799+0000","last_unstale":"2023-11-25T04:22:48.047954+0000","last_undegraded":"2023-11-25T04:22:48.047954+0000","last_fullsized":"2023-11-25T04:22:48.047954+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:45:40.065146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047835+0000","last_change":"2023-11-25T04:22:47.037668+0000","last_active":"2023-11-25T04:22:48.047835+0000","last_peered":"2023-11-25T04:22:48.047835+0000","last_clean":"2023-11-25T04:22:48.047835+0000","last_became_active":"2023-11-25T04:22:47.036823+0000","last_became_peered":"2023-11-25T04:22:47.036823+0000","last_unstale":"2023-11-25T04:22:48.047835+0000","last_undegraded":"2023-11-25T04:22:48.047835+0000","last_fullsized":"2023-11-25T04:22:48.047835+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:02:07.088241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047758+0000","last_change":"2023-11-25T04:22:47.038117+0000","last_active":"2023-11-25T04:22:48.047758+0000","last_peered":"2023-11-25T04:22:48.047758+0000","last_clean":"2023-11-25T04:22:48.047758+0000","last_became_active":"2023-11-25T04:22:47.037751+0000","last_became_peered":"2023-11-25T04:22:47.037751+0000","last_unstale":"2023-11-25T04:22:48.047758+0000","last_undegraded":"2023-11-25T04:22:48.047758+0000","last_fullsized":"2023-11-25T04:22:48.047758+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:30:04.568790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047184+0000","last_change":"2023-11-25T04:22:47.037786+0000","last_active":"2023-11-25T04:22:48.047184+0000","last_peered":"2023-11-25T04:22:48.047184+0000","last_clean":"2023-11-25T04:22:48.047184+0000","last_became_active":"2023-11-25T04:22:47.037206+0000","last_became_peered":"2023-11-25T04:22:47.037206+0000","last_unstale":"2023-11-25T04:22:48.047184+0000","last_undegraded":"2023-11-25T04:22:48.047184+0000","last_fullsized":"2023-11-25T04:22:48.047184+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:11:40.532140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047505+0000","last_change":"2023-11-25T04:22:47.041325+0000","last_active":"2023-11-25T04:22:48.047505+0000","last_peered":"2023-11-25T04:22:48.047505+0000","last_clean":"2023-11-25T04:22:48.047505+0000","last_became_active":"2023-11-25T04:22:47.040086+0000","last_became_peered":"2023-11-25T04:22:47.040086+0000","last_unstale":"2023-11-25T04:22:48.047505+0000","last_undegraded":"2023-11-25T04:22:48.047505+0000","last_fullsized":"2023-11-25T04:22:48.047505+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:15:39.511043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047822+0000","last_change":"2023-11-25T04:22:47.036689+0000","last_active":"2023-11-25T04:22:48.047822+0000","last_peered":"2023-11-25T04:22:48.047822+0000","last_clean":"2023-11-25T04:22:48.047822+0000","last_became_active":"2023-11-25T04:22:47.036107+0000","last_became_peered":"2023-11-25T04:22:47.036107+0000","last_unstale":"2023-11-25T04:22:48.047822+0000","last_undegraded":"2023-11-25T04:22:48.047822+0000","last_fullsized":"2023-11-25T04:22:48.047822+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:02:07.088241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"18'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048313+0000","last_change":"2023-11-25T04:22:46.040844+0000","last_active":"2023-11-25T04:22:48.048313+0000","last_peered":"2023-11-25T04:22:48.048313+0000","last_clean":"2023-11-25T04:22:48.048313+0000","last_became_active":"2023-11-25T04:22:46.040395+0000","last_became_peered":"2023-11-25T04:22:46.040395+0000","last_unstale":"2023-11-25T04:22:48.048313+0000","last_undegraded":"2023-11-25T04:22:48.048313+0000","last_fullsized":"2023-11-25T04:22:48.048313+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:11:39.522677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048200+0000","last_change":"2023-11-25T04:22:47.037888+0000","last_active":"2023-11-25T04:22:48.048200+0000","last_peered":"2023-11-25T04:22:48.048200+0000","last_clean":"2023-11-25T04:22:48.048200+0000","last_became_active":"2023-11-25T04:22:47.037158+0000","last_became_peered":"2023-11-25T04:22:47.037158+0000","last_unstale":"2023-11-25T04:22:48.048200+0000","last_undegraded":"2023-11-25T04:22:48.048200+0000","last_fullsized":"2023-11-25T04:22:48.048200+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:56:59.689578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047391+0000","last_change":"2023-11-25T04:22:46.036054+0000","last_active":"2023-11-25T04:22:48.047391+0000","last_peered":"2023-11-25T04:22:48.047391+0000","last_clean":"2023-11-25T04:22:48.047391+0000","last_became_active":"2023-11-25T04:22:46.035620+0000","last_became_peered":"2023-11-25T04:22:46.035620+0000","last_unstale":"2023-11-25T04:22:48.047391+0000","last_undegraded":"2023-11-25T04:22:48.047391+0000","last_fullsized":"2023-11-25T04:22:48.047391+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:19:48.371980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048391+0000","last_change":"2023-11-25T04:22:47.045025+0000","last_active":"2023-11-25T04:22:48.048391+0000","last_peered":"2023-11-25T04:22:48.048391+0000","last_clean":"2023-11-25T04:22:48.048391+0000","last_became_active":"2023-11-25T04:22:47.044697+0000","last_became_peered":"2023-11-25T04:22:47.044697+0000","last_unstale":"2023-11-25T04:22:48.048391+0000","last_undegraded":"2023-11-25T04:22:48.048391+0000","last_fullsized":"2023-11-25T04:22:48.048391+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:16:29.085050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047517+0000","last_change":"2023-11-25T04:22:46.037169+0000","last_active":"2023-11-25T04:22:48.047517+0000","last_peered":"2023-11-25T04:22:48.047517+0000","last_clean":"2023-11-25T04:22:48.047517+0000","last_became_active":"2023-11-25T04:22:46.036948+0000","last_became_peered":"2023-11-25T04:22:46.036948+0000","last_unstale":"2023-11-25T04:22:48.047517+0000","last_undegraded":"2023-11-25T04:22:48.047517+0000","last_fullsized":"2023-11-25T04:22:48.047517+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:45:14.670326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047715+0000","last_change":"2023-11-25T04:22:47.044860+0000","last_active":"2023-11-25T04:22:48.047715+0000","last_peered":"2023-11-25T04:22:48.047715+0000","last_clean":"2023-11-25T04:22:48.047715+0000","last_became_active":"2023-11-25T04:22:47.044551+0000","last_became_peered":"2023-11-25T04:22:47.044551+0000","last_unstale":"2023-11-25T04:22:48.047715+0000","last_undegraded":"2023-11-25T04:22:48.047715+0000","last_fullsized":"2023-11-25T04:22:48.047715+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:09:55.357642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.10","version":"18'8","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047593+0000","last_change":"2023-11-25T04:22:46.040356+0000","last_active":"2023-11-25T04:22:48.047593+0000","last_peered":"2023-11-25T04:22:48.047593+0000","last_clean":"2023-11-25T04:22:48.047593+0000","last_became_active":"2023-11-25T04:22:46.039802+0000","last_became_peered":"2023-11-25T04:22:46.039802+0000","last_unstale":"2023-11-25T04:22:48.047593+0000","last_undegraded":"2023-11-25T04:22:48.047593+0000","last_fullsized":"2023-11-25T04:22:48.047593+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:09:54.348179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"18'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048372+0000","last_change":"2023-11-25T04:22:46.040082+0000","last_active":"2023-11-25T04:22:48.048372+0000","last_peered":"2023-11-25T04:22:48.048372+0000","last_clean":"2023-11-25T04:22:48.048372+0000","last_became_active":"2023-11-25T04:22:46.039763+0000","last_became_peered":"2023-11-25T04:22:46.039763+0000","last_unstale":"2023-11-25T04:22:48.048372+0000","last_undegraded":"2023-11-25T04:22:48.048372+0000","last_fullsized":"2023-11-25T04:22:48.048372+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:10:20.178136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047265+0000","last_change":"2023-11-25T04:22:47.043644+0000","last_active":"2023-11-25T04:22:48.047265+0000","last_peered":"2023-11-25T04:22:48.047265+0000","last_clean":"2023-11-25T04:22:48.047265+0000","last_became_active":"2023-11-25T04:22:47.043221+0000","last_became_peered":"2023-11-25T04:22:47.043221+0000","last_unstale":"2023-11-25T04:22:48.047265+0000","last_undegraded":"2023-11-25T04:22:48.047265+0000","last_fullsized":"2023-11-25T04:22:48.047265+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:41:16.672268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"18'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048462+0000","last_change":"2023-11-25T04:22:46.035806+0000","last_active":"2023-11-25T04:22:48.048462+0000","last_peered":"2023-11-25T04:22:48.048462+0000","last_clean":"2023-11-25T04:22:48.048462+0000","last_became_active":"2023-11-25T04:22:46.035418+0000","last_became_peered":"2023-11-25T04:22:46.035418+0000","last_unstale":"2023-11-25T04:22:48.048462+0000","last_undegraded":"2023-11-25T04:22:48.048462+0000","last_fullsized":"2023-11-25T04:22:48.048462+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:31:53.068767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047508+0000","last_change":"2023-11-25T04:22:47.039685+0000","last_active":"2023-11-25T04:22:48.047508+0000","last_peered":"2023-11-25T04:22:48.047508+0000","last_clean":"2023-11-25T04:22:48.047508+0000","last_became_active":"2023-11-25T04:22:47.039296+0000","last_became_peered":"2023-11-25T04:22:47.039296+0000","last_unstale":"2023-11-25T04:22:48.047508+0000","last_undegraded":"2023-11-25T04:22:48.047508+0000","last_fullsized":"2023-11-25T04:22:48.047508+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:11:40.532140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"18'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048658+0000","last_change":"2023-11-25T04:22:46.039953+0000","last_active":"2023-11-25T04:22:48.048658+0000","last_peered":"2023-11-25T04:22:48.048658+0000","last_clean":"2023-11-25T04:22:48.048658+0000","last_became_active":"2023-11-25T04:22:46.039660+0000","last_became_peered":"2023-11-25T04:22:46.039660+0000","last_unstale":"2023-11-25T04:22:48.048658+0000","last_undegraded":"2023-11-25T04:22:48.048658+0000","last_fullsized":"2023-11-25T04:22:48.048658+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:59:44.482613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047887+0000","last_change":"2023-11-25T04:22:47.037912+0000","last_active":"2023-11-25T04:22:48.047887+0000","last_peered":"2023-11-25T04:22:48.047887+0000","last_clean":"2023-11-25T04:22:48.047887+0000","last_became_active":"2023-11-25T04:22:47.036552+0000","last_became_peered":"2023-11-25T04:22:47.036552+0000","last_unstale":"2023-11-25T04:22:48.047887+0000","last_undegraded":"2023-11-25T04:22:48.047887+0000","last_fullsized":"2023-11-25T04:22:48.047887+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:38:09.176582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048295+0000","last_change":"2023-11-25T04:22:46.039176+0000","last_active":"2023-11-25T04:22:48.048295+0000","last_peered":"2023-11-25T04:22:48.048295+0000","last_clean":"2023-11-25T04:22:48.048295+0000","last_became_active":"2023-11-25T04:22:46.038842+0000","last_became_peered":"2023-11-25T04:22:46.038842+0000","last_unstale":"2023-11-25T04:22:48.048295+0000","last_undegraded":"2023-11-25T04:22:48.048295+0000","last_fullsized":"2023-11-25T04:22:48.048295+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:47:56.298293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047733+0000","last_change":"2023-11-25T04:22:47.042155+0000","last_active":"2023-11-25T04:22:48.047733+0000","last_peered":"2023-11-25T04:22:48.047733+0000","last_clean":"2023-11-25T04:22:48.047733+0000","last_became_active":"2023-11-25T04:22:47.041433+0000","last_became_peered":"2023-11-25T04:22:47.041433+0000","last_unstale":"2023-11-25T04:22:48.047733+0000","last_undegraded":"2023-11-25T04:22:48.047733+0000","last_fullsized":"2023-11-25T04:22:48.047733+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:57:28.855151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"18'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048201+0000","last_change":"2023-11-25T04:22:46.037827+0000","last_active":"2023-11-25T04:22:48.048201+0000","last_peered":"2023-11-25T04:22:48.048201+0000","last_clean":"2023-11-25T04:22:48.048201+0000","last_became_active":"2023-11-25T04:22:46.036515+0000","last_became_peered":"2023-11-25T04:22:46.036515+0000","last_unstale":"2023-11-25T04:22:48.048201+0000","last_undegraded":"2023-11-25T04:22:48.048201+0000","last_fullsized":"2023-11-25T04:22:48.048201+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:10:20.178136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048154+0000","last_change":"2023-11-25T04:22:47.038856+0000","last_active":"2023-11-25T04:22:48.048154+0000","last_peered":"2023-11-25T04:22:48.048154+0000","last_clean":"2023-11-25T04:22:48.048154+0000","last_became_active":"2023-11-25T04:22:47.038190+0000","last_became_peered":"2023-11-25T04:22:47.038190+0000","last_unstale":"2023-11-25T04:22:48.048154+0000","last_undegraded":"2023-11-25T04:22:48.048154+0000","last_fullsized":"2023-11-25T04:22:48.048154+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:29:37.268624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047789+0000","last_change":"2023-11-25T04:22:46.038473+0000","last_active":"2023-11-25T04:22:48.047789+0000","last_peered":"2023-11-25T04:22:48.047789+0000","last_clean":"2023-11-25T04:22:48.047789+0000","last_became_active":"2023-11-25T04:22:46.038159+0000","last_became_peered":"2023-11-25T04:22:46.038159+0000","last_unstale":"2023-11-25T04:22:48.047789+0000","last_undegraded":"2023-11-25T04:22:48.047789+0000","last_fullsized":"2023-11-25T04:22:48.047789+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:31:53.847807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048568+0000","last_change":"2023-11-25T04:22:47.036222+0000","last_active":"2023-11-25T04:22:48.048568+0000","last_peered":"2023-11-25T04:22:48.048568+0000","last_clean":"2023-11-25T04:22:48.048568+0000","last_became_active":"2023-11-25T04:22:47.035871+0000","last_became_peered":"2023-11-25T04:22:47.035871+0000","last_unstale":"2023-11-25T04:22:48.048568+0000","last_undegraded":"2023-11-25T04:22:48.048568+0000","last_fullsized":"2023-11-25T04:22:48.048568+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:13:35.199538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047681+0000","last_change":"2023-11-25T04:22:46.035907+0000","last_active":"2023-11-25T04:22:48.047681+0000","last_peered":"2023-11-25T04:22:48.047681+0000","last_clean":"2023-11-25T04:22:48.047681+0000","last_became_active":"2023-11-25T04:22:46.035612+0000","last_became_peered":"2023-11-25T04:22:46.035612+0000","last_unstale":"2023-11-25T04:22:48.047681+0000","last_undegraded":"2023-11-25T04:22:48.047681+0000","last_fullsized":"2023-11-25T04:22:48.047681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:00:35.249875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048441+0000","last_change":"2023-11-25T04:22:47.038725+0000","last_active":"2023-11-25T04:22:48.048441+0000","last_peered":"2023-11-25T04:22:48.048441+0000","last_clean":"2023-11-25T04:22:48.048441+0000","last_became_active":"2023-11-25T04:22:47.038124+0000","last_became_peered":"2023-11-25T04:22:47.038124+0000","last_unstale":"2023-11-25T04:22:48.048441+0000","last_undegraded":"2023-11-25T04:22:48.048441+0000","last_fullsized":"2023-11-25T04:22:48.048441+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:00:26.187878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047437+0000","last_change":"2023-11-25T04:22:46.034509+0000","last_active":"2023-11-25T04:22:48.047437+0000","last_peered":"2023-11-25T04:22:48.047437+0000","last_clean":"2023-11-25T04:22:48.047437+0000","last_became_active":"2023-11-25T04:22:46.034158+0000","last_became_peered":"2023-11-25T04:22:46.034158+0000","last_unstale":"2023-11-25T04:22:48.047437+0000","last_undegraded":"2023-11-25T04:22:48.047437+0000","last_fullsized":"2023-11-25T04:22:48.047437+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:26:25.338117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048005+0000","last_change":"2023-11-25T04:22:47.038800+0000","last_active":"2023-11-25T04:22:48.048005+0000","last_peered":"2023-11-25T04:22:48.048005+0000","last_clean":"2023-11-25T04:22:48.048005+0000","last_became_active":"2023-11-25T04:22:47.038081+0000","last_became_peered":"2023-11-25T04:22:47.038081+0000","last_unstale":"2023-11-25T04:22:48.048005+0000","last_undegraded":"2023-11-25T04:22:48.048005+0000","last_fullsized":"2023-11-25T04:22:48.048005+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:45:40.065146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"12'157","reported_seq":214,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-25T04:22:47.033945+0000","last_change":"2023-11-25T04:22:36.931828+0000","last_active":"2023-11-25T04:22:47.033945+0000","last_peered":"2023-11-25T04:22:47.033945+0000","last_clean":"2023-11-25T04:22:47.033945+0000","last_became_active":"2023-11-25T04:22:36.931582+0000","last_became_peered":"2023-11-25T04:22:36.931582+0000","last_unstale":"2023-11-25T04:22:47.033945+0000","last_undegraded":"2023-11-25T04:22:47.033945+0000","last_fullsized":"2023-11-25T04:22:47.033945+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:35.919850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:35.919850+0000","last_clean_scrub_stamp":"2023-11-25T04:22:35.919850+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:59:59.198309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-25T04:22:47.031925+0000","last_change":"2023-11-25T04:22:41.999590+0000","last_active":"2023-11-25T04:22:47.031925+0000","last_peered":"2023-11-25T04:22:47.031925+0000","last_clean":"2023-11-25T04:22:47.031925+0000","last_became_active":"2023-11-25T04:22:41.999314+0000","last_became_peered":"2023-11-25T04:22:41.999314+0000","last_unstale":"2023-11-25T04:22:47.031925+0000","last_undegraded":"2023-11-25T04:22:47.031925+0000","last_fullsized":"2023-11-25T04:22:47.031925+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:20:12.785425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047840+0000","last_change":"2023-11-25T04:22:46.037044+0000","last_active":"2023-11-25T04:22:48.047840+0000","last_peered":"2023-11-25T04:22:48.047840+0000","last_clean":"2023-11-25T04:22:48.047840+0000","last_became_active":"2023-11-25T04:22:46.035916+0000","last_became_peered":"2023-11-25T04:22:46.035916+0000","last_unstale":"2023-11-25T04:22:48.047840+0000","last_undegraded":"2023-11-25T04:22:48.047840+0000","last_fullsized":"2023-11-25T04:22:48.047840+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:38:20.501102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048611+0000","last_change":"2023-11-25T04:22:47.036222+0000","last_active":"2023-11-25T04:22:48.048611+0000","last_peered":"2023-11-25T04:22:48.048611+0000","last_clean":"2023-11-25T04:22:48.048611+0000","last_became_active":"2023-11-25T04:22:47.035484+0000","last_became_peered":"2023-11-25T04:22:47.035484+0000","last_unstale":"2023-11-25T04:22:48.048611+0000","last_undegraded":"2023-11-25T04:22:48.048611+0000","last_fullsized":"2023-11-25T04:22:48.048611+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:18:56.845916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-25T04:22:47.032801+0000","last_change":"2023-11-25T04:22:42.000257+0000","last_active":"2023-11-25T04:22:47.032801+0000","last_peered":"2023-11-25T04:22:47.032801+0000","last_clean":"2023-11-25T04:22:47.032801+0000","last_became_active":"2023-11-25T04:22:41.999969+0000","last_became_peered":"2023-11-25T04:22:41.999969+0000","last_unstale":"2023-11-25T04:22:47.032801+0000","last_undegraded":"2023-11-25T04:22:47.032801+0000","last_fullsized":"2023-11-25T04:22:47.032801+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:21:34.424091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047780+0000","last_change":"2023-11-25T04:22:46.036121+0000","last_active":"2023-11-25T04:22:48.047780+0000","last_peered":"2023-11-25T04:22:48.047780+0000","last_clean":"2023-11-25T04:22:48.047780+0000","last_became_active":"2023-11-25T04:22:46.035654+0000","last_became_peered":"2023-11-25T04:22:46.035654+0000","last_unstale":"2023-11-25T04:22:48.047780+0000","last_undegraded":"2023-11-25T04:22:48.047780+0000","last_fullsized":"2023-11-25T04:22:48.047780+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:27:31.901618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048273+0000","last_change":"2023-11-25T04:22:47.039583+0000","last_active":"2023-11-25T04:22:48.048273+0000","last_peered":"2023-11-25T04:22:48.048273+0000","last_clean":"2023-11-25T04:22:48.048273+0000","last_became_active":"2023-11-25T04:22:47.039197+0000","last_became_peered":"2023-11-25T04:22:47.039197+0000","last_unstale":"2023-11-25T04:22:48.048273+0000","last_undegraded":"2023-11-25T04:22:48.048273+0000","last_fullsized":"2023-11-25T04:22:48.048273+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:37:30.959147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-25T04:22:47.029239+0000","last_change":"2023-11-25T04:22:42.000382+0000","last_active":"2023-11-25T04:22:47.029239+0000","last_peered":"2023-11-25T04:22:47.029239+0000","last_clean":"2023-11-25T04:22:47.029239+0000","last_became_active":"2023-11-25T04:22:42.000041+0000","last_became_peered":"2023-11-25T04:22:42.000041+0000","last_unstale":"2023-11-25T04:22:47.029239+0000","last_undegraded":"2023-11-25T04:22:47.029239+0000","last_fullsized":"2023-11-25T04:22:47.029239+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:08:19.655864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047444+0000","last_change":"2023-11-25T04:22:46.037431+0000","last_active":"2023-11-25T04:22:48.047444+0000","last_peered":"2023-11-25T04:22:48.047444+0000","last_clean":"2023-11-25T04:22:48.047444+0000","last_became_active":"2023-11-25T04:22:46.037141+0000","last_became_peered":"2023-11-25T04:22:46.037141+0000","last_unstale":"2023-11-25T04:22:48.047444+0000","last_undegraded":"2023-11-25T04:22:48.047444+0000","last_fullsized":"2023-11-25T04:22:48.047444+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:40:42.500663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048182+0000","last_change":"2023-11-25T04:22:47.038454+0000","last_active":"2023-11-25T04:22:48.048182+0000","last_peered":"2023-11-25T04:22:48.048182+0000","last_clean":"2023-11-25T04:22:48.048182+0000","last_became_active":"2023-11-25T04:22:47.038046+0000","last_became_peered":"2023-11-25T04:22:47.038046+0000","last_unstale":"2023-11-25T04:22:48.048182+0000","last_undegraded":"2023-11-25T04:22:48.048182+0000","last_fullsized":"2023-11-25T04:22:48.048182+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:38:38.839941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-25T04:22:47.031534+0000","last_change":"2023-11-25T04:22:41.999424+0000","last_active":"2023-11-25T04:22:47.031534+0000","last_peered":"2023-11-25T04:22:47.031534+0000","last_clean":"2023-11-25T04:22:47.031534+0000","last_became_active":"2023-11-25T04:22:41.999179+0000","last_became_peered":"2023-11-25T04:22:41.999179+0000","last_unstale":"2023-11-25T04:22:47.031534+0000","last_undegraded":"2023-11-25T04:22:47.031534+0000","last_fullsized":"2023-11-25T04:22:47.031534+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:12:04.291696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048523+0000","last_change":"2023-11-25T04:22:46.038758+0000","last_active":"2023-11-25T04:22:48.048523+0000","last_peered":"2023-11-25T04:22:48.048523+0000","last_clean":"2023-11-25T04:22:48.048523+0000","last_became_active":"2023-11-25T04:22:46.038470+0000","last_became_peered":"2023-11-25T04:22:46.038470+0000","last_unstale":"2023-11-25T04:22:48.048523+0000","last_undegraded":"2023-11-25T04:22:48.048523+0000","last_fullsized":"2023-11-25T04:22:48.048523+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:49:01.490558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048075+0000","last_change":"2023-11-25T04:22:47.037355+0000","last_active":"2023-11-25T04:22:48.048075+0000","last_peered":"2023-11-25T04:22:48.048075+0000","last_clean":"2023-11-25T04:22:48.048075+0000","last_became_active":"2023-11-25T04:22:47.036924+0000","last_became_peered":"2023-11-25T04:22:47.036924+0000","last_unstale":"2023-11-25T04:22:48.048075+0000","last_undegraded":"2023-11-25T04:22:48.048075+0000","last_fullsized":"2023-11-25T04:22:48.048075+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:15:39.511043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048677+0000","last_change":"2023-11-25T04:22:46.039025+0000","last_active":"2023-11-25T04:22:48.048677+0000","last_peered":"2023-11-25T04:22:48.048677+0000","last_clean":"2023-11-25T04:22:48.048677+0000","last_became_active":"2023-11-25T04:22:46.038762+0000","last_became_peered":"2023-11-25T04:22:46.038762+0000","last_unstale":"2023-11-25T04:22:48.048677+0000","last_undegraded":"2023-11-25T04:22:48.048677+0000","last_fullsized":"2023-11-25T04:22:48.048677+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:44:47.544509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-25T04:22:47.029103+0000","last_change":"2023-11-25T04:22:42.000204+0000","last_active":"2023-11-25T04:22:47.029103+0000","last_peered":"2023-11-25T04:22:47.029103+0000","last_clean":"2023-11-25T04:22:47.029103+0000","last_became_active":"2023-11-25T04:22:41.999915+0000","last_became_peered":"2023-11-25T04:22:41.999915+0000","last_unstale":"2023-11-25T04:22:47.029103+0000","last_undegraded":"2023-11-25T04:22:47.029103+0000","last_fullsized":"2023-11-25T04:22:47.029103+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:07:32.457905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047409+0000","last_change":"2023-11-25T04:22:47.038119+0000","last_active":"2023-11-25T04:22:48.047409+0000","last_peered":"2023-11-25T04:22:48.047409+0000","last_clean":"2023-11-25T04:22:48.047409+0000","last_became_active":"2023-11-25T04:22:47.037732+0000","last_became_peered":"2023-11-25T04:22:47.037732+0000","last_unstale":"2023-11-25T04:22:48.047409+0000","last_undegraded":"2023-11-25T04:22:48.047409+0000","last_fullsized":"2023-11-25T04:22:48.047409+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:49:02.500022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048004+0000","last_change":"2023-11-25T04:22:46.035885+0000","last_active":"2023-11-25T04:22:48.048004+0000","last_peered":"2023-11-25T04:22:48.048004+0000","last_clean":"2023-11-25T04:22:48.048004+0000","last_became_active":"2023-11-25T04:22:46.035405+0000","last_became_peered":"2023-11-25T04:22:46.035405+0000","last_unstale":"2023-11-25T04:22:48.048004+0000","last_undegraded":"2023-11-25T04:22:48.048004+0000","last_fullsized":"2023-11-25T04:22:48.048004+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:53:47.398746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-25T04:22:47.032524+0000","last_change":"2023-11-25T04:22:41.999067+0000","last_active":"2023-11-25T04:22:47.032524+0000","last_peered":"2023-11-25T04:22:47.032524+0000","last_clean":"2023-11-25T04:22:47.032524+0000","last_became_active":"2023-11-25T04:22:41.998798+0000","last_became_peered":"2023-11-25T04:22:41.998798+0000","last_unstale":"2023-11-25T04:22:47.032524+0000","last_undegraded":"2023-11-25T04:22:47.032524+0000","last_fullsized":"2023-11-25T04:22:47.032524+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:00:04.266051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047617+0000","last_change":"2023-11-25T04:22:47.043464+0000","last_active":"2023-11-25T04:22:48.047617+0000","last_peered":"2023-11-25T04:22:48.047617+0000","last_clean":"2023-11-25T04:22:48.047617+0000","last_became_active":"2023-11-25T04:22:47.043050+0000","last_became_peered":"2023-11-25T04:22:47.043050+0000","last_unstale":"2023-11-25T04:22:48.047617+0000","last_undegraded":"2023-11-25T04:22:48.047617+0000","last_fullsized":"2023-11-25T04:22:48.047617+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:59:45.492077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"18'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048417+0000","last_change":"2023-11-25T04:22:46.034606+0000","last_active":"2023-11-25T04:22:48.048417+0000","last_peered":"2023-11-25T04:22:48.048417+0000","last_clean":"2023-11-25T04:22:48.048417+0000","last_became_active":"2023-11-25T04:22:46.034229+0000","last_became_peered":"2023-11-25T04:22:46.034229+0000","last_unstale":"2023-11-25T04:22:48.048417+0000","last_undegraded":"2023-11-25T04:22:48.048417+0000","last_fullsized":"2023-11-25T04:22:48.048417+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:21:42.146264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-25T04:22:47.031216+0000","last_change":"2023-11-25T04:22:42.000258+0000","last_active":"2023-11-25T04:22:47.031216+0000","last_peered":"2023-11-25T04:22:47.031216+0000","last_clean":"2023-11-25T04:22:47.031216+0000","last_became_active":"2023-11-25T04:22:41.999992+0000","last_became_peered":"2023-11-25T04:22:41.999992+0000","last_unstale":"2023-11-25T04:22:47.031216+0000","last_undegraded":"2023-11-25T04:22:47.031216+0000","last_fullsized":"2023-11-25T04:22:47.031216+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:31:13.715992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047510+0000","last_change":"2023-11-25T04:22:47.042860+0000","last_active":"2023-11-25T04:22:48.047510+0000","last_peered":"2023-11-25T04:22:48.047510+0000","last_clean":"2023-11-25T04:22:48.047510+0000","last_became_active":"2023-11-25T04:22:47.042472+0000","last_became_peered":"2023-11-25T04:22:47.042472+0000","last_unstale":"2023-11-25T04:22:48.047510+0000","last_undegraded":"2023-11-25T04:22:48.047510+0000","last_fullsized":"2023-11-25T04:22:48.047510+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:44:48.553972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048418+0000","last_change":"2023-11-25T04:22:46.038417+0000","last_active":"2023-11-25T04:22:48.048418+0000","last_peered":"2023-11-25T04:22:48.048418+0000","last_clean":"2023-11-25T04:22:48.048418+0000","last_became_active":"2023-11-25T04:22:46.037513+0000","last_became_peered":"2023-11-25T04:22:46.037513+0000","last_unstale":"2023-11-25T04:22:48.048418+0000","last_undegraded":"2023-11-25T04:22:48.048418+0000","last_fullsized":"2023-11-25T04:22:48.048418+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:58:14.877413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-25T04:22:47.030950+0000","last_change":"2023-11-25T04:22:42.000217+0000","last_active":"2023-11-25T04:22:47.030950+0000","last_peered":"2023-11-25T04:22:47.030950+0000","last_clean":"2023-11-25T04:22:47.030950+0000","last_became_active":"2023-11-25T04:22:41.999987+0000","last_became_peered":"2023-11-25T04:22:41.999987+0000","last_unstale":"2023-11-25T04:22:47.030950+0000","last_undegraded":"2023-11-25T04:22:47.030950+0000","last_fullsized":"2023-11-25T04:22:47.030950+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:11:51.100459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047385+0000","last_change":"2023-11-25T04:22:47.036099+0000","last_active":"2023-11-25T04:22:48.047385+0000","last_peered":"2023-11-25T04:22:48.047385+0000","last_clean":"2023-11-25T04:22:48.047385+0000","last_became_active":"2023-11-25T04:22:47.035581+0000","last_became_peered":"2023-11-25T04:22:47.035581+0000","last_unstale":"2023-11-25T04:22:48.047385+0000","last_undegraded":"2023-11-25T04:22:48.047385+0000","last_fullsized":"2023-11-25T04:22:48.047385+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:21:43.155728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048171+0000","last_change":"2023-11-25T04:22:47.040343+0000","last_active":"2023-11-25T04:22:48.048171+0000","last_peered":"2023-11-25T04:22:48.048171+0000","last_clean":"2023-11-25T04:22:48.048171+0000","last_became_active":"2023-11-25T04:22:47.039151+0000","last_became_peered":"2023-11-25T04:22:47.039151+0000","last_unstale":"2023-11-25T04:22:48.048171+0000","last_undegraded":"2023-11-25T04:22:48.048171+0000","last_fullsized":"2023-11-25T04:22:48.048171+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:21:15.981901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"18'6","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048260+0000","last_change":"2023-11-25T04:22:46.040005+0000","last_active":"2023-11-25T04:22:48.048260+0000","last_peered":"2023-11-25T04:22:48.048260+0000","last_clean":"2023-11-25T04:22:48.048260+0000","last_became_active":"2023-11-25T04:22:46.039655+0000","last_became_peered":"2023-11-25T04:22:46.039655+0000","last_unstale":"2023-11-25T04:22:48.048260+0000","last_undegraded":"2023-11-25T04:22:48.048260+0000","last_fullsized":"2023-11-25T04:22:48.048260+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:41:15.662804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047237+0000","last_change":"2023-11-25T04:22:47.037181+0000","last_active":"2023-11-25T04:22:48.047237+0000","last_peered":"2023-11-25T04:22:48.047237+0000","last_clean":"2023-11-25T04:22:48.047237+0000","last_became_active":"2023-11-25T04:22:47.036821+0000","last_became_peered":"2023-11-25T04:22:47.036821+0000","last_unstale":"2023-11-25T04:22:48.047237+0000","last_undegraded":"2023-11-25T04:22:48.047237+0000","last_fullsized":"2023-11-25T04:22:48.047237+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:12:51.990448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"18'3","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048375+0000","last_change":"2023-11-25T04:22:46.035816+0000","last_active":"2023-11-25T04:22:48.048375+0000","last_peered":"2023-11-25T04:22:48.048375+0000","last_clean":"2023-11-25T04:22:48.048375+0000","last_became_active":"2023-11-25T04:22:46.035461+0000","last_became_peered":"2023-11-25T04:22:46.035461+0000","last_unstale":"2023-11-25T04:22:48.048375+0000","last_undegraded":"2023-11-25T04:22:48.048375+0000","last_fullsized":"2023-11-25T04:22:48.048375+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:58:26.155063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048370+0000","last_change":"2023-11-25T04:22:46.040458+0000","last_active":"2023-11-25T04:22:48.048370+0000","last_peered":"2023-11-25T04:22:48.048370+0000","last_clean":"2023-11-25T04:22:48.048370+0000","last_became_active":"2023-11-25T04:22:46.040143+0000","last_became_peered":"2023-11-25T04:22:46.040143+0000","last_unstale":"2023-11-25T04:22:48.048370+0000","last_undegraded":"2023-11-25T04:22:48.048370+0000","last_fullsized":"2023-11-25T04:22:48.048370+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:27:31.901618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047387+0000","last_change":"2023-11-25T04:22:47.040346+0000","last_active":"2023-11-25T04:22:48.047387+0000","last_peered":"2023-11-25T04:22:48.047387+0000","last_clean":"2023-11-25T04:22:48.047387+0000","last_became_active":"2023-11-25T04:22:47.040084+0000","last_became_peered":"2023-11-25T04:22:47.040084+0000","last_unstale":"2023-11-25T04:22:48.047387+0000","last_undegraded":"2023-11-25T04:22:48.047387+0000","last_fullsized":"2023-11-25T04:22:48.047387+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:59:45.492077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048308+0000","last_change":"2023-11-25T04:22:46.038721+0000","last_active":"2023-11-25T04:22:48.048308+0000","last_peered":"2023-11-25T04:22:48.048308+0000","last_clean":"2023-11-25T04:22:48.048308+0000","last_became_active":"2023-11-25T04:22:46.037698+0000","last_became_peered":"2023-11-25T04:22:46.037698+0000","last_unstale":"2023-11-25T04:22:48.048308+0000","last_undegraded":"2023-11-25T04:22:48.048308+0000","last_fullsized":"2023-11-25T04:22:48.048308+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:25:20.721529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047199+0000","last_change":"2023-11-25T04:22:47.037790+0000","last_active":"2023-11-25T04:22:48.047199+0000","last_peered":"2023-11-25T04:22:48.047199+0000","last_clean":"2023-11-25T04:22:48.047199+0000","last_became_active":"2023-11-25T04:22:47.037149+0000","last_became_peered":"2023-11-25T04:22:47.037149+0000","last_unstale":"2023-11-25T04:22:48.047199+0000","last_undegraded":"2023-11-25T04:22:48.047199+0000","last_fullsized":"2023-11-25T04:22:48.047199+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:31:54.857270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048584+0000","last_change":"2023-11-25T04:22:47.036079+0000","last_active":"2023-11-25T04:22:48.048584+0000","last_peered":"2023-11-25T04:22:48.048584+0000","last_clean":"2023-11-25T04:22:48.048584+0000","last_became_active":"2023-11-25T04:22:47.035625+0000","last_became_peered":"2023-11-25T04:22:47.035625+0000","last_unstale":"2023-11-25T04:22:48.048584+0000","last_undegraded":"2023-11-25T04:22:48.048584+0000","last_fullsized":"2023-11-25T04:22:48.048584+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:21:15.981901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047749+0000","last_change":"2023-11-25T04:22:46.036085+0000","last_active":"2023-11-25T04:22:48.047749+0000","last_peered":"2023-11-25T04:22:48.047749+0000","last_clean":"2023-11-25T04:22:48.047749+0000","last_became_active":"2023-11-25T04:22:46.035495+0000","last_became_peered":"2023-11-25T04:22:46.035495+0000","last_unstale":"2023-11-25T04:22:48.047749+0000","last_undegraded":"2023-11-25T04:22:48.047749+0000","last_fullsized":"2023-11-25T04:22:48.047749+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:14:56.735365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048251+0000","last_change":"2023-11-25T04:22:47.043064+0000","last_active":"2023-11-25T04:22:48.048251+0000","last_peered":"2023-11-25T04:22:48.048251+0000","last_clean":"2023-11-25T04:22:48.048251+0000","last_became_active":"2023-11-25T04:22:47.042760+0000","last_became_peered":"2023-11-25T04:22:47.042760+0000","last_unstale":"2023-11-25T04:22:48.048251+0000","last_undegraded":"2023-11-25T04:22:48.048251+0000","last_fullsized":"2023-11-25T04:22:48.048251+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:37:30.959147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047514+0000","last_change":"2023-11-25T04:22:46.038298+0000","last_active":"2023-11-25T04:22:48.047514+0000","last_peered":"2023-11-25T04:22:48.047514+0000","last_clean":"2023-11-25T04:22:48.047514+0000","last_became_active":"2023-11-25T04:22:46.036629+0000","last_became_peered":"2023-11-25T04:22:46.036629+0000","last_unstale":"2023-11-25T04:22:48.047514+0000","last_undegraded":"2023-11-25T04:22:48.047514+0000","last_fullsized":"2023-11-25T04:22:48.047514+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:45:14.670326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048209+0000","last_change":"2023-11-25T04:22:47.043156+0000","last_active":"2023-11-25T04:22:48.048209+0000","last_peered":"2023-11-25T04:22:48.048209+0000","last_clean":"2023-11-25T04:22:48.048209+0000","last_became_active":"2023-11-25T04:22:47.042693+0000","last_became_peered":"2023-11-25T04:22:47.042693+0000","last_unstale":"2023-11-25T04:22:48.048209+0000","last_undegraded":"2023-11-25T04:22:48.048209+0000","last_fullsized":"2023-11-25T04:22:48.048209+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:38:38.839941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1a","version":"18'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047866+0000","last_change":"2023-11-25T04:22:46.034340+0000","last_active":"2023-11-25T04:22:48.047866+0000","last_peered":"2023-11-25T04:22:48.047866+0000","last_clean":"2023-11-25T04:22:48.047866+0000","last_became_active":"2023-11-25T04:22:46.033944+0000","last_became_peered":"2023-11-25T04:22:46.033944+0000","last_unstale":"2023-11-25T04:22:48.047866+0000","last_undegraded":"2023-11-25T04:22:48.047866+0000","last_fullsized":"2023-11-25T04:22:48.047866+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:59:30.291004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047694+0000","last_change":"2023-11-25T04:22:47.039547+0000","last_active":"2023-11-25T04:22:48.047694+0000","last_peered":"2023-11-25T04:22:48.047694+0000","last_clean":"2023-11-25T04:22:48.047694+0000","last_became_active":"2023-11-25T04:22:47.039218+0000","last_became_peered":"2023-11-25T04:22:47.039218+0000","last_unstale":"2023-11-25T04:22:48.047694+0000","last_undegraded":"2023-11-25T04:22:48.047694+0000","last_fullsized":"2023-11-25T04:22:48.047694+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:44:48.553972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"18'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047860+0000","last_change":"2023-11-25T04:22:46.035882+0000","last_active":"2023-11-25T04:22:48.047860+0000","last_peered":"2023-11-25T04:22:48.047860+0000","last_clean":"2023-11-25T04:22:48.047860+0000","last_became_active":"2023-11-25T04:22:46.035588+0000","last_became_peered":"2023-11-25T04:22:46.035588+0000","last_unstale":"2023-11-25T04:22:48.047860+0000","last_undegraded":"2023-11-25T04:22:48.047860+0000","last_fullsized":"2023-11-25T04:22:48.047860+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:43:10.702849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047831+0000","last_change":"2023-11-25T04:22:47.041267+0000","last_active":"2023-11-25T04:22:48.047831+0000","last_peered":"2023-11-25T04:22:48.047831+0000","last_clean":"2023-11-25T04:22:48.047831+0000","last_became_active":"2023-11-25T04:22:47.040408+0000","last_became_peered":"2023-11-25T04:22:47.040408+0000","last_unstale":"2023-11-25T04:22:48.047831+0000","last_undegraded":"2023-11-25T04:22:48.047831+0000","last_fullsized":"2023-11-25T04:22:48.047831+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:37:30.959147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"18'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048131+0000","last_change":"2023-11-25T04:22:46.040793+0000","last_active":"2023-11-25T04:22:48.048131+0000","last_peered":"2023-11-25T04:22:48.048131+0000","last_clean":"2023-11-25T04:22:48.048131+0000","last_became_active":"2023-11-25T04:22:46.040493+0000","last_became_peered":"2023-11-25T04:22:46.040493+0000","last_unstale":"2023-11-25T04:22:48.048131+0000","last_undegraded":"2023-11-25T04:22:48.048131+0000","last_fullsized":"2023-11-25T04:22:48.048131+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:58:26.155063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048056+0000","last_change":"2023-11-25T04:22:47.040067+0000","last_active":"2023-11-25T04:22:48.048056+0000","last_peered":"2023-11-25T04:22:48.048056+0000","last_clean":"2023-11-25T04:22:48.048056+0000","last_became_active":"2023-11-25T04:22:47.038425+0000","last_became_peered":"2023-11-25T04:22:47.038425+0000","last_unstale":"2023-11-25T04:22:48.048056+0000","last_undegraded":"2023-11-25T04:22:48.048056+0000","last_fullsized":"2023-11-25T04:22:48.048056+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:25:06.132517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"18'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048297+0000","last_change":"2023-11-25T04:22:46.035872+0000","last_active":"2023-11-25T04:22:48.048297+0000","last_peered":"2023-11-25T04:22:48.048297+0000","last_clean":"2023-11-25T04:22:48.048297+0000","last_became_active":"2023-11-25T04:22:46.035575+0000","last_became_peered":"2023-11-25T04:22:46.035575+0000","last_unstale":"2023-11-25T04:22:48.048297+0000","last_undegraded":"2023-11-25T04:22:48.048297+0000","last_fullsized":"2023-11-25T04:22:48.048297+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:58:14.877413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047725+0000","last_change":"2023-11-25T04:22:47.036764+0000","last_active":"2023-11-25T04:22:48.047725+0000","last_peered":"2023-11-25T04:22:48.047725+0000","last_clean":"2023-11-25T04:22:48.047725+0000","last_became_active":"2023-11-25T04:22:47.036406+0000","last_became_peered":"2023-11-25T04:22:47.036406+0000","last_unstale":"2023-11-25T04:22:48.047725+0000","last_undegraded":"2023-11-25T04:22:48.047725+0000","last_fullsized":"2023-11-25T04:22:48.047725+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:09:55.357642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1e","version":"18'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048335+0000","last_change":"2023-11-25T04:22:46.035659+0000","last_active":"2023-11-25T04:22:48.048335+0000","last_peered":"2023-11-25T04:22:48.048335+0000","last_clean":"2023-11-25T04:22:48.048335+0000","last_became_active":"2023-11-25T04:22:46.035113+0000","last_became_peered":"2023-11-25T04:22:46.035113+0000","last_unstale":"2023-11-25T04:22:48.048335+0000","last_undegraded":"2023-11-25T04:22:48.048335+0000","last_fullsized":"2023-11-25T04:22:48.048335+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:26:25.338117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048474+0000","last_change":"2023-11-25T04:22:47.041554+0000","last_active":"2023-11-25T04:22:48.048474+0000","last_peered":"2023-11-25T04:22:48.048474+0000","last_clean":"2023-11-25T04:22:48.048474+0000","last_became_active":"2023-11-25T04:22:47.040314+0000","last_became_peered":"2023-11-25T04:22:47.040314+0000","last_unstale":"2023-11-25T04:22:48.048474+0000","last_undegraded":"2023-11-25T04:22:48.048474+0000","last_fullsized":"2023-11-25T04:22:48.048474+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:24:56.315251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"18'4","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048204+0000","last_change":"2023-11-25T04:22:46.038702+0000","last_active":"2023-11-25T04:22:48.048204+0000","last_peered":"2023-11-25T04:22:48.048204+0000","last_clean":"2023-11-25T04:22:48.048204+0000","last_became_active":"2023-11-25T04:22:46.037567+0000","last_became_peered":"2023-11-25T04:22:46.037567+0000","last_unstale":"2023-11-25T04:22:48.048204+0000","last_undegraded":"2023-11-25T04:22:48.048204+0000","last_fullsized":"2023-11-25T04:22:48.048204+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:58:26.155063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738375,"num_pgs":72,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35508,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336332,"statfs":{"total":96636764160,"available":96600403968,"internally_reserved":0,"allocated":172032,"data_stored":58325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.307}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]}]},{"osd":1,"up_from":8,"seq":34359738375,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31944,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339896,"statfs":{"total":96636764160,"available":96604053504,"internally_reserved":0,"allocated":782336,"data_stored":649871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738374,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31952,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339888,"statfs":{"total":96636764160,"available":96604045312,"internally_reserved":0,"allocated":790528,"data_stored":650487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-25T04:22:55.028 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-25T04:22:55.361 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:22:55.363 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2023-11-25T04:22:55.372 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-11-25T04:22:53.494202+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":247,"num_write_kb":3474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":195,"ondisk_log_size":195,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":192,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":99404,"kb_used_data":1704,"kb_used_omap":4,"kb_used_meta":97659,"kb_avail":283016116,"statfs":{"total":289910292480,"available":289808502784,"internally_reserved":0,"allocated":1744896,"data_stored":1358683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":100003151},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.469702"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048437+0000","last_change":"2023-11-25T04:22:47.039847+0000","last_active":"2023-11-25T04:22:48.048437+0000","last_peered":"2023-11-25T04:22:48.048437+0000","last_clean":"2023-11-25T04:22:48.048437+0000","last_became_active":"2023-11-25T04:22:47.039490+0000","last_became_peered":"2023-11-25T04:22:47.039490+0000","last_unstale":"2023-11-25T04:22:48.048437+0000","last_undegraded":"2023-11-25T04:22:48.048437+0000","last_fullsized":"2023-11-25T04:22:48.048437+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:25:06.132517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048105+0000","last_change":"2023-11-25T04:22:47.040836+0000","last_active":"2023-11-25T04:22:48.048105+0000","last_peered":"2023-11-25T04:22:48.048105+0000","last_clean":"2023-11-25T04:22:48.048105+0000","last_became_active":"2023-11-25T04:22:47.040511+0000","last_became_peered":"2023-11-25T04:22:47.040511+0000","last_unstale":"2023-11-25T04:22:48.048105+0000","last_undegraded":"2023-11-25T04:22:48.048105+0000","last_fullsized":"2023-11-25T04:22:48.048105+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:15:39.511043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048297+0000","last_change":"2023-11-25T04:22:47.043529+0000","last_active":"2023-11-25T04:22:48.048297+0000","last_peered":"2023-11-25T04:22:48.048297+0000","last_clean":"2023-11-25T04:22:48.048297+0000","last_became_active":"2023-11-25T04:22:47.042733+0000","last_became_peered":"2023-11-25T04:22:47.042733+0000","last_unstale":"2023-11-25T04:22:48.048297+0000","last_undegraded":"2023-11-25T04:22:48.048297+0000","last_fullsized":"2023-11-25T04:22:48.048297+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:47:34.764521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047862+0000","last_change":"2023-11-25T04:22:47.040314+0000","last_active":"2023-11-25T04:22:48.047862+0000","last_peered":"2023-11-25T04:22:48.047862+0000","last_clean":"2023-11-25T04:22:48.047862+0000","last_became_active":"2023-11-25T04:22:47.039185+0000","last_became_peered":"2023-11-25T04:22:47.039185+0000","last_unstale":"2023-11-25T04:22:48.047862+0000","last_undegraded":"2023-11-25T04:22:48.047862+0000","last_fullsized":"2023-11-25T04:22:48.047862+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:16:29.085050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047841+0000","last_change":"2023-11-25T04:22:47.039499+0000","last_active":"2023-11-25T04:22:48.047841+0000","last_peered":"2023-11-25T04:22:48.047841+0000","last_clean":"2023-11-25T04:22:48.047841+0000","last_became_active":"2023-11-25T04:22:47.039165+0000","last_became_peered":"2023-11-25T04:22:47.039165+0000","last_unstale":"2023-11-25T04:22:48.047841+0000","last_undegraded":"2023-11-25T04:22:48.047841+0000","last_fullsized":"2023-11-25T04:22:48.047841+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:57:28.855151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047565+0000","last_change":"2023-11-25T04:22:47.039992+0000","last_active":"2023-11-25T04:22:48.047565+0000","last_peered":"2023-11-25T04:22:48.047565+0000","last_clean":"2023-11-25T04:22:48.047565+0000","last_became_active":"2023-11-25T04:22:47.039405+0000","last_became_peered":"2023-11-25T04:22:47.039405+0000","last_unstale":"2023-11-25T04:22:48.047565+0000","last_undegraded":"2023-11-25T04:22:48.047565+0000","last_fullsized":"2023-11-25T04:22:48.047565+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:56:59.689578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047396+0000","last_change":"2023-11-25T04:22:47.037971+0000","last_active":"2023-11-25T04:22:48.047396+0000","last_peered":"2023-11-25T04:22:48.047396+0000","last_clean":"2023-11-25T04:22:48.047396+0000","last_became_active":"2023-11-25T04:22:47.036736+0000","last_became_peered":"2023-11-25T04:22:47.036736+0000","last_unstale":"2023-11-25T04:22:48.047396+0000","last_undegraded":"2023-11-25T04:22:48.047396+0000","last_fullsized":"2023-11-25T04:22:48.047396+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:02:07.088241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047307+0000","last_change":"2023-11-25T04:22:47.041728+0000","last_active":"2023-11-25T04:22:48.047307+0000","last_peered":"2023-11-25T04:22:48.047307+0000","last_clean":"2023-11-25T04:22:48.047307+0000","last_became_active":"2023-11-25T04:22:47.041017+0000","last_became_peered":"2023-11-25T04:22:47.041017+0000","last_unstale":"2023-11-25T04:22:48.047307+0000","last_undegraded":"2023-11-25T04:22:48.047307+0000","last_fullsized":"2023-11-25T04:22:48.047307+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:57:28.855151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048097+0000","last_change":"2023-11-25T04:22:47.038397+0000","last_active":"2023-11-25T04:22:48.048097+0000","last_peered":"2023-11-25T04:22:48.048097+0000","last_clean":"2023-11-25T04:22:48.048097+0000","last_became_active":"2023-11-25T04:22:47.037328+0000","last_became_peered":"2023-11-25T04:22:47.037328+0000","last_unstale":"2023-11-25T04:22:48.048097+0000","last_undegraded":"2023-11-25T04:22:48.048097+0000","last_fullsized":"2023-11-25T04:22:48.048097+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:56:59.689578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048223+0000","last_change":"2023-11-25T04:22:47.040182+0000","last_active":"2023-11-25T04:22:48.048223+0000","last_peered":"2023-11-25T04:22:48.048223+0000","last_clean":"2023-11-25T04:22:48.048223+0000","last_became_active":"2023-11-25T04:22:47.039892+0000","last_became_peered":"2023-11-25T04:22:47.039892+0000","last_unstale":"2023-11-25T04:22:48.048223+0000","last_undegraded":"2023-11-25T04:22:48.048223+0000","last_fullsized":"2023-11-25T04:22:48.048223+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:38:09.176582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048202+0000","last_change":"2023-11-25T04:22:47.039907+0000","last_active":"2023-11-25T04:22:48.048202+0000","last_peered":"2023-11-25T04:22:48.048202+0000","last_clean":"2023-11-25T04:22:48.048202+0000","last_became_active":"2023-11-25T04:22:47.039046+0000","last_became_peered":"2023-11-25T04:22:47.039046+0000","last_unstale":"2023-11-25T04:22:48.048202+0000","last_undegraded":"2023-11-25T04:22:48.048202+0000","last_fullsized":"2023-11-25T04:22:48.048202+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:01:51.418774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047609+0000","last_change":"2023-11-25T04:22:47.037618+0000","last_active":"2023-11-25T04:22:48.047609+0000","last_peered":"2023-11-25T04:22:48.047609+0000","last_clean":"2023-11-25T04:22:48.047609+0000","last_became_active":"2023-11-25T04:22:47.036806+0000","last_became_peered":"2023-11-25T04:22:47.036806+0000","last_unstale":"2023-11-25T04:22:48.047609+0000","last_undegraded":"2023-11-25T04:22:48.047609+0000","last_fullsized":"2023-11-25T04:22:48.047609+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:12:51.990448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047608+0000","last_change":"2023-11-25T04:22:47.040388+0000","last_active":"2023-11-25T04:22:48.047608+0000","last_peered":"2023-11-25T04:22:48.047608+0000","last_clean":"2023-11-25T04:22:48.047608+0000","last_became_active":"2023-11-25T04:22:47.040129+0000","last_became_peered":"2023-11-25T04:22:47.040129+0000","last_unstale":"2023-11-25T04:22:48.047608+0000","last_undegraded":"2023-11-25T04:22:48.047608+0000","last_fullsized":"2023-11-25T04:22:48.047608+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:31:54.857270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047643+0000","last_change":"2023-11-25T04:22:47.038887+0000","last_active":"2023-11-25T04:22:48.047643+0000","last_peered":"2023-11-25T04:22:48.047643+0000","last_clean":"2023-11-25T04:22:48.047643+0000","last_became_active":"2023-11-25T04:22:47.038014+0000","last_became_peered":"2023-11-25T04:22:47.038014+0000","last_unstale":"2023-11-25T04:22:48.047643+0000","last_undegraded":"2023-11-25T04:22:48.047643+0000","last_fullsized":"2023-11-25T04:22:48.047643+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:38:38.839941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047471+0000","last_change":"2023-11-25T04:22:47.035843+0000","last_active":"2023-11-25T04:22:48.047471+0000","last_peered":"2023-11-25T04:22:48.047471+0000","last_clean":"2023-11-25T04:22:48.047471+0000","last_became_active":"2023-11-25T04:22:47.035453+0000","last_became_peered":"2023-11-25T04:22:47.035453+0000","last_unstale":"2023-11-25T04:22:48.047471+0000","last_undegraded":"2023-11-25T04:22:48.047471+0000","last_fullsized":"2023-11-25T04:22:48.047471+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:45:40.065146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047308+0000","last_change":"2023-11-25T04:22:47.041778+0000","last_active":"2023-11-25T04:22:48.047308+0000","last_peered":"2023-11-25T04:22:48.047308+0000","last_clean":"2023-11-25T04:22:48.047308+0000","last_became_active":"2023-11-25T04:22:47.041115+0000","last_became_peered":"2023-11-25T04:22:47.041115+0000","last_unstale":"2023-11-25T04:22:48.047308+0000","last_undegraded":"2023-11-25T04:22:48.047308+0000","last_fullsized":"2023-11-25T04:22:48.047308+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:30:04.568790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048422+0000","last_change":"2023-11-25T04:22:47.039389+0000","last_active":"2023-11-25T04:22:48.048422+0000","last_peered":"2023-11-25T04:22:48.048422+0000","last_clean":"2023-11-25T04:22:48.048422+0000","last_became_active":"2023-11-25T04:22:47.039123+0000","last_became_peered":"2023-11-25T04:22:47.039123+0000","last_unstale":"2023-11-25T04:22:48.048422+0000","last_undegraded":"2023-11-25T04:22:48.048422+0000","last_fullsized":"2023-11-25T04:22:48.048422+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:09:11.570561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048403+0000","last_change":"2023-11-25T04:22:47.036946+0000","last_active":"2023-11-25T04:22:48.048403+0000","last_peered":"2023-11-25T04:22:48.048403+0000","last_clean":"2023-11-25T04:22:48.048403+0000","last_became_active":"2023-11-25T04:22:47.036411+0000","last_became_peered":"2023-11-25T04:22:47.036411+0000","last_unstale":"2023-11-25T04:22:48.048403+0000","last_undegraded":"2023-11-25T04:22:48.048403+0000","last_fullsized":"2023-11-25T04:22:48.048403+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:16:29.085050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047729+0000","last_change":"2023-11-25T04:22:47.038983+0000","last_active":"2023-11-25T04:22:48.047729+0000","last_peered":"2023-11-25T04:22:48.047729+0000","last_clean":"2023-11-25T04:22:48.047729+0000","last_became_active":"2023-11-25T04:22:47.038477+0000","last_became_peered":"2023-11-25T04:22:47.038477+0000","last_unstale":"2023-11-25T04:22:48.047729+0000","last_undegraded":"2023-11-25T04:22:48.047729+0000","last_fullsized":"2023-11-25T04:22:48.047729+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:30:04.568790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047891+0000","last_change":"2023-11-25T04:22:47.038155+0000","last_active":"2023-11-25T04:22:48.047891+0000","last_peered":"2023-11-25T04:22:48.047891+0000","last_clean":"2023-11-25T04:22:48.047891+0000","last_became_active":"2023-11-25T04:22:47.037740+0000","last_became_peered":"2023-11-25T04:22:47.037740+0000","last_unstale":"2023-11-25T04:22:48.047891+0000","last_undegraded":"2023-11-25T04:22:48.047891+0000","last_fullsized":"2023-11-25T04:22:48.047891+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:09:11.570561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048543+0000","last_change":"2023-11-25T04:22:47.040641+0000","last_active":"2023-11-25T04:22:48.048543+0000","last_peered":"2023-11-25T04:22:48.048543+0000","last_clean":"2023-11-25T04:22:48.048543+0000","last_became_active":"2023-11-25T04:22:47.039409+0000","last_became_peered":"2023-11-25T04:22:47.039409+0000","last_unstale":"2023-11-25T04:22:48.048543+0000","last_undegraded":"2023-11-25T04:22:48.048543+0000","last_fullsized":"2023-11-25T04:22:48.048543+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:23:15.914817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047384+0000","last_change":"2023-11-25T04:22:47.043456+0000","last_active":"2023-11-25T04:22:48.047384+0000","last_peered":"2023-11-25T04:22:48.047384+0000","last_clean":"2023-11-25T04:22:48.047384+0000","last_became_active":"2023-11-25T04:22:47.043098+0000","last_became_peered":"2023-11-25T04:22:47.043098+0000","last_unstale":"2023-11-25T04:22:48.047384+0000","last_undegraded":"2023-11-25T04:22:48.047384+0000","last_fullsized":"2023-11-25T04:22:48.047384+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:47:57.307757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047103+0000","last_change":"2023-11-25T04:22:47.035260+0000","last_active":"2023-11-25T04:22:48.047103+0000","last_peered":"2023-11-25T04:22:48.047103+0000","last_clean":"2023-11-25T04:22:48.047103+0000","last_became_active":"2023-11-25T04:22:47.034702+0000","last_became_peered":"2023-11-25T04:22:47.034702+0000","last_unstale":"2023-11-25T04:22:48.047103+0000","last_undegraded":"2023-11-25T04:22:48.047103+0000","last_fullsized":"2023-11-25T04:22:48.047103+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:49:02.500022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047259+0000","last_change":"2023-11-25T04:22:47.036805+0000","last_active":"2023-11-25T04:22:48.047259+0000","last_peered":"2023-11-25T04:22:48.047259+0000","last_clean":"2023-11-25T04:22:48.047259+0000","last_became_active":"2023-11-25T04:22:47.036135+0000","last_became_peered":"2023-11-25T04:22:47.036135+0000","last_unstale":"2023-11-25T04:22:48.047259+0000","last_undegraded":"2023-11-25T04:22:48.047259+0000","last_fullsized":"2023-11-25T04:22:48.047259+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:25:21.730992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048430+0000","last_change":"2023-11-25T04:22:47.036611+0000","last_active":"2023-11-25T04:22:48.048430+0000","last_peered":"2023-11-25T04:22:48.048430+0000","last_clean":"2023-11-25T04:22:48.048430+0000","last_became_active":"2023-11-25T04:22:47.035762+0000","last_became_peered":"2023-11-25T04:22:47.035762+0000","last_unstale":"2023-11-25T04:22:48.048430+0000","last_undegraded":"2023-11-25T04:22:48.048430+0000","last_fullsized":"2023-11-25T04:22:48.048430+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:09:11.570561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048216+0000","last_change":"2023-11-25T04:22:47.042423+0000","last_active":"2023-11-25T04:22:48.048216+0000","last_peered":"2023-11-25T04:22:48.048216+0000","last_clean":"2023-11-25T04:22:48.048216+0000","last_became_active":"2023-11-25T04:22:47.042022+0000","last_became_peered":"2023-11-25T04:22:47.042022+0000","last_unstale":"2023-11-25T04:22:48.048216+0000","last_undegraded":"2023-11-25T04:22:48.048216+0000","last_fullsized":"2023-11-25T04:22:48.048216+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:38:09.176582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048337+0000","last_change":"2023-11-25T04:22:47.044040+0000","last_active":"2023-11-25T04:22:48.048337+0000","last_peered":"2023-11-25T04:22:48.048337+0000","last_clean":"2023-11-25T04:22:48.048337+0000","last_became_active":"2023-11-25T04:22:47.043329+0000","last_became_peered":"2023-11-25T04:22:47.043329+0000","last_unstale":"2023-11-25T04:22:48.048337+0000","last_undegraded":"2023-11-25T04:22:48.048337+0000","last_fullsized":"2023-11-25T04:22:48.048337+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:10:59.482775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047954+0000","last_change":"2023-11-25T04:22:47.036485+0000","last_active":"2023-11-25T04:22:48.047954+0000","last_peered":"2023-11-25T04:22:48.047954+0000","last_clean":"2023-11-25T04:22:48.047954+0000","last_became_active":"2023-11-25T04:22:47.035799+0000","last_became_peered":"2023-11-25T04:22:47.035799+0000","last_unstale":"2023-11-25T04:22:48.047954+0000","last_undegraded":"2023-11-25T04:22:48.047954+0000","last_fullsized":"2023-11-25T04:22:48.047954+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:45:40.065146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047835+0000","last_change":"2023-11-25T04:22:47.037668+0000","last_active":"2023-11-25T04:22:48.047835+0000","last_peered":"2023-11-25T04:22:48.047835+0000","last_clean":"2023-11-25T04:22:48.047835+0000","last_became_active":"2023-11-25T04:22:47.036823+0000","last_became_peered":"2023-11-25T04:22:47.036823+0000","last_unstale":"2023-11-25T04:22:48.047835+0000","last_undegraded":"2023-11-25T04:22:48.047835+0000","last_fullsized":"2023-11-25T04:22:48.047835+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:02:07.088241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047758+0000","last_change":"2023-11-25T04:22:47.038117+0000","last_active":"2023-11-25T04:22:48.047758+0000","last_peered":"2023-11-25T04:22:48.047758+0000","last_clean":"2023-11-25T04:22:48.047758+0000","last_became_active":"2023-11-25T04:22:47.037751+0000","last_became_peered":"2023-11-25T04:22:47.037751+0000","last_unstale":"2023-11-25T04:22:48.047758+0000","last_undegraded":"2023-11-25T04:22:48.047758+0000","last_fullsized":"2023-11-25T04:22:48.047758+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:30:04.568790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047184+0000","last_change":"2023-11-25T04:22:47.037786+0000","last_active":"2023-11-25T04:22:48.047184+0000","last_peered":"2023-11-25T04:22:48.047184+0000","last_clean":"2023-11-25T04:22:48.047184+0000","last_became_active":"2023-11-25T04:22:47.037206+0000","last_became_peered":"2023-11-25T04:22:47.037206+0000","last_unstale":"2023-11-25T04:22:48.047184+0000","last_undegraded":"2023-11-25T04:22:48.047184+0000","last_fullsized":"2023-11-25T04:22:48.047184+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:11:40.532140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047505+0000","last_change":"2023-11-25T04:22:47.041325+0000","last_active":"2023-11-25T04:22:48.047505+0000","last_peered":"2023-11-25T04:22:48.047505+0000","last_clean":"2023-11-25T04:22:48.047505+0000","last_became_active":"2023-11-25T04:22:47.040086+0000","last_became_peered":"2023-11-25T04:22:47.040086+0000","last_unstale":"2023-11-25T04:22:48.047505+0000","last_undegraded":"2023-11-25T04:22:48.047505+0000","last_fullsized":"2023-11-25T04:22:48.047505+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:15:39.511043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047822+0000","last_change":"2023-11-25T04:22:47.036689+0000","last_active":"2023-11-25T04:22:48.047822+0000","last_peered":"2023-11-25T04:22:48.047822+0000","last_clean":"2023-11-25T04:22:48.047822+0000","last_became_active":"2023-11-25T04:22:47.036107+0000","last_became_peered":"2023-11-25T04:22:47.036107+0000","last_unstale":"2023-11-25T04:22:48.047822+0000","last_undegraded":"2023-11-25T04:22:48.047822+0000","last_fullsized":"2023-11-25T04:22:48.047822+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:02:07.088241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"18'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048313+0000","last_change":"2023-11-25T04:22:46.040844+0000","last_active":"2023-11-25T04:22:48.048313+0000","last_peered":"2023-11-25T04:22:48.048313+0000","last_clean":"2023-11-25T04:22:48.048313+0000","last_became_active":"2023-11-25T04:22:46.040395+0000","last_became_peered":"2023-11-25T04:22:46.040395+0000","last_unstale":"2023-11-25T04:22:48.048313+0000","last_undegraded":"2023-11-25T04:22:48.048313+0000","last_fullsized":"2023-11-25T04:22:48.048313+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:11:39.522677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048200+0000","last_change":"2023-11-25T04:22:47.037888+0000","last_active":"2023-11-25T04:22:48.048200+0000","last_peered":"2023-11-25T04:22:48.048200+0000","last_clean":"2023-11-25T04:22:48.048200+0000","last_became_active":"2023-11-25T04:22:47.037158+0000","last_became_peered":"2023-11-25T04:22:47.037158+0000","last_unstale":"2023-11-25T04:22:48.048200+0000","last_undegraded":"2023-11-25T04:22:48.048200+0000","last_fullsized":"2023-11-25T04:22:48.048200+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:56:59.689578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047391+0000","last_change":"2023-11-25T04:22:46.036054+0000","last_active":"2023-11-25T04:22:48.047391+0000","last_peered":"2023-11-25T04:22:48.047391+0000","last_clean":"2023-11-25T04:22:48.047391+0000","last_became_active":"2023-11-25T04:22:46.035620+0000","last_became_peered":"2023-11-25T04:22:46.035620+0000","last_unstale":"2023-11-25T04:22:48.047391+0000","last_undegraded":"2023-11-25T04:22:48.047391+0000","last_fullsized":"2023-11-25T04:22:48.047391+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:19:48.371980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048391+0000","last_change":"2023-11-25T04:22:47.045025+0000","last_active":"2023-11-25T04:22:48.048391+0000","last_peered":"2023-11-25T04:22:48.048391+0000","last_clean":"2023-11-25T04:22:48.048391+0000","last_became_active":"2023-11-25T04:22:47.044697+0000","last_became_peered":"2023-11-25T04:22:47.044697+0000","last_unstale":"2023-11-25T04:22:48.048391+0000","last_undegraded":"2023-11-25T04:22:48.048391+0000","last_fullsized":"2023-11-25T04:22:48.048391+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:16:29.085050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047517+0000","last_change":"2023-11-25T04:22:46.037169+0000","last_active":"2023-11-25T04:22:48.047517+0000","last_peered":"2023-11-25T04:22:48.047517+0000","last_clean":"2023-11-25T04:22:48.047517+0000","last_became_active":"2023-11-25T04:22:46.036948+0000","last_became_peered":"2023-11-25T04:22:46.036948+0000","last_unstale":"2023-11-25T04:22:48.047517+0000","last_undegraded":"2023-11-25T04:22:48.047517+0000","last_fullsized":"2023-11-25T04:22:48.047517+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:45:14.670326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047715+0000","last_change":"2023-11-25T04:22:47.044860+0000","last_active":"2023-11-25T04:22:48.047715+0000","last_peered":"2023-11-25T04:22:48.047715+0000","last_clean":"2023-11-25T04:22:48.047715+0000","last_became_active":"2023-11-25T04:22:47.044551+0000","last_became_peered":"2023-11-25T04:22:47.044551+0000","last_unstale":"2023-11-25T04:22:48.047715+0000","last_undegraded":"2023-11-25T04:22:48.047715+0000","last_fullsized":"2023-11-25T04:22:48.047715+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:09:55.357642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.10","version":"18'8","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047593+0000","last_change":"2023-11-25T04:22:46.040356+0000","last_active":"2023-11-25T04:22:48.047593+0000","last_peered":"2023-11-25T04:22:48.047593+0000","last_clean":"2023-11-25T04:22:48.047593+0000","last_became_active":"2023-11-25T04:22:46.039802+0000","last_became_peered":"2023-11-25T04:22:46.039802+0000","last_unstale":"2023-11-25T04:22:48.047593+0000","last_undegraded":"2023-11-25T04:22:48.047593+0000","last_fullsized":"2023-11-25T04:22:48.047593+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:09:54.348179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"18'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048372+0000","last_change":"2023-11-25T04:22:46.040082+0000","last_active":"2023-11-25T04:22:48.048372+0000","last_peered":"2023-11-25T04:22:48.048372+0000","last_clean":"2023-11-25T04:22:48.048372+0000","last_became_active":"2023-11-25T04:22:46.039763+0000","last_became_peered":"2023-11-25T04:22:46.039763+0000","last_unstale":"2023-11-25T04:22:48.048372+0000","last_undegraded":"2023-11-25T04:22:48.048372+0000","last_fullsized":"2023-11-25T04:22:48.048372+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:10:20.178136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047265+0000","last_change":"2023-11-25T04:22:47.043644+0000","last_active":"2023-11-25T04:22:48.047265+0000","last_peered":"2023-11-25T04:22:48.047265+0000","last_clean":"2023-11-25T04:22:48.047265+0000","last_became_active":"2023-11-25T04:22:47.043221+0000","last_became_peered":"2023-11-25T04:22:47.043221+0000","last_unstale":"2023-11-25T04:22:48.047265+0000","last_undegraded":"2023-11-25T04:22:48.047265+0000","last_fullsized":"2023-11-25T04:22:48.047265+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:41:16.672268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"18'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048462+0000","last_change":"2023-11-25T04:22:46.035806+0000","last_active":"2023-11-25T04:22:48.048462+0000","last_peered":"2023-11-25T04:22:48.048462+0000","last_clean":"2023-11-25T04:22:48.048462+0000","last_became_active":"2023-11-25T04:22:46.035418+0000","last_became_peered":"2023-11-25T04:22:46.035418+0000","last_unstale":"2023-11-25T04:22:48.048462+0000","last_undegraded":"2023-11-25T04:22:48.048462+0000","last_fullsized":"2023-11-25T04:22:48.048462+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:31:53.068767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047508+0000","last_change":"2023-11-25T04:22:47.039685+0000","last_active":"2023-11-25T04:22:48.047508+0000","last_peered":"2023-11-25T04:22:48.047508+0000","last_clean":"2023-11-25T04:22:48.047508+0000","last_became_active":"2023-11-25T04:22:47.039296+0000","last_became_peered":"2023-11-25T04:22:47.039296+0000","last_unstale":"2023-11-25T04:22:48.047508+0000","last_undegraded":"2023-11-25T04:22:48.047508+0000","last_fullsized":"2023-11-25T04:22:48.047508+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:11:40.532140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"18'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048658+0000","last_change":"2023-11-25T04:22:46.039953+0000","last_active":"2023-11-25T04:22:48.048658+0000","last_peered":"2023-11-25T04:22:48.048658+0000","last_clean":"2023-11-25T04:22:48.048658+0000","last_became_active":"2023-11-25T04:22:46.039660+0000","last_became_peered":"2023-11-25T04:22:46.039660+0000","last_unstale":"2023-11-25T04:22:48.048658+0000","last_undegraded":"2023-11-25T04:22:48.048658+0000","last_fullsized":"2023-11-25T04:22:48.048658+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:59:44.482613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047887+0000","last_change":"2023-11-25T04:22:47.037912+0000","last_active":"2023-11-25T04:22:48.047887+0000","last_peered":"2023-11-25T04:22:48.047887+0000","last_clean":"2023-11-25T04:22:48.047887+0000","last_became_active":"2023-11-25T04:22:47.036552+0000","last_became_peered":"2023-11-25T04:22:47.036552+0000","last_unstale":"2023-11-25T04:22:48.047887+0000","last_undegraded":"2023-11-25T04:22:48.047887+0000","last_fullsized":"2023-11-25T04:22:48.047887+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:38:09.176582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048295+0000","last_change":"2023-11-25T04:22:46.039176+0000","last_active":"2023-11-25T04:22:48.048295+0000","last_peered":"2023-11-25T04:22:48.048295+0000","last_clean":"2023-11-25T04:22:48.048295+0000","last_became_active":"2023-11-25T04:22:46.038842+0000","last_became_peered":"2023-11-25T04:22:46.038842+0000","last_unstale":"2023-11-25T04:22:48.048295+0000","last_undegraded":"2023-11-25T04:22:48.048295+0000","last_fullsized":"2023-11-25T04:22:48.048295+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:47:56.298293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047733+0000","last_change":"2023-11-25T04:22:47.042155+0000","last_active":"2023-11-25T04:22:48.047733+0000","last_peered":"2023-11-25T04:22:48.047733+0000","last_clean":"2023-11-25T04:22:48.047733+0000","last_became_active":"2023-11-25T04:22:47.041433+0000","last_became_peered":"2023-11-25T04:22:47.041433+0000","last_unstale":"2023-11-25T04:22:48.047733+0000","last_undegraded":"2023-11-25T04:22:48.047733+0000","last_fullsized":"2023-11-25T04:22:48.047733+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:57:28.855151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"18'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048201+0000","last_change":"2023-11-25T04:22:46.037827+0000","last_active":"2023-11-25T04:22:48.048201+0000","last_peered":"2023-11-25T04:22:48.048201+0000","last_clean":"2023-11-25T04:22:48.048201+0000","last_became_active":"2023-11-25T04:22:46.036515+0000","last_became_peered":"2023-11-25T04:22:46.036515+0000","last_unstale":"2023-11-25T04:22:48.048201+0000","last_undegraded":"2023-11-25T04:22:48.048201+0000","last_fullsized":"2023-11-25T04:22:48.048201+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:10:20.178136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048154+0000","last_change":"2023-11-25T04:22:47.038856+0000","last_active":"2023-11-25T04:22:48.048154+0000","last_peered":"2023-11-25T04:22:48.048154+0000","last_clean":"2023-11-25T04:22:48.048154+0000","last_became_active":"2023-11-25T04:22:47.038190+0000","last_became_peered":"2023-11-25T04:22:47.038190+0000","last_unstale":"2023-11-25T04:22:48.048154+0000","last_undegraded":"2023-11-25T04:22:48.048154+0000","last_fullsized":"2023-11-25T04:22:48.048154+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:29:37.268624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047789+0000","last_change":"2023-11-25T04:22:46.038473+0000","last_active":"2023-11-25T04:22:48.047789+0000","last_peered":"2023-11-25T04:22:48.047789+0000","last_clean":"2023-11-25T04:22:48.047789+0000","last_became_active":"2023-11-25T04:22:46.038159+0000","last_became_peered":"2023-11-25T04:22:46.038159+0000","last_unstale":"2023-11-25T04:22:48.047789+0000","last_undegraded":"2023-11-25T04:22:48.047789+0000","last_fullsized":"2023-11-25T04:22:48.047789+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:31:53.847807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048568+0000","last_change":"2023-11-25T04:22:47.036222+0000","last_active":"2023-11-25T04:22:48.048568+0000","last_peered":"2023-11-25T04:22:48.048568+0000","last_clean":"2023-11-25T04:22:48.048568+0000","last_became_active":"2023-11-25T04:22:47.035871+0000","last_became_peered":"2023-11-25T04:22:47.035871+0000","last_unstale":"2023-11-25T04:22:48.048568+0000","last_undegraded":"2023-11-25T04:22:48.048568+0000","last_fullsized":"2023-11-25T04:22:48.048568+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:13:35.199538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047681+0000","last_change":"2023-11-25T04:22:46.035907+0000","last_active":"2023-11-25T04:22:48.047681+0000","last_peered":"2023-11-25T04:22:48.047681+0000","last_clean":"2023-11-25T04:22:48.047681+0000","last_became_active":"2023-11-25T04:22:46.035612+0000","last_became_peered":"2023-11-25T04:22:46.035612+0000","last_unstale":"2023-11-25T04:22:48.047681+0000","last_undegraded":"2023-11-25T04:22:48.047681+0000","last_fullsized":"2023-11-25T04:22:48.047681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:00:35.249875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048441+0000","last_change":"2023-11-25T04:22:47.038725+0000","last_active":"2023-11-25T04:22:48.048441+0000","last_peered":"2023-11-25T04:22:48.048441+0000","last_clean":"2023-11-25T04:22:48.048441+0000","last_became_active":"2023-11-25T04:22:47.038124+0000","last_became_peered":"2023-11-25T04:22:47.038124+0000","last_unstale":"2023-11-25T04:22:48.048441+0000","last_undegraded":"2023-11-25T04:22:48.048441+0000","last_fullsized":"2023-11-25T04:22:48.048441+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:00:26.187878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047437+0000","last_change":"2023-11-25T04:22:46.034509+0000","last_active":"2023-11-25T04:22:48.047437+0000","last_peered":"2023-11-25T04:22:48.047437+0000","last_clean":"2023-11-25T04:22:48.047437+0000","last_became_active":"2023-11-25T04:22:46.034158+0000","last_became_peered":"2023-11-25T04:22:46.034158+0000","last_unstale":"2023-11-25T04:22:48.047437+0000","last_undegraded":"2023-11-25T04:22:48.047437+0000","last_fullsized":"2023-11-25T04:22:48.047437+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:26:25.338117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048005+0000","last_change":"2023-11-25T04:22:47.038800+0000","last_active":"2023-11-25T04:22:48.048005+0000","last_peered":"2023-11-25T04:22:48.048005+0000","last_clean":"2023-11-25T04:22:48.048005+0000","last_became_active":"2023-11-25T04:22:47.038081+0000","last_became_peered":"2023-11-25T04:22:47.038081+0000","last_unstale":"2023-11-25T04:22:48.048005+0000","last_undegraded":"2023-11-25T04:22:48.048005+0000","last_fullsized":"2023-11-25T04:22:48.048005+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:45:40.065146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"12'157","reported_seq":214,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-25T04:22:47.033945+0000","last_change":"2023-11-25T04:22:36.931828+0000","last_active":"2023-11-25T04:22:47.033945+0000","last_peered":"2023-11-25T04:22:47.033945+0000","last_clean":"2023-11-25T04:22:47.033945+0000","last_became_active":"2023-11-25T04:22:36.931582+0000","last_became_peered":"2023-11-25T04:22:36.931582+0000","last_unstale":"2023-11-25T04:22:47.033945+0000","last_undegraded":"2023-11-25T04:22:47.033945+0000","last_fullsized":"2023-11-25T04:22:47.033945+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:35.919850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:35.919850+0000","last_clean_scrub_stamp":"2023-11-25T04:22:35.919850+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:59:59.198309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-25T04:22:47.031925+0000","last_change":"2023-11-25T04:22:41.999590+0000","last_active":"2023-11-25T04:22:47.031925+0000","last_peered":"2023-11-25T04:22:47.031925+0000","last_clean":"2023-11-25T04:22:47.031925+0000","last_became_active":"2023-11-25T04:22:41.999314+0000","last_became_peered":"2023-11-25T04:22:41.999314+0000","last_unstale":"2023-11-25T04:22:47.031925+0000","last_undegraded":"2023-11-25T04:22:47.031925+0000","last_fullsized":"2023-11-25T04:22:47.031925+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:20:12.785425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047840+0000","last_change":"2023-11-25T04:22:46.037044+0000","last_active":"2023-11-25T04:22:48.047840+0000","last_peered":"2023-11-25T04:22:48.047840+0000","last_clean":"2023-11-25T04:22:48.047840+0000","last_became_active":"2023-11-25T04:22:46.035916+0000","last_became_peered":"2023-11-25T04:22:46.035916+0000","last_unstale":"2023-11-25T04:22:48.047840+0000","last_undegraded":"2023-11-25T04:22:48.047840+0000","last_fullsized":"2023-11-25T04:22:48.047840+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:38:20.501102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048611+0000","last_change":"2023-11-25T04:22:47.036222+0000","last_active":"2023-11-25T04:22:48.048611+0000","last_peered":"2023-11-25T04:22:48.048611+0000","last_clean":"2023-11-25T04:22:48.048611+0000","last_became_active":"2023-11-25T04:22:47.035484+0000","last_became_peered":"2023-11-25T04:22:47.035484+0000","last_unstale":"2023-11-25T04:22:48.048611+0000","last_undegraded":"2023-11-25T04:22:48.048611+0000","last_fullsized":"2023-11-25T04:22:48.048611+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:18:56.845916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-25T04:22:47.032801+0000","last_change":"2023-11-25T04:22:42.000257+0000","last_active":"2023-11-25T04:22:47.032801+0000","last_peered":"2023-11-25T04:22:47.032801+0000","last_clean":"2023-11-25T04:22:47.032801+0000","last_became_active":"2023-11-25T04:22:41.999969+0000","last_became_peered":"2023-11-25T04:22:41.999969+0000","last_unstale":"2023-11-25T04:22:47.032801+0000","last_undegraded":"2023-11-25T04:22:47.032801+0000","last_fullsized":"2023-11-25T04:22:47.032801+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:21:34.424091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047780+0000","last_change":"2023-11-25T04:22:46.036121+0000","last_active":"2023-11-25T04:22:48.047780+0000","last_peered":"2023-11-25T04:22:48.047780+0000","last_clean":"2023-11-25T04:22:48.047780+0000","last_became_active":"2023-11-25T04:22:46.035654+0000","last_became_peered":"2023-11-25T04:22:46.035654+0000","last_unstale":"2023-11-25T04:22:48.047780+0000","last_undegraded":"2023-11-25T04:22:48.047780+0000","last_fullsized":"2023-11-25T04:22:48.047780+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:27:31.901618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048273+0000","last_change":"2023-11-25T04:22:47.039583+0000","last_active":"2023-11-25T04:22:48.048273+0000","last_peered":"2023-11-25T04:22:48.048273+0000","last_clean":"2023-11-25T04:22:48.048273+0000","last_became_active":"2023-11-25T04:22:47.039197+0000","last_became_peered":"2023-11-25T04:22:47.039197+0000","last_unstale":"2023-11-25T04:22:48.048273+0000","last_undegraded":"2023-11-25T04:22:48.048273+0000","last_fullsized":"2023-11-25T04:22:48.048273+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:37:30.959147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-25T04:22:47.029239+0000","last_change":"2023-11-25T04:22:42.000382+0000","last_active":"2023-11-25T04:22:47.029239+0000","last_peered":"2023-11-25T04:22:47.029239+0000","last_clean":"2023-11-25T04:22:47.029239+0000","last_became_active":"2023-11-25T04:22:42.000041+0000","last_became_peered":"2023-11-25T04:22:42.000041+0000","last_unstale":"2023-11-25T04:22:47.029239+0000","last_undegraded":"2023-11-25T04:22:47.029239+0000","last_fullsized":"2023-11-25T04:22:47.029239+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:08:19.655864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047444+0000","last_change":"2023-11-25T04:22:46.037431+0000","last_active":"2023-11-25T04:22:48.047444+0000","last_peered":"2023-11-25T04:22:48.047444+0000","last_clean":"2023-11-25T04:22:48.047444+0000","last_became_active":"2023-11-25T04:22:46.037141+0000","last_became_peered":"2023-11-25T04:22:46.037141+0000","last_unstale":"2023-11-25T04:22:48.047444+0000","last_undegraded":"2023-11-25T04:22:48.047444+0000","last_fullsized":"2023-11-25T04:22:48.047444+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:40:42.500663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048182+0000","last_change":"2023-11-25T04:22:47.038454+0000","last_active":"2023-11-25T04:22:48.048182+0000","last_peered":"2023-11-25T04:22:48.048182+0000","last_clean":"2023-11-25T04:22:48.048182+0000","last_became_active":"2023-11-25T04:22:47.038046+0000","last_became_peered":"2023-11-25T04:22:47.038046+0000","last_unstale":"2023-11-25T04:22:48.048182+0000","last_undegraded":"2023-11-25T04:22:48.048182+0000","last_fullsized":"2023-11-25T04:22:48.048182+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:38:38.839941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-25T04:22:47.031534+0000","last_change":"2023-11-25T04:22:41.999424+0000","last_active":"2023-11-25T04:22:47.031534+0000","last_peered":"2023-11-25T04:22:47.031534+0000","last_clean":"2023-11-25T04:22:47.031534+0000","last_became_active":"2023-11-25T04:22:41.999179+0000","last_became_peered":"2023-11-25T04:22:41.999179+0000","last_unstale":"2023-11-25T04:22:47.031534+0000","last_undegraded":"2023-11-25T04:22:47.031534+0000","last_fullsized":"2023-11-25T04:22:47.031534+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:12:04.291696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048523+0000","last_change":"2023-11-25T04:22:46.038758+0000","last_active":"2023-11-25T04:22:48.048523+0000","last_peered":"2023-11-25T04:22:48.048523+0000","last_clean":"2023-11-25T04:22:48.048523+0000","last_became_active":"2023-11-25T04:22:46.038470+0000","last_became_peered":"2023-11-25T04:22:46.038470+0000","last_unstale":"2023-11-25T04:22:48.048523+0000","last_undegraded":"2023-11-25T04:22:48.048523+0000","last_fullsized":"2023-11-25T04:22:48.048523+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:49:01.490558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048075+0000","last_change":"2023-11-25T04:22:47.037355+0000","last_active":"2023-11-25T04:22:48.048075+0000","last_peered":"2023-11-25T04:22:48.048075+0000","last_clean":"2023-11-25T04:22:48.048075+0000","last_became_active":"2023-11-25T04:22:47.036924+0000","last_became_peered":"2023-11-25T04:22:47.036924+0000","last_unstale":"2023-11-25T04:22:48.048075+0000","last_undegraded":"2023-11-25T04:22:48.048075+0000","last_fullsized":"2023-11-25T04:22:48.048075+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:15:39.511043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048677+0000","last_change":"2023-11-25T04:22:46.039025+0000","last_active":"2023-11-25T04:22:48.048677+0000","last_peered":"2023-11-25T04:22:48.048677+0000","last_clean":"2023-11-25T04:22:48.048677+0000","last_became_active":"2023-11-25T04:22:46.038762+0000","last_became_peered":"2023-11-25T04:22:46.038762+0000","last_unstale":"2023-11-25T04:22:48.048677+0000","last_undegraded":"2023-11-25T04:22:48.048677+0000","last_fullsized":"2023-11-25T04:22:48.048677+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:44:47.544509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-25T04:22:47.029103+0000","last_change":"2023-11-25T04:22:42.000204+0000","last_active":"2023-11-25T04:22:47.029103+0000","last_peered":"2023-11-25T04:22:47.029103+0000","last_clean":"2023-11-25T04:22:47.029103+0000","last_became_active":"2023-11-25T04:22:41.999915+0000","last_became_peered":"2023-11-25T04:22:41.999915+0000","last_unstale":"2023-11-25T04:22:47.029103+0000","last_undegraded":"2023-11-25T04:22:47.029103+0000","last_fullsized":"2023-11-25T04:22:47.029103+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:07:32.457905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047409+0000","last_change":"2023-11-25T04:22:47.038119+0000","last_active":"2023-11-25T04:22:48.047409+0000","last_peered":"2023-11-25T04:22:48.047409+0000","last_clean":"2023-11-25T04:22:48.047409+0000","last_became_active":"2023-11-25T04:22:47.037732+0000","last_became_peered":"2023-11-25T04:22:47.037732+0000","last_unstale":"2023-11-25T04:22:48.047409+0000","last_undegraded":"2023-11-25T04:22:48.047409+0000","last_fullsized":"2023-11-25T04:22:48.047409+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:49:02.500022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048004+0000","last_change":"2023-11-25T04:22:46.035885+0000","last_active":"2023-11-25T04:22:48.048004+0000","last_peered":"2023-11-25T04:22:48.048004+0000","last_clean":"2023-11-25T04:22:48.048004+0000","last_became_active":"2023-11-25T04:22:46.035405+0000","last_became_peered":"2023-11-25T04:22:46.035405+0000","last_unstale":"2023-11-25T04:22:48.048004+0000","last_undegraded":"2023-11-25T04:22:48.048004+0000","last_fullsized":"2023-11-25T04:22:48.048004+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:53:47.398746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-25T04:22:47.032524+0000","last_change":"2023-11-25T04:22:41.999067+0000","last_active":"2023-11-25T04:22:47.032524+0000","last_peered":"2023-11-25T04:22:47.032524+0000","last_clean":"2023-11-25T04:22:47.032524+0000","last_became_active":"2023-11-25T04:22:41.998798+0000","last_became_peered":"2023-11-25T04:22:41.998798+0000","last_unstale":"2023-11-25T04:22:47.032524+0000","last_undegraded":"2023-11-25T04:22:47.032524+0000","last_fullsized":"2023-11-25T04:22:47.032524+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:00:04.266051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047617+0000","last_change":"2023-11-25T04:22:47.043464+0000","last_active":"2023-11-25T04:22:48.047617+0000","last_peered":"2023-11-25T04:22:48.047617+0000","last_clean":"2023-11-25T04:22:48.047617+0000","last_became_active":"2023-11-25T04:22:47.043050+0000","last_became_peered":"2023-11-25T04:22:47.043050+0000","last_unstale":"2023-11-25T04:22:48.047617+0000","last_undegraded":"2023-11-25T04:22:48.047617+0000","last_fullsized":"2023-11-25T04:22:48.047617+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:59:45.492077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"18'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048417+0000","last_change":"2023-11-25T04:22:46.034606+0000","last_active":"2023-11-25T04:22:48.048417+0000","last_peered":"2023-11-25T04:22:48.048417+0000","last_clean":"2023-11-25T04:22:48.048417+0000","last_became_active":"2023-11-25T04:22:46.034229+0000","last_became_peered":"2023-11-25T04:22:46.034229+0000","last_unstale":"2023-11-25T04:22:48.048417+0000","last_undegraded":"2023-11-25T04:22:48.048417+0000","last_fullsized":"2023-11-25T04:22:48.048417+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:21:42.146264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-25T04:22:47.031216+0000","last_change":"2023-11-25T04:22:42.000258+0000","last_active":"2023-11-25T04:22:47.031216+0000","last_peered":"2023-11-25T04:22:47.031216+0000","last_clean":"2023-11-25T04:22:47.031216+0000","last_became_active":"2023-11-25T04:22:41.999992+0000","last_became_peered":"2023-11-25T04:22:41.999992+0000","last_unstale":"2023-11-25T04:22:47.031216+0000","last_undegraded":"2023-11-25T04:22:47.031216+0000","last_fullsized":"2023-11-25T04:22:47.031216+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:31:13.715992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047510+0000","last_change":"2023-11-25T04:22:47.042860+0000","last_active":"2023-11-25T04:22:48.047510+0000","last_peered":"2023-11-25T04:22:48.047510+0000","last_clean":"2023-11-25T04:22:48.047510+0000","last_became_active":"2023-11-25T04:22:47.042472+0000","last_became_peered":"2023-11-25T04:22:47.042472+0000","last_unstale":"2023-11-25T04:22:48.047510+0000","last_undegraded":"2023-11-25T04:22:48.047510+0000","last_fullsized":"2023-11-25T04:22:48.047510+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:44:48.553972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048418+0000","last_change":"2023-11-25T04:22:46.038417+0000","last_active":"2023-11-25T04:22:48.048418+0000","last_peered":"2023-11-25T04:22:48.048418+0000","last_clean":"2023-11-25T04:22:48.048418+0000","last_became_active":"2023-11-25T04:22:46.037513+0000","last_became_peered":"2023-11-25T04:22:46.037513+0000","last_unstale":"2023-11-25T04:22:48.048418+0000","last_undegraded":"2023-11-25T04:22:48.048418+0000","last_fullsized":"2023-11-25T04:22:48.048418+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:58:14.877413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-25T04:22:47.030950+0000","last_change":"2023-11-25T04:22:42.000217+0000","last_active":"2023-11-25T04:22:47.030950+0000","last_peered":"2023-11-25T04:22:47.030950+0000","last_clean":"2023-11-25T04:22:47.030950+0000","last_became_active":"2023-11-25T04:22:41.999987+0000","last_became_peered":"2023-11-25T04:22:41.999987+0000","last_unstale":"2023-11-25T04:22:47.030950+0000","last_undegraded":"2023-11-25T04:22:47.030950+0000","last_fullsized":"2023-11-25T04:22:47.030950+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:11:51.100459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047385+0000","last_change":"2023-11-25T04:22:47.036099+0000","last_active":"2023-11-25T04:22:48.047385+0000","last_peered":"2023-11-25T04:22:48.047385+0000","last_clean":"2023-11-25T04:22:48.047385+0000","last_became_active":"2023-11-25T04:22:47.035581+0000","last_became_peered":"2023-11-25T04:22:47.035581+0000","last_unstale":"2023-11-25T04:22:48.047385+0000","last_undegraded":"2023-11-25T04:22:48.047385+0000","last_fullsized":"2023-11-25T04:22:48.047385+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:21:43.155728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048171+0000","last_change":"2023-11-25T04:22:47.040343+0000","last_active":"2023-11-25T04:22:48.048171+0000","last_peered":"2023-11-25T04:22:48.048171+0000","last_clean":"2023-11-25T04:22:48.048171+0000","last_became_active":"2023-11-25T04:22:47.039151+0000","last_became_peered":"2023-11-25T04:22:47.039151+0000","last_unstale":"2023-11-25T04:22:48.048171+0000","last_undegraded":"2023-11-25T04:22:48.048171+0000","last_fullsized":"2023-11-25T04:22:48.048171+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:21:15.981901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"18'6","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048260+0000","last_change":"2023-11-25T04:22:46.040005+0000","last_active":"2023-11-25T04:22:48.048260+0000","last_peered":"2023-11-25T04:22:48.048260+0000","last_clean":"2023-11-25T04:22:48.048260+0000","last_became_active":"2023-11-25T04:22:46.039655+0000","last_became_peered":"2023-11-25T04:22:46.039655+0000","last_unstale":"2023-11-25T04:22:48.048260+0000","last_undegraded":"2023-11-25T04:22:48.048260+0000","last_fullsized":"2023-11-25T04:22:48.048260+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:41:15.662804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047237+0000","last_change":"2023-11-25T04:22:47.037181+0000","last_active":"2023-11-25T04:22:48.047237+0000","last_peered":"2023-11-25T04:22:48.047237+0000","last_clean":"2023-11-25T04:22:48.047237+0000","last_became_active":"2023-11-25T04:22:47.036821+0000","last_became_peered":"2023-11-25T04:22:47.036821+0000","last_unstale":"2023-11-25T04:22:48.047237+0000","last_undegraded":"2023-11-25T04:22:48.047237+0000","last_fullsized":"2023-11-25T04:22:48.047237+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:12:51.990448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"18'3","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048375+0000","last_change":"2023-11-25T04:22:46.035816+0000","last_active":"2023-11-25T04:22:48.048375+0000","last_peered":"2023-11-25T04:22:48.048375+0000","last_clean":"2023-11-25T04:22:48.048375+0000","last_became_active":"2023-11-25T04:22:46.035461+0000","last_became_peered":"2023-11-25T04:22:46.035461+0000","last_unstale":"2023-11-25T04:22:48.048375+0000","last_undegraded":"2023-11-25T04:22:48.048375+0000","last_fullsized":"2023-11-25T04:22:48.048375+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:58:26.155063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048370+0000","last_change":"2023-11-25T04:22:46.040458+0000","last_active":"2023-11-25T04:22:48.048370+0000","last_peered":"2023-11-25T04:22:48.048370+0000","last_clean":"2023-11-25T04:22:48.048370+0000","last_became_active":"2023-11-25T04:22:46.040143+0000","last_became_peered":"2023-11-25T04:22:46.040143+0000","last_unstale":"2023-11-25T04:22:48.048370+0000","last_undegraded":"2023-11-25T04:22:48.048370+0000","last_fullsized":"2023-11-25T04:22:48.048370+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:27:31.901618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047387+0000","last_change":"2023-11-25T04:22:47.040346+0000","last_active":"2023-11-25T04:22:48.047387+0000","last_peered":"2023-11-25T04:22:48.047387+0000","last_clean":"2023-11-25T04:22:48.047387+0000","last_became_active":"2023-11-25T04:22:47.040084+0000","last_became_peered":"2023-11-25T04:22:47.040084+0000","last_unstale":"2023-11-25T04:22:48.047387+0000","last_undegraded":"2023-11-25T04:22:48.047387+0000","last_fullsized":"2023-11-25T04:22:48.047387+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:59:45.492077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048308+0000","last_change":"2023-11-25T04:22:46.038721+0000","last_active":"2023-11-25T04:22:48.048308+0000","last_peered":"2023-11-25T04:22:48.048308+0000","last_clean":"2023-11-25T04:22:48.048308+0000","last_became_active":"2023-11-25T04:22:46.037698+0000","last_became_peered":"2023-11-25T04:22:46.037698+0000","last_unstale":"2023-11-25T04:22:48.048308+0000","last_undegraded":"2023-11-25T04:22:48.048308+0000","last_fullsized":"2023-11-25T04:22:48.048308+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:25:20.721529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047199+0000","last_change":"2023-11-25T04:22:47.037790+0000","last_active":"2023-11-25T04:22:48.047199+0000","last_peered":"2023-11-25T04:22:48.047199+0000","last_clean":"2023-11-25T04:22:48.047199+0000","last_became_active":"2023-11-25T04:22:47.037149+0000","last_became_peered":"2023-11-25T04:22:47.037149+0000","last_unstale":"2023-11-25T04:22:48.047199+0000","last_undegraded":"2023-11-25T04:22:48.047199+0000","last_fullsized":"2023-11-25T04:22:48.047199+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:31:54.857270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048584+0000","last_change":"2023-11-25T04:22:47.036079+0000","last_active":"2023-11-25T04:22:48.048584+0000","last_peered":"2023-11-25T04:22:48.048584+0000","last_clean":"2023-11-25T04:22:48.048584+0000","last_became_active":"2023-11-25T04:22:47.035625+0000","last_became_peered":"2023-11-25T04:22:47.035625+0000","last_unstale":"2023-11-25T04:22:48.048584+0000","last_undegraded":"2023-11-25T04:22:48.048584+0000","last_fullsized":"2023-11-25T04:22:48.048584+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:21:15.981901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047749+0000","last_change":"2023-11-25T04:22:46.036085+0000","last_active":"2023-11-25T04:22:48.047749+0000","last_peered":"2023-11-25T04:22:48.047749+0000","last_clean":"2023-11-25T04:22:48.047749+0000","last_became_active":"2023-11-25T04:22:46.035495+0000","last_became_peered":"2023-11-25T04:22:46.035495+0000","last_unstale":"2023-11-25T04:22:48.047749+0000","last_undegraded":"2023-11-25T04:22:48.047749+0000","last_fullsized":"2023-11-25T04:22:48.047749+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:14:56.735365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048251+0000","last_change":"2023-11-25T04:22:47.043064+0000","last_active":"2023-11-25T04:22:48.048251+0000","last_peered":"2023-11-25T04:22:48.048251+0000","last_clean":"2023-11-25T04:22:48.048251+0000","last_became_active":"2023-11-25T04:22:47.042760+0000","last_became_peered":"2023-11-25T04:22:47.042760+0000","last_unstale":"2023-11-25T04:22:48.048251+0000","last_undegraded":"2023-11-25T04:22:48.048251+0000","last_fullsized":"2023-11-25T04:22:48.048251+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:37:30.959147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047514+0000","last_change":"2023-11-25T04:22:46.038298+0000","last_active":"2023-11-25T04:22:48.047514+0000","last_peered":"2023-11-25T04:22:48.047514+0000","last_clean":"2023-11-25T04:22:48.047514+0000","last_became_active":"2023-11-25T04:22:46.036629+0000","last_became_peered":"2023-11-25T04:22:46.036629+0000","last_unstale":"2023-11-25T04:22:48.047514+0000","last_undegraded":"2023-11-25T04:22:48.047514+0000","last_fullsized":"2023-11-25T04:22:48.047514+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:45:14.670326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048209+0000","last_change":"2023-11-25T04:22:47.043156+0000","last_active":"2023-11-25T04:22:48.048209+0000","last_peered":"2023-11-25T04:22:48.048209+0000","last_clean":"2023-11-25T04:22:48.048209+0000","last_became_active":"2023-11-25T04:22:47.042693+0000","last_became_peered":"2023-11-25T04:22:47.042693+0000","last_unstale":"2023-11-25T04:22:48.048209+0000","last_undegraded":"2023-11-25T04:22:48.048209+0000","last_fullsized":"2023-11-25T04:22:48.048209+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:38:38.839941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1a","version":"18'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047866+0000","last_change":"2023-11-25T04:22:46.034340+0000","last_active":"2023-11-25T04:22:48.047866+0000","last_peered":"2023-11-25T04:22:48.047866+0000","last_clean":"2023-11-25T04:22:48.047866+0000","last_became_active":"2023-11-25T04:22:46.033944+0000","last_became_peered":"2023-11-25T04:22:46.033944+0000","last_unstale":"2023-11-25T04:22:48.047866+0000","last_undegraded":"2023-11-25T04:22:48.047866+0000","last_fullsized":"2023-11-25T04:22:48.047866+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:59:30.291004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047694+0000","last_change":"2023-11-25T04:22:47.039547+0000","last_active":"2023-11-25T04:22:48.047694+0000","last_peered":"2023-11-25T04:22:48.047694+0000","last_clean":"2023-11-25T04:22:48.047694+0000","last_became_active":"2023-11-25T04:22:47.039218+0000","last_became_peered":"2023-11-25T04:22:47.039218+0000","last_unstale":"2023-11-25T04:22:48.047694+0000","last_undegraded":"2023-11-25T04:22:48.047694+0000","last_fullsized":"2023-11-25T04:22:48.047694+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:44:48.553972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"18'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047860+0000","last_change":"2023-11-25T04:22:46.035882+0000","last_active":"2023-11-25T04:22:48.047860+0000","last_peered":"2023-11-25T04:22:48.047860+0000","last_clean":"2023-11-25T04:22:48.047860+0000","last_became_active":"2023-11-25T04:22:46.035588+0000","last_became_peered":"2023-11-25T04:22:46.035588+0000","last_unstale":"2023-11-25T04:22:48.047860+0000","last_undegraded":"2023-11-25T04:22:48.047860+0000","last_fullsized":"2023-11-25T04:22:48.047860+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:43:10.702849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047831+0000","last_change":"2023-11-25T04:22:47.041267+0000","last_active":"2023-11-25T04:22:48.047831+0000","last_peered":"2023-11-25T04:22:48.047831+0000","last_clean":"2023-11-25T04:22:48.047831+0000","last_became_active":"2023-11-25T04:22:47.040408+0000","last_became_peered":"2023-11-25T04:22:47.040408+0000","last_unstale":"2023-11-25T04:22:48.047831+0000","last_undegraded":"2023-11-25T04:22:48.047831+0000","last_fullsized":"2023-11-25T04:22:48.047831+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:37:30.959147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"18'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048131+0000","last_change":"2023-11-25T04:22:46.040793+0000","last_active":"2023-11-25T04:22:48.048131+0000","last_peered":"2023-11-25T04:22:48.048131+0000","last_clean":"2023-11-25T04:22:48.048131+0000","last_became_active":"2023-11-25T04:22:46.040493+0000","last_became_peered":"2023-11-25T04:22:46.040493+0000","last_unstale":"2023-11-25T04:22:48.048131+0000","last_undegraded":"2023-11-25T04:22:48.048131+0000","last_fullsized":"2023-11-25T04:22:48.048131+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:58:26.155063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048056+0000","last_change":"2023-11-25T04:22:47.040067+0000","last_active":"2023-11-25T04:22:48.048056+0000","last_peered":"2023-11-25T04:22:48.048056+0000","last_clean":"2023-11-25T04:22:48.048056+0000","last_became_active":"2023-11-25T04:22:47.038425+0000","last_became_peered":"2023-11-25T04:22:47.038425+0000","last_unstale":"2023-11-25T04:22:48.048056+0000","last_undegraded":"2023-11-25T04:22:48.048056+0000","last_fullsized":"2023-11-25T04:22:48.048056+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:25:06.132517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"18'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048297+0000","last_change":"2023-11-25T04:22:46.035872+0000","last_active":"2023-11-25T04:22:48.048297+0000","last_peered":"2023-11-25T04:22:48.048297+0000","last_clean":"2023-11-25T04:22:48.048297+0000","last_became_active":"2023-11-25T04:22:46.035575+0000","last_became_peered":"2023-11-25T04:22:46.035575+0000","last_unstale":"2023-11-25T04:22:48.048297+0000","last_undegraded":"2023-11-25T04:22:48.048297+0000","last_fullsized":"2023-11-25T04:22:48.048297+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:58:14.877413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047725+0000","last_change":"2023-11-25T04:22:47.036764+0000","last_active":"2023-11-25T04:22:48.047725+0000","last_peered":"2023-11-25T04:22:48.047725+0000","last_clean":"2023-11-25T04:22:48.047725+0000","last_became_active":"2023-11-25T04:22:47.036406+0000","last_became_peered":"2023-11-25T04:22:47.036406+0000","last_unstale":"2023-11-25T04:22:48.047725+0000","last_undegraded":"2023-11-25T04:22:48.047725+0000","last_fullsized":"2023-11-25T04:22:48.047725+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:09:55.357642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1e","version":"18'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048335+0000","last_change":"2023-11-25T04:22:46.035659+0000","last_active":"2023-11-25T04:22:48.048335+0000","last_peered":"2023-11-25T04:22:48.048335+0000","last_clean":"2023-11-25T04:22:48.048335+0000","last_became_active":"2023-11-25T04:22:46.035113+0000","last_became_peered":"2023-11-25T04:22:46.035113+0000","last_unstale":"2023-11-25T04:22:48.048335+0000","last_undegraded":"2023-11-25T04:22:48.048335+0000","last_fullsized":"2023-11-25T04:22:48.048335+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:26:25.338117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048474+0000","last_change":"2023-11-25T04:22:47.041554+0000","last_active":"2023-11-25T04:22:48.048474+0000","last_peered":"2023-11-25T04:22:48.048474+0000","last_clean":"2023-11-25T04:22:48.048474+0000","last_became_active":"2023-11-25T04:22:47.040314+0000","last_became_peered":"2023-11-25T04:22:47.040314+0000","last_unstale":"2023-11-25T04:22:48.048474+0000","last_undegraded":"2023-11-25T04:22:48.048474+0000","last_fullsized":"2023-11-25T04:22:48.048474+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:24:56.315251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"18'4","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048204+0000","last_change":"2023-11-25T04:22:46.038702+0000","last_active":"2023-11-25T04:22:48.048204+0000","last_peered":"2023-11-25T04:22:48.048204+0000","last_clean":"2023-11-25T04:22:48.048204+0000","last_became_active":"2023-11-25T04:22:46.037567+0000","last_became_peered":"2023-11-25T04:22:46.037567+0000","last_unstale":"2023-11-25T04:22:48.048204+0000","last_undegraded":"2023-11-25T04:22:48.048204+0000","last_fullsized":"2023-11-25T04:22:48.048204+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:58:26.155063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738375,"num_pgs":72,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35508,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336332,"statfs":{"total":96636764160,"available":96600403968,"internally_reserved":0,"allocated":172032,"data_stored":58325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.307}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]}]},{"osd":1,"up_from":8,"seq":34359738375,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31944,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339896,"statfs":{"total":96636764160,"available":96604053504,"internally_reserved":0,"allocated":782336,"data_stored":649871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738374,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31952,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339888,"statfs":{"total":96636764160,"available":96604045312,"internally_reserved":0,"allocated":790528,"data_stored":650487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-25T04:22:55.376 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-25T04:22:55.376 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-11-25T04:22:55.377 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-11-25T04:22:55.377 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-25T04:22:55.744 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:22:55.744 INFO:teuthology.orchestra.run.smithi067.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-11-25T04:22:55.757 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-11-25T04:22:55.757 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-25T04:22:56.052 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:22:56.052 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":19,"fsid":"a677f511-1143-4e76-8da6-5ddfadaff077","created":"2023-11-25T04:22:25.140474+0000","modified":"2023-11-25T04:22:48.032728+0000","last_up_change":"2023-11-25T04:22:33.677691+0000","last_in_change":"2023-11-25T04:22:25.985372+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-25T04:22:35.548359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-25T04:22:40.218527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-25T04:22:44.844906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-25T04:22:45.365536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bc397d35-53ea-430c-a96a-63a4160e64c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":181847435},{"type":"v1","addr":"172.21.15.76:6801","nonce":181847435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":181847435},{"type":"v1","addr":"172.21.15.76:6803","nonce":181847435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":181847435},{"type":"v1","addr":"172.21.15.76:6807","nonce":181847435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":181847435},{"type":"v1","addr":"172.21.15.76:6805","nonce":181847435}]},"public_addr":"172.21.15.76:6801/181847435","cluster_addr":"172.21.15.76:6803/181847435","heartbeat_back_addr":"172.21.15.76:6807/181847435","heartbeat_front_addr":"172.21.15.76:6805/181847435","state":["exists","up"]},{"osd":1,"uuid":"d4f1b88a-bbd4-4e58-9519-569489ced7b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6801","nonce":3130474108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6803","nonce":3130474108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6807","nonce":3130474108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6805","nonce":3130474108}]},"public_addr":"172.21.15.92:6801/3130474108","cluster_addr":"172.21.15.92:6803/3130474108","heartbeat_back_addr":"172.21.15.92:6807/3130474108","heartbeat_front_addr":"172.21.15.92:6805/3130474108","state":["exists","up"]},{"osd":2,"uuid":"55c82801-e090-4c10-91e5-5fa9dbfba36d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6801","nonce":4099978417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6803","nonce":4099978417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6807","nonce":4099978417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6805","nonce":4099978417}]},"public_addr":"172.21.15.195:6801/4099978417","cluster_addr":"172.21.15.195:6803/4099978417","heartbeat_back_addr":"172.21.15.195:6807/4099978417","heartbeat_front_addr":"172.21.15.195:6805/4099978417","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-25T04:22:31.039867+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-25T04:22:31.021863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-25T04:22:31.009821+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-25T04:22:56.065 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-25T04:22:35.548359+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-25T04:22:40.218527+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-25T04:22:44.844906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-25T04:22:45.365536+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-25T04:22:56.065 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-25T04:22:56.359 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2023-11-25T04:22:56.371 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-25T04:22:56.665 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 8 2023-11-25T04:22:56.678 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-25T04:22:56.977 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2023-11-25T04:22:56.990 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-25T04:22:57.287 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2023-11-25T04:22:57.300 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-25T04:22:57.629 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:22:57.629 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-25T04:22:46.384357+0000","modified":"2023-11-25T04:22:48.396173+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4233},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4233":{"gid":4233,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":3,"addr":"172.21.15.67:6803/2443032509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2443032509},{"type":"v1","addr":"172.21.15.67:6803","nonce":2443032509}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-25T04:22:57.629 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 5 2023-11-25T04:22:57.642 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-25T04:22:57.936 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:22:57.936 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":19,"fsid":"a677f511-1143-4e76-8da6-5ddfadaff077","created":"2023-11-25T04:22:25.140474+0000","modified":"2023-11-25T04:22:48.032728+0000","last_up_change":"2023-11-25T04:22:33.677691+0000","last_in_change":"2023-11-25T04:22:25.985372+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-25T04:22:35.548359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-25T04:22:40.218527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-25T04:22:44.844906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-25T04:22:45.365536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bc397d35-53ea-430c-a96a-63a4160e64c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":181847435},{"type":"v1","addr":"172.21.15.76:6801","nonce":181847435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":181847435},{"type":"v1","addr":"172.21.15.76:6803","nonce":181847435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":181847435},{"type":"v1","addr":"172.21.15.76:6807","nonce":181847435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":181847435},{"type":"v1","addr":"172.21.15.76:6805","nonce":181847435}]},"public_addr":"172.21.15.76:6801/181847435","cluster_addr":"172.21.15.76:6803/181847435","heartbeat_back_addr":"172.21.15.76:6807/181847435","heartbeat_front_addr":"172.21.15.76:6805/181847435","state":["exists","up"]},{"osd":1,"uuid":"d4f1b88a-bbd4-4e58-9519-569489ced7b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6801","nonce":3130474108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6803","nonce":3130474108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6807","nonce":3130474108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6805","nonce":3130474108}]},"public_addr":"172.21.15.92:6801/3130474108","cluster_addr":"172.21.15.92:6803/3130474108","heartbeat_back_addr":"172.21.15.92:6807/3130474108","heartbeat_front_addr":"172.21.15.92:6805/3130474108","state":["exists","up"]},{"osd":2,"uuid":"55c82801-e090-4c10-91e5-5fa9dbfba36d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6801","nonce":4099978417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6803","nonce":4099978417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6807","nonce":4099978417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6805","nonce":4099978417}]},"public_addr":"172.21.15.195:6801/4099978417","cluster_addr":"172.21.15.195:6803/4099978417","heartbeat_back_addr":"172.21.15.195:6807/4099978417","heartbeat_front_addr":"172.21.15.195:6805/4099978417","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-25T04:22:31.039867+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-25T04:22:31.021863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-25T04:22:31.009821+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-25T04:22:57.949 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-25T04:22:35.548359+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-25T04:22:40.218527+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-25T04:22:44.844906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-25T04:22:45.365536+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-25T04:22:57.949 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-25T04:22:58.244 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2023-11-25T04:22:58.257 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-25T04:22:58.551 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 8 2023-11-25T04:22:58.564 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-25T04:22:58.860 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2023-11-25T04:22:58.873 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-25T04:22:59.171 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2023-11-25T04:22:59.184 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-25T04:22:59.514 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:22:59.515 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-25T04:22:46.384357+0000","modified":"2023-11-25T04:22:48.396173+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4233},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4233":{"gid":4233,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":3,"addr":"172.21.15.67:6803/2443032509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2443032509},{"type":"v1","addr":"172.21.15.67:6803","nonce":2443032509}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-25T04:22:59.515 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 5 2023-11-25T04:22:59.527 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-25T04:22:59.821 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:22:59.822 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":19,"fsid":"a677f511-1143-4e76-8da6-5ddfadaff077","created":"2023-11-25T04:22:25.140474+0000","modified":"2023-11-25T04:22:48.032728+0000","last_up_change":"2023-11-25T04:22:33.677691+0000","last_in_change":"2023-11-25T04:22:25.985372+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-25T04:22:35.548359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-25T04:22:40.218527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-25T04:22:44.844906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-25T04:22:45.365536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bc397d35-53ea-430c-a96a-63a4160e64c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":181847435},{"type":"v1","addr":"172.21.15.76:6801","nonce":181847435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":181847435},{"type":"v1","addr":"172.21.15.76:6803","nonce":181847435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":181847435},{"type":"v1","addr":"172.21.15.76:6807","nonce":181847435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":181847435},{"type":"v1","addr":"172.21.15.76:6805","nonce":181847435}]},"public_addr":"172.21.15.76:6801/181847435","cluster_addr":"172.21.15.76:6803/181847435","heartbeat_back_addr":"172.21.15.76:6807/181847435","heartbeat_front_addr":"172.21.15.76:6805/181847435","state":["exists","up"]},{"osd":1,"uuid":"d4f1b88a-bbd4-4e58-9519-569489ced7b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6801","nonce":3130474108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6803","nonce":3130474108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6807","nonce":3130474108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6805","nonce":3130474108}]},"public_addr":"172.21.15.92:6801/3130474108","cluster_addr":"172.21.15.92:6803/3130474108","heartbeat_back_addr":"172.21.15.92:6807/3130474108","heartbeat_front_addr":"172.21.15.92:6805/3130474108","state":["exists","up"]},{"osd":2,"uuid":"55c82801-e090-4c10-91e5-5fa9dbfba36d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6801","nonce":4099978417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6803","nonce":4099978417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6807","nonce":4099978417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6805","nonce":4099978417}]},"public_addr":"172.21.15.195:6801/4099978417","cluster_addr":"172.21.15.195:6803/4099978417","heartbeat_back_addr":"172.21.15.195:6807/4099978417","heartbeat_front_addr":"172.21.15.195:6805/4099978417","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-25T04:22:31.039867+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-25T04:22:31.021863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-25T04:22:31.009821+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-25T04:22:59.834 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-25T04:23:00.165 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:23:00.165 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-25T04:22:46.384357+0000","modified":"2023-11-25T04:22:48.396173+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4233},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4233":{"gid":4233,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":3,"addr":"172.21.15.67:6803/2443032509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2443032509},{"type":"v1","addr":"172.21.15.67:6803","nonce":2443032509}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-25T04:23:00.165 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 5 2023-11-25T04:23:00.278 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 5, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-25T04:22:46.384357+0000', 'modified': '2023-11-25T04:22:48.396173+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4233}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4233': {'gid': 4233, 'name': 'a', 'rank': 0, 'incarnation': 3, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.67:6803/2443032509', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6802', 'nonce': 2443032509}, {'type': 'v1', 'addr': '172.21.15.67:6803', 'nonce': 2443032509}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-25T04:23:00.279 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-25T04:23:00.279 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-25T04:23:00.398 INFO:teuthology.orchestra.run.smithi067.stderr:2023-11-25T04:23:00.396+0000 7f2f827fc640 0 client.4498 ms_handle_reset on v2:172.21.15.67:6802/2443032509 2023-11-25T04:23:00.418 INFO:teuthology.orchestra.run.smithi067.stderr:2023-11-25T04:23:00.416+0000 7f2f827fc640 0 client.4617 ms_handle_reset on v2:172.21.15.67:6802/2443032509 2023-11-25T04:23:00.418 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2023-11-25T04:23:00.419 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "a677f511-1143-4e76-8da6-5ddfadaff077", 2023-11-25T04:23:00.419 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 0, 2023-11-25T04:23:00.419 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 4233, 2023-11-25T04:23:00.419 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2023-11-25T04:23:00.419 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2023-11-25T04:23:00.419 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2023-11-25T04:23:00.419 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 13.393895563999999, 2023-11-25T04:23:00.420 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 5, 2023-11-25T04:23:00.420 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 19, 2023-11-25T04:23:00.420 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 19, 2023-11-25T04:23:00.420 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 18.315444105000001 2023-11-25T04:23:00.420 INFO:teuthology.orchestra.run.smithi067.stdout:} 2023-11-25T04:23:00.429 INFO:teuthology.run_tasks:Running task thrashosds... 2023-11-25T04:23:00.439 INFO:tasks.thrashosds:config is {'chance_down': 1.0, 'powercycle': True, 'timeout': 600, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2023-11-25T04:23:00.440 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'main'}, 'ceph': {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(MDS_TRIM\\)', '\\(MDS_SLOW_REQUEST\\)', 'MDS_SLOW_METADATA_IO', 'Behind on trimming', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from'], 'sha1': 'ec928368edcc5fc71884a56fd9fd821b148f4e30'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'ec928368edcc5fc71884a56fd9fd821b148f4e30'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'main', 'sha1': 'ec928368edcc5fc71884a56fd9fd821b148f4e30'}} 2023-11-25T04:23:00.440 INFO:tasks.thrashosds:config is {'chance_down': 1.0, 'powercycle': True, 'timeout': 600, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2023-11-25T04:23:00.440 INFO:tasks.thrashosds:Doing preliminary sync to avoid collateral damage... 2023-11-25T04:23:00.440 DEBUG:teuthology.orchestra.run.smithi067:> sync 2023-11-25T04:23:00.585 DEBUG:teuthology.orchestra.run.smithi076:> sync 2023-11-25T04:23:00.778 DEBUG:teuthology.orchestra.run.smithi092:> sync 2023-11-25T04:23:00.881 DEBUG:teuthology.orchestra.run.smithi195:> sync 2023-11-25T04:23:00.991 DEBUG:tasks.thrashosds:checking console status of smithi067 2023-11-25T04:23:00.991 DEBUG:teuthology.orchestra.console.smithi067:Waiting for login prompt 2023-11-25T04:23:00.991 DEBUG:teuthology.orchestra.console.smithi067:pexpect command: console -M conserver.front.sepia.ceph.com -p 3109 -f smithi067 2023-11-25T04:23:01.076 DEBUG:teuthology.orchestra.console.smithi067:expect before: [Enter `^Ec?' for help] 2023-11-25T04:23:01.076 DEBUG:teuthology.orchestra.console.smithi067:expect after: smithi067 login: 2023-11-25T04:23:01.128 DEBUG:teuthology.orchestra.console.smithi067:console disconnect output: [Enter `^Ec?' for help] smithi067 login: [disconnect] 2023-11-25T04:23:01.230 DEBUG:tasks.thrashosds:checking console status of smithi076 2023-11-25T04:23:01.230 DEBUG:teuthology.orchestra.console.smithi076:Waiting for login prompt 2023-11-25T04:23:01.230 DEBUG:teuthology.orchestra.console.smithi076:pexpect command: console -M conserver.front.sepia.ceph.com -p 3109 -f smithi076 2023-11-25T04:23:01.355 DEBUG:teuthology.orchestra.console.smithi076:expect before: [Enter `^Ec?' for help] 2023-11-25T04:23:01.355 DEBUG:teuthology.orchestra.console.smithi076:expect after: smithi076 login: 2023-11-25T04:23:01.407 DEBUG:teuthology.orchestra.console.smithi076:console disconnect output: [Enter `^Ec?' for help] smithi076 login: [disconnect] 2023-11-25T04:23:01.509 DEBUG:tasks.thrashosds:checking console status of smithi092 2023-11-25T04:23:01.509 DEBUG:teuthology.orchestra.console.smithi092:Waiting for login prompt 2023-11-25T04:23:01.509 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: console -M conserver.front.sepia.ceph.com -p 3109 -f smithi092 2023-11-25T04:23:01.655 DEBUG:teuthology.orchestra.console.smithi092:expect before: [Enter `^Ec?' for help] 2023-11-25T04:23:01.656 DEBUG:teuthology.orchestra.console.smithi092:expect after: smithi092 login: 2023-11-25T04:23:01.710 DEBUG:teuthology.orchestra.console.smithi092:console disconnect output: [Enter `^Ec?' for help] smithi092 login: [disconnect] 2023-11-25T04:23:01.811 DEBUG:tasks.thrashosds:checking console status of smithi195 2023-11-25T04:23:01.811 DEBUG:teuthology.orchestra.console.smithi195:Waiting for login prompt 2023-11-25T04:23:01.811 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: console -M conserver.front.sepia.ceph.com -p 3109 -f smithi195 2023-11-25T04:23:01.890 DEBUG:teuthology.orchestra.console.smithi195:expect before: [Enter `^Ec?' for help] 2023-11-25T04:23:01.890 DEBUG:teuthology.orchestra.console.smithi195:expect after: smithi195 login: 2023-11-25T04:23:01.942 DEBUG:teuthology.orchestra.console.smithi195:console disconnect output: [Enter `^Ec?' for help] smithi195 login: [disconnect] 2023-11-25T04:23:02.044 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2023-11-25T04:23:02.374 INFO:tasks.thrashosds:Beginning thrashosds... 2023-11-25T04:23:02.374 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-25T04:23:02.375 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-25T04:23:02.679 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:23:02.682 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2023-11-25T04:23:02.684 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2023-11-25T04:23:01.496402+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":248,"num_write_kb":3474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":196,"ondisk_log_size":196,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":99404,"kb_used_data":1704,"kb_used_omap":4,"kb_used_meta":97659,"kb_avail":283016116,"statfs":{"total":289910292480,"available":289808502784,"internally_reserved":0,"allocated":1744896,"data_stored":1358683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":100003151},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003034"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048437+0000","last_change":"2023-11-25T04:22:47.039847+0000","last_active":"2023-11-25T04:22:48.048437+0000","last_peered":"2023-11-25T04:22:48.048437+0000","last_clean":"2023-11-25T04:22:48.048437+0000","last_became_active":"2023-11-25T04:22:47.039490+0000","last_became_peered":"2023-11-25T04:22:47.039490+0000","last_unstale":"2023-11-25T04:22:48.048437+0000","last_undegraded":"2023-11-25T04:22:48.048437+0000","last_fullsized":"2023-11-25T04:22:48.048437+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:25:06.132517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048105+0000","last_change":"2023-11-25T04:22:47.040836+0000","last_active":"2023-11-25T04:22:48.048105+0000","last_peered":"2023-11-25T04:22:48.048105+0000","last_clean":"2023-11-25T04:22:48.048105+0000","last_became_active":"2023-11-25T04:22:47.040511+0000","last_became_peered":"2023-11-25T04:22:47.040511+0000","last_unstale":"2023-11-25T04:22:48.048105+0000","last_undegraded":"2023-11-25T04:22:48.048105+0000","last_fullsized":"2023-11-25T04:22:48.048105+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:15:39.511043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048297+0000","last_change":"2023-11-25T04:22:47.043529+0000","last_active":"2023-11-25T04:22:48.048297+0000","last_peered":"2023-11-25T04:22:48.048297+0000","last_clean":"2023-11-25T04:22:48.048297+0000","last_became_active":"2023-11-25T04:22:47.042733+0000","last_became_peered":"2023-11-25T04:22:47.042733+0000","last_unstale":"2023-11-25T04:22:48.048297+0000","last_undegraded":"2023-11-25T04:22:48.048297+0000","last_fullsized":"2023-11-25T04:22:48.048297+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:47:34.764521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047862+0000","last_change":"2023-11-25T04:22:47.040314+0000","last_active":"2023-11-25T04:22:48.047862+0000","last_peered":"2023-11-25T04:22:48.047862+0000","last_clean":"2023-11-25T04:22:48.047862+0000","last_became_active":"2023-11-25T04:22:47.039185+0000","last_became_peered":"2023-11-25T04:22:47.039185+0000","last_unstale":"2023-11-25T04:22:48.047862+0000","last_undegraded":"2023-11-25T04:22:48.047862+0000","last_fullsized":"2023-11-25T04:22:48.047862+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:16:29.085050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047841+0000","last_change":"2023-11-25T04:22:47.039499+0000","last_active":"2023-11-25T04:22:48.047841+0000","last_peered":"2023-11-25T04:22:48.047841+0000","last_clean":"2023-11-25T04:22:48.047841+0000","last_became_active":"2023-11-25T04:22:47.039165+0000","last_became_peered":"2023-11-25T04:22:47.039165+0000","last_unstale":"2023-11-25T04:22:48.047841+0000","last_undegraded":"2023-11-25T04:22:48.047841+0000","last_fullsized":"2023-11-25T04:22:48.047841+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:57:28.855151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047565+0000","last_change":"2023-11-25T04:22:47.039992+0000","last_active":"2023-11-25T04:22:48.047565+0000","last_peered":"2023-11-25T04:22:48.047565+0000","last_clean":"2023-11-25T04:22:48.047565+0000","last_became_active":"2023-11-25T04:22:47.039405+0000","last_became_peered":"2023-11-25T04:22:47.039405+0000","last_unstale":"2023-11-25T04:22:48.047565+0000","last_undegraded":"2023-11-25T04:22:48.047565+0000","last_fullsized":"2023-11-25T04:22:48.047565+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:56:59.689578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047396+0000","last_change":"2023-11-25T04:22:47.037971+0000","last_active":"2023-11-25T04:22:48.047396+0000","last_peered":"2023-11-25T04:22:48.047396+0000","last_clean":"2023-11-25T04:22:48.047396+0000","last_became_active":"2023-11-25T04:22:47.036736+0000","last_became_peered":"2023-11-25T04:22:47.036736+0000","last_unstale":"2023-11-25T04:22:48.047396+0000","last_undegraded":"2023-11-25T04:22:48.047396+0000","last_fullsized":"2023-11-25T04:22:48.047396+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:02:07.088241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047307+0000","last_change":"2023-11-25T04:22:47.041728+0000","last_active":"2023-11-25T04:22:48.047307+0000","last_peered":"2023-11-25T04:22:48.047307+0000","last_clean":"2023-11-25T04:22:48.047307+0000","last_became_active":"2023-11-25T04:22:47.041017+0000","last_became_peered":"2023-11-25T04:22:47.041017+0000","last_unstale":"2023-11-25T04:22:48.047307+0000","last_undegraded":"2023-11-25T04:22:48.047307+0000","last_fullsized":"2023-11-25T04:22:48.047307+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:57:28.855151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048097+0000","last_change":"2023-11-25T04:22:47.038397+0000","last_active":"2023-11-25T04:22:48.048097+0000","last_peered":"2023-11-25T04:22:48.048097+0000","last_clean":"2023-11-25T04:22:48.048097+0000","last_became_active":"2023-11-25T04:22:47.037328+0000","last_became_peered":"2023-11-25T04:22:47.037328+0000","last_unstale":"2023-11-25T04:22:48.048097+0000","last_undegraded":"2023-11-25T04:22:48.048097+0000","last_fullsized":"2023-11-25T04:22:48.048097+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:56:59.689578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048223+0000","last_change":"2023-11-25T04:22:47.040182+0000","last_active":"2023-11-25T04:22:48.048223+0000","last_peered":"2023-11-25T04:22:48.048223+0000","last_clean":"2023-11-25T04:22:48.048223+0000","last_became_active":"2023-11-25T04:22:47.039892+0000","last_became_peered":"2023-11-25T04:22:47.039892+0000","last_unstale":"2023-11-25T04:22:48.048223+0000","last_undegraded":"2023-11-25T04:22:48.048223+0000","last_fullsized":"2023-11-25T04:22:48.048223+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:38:09.176582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048202+0000","last_change":"2023-11-25T04:22:47.039907+0000","last_active":"2023-11-25T04:22:48.048202+0000","last_peered":"2023-11-25T04:22:48.048202+0000","last_clean":"2023-11-25T04:22:48.048202+0000","last_became_active":"2023-11-25T04:22:47.039046+0000","last_became_peered":"2023-11-25T04:22:47.039046+0000","last_unstale":"2023-11-25T04:22:48.048202+0000","last_undegraded":"2023-11-25T04:22:48.048202+0000","last_fullsized":"2023-11-25T04:22:48.048202+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:01:51.418774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047609+0000","last_change":"2023-11-25T04:22:47.037618+0000","last_active":"2023-11-25T04:22:48.047609+0000","last_peered":"2023-11-25T04:22:48.047609+0000","last_clean":"2023-11-25T04:22:48.047609+0000","last_became_active":"2023-11-25T04:22:47.036806+0000","last_became_peered":"2023-11-25T04:22:47.036806+0000","last_unstale":"2023-11-25T04:22:48.047609+0000","last_undegraded":"2023-11-25T04:22:48.047609+0000","last_fullsized":"2023-11-25T04:22:48.047609+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:12:51.990448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047608+0000","last_change":"2023-11-25T04:22:47.040388+0000","last_active":"2023-11-25T04:22:48.047608+0000","last_peered":"2023-11-25T04:22:48.047608+0000","last_clean":"2023-11-25T04:22:48.047608+0000","last_became_active":"2023-11-25T04:22:47.040129+0000","last_became_peered":"2023-11-25T04:22:47.040129+0000","last_unstale":"2023-11-25T04:22:48.047608+0000","last_undegraded":"2023-11-25T04:22:48.047608+0000","last_fullsized":"2023-11-25T04:22:48.047608+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:31:54.857270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047643+0000","last_change":"2023-11-25T04:22:47.038887+0000","last_active":"2023-11-25T04:22:48.047643+0000","last_peered":"2023-11-25T04:22:48.047643+0000","last_clean":"2023-11-25T04:22:48.047643+0000","last_became_active":"2023-11-25T04:22:47.038014+0000","last_became_peered":"2023-11-25T04:22:47.038014+0000","last_unstale":"2023-11-25T04:22:48.047643+0000","last_undegraded":"2023-11-25T04:22:48.047643+0000","last_fullsized":"2023-11-25T04:22:48.047643+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:38:38.839941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047471+0000","last_change":"2023-11-25T04:22:47.035843+0000","last_active":"2023-11-25T04:22:48.047471+0000","last_peered":"2023-11-25T04:22:48.047471+0000","last_clean":"2023-11-25T04:22:48.047471+0000","last_became_active":"2023-11-25T04:22:47.035453+0000","last_became_peered":"2023-11-25T04:22:47.035453+0000","last_unstale":"2023-11-25T04:22:48.047471+0000","last_undegraded":"2023-11-25T04:22:48.047471+0000","last_fullsized":"2023-11-25T04:22:48.047471+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:45:40.065146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047308+0000","last_change":"2023-11-25T04:22:47.041778+0000","last_active":"2023-11-25T04:22:48.047308+0000","last_peered":"2023-11-25T04:22:48.047308+0000","last_clean":"2023-11-25T04:22:48.047308+0000","last_became_active":"2023-11-25T04:22:47.041115+0000","last_became_peered":"2023-11-25T04:22:47.041115+0000","last_unstale":"2023-11-25T04:22:48.047308+0000","last_undegraded":"2023-11-25T04:22:48.047308+0000","last_fullsized":"2023-11-25T04:22:48.047308+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:30:04.568790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048422+0000","last_change":"2023-11-25T04:22:47.039389+0000","last_active":"2023-11-25T04:22:48.048422+0000","last_peered":"2023-11-25T04:22:48.048422+0000","last_clean":"2023-11-25T04:22:48.048422+0000","last_became_active":"2023-11-25T04:22:47.039123+0000","last_became_peered":"2023-11-25T04:22:47.039123+0000","last_unstale":"2023-11-25T04:22:48.048422+0000","last_undegraded":"2023-11-25T04:22:48.048422+0000","last_fullsized":"2023-11-25T04:22:48.048422+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:09:11.570561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048403+0000","last_change":"2023-11-25T04:22:47.036946+0000","last_active":"2023-11-25T04:22:48.048403+0000","last_peered":"2023-11-25T04:22:48.048403+0000","last_clean":"2023-11-25T04:22:48.048403+0000","last_became_active":"2023-11-25T04:22:47.036411+0000","last_became_peered":"2023-11-25T04:22:47.036411+0000","last_unstale":"2023-11-25T04:22:48.048403+0000","last_undegraded":"2023-11-25T04:22:48.048403+0000","last_fullsized":"2023-11-25T04:22:48.048403+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:16:29.085050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047729+0000","last_change":"2023-11-25T04:22:47.038983+0000","last_active":"2023-11-25T04:22:48.047729+0000","last_peered":"2023-11-25T04:22:48.047729+0000","last_clean":"2023-11-25T04:22:48.047729+0000","last_became_active":"2023-11-25T04:22:47.038477+0000","last_became_peered":"2023-11-25T04:22:47.038477+0000","last_unstale":"2023-11-25T04:22:48.047729+0000","last_undegraded":"2023-11-25T04:22:48.047729+0000","last_fullsized":"2023-11-25T04:22:48.047729+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:30:04.568790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047891+0000","last_change":"2023-11-25T04:22:47.038155+0000","last_active":"2023-11-25T04:22:48.047891+0000","last_peered":"2023-11-25T04:22:48.047891+0000","last_clean":"2023-11-25T04:22:48.047891+0000","last_became_active":"2023-11-25T04:22:47.037740+0000","last_became_peered":"2023-11-25T04:22:47.037740+0000","last_unstale":"2023-11-25T04:22:48.047891+0000","last_undegraded":"2023-11-25T04:22:48.047891+0000","last_fullsized":"2023-11-25T04:22:48.047891+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:09:11.570561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048543+0000","last_change":"2023-11-25T04:22:47.040641+0000","last_active":"2023-11-25T04:22:48.048543+0000","last_peered":"2023-11-25T04:22:48.048543+0000","last_clean":"2023-11-25T04:22:48.048543+0000","last_became_active":"2023-11-25T04:22:47.039409+0000","last_became_peered":"2023-11-25T04:22:47.039409+0000","last_unstale":"2023-11-25T04:22:48.048543+0000","last_undegraded":"2023-11-25T04:22:48.048543+0000","last_fullsized":"2023-11-25T04:22:48.048543+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:23:15.914817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047384+0000","last_change":"2023-11-25T04:22:47.043456+0000","last_active":"2023-11-25T04:22:48.047384+0000","last_peered":"2023-11-25T04:22:48.047384+0000","last_clean":"2023-11-25T04:22:48.047384+0000","last_became_active":"2023-11-25T04:22:47.043098+0000","last_became_peered":"2023-11-25T04:22:47.043098+0000","last_unstale":"2023-11-25T04:22:48.047384+0000","last_undegraded":"2023-11-25T04:22:48.047384+0000","last_fullsized":"2023-11-25T04:22:48.047384+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:47:57.307757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047103+0000","last_change":"2023-11-25T04:22:47.035260+0000","last_active":"2023-11-25T04:22:48.047103+0000","last_peered":"2023-11-25T04:22:48.047103+0000","last_clean":"2023-11-25T04:22:48.047103+0000","last_became_active":"2023-11-25T04:22:47.034702+0000","last_became_peered":"2023-11-25T04:22:47.034702+0000","last_unstale":"2023-11-25T04:22:48.047103+0000","last_undegraded":"2023-11-25T04:22:48.047103+0000","last_fullsized":"2023-11-25T04:22:48.047103+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:49:02.500022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047259+0000","last_change":"2023-11-25T04:22:47.036805+0000","last_active":"2023-11-25T04:22:48.047259+0000","last_peered":"2023-11-25T04:22:48.047259+0000","last_clean":"2023-11-25T04:22:48.047259+0000","last_became_active":"2023-11-25T04:22:47.036135+0000","last_became_peered":"2023-11-25T04:22:47.036135+0000","last_unstale":"2023-11-25T04:22:48.047259+0000","last_undegraded":"2023-11-25T04:22:48.047259+0000","last_fullsized":"2023-11-25T04:22:48.047259+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:25:21.730992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048430+0000","last_change":"2023-11-25T04:22:47.036611+0000","last_active":"2023-11-25T04:22:48.048430+0000","last_peered":"2023-11-25T04:22:48.048430+0000","last_clean":"2023-11-25T04:22:48.048430+0000","last_became_active":"2023-11-25T04:22:47.035762+0000","last_became_peered":"2023-11-25T04:22:47.035762+0000","last_unstale":"2023-11-25T04:22:48.048430+0000","last_undegraded":"2023-11-25T04:22:48.048430+0000","last_fullsized":"2023-11-25T04:22:48.048430+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:09:11.570561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048216+0000","last_change":"2023-11-25T04:22:47.042423+0000","last_active":"2023-11-25T04:22:48.048216+0000","last_peered":"2023-11-25T04:22:48.048216+0000","last_clean":"2023-11-25T04:22:48.048216+0000","last_became_active":"2023-11-25T04:22:47.042022+0000","last_became_peered":"2023-11-25T04:22:47.042022+0000","last_unstale":"2023-11-25T04:22:48.048216+0000","last_undegraded":"2023-11-25T04:22:48.048216+0000","last_fullsized":"2023-11-25T04:22:48.048216+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:38:09.176582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048337+0000","last_change":"2023-11-25T04:22:47.044040+0000","last_active":"2023-11-25T04:22:48.048337+0000","last_peered":"2023-11-25T04:22:48.048337+0000","last_clean":"2023-11-25T04:22:48.048337+0000","last_became_active":"2023-11-25T04:22:47.043329+0000","last_became_peered":"2023-11-25T04:22:47.043329+0000","last_unstale":"2023-11-25T04:22:48.048337+0000","last_undegraded":"2023-11-25T04:22:48.048337+0000","last_fullsized":"2023-11-25T04:22:48.048337+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:10:59.482775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047954+0000","last_change":"2023-11-25T04:22:47.036485+0000","last_active":"2023-11-25T04:22:48.047954+0000","last_peered":"2023-11-25T04:22:48.047954+0000","last_clean":"2023-11-25T04:22:48.047954+0000","last_became_active":"2023-11-25T04:22:47.035799+0000","last_became_peered":"2023-11-25T04:22:47.035799+0000","last_unstale":"2023-11-25T04:22:48.047954+0000","last_undegraded":"2023-11-25T04:22:48.047954+0000","last_fullsized":"2023-11-25T04:22:48.047954+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:45:40.065146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047835+0000","last_change":"2023-11-25T04:22:47.037668+0000","last_active":"2023-11-25T04:22:48.047835+0000","last_peered":"2023-11-25T04:22:48.047835+0000","last_clean":"2023-11-25T04:22:48.047835+0000","last_became_active":"2023-11-25T04:22:47.036823+0000","last_became_peered":"2023-11-25T04:22:47.036823+0000","last_unstale":"2023-11-25T04:22:48.047835+0000","last_undegraded":"2023-11-25T04:22:48.047835+0000","last_fullsized":"2023-11-25T04:22:48.047835+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:02:07.088241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047758+0000","last_change":"2023-11-25T04:22:47.038117+0000","last_active":"2023-11-25T04:22:48.047758+0000","last_peered":"2023-11-25T04:22:48.047758+0000","last_clean":"2023-11-25T04:22:48.047758+0000","last_became_active":"2023-11-25T04:22:47.037751+0000","last_became_peered":"2023-11-25T04:22:47.037751+0000","last_unstale":"2023-11-25T04:22:48.047758+0000","last_undegraded":"2023-11-25T04:22:48.047758+0000","last_fullsized":"2023-11-25T04:22:48.047758+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:30:04.568790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047184+0000","last_change":"2023-11-25T04:22:47.037786+0000","last_active":"2023-11-25T04:22:48.047184+0000","last_peered":"2023-11-25T04:22:48.047184+0000","last_clean":"2023-11-25T04:22:48.047184+0000","last_became_active":"2023-11-25T04:22:47.037206+0000","last_became_peered":"2023-11-25T04:22:47.037206+0000","last_unstale":"2023-11-25T04:22:48.047184+0000","last_undegraded":"2023-11-25T04:22:48.047184+0000","last_fullsized":"2023-11-25T04:22:48.047184+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:11:40.532140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047505+0000","last_change":"2023-11-25T04:22:47.041325+0000","last_active":"2023-11-25T04:22:48.047505+0000","last_peered":"2023-11-25T04:22:48.047505+0000","last_clean":"2023-11-25T04:22:48.047505+0000","last_became_active":"2023-11-25T04:22:47.040086+0000","last_became_peered":"2023-11-25T04:22:47.040086+0000","last_unstale":"2023-11-25T04:22:48.047505+0000","last_undegraded":"2023-11-25T04:22:48.047505+0000","last_fullsized":"2023-11-25T04:22:48.047505+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:15:39.511043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047822+0000","last_change":"2023-11-25T04:22:47.036689+0000","last_active":"2023-11-25T04:22:48.047822+0000","last_peered":"2023-11-25T04:22:48.047822+0000","last_clean":"2023-11-25T04:22:48.047822+0000","last_became_active":"2023-11-25T04:22:47.036107+0000","last_became_peered":"2023-11-25T04:22:47.036107+0000","last_unstale":"2023-11-25T04:22:48.047822+0000","last_undegraded":"2023-11-25T04:22:48.047822+0000","last_fullsized":"2023-11-25T04:22:48.047822+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:02:07.088241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"18'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048313+0000","last_change":"2023-11-25T04:22:46.040844+0000","last_active":"2023-11-25T04:22:48.048313+0000","last_peered":"2023-11-25T04:22:48.048313+0000","last_clean":"2023-11-25T04:22:48.048313+0000","last_became_active":"2023-11-25T04:22:46.040395+0000","last_became_peered":"2023-11-25T04:22:46.040395+0000","last_unstale":"2023-11-25T04:22:48.048313+0000","last_undegraded":"2023-11-25T04:22:48.048313+0000","last_fullsized":"2023-11-25T04:22:48.048313+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:11:39.522677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048200+0000","last_change":"2023-11-25T04:22:47.037888+0000","last_active":"2023-11-25T04:22:48.048200+0000","last_peered":"2023-11-25T04:22:48.048200+0000","last_clean":"2023-11-25T04:22:48.048200+0000","last_became_active":"2023-11-25T04:22:47.037158+0000","last_became_peered":"2023-11-25T04:22:47.037158+0000","last_unstale":"2023-11-25T04:22:48.048200+0000","last_undegraded":"2023-11-25T04:22:48.048200+0000","last_fullsized":"2023-11-25T04:22:48.048200+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:56:59.689578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047391+0000","last_change":"2023-11-25T04:22:46.036054+0000","last_active":"2023-11-25T04:22:48.047391+0000","last_peered":"2023-11-25T04:22:48.047391+0000","last_clean":"2023-11-25T04:22:48.047391+0000","last_became_active":"2023-11-25T04:22:46.035620+0000","last_became_peered":"2023-11-25T04:22:46.035620+0000","last_unstale":"2023-11-25T04:22:48.047391+0000","last_undegraded":"2023-11-25T04:22:48.047391+0000","last_fullsized":"2023-11-25T04:22:48.047391+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:19:48.371980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048391+0000","last_change":"2023-11-25T04:22:47.045025+0000","last_active":"2023-11-25T04:22:48.048391+0000","last_peered":"2023-11-25T04:22:48.048391+0000","last_clean":"2023-11-25T04:22:48.048391+0000","last_became_active":"2023-11-25T04:22:47.044697+0000","last_became_peered":"2023-11-25T04:22:47.044697+0000","last_unstale":"2023-11-25T04:22:48.048391+0000","last_undegraded":"2023-11-25T04:22:48.048391+0000","last_fullsized":"2023-11-25T04:22:48.048391+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:16:29.085050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047517+0000","last_change":"2023-11-25T04:22:46.037169+0000","last_active":"2023-11-25T04:22:48.047517+0000","last_peered":"2023-11-25T04:22:48.047517+0000","last_clean":"2023-11-25T04:22:48.047517+0000","last_became_active":"2023-11-25T04:22:46.036948+0000","last_became_peered":"2023-11-25T04:22:46.036948+0000","last_unstale":"2023-11-25T04:22:48.047517+0000","last_undegraded":"2023-11-25T04:22:48.047517+0000","last_fullsized":"2023-11-25T04:22:48.047517+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:45:14.670326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047715+0000","last_change":"2023-11-25T04:22:47.044860+0000","last_active":"2023-11-25T04:22:48.047715+0000","last_peered":"2023-11-25T04:22:48.047715+0000","last_clean":"2023-11-25T04:22:48.047715+0000","last_became_active":"2023-11-25T04:22:47.044551+0000","last_became_peered":"2023-11-25T04:22:47.044551+0000","last_unstale":"2023-11-25T04:22:48.047715+0000","last_undegraded":"2023-11-25T04:22:48.047715+0000","last_fullsized":"2023-11-25T04:22:48.047715+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:09:55.357642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.10","version":"18'8","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047593+0000","last_change":"2023-11-25T04:22:46.040356+0000","last_active":"2023-11-25T04:22:48.047593+0000","last_peered":"2023-11-25T04:22:48.047593+0000","last_clean":"2023-11-25T04:22:48.047593+0000","last_became_active":"2023-11-25T04:22:46.039802+0000","last_became_peered":"2023-11-25T04:22:46.039802+0000","last_unstale":"2023-11-25T04:22:48.047593+0000","last_undegraded":"2023-11-25T04:22:48.047593+0000","last_fullsized":"2023-11-25T04:22:48.047593+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:09:54.348179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"18'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048372+0000","last_change":"2023-11-25T04:22:46.040082+0000","last_active":"2023-11-25T04:22:48.048372+0000","last_peered":"2023-11-25T04:22:48.048372+0000","last_clean":"2023-11-25T04:22:48.048372+0000","last_became_active":"2023-11-25T04:22:46.039763+0000","last_became_peered":"2023-11-25T04:22:46.039763+0000","last_unstale":"2023-11-25T04:22:48.048372+0000","last_undegraded":"2023-11-25T04:22:48.048372+0000","last_fullsized":"2023-11-25T04:22:48.048372+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:10:20.178136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047265+0000","last_change":"2023-11-25T04:22:47.043644+0000","last_active":"2023-11-25T04:22:48.047265+0000","last_peered":"2023-11-25T04:22:48.047265+0000","last_clean":"2023-11-25T04:22:48.047265+0000","last_became_active":"2023-11-25T04:22:47.043221+0000","last_became_peered":"2023-11-25T04:22:47.043221+0000","last_unstale":"2023-11-25T04:22:48.047265+0000","last_undegraded":"2023-11-25T04:22:48.047265+0000","last_fullsized":"2023-11-25T04:22:48.047265+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:41:16.672268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"18'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048462+0000","last_change":"2023-11-25T04:22:46.035806+0000","last_active":"2023-11-25T04:22:48.048462+0000","last_peered":"2023-11-25T04:22:48.048462+0000","last_clean":"2023-11-25T04:22:48.048462+0000","last_became_active":"2023-11-25T04:22:46.035418+0000","last_became_peered":"2023-11-25T04:22:46.035418+0000","last_unstale":"2023-11-25T04:22:48.048462+0000","last_undegraded":"2023-11-25T04:22:48.048462+0000","last_fullsized":"2023-11-25T04:22:48.048462+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:31:53.068767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047508+0000","last_change":"2023-11-25T04:22:47.039685+0000","last_active":"2023-11-25T04:22:48.047508+0000","last_peered":"2023-11-25T04:22:48.047508+0000","last_clean":"2023-11-25T04:22:48.047508+0000","last_became_active":"2023-11-25T04:22:47.039296+0000","last_became_peered":"2023-11-25T04:22:47.039296+0000","last_unstale":"2023-11-25T04:22:48.047508+0000","last_undegraded":"2023-11-25T04:22:48.047508+0000","last_fullsized":"2023-11-25T04:22:48.047508+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:11:40.532140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"18'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048658+0000","last_change":"2023-11-25T04:22:46.039953+0000","last_active":"2023-11-25T04:22:48.048658+0000","last_peered":"2023-11-25T04:22:48.048658+0000","last_clean":"2023-11-25T04:22:48.048658+0000","last_became_active":"2023-11-25T04:22:46.039660+0000","last_became_peered":"2023-11-25T04:22:46.039660+0000","last_unstale":"2023-11-25T04:22:48.048658+0000","last_undegraded":"2023-11-25T04:22:48.048658+0000","last_fullsized":"2023-11-25T04:22:48.048658+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:59:44.482613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047887+0000","last_change":"2023-11-25T04:22:47.037912+0000","last_active":"2023-11-25T04:22:48.047887+0000","last_peered":"2023-11-25T04:22:48.047887+0000","last_clean":"2023-11-25T04:22:48.047887+0000","last_became_active":"2023-11-25T04:22:47.036552+0000","last_became_peered":"2023-11-25T04:22:47.036552+0000","last_unstale":"2023-11-25T04:22:48.047887+0000","last_undegraded":"2023-11-25T04:22:48.047887+0000","last_fullsized":"2023-11-25T04:22:48.047887+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:38:09.176582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048295+0000","last_change":"2023-11-25T04:22:46.039176+0000","last_active":"2023-11-25T04:22:48.048295+0000","last_peered":"2023-11-25T04:22:48.048295+0000","last_clean":"2023-11-25T04:22:48.048295+0000","last_became_active":"2023-11-25T04:22:46.038842+0000","last_became_peered":"2023-11-25T04:22:46.038842+0000","last_unstale":"2023-11-25T04:22:48.048295+0000","last_undegraded":"2023-11-25T04:22:48.048295+0000","last_fullsized":"2023-11-25T04:22:48.048295+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:47:56.298293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047733+0000","last_change":"2023-11-25T04:22:47.042155+0000","last_active":"2023-11-25T04:22:48.047733+0000","last_peered":"2023-11-25T04:22:48.047733+0000","last_clean":"2023-11-25T04:22:48.047733+0000","last_became_active":"2023-11-25T04:22:47.041433+0000","last_became_peered":"2023-11-25T04:22:47.041433+0000","last_unstale":"2023-11-25T04:22:48.047733+0000","last_undegraded":"2023-11-25T04:22:48.047733+0000","last_fullsized":"2023-11-25T04:22:48.047733+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:57:28.855151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"18'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048201+0000","last_change":"2023-11-25T04:22:46.037827+0000","last_active":"2023-11-25T04:22:48.048201+0000","last_peered":"2023-11-25T04:22:48.048201+0000","last_clean":"2023-11-25T04:22:48.048201+0000","last_became_active":"2023-11-25T04:22:46.036515+0000","last_became_peered":"2023-11-25T04:22:46.036515+0000","last_unstale":"2023-11-25T04:22:48.048201+0000","last_undegraded":"2023-11-25T04:22:48.048201+0000","last_fullsized":"2023-11-25T04:22:48.048201+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:10:20.178136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048154+0000","last_change":"2023-11-25T04:22:47.038856+0000","last_active":"2023-11-25T04:22:48.048154+0000","last_peered":"2023-11-25T04:22:48.048154+0000","last_clean":"2023-11-25T04:22:48.048154+0000","last_became_active":"2023-11-25T04:22:47.038190+0000","last_became_peered":"2023-11-25T04:22:47.038190+0000","last_unstale":"2023-11-25T04:22:48.048154+0000","last_undegraded":"2023-11-25T04:22:48.048154+0000","last_fullsized":"2023-11-25T04:22:48.048154+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:29:37.268624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047789+0000","last_change":"2023-11-25T04:22:46.038473+0000","last_active":"2023-11-25T04:22:48.047789+0000","last_peered":"2023-11-25T04:22:48.047789+0000","last_clean":"2023-11-25T04:22:48.047789+0000","last_became_active":"2023-11-25T04:22:46.038159+0000","last_became_peered":"2023-11-25T04:22:46.038159+0000","last_unstale":"2023-11-25T04:22:48.047789+0000","last_undegraded":"2023-11-25T04:22:48.047789+0000","last_fullsized":"2023-11-25T04:22:48.047789+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:31:53.847807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048568+0000","last_change":"2023-11-25T04:22:47.036222+0000","last_active":"2023-11-25T04:22:48.048568+0000","last_peered":"2023-11-25T04:22:48.048568+0000","last_clean":"2023-11-25T04:22:48.048568+0000","last_became_active":"2023-11-25T04:22:47.035871+0000","last_became_peered":"2023-11-25T04:22:47.035871+0000","last_unstale":"2023-11-25T04:22:48.048568+0000","last_undegraded":"2023-11-25T04:22:48.048568+0000","last_fullsized":"2023-11-25T04:22:48.048568+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:13:35.199538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047681+0000","last_change":"2023-11-25T04:22:46.035907+0000","last_active":"2023-11-25T04:22:48.047681+0000","last_peered":"2023-11-25T04:22:48.047681+0000","last_clean":"2023-11-25T04:22:48.047681+0000","last_became_active":"2023-11-25T04:22:46.035612+0000","last_became_peered":"2023-11-25T04:22:46.035612+0000","last_unstale":"2023-11-25T04:22:48.047681+0000","last_undegraded":"2023-11-25T04:22:48.047681+0000","last_fullsized":"2023-11-25T04:22:48.047681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:00:35.249875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048441+0000","last_change":"2023-11-25T04:22:47.038725+0000","last_active":"2023-11-25T04:22:48.048441+0000","last_peered":"2023-11-25T04:22:48.048441+0000","last_clean":"2023-11-25T04:22:48.048441+0000","last_became_active":"2023-11-25T04:22:47.038124+0000","last_became_peered":"2023-11-25T04:22:47.038124+0000","last_unstale":"2023-11-25T04:22:48.048441+0000","last_undegraded":"2023-11-25T04:22:48.048441+0000","last_fullsized":"2023-11-25T04:22:48.048441+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:00:26.187878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047437+0000","last_change":"2023-11-25T04:22:46.034509+0000","last_active":"2023-11-25T04:22:48.047437+0000","last_peered":"2023-11-25T04:22:48.047437+0000","last_clean":"2023-11-25T04:22:48.047437+0000","last_became_active":"2023-11-25T04:22:46.034158+0000","last_became_peered":"2023-11-25T04:22:46.034158+0000","last_unstale":"2023-11-25T04:22:48.047437+0000","last_undegraded":"2023-11-25T04:22:48.047437+0000","last_fullsized":"2023-11-25T04:22:48.047437+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:26:25.338117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048005+0000","last_change":"2023-11-25T04:22:47.038800+0000","last_active":"2023-11-25T04:22:48.048005+0000","last_peered":"2023-11-25T04:22:48.048005+0000","last_clean":"2023-11-25T04:22:48.048005+0000","last_became_active":"2023-11-25T04:22:47.038081+0000","last_became_peered":"2023-11-25T04:22:47.038081+0000","last_unstale":"2023-11-25T04:22:48.048005+0000","last_undegraded":"2023-11-25T04:22:48.048005+0000","last_fullsized":"2023-11-25T04:22:48.048005+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:45:40.065146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"12'157","reported_seq":214,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-25T04:22:47.033945+0000","last_change":"2023-11-25T04:22:36.931828+0000","last_active":"2023-11-25T04:22:47.033945+0000","last_peered":"2023-11-25T04:22:47.033945+0000","last_clean":"2023-11-25T04:22:47.033945+0000","last_became_active":"2023-11-25T04:22:36.931582+0000","last_became_peered":"2023-11-25T04:22:36.931582+0000","last_unstale":"2023-11-25T04:22:47.033945+0000","last_undegraded":"2023-11-25T04:22:47.033945+0000","last_fullsized":"2023-11-25T04:22:47.033945+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:35.919850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:35.919850+0000","last_clean_scrub_stamp":"2023-11-25T04:22:35.919850+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:59:59.198309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-25T04:22:47.031925+0000","last_change":"2023-11-25T04:22:41.999590+0000","last_active":"2023-11-25T04:22:47.031925+0000","last_peered":"2023-11-25T04:22:47.031925+0000","last_clean":"2023-11-25T04:22:47.031925+0000","last_became_active":"2023-11-25T04:22:41.999314+0000","last_became_peered":"2023-11-25T04:22:41.999314+0000","last_unstale":"2023-11-25T04:22:47.031925+0000","last_undegraded":"2023-11-25T04:22:47.031925+0000","last_fullsized":"2023-11-25T04:22:47.031925+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:20:12.785425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047840+0000","last_change":"2023-11-25T04:22:46.037044+0000","last_active":"2023-11-25T04:22:48.047840+0000","last_peered":"2023-11-25T04:22:48.047840+0000","last_clean":"2023-11-25T04:22:48.047840+0000","last_became_active":"2023-11-25T04:22:46.035916+0000","last_became_peered":"2023-11-25T04:22:46.035916+0000","last_unstale":"2023-11-25T04:22:48.047840+0000","last_undegraded":"2023-11-25T04:22:48.047840+0000","last_fullsized":"2023-11-25T04:22:48.047840+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:38:20.501102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048611+0000","last_change":"2023-11-25T04:22:47.036222+0000","last_active":"2023-11-25T04:22:48.048611+0000","last_peered":"2023-11-25T04:22:48.048611+0000","last_clean":"2023-11-25T04:22:48.048611+0000","last_became_active":"2023-11-25T04:22:47.035484+0000","last_became_peered":"2023-11-25T04:22:47.035484+0000","last_unstale":"2023-11-25T04:22:48.048611+0000","last_undegraded":"2023-11-25T04:22:48.048611+0000","last_fullsized":"2023-11-25T04:22:48.048611+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:18:56.845916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-25T04:22:47.032801+0000","last_change":"2023-11-25T04:22:42.000257+0000","last_active":"2023-11-25T04:22:47.032801+0000","last_peered":"2023-11-25T04:22:47.032801+0000","last_clean":"2023-11-25T04:22:47.032801+0000","last_became_active":"2023-11-25T04:22:41.999969+0000","last_became_peered":"2023-11-25T04:22:41.999969+0000","last_unstale":"2023-11-25T04:22:47.032801+0000","last_undegraded":"2023-11-25T04:22:47.032801+0000","last_fullsized":"2023-11-25T04:22:47.032801+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:21:34.424091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047780+0000","last_change":"2023-11-25T04:22:46.036121+0000","last_active":"2023-11-25T04:22:48.047780+0000","last_peered":"2023-11-25T04:22:48.047780+0000","last_clean":"2023-11-25T04:22:48.047780+0000","last_became_active":"2023-11-25T04:22:46.035654+0000","last_became_peered":"2023-11-25T04:22:46.035654+0000","last_unstale":"2023-11-25T04:22:48.047780+0000","last_undegraded":"2023-11-25T04:22:48.047780+0000","last_fullsized":"2023-11-25T04:22:48.047780+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:27:31.901618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048273+0000","last_change":"2023-11-25T04:22:47.039583+0000","last_active":"2023-11-25T04:22:48.048273+0000","last_peered":"2023-11-25T04:22:48.048273+0000","last_clean":"2023-11-25T04:22:48.048273+0000","last_became_active":"2023-11-25T04:22:47.039197+0000","last_became_peered":"2023-11-25T04:22:47.039197+0000","last_unstale":"2023-11-25T04:22:48.048273+0000","last_undegraded":"2023-11-25T04:22:48.048273+0000","last_fullsized":"2023-11-25T04:22:48.048273+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:37:30.959147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-25T04:22:47.029239+0000","last_change":"2023-11-25T04:22:42.000382+0000","last_active":"2023-11-25T04:22:47.029239+0000","last_peered":"2023-11-25T04:22:47.029239+0000","last_clean":"2023-11-25T04:22:47.029239+0000","last_became_active":"2023-11-25T04:22:42.000041+0000","last_became_peered":"2023-11-25T04:22:42.000041+0000","last_unstale":"2023-11-25T04:22:47.029239+0000","last_undegraded":"2023-11-25T04:22:47.029239+0000","last_fullsized":"2023-11-25T04:22:47.029239+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:08:19.655864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047444+0000","last_change":"2023-11-25T04:22:46.037431+0000","last_active":"2023-11-25T04:22:48.047444+0000","last_peered":"2023-11-25T04:22:48.047444+0000","last_clean":"2023-11-25T04:22:48.047444+0000","last_became_active":"2023-11-25T04:22:46.037141+0000","last_became_peered":"2023-11-25T04:22:46.037141+0000","last_unstale":"2023-11-25T04:22:48.047444+0000","last_undegraded":"2023-11-25T04:22:48.047444+0000","last_fullsized":"2023-11-25T04:22:48.047444+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:40:42.500663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048182+0000","last_change":"2023-11-25T04:22:47.038454+0000","last_active":"2023-11-25T04:22:48.048182+0000","last_peered":"2023-11-25T04:22:48.048182+0000","last_clean":"2023-11-25T04:22:48.048182+0000","last_became_active":"2023-11-25T04:22:47.038046+0000","last_became_peered":"2023-11-25T04:22:47.038046+0000","last_unstale":"2023-11-25T04:22:48.048182+0000","last_undegraded":"2023-11-25T04:22:48.048182+0000","last_fullsized":"2023-11-25T04:22:48.048182+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:38:38.839941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-25T04:22:47.031534+0000","last_change":"2023-11-25T04:22:41.999424+0000","last_active":"2023-11-25T04:22:47.031534+0000","last_peered":"2023-11-25T04:22:47.031534+0000","last_clean":"2023-11-25T04:22:47.031534+0000","last_became_active":"2023-11-25T04:22:41.999179+0000","last_became_peered":"2023-11-25T04:22:41.999179+0000","last_unstale":"2023-11-25T04:22:47.031534+0000","last_undegraded":"2023-11-25T04:22:47.031534+0000","last_fullsized":"2023-11-25T04:22:47.031534+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:12:04.291696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048523+0000","last_change":"2023-11-25T04:22:46.038758+0000","last_active":"2023-11-25T04:22:48.048523+0000","last_peered":"2023-11-25T04:22:48.048523+0000","last_clean":"2023-11-25T04:22:48.048523+0000","last_became_active":"2023-11-25T04:22:46.038470+0000","last_became_peered":"2023-11-25T04:22:46.038470+0000","last_unstale":"2023-11-25T04:22:48.048523+0000","last_undegraded":"2023-11-25T04:22:48.048523+0000","last_fullsized":"2023-11-25T04:22:48.048523+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:49:01.490558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048075+0000","last_change":"2023-11-25T04:22:47.037355+0000","last_active":"2023-11-25T04:22:48.048075+0000","last_peered":"2023-11-25T04:22:48.048075+0000","last_clean":"2023-11-25T04:22:48.048075+0000","last_became_active":"2023-11-25T04:22:47.036924+0000","last_became_peered":"2023-11-25T04:22:47.036924+0000","last_unstale":"2023-11-25T04:22:48.048075+0000","last_undegraded":"2023-11-25T04:22:48.048075+0000","last_fullsized":"2023-11-25T04:22:48.048075+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:15:39.511043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048677+0000","last_change":"2023-11-25T04:22:46.039025+0000","last_active":"2023-11-25T04:22:48.048677+0000","last_peered":"2023-11-25T04:22:48.048677+0000","last_clean":"2023-11-25T04:22:48.048677+0000","last_became_active":"2023-11-25T04:22:46.038762+0000","last_became_peered":"2023-11-25T04:22:46.038762+0000","last_unstale":"2023-11-25T04:22:48.048677+0000","last_undegraded":"2023-11-25T04:22:48.048677+0000","last_fullsized":"2023-11-25T04:22:48.048677+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:44:47.544509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-25T04:22:47.029103+0000","last_change":"2023-11-25T04:22:42.000204+0000","last_active":"2023-11-25T04:22:47.029103+0000","last_peered":"2023-11-25T04:22:47.029103+0000","last_clean":"2023-11-25T04:22:47.029103+0000","last_became_active":"2023-11-25T04:22:41.999915+0000","last_became_peered":"2023-11-25T04:22:41.999915+0000","last_unstale":"2023-11-25T04:22:47.029103+0000","last_undegraded":"2023-11-25T04:22:47.029103+0000","last_fullsized":"2023-11-25T04:22:47.029103+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:07:32.457905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047409+0000","last_change":"2023-11-25T04:22:47.038119+0000","last_active":"2023-11-25T04:22:48.047409+0000","last_peered":"2023-11-25T04:22:48.047409+0000","last_clean":"2023-11-25T04:22:48.047409+0000","last_became_active":"2023-11-25T04:22:47.037732+0000","last_became_peered":"2023-11-25T04:22:47.037732+0000","last_unstale":"2023-11-25T04:22:48.047409+0000","last_undegraded":"2023-11-25T04:22:48.047409+0000","last_fullsized":"2023-11-25T04:22:48.047409+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:49:02.500022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048004+0000","last_change":"2023-11-25T04:22:46.035885+0000","last_active":"2023-11-25T04:22:48.048004+0000","last_peered":"2023-11-25T04:22:48.048004+0000","last_clean":"2023-11-25T04:22:48.048004+0000","last_became_active":"2023-11-25T04:22:46.035405+0000","last_became_peered":"2023-11-25T04:22:46.035405+0000","last_unstale":"2023-11-25T04:22:48.048004+0000","last_undegraded":"2023-11-25T04:22:48.048004+0000","last_fullsized":"2023-11-25T04:22:48.048004+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:53:47.398746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-25T04:22:47.032524+0000","last_change":"2023-11-25T04:22:41.999067+0000","last_active":"2023-11-25T04:22:47.032524+0000","last_peered":"2023-11-25T04:22:47.032524+0000","last_clean":"2023-11-25T04:22:47.032524+0000","last_became_active":"2023-11-25T04:22:41.998798+0000","last_became_peered":"2023-11-25T04:22:41.998798+0000","last_unstale":"2023-11-25T04:22:47.032524+0000","last_undegraded":"2023-11-25T04:22:47.032524+0000","last_fullsized":"2023-11-25T04:22:47.032524+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:00:04.266051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047617+0000","last_change":"2023-11-25T04:22:47.043464+0000","last_active":"2023-11-25T04:22:48.047617+0000","last_peered":"2023-11-25T04:22:48.047617+0000","last_clean":"2023-11-25T04:22:48.047617+0000","last_became_active":"2023-11-25T04:22:47.043050+0000","last_became_peered":"2023-11-25T04:22:47.043050+0000","last_unstale":"2023-11-25T04:22:48.047617+0000","last_undegraded":"2023-11-25T04:22:48.047617+0000","last_fullsized":"2023-11-25T04:22:48.047617+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:59:45.492077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"18'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048417+0000","last_change":"2023-11-25T04:22:46.034606+0000","last_active":"2023-11-25T04:22:48.048417+0000","last_peered":"2023-11-25T04:22:48.048417+0000","last_clean":"2023-11-25T04:22:48.048417+0000","last_became_active":"2023-11-25T04:22:46.034229+0000","last_became_peered":"2023-11-25T04:22:46.034229+0000","last_unstale":"2023-11-25T04:22:48.048417+0000","last_undegraded":"2023-11-25T04:22:48.048417+0000","last_fullsized":"2023-11-25T04:22:48.048417+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:21:42.146264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-25T04:22:47.031216+0000","last_change":"2023-11-25T04:22:42.000258+0000","last_active":"2023-11-25T04:22:47.031216+0000","last_peered":"2023-11-25T04:22:47.031216+0000","last_clean":"2023-11-25T04:22:47.031216+0000","last_became_active":"2023-11-25T04:22:41.999992+0000","last_became_peered":"2023-11-25T04:22:41.999992+0000","last_unstale":"2023-11-25T04:22:47.031216+0000","last_undegraded":"2023-11-25T04:22:47.031216+0000","last_fullsized":"2023-11-25T04:22:47.031216+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:31:13.715992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047510+0000","last_change":"2023-11-25T04:22:47.042860+0000","last_active":"2023-11-25T04:22:48.047510+0000","last_peered":"2023-11-25T04:22:48.047510+0000","last_clean":"2023-11-25T04:22:48.047510+0000","last_became_active":"2023-11-25T04:22:47.042472+0000","last_became_peered":"2023-11-25T04:22:47.042472+0000","last_unstale":"2023-11-25T04:22:48.047510+0000","last_undegraded":"2023-11-25T04:22:48.047510+0000","last_fullsized":"2023-11-25T04:22:48.047510+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:44:48.553972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.894735+0000","last_change":"2023-11-25T04:22:46.038417+0000","last_active":"2023-11-25T04:22:48.894735+0000","last_peered":"2023-11-25T04:22:48.894735+0000","last_clean":"2023-11-25T04:22:48.894735+0000","last_became_active":"2023-11-25T04:22:46.037513+0000","last_became_peered":"2023-11-25T04:22:46.037513+0000","last_unstale":"2023-11-25T04:22:48.894735+0000","last_undegraded":"2023-11-25T04:22:48.894735+0000","last_fullsized":"2023-11-25T04:22:48.894735+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:58:14.877413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-25T04:22:47.030950+0000","last_change":"2023-11-25T04:22:42.000217+0000","last_active":"2023-11-25T04:22:47.030950+0000","last_peered":"2023-11-25T04:22:47.030950+0000","last_clean":"2023-11-25T04:22:47.030950+0000","last_became_active":"2023-11-25T04:22:41.999987+0000","last_became_peered":"2023-11-25T04:22:41.999987+0000","last_unstale":"2023-11-25T04:22:47.030950+0000","last_undegraded":"2023-11-25T04:22:47.030950+0000","last_fullsized":"2023-11-25T04:22:47.030950+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:11:51.100459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047385+0000","last_change":"2023-11-25T04:22:47.036099+0000","last_active":"2023-11-25T04:22:48.047385+0000","last_peered":"2023-11-25T04:22:48.047385+0000","last_clean":"2023-11-25T04:22:48.047385+0000","last_became_active":"2023-11-25T04:22:47.035581+0000","last_became_peered":"2023-11-25T04:22:47.035581+0000","last_unstale":"2023-11-25T04:22:48.047385+0000","last_undegraded":"2023-11-25T04:22:48.047385+0000","last_fullsized":"2023-11-25T04:22:48.047385+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:21:43.155728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048171+0000","last_change":"2023-11-25T04:22:47.040343+0000","last_active":"2023-11-25T04:22:48.048171+0000","last_peered":"2023-11-25T04:22:48.048171+0000","last_clean":"2023-11-25T04:22:48.048171+0000","last_became_active":"2023-11-25T04:22:47.039151+0000","last_became_peered":"2023-11-25T04:22:47.039151+0000","last_unstale":"2023-11-25T04:22:48.048171+0000","last_undegraded":"2023-11-25T04:22:48.048171+0000","last_fullsized":"2023-11-25T04:22:48.048171+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:21:15.981901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"18'6","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048260+0000","last_change":"2023-11-25T04:22:46.040005+0000","last_active":"2023-11-25T04:22:48.048260+0000","last_peered":"2023-11-25T04:22:48.048260+0000","last_clean":"2023-11-25T04:22:48.048260+0000","last_became_active":"2023-11-25T04:22:46.039655+0000","last_became_peered":"2023-11-25T04:22:46.039655+0000","last_unstale":"2023-11-25T04:22:48.048260+0000","last_undegraded":"2023-11-25T04:22:48.048260+0000","last_fullsized":"2023-11-25T04:22:48.048260+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:41:15.662804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047237+0000","last_change":"2023-11-25T04:22:47.037181+0000","last_active":"2023-11-25T04:22:48.047237+0000","last_peered":"2023-11-25T04:22:48.047237+0000","last_clean":"2023-11-25T04:22:48.047237+0000","last_became_active":"2023-11-25T04:22:47.036821+0000","last_became_peered":"2023-11-25T04:22:47.036821+0000","last_unstale":"2023-11-25T04:22:48.047237+0000","last_undegraded":"2023-11-25T04:22:48.047237+0000","last_fullsized":"2023-11-25T04:22:48.047237+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:12:51.990448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"18'3","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048375+0000","last_change":"2023-11-25T04:22:46.035816+0000","last_active":"2023-11-25T04:22:48.048375+0000","last_peered":"2023-11-25T04:22:48.048375+0000","last_clean":"2023-11-25T04:22:48.048375+0000","last_became_active":"2023-11-25T04:22:46.035461+0000","last_became_peered":"2023-11-25T04:22:46.035461+0000","last_unstale":"2023-11-25T04:22:48.048375+0000","last_undegraded":"2023-11-25T04:22:48.048375+0000","last_fullsized":"2023-11-25T04:22:48.048375+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:58:26.155063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048370+0000","last_change":"2023-11-25T04:22:46.040458+0000","last_active":"2023-11-25T04:22:48.048370+0000","last_peered":"2023-11-25T04:22:48.048370+0000","last_clean":"2023-11-25T04:22:48.048370+0000","last_became_active":"2023-11-25T04:22:46.040143+0000","last_became_peered":"2023-11-25T04:22:46.040143+0000","last_unstale":"2023-11-25T04:22:48.048370+0000","last_undegraded":"2023-11-25T04:22:48.048370+0000","last_fullsized":"2023-11-25T04:22:48.048370+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:27:31.901618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047387+0000","last_change":"2023-11-25T04:22:47.040346+0000","last_active":"2023-11-25T04:22:48.047387+0000","last_peered":"2023-11-25T04:22:48.047387+0000","last_clean":"2023-11-25T04:22:48.047387+0000","last_became_active":"2023-11-25T04:22:47.040084+0000","last_became_peered":"2023-11-25T04:22:47.040084+0000","last_unstale":"2023-11-25T04:22:48.047387+0000","last_undegraded":"2023-11-25T04:22:48.047387+0000","last_fullsized":"2023-11-25T04:22:48.047387+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:59:45.492077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048308+0000","last_change":"2023-11-25T04:22:46.038721+0000","last_active":"2023-11-25T04:22:48.048308+0000","last_peered":"2023-11-25T04:22:48.048308+0000","last_clean":"2023-11-25T04:22:48.048308+0000","last_became_active":"2023-11-25T04:22:46.037698+0000","last_became_peered":"2023-11-25T04:22:46.037698+0000","last_unstale":"2023-11-25T04:22:48.048308+0000","last_undegraded":"2023-11-25T04:22:48.048308+0000","last_fullsized":"2023-11-25T04:22:48.048308+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:25:20.721529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047199+0000","last_change":"2023-11-25T04:22:47.037790+0000","last_active":"2023-11-25T04:22:48.047199+0000","last_peered":"2023-11-25T04:22:48.047199+0000","last_clean":"2023-11-25T04:22:48.047199+0000","last_became_active":"2023-11-25T04:22:47.037149+0000","last_became_peered":"2023-11-25T04:22:47.037149+0000","last_unstale":"2023-11-25T04:22:48.047199+0000","last_undegraded":"2023-11-25T04:22:48.047199+0000","last_fullsized":"2023-11-25T04:22:48.047199+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:31:54.857270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048584+0000","last_change":"2023-11-25T04:22:47.036079+0000","last_active":"2023-11-25T04:22:48.048584+0000","last_peered":"2023-11-25T04:22:48.048584+0000","last_clean":"2023-11-25T04:22:48.048584+0000","last_became_active":"2023-11-25T04:22:47.035625+0000","last_became_peered":"2023-11-25T04:22:47.035625+0000","last_unstale":"2023-11-25T04:22:48.048584+0000","last_undegraded":"2023-11-25T04:22:48.048584+0000","last_fullsized":"2023-11-25T04:22:48.048584+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:21:15.981901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047749+0000","last_change":"2023-11-25T04:22:46.036085+0000","last_active":"2023-11-25T04:22:48.047749+0000","last_peered":"2023-11-25T04:22:48.047749+0000","last_clean":"2023-11-25T04:22:48.047749+0000","last_became_active":"2023-11-25T04:22:46.035495+0000","last_became_peered":"2023-11-25T04:22:46.035495+0000","last_unstale":"2023-11-25T04:22:48.047749+0000","last_undegraded":"2023-11-25T04:22:48.047749+0000","last_fullsized":"2023-11-25T04:22:48.047749+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:14:56.735365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048251+0000","last_change":"2023-11-25T04:22:47.043064+0000","last_active":"2023-11-25T04:22:48.048251+0000","last_peered":"2023-11-25T04:22:48.048251+0000","last_clean":"2023-11-25T04:22:48.048251+0000","last_became_active":"2023-11-25T04:22:47.042760+0000","last_became_peered":"2023-11-25T04:22:47.042760+0000","last_unstale":"2023-11-25T04:22:48.048251+0000","last_undegraded":"2023-11-25T04:22:48.048251+0000","last_fullsized":"2023-11-25T04:22:48.048251+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:37:30.959147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047514+0000","last_change":"2023-11-25T04:22:46.038298+0000","last_active":"2023-11-25T04:22:48.047514+0000","last_peered":"2023-11-25T04:22:48.047514+0000","last_clean":"2023-11-25T04:22:48.047514+0000","last_became_active":"2023-11-25T04:22:46.036629+0000","last_became_peered":"2023-11-25T04:22:46.036629+0000","last_unstale":"2023-11-25T04:22:48.047514+0000","last_undegraded":"2023-11-25T04:22:48.047514+0000","last_fullsized":"2023-11-25T04:22:48.047514+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:45:14.670326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048209+0000","last_change":"2023-11-25T04:22:47.043156+0000","last_active":"2023-11-25T04:22:48.048209+0000","last_peered":"2023-11-25T04:22:48.048209+0000","last_clean":"2023-11-25T04:22:48.048209+0000","last_became_active":"2023-11-25T04:22:47.042693+0000","last_became_peered":"2023-11-25T04:22:47.042693+0000","last_unstale":"2023-11-25T04:22:48.048209+0000","last_undegraded":"2023-11-25T04:22:48.048209+0000","last_fullsized":"2023-11-25T04:22:48.048209+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:38:38.839941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1a","version":"18'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047866+0000","last_change":"2023-11-25T04:22:46.034340+0000","last_active":"2023-11-25T04:22:48.047866+0000","last_peered":"2023-11-25T04:22:48.047866+0000","last_clean":"2023-11-25T04:22:48.047866+0000","last_became_active":"2023-11-25T04:22:46.033944+0000","last_became_peered":"2023-11-25T04:22:46.033944+0000","last_unstale":"2023-11-25T04:22:48.047866+0000","last_undegraded":"2023-11-25T04:22:48.047866+0000","last_fullsized":"2023-11-25T04:22:48.047866+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:59:30.291004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047694+0000","last_change":"2023-11-25T04:22:47.039547+0000","last_active":"2023-11-25T04:22:48.047694+0000","last_peered":"2023-11-25T04:22:48.047694+0000","last_clean":"2023-11-25T04:22:48.047694+0000","last_became_active":"2023-11-25T04:22:47.039218+0000","last_became_peered":"2023-11-25T04:22:47.039218+0000","last_unstale":"2023-11-25T04:22:48.047694+0000","last_undegraded":"2023-11-25T04:22:48.047694+0000","last_fullsized":"2023-11-25T04:22:48.047694+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:44:48.553972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"18'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047860+0000","last_change":"2023-11-25T04:22:46.035882+0000","last_active":"2023-11-25T04:22:48.047860+0000","last_peered":"2023-11-25T04:22:48.047860+0000","last_clean":"2023-11-25T04:22:48.047860+0000","last_became_active":"2023-11-25T04:22:46.035588+0000","last_became_peered":"2023-11-25T04:22:46.035588+0000","last_unstale":"2023-11-25T04:22:48.047860+0000","last_undegraded":"2023-11-25T04:22:48.047860+0000","last_fullsized":"2023-11-25T04:22:48.047860+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:43:10.702849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047831+0000","last_change":"2023-11-25T04:22:47.041267+0000","last_active":"2023-11-25T04:22:48.047831+0000","last_peered":"2023-11-25T04:22:48.047831+0000","last_clean":"2023-11-25T04:22:48.047831+0000","last_became_active":"2023-11-25T04:22:47.040408+0000","last_became_peered":"2023-11-25T04:22:47.040408+0000","last_unstale":"2023-11-25T04:22:48.047831+0000","last_undegraded":"2023-11-25T04:22:48.047831+0000","last_fullsized":"2023-11-25T04:22:48.047831+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:37:30.959147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"18'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048131+0000","last_change":"2023-11-25T04:22:46.040793+0000","last_active":"2023-11-25T04:22:48.048131+0000","last_peered":"2023-11-25T04:22:48.048131+0000","last_clean":"2023-11-25T04:22:48.048131+0000","last_became_active":"2023-11-25T04:22:46.040493+0000","last_became_peered":"2023-11-25T04:22:46.040493+0000","last_unstale":"2023-11-25T04:22:48.048131+0000","last_undegraded":"2023-11-25T04:22:48.048131+0000","last_fullsized":"2023-11-25T04:22:48.048131+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:58:26.155063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048056+0000","last_change":"2023-11-25T04:22:47.040067+0000","last_active":"2023-11-25T04:22:48.048056+0000","last_peered":"2023-11-25T04:22:48.048056+0000","last_clean":"2023-11-25T04:22:48.048056+0000","last_became_active":"2023-11-25T04:22:47.038425+0000","last_became_peered":"2023-11-25T04:22:47.038425+0000","last_unstale":"2023-11-25T04:22:48.048056+0000","last_undegraded":"2023-11-25T04:22:48.048056+0000","last_fullsized":"2023-11-25T04:22:48.048056+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:25:06.132517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"18'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048297+0000","last_change":"2023-11-25T04:22:46.035872+0000","last_active":"2023-11-25T04:22:48.048297+0000","last_peered":"2023-11-25T04:22:48.048297+0000","last_clean":"2023-11-25T04:22:48.048297+0000","last_became_active":"2023-11-25T04:22:46.035575+0000","last_became_peered":"2023-11-25T04:22:46.035575+0000","last_unstale":"2023-11-25T04:22:48.048297+0000","last_undegraded":"2023-11-25T04:22:48.048297+0000","last_fullsized":"2023-11-25T04:22:48.048297+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:58:14.877413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047725+0000","last_change":"2023-11-25T04:22:47.036764+0000","last_active":"2023-11-25T04:22:48.047725+0000","last_peered":"2023-11-25T04:22:48.047725+0000","last_clean":"2023-11-25T04:22:48.047725+0000","last_became_active":"2023-11-25T04:22:47.036406+0000","last_became_peered":"2023-11-25T04:22:47.036406+0000","last_unstale":"2023-11-25T04:22:48.047725+0000","last_undegraded":"2023-11-25T04:22:48.047725+0000","last_fullsized":"2023-11-25T04:22:48.047725+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:09:55.357642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1e","version":"18'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048335+0000","last_change":"2023-11-25T04:22:46.035659+0000","last_active":"2023-11-25T04:22:48.048335+0000","last_peered":"2023-11-25T04:22:48.048335+0000","last_clean":"2023-11-25T04:22:48.048335+0000","last_became_active":"2023-11-25T04:22:46.035113+0000","last_became_peered":"2023-11-25T04:22:46.035113+0000","last_unstale":"2023-11-25T04:22:48.048335+0000","last_undegraded":"2023-11-25T04:22:48.048335+0000","last_fullsized":"2023-11-25T04:22:48.048335+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:26:25.338117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048474+0000","last_change":"2023-11-25T04:22:47.041554+0000","last_active":"2023-11-25T04:22:48.048474+0000","last_peered":"2023-11-25T04:22:48.048474+0000","last_clean":"2023-11-25T04:22:48.048474+0000","last_became_active":"2023-11-25T04:22:47.040314+0000","last_became_peered":"2023-11-25T04:22:47.040314+0000","last_unstale":"2023-11-25T04:22:48.048474+0000","last_undegraded":"2023-11-25T04:22:48.048474+0000","last_fullsized":"2023-11-25T04:22:48.048474+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:24:56.315251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"18'4","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048204+0000","last_change":"2023-11-25T04:22:46.038702+0000","last_active":"2023-11-25T04:22:48.048204+0000","last_peered":"2023-11-25T04:22:48.048204+0000","last_clean":"2023-11-25T04:22:48.048204+0000","last_became_active":"2023-11-25T04:22:46.037567+0000","last_became_peered":"2023-11-25T04:22:46.037567+0000","last_unstale":"2023-11-25T04:22:48.048204+0000","last_undegraded":"2023-11-25T04:22:48.048204+0000","last_fullsized":"2023-11-25T04:22:48.048204+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:58:26.155063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738377,"num_pgs":72,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35508,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336332,"statfs":{"total":96636764160,"available":96600403968,"internally_reserved":0,"allocated":172032,"data_stored":58325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]}]},{"osd":1,"up_from":8,"seq":34359738377,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31944,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339896,"statfs":{"total":96636764160,"available":96604053504,"internally_reserved":0,"allocated":782336,"data_stored":649871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738376,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31952,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339888,"statfs":{"total":96636764160,"available":96604045312,"internally_reserved":0,"allocated":790528,"data_stored":650487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-25T04:23:02.692 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-25T04:23:03.025 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:23:03.026 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2023-11-25T04:23:03.035 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2023-11-25T04:23:01.496402+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":248,"num_write_kb":3474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":196,"ondisk_log_size":196,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":99404,"kb_used_data":1704,"kb_used_omap":4,"kb_used_meta":97659,"kb_avail":283016116,"statfs":{"total":289910292480,"available":289808502784,"internally_reserved":0,"allocated":1744896,"data_stored":1358683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":100003151},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003034"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048437+0000","last_change":"2023-11-25T04:22:47.039847+0000","last_active":"2023-11-25T04:22:48.048437+0000","last_peered":"2023-11-25T04:22:48.048437+0000","last_clean":"2023-11-25T04:22:48.048437+0000","last_became_active":"2023-11-25T04:22:47.039490+0000","last_became_peered":"2023-11-25T04:22:47.039490+0000","last_unstale":"2023-11-25T04:22:48.048437+0000","last_undegraded":"2023-11-25T04:22:48.048437+0000","last_fullsized":"2023-11-25T04:22:48.048437+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:25:06.132517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048105+0000","last_change":"2023-11-25T04:22:47.040836+0000","last_active":"2023-11-25T04:22:48.048105+0000","last_peered":"2023-11-25T04:22:48.048105+0000","last_clean":"2023-11-25T04:22:48.048105+0000","last_became_active":"2023-11-25T04:22:47.040511+0000","last_became_peered":"2023-11-25T04:22:47.040511+0000","last_unstale":"2023-11-25T04:22:48.048105+0000","last_undegraded":"2023-11-25T04:22:48.048105+0000","last_fullsized":"2023-11-25T04:22:48.048105+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:15:39.511043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048297+0000","last_change":"2023-11-25T04:22:47.043529+0000","last_active":"2023-11-25T04:22:48.048297+0000","last_peered":"2023-11-25T04:22:48.048297+0000","last_clean":"2023-11-25T04:22:48.048297+0000","last_became_active":"2023-11-25T04:22:47.042733+0000","last_became_peered":"2023-11-25T04:22:47.042733+0000","last_unstale":"2023-11-25T04:22:48.048297+0000","last_undegraded":"2023-11-25T04:22:48.048297+0000","last_fullsized":"2023-11-25T04:22:48.048297+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:47:34.764521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047862+0000","last_change":"2023-11-25T04:22:47.040314+0000","last_active":"2023-11-25T04:22:48.047862+0000","last_peered":"2023-11-25T04:22:48.047862+0000","last_clean":"2023-11-25T04:22:48.047862+0000","last_became_active":"2023-11-25T04:22:47.039185+0000","last_became_peered":"2023-11-25T04:22:47.039185+0000","last_unstale":"2023-11-25T04:22:48.047862+0000","last_undegraded":"2023-11-25T04:22:48.047862+0000","last_fullsized":"2023-11-25T04:22:48.047862+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:16:29.085050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047841+0000","last_change":"2023-11-25T04:22:47.039499+0000","last_active":"2023-11-25T04:22:48.047841+0000","last_peered":"2023-11-25T04:22:48.047841+0000","last_clean":"2023-11-25T04:22:48.047841+0000","last_became_active":"2023-11-25T04:22:47.039165+0000","last_became_peered":"2023-11-25T04:22:47.039165+0000","last_unstale":"2023-11-25T04:22:48.047841+0000","last_undegraded":"2023-11-25T04:22:48.047841+0000","last_fullsized":"2023-11-25T04:22:48.047841+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:57:28.855151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047565+0000","last_change":"2023-11-25T04:22:47.039992+0000","last_active":"2023-11-25T04:22:48.047565+0000","last_peered":"2023-11-25T04:22:48.047565+0000","last_clean":"2023-11-25T04:22:48.047565+0000","last_became_active":"2023-11-25T04:22:47.039405+0000","last_became_peered":"2023-11-25T04:22:47.039405+0000","last_unstale":"2023-11-25T04:22:48.047565+0000","last_undegraded":"2023-11-25T04:22:48.047565+0000","last_fullsized":"2023-11-25T04:22:48.047565+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:56:59.689578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047396+0000","last_change":"2023-11-25T04:22:47.037971+0000","last_active":"2023-11-25T04:22:48.047396+0000","last_peered":"2023-11-25T04:22:48.047396+0000","last_clean":"2023-11-25T04:22:48.047396+0000","last_became_active":"2023-11-25T04:22:47.036736+0000","last_became_peered":"2023-11-25T04:22:47.036736+0000","last_unstale":"2023-11-25T04:22:48.047396+0000","last_undegraded":"2023-11-25T04:22:48.047396+0000","last_fullsized":"2023-11-25T04:22:48.047396+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:02:07.088241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047307+0000","last_change":"2023-11-25T04:22:47.041728+0000","last_active":"2023-11-25T04:22:48.047307+0000","last_peered":"2023-11-25T04:22:48.047307+0000","last_clean":"2023-11-25T04:22:48.047307+0000","last_became_active":"2023-11-25T04:22:47.041017+0000","last_became_peered":"2023-11-25T04:22:47.041017+0000","last_unstale":"2023-11-25T04:22:48.047307+0000","last_undegraded":"2023-11-25T04:22:48.047307+0000","last_fullsized":"2023-11-25T04:22:48.047307+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:57:28.855151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048097+0000","last_change":"2023-11-25T04:22:47.038397+0000","last_active":"2023-11-25T04:22:48.048097+0000","last_peered":"2023-11-25T04:22:48.048097+0000","last_clean":"2023-11-25T04:22:48.048097+0000","last_became_active":"2023-11-25T04:22:47.037328+0000","last_became_peered":"2023-11-25T04:22:47.037328+0000","last_unstale":"2023-11-25T04:22:48.048097+0000","last_undegraded":"2023-11-25T04:22:48.048097+0000","last_fullsized":"2023-11-25T04:22:48.048097+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:56:59.689578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048223+0000","last_change":"2023-11-25T04:22:47.040182+0000","last_active":"2023-11-25T04:22:48.048223+0000","last_peered":"2023-11-25T04:22:48.048223+0000","last_clean":"2023-11-25T04:22:48.048223+0000","last_became_active":"2023-11-25T04:22:47.039892+0000","last_became_peered":"2023-11-25T04:22:47.039892+0000","last_unstale":"2023-11-25T04:22:48.048223+0000","last_undegraded":"2023-11-25T04:22:48.048223+0000","last_fullsized":"2023-11-25T04:22:48.048223+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:38:09.176582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048202+0000","last_change":"2023-11-25T04:22:47.039907+0000","last_active":"2023-11-25T04:22:48.048202+0000","last_peered":"2023-11-25T04:22:48.048202+0000","last_clean":"2023-11-25T04:22:48.048202+0000","last_became_active":"2023-11-25T04:22:47.039046+0000","last_became_peered":"2023-11-25T04:22:47.039046+0000","last_unstale":"2023-11-25T04:22:48.048202+0000","last_undegraded":"2023-11-25T04:22:48.048202+0000","last_fullsized":"2023-11-25T04:22:48.048202+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:01:51.418774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047609+0000","last_change":"2023-11-25T04:22:47.037618+0000","last_active":"2023-11-25T04:22:48.047609+0000","last_peered":"2023-11-25T04:22:48.047609+0000","last_clean":"2023-11-25T04:22:48.047609+0000","last_became_active":"2023-11-25T04:22:47.036806+0000","last_became_peered":"2023-11-25T04:22:47.036806+0000","last_unstale":"2023-11-25T04:22:48.047609+0000","last_undegraded":"2023-11-25T04:22:48.047609+0000","last_fullsized":"2023-11-25T04:22:48.047609+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:12:51.990448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047608+0000","last_change":"2023-11-25T04:22:47.040388+0000","last_active":"2023-11-25T04:22:48.047608+0000","last_peered":"2023-11-25T04:22:48.047608+0000","last_clean":"2023-11-25T04:22:48.047608+0000","last_became_active":"2023-11-25T04:22:47.040129+0000","last_became_peered":"2023-11-25T04:22:47.040129+0000","last_unstale":"2023-11-25T04:22:48.047608+0000","last_undegraded":"2023-11-25T04:22:48.047608+0000","last_fullsized":"2023-11-25T04:22:48.047608+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:31:54.857270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047643+0000","last_change":"2023-11-25T04:22:47.038887+0000","last_active":"2023-11-25T04:22:48.047643+0000","last_peered":"2023-11-25T04:22:48.047643+0000","last_clean":"2023-11-25T04:22:48.047643+0000","last_became_active":"2023-11-25T04:22:47.038014+0000","last_became_peered":"2023-11-25T04:22:47.038014+0000","last_unstale":"2023-11-25T04:22:48.047643+0000","last_undegraded":"2023-11-25T04:22:48.047643+0000","last_fullsized":"2023-11-25T04:22:48.047643+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:38:38.839941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047471+0000","last_change":"2023-11-25T04:22:47.035843+0000","last_active":"2023-11-25T04:22:48.047471+0000","last_peered":"2023-11-25T04:22:48.047471+0000","last_clean":"2023-11-25T04:22:48.047471+0000","last_became_active":"2023-11-25T04:22:47.035453+0000","last_became_peered":"2023-11-25T04:22:47.035453+0000","last_unstale":"2023-11-25T04:22:48.047471+0000","last_undegraded":"2023-11-25T04:22:48.047471+0000","last_fullsized":"2023-11-25T04:22:48.047471+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:45:40.065146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047308+0000","last_change":"2023-11-25T04:22:47.041778+0000","last_active":"2023-11-25T04:22:48.047308+0000","last_peered":"2023-11-25T04:22:48.047308+0000","last_clean":"2023-11-25T04:22:48.047308+0000","last_became_active":"2023-11-25T04:22:47.041115+0000","last_became_peered":"2023-11-25T04:22:47.041115+0000","last_unstale":"2023-11-25T04:22:48.047308+0000","last_undegraded":"2023-11-25T04:22:48.047308+0000","last_fullsized":"2023-11-25T04:22:48.047308+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:30:04.568790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048422+0000","last_change":"2023-11-25T04:22:47.039389+0000","last_active":"2023-11-25T04:22:48.048422+0000","last_peered":"2023-11-25T04:22:48.048422+0000","last_clean":"2023-11-25T04:22:48.048422+0000","last_became_active":"2023-11-25T04:22:47.039123+0000","last_became_peered":"2023-11-25T04:22:47.039123+0000","last_unstale":"2023-11-25T04:22:48.048422+0000","last_undegraded":"2023-11-25T04:22:48.048422+0000","last_fullsized":"2023-11-25T04:22:48.048422+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:09:11.570561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048403+0000","last_change":"2023-11-25T04:22:47.036946+0000","last_active":"2023-11-25T04:22:48.048403+0000","last_peered":"2023-11-25T04:22:48.048403+0000","last_clean":"2023-11-25T04:22:48.048403+0000","last_became_active":"2023-11-25T04:22:47.036411+0000","last_became_peered":"2023-11-25T04:22:47.036411+0000","last_unstale":"2023-11-25T04:22:48.048403+0000","last_undegraded":"2023-11-25T04:22:48.048403+0000","last_fullsized":"2023-11-25T04:22:48.048403+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:16:29.085050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047729+0000","last_change":"2023-11-25T04:22:47.038983+0000","last_active":"2023-11-25T04:22:48.047729+0000","last_peered":"2023-11-25T04:22:48.047729+0000","last_clean":"2023-11-25T04:22:48.047729+0000","last_became_active":"2023-11-25T04:22:47.038477+0000","last_became_peered":"2023-11-25T04:22:47.038477+0000","last_unstale":"2023-11-25T04:22:48.047729+0000","last_undegraded":"2023-11-25T04:22:48.047729+0000","last_fullsized":"2023-11-25T04:22:48.047729+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:30:04.568790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047891+0000","last_change":"2023-11-25T04:22:47.038155+0000","last_active":"2023-11-25T04:22:48.047891+0000","last_peered":"2023-11-25T04:22:48.047891+0000","last_clean":"2023-11-25T04:22:48.047891+0000","last_became_active":"2023-11-25T04:22:47.037740+0000","last_became_peered":"2023-11-25T04:22:47.037740+0000","last_unstale":"2023-11-25T04:22:48.047891+0000","last_undegraded":"2023-11-25T04:22:48.047891+0000","last_fullsized":"2023-11-25T04:22:48.047891+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:09:11.570561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048543+0000","last_change":"2023-11-25T04:22:47.040641+0000","last_active":"2023-11-25T04:22:48.048543+0000","last_peered":"2023-11-25T04:22:48.048543+0000","last_clean":"2023-11-25T04:22:48.048543+0000","last_became_active":"2023-11-25T04:22:47.039409+0000","last_became_peered":"2023-11-25T04:22:47.039409+0000","last_unstale":"2023-11-25T04:22:48.048543+0000","last_undegraded":"2023-11-25T04:22:48.048543+0000","last_fullsized":"2023-11-25T04:22:48.048543+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:23:15.914817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047384+0000","last_change":"2023-11-25T04:22:47.043456+0000","last_active":"2023-11-25T04:22:48.047384+0000","last_peered":"2023-11-25T04:22:48.047384+0000","last_clean":"2023-11-25T04:22:48.047384+0000","last_became_active":"2023-11-25T04:22:47.043098+0000","last_became_peered":"2023-11-25T04:22:47.043098+0000","last_unstale":"2023-11-25T04:22:48.047384+0000","last_undegraded":"2023-11-25T04:22:48.047384+0000","last_fullsized":"2023-11-25T04:22:48.047384+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:47:57.307757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047103+0000","last_change":"2023-11-25T04:22:47.035260+0000","last_active":"2023-11-25T04:22:48.047103+0000","last_peered":"2023-11-25T04:22:48.047103+0000","last_clean":"2023-11-25T04:22:48.047103+0000","last_became_active":"2023-11-25T04:22:47.034702+0000","last_became_peered":"2023-11-25T04:22:47.034702+0000","last_unstale":"2023-11-25T04:22:48.047103+0000","last_undegraded":"2023-11-25T04:22:48.047103+0000","last_fullsized":"2023-11-25T04:22:48.047103+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:49:02.500022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047259+0000","last_change":"2023-11-25T04:22:47.036805+0000","last_active":"2023-11-25T04:22:48.047259+0000","last_peered":"2023-11-25T04:22:48.047259+0000","last_clean":"2023-11-25T04:22:48.047259+0000","last_became_active":"2023-11-25T04:22:47.036135+0000","last_became_peered":"2023-11-25T04:22:47.036135+0000","last_unstale":"2023-11-25T04:22:48.047259+0000","last_undegraded":"2023-11-25T04:22:48.047259+0000","last_fullsized":"2023-11-25T04:22:48.047259+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:25:21.730992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048430+0000","last_change":"2023-11-25T04:22:47.036611+0000","last_active":"2023-11-25T04:22:48.048430+0000","last_peered":"2023-11-25T04:22:48.048430+0000","last_clean":"2023-11-25T04:22:48.048430+0000","last_became_active":"2023-11-25T04:22:47.035762+0000","last_became_peered":"2023-11-25T04:22:47.035762+0000","last_unstale":"2023-11-25T04:22:48.048430+0000","last_undegraded":"2023-11-25T04:22:48.048430+0000","last_fullsized":"2023-11-25T04:22:48.048430+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:09:11.570561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048216+0000","last_change":"2023-11-25T04:22:47.042423+0000","last_active":"2023-11-25T04:22:48.048216+0000","last_peered":"2023-11-25T04:22:48.048216+0000","last_clean":"2023-11-25T04:22:48.048216+0000","last_became_active":"2023-11-25T04:22:47.042022+0000","last_became_peered":"2023-11-25T04:22:47.042022+0000","last_unstale":"2023-11-25T04:22:48.048216+0000","last_undegraded":"2023-11-25T04:22:48.048216+0000","last_fullsized":"2023-11-25T04:22:48.048216+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:38:09.176582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048337+0000","last_change":"2023-11-25T04:22:47.044040+0000","last_active":"2023-11-25T04:22:48.048337+0000","last_peered":"2023-11-25T04:22:48.048337+0000","last_clean":"2023-11-25T04:22:48.048337+0000","last_became_active":"2023-11-25T04:22:47.043329+0000","last_became_peered":"2023-11-25T04:22:47.043329+0000","last_unstale":"2023-11-25T04:22:48.048337+0000","last_undegraded":"2023-11-25T04:22:48.048337+0000","last_fullsized":"2023-11-25T04:22:48.048337+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:10:59.482775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047954+0000","last_change":"2023-11-25T04:22:47.036485+0000","last_active":"2023-11-25T04:22:48.047954+0000","last_peered":"2023-11-25T04:22:48.047954+0000","last_clean":"2023-11-25T04:22:48.047954+0000","last_became_active":"2023-11-25T04:22:47.035799+0000","last_became_peered":"2023-11-25T04:22:47.035799+0000","last_unstale":"2023-11-25T04:22:48.047954+0000","last_undegraded":"2023-11-25T04:22:48.047954+0000","last_fullsized":"2023-11-25T04:22:48.047954+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:45:40.065146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047835+0000","last_change":"2023-11-25T04:22:47.037668+0000","last_active":"2023-11-25T04:22:48.047835+0000","last_peered":"2023-11-25T04:22:48.047835+0000","last_clean":"2023-11-25T04:22:48.047835+0000","last_became_active":"2023-11-25T04:22:47.036823+0000","last_became_peered":"2023-11-25T04:22:47.036823+0000","last_unstale":"2023-11-25T04:22:48.047835+0000","last_undegraded":"2023-11-25T04:22:48.047835+0000","last_fullsized":"2023-11-25T04:22:48.047835+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:02:07.088241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047758+0000","last_change":"2023-11-25T04:22:47.038117+0000","last_active":"2023-11-25T04:22:48.047758+0000","last_peered":"2023-11-25T04:22:48.047758+0000","last_clean":"2023-11-25T04:22:48.047758+0000","last_became_active":"2023-11-25T04:22:47.037751+0000","last_became_peered":"2023-11-25T04:22:47.037751+0000","last_unstale":"2023-11-25T04:22:48.047758+0000","last_undegraded":"2023-11-25T04:22:48.047758+0000","last_fullsized":"2023-11-25T04:22:48.047758+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:30:04.568790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047184+0000","last_change":"2023-11-25T04:22:47.037786+0000","last_active":"2023-11-25T04:22:48.047184+0000","last_peered":"2023-11-25T04:22:48.047184+0000","last_clean":"2023-11-25T04:22:48.047184+0000","last_became_active":"2023-11-25T04:22:47.037206+0000","last_became_peered":"2023-11-25T04:22:47.037206+0000","last_unstale":"2023-11-25T04:22:48.047184+0000","last_undegraded":"2023-11-25T04:22:48.047184+0000","last_fullsized":"2023-11-25T04:22:48.047184+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:11:40.532140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047505+0000","last_change":"2023-11-25T04:22:47.041325+0000","last_active":"2023-11-25T04:22:48.047505+0000","last_peered":"2023-11-25T04:22:48.047505+0000","last_clean":"2023-11-25T04:22:48.047505+0000","last_became_active":"2023-11-25T04:22:47.040086+0000","last_became_peered":"2023-11-25T04:22:47.040086+0000","last_unstale":"2023-11-25T04:22:48.047505+0000","last_undegraded":"2023-11-25T04:22:48.047505+0000","last_fullsized":"2023-11-25T04:22:48.047505+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:15:39.511043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047822+0000","last_change":"2023-11-25T04:22:47.036689+0000","last_active":"2023-11-25T04:22:48.047822+0000","last_peered":"2023-11-25T04:22:48.047822+0000","last_clean":"2023-11-25T04:22:48.047822+0000","last_became_active":"2023-11-25T04:22:47.036107+0000","last_became_peered":"2023-11-25T04:22:47.036107+0000","last_unstale":"2023-11-25T04:22:48.047822+0000","last_undegraded":"2023-11-25T04:22:48.047822+0000","last_fullsized":"2023-11-25T04:22:48.047822+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:02:07.088241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"18'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048313+0000","last_change":"2023-11-25T04:22:46.040844+0000","last_active":"2023-11-25T04:22:48.048313+0000","last_peered":"2023-11-25T04:22:48.048313+0000","last_clean":"2023-11-25T04:22:48.048313+0000","last_became_active":"2023-11-25T04:22:46.040395+0000","last_became_peered":"2023-11-25T04:22:46.040395+0000","last_unstale":"2023-11-25T04:22:48.048313+0000","last_undegraded":"2023-11-25T04:22:48.048313+0000","last_fullsized":"2023-11-25T04:22:48.048313+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:11:39.522677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048200+0000","last_change":"2023-11-25T04:22:47.037888+0000","last_active":"2023-11-25T04:22:48.048200+0000","last_peered":"2023-11-25T04:22:48.048200+0000","last_clean":"2023-11-25T04:22:48.048200+0000","last_became_active":"2023-11-25T04:22:47.037158+0000","last_became_peered":"2023-11-25T04:22:47.037158+0000","last_unstale":"2023-11-25T04:22:48.048200+0000","last_undegraded":"2023-11-25T04:22:48.048200+0000","last_fullsized":"2023-11-25T04:22:48.048200+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:56:59.689578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047391+0000","last_change":"2023-11-25T04:22:46.036054+0000","last_active":"2023-11-25T04:22:48.047391+0000","last_peered":"2023-11-25T04:22:48.047391+0000","last_clean":"2023-11-25T04:22:48.047391+0000","last_became_active":"2023-11-25T04:22:46.035620+0000","last_became_peered":"2023-11-25T04:22:46.035620+0000","last_unstale":"2023-11-25T04:22:48.047391+0000","last_undegraded":"2023-11-25T04:22:48.047391+0000","last_fullsized":"2023-11-25T04:22:48.047391+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:19:48.371980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048391+0000","last_change":"2023-11-25T04:22:47.045025+0000","last_active":"2023-11-25T04:22:48.048391+0000","last_peered":"2023-11-25T04:22:48.048391+0000","last_clean":"2023-11-25T04:22:48.048391+0000","last_became_active":"2023-11-25T04:22:47.044697+0000","last_became_peered":"2023-11-25T04:22:47.044697+0000","last_unstale":"2023-11-25T04:22:48.048391+0000","last_undegraded":"2023-11-25T04:22:48.048391+0000","last_fullsized":"2023-11-25T04:22:48.048391+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:16:29.085050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047517+0000","last_change":"2023-11-25T04:22:46.037169+0000","last_active":"2023-11-25T04:22:48.047517+0000","last_peered":"2023-11-25T04:22:48.047517+0000","last_clean":"2023-11-25T04:22:48.047517+0000","last_became_active":"2023-11-25T04:22:46.036948+0000","last_became_peered":"2023-11-25T04:22:46.036948+0000","last_unstale":"2023-11-25T04:22:48.047517+0000","last_undegraded":"2023-11-25T04:22:48.047517+0000","last_fullsized":"2023-11-25T04:22:48.047517+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:45:14.670326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047715+0000","last_change":"2023-11-25T04:22:47.044860+0000","last_active":"2023-11-25T04:22:48.047715+0000","last_peered":"2023-11-25T04:22:48.047715+0000","last_clean":"2023-11-25T04:22:48.047715+0000","last_became_active":"2023-11-25T04:22:47.044551+0000","last_became_peered":"2023-11-25T04:22:47.044551+0000","last_unstale":"2023-11-25T04:22:48.047715+0000","last_undegraded":"2023-11-25T04:22:48.047715+0000","last_fullsized":"2023-11-25T04:22:48.047715+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:09:55.357642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.10","version":"18'8","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047593+0000","last_change":"2023-11-25T04:22:46.040356+0000","last_active":"2023-11-25T04:22:48.047593+0000","last_peered":"2023-11-25T04:22:48.047593+0000","last_clean":"2023-11-25T04:22:48.047593+0000","last_became_active":"2023-11-25T04:22:46.039802+0000","last_became_peered":"2023-11-25T04:22:46.039802+0000","last_unstale":"2023-11-25T04:22:48.047593+0000","last_undegraded":"2023-11-25T04:22:48.047593+0000","last_fullsized":"2023-11-25T04:22:48.047593+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:09:54.348179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"18'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048372+0000","last_change":"2023-11-25T04:22:46.040082+0000","last_active":"2023-11-25T04:22:48.048372+0000","last_peered":"2023-11-25T04:22:48.048372+0000","last_clean":"2023-11-25T04:22:48.048372+0000","last_became_active":"2023-11-25T04:22:46.039763+0000","last_became_peered":"2023-11-25T04:22:46.039763+0000","last_unstale":"2023-11-25T04:22:48.048372+0000","last_undegraded":"2023-11-25T04:22:48.048372+0000","last_fullsized":"2023-11-25T04:22:48.048372+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:10:20.178136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047265+0000","last_change":"2023-11-25T04:22:47.043644+0000","last_active":"2023-11-25T04:22:48.047265+0000","last_peered":"2023-11-25T04:22:48.047265+0000","last_clean":"2023-11-25T04:22:48.047265+0000","last_became_active":"2023-11-25T04:22:47.043221+0000","last_became_peered":"2023-11-25T04:22:47.043221+0000","last_unstale":"2023-11-25T04:22:48.047265+0000","last_undegraded":"2023-11-25T04:22:48.047265+0000","last_fullsized":"2023-11-25T04:22:48.047265+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:41:16.672268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"18'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048462+0000","last_change":"2023-11-25T04:22:46.035806+0000","last_active":"2023-11-25T04:22:48.048462+0000","last_peered":"2023-11-25T04:22:48.048462+0000","last_clean":"2023-11-25T04:22:48.048462+0000","last_became_active":"2023-11-25T04:22:46.035418+0000","last_became_peered":"2023-11-25T04:22:46.035418+0000","last_unstale":"2023-11-25T04:22:48.048462+0000","last_undegraded":"2023-11-25T04:22:48.048462+0000","last_fullsized":"2023-11-25T04:22:48.048462+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:31:53.068767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047508+0000","last_change":"2023-11-25T04:22:47.039685+0000","last_active":"2023-11-25T04:22:48.047508+0000","last_peered":"2023-11-25T04:22:48.047508+0000","last_clean":"2023-11-25T04:22:48.047508+0000","last_became_active":"2023-11-25T04:22:47.039296+0000","last_became_peered":"2023-11-25T04:22:47.039296+0000","last_unstale":"2023-11-25T04:22:48.047508+0000","last_undegraded":"2023-11-25T04:22:48.047508+0000","last_fullsized":"2023-11-25T04:22:48.047508+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:11:40.532140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"18'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048658+0000","last_change":"2023-11-25T04:22:46.039953+0000","last_active":"2023-11-25T04:22:48.048658+0000","last_peered":"2023-11-25T04:22:48.048658+0000","last_clean":"2023-11-25T04:22:48.048658+0000","last_became_active":"2023-11-25T04:22:46.039660+0000","last_became_peered":"2023-11-25T04:22:46.039660+0000","last_unstale":"2023-11-25T04:22:48.048658+0000","last_undegraded":"2023-11-25T04:22:48.048658+0000","last_fullsized":"2023-11-25T04:22:48.048658+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:59:44.482613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047887+0000","last_change":"2023-11-25T04:22:47.037912+0000","last_active":"2023-11-25T04:22:48.047887+0000","last_peered":"2023-11-25T04:22:48.047887+0000","last_clean":"2023-11-25T04:22:48.047887+0000","last_became_active":"2023-11-25T04:22:47.036552+0000","last_became_peered":"2023-11-25T04:22:47.036552+0000","last_unstale":"2023-11-25T04:22:48.047887+0000","last_undegraded":"2023-11-25T04:22:48.047887+0000","last_fullsized":"2023-11-25T04:22:48.047887+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:38:09.176582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048295+0000","last_change":"2023-11-25T04:22:46.039176+0000","last_active":"2023-11-25T04:22:48.048295+0000","last_peered":"2023-11-25T04:22:48.048295+0000","last_clean":"2023-11-25T04:22:48.048295+0000","last_became_active":"2023-11-25T04:22:46.038842+0000","last_became_peered":"2023-11-25T04:22:46.038842+0000","last_unstale":"2023-11-25T04:22:48.048295+0000","last_undegraded":"2023-11-25T04:22:48.048295+0000","last_fullsized":"2023-11-25T04:22:48.048295+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:47:56.298293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047733+0000","last_change":"2023-11-25T04:22:47.042155+0000","last_active":"2023-11-25T04:22:48.047733+0000","last_peered":"2023-11-25T04:22:48.047733+0000","last_clean":"2023-11-25T04:22:48.047733+0000","last_became_active":"2023-11-25T04:22:47.041433+0000","last_became_peered":"2023-11-25T04:22:47.041433+0000","last_unstale":"2023-11-25T04:22:48.047733+0000","last_undegraded":"2023-11-25T04:22:48.047733+0000","last_fullsized":"2023-11-25T04:22:48.047733+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:57:28.855151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"18'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048201+0000","last_change":"2023-11-25T04:22:46.037827+0000","last_active":"2023-11-25T04:22:48.048201+0000","last_peered":"2023-11-25T04:22:48.048201+0000","last_clean":"2023-11-25T04:22:48.048201+0000","last_became_active":"2023-11-25T04:22:46.036515+0000","last_became_peered":"2023-11-25T04:22:46.036515+0000","last_unstale":"2023-11-25T04:22:48.048201+0000","last_undegraded":"2023-11-25T04:22:48.048201+0000","last_fullsized":"2023-11-25T04:22:48.048201+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:10:20.178136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048154+0000","last_change":"2023-11-25T04:22:47.038856+0000","last_active":"2023-11-25T04:22:48.048154+0000","last_peered":"2023-11-25T04:22:48.048154+0000","last_clean":"2023-11-25T04:22:48.048154+0000","last_became_active":"2023-11-25T04:22:47.038190+0000","last_became_peered":"2023-11-25T04:22:47.038190+0000","last_unstale":"2023-11-25T04:22:48.048154+0000","last_undegraded":"2023-11-25T04:22:48.048154+0000","last_fullsized":"2023-11-25T04:22:48.048154+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:29:37.268624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047789+0000","last_change":"2023-11-25T04:22:46.038473+0000","last_active":"2023-11-25T04:22:48.047789+0000","last_peered":"2023-11-25T04:22:48.047789+0000","last_clean":"2023-11-25T04:22:48.047789+0000","last_became_active":"2023-11-25T04:22:46.038159+0000","last_became_peered":"2023-11-25T04:22:46.038159+0000","last_unstale":"2023-11-25T04:22:48.047789+0000","last_undegraded":"2023-11-25T04:22:48.047789+0000","last_fullsized":"2023-11-25T04:22:48.047789+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:31:53.847807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048568+0000","last_change":"2023-11-25T04:22:47.036222+0000","last_active":"2023-11-25T04:22:48.048568+0000","last_peered":"2023-11-25T04:22:48.048568+0000","last_clean":"2023-11-25T04:22:48.048568+0000","last_became_active":"2023-11-25T04:22:47.035871+0000","last_became_peered":"2023-11-25T04:22:47.035871+0000","last_unstale":"2023-11-25T04:22:48.048568+0000","last_undegraded":"2023-11-25T04:22:48.048568+0000","last_fullsized":"2023-11-25T04:22:48.048568+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:13:35.199538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047681+0000","last_change":"2023-11-25T04:22:46.035907+0000","last_active":"2023-11-25T04:22:48.047681+0000","last_peered":"2023-11-25T04:22:48.047681+0000","last_clean":"2023-11-25T04:22:48.047681+0000","last_became_active":"2023-11-25T04:22:46.035612+0000","last_became_peered":"2023-11-25T04:22:46.035612+0000","last_unstale":"2023-11-25T04:22:48.047681+0000","last_undegraded":"2023-11-25T04:22:48.047681+0000","last_fullsized":"2023-11-25T04:22:48.047681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:00:35.249875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048441+0000","last_change":"2023-11-25T04:22:47.038725+0000","last_active":"2023-11-25T04:22:48.048441+0000","last_peered":"2023-11-25T04:22:48.048441+0000","last_clean":"2023-11-25T04:22:48.048441+0000","last_became_active":"2023-11-25T04:22:47.038124+0000","last_became_peered":"2023-11-25T04:22:47.038124+0000","last_unstale":"2023-11-25T04:22:48.048441+0000","last_undegraded":"2023-11-25T04:22:48.048441+0000","last_fullsized":"2023-11-25T04:22:48.048441+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:00:26.187878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047437+0000","last_change":"2023-11-25T04:22:46.034509+0000","last_active":"2023-11-25T04:22:48.047437+0000","last_peered":"2023-11-25T04:22:48.047437+0000","last_clean":"2023-11-25T04:22:48.047437+0000","last_became_active":"2023-11-25T04:22:46.034158+0000","last_became_peered":"2023-11-25T04:22:46.034158+0000","last_unstale":"2023-11-25T04:22:48.047437+0000","last_undegraded":"2023-11-25T04:22:48.047437+0000","last_fullsized":"2023-11-25T04:22:48.047437+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:26:25.338117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048005+0000","last_change":"2023-11-25T04:22:47.038800+0000","last_active":"2023-11-25T04:22:48.048005+0000","last_peered":"2023-11-25T04:22:48.048005+0000","last_clean":"2023-11-25T04:22:48.048005+0000","last_became_active":"2023-11-25T04:22:47.038081+0000","last_became_peered":"2023-11-25T04:22:47.038081+0000","last_unstale":"2023-11-25T04:22:48.048005+0000","last_undegraded":"2023-11-25T04:22:48.048005+0000","last_fullsized":"2023-11-25T04:22:48.048005+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:45:40.065146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"12'157","reported_seq":214,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-25T04:22:47.033945+0000","last_change":"2023-11-25T04:22:36.931828+0000","last_active":"2023-11-25T04:22:47.033945+0000","last_peered":"2023-11-25T04:22:47.033945+0000","last_clean":"2023-11-25T04:22:47.033945+0000","last_became_active":"2023-11-25T04:22:36.931582+0000","last_became_peered":"2023-11-25T04:22:36.931582+0000","last_unstale":"2023-11-25T04:22:47.033945+0000","last_undegraded":"2023-11-25T04:22:47.033945+0000","last_fullsized":"2023-11-25T04:22:47.033945+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:35.919850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:35.919850+0000","last_clean_scrub_stamp":"2023-11-25T04:22:35.919850+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:59:59.198309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-25T04:22:47.031925+0000","last_change":"2023-11-25T04:22:41.999590+0000","last_active":"2023-11-25T04:22:47.031925+0000","last_peered":"2023-11-25T04:22:47.031925+0000","last_clean":"2023-11-25T04:22:47.031925+0000","last_became_active":"2023-11-25T04:22:41.999314+0000","last_became_peered":"2023-11-25T04:22:41.999314+0000","last_unstale":"2023-11-25T04:22:47.031925+0000","last_undegraded":"2023-11-25T04:22:47.031925+0000","last_fullsized":"2023-11-25T04:22:47.031925+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:20:12.785425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047840+0000","last_change":"2023-11-25T04:22:46.037044+0000","last_active":"2023-11-25T04:22:48.047840+0000","last_peered":"2023-11-25T04:22:48.047840+0000","last_clean":"2023-11-25T04:22:48.047840+0000","last_became_active":"2023-11-25T04:22:46.035916+0000","last_became_peered":"2023-11-25T04:22:46.035916+0000","last_unstale":"2023-11-25T04:22:48.047840+0000","last_undegraded":"2023-11-25T04:22:48.047840+0000","last_fullsized":"2023-11-25T04:22:48.047840+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:38:20.501102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048611+0000","last_change":"2023-11-25T04:22:47.036222+0000","last_active":"2023-11-25T04:22:48.048611+0000","last_peered":"2023-11-25T04:22:48.048611+0000","last_clean":"2023-11-25T04:22:48.048611+0000","last_became_active":"2023-11-25T04:22:47.035484+0000","last_became_peered":"2023-11-25T04:22:47.035484+0000","last_unstale":"2023-11-25T04:22:48.048611+0000","last_undegraded":"2023-11-25T04:22:48.048611+0000","last_fullsized":"2023-11-25T04:22:48.048611+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:18:56.845916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-25T04:22:47.032801+0000","last_change":"2023-11-25T04:22:42.000257+0000","last_active":"2023-11-25T04:22:47.032801+0000","last_peered":"2023-11-25T04:22:47.032801+0000","last_clean":"2023-11-25T04:22:47.032801+0000","last_became_active":"2023-11-25T04:22:41.999969+0000","last_became_peered":"2023-11-25T04:22:41.999969+0000","last_unstale":"2023-11-25T04:22:47.032801+0000","last_undegraded":"2023-11-25T04:22:47.032801+0000","last_fullsized":"2023-11-25T04:22:47.032801+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:21:34.424091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047780+0000","last_change":"2023-11-25T04:22:46.036121+0000","last_active":"2023-11-25T04:22:48.047780+0000","last_peered":"2023-11-25T04:22:48.047780+0000","last_clean":"2023-11-25T04:22:48.047780+0000","last_became_active":"2023-11-25T04:22:46.035654+0000","last_became_peered":"2023-11-25T04:22:46.035654+0000","last_unstale":"2023-11-25T04:22:48.047780+0000","last_undegraded":"2023-11-25T04:22:48.047780+0000","last_fullsized":"2023-11-25T04:22:48.047780+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:27:31.901618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048273+0000","last_change":"2023-11-25T04:22:47.039583+0000","last_active":"2023-11-25T04:22:48.048273+0000","last_peered":"2023-11-25T04:22:48.048273+0000","last_clean":"2023-11-25T04:22:48.048273+0000","last_became_active":"2023-11-25T04:22:47.039197+0000","last_became_peered":"2023-11-25T04:22:47.039197+0000","last_unstale":"2023-11-25T04:22:48.048273+0000","last_undegraded":"2023-11-25T04:22:48.048273+0000","last_fullsized":"2023-11-25T04:22:48.048273+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:37:30.959147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-25T04:22:47.029239+0000","last_change":"2023-11-25T04:22:42.000382+0000","last_active":"2023-11-25T04:22:47.029239+0000","last_peered":"2023-11-25T04:22:47.029239+0000","last_clean":"2023-11-25T04:22:47.029239+0000","last_became_active":"2023-11-25T04:22:42.000041+0000","last_became_peered":"2023-11-25T04:22:42.000041+0000","last_unstale":"2023-11-25T04:22:47.029239+0000","last_undegraded":"2023-11-25T04:22:47.029239+0000","last_fullsized":"2023-11-25T04:22:47.029239+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:08:19.655864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047444+0000","last_change":"2023-11-25T04:22:46.037431+0000","last_active":"2023-11-25T04:22:48.047444+0000","last_peered":"2023-11-25T04:22:48.047444+0000","last_clean":"2023-11-25T04:22:48.047444+0000","last_became_active":"2023-11-25T04:22:46.037141+0000","last_became_peered":"2023-11-25T04:22:46.037141+0000","last_unstale":"2023-11-25T04:22:48.047444+0000","last_undegraded":"2023-11-25T04:22:48.047444+0000","last_fullsized":"2023-11-25T04:22:48.047444+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:40:42.500663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048182+0000","last_change":"2023-11-25T04:22:47.038454+0000","last_active":"2023-11-25T04:22:48.048182+0000","last_peered":"2023-11-25T04:22:48.048182+0000","last_clean":"2023-11-25T04:22:48.048182+0000","last_became_active":"2023-11-25T04:22:47.038046+0000","last_became_peered":"2023-11-25T04:22:47.038046+0000","last_unstale":"2023-11-25T04:22:48.048182+0000","last_undegraded":"2023-11-25T04:22:48.048182+0000","last_fullsized":"2023-11-25T04:22:48.048182+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:38:38.839941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-25T04:22:47.031534+0000","last_change":"2023-11-25T04:22:41.999424+0000","last_active":"2023-11-25T04:22:47.031534+0000","last_peered":"2023-11-25T04:22:47.031534+0000","last_clean":"2023-11-25T04:22:47.031534+0000","last_became_active":"2023-11-25T04:22:41.999179+0000","last_became_peered":"2023-11-25T04:22:41.999179+0000","last_unstale":"2023-11-25T04:22:47.031534+0000","last_undegraded":"2023-11-25T04:22:47.031534+0000","last_fullsized":"2023-11-25T04:22:47.031534+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:12:04.291696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048523+0000","last_change":"2023-11-25T04:22:46.038758+0000","last_active":"2023-11-25T04:22:48.048523+0000","last_peered":"2023-11-25T04:22:48.048523+0000","last_clean":"2023-11-25T04:22:48.048523+0000","last_became_active":"2023-11-25T04:22:46.038470+0000","last_became_peered":"2023-11-25T04:22:46.038470+0000","last_unstale":"2023-11-25T04:22:48.048523+0000","last_undegraded":"2023-11-25T04:22:48.048523+0000","last_fullsized":"2023-11-25T04:22:48.048523+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:49:01.490558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048075+0000","last_change":"2023-11-25T04:22:47.037355+0000","last_active":"2023-11-25T04:22:48.048075+0000","last_peered":"2023-11-25T04:22:48.048075+0000","last_clean":"2023-11-25T04:22:48.048075+0000","last_became_active":"2023-11-25T04:22:47.036924+0000","last_became_peered":"2023-11-25T04:22:47.036924+0000","last_unstale":"2023-11-25T04:22:48.048075+0000","last_undegraded":"2023-11-25T04:22:48.048075+0000","last_fullsized":"2023-11-25T04:22:48.048075+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:15:39.511043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048677+0000","last_change":"2023-11-25T04:22:46.039025+0000","last_active":"2023-11-25T04:22:48.048677+0000","last_peered":"2023-11-25T04:22:48.048677+0000","last_clean":"2023-11-25T04:22:48.048677+0000","last_became_active":"2023-11-25T04:22:46.038762+0000","last_became_peered":"2023-11-25T04:22:46.038762+0000","last_unstale":"2023-11-25T04:22:48.048677+0000","last_undegraded":"2023-11-25T04:22:48.048677+0000","last_fullsized":"2023-11-25T04:22:48.048677+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:44:47.544509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-25T04:22:47.029103+0000","last_change":"2023-11-25T04:22:42.000204+0000","last_active":"2023-11-25T04:22:47.029103+0000","last_peered":"2023-11-25T04:22:47.029103+0000","last_clean":"2023-11-25T04:22:47.029103+0000","last_became_active":"2023-11-25T04:22:41.999915+0000","last_became_peered":"2023-11-25T04:22:41.999915+0000","last_unstale":"2023-11-25T04:22:47.029103+0000","last_undegraded":"2023-11-25T04:22:47.029103+0000","last_fullsized":"2023-11-25T04:22:47.029103+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:07:32.457905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047409+0000","last_change":"2023-11-25T04:22:47.038119+0000","last_active":"2023-11-25T04:22:48.047409+0000","last_peered":"2023-11-25T04:22:48.047409+0000","last_clean":"2023-11-25T04:22:48.047409+0000","last_became_active":"2023-11-25T04:22:47.037732+0000","last_became_peered":"2023-11-25T04:22:47.037732+0000","last_unstale":"2023-11-25T04:22:48.047409+0000","last_undegraded":"2023-11-25T04:22:48.047409+0000","last_fullsized":"2023-11-25T04:22:48.047409+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:49:02.500022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048004+0000","last_change":"2023-11-25T04:22:46.035885+0000","last_active":"2023-11-25T04:22:48.048004+0000","last_peered":"2023-11-25T04:22:48.048004+0000","last_clean":"2023-11-25T04:22:48.048004+0000","last_became_active":"2023-11-25T04:22:46.035405+0000","last_became_peered":"2023-11-25T04:22:46.035405+0000","last_unstale":"2023-11-25T04:22:48.048004+0000","last_undegraded":"2023-11-25T04:22:48.048004+0000","last_fullsized":"2023-11-25T04:22:48.048004+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:53:47.398746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-25T04:22:47.032524+0000","last_change":"2023-11-25T04:22:41.999067+0000","last_active":"2023-11-25T04:22:47.032524+0000","last_peered":"2023-11-25T04:22:47.032524+0000","last_clean":"2023-11-25T04:22:47.032524+0000","last_became_active":"2023-11-25T04:22:41.998798+0000","last_became_peered":"2023-11-25T04:22:41.998798+0000","last_unstale":"2023-11-25T04:22:47.032524+0000","last_undegraded":"2023-11-25T04:22:47.032524+0000","last_fullsized":"2023-11-25T04:22:47.032524+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:00:04.266051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047617+0000","last_change":"2023-11-25T04:22:47.043464+0000","last_active":"2023-11-25T04:22:48.047617+0000","last_peered":"2023-11-25T04:22:48.047617+0000","last_clean":"2023-11-25T04:22:48.047617+0000","last_became_active":"2023-11-25T04:22:47.043050+0000","last_became_peered":"2023-11-25T04:22:47.043050+0000","last_unstale":"2023-11-25T04:22:48.047617+0000","last_undegraded":"2023-11-25T04:22:48.047617+0000","last_fullsized":"2023-11-25T04:22:48.047617+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:59:45.492077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"18'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048417+0000","last_change":"2023-11-25T04:22:46.034606+0000","last_active":"2023-11-25T04:22:48.048417+0000","last_peered":"2023-11-25T04:22:48.048417+0000","last_clean":"2023-11-25T04:22:48.048417+0000","last_became_active":"2023-11-25T04:22:46.034229+0000","last_became_peered":"2023-11-25T04:22:46.034229+0000","last_unstale":"2023-11-25T04:22:48.048417+0000","last_undegraded":"2023-11-25T04:22:48.048417+0000","last_fullsized":"2023-11-25T04:22:48.048417+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:21:42.146264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-25T04:22:47.031216+0000","last_change":"2023-11-25T04:22:42.000258+0000","last_active":"2023-11-25T04:22:47.031216+0000","last_peered":"2023-11-25T04:22:47.031216+0000","last_clean":"2023-11-25T04:22:47.031216+0000","last_became_active":"2023-11-25T04:22:41.999992+0000","last_became_peered":"2023-11-25T04:22:41.999992+0000","last_unstale":"2023-11-25T04:22:47.031216+0000","last_undegraded":"2023-11-25T04:22:47.031216+0000","last_fullsized":"2023-11-25T04:22:47.031216+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:31:13.715992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047510+0000","last_change":"2023-11-25T04:22:47.042860+0000","last_active":"2023-11-25T04:22:48.047510+0000","last_peered":"2023-11-25T04:22:48.047510+0000","last_clean":"2023-11-25T04:22:48.047510+0000","last_became_active":"2023-11-25T04:22:47.042472+0000","last_became_peered":"2023-11-25T04:22:47.042472+0000","last_unstale":"2023-11-25T04:22:48.047510+0000","last_undegraded":"2023-11-25T04:22:48.047510+0000","last_fullsized":"2023-11-25T04:22:48.047510+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:44:48.553972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.894735+0000","last_change":"2023-11-25T04:22:46.038417+0000","last_active":"2023-11-25T04:22:48.894735+0000","last_peered":"2023-11-25T04:22:48.894735+0000","last_clean":"2023-11-25T04:22:48.894735+0000","last_became_active":"2023-11-25T04:22:46.037513+0000","last_became_peered":"2023-11-25T04:22:46.037513+0000","last_unstale":"2023-11-25T04:22:48.894735+0000","last_undegraded":"2023-11-25T04:22:48.894735+0000","last_fullsized":"2023-11-25T04:22:48.894735+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:58:14.877413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-25T04:22:47.030950+0000","last_change":"2023-11-25T04:22:42.000217+0000","last_active":"2023-11-25T04:22:47.030950+0000","last_peered":"2023-11-25T04:22:47.030950+0000","last_clean":"2023-11-25T04:22:47.030950+0000","last_became_active":"2023-11-25T04:22:41.999987+0000","last_became_peered":"2023-11-25T04:22:41.999987+0000","last_unstale":"2023-11-25T04:22:47.030950+0000","last_undegraded":"2023-11-25T04:22:47.030950+0000","last_fullsized":"2023-11-25T04:22:47.030950+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:11:51.100459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047385+0000","last_change":"2023-11-25T04:22:47.036099+0000","last_active":"2023-11-25T04:22:48.047385+0000","last_peered":"2023-11-25T04:22:48.047385+0000","last_clean":"2023-11-25T04:22:48.047385+0000","last_became_active":"2023-11-25T04:22:47.035581+0000","last_became_peered":"2023-11-25T04:22:47.035581+0000","last_unstale":"2023-11-25T04:22:48.047385+0000","last_undegraded":"2023-11-25T04:22:48.047385+0000","last_fullsized":"2023-11-25T04:22:48.047385+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:21:43.155728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048171+0000","last_change":"2023-11-25T04:22:47.040343+0000","last_active":"2023-11-25T04:22:48.048171+0000","last_peered":"2023-11-25T04:22:48.048171+0000","last_clean":"2023-11-25T04:22:48.048171+0000","last_became_active":"2023-11-25T04:22:47.039151+0000","last_became_peered":"2023-11-25T04:22:47.039151+0000","last_unstale":"2023-11-25T04:22:48.048171+0000","last_undegraded":"2023-11-25T04:22:48.048171+0000","last_fullsized":"2023-11-25T04:22:48.048171+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:21:15.981901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"18'6","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048260+0000","last_change":"2023-11-25T04:22:46.040005+0000","last_active":"2023-11-25T04:22:48.048260+0000","last_peered":"2023-11-25T04:22:48.048260+0000","last_clean":"2023-11-25T04:22:48.048260+0000","last_became_active":"2023-11-25T04:22:46.039655+0000","last_became_peered":"2023-11-25T04:22:46.039655+0000","last_unstale":"2023-11-25T04:22:48.048260+0000","last_undegraded":"2023-11-25T04:22:48.048260+0000","last_fullsized":"2023-11-25T04:22:48.048260+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:41:15.662804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047237+0000","last_change":"2023-11-25T04:22:47.037181+0000","last_active":"2023-11-25T04:22:48.047237+0000","last_peered":"2023-11-25T04:22:48.047237+0000","last_clean":"2023-11-25T04:22:48.047237+0000","last_became_active":"2023-11-25T04:22:47.036821+0000","last_became_peered":"2023-11-25T04:22:47.036821+0000","last_unstale":"2023-11-25T04:22:48.047237+0000","last_undegraded":"2023-11-25T04:22:48.047237+0000","last_fullsized":"2023-11-25T04:22:48.047237+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:12:51.990448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"18'3","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048375+0000","last_change":"2023-11-25T04:22:46.035816+0000","last_active":"2023-11-25T04:22:48.048375+0000","last_peered":"2023-11-25T04:22:48.048375+0000","last_clean":"2023-11-25T04:22:48.048375+0000","last_became_active":"2023-11-25T04:22:46.035461+0000","last_became_peered":"2023-11-25T04:22:46.035461+0000","last_unstale":"2023-11-25T04:22:48.048375+0000","last_undegraded":"2023-11-25T04:22:48.048375+0000","last_fullsized":"2023-11-25T04:22:48.048375+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:58:26.155063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048370+0000","last_change":"2023-11-25T04:22:46.040458+0000","last_active":"2023-11-25T04:22:48.048370+0000","last_peered":"2023-11-25T04:22:48.048370+0000","last_clean":"2023-11-25T04:22:48.048370+0000","last_became_active":"2023-11-25T04:22:46.040143+0000","last_became_peered":"2023-11-25T04:22:46.040143+0000","last_unstale":"2023-11-25T04:22:48.048370+0000","last_undegraded":"2023-11-25T04:22:48.048370+0000","last_fullsized":"2023-11-25T04:22:48.048370+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:27:31.901618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047387+0000","last_change":"2023-11-25T04:22:47.040346+0000","last_active":"2023-11-25T04:22:48.047387+0000","last_peered":"2023-11-25T04:22:48.047387+0000","last_clean":"2023-11-25T04:22:48.047387+0000","last_became_active":"2023-11-25T04:22:47.040084+0000","last_became_peered":"2023-11-25T04:22:47.040084+0000","last_unstale":"2023-11-25T04:22:48.047387+0000","last_undegraded":"2023-11-25T04:22:48.047387+0000","last_fullsized":"2023-11-25T04:22:48.047387+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:59:45.492077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048308+0000","last_change":"2023-11-25T04:22:46.038721+0000","last_active":"2023-11-25T04:22:48.048308+0000","last_peered":"2023-11-25T04:22:48.048308+0000","last_clean":"2023-11-25T04:22:48.048308+0000","last_became_active":"2023-11-25T04:22:46.037698+0000","last_became_peered":"2023-11-25T04:22:46.037698+0000","last_unstale":"2023-11-25T04:22:48.048308+0000","last_undegraded":"2023-11-25T04:22:48.048308+0000","last_fullsized":"2023-11-25T04:22:48.048308+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:25:20.721529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047199+0000","last_change":"2023-11-25T04:22:47.037790+0000","last_active":"2023-11-25T04:22:48.047199+0000","last_peered":"2023-11-25T04:22:48.047199+0000","last_clean":"2023-11-25T04:22:48.047199+0000","last_became_active":"2023-11-25T04:22:47.037149+0000","last_became_peered":"2023-11-25T04:22:47.037149+0000","last_unstale":"2023-11-25T04:22:48.047199+0000","last_undegraded":"2023-11-25T04:22:48.047199+0000","last_fullsized":"2023-11-25T04:22:48.047199+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:31:54.857270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048584+0000","last_change":"2023-11-25T04:22:47.036079+0000","last_active":"2023-11-25T04:22:48.048584+0000","last_peered":"2023-11-25T04:22:48.048584+0000","last_clean":"2023-11-25T04:22:48.048584+0000","last_became_active":"2023-11-25T04:22:47.035625+0000","last_became_peered":"2023-11-25T04:22:47.035625+0000","last_unstale":"2023-11-25T04:22:48.048584+0000","last_undegraded":"2023-11-25T04:22:48.048584+0000","last_fullsized":"2023-11-25T04:22:48.048584+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:21:15.981901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047749+0000","last_change":"2023-11-25T04:22:46.036085+0000","last_active":"2023-11-25T04:22:48.047749+0000","last_peered":"2023-11-25T04:22:48.047749+0000","last_clean":"2023-11-25T04:22:48.047749+0000","last_became_active":"2023-11-25T04:22:46.035495+0000","last_became_peered":"2023-11-25T04:22:46.035495+0000","last_unstale":"2023-11-25T04:22:48.047749+0000","last_undegraded":"2023-11-25T04:22:48.047749+0000","last_fullsized":"2023-11-25T04:22:48.047749+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:14:56.735365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048251+0000","last_change":"2023-11-25T04:22:47.043064+0000","last_active":"2023-11-25T04:22:48.048251+0000","last_peered":"2023-11-25T04:22:48.048251+0000","last_clean":"2023-11-25T04:22:48.048251+0000","last_became_active":"2023-11-25T04:22:47.042760+0000","last_became_peered":"2023-11-25T04:22:47.042760+0000","last_unstale":"2023-11-25T04:22:48.048251+0000","last_undegraded":"2023-11-25T04:22:48.048251+0000","last_fullsized":"2023-11-25T04:22:48.048251+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:37:30.959147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047514+0000","last_change":"2023-11-25T04:22:46.038298+0000","last_active":"2023-11-25T04:22:48.047514+0000","last_peered":"2023-11-25T04:22:48.047514+0000","last_clean":"2023-11-25T04:22:48.047514+0000","last_became_active":"2023-11-25T04:22:46.036629+0000","last_became_peered":"2023-11-25T04:22:46.036629+0000","last_unstale":"2023-11-25T04:22:48.047514+0000","last_undegraded":"2023-11-25T04:22:48.047514+0000","last_fullsized":"2023-11-25T04:22:48.047514+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:45:14.670326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048209+0000","last_change":"2023-11-25T04:22:47.043156+0000","last_active":"2023-11-25T04:22:48.048209+0000","last_peered":"2023-11-25T04:22:48.048209+0000","last_clean":"2023-11-25T04:22:48.048209+0000","last_became_active":"2023-11-25T04:22:47.042693+0000","last_became_peered":"2023-11-25T04:22:47.042693+0000","last_unstale":"2023-11-25T04:22:48.048209+0000","last_undegraded":"2023-11-25T04:22:48.048209+0000","last_fullsized":"2023-11-25T04:22:48.048209+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:38:38.839941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1a","version":"18'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047866+0000","last_change":"2023-11-25T04:22:46.034340+0000","last_active":"2023-11-25T04:22:48.047866+0000","last_peered":"2023-11-25T04:22:48.047866+0000","last_clean":"2023-11-25T04:22:48.047866+0000","last_became_active":"2023-11-25T04:22:46.033944+0000","last_became_peered":"2023-11-25T04:22:46.033944+0000","last_unstale":"2023-11-25T04:22:48.047866+0000","last_undegraded":"2023-11-25T04:22:48.047866+0000","last_fullsized":"2023-11-25T04:22:48.047866+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:59:30.291004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047694+0000","last_change":"2023-11-25T04:22:47.039547+0000","last_active":"2023-11-25T04:22:48.047694+0000","last_peered":"2023-11-25T04:22:48.047694+0000","last_clean":"2023-11-25T04:22:48.047694+0000","last_became_active":"2023-11-25T04:22:47.039218+0000","last_became_peered":"2023-11-25T04:22:47.039218+0000","last_unstale":"2023-11-25T04:22:48.047694+0000","last_undegraded":"2023-11-25T04:22:48.047694+0000","last_fullsized":"2023-11-25T04:22:48.047694+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:44:48.553972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"18'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047860+0000","last_change":"2023-11-25T04:22:46.035882+0000","last_active":"2023-11-25T04:22:48.047860+0000","last_peered":"2023-11-25T04:22:48.047860+0000","last_clean":"2023-11-25T04:22:48.047860+0000","last_became_active":"2023-11-25T04:22:46.035588+0000","last_became_peered":"2023-11-25T04:22:46.035588+0000","last_unstale":"2023-11-25T04:22:48.047860+0000","last_undegraded":"2023-11-25T04:22:48.047860+0000","last_fullsized":"2023-11-25T04:22:48.047860+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:43:10.702849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047831+0000","last_change":"2023-11-25T04:22:47.041267+0000","last_active":"2023-11-25T04:22:48.047831+0000","last_peered":"2023-11-25T04:22:48.047831+0000","last_clean":"2023-11-25T04:22:48.047831+0000","last_became_active":"2023-11-25T04:22:47.040408+0000","last_became_peered":"2023-11-25T04:22:47.040408+0000","last_unstale":"2023-11-25T04:22:48.047831+0000","last_undegraded":"2023-11-25T04:22:48.047831+0000","last_fullsized":"2023-11-25T04:22:48.047831+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:37:30.959147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"18'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048131+0000","last_change":"2023-11-25T04:22:46.040793+0000","last_active":"2023-11-25T04:22:48.048131+0000","last_peered":"2023-11-25T04:22:48.048131+0000","last_clean":"2023-11-25T04:22:48.048131+0000","last_became_active":"2023-11-25T04:22:46.040493+0000","last_became_peered":"2023-11-25T04:22:46.040493+0000","last_unstale":"2023-11-25T04:22:48.048131+0000","last_undegraded":"2023-11-25T04:22:48.048131+0000","last_fullsized":"2023-11-25T04:22:48.048131+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:58:26.155063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048056+0000","last_change":"2023-11-25T04:22:47.040067+0000","last_active":"2023-11-25T04:22:48.048056+0000","last_peered":"2023-11-25T04:22:48.048056+0000","last_clean":"2023-11-25T04:22:48.048056+0000","last_became_active":"2023-11-25T04:22:47.038425+0000","last_became_peered":"2023-11-25T04:22:47.038425+0000","last_unstale":"2023-11-25T04:22:48.048056+0000","last_undegraded":"2023-11-25T04:22:48.048056+0000","last_fullsized":"2023-11-25T04:22:48.048056+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:25:06.132517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"18'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048297+0000","last_change":"2023-11-25T04:22:46.035872+0000","last_active":"2023-11-25T04:22:48.048297+0000","last_peered":"2023-11-25T04:22:48.048297+0000","last_clean":"2023-11-25T04:22:48.048297+0000","last_became_active":"2023-11-25T04:22:46.035575+0000","last_became_peered":"2023-11-25T04:22:46.035575+0000","last_unstale":"2023-11-25T04:22:48.048297+0000","last_undegraded":"2023-11-25T04:22:48.048297+0000","last_fullsized":"2023-11-25T04:22:48.048297+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:58:14.877413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.047725+0000","last_change":"2023-11-25T04:22:47.036764+0000","last_active":"2023-11-25T04:22:48.047725+0000","last_peered":"2023-11-25T04:22:48.047725+0000","last_clean":"2023-11-25T04:22:48.047725+0000","last_became_active":"2023-11-25T04:22:47.036406+0000","last_became_peered":"2023-11-25T04:22:47.036406+0000","last_unstale":"2023-11-25T04:22:48.047725+0000","last_undegraded":"2023-11-25T04:22:48.047725+0000","last_fullsized":"2023-11-25T04:22:48.047725+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:09:55.357642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1e","version":"18'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048335+0000","last_change":"2023-11-25T04:22:46.035659+0000","last_active":"2023-11-25T04:22:48.048335+0000","last_peered":"2023-11-25T04:22:48.048335+0000","last_clean":"2023-11-25T04:22:48.048335+0000","last_became_active":"2023-11-25T04:22:46.035113+0000","last_became_peered":"2023-11-25T04:22:46.035113+0000","last_unstale":"2023-11-25T04:22:48.048335+0000","last_undegraded":"2023-11-25T04:22:48.048335+0000","last_fullsized":"2023-11-25T04:22:48.048335+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:26:25.338117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048474+0000","last_change":"2023-11-25T04:22:47.041554+0000","last_active":"2023-11-25T04:22:48.048474+0000","last_peered":"2023-11-25T04:22:48.048474+0000","last_clean":"2023-11-25T04:22:48.048474+0000","last_became_active":"2023-11-25T04:22:47.040314+0000","last_became_peered":"2023-11-25T04:22:47.040314+0000","last_unstale":"2023-11-25T04:22:48.048474+0000","last_undegraded":"2023-11-25T04:22:48.048474+0000","last_fullsized":"2023-11-25T04:22:48.048474+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:24:56.315251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"18'4","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-25T04:22:48.048204+0000","last_change":"2023-11-25T04:22:46.038702+0000","last_active":"2023-11-25T04:22:48.048204+0000","last_peered":"2023-11-25T04:22:48.048204+0000","last_clean":"2023-11-25T04:22:48.048204+0000","last_became_active":"2023-11-25T04:22:46.037567+0000","last_became_peered":"2023-11-25T04:22:46.037567+0000","last_unstale":"2023-11-25T04:22:48.048204+0000","last_undegraded":"2023-11-25T04:22:48.048204+0000","last_fullsized":"2023-11-25T04:22:48.048204+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:58:26.155063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738377,"num_pgs":72,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35508,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336332,"statfs":{"total":96636764160,"available":96600403968,"internally_reserved":0,"allocated":172032,"data_stored":58325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]}]},{"osd":1,"up_from":8,"seq":34359738377,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31944,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339896,"statfs":{"total":96636764160,"available":96604053504,"internally_reserved":0,"allocated":782336,"data_stored":649871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738376,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31952,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339888,"statfs":{"total":96636764160,"available":96604045312,"internally_reserved":0,"allocated":790528,"data_stored":650487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-25T04:23:03.039 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-25T04:23:03.040 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2023-11-25T04:23:03.370 INFO:teuthology.orchestra.run.smithi067.stdout:epoch 19 2023-11-25T04:23:03.370 INFO:teuthology.orchestra.run.smithi067.stdout:fsid a677f511-1143-4e76-8da6-5ddfadaff077 2023-11-25T04:23:03.370 INFO:teuthology.orchestra.run.smithi067.stdout:created 2023-11-25T04:22:25.140474+0000 2023-11-25T04:23:03.370 INFO:teuthology.orchestra.run.smithi067.stdout:modified 2023-11-25T04:22:48.032728+0000 2023-11-25T04:23:03.370 INFO:teuthology.orchestra.run.smithi067.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-11-25T04:23:03.370 INFO:teuthology.orchestra.run.smithi067.stdout:crush_version 4 2023-11-25T04:23:03.370 INFO:teuthology.orchestra.run.smithi067.stdout:full_ratio 0.95 2023-11-25T04:23:03.370 INFO:teuthology.orchestra.run.smithi067.stdout:backfillfull_ratio 0.9 2023-11-25T04:23:03.370 INFO:teuthology.orchestra.run.smithi067.stdout:nearfull_ratio 0.85 2023-11-25T04:23:03.371 INFO:teuthology.orchestra.run.smithi067.stdout:require_min_compat_client luminous 2023-11-25T04:23:03.371 INFO:teuthology.orchestra.run.smithi067.stdout:min_compat_client jewel 2023-11-25T04:23:03.371 INFO:teuthology.orchestra.run.smithi067.stdout:require_osd_release reef 2023-11-25T04:23:03.371 INFO:teuthology.orchestra.run.smithi067.stdout:stretch_mode_enabled false 2023-11-25T04:23:03.371 INFO:teuthology.orchestra.run.smithi067.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 12 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 2.99 2023-11-25T04:23:03.371 INFO:teuthology.orchestra.run.smithi067.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 15 flags hashpspool stripe_width 0 application rbd read_balance_score 1.50 2023-11-25T04:23:03.371 INFO:teuthology.orchestra.run.smithi067.stdout:pool 3 'cephfs_metadata' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 18 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.13 2023-11-25T04:23:03.371 INFO:teuthology.orchestra.run.smithi067.stdout:pool 4 'cephfs_data' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 19 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.13 2023-11-25T04:23:03.371 INFO:teuthology.orchestra.run.smithi067.stdout:max_osd 3 2023-11-25T04:23:03.371 INFO:teuthology.orchestra.run.smithi067.stdout:osd.0 up in weight 1 up_from 8 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:172.21.15.76:6800/181847435,v1:172.21.15.76:6801/181847435] [v2:172.21.15.76:6802/181847435,v1:172.21.15.76:6803/181847435] exists,up bc397d35-53ea-430c-a96a-63a4160e64c3 2023-11-25T04:23:03.371 INFO:teuthology.orchestra.run.smithi067.stdout:osd.1 up in weight 1 up_from 8 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6800/3130474108,v1:172.21.15.92:6801/3130474108] [v2:172.21.15.92:6802/3130474108,v1:172.21.15.92:6803/3130474108] exists,up d4f1b88a-bbd4-4e58-9519-569489ced7b2 2023-11-25T04:23:03.371 INFO:teuthology.orchestra.run.smithi067.stdout:osd.2 up in weight 1 up_from 8 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:172.21.15.195:6800/4099978417,v1:172.21.15.195:6801/4099978417] [v2:172.21.15.195:6802/4099978417,v1:172.21.15.195:6803/4099978417] exists,up 55c82801-e090-4c10-91e5-5fa9dbfba36d 2023-11-25T04:23:03.382 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 8 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:172.21.15.76:6800/181847435,v1:172.21.15.76:6801/181847435] [v2:172.21.15.76:6802/181847435,v1:172.21.15.76:6803/181847435] exists,up bc397d35-53ea-430c-a96a-63a4160e64c3', 'osd.1 up in weight 1 up_from 8 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6800/3130474108,v1:172.21.15.92:6801/3130474108] [v2:172.21.15.92:6802/3130474108,v1:172.21.15.92:6803/3130474108] exists,up d4f1b88a-bbd4-4e58-9519-569489ced7b2', 'osd.2 up in weight 1 up_from 8 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:172.21.15.195:6800/4099978417,v1:172.21.15.195:6801/4099978417] [v2:172.21.15.195:6802/4099978417,v1:172.21.15.195:6803/4099978417] exists,up 55c82801-e090-4c10-91e5-5fa9dbfba36d'] 2023-11-25T04:23:03.383 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2023-11-25T04:23:03.489 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2023-11-25T04:23:03.489 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mon.a", 2023-11-25T04:23:03.489 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster": "ceph", 2023-11-25T04:23:03.489 INFO:teuthology.orchestra.run.smithi067.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2023-11-25T04:23:03.490 INFO:teuthology.orchestra.run.smithi067.stdout: "admin_socket_mode": "", 2023-11-25T04:23:03.490 INFO:teuthology.orchestra.run.smithi067.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2023-11-25T04:23:03.490 INFO:teuthology.orchestra.run.smithi067.stdout: "auth_client_required": "cephx, none", 2023-11-25T04:23:03.490 INFO:teuthology.orchestra.run.smithi067.stdout: "auth_cluster_required": "cephx", 2023-11-25T04:23:03.490 INFO:teuthology.orchestra.run.smithi067.stdout: "auth_debug": "true", 2023-11-25T04:23:03.490 INFO:teuthology.orchestra.run.smithi067.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2023-11-25T04:23:03.490 INFO:teuthology.orchestra.run.smithi067.stdout: "auth_mon_ticket_ttl": "660.000000", 2023-11-25T04:23:03.490 INFO:teuthology.orchestra.run.smithi067.stdout: "auth_service_required": "cephx", 2023-11-25T04:23:03.490 INFO:teuthology.orchestra.run.smithi067.stdout: "auth_service_ticket_ttl": "240.000000", 2023-11-25T04:23:03.490 INFO:teuthology.orchestra.run.smithi067.stdout: "auth_supported": "cephx", 2023-11-25T04:23:03.490 INFO:teuthology.orchestra.run.smithi067.stdout: "bdev_aio": "true", 2023-11-25T04:23:03.491 INFO:teuthology.orchestra.run.smithi067.stdout: "bdev_aio_max_queue_depth": "1024", 2023-11-25T04:23:03.491 INFO:teuthology.orchestra.run.smithi067.stdout: "bdev_aio_poll_ms": "250", 2023-11-25T04:23:03.491 INFO:teuthology.orchestra.run.smithi067.stdout: "bdev_aio_reap_max": "16", 2023-11-25T04:23:03.491 INFO:teuthology.orchestra.run.smithi067.stdout: "bdev_async_discard": "false", 2023-11-25T04:23:03.491 INFO:teuthology.orchestra.run.smithi067.stdout: "bdev_block_size": "4096", 2023-11-25T04:23:03.491 INFO:teuthology.orchestra.run.smithi067.stdout: "bdev_debug_aio": "false", 2023-11-25T04:23:03.491 INFO:teuthology.orchestra.run.smithi067.stdout: "bdev_debug_aio_log_age": "5.000000", 2023-11-25T04:23:03.491 INFO:teuthology.orchestra.run.smithi067.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2023-11-25T04:23:03.491 INFO:teuthology.orchestra.run.smithi067.stdout: "bdev_debug_inflight_ios": "false", 2023-11-25T04:23:03.491 INFO:teuthology.orchestra.run.smithi067.stdout: "bdev_enable_discard": "false", 2023-11-25T04:23:03.491 INFO:teuthology.orchestra.run.smithi067.stdout: "bdev_flock_retry": "3", 2023-11-25T04:23:03.491 INFO:teuthology.orchestra.run.smithi067.stdout: "bdev_flock_retry_interval": "0.100000", 2023-11-25T04:23:03.491 INFO:teuthology.orchestra.run.smithi067.stdout: "bdev_inject_crash": "0", 2023-11-25T04:23:03.492 INFO:teuthology.orchestra.run.smithi067.stdout: "bdev_inject_crash_flush_delay": "2", 2023-11-25T04:23:03.492 INFO:teuthology.orchestra.run.smithi067.stdout: "bdev_ioring": "false", 2023-11-25T04:23:03.492 INFO:teuthology.orchestra.run.smithi067.stdout: "bdev_ioring_hipri": "false", 2023-11-25T04:23:03.492 INFO:teuthology.orchestra.run.smithi067.stdout: "bdev_ioring_sqthread_poll": "false", 2023-11-25T04:23:03.492 INFO:teuthology.orchestra.run.smithi067.stdout: "bdev_nvme_unbind_from_kernel": "false", 2023-11-25T04:23:03.492 INFO:teuthology.orchestra.run.smithi067.stdout: "bdev_read_buffer_alignment": "4096", 2023-11-25T04:23:03.492 INFO:teuthology.orchestra.run.smithi067.stdout: "bdev_read_preallocated_huge_buffers": "", 2023-11-25T04:23:03.492 INFO:teuthology.orchestra.run.smithi067.stdout: "bdev_type": "", 2023-11-25T04:23:03.492 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_alloc_size": "1048576", 2023-11-25T04:23:03.492 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_allocator": "hybrid", 2023-11-25T04:23:03.492 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_buffered_io": "true", 2023-11-25T04:23:03.492 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_check_for_zeros": "false", 2023-11-25T04:23:03.493 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_check_volume_selector_often": "false", 2023-11-25T04:23:03.493 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_check_volume_selector_on_umount": "false", 2023-11-25T04:23:03.493 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_compact_log_sync": "false", 2023-11-25T04:23:03.493 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2023-11-25T04:23:03.493 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2023-11-25T04:23:03.493 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_log_compact_min_size": "16777216", 2023-11-25T04:23:03.493 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_log_replay_check_allocations": "true", 2023-11-25T04:23:03.493 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_max_log_runway": "4194304", 2023-11-25T04:23:03.493 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_max_prefetch": "1048576", 2023-11-25T04:23:03.493 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_min_flush_size": "524288", 2023-11-25T04:23:03.493 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_min_log_runway": "1048576", 2023-11-25T04:23:03.493 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_replay_recovery": "false", 2023-11-25T04:23:03.494 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_replay_recovery_disable_compact": "false", 2023-11-25T04:23:03.494 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_shared_alloc_size": "65536", 2023-11-25T04:23:03.494 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_sync_write": "false", 2023-11-25T04:23:03.494 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2023-11-25T04:23:03.494 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2023-11-25T04:23:03.494 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2023-11-25T04:23:03.494 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_allocation_from_file": "true", 2023-11-25T04:23:03.494 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_allocator": "hybrid", 2023-11-25T04:23:03.494 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2023-11-25T04:23:03.494 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2023-11-25T04:23:03.494 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2023-11-25T04:23:03.494 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2023-11-25T04:23:03.495 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2023-11-25T04:23:03.495 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_bitmapallocator_span_size": "1024", 2023-11-25T04:23:03.495 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_blobid_prealloc": "10240", 2023-11-25T04:23:03.495 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_block_create": "true", 2023-11-25T04:23:03.495 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_block_db_create": "false", 2023-11-25T04:23:03.495 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_block_db_path": "", 2023-11-25T04:23:03.495 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_block_db_size": "0", 2023-11-25T04:23:03.495 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_block_path": "", 2023-11-25T04:23:03.495 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_block_preallocate_file": "false", 2023-11-25T04:23:03.495 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_block_size": "107374182400", 2023-11-25T04:23:03.495 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_block_wal_create": "false", 2023-11-25T04:23:03.495 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_block_wal_path": "", 2023-11-25T04:23:03.496 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_block_wal_size": "100663296", 2023-11-25T04:23:03.496 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_bluefs": "true", 2023-11-25T04:23:03.496 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2023-11-25T04:23:03.496 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_bluefs_env_mirror": "false", 2023-11-25T04:23:03.496 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_bluefs_max_free": "10737418240", 2023-11-25T04:23:03.496 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2023-11-25T04:23:03.496 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2023-11-25T04:23:03.496 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2023-11-25T04:23:03.496 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2023-11-25T04:23:03.496 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2023-11-25T04:23:03.496 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_autotune": "true", 2023-11-25T04:23:03.496 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_autotune_interval": "5.000000", 2023-11-25T04:23:03.496 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2023-11-25T04:23:03.497 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_kv_ratio": "0.450000", 2023-11-25T04:23:03.497 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_meta_ratio": "0.450000", 2023-11-25T04:23:03.497 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_size": "0", 2023-11-25T04:23:03.497 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_size_hdd": "1073741824", 2023-11-25T04:23:03.497 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_size_ssd": "3221225472", 2023-11-25T04:23:03.497 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_trim_interval": "0.050000", 2023-11-25T04:23:03.497 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2023-11-25T04:23:03.497 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_type": "2q", 2023-11-25T04:23:03.497 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2023-11-25T04:23:03.497 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_clone_cow": "true", 2023-11-25T04:23:03.497 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_compression_algorithm": "snappy", 2023-11-25T04:23:03.497 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_compression_max_blob_size": "0", 2023-11-25T04:23:03.498 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2023-11-25T04:23:03.498 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2023-11-25T04:23:03.498 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_compression_min_blob_size": "0", 2023-11-25T04:23:03.498 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2023-11-25T04:23:03.498 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2023-11-25T04:23:03.498 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_compression_mode": "none", 2023-11-25T04:23:03.498 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_compression_required_ratio": "0.875000", 2023-11-25T04:23:03.498 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_csum_type": "crc32c", 2023-11-25T04:23:03.498 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_debug_enforce_settings": "default", 2023-11-25T04:23:03.498 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_debug_freelist": "false", 2023-11-25T04:23:03.498 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_debug_fsck_abort": "false", 2023-11-25T04:23:03.498 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2023-11-25T04:23:03.498 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2023-11-25T04:23:03.499 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_debug_inject_read_err": "false", 2023-11-25T04:23:03.499 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_debug_legacy_omap": "false", 2023-11-25T04:23:03.499 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_debug_no_reuse_blocks": "false", 2023-11-25T04:23:03.499 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_debug_omit_block_device_write": "false", 2023-11-25T04:23:03.499 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_debug_omit_kv_commit": "false", 2023-11-25T04:23:03.499 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2023-11-25T04:23:03.499 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_debug_prefill": "0.000000", 2023-11-25T04:23:03.499 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_debug_prefragment_max": "1048576", 2023-11-25T04:23:03.499 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_debug_random_read_err": "0.000000", 2023-11-25T04:23:03.499 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2023-11-25T04:23:03.499 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_debug_small_allocations": "0", 2023-11-25T04:23:03.499 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2023-11-25T04:23:03.500 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_default_buffered_read": "true", 2023-11-25T04:23:03.500 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_default_buffered_write": "false", 2023-11-25T04:23:03.500 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_deferred_batch_ops": "0", 2023-11-25T04:23:03.500 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2023-11-25T04:23:03.500 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2023-11-25T04:23:03.500 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_elastic_shared_blobs": "true", 2023-11-25T04:23:03.500 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2023-11-25T04:23:03.500 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_extent_map_shard_max_size": "1200", 2023-11-25T04:23:03.500 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_extent_map_shard_min_size": "150", 2023-11-25T04:23:03.500 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_extent_map_shard_target_size": "500", 2023-11-25T04:23:03.500 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2023-11-25T04:23:03.500 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_fail_eio": "false", 2023-11-25T04:23:03.500 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_freelist_blocks_per_key": "128", 2023-11-25T04:23:03.501 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2023-11-25T04:23:03.501 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2023-11-25T04:23:03.501 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2023-11-25T04:23:03.501 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_fsck_on_mkfs": "true", 2023-11-25T04:23:03.501 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2023-11-25T04:23:03.501 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_fsck_on_mount": "false", 2023-11-25T04:23:03.501 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_fsck_on_mount_deep": "false", 2023-11-25T04:23:03.501 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_fsck_on_umount": "false", 2023-11-25T04:23:03.501 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_fsck_on_umount_deep": "false", 2023-11-25T04:23:03.501 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2023-11-25T04:23:03.501 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_fsck_quick_fix_threads": "2", 2023-11-25T04:23:03.501 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2023-11-25T04:23:03.502 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2023-11-25T04:23:03.502 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_gc_enable_blob_threshold": "0", 2023-11-25T04:23:03.502 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_gc_enable_total_threshold": "0", 2023-11-25T04:23:03.502 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2023-11-25T04:23:03.502 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_ignore_data_csum": "false", 2023-11-25T04:23:03.502 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2023-11-25T04:23:03.502 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_kvbackend": "rocksdb", 2023-11-25T04:23:03.502 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_log_collection_list_age": "60.000000", 2023-11-25T04:23:03.502 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2023-11-25T04:23:03.502 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_log_op_age": "5.000000", 2023-11-25T04:23:03.502 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_max_alloc_size": "0", 2023-11-25T04:23:03.503 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_max_blob_size": "0", 2023-11-25T04:23:03.503 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_max_blob_size_hdd": "65536", 2023-11-25T04:23:03.503 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_max_blob_size_ssd": "65536", 2023-11-25T04:23:03.503 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_max_defer_interval": "3.000000", 2023-11-25T04:23:03.503 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_max_deferred_txc": "32", 2023-11-25T04:23:03.503 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_min_alloc_size": "0", 2023-11-25T04:23:03.503 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_min_alloc_size_hdd": "4096", 2023-11-25T04:23:03.503 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_min_alloc_size_ssd": "4096", 2023-11-25T04:23:03.503 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_nid_prealloc": "1024", 2023-11-25T04:23:03.503 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_prefer_deferred_size": "0", 2023-11-25T04:23:03.503 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2023-11-25T04:23:03.503 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2023-11-25T04:23:03.504 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_qfsck_on_mount": "true", 2023-11-25T04:23:03.504 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_retry_disk_reads": "3", 2023-11-25T04:23:03.504 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_rocksdb_cf": "true", 2023-11-25T04:23:03.504 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2023-11-25T04:23:03.504 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2023-11-25T04:23:03.504 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_rocksdb_options_annex": "", 2023-11-25T04:23:03.504 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_spdk_coremask": "0x1", 2023-11-25T04:23:03.504 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_spdk_io_sleep": "5", 2023-11-25T04:23:03.504 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_spdk_max_io_completion": "0", 2023-11-25T04:23:03.504 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_spdk_mem": "512", 2023-11-25T04:23:03.504 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_sync_submit_transaction": "false", 2023-11-25T04:23:03.504 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_throttle_bytes": "67108864", 2023-11-25T04:23:03.505 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_throttle_cost_per_io": "0", 2023-11-25T04:23:03.505 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2023-11-25T04:23:03.505 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2023-11-25T04:23:03.505 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2023-11-25T04:23:03.505 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_throttle_trace_rate": "0.000000", 2023-11-25T04:23:03.505 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_tracing": "false", 2023-11-25T04:23:03.505 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2023-11-25T04:23:03.505 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2023-11-25T04:23:03.505 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_volume_selection_reserved": "0", 2023-11-25T04:23:03.505 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2023-11-25T04:23:03.505 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2023-11-25T04:23:03.505 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_warn_on_legacy_statfs": "true", 2023-11-25T04:23:03.505 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2023-11-25T04:23:03.506 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2023-11-25T04:23:03.506 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2023-11-25T04:23:03.506 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_zero_block_detection": "false", 2023-11-25T04:23:03.506 INFO:teuthology.orchestra.run.smithi067.stdout: "cephadm_path": "/usr/sbin/cephadm", 2023-11-25T04:23:03.506 INFO:teuthology.orchestra.run.smithi067.stdout: "cephfs_mirror_action_update_interval": "2", 2023-11-25T04:23:03.506 INFO:teuthology.orchestra.run.smithi067.stdout: "cephfs_mirror_directory_scan_interval": "10", 2023-11-25T04:23:03.506 INFO:teuthology.orchestra.run.smithi067.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2023-11-25T04:23:03.506 INFO:teuthology.orchestra.run.smithi067.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2023-11-25T04:23:03.506 INFO:teuthology.orchestra.run.smithi067.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2023-11-25T04:23:03.506 INFO:teuthology.orchestra.run.smithi067.stdout: "cephfs_mirror_mount_timeout": "10", 2023-11-25T04:23:03.506 INFO:teuthology.orchestra.run.smithi067.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2023-11-25T04:23:03.506 INFO:teuthology.orchestra.run.smithi067.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2023-11-25T04:23:03.507 INFO:teuthology.orchestra.run.smithi067.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2023-11-25T04:23:03.507 INFO:teuthology.orchestra.run.smithi067.stdout: "cephsqlite_blocklist_dead_locker": "true", 2023-11-25T04:23:03.507 INFO:teuthology.orchestra.run.smithi067.stdout: "cephsqlite_lock_renewal_interval": "2000", 2023-11-25T04:23:03.507 INFO:teuthology.orchestra.run.smithi067.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2023-11-25T04:23:03.507 INFO:teuthology.orchestra.run.smithi067.stdout: "cephx_cluster_require_signatures": "false", 2023-11-25T04:23:03.507 INFO:teuthology.orchestra.run.smithi067.stdout: "cephx_cluster_require_version": "2", 2023-11-25T04:23:03.507 INFO:teuthology.orchestra.run.smithi067.stdout: "cephx_require_signatures": "false", 2023-11-25T04:23:03.507 INFO:teuthology.orchestra.run.smithi067.stdout: "cephx_require_version": "2", 2023-11-25T04:23:03.507 INFO:teuthology.orchestra.run.smithi067.stdout: "cephx_service_require_signatures": "false", 2023-11-25T04:23:03.507 INFO:teuthology.orchestra.run.smithi067.stdout: "cephx_service_require_version": "2", 2023-11-25T04:23:03.507 INFO:teuthology.orchestra.run.smithi067.stdout: "cephx_sign_messages": "true", 2023-11-25T04:23:03.507 INFO:teuthology.orchestra.run.smithi067.stdout: "chdir": "", 2023-11-25T04:23:03.508 INFO:teuthology.orchestra.run.smithi067.stdout: "client_acl_type": "", 2023-11-25T04:23:03.508 INFO:teuthology.orchestra.run.smithi067.stdout: "client_asio_thread_count": "2", 2023-11-25T04:23:03.508 INFO:teuthology.orchestra.run.smithi067.stdout: "client_cache_mid": "0.750000", 2023-11-25T04:23:03.508 INFO:teuthology.orchestra.run.smithi067.stdout: "client_cache_size": "16384", 2023-11-25T04:23:03.508 INFO:teuthology.orchestra.run.smithi067.stdout: "client_caps_release_delay": "5", 2023-11-25T04:23:03.508 INFO:teuthology.orchestra.run.smithi067.stdout: "client_check_pool_perm": "true", 2023-11-25T04:23:03.508 INFO:teuthology.orchestra.run.smithi067.stdout: "client_collect_and_send_global_metrics": "false", 2023-11-25T04:23:03.508 INFO:teuthology.orchestra.run.smithi067.stdout: "client_debug_force_sync_read": "false", 2023-11-25T04:23:03.508 INFO:teuthology.orchestra.run.smithi067.stdout: "client_debug_getattr_caps": "false", 2023-11-25T04:23:03.508 INFO:teuthology.orchestra.run.smithi067.stdout: "client_debug_inject_tick_delay": "0", 2023-11-25T04:23:03.508 INFO:teuthology.orchestra.run.smithi067.stdout: "client_die_on_failed_dentry_invalidate": "true", 2023-11-25T04:23:03.508 INFO:teuthology.orchestra.run.smithi067.stdout: "client_die_on_failed_remount": "false", 2023-11-25T04:23:03.508 INFO:teuthology.orchestra.run.smithi067.stdout: "client_dirsize_rbytes": "true", 2023-11-25T04:23:03.509 INFO:teuthology.orchestra.run.smithi067.stdout: "client_force_lazyio": "false", 2023-11-25T04:23:03.509 INFO:teuthology.orchestra.run.smithi067.stdout: "client_fs": "", 2023-11-25T04:23:03.509 INFO:teuthology.orchestra.run.smithi067.stdout: "client_inject_fixed_oldest_tid": "false", 2023-11-25T04:23:03.509 INFO:teuthology.orchestra.run.smithi067.stdout: "client_inject_release_failure": "false", 2023-11-25T04:23:03.509 INFO:teuthology.orchestra.run.smithi067.stdout: "client_max_inline_size": "4096", 2023-11-25T04:23:03.509 INFO:teuthology.orchestra.run.smithi067.stdout: "client_max_retries_on_remount_failure": "5", 2023-11-25T04:23:03.509 INFO:teuthology.orchestra.run.smithi067.stdout: "client_mds_namespace": "", 2023-11-25T04:23:03.509 INFO:teuthology.orchestra.run.smithi067.stdout: "client_metadata": "", 2023-11-25T04:23:03.509 INFO:teuthology.orchestra.run.smithi067.stdout: "client_mount_gid": "-1", 2023-11-25T04:23:03.509 INFO:teuthology.orchestra.run.smithi067.stdout: "client_mount_timeout": "300", 2023-11-25T04:23:03.509 INFO:teuthology.orchestra.run.smithi067.stdout: "client_mount_uid": "-1", 2023-11-25T04:23:03.509 INFO:teuthology.orchestra.run.smithi067.stdout: "client_mountpoint": "/", 2023-11-25T04:23:03.510 INFO:teuthology.orchestra.run.smithi067.stdout: "client_notify_timeout": "10", 2023-11-25T04:23:03.510 INFO:teuthology.orchestra.run.smithi067.stdout: "client_oc": "true", 2023-11-25T04:23:03.510 INFO:teuthology.orchestra.run.smithi067.stdout: "client_oc_max_dirty": "104857600", 2023-11-25T04:23:03.510 INFO:teuthology.orchestra.run.smithi067.stdout: "client_oc_max_dirty_age": "5.000000", 2023-11-25T04:23:03.510 INFO:teuthology.orchestra.run.smithi067.stdout: "client_oc_max_objects": "1000", 2023-11-25T04:23:03.510 INFO:teuthology.orchestra.run.smithi067.stdout: "client_oc_size": "209715200", 2023-11-25T04:23:03.510 INFO:teuthology.orchestra.run.smithi067.stdout: "client_oc_target_dirty": "8388608", 2023-11-25T04:23:03.510 INFO:teuthology.orchestra.run.smithi067.stdout: "client_permissions": "true", 2023-11-25T04:23:03.510 INFO:teuthology.orchestra.run.smithi067.stdout: "client_quota": "true", 2023-11-25T04:23:03.510 INFO:teuthology.orchestra.run.smithi067.stdout: "client_quota_df": "true", 2023-11-25T04:23:03.510 INFO:teuthology.orchestra.run.smithi067.stdout: "client_readahead_max_bytes": "0", 2023-11-25T04:23:03.510 INFO:teuthology.orchestra.run.smithi067.stdout: "client_readahead_max_periods": "4", 2023-11-25T04:23:03.511 INFO:teuthology.orchestra.run.smithi067.stdout: "client_readahead_min": "131072", 2023-11-25T04:23:03.511 INFO:teuthology.orchestra.run.smithi067.stdout: "client_reconnect_stale": "false", 2023-11-25T04:23:03.511 INFO:teuthology.orchestra.run.smithi067.stdout: "client_shutdown_timeout": "30", 2023-11-25T04:23:03.511 INFO:teuthology.orchestra.run.smithi067.stdout: "client_snapdir": ".snap", 2023-11-25T04:23:03.511 INFO:teuthology.orchestra.run.smithi067.stdout: "client_tick_interval": "1", 2023-11-25T04:23:03.511 INFO:teuthology.orchestra.run.smithi067.stdout: "client_trace": "", 2023-11-25T04:23:03.511 INFO:teuthology.orchestra.run.smithi067.stdout: "client_try_dentry_invalidate": "false", 2023-11-25T04:23:03.511 INFO:teuthology.orchestra.run.smithi067.stdout: "client_use_faked_inos": "false", 2023-11-25T04:23:03.511 INFO:teuthology.orchestra.run.smithi067.stdout: "client_use_random_mds": "false", 2023-11-25T04:23:03.511 INFO:teuthology.orchestra.run.smithi067.stdout: "clog_to_graylog": "false", 2023-11-25T04:23:03.511 INFO:teuthology.orchestra.run.smithi067.stdout: "clog_to_graylog_host": "127.0.0.1", 2023-11-25T04:23:03.511 INFO:teuthology.orchestra.run.smithi067.stdout: "clog_to_graylog_port": "12201", 2023-11-25T04:23:03.512 INFO:teuthology.orchestra.run.smithi067.stdout: "clog_to_monitors": "default=true", 2023-11-25T04:23:03.512 INFO:teuthology.orchestra.run.smithi067.stdout: "clog_to_syslog": "false", 2023-11-25T04:23:03.512 INFO:teuthology.orchestra.run.smithi067.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2023-11-25T04:23:03.512 INFO:teuthology.orchestra.run.smithi067.stdout: "clog_to_syslog_level": "info", 2023-11-25T04:23:03.512 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_addr": "-", 2023-11-25T04:23:03.512 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_network": "", 2023-11-25T04:23:03.512 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_network_interface": "", 2023-11-25T04:23:03.513 INFO:teuthology.orchestra.run.smithi067.stdout: "compressor_zlib_isal": "false", 2023-11-25T04:23:03.513 INFO:teuthology.orchestra.run.smithi067.stdout: "compressor_zlib_level": "5", 2023-11-25T04:23:03.513 INFO:teuthology.orchestra.run.smithi067.stdout: "compressor_zlib_winsize": "-15", 2023-11-25T04:23:03.513 INFO:teuthology.orchestra.run.smithi067.stdout: "compressor_zstd_level": "1", 2023-11-25T04:23:03.513 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2023-11-25T04:23:03.513 INFO:teuthology.orchestra.run.smithi067.stdout: "crash_dir": "/var/lib/ceph/crash", 2023-11-25T04:23:03.513 INFO:teuthology.orchestra.run.smithi067.stdout: "crimson_alien_op_num_threads": "6", 2023-11-25T04:23:03.513 INFO:teuthology.orchestra.run.smithi067.stdout: "crimson_alien_thread_cpu_cores": "", 2023-11-25T04:23:03.513 INFO:teuthology.orchestra.run.smithi067.stdout: "crimson_osd_obc_lru_size": "10", 2023-11-25T04:23:03.513 INFO:teuthology.orchestra.run.smithi067.stdout: "crimson_osd_scheduler_concurrency": "0", 2023-11-25T04:23:03.513 INFO:teuthology.orchestra.run.smithi067.stdout: "crimson_seastar_smp": "1", 2023-11-25T04:23:03.514 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location": "", 2023-11-25T04:23:03.514 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location_hook": "", 2023-11-25T04:23:03.514 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_location_hook_timeout": "10", 2023-11-25T04:23:03.514 INFO:teuthology.orchestra.run.smithi067.stdout: "daemonize": "false", 2023-11-25T04:23:03.514 INFO:teuthology.orchestra.run.smithi067.stdout: "daos_pool": "tank", 2023-11-25T04:23:03.514 INFO:teuthology.orchestra.run.smithi067.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2023-11-25T04:23:03.514 INFO:teuthology.orchestra.run.smithi067.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2023-11-25T04:23:03.514 INFO:teuthology.orchestra.run.smithi067.stdout: "dbstore_db_name_prefix": "dbstore", 2023-11-25T04:23:03.514 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_alienstore": "0/5", 2023-11-25T04:23:03.514 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_allow_any_pool_priority": "false", 2023-11-25T04:23:03.514 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_asok": "1/5", 2023-11-25T04:23:03.514 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_asok_assert_abort": "false", 2023-11-25T04:23:03.515 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_asserts_on_shutdown": "true", 2023-11-25T04:23:03.515 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_auth": "20/20", 2023-11-25T04:23:03.515 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_bdev": "1/3", 2023-11-25T04:23:03.515 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_bluefs": "1/5", 2023-11-25T04:23:03.515 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_bluestore": "1/5", 2023-11-25T04:23:03.515 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_buffer": "0/1", 2023-11-25T04:23:03.515 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_ceph_exporter": "1/5", 2023-11-25T04:23:03.515 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_cephfs_mirror": "0/5", 2023-11-25T04:23:03.515 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_cephsqlite": "0/5", 2023-11-25T04:23:03.515 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_client": "0/5", 2023-11-25T04:23:03.515 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_compressor": "1/5", 2023-11-25T04:23:03.515 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_context": "0/1", 2023-11-25T04:23:03.515 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_crimson_interrupt": "0/5", 2023-11-25T04:23:03.516 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_crush": "1/1", 2023-11-25T04:23:03.516 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_crypto": "1/5", 2023-11-25T04:23:03.516 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_cyanstore": "0/5", 2023-11-25T04:23:03.516 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_deliberately_leak_memory": "false", 2023-11-25T04:23:03.516 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_disable_randomized_ping": "false", 2023-11-25T04:23:03.516 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_dpdk": "1/5", 2023-11-25T04:23:03.516 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_eventtrace": "1/5", 2023-11-25T04:23:03.516 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_filer": "0/1", 2023-11-25T04:23:03.516 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_filestore": "1/3", 2023-11-25T04:23:03.516 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_finisher": "1/1", 2023-11-25T04:23:03.516 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_fuse": "1/5", 2023-11-25T04:23:03.516 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_heartbeat_testing_span": "0", 2023-11-25T04:23:03.517 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_heartbeatmap": "1/5", 2023-11-25T04:23:03.517 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_immutable_obj_cache": "0/5", 2023-11-25T04:23:03.517 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_javaclient": "1/5", 2023-11-25T04:23:03.517 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_journal": "1/3", 2023-11-25T04:23:03.517 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_journaler": "0/5", 2023-11-25T04:23:03.517 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_kstore": "1/5", 2023-11-25T04:23:03.517 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_lockdep": "0/1", 2023-11-25T04:23:03.517 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_mclock": "1/5", 2023-11-25T04:23:03.517 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_mds": "1/5", 2023-11-25T04:23:03.517 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_mds_balancer": "1/5", 2023-11-25T04:23:03.517 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_mds_locker": "1/5", 2023-11-25T04:23:03.517 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_mds_log": "1/5", 2023-11-25T04:23:03.517 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_mds_log_expire": "1/5", 2023-11-25T04:23:03.518 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_mds_migrator": "1/5", 2023-11-25T04:23:03.518 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_memstore": "1/5", 2023-11-25T04:23:03.518 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_mgr": "2/5", 2023-11-25T04:23:03.518 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_mgrc": "1/5", 2023-11-25T04:23:03.518 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_mon": "20/20", 2023-11-25T04:23:03.518 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_monc": "0/10", 2023-11-25T04:23:03.518 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_ms": "1/1", 2023-11-25T04:23:03.518 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_none": "0/5", 2023-11-25T04:23:03.518 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_objclass": "0/5", 2023-11-25T04:23:03.518 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_objectcacher": "0/5", 2023-11-25T04:23:03.518 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_objecter": "0/1", 2023-11-25T04:23:03.518 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_optracker": "0/5", 2023-11-25T04:23:03.519 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_osd": "1/5", 2023-11-25T04:23:03.519 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_paxos": "20/20", 2023-11-25T04:23:03.519 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_perfcounter": "1/5", 2023-11-25T04:23:03.519 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_prioritycache": "1/5", 2023-11-25T04:23:03.519 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_rados": "0/5", 2023-11-25T04:23:03.519 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_rbd": "0/5", 2023-11-25T04:23:03.519 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_rbd_mirror": "0/5", 2023-11-25T04:23:03.519 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_rbd_pwl": "0/5", 2023-11-25T04:23:03.519 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_rbd_replay": "0/5", 2023-11-25T04:23:03.519 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_refs": "0/0", 2023-11-25T04:23:03.519 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_reserver": "1/1", 2023-11-25T04:23:03.519 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_rgw": "1/5", 2023-11-25T04:23:03.520 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_rgw_access": "1/5", 2023-11-25T04:23:03.520 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_rgw_datacache": "1/5", 2023-11-25T04:23:03.520 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_rgw_dbstore": "1/5", 2023-11-25T04:23:03.520 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_rgw_flight": "1/5", 2023-11-25T04:23:03.520 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_rgw_sync": "1/5", 2023-11-25T04:23:03.520 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_rocksdb": "4/5", 2023-11-25T04:23:03.520 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_seastore": "0/5", 2023-11-25T04:23:03.520 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_seastore_backref": "0/5", 2023-11-25T04:23:03.520 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_seastore_cache": "0/5", 2023-11-25T04:23:03.520 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_seastore_cleaner": "0/5", 2023-11-25T04:23:03.520 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_seastore_device": "0/5", 2023-11-25T04:23:03.520 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_seastore_epm": "0/5", 2023-11-25T04:23:03.521 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_seastore_fixedkv_tree": "0/5", 2023-11-25T04:23:03.521 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_seastore_journal": "0/5", 2023-11-25T04:23:03.521 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_seastore_lba": "0/5", 2023-11-25T04:23:03.521 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_seastore_odata": "0/5", 2023-11-25T04:23:03.521 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_seastore_omap": "0/5", 2023-11-25T04:23:03.521 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_seastore_onode": "0/5", 2023-11-25T04:23:03.521 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_seastore_t": "0/5", 2023-11-25T04:23:03.521 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_seastore_tm": "0/5", 2023-11-25T04:23:03.521 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_striper": "0/1", 2023-11-25T04:23:03.521 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_test": "0/5", 2023-11-25T04:23:03.521 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_throttle": "1/1", 2023-11-25T04:23:03.521 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_timer": "0/1", 2023-11-25T04:23:03.521 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_tp": "0/5", 2023-11-25T04:23:03.522 INFO:teuthology.orchestra.run.smithi067.stdout: "debug_trace": "1/5", 2023-11-25T04:23:03.522 INFO:teuthology.orchestra.run.smithi067.stdout: "defer_client_eviction_on_laggy_osds": "true", 2023-11-25T04:23:03.522 INFO:teuthology.orchestra.run.smithi067.stdout: "device_failure_prediction_mode": "none", 2023-11-25T04:23:03.522 INFO:teuthology.orchestra.run.smithi067.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2023-11-25T04:23:03.522 INFO:teuthology.orchestra.run.smithi067.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2023-11-25T04:23:03.522 INFO:teuthology.orchestra.run.smithi067.stdout: "err_to_graylog": "false", 2023-11-25T04:23:03.522 INFO:teuthology.orchestra.run.smithi067.stdout: "err_to_journald": "false", 2023-11-25T04:23:03.522 INFO:teuthology.orchestra.run.smithi067.stdout: "err_to_stderr": "true", 2023-11-25T04:23:03.522 INFO:teuthology.orchestra.run.smithi067.stdout: "err_to_syslog": "false", 2023-11-25T04:23:03.522 INFO:teuthology.orchestra.run.smithi067.stdout: "event_tracing": "false", 2023-11-25T04:23:03.522 INFO:teuthology.orchestra.run.smithi067.stdout: "exporter_addr": "0.0.0.0", 2023-11-25T04:23:03.523 INFO:teuthology.orchestra.run.smithi067.stdout: "exporter_http_port": "9926", 2023-11-25T04:23:03.523 INFO:teuthology.orchestra.run.smithi067.stdout: "exporter_prio_limit": "5", 2023-11-25T04:23:03.523 INFO:teuthology.orchestra.run.smithi067.stdout: "exporter_sock_dir": "/var/run/ceph/", 2023-11-25T04:23:03.523 INFO:teuthology.orchestra.run.smithi067.stdout: "exporter_sort_metrics": "true", 2023-11-25T04:23:03.523 INFO:teuthology.orchestra.run.smithi067.stdout: "exporter_stats_period": "5", 2023-11-25T04:23:03.523 INFO:teuthology.orchestra.run.smithi067.stdout: "fake_statfs_for_testing": "0", 2023-11-25T04:23:03.523 INFO:teuthology.orchestra.run.smithi067.stdout: "fatal_signal_handlers": "true", 2023-11-25T04:23:03.523 INFO:teuthology.orchestra.run.smithi067.stdout: "filer_max_purge_ops": "10", 2023-11-25T04:23:03.523 INFO:teuthology.orchestra.run.smithi067.stdout: "filer_max_truncate_ops": "128", 2023-11-25T04:23:03.523 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_apply_finisher_threads": "1", 2023-11-25T04:23:03.523 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_blackhole": "false", 2023-11-25T04:23:03.523 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_btrfs_clone_range": "true", 2023-11-25T04:23:03.523 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_btrfs_snap": "true", 2023-11-25T04:23:03.524 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_caller_concurrency": "10", 2023-11-25T04:23:03.524 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_collect_device_partition_information": "true", 2023-11-25T04:23:03.524 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_commit_timeout": "600.000000", 2023-11-25T04:23:03.524 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_debug_inject_read_err": "false", 2023-11-25T04:23:03.524 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_debug_omap_check": "false", 2023-11-25T04:23:03.524 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_debug_random_read_err": "0.000000", 2023-11-25T04:23:03.524 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_debug_verify_split": "false", 2023-11-25T04:23:03.524 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_dump_file": "", 2023-11-25T04:23:03.524 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2023-11-25T04:23:03.524 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_expected_throughput_ops": "200.000000", 2023-11-25T04:23:03.524 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_fadvise": "true", 2023-11-25T04:23:03.524 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_fail_eio": "true", 2023-11-25T04:23:03.525 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_fd_cache_shards": "16", 2023-11-25T04:23:03.525 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_fd_cache_size": "128", 2023-11-25T04:23:03.525 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_fiemap": "false", 2023-11-25T04:23:03.525 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_fiemap_threshold": "4096", 2023-11-25T04:23:03.525 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_fsync_flushes_journal_data": "false", 2023-11-25T04:23:03.525 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_index_retry_probability": "0.000000", 2023-11-25T04:23:03.525 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_inject_stall": "0", 2023-11-25T04:23:03.525 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_journal_parallel": "false", 2023-11-25T04:23:03.525 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_journal_trailing": "false", 2023-11-25T04:23:03.525 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_journal_writeahead": "false", 2023-11-25T04:23:03.525 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_kill_at": "0", 2023-11-25T04:23:03.525 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_max_alloc_hint_size": "1048576", 2023-11-25T04:23:03.526 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_max_inline_xattr_size": "0", 2023-11-25T04:23:03.526 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2023-11-25T04:23:03.526 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_max_inline_xattr_size_other": "512", 2023-11-25T04:23:03.526 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2023-11-25T04:23:03.526 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_max_inline_xattrs": "0", 2023-11-25T04:23:03.526 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2023-11-25T04:23:03.526 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_max_inline_xattrs_other": "2", 2023-11-25T04:23:03.526 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_max_inline_xattrs_xfs": "10", 2023-11-25T04:23:03.526 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_max_sync_interval": "5.000000", 2023-11-25T04:23:03.526 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_max_xattr_value_size": "0", 2023-11-25T04:23:03.526 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2023-11-25T04:23:03.526 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_max_xattr_value_size_other": "1024", 2023-11-25T04:23:03.527 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2023-11-25T04:23:03.527 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_merge_threshold": "-10", 2023-11-25T04:23:03.527 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_min_sync_interval": "0.010000", 2023-11-25T04:23:03.527 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_odsync_write": "false", 2023-11-25T04:23:03.527 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_omap_backend": "rocksdb", 2023-11-25T04:23:03.527 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_omap_backend_path": "", 2023-11-25T04:23:03.527 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_omap_header_cache_size": "1024", 2023-11-25T04:23:03.527 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_ondisk_finisher_threads": "1", 2023-11-25T04:23:03.527 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_op_thread_suicide_timeout": "180", 2023-11-25T04:23:03.527 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_op_thread_timeout": "60", 2023-11-25T04:23:03.527 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_op_threads": "2", 2023-11-25T04:23:03.527 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_punch_hole": "false", 2023-11-25T04:23:03.528 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2023-11-25T04:23:03.528 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2023-11-25T04:23:03.528 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2023-11-25T04:23:03.528 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_queue_high_threshhold": "0.900000", 2023-11-25T04:23:03.528 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_queue_low_threshhold": "0.300000", 2023-11-25T04:23:03.528 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_queue_max_bytes": "104857600", 2023-11-25T04:23:03.528 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2023-11-25T04:23:03.528 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2023-11-25T04:23:03.528 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2023-11-25T04:23:03.528 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_queue_max_ops": "50", 2023-11-25T04:23:03.528 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2023-11-25T04:23:03.528 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_seek_data_hole": "false", 2023-11-25T04:23:03.528 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_sloppy_crc": "false", 2023-11-25T04:23:03.529 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_sloppy_crc_block_size": "65536", 2023-11-25T04:23:03.529 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_splice": "false", 2023-11-25T04:23:03.529 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_split_multiple": "2", 2023-11-25T04:23:03.529 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_split_rand_factor": "20", 2023-11-25T04:23:03.529 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_update_to": "1000", 2023-11-25T04:23:03.529 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2023-11-25T04:23:03.529 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2023-11-25T04:23:03.529 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2023-11-25T04:23:03.529 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2023-11-25T04:23:03.529 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2023-11-25T04:23:03.529 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2023-11-25T04:23:03.529 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_wbthrottle_enable": "true", 2023-11-25T04:23:03.530 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2023-11-25T04:23:03.530 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2023-11-25T04:23:03.530 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2023-11-25T04:23:03.530 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2023-11-25T04:23:03.530 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2023-11-25T04:23:03.530 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2023-11-25T04:23:03.530 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_xfs_extsize": "false", 2023-11-25T04:23:03.530 INFO:teuthology.orchestra.run.smithi067.stdout: "filestore_zfs_snap": "false", 2023-11-25T04:23:03.530 INFO:teuthology.orchestra.run.smithi067.stdout: "fio_dir": "/tmp/fio", 2023-11-25T04:23:03.530 INFO:teuthology.orchestra.run.smithi067.stdout: "fsid": "a677f511-1143-4e76-8da6-5ddfadaff077", 2023-11-25T04:23:03.530 INFO:teuthology.orchestra.run.smithi067.stdout: "fuse_allow_other": "true", 2023-11-25T04:23:03.530 INFO:teuthology.orchestra.run.smithi067.stdout: "fuse_atomic_o_trunc": "true", 2023-11-25T04:23:03.530 INFO:teuthology.orchestra.run.smithi067.stdout: "fuse_big_writes": "true", 2023-11-25T04:23:03.531 INFO:teuthology.orchestra.run.smithi067.stdout: "fuse_debug": "false", 2023-11-25T04:23:03.531 INFO:teuthology.orchestra.run.smithi067.stdout: "fuse_default_permissions": "false", 2023-11-25T04:23:03.531 INFO:teuthology.orchestra.run.smithi067.stdout: "fuse_disable_pagecache": "false", 2023-11-25T04:23:03.531 INFO:teuthology.orchestra.run.smithi067.stdout: "fuse_max_write": "0", 2023-11-25T04:23:03.531 INFO:teuthology.orchestra.run.smithi067.stdout: "fuse_multithreaded": "true", 2023-11-25T04:23:03.531 INFO:teuthology.orchestra.run.smithi067.stdout: "fuse_require_active_mds": "true", 2023-11-25T04:23:03.531 INFO:teuthology.orchestra.run.smithi067.stdout: "fuse_set_user_groups": "true", 2023-11-25T04:23:03.531 INFO:teuthology.orchestra.run.smithi067.stdout: "fuse_splice_move": "true", 2023-11-25T04:23:03.531 INFO:teuthology.orchestra.run.smithi067.stdout: "fuse_splice_read": "true", 2023-11-25T04:23:03.531 INFO:teuthology.orchestra.run.smithi067.stdout: "fuse_splice_write": "true", 2023-11-25T04:23:03.531 INFO:teuthology.orchestra.run.smithi067.stdout: "fuse_syncfs_on_mksnap": "true", 2023-11-25T04:23:03.531 INFO:teuthology.orchestra.run.smithi067.stdout: "fuse_use_invalidate_cb": "true", 2023-11-25T04:23:03.532 INFO:teuthology.orchestra.run.smithi067.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2023-11-25T04:23:03.532 INFO:teuthology.orchestra.run.smithi067.stdout: "gss_target_name": "ceph", 2023-11-25T04:23:03.532 INFO:teuthology.orchestra.run.smithi067.stdout: "heartbeat_file": "", 2023-11-25T04:23:03.532 INFO:teuthology.orchestra.run.smithi067.stdout: "heartbeat_inject_failure": "0", 2023-11-25T04:23:03.532 INFO:teuthology.orchestra.run.smithi067.stdout: "heartbeat_interval": "5", 2023-11-25T04:23:03.532 INFO:teuthology.orchestra.run.smithi067.stdout: "host": "smithi067", 2023-11-25T04:23:03.532 INFO:teuthology.orchestra.run.smithi067.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2023-11-25T04:23:03.532 INFO:teuthology.orchestra.run.smithi067.stdout: "immutable_object_cache_max_inflight_ops": "128", 2023-11-25T04:23:03.532 INFO:teuthology.orchestra.run.smithi067.stdout: "immutable_object_cache_max_size": "1073741824", 2023-11-25T04:23:03.532 INFO:teuthology.orchestra.run.smithi067.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2023-11-25T04:23:03.532 INFO:teuthology.orchestra.run.smithi067.stdout: "immutable_object_cache_qos_bps_burst": "0", 2023-11-25T04:23:03.532 INFO:teuthology.orchestra.run.smithi067.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2023-11-25T04:23:03.533 INFO:teuthology.orchestra.run.smithi067.stdout: "immutable_object_cache_qos_bps_limit": "0", 2023-11-25T04:23:03.533 INFO:teuthology.orchestra.run.smithi067.stdout: "immutable_object_cache_qos_iops_burst": "0", 2023-11-25T04:23:03.533 INFO:teuthology.orchestra.run.smithi067.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2023-11-25T04:23:03.533 INFO:teuthology.orchestra.run.smithi067.stdout: "immutable_object_cache_qos_iops_limit": "0", 2023-11-25T04:23:03.533 INFO:teuthology.orchestra.run.smithi067.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2023-11-25T04:23:03.533 INFO:teuthology.orchestra.run.smithi067.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2023-11-25T04:23:03.533 INFO:teuthology.orchestra.run.smithi067.stdout: "immutable_object_cache_watermark": "0.900000", 2023-11-25T04:23:03.533 INFO:teuthology.orchestra.run.smithi067.stdout: "inject_early_sigterm": "false", 2023-11-25T04:23:03.533 INFO:teuthology.orchestra.run.smithi067.stdout: "jaeger_agent_port": "6799", 2023-11-25T04:23:03.533 INFO:teuthology.orchestra.run.smithi067.stdout: "jaeger_tracing_enable": "false", 2023-11-25T04:23:03.533 INFO:teuthology.orchestra.run.smithi067.stdout: "journal_aio": "true", 2023-11-25T04:23:03.533 INFO:teuthology.orchestra.run.smithi067.stdout: "journal_align_min_size": "65536", 2023-11-25T04:23:03.533 INFO:teuthology.orchestra.run.smithi067.stdout: "journal_block_align": "true", 2023-11-25T04:23:03.534 INFO:teuthology.orchestra.run.smithi067.stdout: "journal_block_size": "4096", 2023-11-25T04:23:03.534 INFO:teuthology.orchestra.run.smithi067.stdout: "journal_dio": "true", 2023-11-25T04:23:03.534 INFO:teuthology.orchestra.run.smithi067.stdout: "journal_discard": "false", 2023-11-25T04:23:03.534 INFO:teuthology.orchestra.run.smithi067.stdout: "journal_force_aio": "false", 2023-11-25T04:23:03.534 INFO:teuthology.orchestra.run.smithi067.stdout: "journal_ignore_corruption": "false", 2023-11-25T04:23:03.534 INFO:teuthology.orchestra.run.smithi067.stdout: "journal_max_write_bytes": "10485760", 2023-11-25T04:23:03.534 INFO:teuthology.orchestra.run.smithi067.stdout: "journal_max_write_entries": "100", 2023-11-25T04:23:03.534 INFO:teuthology.orchestra.run.smithi067.stdout: "journal_replay_from": "0", 2023-11-25T04:23:03.534 INFO:teuthology.orchestra.run.smithi067.stdout: "journal_throttle_high_multiple": "0.000000", 2023-11-25T04:23:03.534 INFO:teuthology.orchestra.run.smithi067.stdout: "journal_throttle_high_threshhold": "0.900000", 2023-11-25T04:23:03.534 INFO:teuthology.orchestra.run.smithi067.stdout: "journal_throttle_low_threshhold": "0.600000", 2023-11-25T04:23:03.534 INFO:teuthology.orchestra.run.smithi067.stdout: "journal_throttle_max_multiple": "0.000000", 2023-11-25T04:23:03.535 INFO:teuthology.orchestra.run.smithi067.stdout: "journal_write_header_frequency": "0", 2023-11-25T04:23:03.535 INFO:teuthology.orchestra.run.smithi067.stdout: "journal_zero_on_create": "false", 2023-11-25T04:23:03.535 INFO:teuthology.orchestra.run.smithi067.stdout: "journaler_prefetch_periods": "10", 2023-11-25T04:23:03.535 INFO:teuthology.orchestra.run.smithi067.stdout: "journaler_prezero_periods": "5", 2023-11-25T04:23:03.535 INFO:teuthology.orchestra.run.smithi067.stdout: "journaler_write_head_interval": "15", 2023-11-25T04:23:03.535 INFO:teuthology.orchestra.run.smithi067.stdout: "key": "", 2023-11-25T04:23:03.535 INFO:teuthology.orchestra.run.smithi067.stdout: "keyfile": "", 2023-11-25T04:23:03.535 INFO:teuthology.orchestra.run.smithi067.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2023-11-25T04:23:03.535 INFO:teuthology.orchestra.run.smithi067.stdout: "kstore_backend": "rocksdb", 2023-11-25T04:23:03.535 INFO:teuthology.orchestra.run.smithi067.stdout: "kstore_default_stripe_size": "65536", 2023-11-25T04:23:03.535 INFO:teuthology.orchestra.run.smithi067.stdout: "kstore_fsck_on_mount": "false", 2023-11-25T04:23:03.535 INFO:teuthology.orchestra.run.smithi067.stdout: "kstore_fsck_on_mount_deep": "true", 2023-11-25T04:23:03.536 INFO:teuthology.orchestra.run.smithi067.stdout: "kstore_max_bytes": "67108864", 2023-11-25T04:23:03.536 INFO:teuthology.orchestra.run.smithi067.stdout: "kstore_max_ops": "512", 2023-11-25T04:23:03.536 INFO:teuthology.orchestra.run.smithi067.stdout: "kstore_nid_prealloc": "1024", 2023-11-25T04:23:03.536 INFO:teuthology.orchestra.run.smithi067.stdout: "kstore_onode_map_size": "1024", 2023-11-25T04:23:03.536 INFO:teuthology.orchestra.run.smithi067.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2023-11-25T04:23:03.536 INFO:teuthology.orchestra.run.smithi067.stdout: "kstore_sync_submit_transaction": "false", 2023-11-25T04:23:03.536 INFO:teuthology.orchestra.run.smithi067.stdout: "kstore_sync_transaction": "false", 2023-11-25T04:23:03.536 INFO:teuthology.orchestra.run.smithi067.stdout: "librados_thread_count": "2", 2023-11-25T04:23:03.536 INFO:teuthology.orchestra.run.smithi067.stdout: "lockdep": "false", 2023-11-25T04:23:03.536 INFO:teuthology.orchestra.run.smithi067.stdout: "lockdep_force_backtrace": "false", 2023-11-25T04:23:03.536 INFO:teuthology.orchestra.run.smithi067.stdout: "log_coarse_timestamps": "true", 2023-11-25T04:23:03.536 INFO:teuthology.orchestra.run.smithi067.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2023-11-25T04:23:03.537 INFO:teuthology.orchestra.run.smithi067.stdout: "log_flush_on_exit": "false", 2023-11-25T04:23:03.537 INFO:teuthology.orchestra.run.smithi067.stdout: "log_graylog_host": "127.0.0.1", 2023-11-25T04:23:03.537 INFO:teuthology.orchestra.run.smithi067.stdout: "log_graylog_port": "12201", 2023-11-25T04:23:03.537 INFO:teuthology.orchestra.run.smithi067.stdout: "log_max_new": "1000", 2023-11-25T04:23:03.537 INFO:teuthology.orchestra.run.smithi067.stdout: "log_max_recent": "10000", 2023-11-25T04:23:03.537 INFO:teuthology.orchestra.run.smithi067.stdout: "log_stderr_prefix": "", 2023-11-25T04:23:03.537 INFO:teuthology.orchestra.run.smithi067.stdout: "log_stop_at_utilization": "0.970000", 2023-11-25T04:23:03.537 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": "true", 2023-11-25T04:23:03.537 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_graylog": "false", 2023-11-25T04:23:03.537 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_journald": "false", 2023-11-25T04:23:03.537 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_stderr": "false", 2023-11-25T04:23:03.537 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_syslog": "false", 2023-11-25T04:23:03.537 INFO:teuthology.orchestra.run.smithi067.stdout: "mandatory_topic_permissions": "false", 2023-11-25T04:23:03.538 INFO:teuthology.orchestra.run.smithi067.stdout: "max_rotating_auth_attempts": "10", 2023-11-25T04:23:03.538 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2023-11-25T04:23:03.538 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_action_on_write_error": "1", 2023-11-25T04:23:03.538 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_alternate_name_max": "8192", 2023-11-25T04:23:03.538 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_asio_thread_count": "2", 2023-11-25T04:23:03.538 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_bal_export_pin": "true", 2023-11-25T04:23:03.538 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_bal_fragment_dirs": "true", 2023-11-25T04:23:03.538 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2023-11-25T04:23:03.538 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_bal_fragment_interval": "5", 2023-11-25T04:23:03.539 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_bal_fragment_size_max": "100000", 2023-11-25T04:23:03.539 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_bal_idle_threshold": "0.000000", 2023-11-25T04:23:03.539 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_bal_interval": "10", 2023-11-25T04:23:03.539 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_bal_max": "-1", 2023-11-25T04:23:03.539 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_bal_max_until": "-1", 2023-11-25T04:23:03.539 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_bal_merge_size": "50", 2023-11-25T04:23:03.539 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_bal_midchunk": "0.300000", 2023-11-25T04:23:03.539 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_bal_min_rebalance": "0.100000", 2023-11-25T04:23:03.539 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_bal_min_start": "0.200000", 2023-11-25T04:23:03.539 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_bal_minchunk": "0.001000", 2023-11-25T04:23:03.539 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_bal_mode": "0", 2023-11-25T04:23:03.539 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_bal_need_max": "1.200000", 2023-11-25T04:23:03.539 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_bal_need_min": "0.800000", 2023-11-25T04:23:03.540 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_bal_overload_epochs": "2", 2023-11-25T04:23:03.540 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_bal_replicate_threshold": "8000.000000", 2023-11-25T04:23:03.540 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_bal_sample_interval": "3.000000", 2023-11-25T04:23:03.540 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_bal_split_bits": "3", 2023-11-25T04:23:03.540 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_bal_split_rd": "25000.000000", 2023-11-25T04:23:03.540 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_bal_split_size": "10000", 2023-11-25T04:23:03.540 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_bal_split_wr": "10000.000000", 2023-11-25T04:23:03.540 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_bal_target_decay": "10.000000", 2023-11-25T04:23:03.540 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2023-11-25T04:23:03.540 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_beacon_grace": "15.000000", 2023-11-25T04:23:03.540 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_beacon_interval": "4.000000", 2023-11-25T04:23:03.540 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_beacon_mon_down_grace": "60", 2023-11-25T04:23:03.541 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_cache_memory_limit": "4294967296", 2023-11-25T04:23:03.541 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_cache_mid": "0.700000", 2023-11-25T04:23:03.541 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_cache_release_free_interval": "10", 2023-11-25T04:23:03.541 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_cache_reservation": "0.050000", 2023-11-25T04:23:03.541 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_cache_trim_decay_rate": "1.000000", 2023-11-25T04:23:03.541 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_cache_trim_interval": "1", 2023-11-25T04:23:03.541 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_cache_trim_threshold": "262144", 2023-11-25T04:23:03.541 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2023-11-25T04:23:03.541 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2023-11-25T04:23:03.541 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_client_delegate_inos_pct": "50", 2023-11-25T04:23:03.541 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_client_prealloc_inos": "1000", 2023-11-25T04:23:03.541 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2023-11-25T04:23:03.541 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_connect_bootstrapping": "false", 2023-11-25T04:23:03.542 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_damage_table_max_entries": "10000", 2023-11-25T04:23:03.542 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2023-11-25T04:23:03.542 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_debug_auth_pins": "false", 2023-11-25T04:23:03.542 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_debug_frag": "false", 2023-11-25T04:23:03.542 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_debug_scatterstat": "false", 2023-11-25T04:23:03.542 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_debug_subtrees": "false", 2023-11-25T04:23:03.542 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_decay_halflife": "5.000000", 2023-11-25T04:23:03.542 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_default_dir_hash": "2", 2023-11-25T04:23:03.542 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_defer_session_stale": "true", 2023-11-25T04:23:03.542 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_deny_all_reconnect": "false", 2023-11-25T04:23:03.542 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_dir_keys_per_op": "16384", 2023-11-25T04:23:03.542 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_dir_max_commit_size": "10", 2023-11-25T04:23:03.543 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_dir_max_entries": "0", 2023-11-25T04:23:03.543 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_dir_prefetch": "true", 2023-11-25T04:23:03.543 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_dirstat_min_interval": "1.000000", 2023-11-25T04:23:03.543 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_dump_cache_after_rejoin": "false", 2023-11-25T04:23:03.543 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_dump_cache_on_map": "false", 2023-11-25T04:23:03.543 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_dump_cache_threshold_file": "0", 2023-11-25T04:23:03.543 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2023-11-25T04:23:03.543 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_early_reply": "true", 2023-11-25T04:23:03.543 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_enable_op_tracker": "true", 2023-11-25T04:23:03.543 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_enforce_unique_name": "true", 2023-11-25T04:23:03.543 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_export_ephemeral_distributed": "true", 2023-11-25T04:23:03.543 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2023-11-25T04:23:03.543 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_export_ephemeral_random": "true", 2023-11-25T04:23:03.544 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_export_ephemeral_random_max": "0.010000", 2023-11-25T04:23:03.544 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_extraordinary_events_dump_interval": "0", 2023-11-25T04:23:03.544 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_forward_all_requests_to_auth": "false", 2023-11-25T04:23:03.544 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_freeze_tree_timeout": "30.000000", 2023-11-25T04:23:03.544 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_fscrypt_last_block_max_size": "4096", 2023-11-25T04:23:03.544 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_go_bad_corrupt_dentry": "true", 2023-11-25T04:23:03.544 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2023-11-25T04:23:03.544 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_health_cache_threshold": "1.500000", 2023-11-25T04:23:03.544 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_health_summarize_threshold": "10", 2023-11-25T04:23:03.544 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_heartbeat_grace": "15.000000", 2023-11-25T04:23:03.544 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_heartbeat_reset_grace": "1000", 2023-11-25T04:23:03.544 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_inject_health_dummy": "false", 2023-11-25T04:23:03.545 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2023-11-25T04:23:03.545 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_inject_migrator_session_race": "false", 2023-11-25T04:23:03.545 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2023-11-25T04:23:03.545 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_inject_skip_replaying_inotable": "false", 2023-11-25T04:23:03.545 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2023-11-25T04:23:03.545 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_join_fs": "", 2023-11-25T04:23:03.545 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_journal_format": "1", 2023-11-25T04:23:03.545 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_kill_create_at": "0", 2023-11-25T04:23:03.545 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_kill_export_at": "0", 2023-11-25T04:23:03.545 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_kill_import_at": "0", 2023-11-25T04:23:03.545 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_kill_journal_at": "0", 2023-11-25T04:23:03.545 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_kill_journal_expire_at": "0", 2023-11-25T04:23:03.545 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_kill_journal_replay_at": "0", 2023-11-25T04:23:03.546 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_kill_link_at": "0", 2023-11-25T04:23:03.546 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_kill_mdstable_at": "0", 2023-11-25T04:23:03.546 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_kill_openc_at": "0", 2023-11-25T04:23:03.546 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_kill_rename_at": "0", 2023-11-25T04:23:03.546 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_kill_shutdown_at": "0", 2023-11-25T04:23:03.546 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_kill_skip_replaying_inotable": "false", 2023-11-25T04:23:03.546 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_log_event_large_threshold": "524288", 2023-11-25T04:23:03.546 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_log_events_per_segment": "1024", 2023-11-25T04:23:03.546 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_log_major_segment_event_ratio": "12", 2023-11-25T04:23:03.546 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_log_max_events": "-1", 2023-11-25T04:23:03.546 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_log_max_segments": "128", 2023-11-25T04:23:03.546 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_log_pause": "false", 2023-11-25T04:23:03.547 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_log_segment_size": "0", 2023-11-25T04:23:03.547 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_log_skip_corrupt_events": "false", 2023-11-25T04:23:03.547 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_log_skip_unbounded_events": "false", 2023-11-25T04:23:03.547 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_log_warn_factor": "2.000000", 2023-11-25T04:23:03.547 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_max_caps_per_client": "1048576", 2023-11-25T04:23:03.547 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_max_completed_flushes": "100000", 2023-11-25T04:23:03.547 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_max_completed_requests": "100000", 2023-11-25T04:23:03.547 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_max_export_size": "20971520", 2023-11-25T04:23:03.547 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_max_file_recover": "32", 2023-11-25T04:23:03.547 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_max_purge_files": "64", 2023-11-25T04:23:03.547 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_max_purge_ops": "8192", 2023-11-25T04:23:03.547 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2023-11-25T04:23:03.548 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_max_scrub_ops_in_progress": "5", 2023-11-25T04:23:03.548 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_max_snaps_per_dir": "100", 2023-11-25T04:23:03.548 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_metrics_update_interval": "2", 2023-11-25T04:23:03.548 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_min_caps_per_client": "100", 2023-11-25T04:23:03.548 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_min_caps_working_set": "10000", 2023-11-25T04:23:03.548 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_mon_shutdown_timeout": "5.000000", 2023-11-25T04:23:03.548 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_numa_node": "-1", 2023-11-25T04:23:03.548 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_oft_prefetch_dirfrags": "false", 2023-11-25T04:23:03.548 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_op_complaint_time": "30.000000", 2023-11-25T04:23:03.548 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_op_history_duration": "600", 2023-11-25T04:23:03.548 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_op_history_size": "20", 2023-11-25T04:23:03.548 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_op_history_slow_op_size": "20", 2023-11-25T04:23:03.548 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_op_history_slow_op_threshold": "10", 2023-11-25T04:23:03.549 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_op_log_threshold": "5", 2023-11-25T04:23:03.549 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_ping_grace": "15", 2023-11-25T04:23:03.549 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_ping_interval": "5", 2023-11-25T04:23:03.549 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2023-11-25T04:23:03.549 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_recall_global_max_decay_threshold": "131072", 2023-11-25T04:23:03.549 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_recall_max_caps": "30000", 2023-11-25T04:23:03.549 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_recall_max_decay_rate": "1.500000", 2023-11-25T04:23:03.549 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_recall_max_decay_threshold": "131072", 2023-11-25T04:23:03.549 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_recall_warning_decay_rate": "60.000000", 2023-11-25T04:23:03.549 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_recall_warning_threshold": "262144", 2023-11-25T04:23:03.549 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_reconnect_timeout": "45.000000", 2023-11-25T04:23:03.549 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_replay_interval": "1.000000", 2023-11-25T04:23:03.550 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_replay_unsafe_with_closed_session": "false", 2023-11-25T04:23:03.550 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_request_load_average_decay_rate": "60.000000", 2023-11-25T04:23:03.550 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_root_ino_gid": "0", 2023-11-25T04:23:03.550 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_root_ino_uid": "0", 2023-11-25T04:23:03.550 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_scatter_nudge_interval": "5.000000", 2023-11-25T04:23:03.550 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_session_blocklist_on_evict": "true", 2023-11-25T04:23:03.550 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_session_blocklist_on_timeout": "true", 2023-11-25T04:23:03.550 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2023-11-25T04:23:03.550 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_session_cache_liveness_magnitude": "10", 2023-11-25T04:23:03.550 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2023-11-25T04:23:03.550 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_session_cap_acquisition_throttle": "100000", 2023-11-25T04:23:03.550 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2023-11-25T04:23:03.551 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_session_metadata_threshold": "16777216", 2023-11-25T04:23:03.551 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_sessionmap_keys_per_op": "1024", 2023-11-25T04:23:03.551 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_shutdown_check": "0", 2023-11-25T04:23:03.551 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_skip_ino": "0", 2023-11-25T04:23:03.551 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_sleep_rank_change": "0.000000", 2023-11-25T04:23:03.551 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_snap_max_uid": "4294967294", 2023-11-25T04:23:03.551 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_snap_min_uid": "0", 2023-11-25T04:23:03.551 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_snap_rstat": "false", 2023-11-25T04:23:03.551 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_standby_replay_damaged": "false", 2023-11-25T04:23:03.551 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_symlink_recovery": "true", 2023-11-25T04:23:03.551 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_task_status_update_interval": "2.000000", 2023-11-25T04:23:03.551 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_thrash_exports": "0", 2023-11-25T04:23:03.551 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_thrash_fragments": "0", 2023-11-25T04:23:03.552 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_tick_interval": "5.000000", 2023-11-25T04:23:03.552 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_valgrind_exit": "false", 2023-11-25T04:23:03.552 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_verify_backtrace": "1", 2023-11-25T04:23:03.552 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_verify_scatter": "false", 2023-11-25T04:23:03.552 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_wipe_ino_prealloc": "false", 2023-11-25T04:23:03.552 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_wipe_sessions": "false", 2023-11-25T04:23:03.552 INFO:teuthology.orchestra.run.smithi067.stdout: "mempool_debug": "false", 2023-11-25T04:23:03.552 INFO:teuthology.orchestra.run.smithi067.stdout: "memstore_debug_omit_block_device_write": "false", 2023-11-25T04:23:03.552 INFO:teuthology.orchestra.run.smithi067.stdout: "memstore_device_bytes": "1073741824", 2023-11-25T04:23:03.552 INFO:teuthology.orchestra.run.smithi067.stdout: "memstore_page_set": "false", 2023-11-25T04:23:03.552 INFO:teuthology.orchestra.run.smithi067.stdout: "memstore_page_size": "65536", 2023-11-25T04:23:03.552 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr_client_bytes": "134217728", 2023-11-25T04:23:03.553 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr_client_messages": "512", 2023-11-25T04:23:03.553 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2023-11-25T04:23:03.553 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr_connect_retry_interval": "1.000000", 2023-11-25T04:23:03.553 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2023-11-25T04:23:03.553 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2023-11-25T04:23:03.553 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr_disabled_modules": "", 2023-11-25T04:23:03.553 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr_initial_modules": "restful iostat nfs", 2023-11-25T04:23:03.553 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr_max_pg_num_change": "128", 2023-11-25T04:23:03.553 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr_mds_bytes": "134217728", 2023-11-25T04:23:03.553 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr_mds_messages": "128", 2023-11-25T04:23:03.553 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2023-11-25T04:23:03.553 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr_mon_bytes": "134217728", 2023-11-25T04:23:03.553 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr_mon_messages": "128", 2023-11-25T04:23:03.554 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr_osd_bytes": "536870912", 2023-11-25T04:23:03.554 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr_osd_messages": "8192", 2023-11-25T04:23:03.554 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr_pool": "true", 2023-11-25T04:23:03.554 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr_service_beacon_grace": "60.000000", 2023-11-25T04:23:03.554 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr_standby_modules": "true", 2023-11-25T04:23:03.554 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr_stats_period": "5", 2023-11-25T04:23:03.554 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr_stats_threshold": "5", 2023-11-25T04:23:03.554 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr_tick_period": "2", 2023-11-25T04:23:03.554 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr_ttl_cache_expire_seconds": "0", 2023-11-25T04:23:03.554 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_accept_timeout_factor": "2.000000", 2023-11-25T04:23:03.554 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_allow_pool_delete": "true", 2023-11-25T04:23:03.554 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_allow_pool_size_one": "true", 2023-11-25T04:23:03.555 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_auth_validate_all_caps": "true", 2023-11-25T04:23:03.555 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2023-11-25T04:23:03.555 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2023-11-25T04:23:03.555 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_client_bytes": "104857600", 2023-11-25T04:23:03.555 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_client_directed_command_retry": "2", 2023-11-25T04:23:03.555 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_client_hunt_interval": "3.000000", 2023-11-25T04:23:03.555 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2023-11-25T04:23:03.555 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2023-11-25T04:23:03.555 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2023-11-25T04:23:03.555 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_client_hunt_parallel": "3", 2023-11-25T04:23:03.555 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_client_log_interval": "1.000000", 2023-11-25T04:23:03.555 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_client_max_log_entries_per_message": "1000", 2023-11-25T04:23:03.556 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_client_ping_interval": "10.000000", 2023-11-25T04:23:03.556 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_client_ping_timeout": "30.000000", 2023-11-25T04:23:03.556 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_client_target_rank": "-1", 2023-11-25T04:23:03.556 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_clock_drift_allowed": "1.000000", 2023-11-25T04:23:03.556 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2023-11-25T04:23:03.556 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2023-11-25T04:23:03.556 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_cluster_log_file_level": "debug", 2023-11-25T04:23:03.556 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_cluster_log_to_file": "true", 2023-11-25T04:23:03.556 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_cluster_log_to_graylog": "false", 2023-11-25T04:23:03.556 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2023-11-25T04:23:03.556 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_cluster_log_to_graylog_port": "12201", 2023-11-25T04:23:03.556 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_cluster_log_to_journald": "false", 2023-11-25T04:23:03.557 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_cluster_log_to_stderr": "false", 2023-11-25T04:23:03.557 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_cluster_log_to_syslog": "default=false", 2023-11-25T04:23:03.557 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2023-11-25T04:23:03.557 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_cluster_log_to_syslog_level": "info", 2023-11-25T04:23:03.557 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_compact_on_bootstrap": "false", 2023-11-25T04:23:03.557 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_compact_on_start": "false", 2023-11-25T04:23:03.557 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_compact_on_trim": "true", 2023-11-25T04:23:03.557 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_con_tracker_persist_interval": "10", 2023-11-25T04:23:03.557 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_con_tracker_score_halflife": "43200", 2023-11-25T04:23:03.557 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_config_key_max_entry_size": "65536", 2023-11-25T04:23:03.557 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_cpu_threads": "4", 2023-11-25T04:23:03.557 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_crush_min_required_version": "hammer", 2023-11-25T04:23:03.557 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_daemon_bytes": "419430400", 2023-11-25T04:23:03.558 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2023-11-25T04:23:03.558 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_data_avail_crit": "5", 2023-11-25T04:23:03.558 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_data_avail_warn": "5", 2023-11-25T04:23:03.558 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_data_size_warn": "16106127360", 2023-11-25T04:23:03.558 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_debug_block_osdmap_trim": "false", 2023-11-25T04:23:03.558 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_debug_deprecated_as_obsolete": "false", 2023-11-25T04:23:03.558 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_debug_dump_json": "false", 2023-11-25T04:23:03.558 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2023-11-25T04:23:03.558 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_debug_dump_transactions": "false", 2023-11-25T04:23:03.558 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_debug_extra_checks": "false", 2023-11-25T04:23:03.558 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_debug_no_initial_persistent_features": "false", 2023-11-25T04:23:03.558 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2023-11-25T04:23:03.559 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_debug_no_require_quincy": "false", 2023-11-25T04:23:03.559 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_debug_no_require_reef": "false", 2023-11-25T04:23:03.559 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2023-11-25T04:23:03.559 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_delta_reset_interval": "10.000000", 2023-11-25T04:23:03.559 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_dns_srv_name": "ceph-mon", 2023-11-25T04:23:03.559 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_down_mkfs_grace": "120", 2023-11-25T04:23:03.559 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_election_default_strategy": "1", 2023-11-25T04:23:03.559 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_election_timeout": "5.000000", 2023-11-25T04:23:03.559 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2023-11-25T04:23:03.559 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_elector_ping_divisor": "2", 2023-11-25T04:23:03.559 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_elector_ping_timeout": "2.000000", 2023-11-25T04:23:03.559 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_enable_op_tracker": "true", 2023-11-25T04:23:03.559 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_fake_pool_delete": "false", 2023-11-25T04:23:03.560 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_force_quorum_join": "false", 2023-11-25T04:23:03.560 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_globalid_prealloc": "10000", 2023-11-25T04:23:03.560 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_health_detail_to_clog": "false", 2023-11-25T04:23:03.560 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_health_log_update_period": "5", 2023-11-25T04:23:03.560 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_health_max_detail": "50", 2023-11-25T04:23:03.560 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_health_to_clog": "true", 2023-11-25T04:23:03.560 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_health_to_clog_interval": "600", 2023-11-25T04:23:03.560 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2023-11-25T04:23:03.560 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_host": "172.21.15.67,[v2:172.21.15.67:3301,v1:172.21.15.67:6790],[v2:172.21.15.67:3302,v1:172.21.15.67:6791]", 2023-11-25T04:23:03.560 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_host_override": "", 2023-11-25T04:23:03.560 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_initial_members": "", 2023-11-25T04:23:03.560 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2023-11-25T04:23:03.561 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2023-11-25T04:23:03.561 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_inject_transaction_delay_max": "10.000000", 2023-11-25T04:23:03.561 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2023-11-25T04:23:03.561 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_keyvaluedb": "rocksdb", 2023-11-25T04:23:03.561 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_lease": "5.000000", 2023-11-25T04:23:03.561 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2023-11-25T04:23:03.561 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_lease_renew_interval_factor": "0.600000", 2023-11-25T04:23:03.561 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_log_full_interval": "50", 2023-11-25T04:23:03.561 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_log_max": "10000", 2023-11-25T04:23:03.561 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_log_max_summary": "50", 2023-11-25T04:23:03.561 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_max_log_entries_per_event": "4096", 2023-11-25T04:23:03.561 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_max_log_epochs": "500", 2023-11-25T04:23:03.561 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_max_mdsmap_epochs": "500", 2023-11-25T04:23:03.562 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_max_mgrmap_epochs": "500", 2023-11-25T04:23:03.562 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_max_osd": "10000", 2023-11-25T04:23:03.562 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_max_pg_per_osd": "10000", 2023-11-25T04:23:03.562 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_max_pool_pg_num": "65536", 2023-11-25T04:23:03.562 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_max_snap_prune_per_epoch": "100", 2023-11-25T04:23:03.562 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_mds_blocklist_interval": "86400.000000", 2023-11-25T04:23:03.562 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_mds_force_trim_to": "0", 2023-11-25T04:23:03.562 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_mds_skip_sanity": "false", 2023-11-25T04:23:03.562 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_memory_autotune": "true", 2023-11-25T04:23:03.562 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_memory_target": "2147483648", 2023-11-25T04:23:03.562 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_mgr_beacon_grace": "30", 2023-11-25T04:23:03.562 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2023-11-25T04:23:03.563 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_mgr_digest_period": "5", 2023-11-25T04:23:03.563 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_mgr_inactive_grace": "60", 2023-11-25T04:23:03.563 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_mgr_mkfs_grace": "240", 2023-11-25T04:23:03.563 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2023-11-25T04:23:03.563 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_min_osdmap_epochs": "500", 2023-11-25T04:23:03.563 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_op_complaint_time": "30", 2023-11-25T04:23:03.563 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_op_history_duration": "600", 2023-11-25T04:23:03.563 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_op_history_size": "20", 2023-11-25T04:23:03.563 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_op_history_slow_op_size": "20", 2023-11-25T04:23:03.563 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_op_history_slow_op_threshold": "10", 2023-11-25T04:23:03.563 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_op_log_threshold": "5", 2023-11-25T04:23:03.563 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_adjust_down_out_interval": "true", 2023-11-25T04:23:03.563 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2023-11-25T04:23:03.564 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2023-11-25T04:23:03.564 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_auto_mark_in": "false", 2023-11-25T04:23:03.564 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_auto_mark_new_in": "true", 2023-11-25T04:23:03.564 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2023-11-25T04:23:03.564 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2023-11-25T04:23:03.564 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_cache_size": "500", 2023-11-25T04:23:03.564 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_cache_size_min": "134217728", 2023-11-25T04:23:03.564 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_crush_smoke_test": "true", 2023-11-25T04:23:03.564 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_destroyed_out_interval": "600", 2023-11-25T04:23:03.564 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_down_out_interval": "600", 2023-11-25T04:23:03.564 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_down_out_subtree_limit": "rack", 2023-11-25T04:23:03.564 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2023-11-25T04:23:03.565 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_force_trim_to": "0", 2023-11-25T04:23:03.565 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_full_ratio": "0.950000", 2023-11-25T04:23:03.565 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2023-11-25T04:23:03.565 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_laggy_halflife": "3600", 2023-11-25T04:23:03.565 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_laggy_max_interval": "300", 2023-11-25T04:23:03.565 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_laggy_weight": "0.300000", 2023-11-25T04:23:03.565 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2023-11-25T04:23:03.565 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_max_creating_pgs": "1024", 2023-11-25T04:23:03.565 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_max_initial_pgs": "1024", 2023-11-25T04:23:03.565 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_min_down_reporters": "2", 2023-11-25T04:23:03.565 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_min_in_ratio": "0.750000", 2023-11-25T04:23:03.565 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_min_up_ratio": "0.300000", 2023-11-25T04:23:03.565 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_nearfull_ratio": "0.850000", 2023-11-25T04:23:03.566 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_prime_pg_temp": "true", 2023-11-25T04:23:03.566 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2023-11-25T04:23:03.566 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2023-11-25T04:23:03.566 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_report_timeout": "900", 2023-11-25T04:23:03.566 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_reporter_subtree_level": "osd", 2023-11-25T04:23:03.566 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2023-11-25T04:23:03.566 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_warn_num_repaired": "10", 2023-11-25T04:23:03.566 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osd_warn_op_age": "32.000000", 2023-11-25T04:23:03.566 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osdmap_full_prune_enabled": "true", 2023-11-25T04:23:03.566 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osdmap_full_prune_interval": "10", 2023-11-25T04:23:03.566 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osdmap_full_prune_min": "10000", 2023-11-25T04:23:03.566 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_osdmap_full_prune_txsize": "100", 2023-11-25T04:23:03.567 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2023-11-25T04:23:03.567 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_pg_stuck_threshold": "60", 2023-11-25T04:23:03.567 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2023-11-25T04:23:03.567 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_pg_warn_min_objects": "10000", 2023-11-25T04:23:03.567 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_pg_warn_min_per_osd": "0", 2023-11-25T04:23:03.567 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_pg_warn_min_pool_objects": "1000", 2023-11-25T04:23:03.567 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_pool_quota_crit_threshold": "0", 2023-11-25T04:23:03.567 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_pool_quota_warn_threshold": "0", 2023-11-25T04:23:03.567 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_probe_timeout": "2.000000", 2023-11-25T04:23:03.568 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_reweight_max_change": "0.050000", 2023-11-25T04:23:03.568 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_reweight_max_osds": "4", 2023-11-25T04:23:03.568 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_reweight_min_bytes_per_osd": "10", 2023-11-25T04:23:03.568 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_reweight_min_pgs_per_osd": "4", 2023-11-25T04:23:03.568 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2023-11-25T04:23:03.568 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2023-11-25T04:23:03.568 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2023-11-25T04:23:03.568 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_scrub_interval": "86400", 2023-11-25T04:23:03.568 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_scrub_max_keys": "100", 2023-11-25T04:23:03.568 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_scrub_timeout": "300", 2023-11-25T04:23:03.568 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_session_timeout": "300", 2023-11-25T04:23:03.568 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_smart_report_timeout": "5", 2023-11-25T04:23:03.568 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_stat_smooth_intervals": "6", 2023-11-25T04:23:03.569 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2023-11-25T04:23:03.569 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_stretch_pool_min_size": "2", 2023-11-25T04:23:03.569 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_stretch_pool_size": "4", 2023-11-25T04:23:03.569 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2023-11-25T04:23:03.569 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_subscribe_interval": "86400.000000", 2023-11-25T04:23:03.569 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_sync_debug": "false", 2023-11-25T04:23:03.569 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_sync_max_payload_keys": "2000", 2023-11-25T04:23:03.569 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_sync_max_payload_size": "1048576", 2023-11-25T04:23:03.569 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_sync_provider_kill_at": "0", 2023-11-25T04:23:03.569 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_sync_requester_kill_at": "0", 2023-11-25T04:23:03.569 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_sync_timeout": "60.000000", 2023-11-25T04:23:03.569 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_target_pg_per_osd": "100", 2023-11-25T04:23:03.570 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_tick_interval": "5", 2023-11-25T04:23:03.570 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_timecheck_interval": "300.000000", 2023-11-25T04:23:03.570 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_timecheck_skew_interval": "30.000000", 2023-11-25T04:23:03.570 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_use_min_delay_socket": "false", 2023-11-25T04:23:03.570 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_warn_older_version_delay": "604800", 2023-11-25T04:23:03.570 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2023-11-25T04:23:03.570 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2023-11-25T04:23:03.570 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2023-11-25T04:23:03.570 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_warn_on_filestore_osds": "false", 2023-11-25T04:23:03.570 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2023-11-25T04:23:03.570 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2023-11-25T04:23:03.571 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2023-11-25T04:23:03.571 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_warn_on_misplaced": "false", 2023-11-25T04:23:03.571 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2023-11-25T04:23:03.571 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_warn_on_older_version": "true", 2023-11-25T04:23:03.571 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2023-11-25T04:23:03.571 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_warn_on_pool_no_app": "true", 2023-11-25T04:23:03.571 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_warn_on_pool_no_redundancy": "false", 2023-11-25T04:23:03.571 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2023-11-25T04:23:03.571 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2023-11-25T04:23:03.571 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2023-11-25T04:23:03.571 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_warn_on_too_few_osds": "false", 2023-11-25T04:23:03.571 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2023-11-25T04:23:03.571 INFO:teuthology.orchestra.run.smithi067.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2023-11-25T04:23:03.572 INFO:teuthology.orchestra.run.smithi067.stdout: "monmap": "", 2023-11-25T04:23:03.572 INFO:teuthology.orchestra.run.smithi067.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2023-11-25T04:23:03.572 INFO:teuthology.orchestra.run.smithi067.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2023-11-25T04:23:03.572 INFO:teuthology.orchestra.run.smithi067.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2023-11-25T04:23:03.572 INFO:teuthology.orchestra.run.smithi067.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2023-11-25T04:23:03.572 INFO:teuthology.orchestra.run.smithi067.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2023-11-25T04:23:03.572 INFO:teuthology.orchestra.run.smithi067.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2023-11-25T04:23:03.572 INFO:teuthology.orchestra.run.smithi067.stdout: "motr_tracing_enabled": "false", 2023-11-25T04:23:03.572 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_async_op_threads": "3", 2023-11-25T04:23:03.572 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_async_rdma_buffer_size": "131072", 2023-11-25T04:23:03.572 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_async_rdma_cm": "false", 2023-11-25T04:23:03.572 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_async_rdma_device_name": "", 2023-11-25T04:23:03.573 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_async_rdma_dscp": "96", 2023-11-25T04:23:03.573 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_async_rdma_enable_hugepage": "false", 2023-11-25T04:23:03.573 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_async_rdma_gid_idx": "0", 2023-11-25T04:23:03.573 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_async_rdma_local_gid": "", 2023-11-25T04:23:03.573 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_async_rdma_polling_us": "1000", 2023-11-25T04:23:03.573 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_async_rdma_port_num": "1", 2023-11-25T04:23:03.573 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_async_rdma_receive_buffers": "32768", 2023-11-25T04:23:03.573 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_async_rdma_receive_queue_len": "4096", 2023-11-25T04:23:03.573 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_async_rdma_roce_ver": "1", 2023-11-25T04:23:03.573 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_async_rdma_send_buffers": "1024", 2023-11-25T04:23:03.573 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_async_rdma_sl": "3", 2023-11-25T04:23:03.573 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_async_rdma_support_srq": "true", 2023-11-25T04:23:03.573 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_async_rdma_type": "ib", 2023-11-25T04:23:03.574 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_async_reap_threshold": "5", 2023-11-25T04:23:03.574 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_bind_before_connect": "false", 2023-11-25T04:23:03.574 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_bind_ipv4": "true", 2023-11-25T04:23:03.574 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_bind_ipv6": "false", 2023-11-25T04:23:03.574 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_bind_msgr1": "true", 2023-11-25T04:23:03.574 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_bind_msgr2": "true", 2023-11-25T04:23:03.574 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_bind_port_max": "7568", 2023-11-25T04:23:03.574 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_bind_port_min": "6800", 2023-11-25T04:23:03.574 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_bind_prefer_ipv4": "false", 2023-11-25T04:23:03.574 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_bind_retry_count": "3", 2023-11-25T04:23:03.574 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_bind_retry_delay": "5", 2023-11-25T04:23:03.574 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_blackhole_client": "false", 2023-11-25T04:23:03.574 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_blackhole_mds": "false", 2023-11-25T04:23:03.575 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_blackhole_mgr": "false", 2023-11-25T04:23:03.575 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_blackhole_mon": "false", 2023-11-25T04:23:03.575 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_blackhole_osd": "false", 2023-11-25T04:23:03.575 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_client_mode": "crc secure", 2023-11-25T04:23:03.575 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_cluster_mode": "crc secure", 2023-11-25T04:23:03.575 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_cluster_type": "", 2023-11-25T04:23:03.575 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_compress_secure": "false", 2023-11-25T04:23:03.575 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_connection_idle_timeout": "900", 2023-11-25T04:23:03.575 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_connection_ready_timeout": "10", 2023-11-25T04:23:03.575 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_crc_data": "true", 2023-11-25T04:23:03.575 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_crc_header": "true", 2023-11-25T04:23:03.575 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_die_on_bad_msg": "false", 2023-11-25T04:23:03.576 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_die_on_bug": "true", 2023-11-25T04:23:03.576 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_die_on_old_message": "true", 2023-11-25T04:23:03.576 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_die_on_skipped_message": "false", 2023-11-25T04:23:03.576 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_die_on_unhandled_msg": "false", 2023-11-25T04:23:03.576 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_dispatch_throttle_bytes": "104857600", 2023-11-25T04:23:03.576 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_dpdk_coremask": "0xF", 2023-11-25T04:23:03.576 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_dpdk_debug_allow_loopback": "false", 2023-11-25T04:23:03.576 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_dpdk_devs_allowlist": "", 2023-11-25T04:23:03.576 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_dpdk_enable_tso": "true", 2023-11-25T04:23:03.576 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2023-11-25T04:23:03.576 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_dpdk_host_ipv4_addr": "", 2023-11-25T04:23:03.576 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_dpdk_hugepages": "", 2023-11-25T04:23:03.576 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_dpdk_hw_flow_control": "true", 2023-11-25T04:23:03.577 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2023-11-25T04:23:03.577 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_dpdk_lro": "true", 2023-11-25T04:23:03.577 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_dpdk_memory_channel": "4", 2023-11-25T04:23:03.577 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2023-11-25T04:23:03.577 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_dpdk_pmd": "", 2023-11-25T04:23:03.577 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_dpdk_port_id": "0", 2023-11-25T04:23:03.577 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2023-11-25T04:23:03.577 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_dump_corrupt_message_level": "1", 2023-11-25T04:23:03.577 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_dump_on_send": "false", 2023-11-25T04:23:03.577 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_initial_backoff": "0.200000", 2023-11-25T04:23:03.577 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_inject_delay_max": "1.000000", 2023-11-25T04:23:03.577 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_inject_delay_probability": "0.000000", 2023-11-25T04:23:03.577 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_inject_delay_type": "", 2023-11-25T04:23:03.578 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_inject_internal_delays": "0.000000", 2023-11-25T04:23:03.578 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_inject_network_congestion": "0", 2023-11-25T04:23:03.578 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_inject_socket_failures": "0", 2023-11-25T04:23:03.578 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_learn_addr_from_peer": "true", 2023-11-25T04:23:03.578 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_max_accept_failures": "4", 2023-11-25T04:23:03.578 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_max_backoff": "15.000000", 2023-11-25T04:23:03.578 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_mon_client_mode": "secure crc", 2023-11-25T04:23:03.578 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_mon_cluster_mode": "secure crc", 2023-11-25T04:23:03.578 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_mon_service_mode": "secure crc", 2023-11-25T04:23:03.578 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_osd_compress_min_size": "1024", 2023-11-25T04:23:03.578 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_osd_compress_mode": "none", 2023-11-25T04:23:03.578 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_osd_compression_algorithm": "snappy", 2023-11-25T04:23:03.579 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2023-11-25T04:23:03.579 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_pq_min_cost": "65536", 2023-11-25T04:23:03.579 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_public_type": "", 2023-11-25T04:23:03.579 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_service_mode": "crc secure", 2023-11-25T04:23:03.579 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_tcp_listen_backlog": "512", 2023-11-25T04:23:03.579 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_tcp_nodelay": "true", 2023-11-25T04:23:03.579 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_tcp_prefetch_max_size": "4096", 2023-11-25T04:23:03.579 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_tcp_rcvbuf": "0", 2023-11-25T04:23:03.579 INFO:teuthology.orchestra.run.smithi067.stdout: "ms_type": "async+posix", 2023-11-25T04:23:03.579 INFO:teuthology.orchestra.run.smithi067.stdout: "no_config_file": "false", 2023-11-25T04:23:03.579 INFO:teuthology.orchestra.run.smithi067.stdout: "objecter_completion_locks_per_session": "32", 2023-11-25T04:23:03.579 INFO:teuthology.orchestra.run.smithi067.stdout: "objecter_debug_inject_relock_delay": "false", 2023-11-25T04:23:03.579 INFO:teuthology.orchestra.run.smithi067.stdout: "objecter_inflight_op_bytes": "104857600", 2023-11-25T04:23:03.580 INFO:teuthology.orchestra.run.smithi067.stdout: "objecter_inflight_ops": "1024", 2023-11-25T04:23:03.580 INFO:teuthology.orchestra.run.smithi067.stdout: "objecter_inject_no_watch_ping": "false", 2023-11-25T04:23:03.580 INFO:teuthology.orchestra.run.smithi067.stdout: "objecter_retry_writes_after_first_reply": "false", 2023-11-25T04:23:03.580 INFO:teuthology.orchestra.run.smithi067.stdout: "objecter_tick_interval": "5.000000", 2023-11-25T04:23:03.580 INFO:teuthology.orchestra.run.smithi067.stdout: "objecter_timeout": "10.000000", 2023-11-25T04:23:03.580 INFO:teuthology.orchestra.run.smithi067.stdout: "objectstore_blackhole": "false", 2023-11-25T04:23:03.580 INFO:teuthology.orchestra.run.smithi067.stdout: "openssl_engine_opts": "", 2023-11-25T04:23:03.580 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_agent_delay_time": "5.000000", 2023-11-25T04:23:03.580 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_agent_hist_halflife": "1000", 2023-11-25T04:23:03.580 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_agent_max_low_ops": "2", 2023-11-25T04:23:03.580 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_agent_max_ops": "4", 2023-11-25T04:23:03.581 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_agent_min_evict_effort": "0.100000", 2023-11-25T04:23:03.581 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_agent_quantize_effort": "0.100000", 2023-11-25T04:23:03.581 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_agent_slop": "0.020000", 2023-11-25T04:23:03.581 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_aggregated_slow_ops_logging": "true", 2023-11-25T04:23:03.581 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_allow_recovery_below_min_size": "true", 2023-11-25T04:23:03.581 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_asio_thread_count": "2", 2023-11-25T04:23:03.581 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_async_recovery_min_cost": "100", 2023-11-25T04:23:03.581 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_auto_mark_unfound_lost": "false", 2023-11-25T04:23:03.581 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_backfill_retry_interval": "30.000000", 2023-11-25T04:23:03.581 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_backfill_scan_max": "512", 2023-11-25T04:23:03.581 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_backfill_scan_min": "64", 2023-11-25T04:23:03.581 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_backoff_on_degraded": "false", 2023-11-25T04:23:03.582 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_backoff_on_peering": "false", 2023-11-25T04:23:03.582 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_backoff_on_unfound": "true", 2023-11-25T04:23:03.582 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_beacon_report_interval": "300", 2023-11-25T04:23:03.582 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_bench_duration": "30", 2023-11-25T04:23:03.582 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_bench_large_size_max_throughput": "104857600", 2023-11-25T04:23:03.582 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_bench_max_block_size": "67108864", 2023-11-25T04:23:03.582 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_bench_small_size_max_iops": "100", 2023-11-25T04:23:03.582 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_blkin_trace_all": "false", 2023-11-25T04:23:03.582 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_blocked_scrub_grace_period": "120", 2023-11-25T04:23:03.582 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2023-11-25T04:23:03.582 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2023-11-25T04:23:03.583 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2023-11-25T04:23:03.583 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_check_for_log_corruption": "false", 2023-11-25T04:23:03.583 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_check_max_object_name_len_on_startup": "true", 2023-11-25T04:23:03.583 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2023-11-25T04:23:03.583 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2023-11-25T04:23:03.583 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2023-11-25T04:23:03.583 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_class_update_on_start": "true", 2023-11-25T04:23:03.583 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_client_message_cap": "256", 2023-11-25T04:23:03.583 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_client_message_size_cap": "524288000", 2023-11-25T04:23:03.583 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_client_op_priority": "63", 2023-11-25T04:23:03.583 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_client_watch_timeout": "30", 2023-11-25T04:23:03.583 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_command_max_records": "256", 2023-11-25T04:23:03.583 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_compact_on_start": "false", 2023-11-25T04:23:03.584 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_copyfrom_max_chunk": "8388608", 2023-11-25T04:23:03.584 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_crush_chooseleaf_type": "0", 2023-11-25T04:23:03.584 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_crush_initial_weight": "-1.000000", 2023-11-25T04:23:03.584 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_crush_update_on_start": "true", 2023-11-25T04:23:03.584 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_crush_update_weight_set": "true", 2023-11-25T04:23:03.584 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2023-11-25T04:23:03.584 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2023-11-25T04:23:03.584 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2023-11-25T04:23:03.584 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_debug_drop_ping_duration": "0", 2023-11-25T04:23:03.584 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_debug_drop_ping_probability": "0.000000", 2023-11-25T04:23:03.584 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_debug_feed_pullee": "-1", 2023-11-25T04:23:03.584 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_debug_inject_copyfrom_error": "false", 2023-11-25T04:23:03.584 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2023-11-25T04:23:03.585 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2023-11-25T04:23:03.585 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_debug_misdirected_ops": "false", 2023-11-25T04:23:03.585 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_debug_no_acting_change": "false", 2023-11-25T04:23:03.585 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_debug_no_purge_strays": "false", 2023-11-25T04:23:03.585 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_debug_op_order": "false", 2023-11-25T04:23:03.585 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_debug_pg_log_writeout": "false", 2023-11-25T04:23:03.585 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_debug_pretend_recovery_active": "false", 2023-11-25T04:23:03.585 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_debug_random_push_read_error": "0.000000", 2023-11-25T04:23:03.585 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2023-11-25T04:23:03.585 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_debug_shutdown": "false", 2023-11-25T04:23:03.585 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2023-11-25T04:23:03.585 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2023-11-25T04:23:03.586 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_debug_trim_objects": "false", 2023-11-25T04:23:03.586 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_debug_verify_cached_snaps": "false", 2023-11-25T04:23:03.586 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_debug_verify_missing_on_start": "false", 2023-11-25T04:23:03.586 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_debug_verify_snaps": "false", 2023-11-25T04:23:03.586 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_debug_verify_stray_on_activate": "false", 2023-11-25T04:23:03.586 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_deep_scrub_interval": "604800.000000", 2023-11-25T04:23:03.586 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_deep_scrub_keys": "1024", 2023-11-25T04:23:03.586 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2023-11-25T04:23:03.586 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2023-11-25T04:23:03.586 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2023-11-25T04:23:03.586 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_deep_scrub_stride": "524288", 2023-11-25T04:23:03.586 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2023-11-25T04:23:03.587 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_default_data_pool_replay_window": "5", 2023-11-25T04:23:03.587 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_default_notify_timeout": "30", 2023-11-25T04:23:03.587 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_delete_sleep": "0.000000", 2023-11-25T04:23:03.587 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_delete_sleep_hdd": "5.000000", 2023-11-25T04:23:03.587 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_delete_sleep_hybrid": "1.000000", 2023-11-25T04:23:03.587 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_delete_sleep_ssd": "1.000000", 2023-11-25T04:23:03.587 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_discard_disconnected_ops": "true", 2023-11-25T04:23:03.587 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_enable_op_tracker": "true", 2023-11-25T04:23:03.587 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2023-11-25T04:23:03.587 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_extblkdev_plugins": "vdo", 2023-11-25T04:23:03.587 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_failsafe_full_ratio": "0.970000", 2023-11-25T04:23:03.587 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_fast_fail_on_connection_refused": "true", 2023-11-25T04:23:03.587 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_fast_info": "true", 2023-11-25T04:23:03.588 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_fast_shutdown": "true", 2023-11-25T04:23:03.588 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_fast_shutdown_notify_mon": "true", 2023-11-25T04:23:03.588 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_fast_shutdown_timeout": "15", 2023-11-25T04:23:03.588 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_find_best_info_ignore_history_les": "false", 2023-11-25T04:23:03.588 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_force_auth_primary_missing_objects": "100", 2023-11-25T04:23:03.588 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2023-11-25T04:23:03.588 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_function_tracing": "false", 2023-11-25T04:23:03.588 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_heartbeat_grace": "20", 2023-11-25T04:23:03.588 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_heartbeat_interval": "6", 2023-11-25T04:23:03.588 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2023-11-25T04:23:03.588 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_heartbeat_min_peers": "10", 2023-11-25T04:23:03.588 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_heartbeat_min_size": "2000", 2023-11-25T04:23:03.589 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_heartbeat_stale": "600", 2023-11-25T04:23:03.589 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2023-11-25T04:23:03.589 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_hit_set_max_size": "100000", 2023-11-25T04:23:03.589 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_hit_set_min_size": "1000", 2023-11-25T04:23:03.589 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_hit_set_namespace": ".ceph-internal", 2023-11-25T04:23:03.589 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_ignore_stale_divergent_priors": "false", 2023-11-25T04:23:03.589 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2023-11-25T04:23:03.589 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_inject_failure_on_pg_removal": "false", 2023-11-25T04:23:03.589 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2023-11-25T04:23:03.589 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_journal_flush_on_shutdown": "true", 2023-11-25T04:23:03.589 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_journal_size": "5120", 2023-11-25T04:23:03.589 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_kill_backfill_at": "0", 2023-11-25T04:23:03.589 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_loop_before_reset_tphandle": "64", 2023-11-25T04:23:03.590 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_map_cache_size": "50", 2023-11-25T04:23:03.590 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_map_dedup": "true", 2023-11-25T04:23:03.590 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_map_message_max": "40", 2023-11-25T04:23:03.590 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_map_message_max_bytes": "10485760", 2023-11-25T04:23:03.590 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_map_share_max_epochs": "40", 2023-11-25T04:23:03.590 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_max_attr_name_len": "100", 2023-11-25T04:23:03.590 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_max_attr_size": "0", 2023-11-25T04:23:03.590 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_max_backfills": "1", 2023-11-25T04:23:03.590 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_max_markdown_count": "5", 2023-11-25T04:23:03.590 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_max_markdown_period": "600", 2023-11-25T04:23:03.590 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_max_object_name_len": "2048", 2023-11-25T04:23:03.590 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_max_object_namespace_len": "256", 2023-11-25T04:23:03.591 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_max_object_size": "134217728", 2023-11-25T04:23:03.591 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2023-11-25T04:23:03.591 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_max_omap_entries_per_request": "1024", 2023-11-25T04:23:03.591 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_max_pg_blocked_by": "16", 2023-11-25T04:23:03.591 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_max_pg_log_entries": "10000", 2023-11-25T04:23:03.591 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2023-11-25T04:23:03.591 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_max_pgls": "1024", 2023-11-25T04:23:03.591 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_max_push_cost": "8388608", 2023-11-25T04:23:03.591 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_max_push_objects": "10", 2023-11-25T04:23:03.591 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_max_scrubs": "3", 2023-11-25T04:23:03.591 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2023-11-25T04:23:03.591 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_max_trimming_pgs": "2", 2023-11-25T04:23:03.591 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_max_write_op_reply_len": "64", 2023-11-25T04:23:03.592 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_max_write_size": "90", 2023-11-25T04:23:03.592 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2023-11-25T04:23:03.592 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2023-11-25T04:23:03.592 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2023-11-25T04:23:03.592 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2023-11-25T04:23:03.592 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2023-11-25T04:23:03.592 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2023-11-25T04:23:03.592 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2023-11-25T04:23:03.592 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_mclock_override_recovery_settings": "false", 2023-11-25T04:23:03.592 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_mclock_profile": "balanced", 2023-11-25T04:23:03.592 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2023-11-25T04:23:03.592 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2023-11-25T04:23:03.593 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2023-11-25T04:23:03.593 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2023-11-25T04:23:03.593 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2023-11-25T04:23:03.593 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2023-11-25T04:23:03.593 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2023-11-25T04:23:03.593 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2023-11-25T04:23:03.593 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2023-11-25T04:23:03.593 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_mclock_scheduler_client_wgt": "1", 2023-11-25T04:23:03.593 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_mclock_skip_benchmark": "false", 2023-11-25T04:23:03.593 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_memory_base": "805306368", 2023-11-25T04:23:03.593 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_memory_cache_min": "134217728", 2023-11-25T04:23:03.593 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_memory_cache_resize_interval": "1.000000", 2023-11-25T04:23:03.593 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_memory_expected_fragmentation": "0.150000", 2023-11-25T04:23:03.594 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_memory_target": "4294967296", 2023-11-25T04:23:03.594 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_memory_target_autotune": "false", 2023-11-25T04:23:03.594 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2023-11-25T04:23:03.594 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_min_pg_log_entries": "250", 2023-11-25T04:23:03.594 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_min_recovery_priority": "0", 2023-11-25T04:23:03.594 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_mon_heartbeat_interval": "30", 2023-11-25T04:23:03.594 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2023-11-25T04:23:03.594 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_mon_report_interval": "5", 2023-11-25T04:23:03.594 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_mon_report_max_in_flight": "2", 2023-11-25T04:23:03.594 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_mon_shutdown_timeout": "5.000000", 2023-11-25T04:23:03.594 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_num_cache_shards": "32", 2023-11-25T04:23:03.595 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_num_op_tracker_shard": "32", 2023-11-25T04:23:03.595 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_numa_auto_affinity": "true", 2023-11-25T04:23:03.595 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_numa_node": "-1", 2023-11-25T04:23:03.595 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_numa_prefer_iface": "true", 2023-11-25T04:23:03.595 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_object_clean_region_max_num_intervals": "10", 2023-11-25T04:23:03.595 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_objecter_finishers": "1", 2023-11-25T04:23:03.595 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_objectstore": "bluestore", 2023-11-25T04:23:03.595 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_objectstore_fuse": "false", 2023-11-25T04:23:03.595 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_objectstore_tracing": "false", 2023-11-25T04:23:03.595 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_op_complaint_time": "30.000000", 2023-11-25T04:23:03.595 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_op_history_duration": "600", 2023-11-25T04:23:03.595 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_op_history_size": "20", 2023-11-25T04:23:03.596 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_op_history_slow_op_size": "20", 2023-11-25T04:23:03.596 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2023-11-25T04:23:03.596 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_op_log_threshold": "5", 2023-11-25T04:23:03.596 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_op_num_shards": "0", 2023-11-25T04:23:03.596 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_op_num_shards_hdd": "5", 2023-11-25T04:23:03.596 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_op_num_shards_ssd": "8", 2023-11-25T04:23:03.596 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_op_num_threads_per_shard": "0", 2023-11-25T04:23:03.596 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2023-11-25T04:23:03.596 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2023-11-25T04:23:03.596 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2023-11-25T04:23:03.596 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_op_pq_min_cost": "65536", 2023-11-25T04:23:03.596 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_op_queue": "mclock_scheduler", 2023-11-25T04:23:03.597 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_op_queue_cut_off": "high", 2023-11-25T04:23:03.597 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_op_thread_suicide_timeout": "150", 2023-11-25T04:23:03.597 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_op_thread_timeout": "15", 2023-11-25T04:23:03.597 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_open_classes_on_start": "true", 2023-11-25T04:23:03.597 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_os_flags": "0", 2023-11-25T04:23:03.597 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_peering_op_priority": "255", 2023-11-25T04:23:03.597 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pg_delete_cost": "1048576", 2023-11-25T04:23:03.597 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pg_delete_priority": "5", 2023-11-25T04:23:03.597 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2023-11-25T04:23:03.597 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2023-11-25T04:23:03.597 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pg_log_dups_tracked": "3000", 2023-11-25T04:23:03.597 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pg_log_trim_max": "10000", 2023-11-25T04:23:03.597 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pg_log_trim_min": "100", 2023-11-25T04:23:03.598 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2023-11-25T04:23:03.598 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pg_object_context_cache_count": "64", 2023-11-25T04:23:03.598 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pg_stat_report_interval_max": "500", 2023-11-25T04:23:03.598 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2023-11-25T04:23:03.598 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pool_default_cache_min_evict_age": "0", 2023-11-25T04:23:03.598 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pool_default_cache_min_flush_age": "0", 2023-11-25T04:23:03.598 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2023-11-25T04:23:03.598 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2023-11-25T04:23:03.598 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2023-11-25T04:23:03.598 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pool_default_crimson": "false", 2023-11-25T04:23:03.598 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pool_default_crush_rule": "-1", 2023-11-25T04:23:03.598 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pool_default_ec_fast_read": "false", 2023-11-25T04:23:03.599 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2023-11-25T04:23:03.599 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pool_default_flag_bulk": "false", 2023-11-25T04:23:03.599 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pool_default_flag_hashpspool": "true", 2023-11-25T04:23:03.599 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pool_default_flag_nodelete": "false", 2023-11-25T04:23:03.599 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pool_default_flag_nopgchange": "false", 2023-11-25T04:23:03.599 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pool_default_flag_nosizechange": "false", 2023-11-25T04:23:03.599 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pool_default_flags": "0", 2023-11-25T04:23:03.599 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2023-11-25T04:23:03.599 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pool_default_min_size": "0", 2023-11-25T04:23:03.599 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2023-11-25T04:23:03.599 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pool_default_pg_num": "32", 2023-11-25T04:23:03.599 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pool_default_pgp_num": "0", 2023-11-25T04:23:03.599 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2023-11-25T04:23:03.600 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pool_default_size": "2", 2023-11-25T04:23:03.600 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pool_default_type": "replicated", 2023-11-25T04:23:03.600 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2023-11-25T04:23:03.600 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pool_use_gmt_hitset": "true", 2023-11-25T04:23:03.600 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_push_per_object_cost": "1000", 2023-11-25T04:23:03.600 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_read_ec_check_for_errors": "false", 2023-11-25T04:23:03.600 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_recover_clone_overlap": "true", 2023-11-25T04:23:03.600 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_recover_clone_overlap_limit": "10", 2023-11-25T04:23:03.600 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_recovery_cost": "20971520", 2023-11-25T04:23:03.600 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_recovery_delay_start": "0.000000", 2023-11-25T04:23:03.600 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_recovery_max_active": "0", 2023-11-25T04:23:03.600 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_recovery_max_active_hdd": "3", 2023-11-25T04:23:03.601 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_recovery_max_active_ssd": "10", 2023-11-25T04:23:03.601 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_recovery_max_chunk": "8388608", 2023-11-25T04:23:03.601 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2023-11-25T04:23:03.601 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_recovery_max_single_start": "1", 2023-11-25T04:23:03.601 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_recovery_op_priority": "3", 2023-11-25T04:23:03.601 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_recovery_op_warn_multiple": "16", 2023-11-25T04:23:03.601 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_recovery_priority": "5", 2023-11-25T04:23:03.601 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_recovery_retry_interval": "30.000000", 2023-11-25T04:23:03.601 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_recovery_sleep": "0.000000", 2023-11-25T04:23:03.601 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_recovery_sleep_hdd": "0.100000", 2023-11-25T04:23:03.601 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2023-11-25T04:23:03.601 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_recovery_sleep_ssd": "0.000000", 2023-11-25T04:23:03.602 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_repair_during_recovery": "false", 2023-11-25T04:23:03.602 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_requested_scrub_priority": "120", 2023-11-25T04:23:03.602 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2023-11-25T04:23:03.602 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_rollback_to_cluster_snap": "", 2023-11-25T04:23:03.602 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_scrub_auto_repair": "false", 2023-11-25T04:23:03.602 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_scrub_auto_repair_num_errors": "5", 2023-11-25T04:23:03.602 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_scrub_backoff_ratio": "0.660000", 2023-11-25T04:23:03.602 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_scrub_begin_hour": "0", 2023-11-25T04:23:03.602 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_scrub_begin_week_day": "0", 2023-11-25T04:23:03.602 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_scrub_chunk_max": "25", 2023-11-25T04:23:03.602 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_scrub_chunk_min": "5", 2023-11-25T04:23:03.602 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_scrub_cost": "52428800", 2023-11-25T04:23:03.603 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_scrub_during_recovery": "false", 2023-11-25T04:23:03.603 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_scrub_end_hour": "0", 2023-11-25T04:23:03.603 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_scrub_end_week_day": "0", 2023-11-25T04:23:03.603 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_scrub_event_cost": "4096", 2023-11-25T04:23:03.603 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_scrub_extended_sleep": "0.000000", 2023-11-25T04:23:03.603 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2023-11-25T04:23:03.603 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_scrub_invalid_stats": "true", 2023-11-25T04:23:03.603 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_scrub_load_threshold": "0.500000", 2023-11-25T04:23:03.603 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_scrub_max_interval": "604800.000000", 2023-11-25T04:23:03.603 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_scrub_max_preemptions": "5", 2023-11-25T04:23:03.603 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_scrub_min_interval": "86400.000000", 2023-11-25T04:23:03.603 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_scrub_priority": "5", 2023-11-25T04:23:03.604 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_scrub_reservation_timeout": "300000", 2023-11-25T04:23:03.604 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_scrub_sleep": "0.000000", 2023-11-25T04:23:03.604 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_scrub_slow_reservation_response": "30000", 2023-11-25T04:23:03.604 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_shallow_scrub_chunk_max": "100", 2023-11-25T04:23:03.604 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_shallow_scrub_chunk_min": "50", 2023-11-25T04:23:03.604 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_shutdown_pgref_assert": "false", 2023-11-25T04:23:03.604 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_skip_data_digest": "false", 2023-11-25T04:23:03.604 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_smart_report_timeout": "5", 2023-11-25T04:23:03.604 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_snap_trim_cost": "1048576", 2023-11-25T04:23:03.604 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_snap_trim_priority": "5", 2023-11-25T04:23:03.604 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_snap_trim_sleep": "0.000000", 2023-11-25T04:23:03.604 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2023-11-25T04:23:03.604 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2023-11-25T04:23:03.605 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2023-11-25T04:23:03.605 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_stats_update_period_not_scrubbing": "120", 2023-11-25T04:23:03.605 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_stats_update_period_scrubbing": "15", 2023-11-25T04:23:03.605 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2023-11-25T04:23:03.605 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_target_transaction_size": "30", 2023-11-25T04:23:03.605 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_tier_default_cache_hit_set_count": "4", 2023-11-25T04:23:03.605 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2023-11-25T04:23:03.605 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2023-11-25T04:23:03.605 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2023-11-25T04:23:03.605 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2023-11-25T04:23:03.605 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2023-11-25T04:23:03.605 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2023-11-25T04:23:03.606 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_tier_default_cache_mode": "writeback", 2023-11-25T04:23:03.606 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2023-11-25T04:23:03.606 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_tier_promote_max_objects_sec": "25", 2023-11-25T04:23:03.606 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_tracing": "false", 2023-11-25T04:23:03.606 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_use_stale_snap": "false", 2023-11-25T04:23:03.606 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2023-11-25T04:23:03.606 INFO:teuthology.orchestra.run.smithi067.stdout: "osdc_blkin_trace_all": "false", 2023-11-25T04:23:03.606 INFO:teuthology.orchestra.run.smithi067.stdout: "paxos_kill_at": "0", 2023-11-25T04:23:03.606 INFO:teuthology.orchestra.run.smithi067.stdout: "paxos_max_join_drift": "10", 2023-11-25T04:23:03.606 INFO:teuthology.orchestra.run.smithi067.stdout: "paxos_min": "500", 2023-11-25T04:23:03.606 INFO:teuthology.orchestra.run.smithi067.stdout: "paxos_min_wait": "0.050000", 2023-11-25T04:23:03.606 INFO:teuthology.orchestra.run.smithi067.stdout: "paxos_propose_interval": "1.000000", 2023-11-25T04:23:03.607 INFO:teuthology.orchestra.run.smithi067.stdout: "paxos_service_trim_max": "500", 2023-11-25T04:23:03.607 INFO:teuthology.orchestra.run.smithi067.stdout: "paxos_service_trim_max_multiplier": "20", 2023-11-25T04:23:03.607 INFO:teuthology.orchestra.run.smithi067.stdout: "paxos_service_trim_min": "250", 2023-11-25T04:23:03.607 INFO:teuthology.orchestra.run.smithi067.stdout: "paxos_stash_full_interval": "25", 2023-11-25T04:23:03.607 INFO:teuthology.orchestra.run.smithi067.stdout: "paxos_trim_max": "500", 2023-11-25T04:23:03.607 INFO:teuthology.orchestra.run.smithi067.stdout: "paxos_trim_min": "250", 2023-11-25T04:23:03.607 INFO:teuthology.orchestra.run.smithi067.stdout: "perf": "true", 2023-11-25T04:23:03.607 INFO:teuthology.orchestra.run.smithi067.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2023-11-25T04:23:03.607 INFO:teuthology.orchestra.run.smithi067.stdout: "plugin_crypto_accelerator": "crypto_isal", 2023-11-25T04:23:03.607 INFO:teuthology.orchestra.run.smithi067.stdout: "plugin_dir": "/usr/lib64/ceph", 2023-11-25T04:23:03.607 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addr": "-", 2023-11-25T04:23:03.607 INFO:teuthology.orchestra.run.smithi067.stdout: "public_addrv": "", 2023-11-25T04:23:03.608 INFO:teuthology.orchestra.run.smithi067.stdout: "public_bind_addr": "-", 2023-11-25T04:23:03.608 INFO:teuthology.orchestra.run.smithi067.stdout: "public_network": "", 2023-11-25T04:23:03.608 INFO:teuthology.orchestra.run.smithi067.stdout: "public_network_interface": "", 2023-11-25T04:23:03.608 INFO:teuthology.orchestra.run.smithi067.stdout: "qat_compressor_enabled": "false", 2023-11-25T04:23:03.608 INFO:teuthology.orchestra.run.smithi067.stdout: "qat_compressor_session_max_number": "256", 2023-11-25T04:23:03.608 INFO:teuthology.orchestra.run.smithi067.stdout: "rados_mon_op_timeout": "0", 2023-11-25T04:23:03.608 INFO:teuthology.orchestra.run.smithi067.stdout: "rados_osd_op_timeout": "0", 2023-11-25T04:23:03.608 INFO:teuthology.orchestra.run.smithi067.stdout: "rados_tracing": "false", 2023-11-25T04:23:03.608 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_atime_update_interval": "60", 2023-11-25T04:23:03.608 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2023-11-25T04:23:03.608 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_balance_parent_reads": "false", 2023-11-25T04:23:03.608 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_balance_snap_reads": "false", 2023-11-25T04:23:03.609 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_blkin_trace_all": "false", 2023-11-25T04:23:03.609 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_blocklist_expire_seconds": "0", 2023-11-25T04:23:03.609 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_blocklist_on_break_lock": "true", 2023-11-25T04:23:03.609 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_cache": "true", 2023-11-25T04:23:03.609 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_cache_block_writes_upfront": "false", 2023-11-25T04:23:03.609 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_cache_max_dirty": "25165824", 2023-11-25T04:23:03.609 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_cache_max_dirty_age": "1.000000", 2023-11-25T04:23:03.609 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_cache_max_dirty_object": "0", 2023-11-25T04:23:03.609 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_cache_policy": "writearound", 2023-11-25T04:23:03.609 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_cache_size": "33554432", 2023-11-25T04:23:03.609 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_cache_target_dirty": "16777216", 2023-11-25T04:23:03.609 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_cache_writethrough_until_flush": "true", 2023-11-25T04:23:03.609 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_clone_copy_on_read": "false", 2023-11-25T04:23:03.610 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_compression_hint": "none", 2023-11-25T04:23:03.610 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_concurrent_management_ops": "10", 2023-11-25T04:23:03.610 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_config_pool_override_update_timestamp": "0", 2023-11-25T04:23:03.610 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_default_clone_format": "auto", 2023-11-25T04:23:03.610 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_default_data_pool": "", 2023-11-25T04:23:03.610 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_default_features": "61", 2023-11-25T04:23:03.610 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_default_format": "2", 2023-11-25T04:23:03.610 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_default_map_options": "", 2023-11-25T04:23:03.610 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_default_order": "22", 2023-11-25T04:23:03.610 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_default_pool": "rbd", 2023-11-25T04:23:03.610 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2023-11-25T04:23:03.610 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_default_stripe_count": "0", 2023-11-25T04:23:03.611 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_default_stripe_unit": "0", 2023-11-25T04:23:03.611 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_disable_zero_copy_writes": "true", 2023-11-25T04:23:03.611 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_discard_granularity_bytes": "65536", 2023-11-25T04:23:03.611 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_discard_on_zeroed_write_same": "true", 2023-11-25T04:23:03.611 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_enable_alloc_hint": "true", 2023-11-25T04:23:03.611 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2023-11-25T04:23:03.611 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_io_scheduler": "simple", 2023-11-25T04:23:03.611 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2023-11-25T04:23:03.611 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_journal_commit_age": "5.000000", 2023-11-25T04:23:03.611 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2023-11-25T04:23:03.611 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_journal_max_payload_bytes": "16384", 2023-11-25T04:23:03.611 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_journal_object_flush_age": "0.000000", 2023-11-25T04:23:03.611 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_journal_object_flush_bytes": "1048576", 2023-11-25T04:23:03.612 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_journal_object_flush_interval": "0", 2023-11-25T04:23:03.612 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2023-11-25T04:23:03.612 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2023-11-25T04:23:03.612 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_journal_order": "24", 2023-11-25T04:23:03.612 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_journal_pool": "", 2023-11-25T04:23:03.612 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_journal_splay_width": "4", 2023-11-25T04:23:03.612 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_localize_parent_reads": "false", 2023-11-25T04:23:03.612 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_localize_snap_reads": "false", 2023-11-25T04:23:03.612 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2023-11-25T04:23:03.612 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2023-11-25T04:23:03.612 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2023-11-25T04:23:03.612 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2023-11-25T04:23:03.612 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2023-11-25T04:23:03.613 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2023-11-25T04:23:03.613 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_mirror_image_policy_type": "simple", 2023-11-25T04:23:03.613 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2023-11-25T04:23:03.613 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_mirror_image_state_check_interval": "30", 2023-11-25T04:23:03.613 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2023-11-25T04:23:03.613 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2023-11-25T04:23:03.613 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2023-11-25T04:23:03.613 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2023-11-25T04:23:03.613 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2023-11-25T04:23:03.613 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_mirror_memory_autotune": "true", 2023-11-25T04:23:03.613 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_mirror_memory_base": "805306368", 2023-11-25T04:23:03.613 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2023-11-25T04:23:03.613 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_mirror_memory_cache_min": "134217728", 2023-11-25T04:23:03.614 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2023-11-25T04:23:03.614 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2023-11-25T04:23:03.614 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_mirror_memory_target": "4294967296", 2023-11-25T04:23:03.614 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_mirror_perf_stats_prio": "5", 2023-11-25T04:23:03.614 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2023-11-25T04:23:03.614 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2023-11-25T04:23:03.614 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_mirroring_delete_delay": "0", 2023-11-25T04:23:03.614 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2023-11-25T04:23:03.614 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_mirroring_replay_delay": "0", 2023-11-25T04:23:03.614 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2023-11-25T04:23:03.614 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2023-11-25T04:23:03.614 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_move_to_trash_on_remove": "false", 2023-11-25T04:23:03.615 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2023-11-25T04:23:03.615 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_mtime_update_interval": "60", 2023-11-25T04:23:03.615 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_non_blocking_aio": "true", 2023-11-25T04:23:03.615 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_op_thread_timeout": "60", 2023-11-25T04:23:03.615 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_op_threads": "1", 2023-11-25T04:23:03.615 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_parent_cache_enabled": "false", 2023-11-25T04:23:03.615 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_persistent_cache_mode": "disabled", 2023-11-25T04:23:03.615 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_persistent_cache_path": "/tmp", 2023-11-25T04:23:03.615 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_persistent_cache_size": "1073741824", 2023-11-25T04:23:03.615 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_plugins": "", 2023-11-25T04:23:03.615 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_qos_bps_burst": "0", 2023-11-25T04:23:03.615 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_qos_bps_burst_seconds": "1", 2023-11-25T04:23:03.616 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_qos_bps_limit": "0", 2023-11-25T04:23:03.616 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_qos_exclude_ops": "0", 2023-11-25T04:23:03.616 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_qos_iops_burst": "0", 2023-11-25T04:23:03.616 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_qos_iops_burst_seconds": "1", 2023-11-25T04:23:03.616 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_qos_iops_limit": "0", 2023-11-25T04:23:03.616 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_qos_read_bps_burst": "0", 2023-11-25T04:23:03.616 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2023-11-25T04:23:03.616 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_qos_read_bps_limit": "0", 2023-11-25T04:23:03.616 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_qos_read_iops_burst": "0", 2023-11-25T04:23:03.616 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2023-11-25T04:23:03.616 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_qos_read_iops_limit": "0", 2023-11-25T04:23:03.616 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_qos_schedule_tick_min": "50", 2023-11-25T04:23:03.616 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_qos_write_bps_burst": "0", 2023-11-25T04:23:03.617 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2023-11-25T04:23:03.617 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_qos_write_bps_limit": "0", 2023-11-25T04:23:03.617 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_qos_write_iops_burst": "0", 2023-11-25T04:23:03.617 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2023-11-25T04:23:03.617 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_qos_write_iops_limit": "0", 2023-11-25T04:23:03.617 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_quiesce_notification_attempts": "10", 2023-11-25T04:23:03.617 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_read_from_replica_policy": "default", 2023-11-25T04:23:03.617 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2023-11-25T04:23:03.617 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_readahead_max_bytes": "524288", 2023-11-25T04:23:03.617 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_readahead_trigger_requests": "10", 2023-11-25T04:23:03.617 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_request_timed_out_seconds": "30", 2023-11-25T04:23:03.617 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_skip_partial_discard": "true", 2023-11-25T04:23:03.617 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2023-11-25T04:23:03.618 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_tracing": "false", 2023-11-25T04:23:03.618 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_validate_names": "true", 2023-11-25T04:23:03.618 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_validate_pool": "true", 2023-11-25T04:23:03.618 INFO:teuthology.orchestra.run.smithi067.stdout: "restapi_base_url": "", 2023-11-25T04:23:03.618 INFO:teuthology.orchestra.run.smithi067.stdout: "restapi_log_level": "", 2023-11-25T04:23:03.618 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_acl_grants_max_num": "100", 2023-11-25T04:23:03.618 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_admin_entry": "admin", 2023-11-25T04:23:03.618 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2023-11-25T04:23:03.618 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_backend_store": "rados", 2023-11-25T04:23:03.618 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_barbican_url": "", 2023-11-25T04:23:03.618 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_beast_enable_async": "true", 2023-11-25T04:23:03.618 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_bucket_counters_cache": "false", 2023-11-25T04:23:03.619 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_bucket_counters_cache_size": "10000", 2023-11-25T04:23:03.619 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2023-11-25T04:23:03.619 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_bucket_default_quota_max_size": "-1", 2023-11-25T04:23:03.619 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_bucket_index_max_aio": "128", 2023-11-25T04:23:03.619 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2023-11-25T04:23:03.619 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_bucket_quota_cache_size": "10000", 2023-11-25T04:23:03.619 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_bucket_quota_ttl": "600", 2023-11-25T04:23:03.619 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_bucket_sync_spawn_window": "20", 2023-11-25T04:23:03.619 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_cache_enabled": "true", 2023-11-25T04:23:03.619 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_cache_expiry_interval": "900", 2023-11-25T04:23:03.619 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_cache_lru_size": "10000", 2023-11-25T04:23:03.619 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_config_store": "rados", 2023-11-25T04:23:03.619 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_content_length_compat": "false", 2023-11-25T04:23:03.620 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_copy_obj_progress": "true", 2023-11-25T04:23:03.620 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2023-11-25T04:23:03.620 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_cors_rules_max_num": "100", 2023-11-25T04:23:03.620 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_cross_domain_policy": "", 2023-11-25T04:23:03.620 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_crypt_default_encryption_key": "", 2023-11-25T04:23:03.620 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_crypt_kmip_addr": "", 2023-11-25T04:23:03.620 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_crypt_kmip_ca_path": "", 2023-11-25T04:23:03.620 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_crypt_kmip_client_cert": "", 2023-11-25T04:23:03.620 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_crypt_kmip_client_key": "", 2023-11-25T04:23:03.620 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_crypt_kmip_kms_key_template": "", 2023-11-25T04:23:03.620 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_crypt_kmip_password": "", 2023-11-25T04:23:03.620 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2023-11-25T04:23:03.620 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_crypt_kmip_username": "", 2023-11-25T04:23:03.621 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_crypt_require_ssl": "true", 2023-11-25T04:23:03.621 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2023-11-25T04:23:03.621 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2023-11-25T04:23:03.621 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_crypt_sse_s3_backend": "vault", 2023-11-25T04:23:03.621 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2023-11-25T04:23:03.621 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2023-11-25T04:23:03.621 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2023-11-25T04:23:03.621 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2023-11-25T04:23:03.621 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2023-11-25T04:23:03.621 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2023-11-25T04:23:03.621 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2023-11-25T04:23:03.621 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2023-11-25T04:23:03.621 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2023-11-25T04:23:03.622 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2023-11-25T04:23:03.622 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2023-11-25T04:23:03.622 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_crypt_suppress_logs": "true", 2023-11-25T04:23:03.622 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_crypt_vault_addr": "", 2023-11-25T04:23:03.622 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_crypt_vault_auth": "token", 2023-11-25T04:23:03.622 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_crypt_vault_namespace": "", 2023-11-25T04:23:03.622 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_crypt_vault_prefix": "", 2023-11-25T04:23:03.622 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_crypt_vault_secret_engine": "transit", 2023-11-25T04:23:03.622 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_crypt_vault_ssl_cacert": "", 2023-11-25T04:23:03.622 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2023-11-25T04:23:03.622 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2023-11-25T04:23:03.622 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_crypt_vault_token_file": "", 2023-11-25T04:23:03.623 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_crypt_vault_verify_ssl": "true", 2023-11-25T04:23:03.623 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_curl_buffersize": "524288", 2023-11-25T04:23:03.623 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_curl_low_speed_limit": "1024", 2023-11-25T04:23:03.623 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_curl_low_speed_time": "300", 2023-11-25T04:23:03.623 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_curl_tcp_keepalive": "0", 2023-11-25T04:23:03.623 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_curl_wait_timeout_ms": "1000", 2023-11-25T04:23:03.623 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2023-11-25T04:23:03.623 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2023-11-25T04:23:03.623 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2023-11-25T04:23:03.623 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2023-11-25T04:23:03.623 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_d3n_l1_fadvise": "4", 2023-11-25T04:23:03.623 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2023-11-25T04:23:03.623 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_d3n_libaio_aio_num": "64", 2023-11-25T04:23:03.624 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_d3n_libaio_aio_threads": "20", 2023-11-25T04:23:03.624 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_d4n_host": "127.0.0.1", 2023-11-25T04:23:03.624 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_d4n_port": "6379", 2023-11-25T04:23:03.624 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2023-11-25T04:23:03.624 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_data_log_changes_size": "1000", 2023-11-25T04:23:03.624 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_data_log_num_shards": "128", 2023-11-25T04:23:03.624 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_data_log_obj_prefix": "data_log", 2023-11-25T04:23:03.624 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_data_log_window": "30", 2023-11-25T04:23:03.624 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_data_notify_interval_msec": "0", 2023-11-25T04:23:03.624 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_data_sync_poll_interval": "20", 2023-11-25T04:23:03.624 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_data_sync_spawn_window": "20", 2023-11-25T04:23:03.624 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2023-11-25T04:23:03.624 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_debug_inject_set_olh_err": "0", 2023-11-25T04:23:03.625 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_default_data_log_backing": "fifo", 2023-11-25T04:23:03.625 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_default_realm_info_oid": "default.realm", 2023-11-25T04:23:03.625 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_default_region_info_oid": "default.region", 2023-11-25T04:23:03.625 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_default_zone_info_oid": "default.zone", 2023-11-25T04:23:03.625 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2023-11-25T04:23:03.625 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_defer_to_bucket_acls": "", 2023-11-25T04:23:03.625 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_delete_multi_obj_max_num": "1000", 2023-11-25T04:23:03.625 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_dmclock_admin_lim": "0.000000", 2023-11-25T04:23:03.625 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_dmclock_admin_res": "100.000000", 2023-11-25T04:23:03.625 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2023-11-25T04:23:03.625 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_dmclock_auth_lim": "0.000000", 2023-11-25T04:23:03.625 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_dmclock_auth_res": "200.000000", 2023-11-25T04:23:03.626 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2023-11-25T04:23:03.626 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_dmclock_data_lim": "0.000000", 2023-11-25T04:23:03.626 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_dmclock_data_res": "500.000000", 2023-11-25T04:23:03.626 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_dmclock_data_wgt": "500.000000", 2023-11-25T04:23:03.626 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2023-11-25T04:23:03.626 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_dmclock_metadata_res": "500.000000", 2023-11-25T04:23:03.626 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2023-11-25T04:23:03.626 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_dns_name": "", 2023-11-25T04:23:03.626 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_dns_s3website_name": "", 2023-11-25T04:23:03.626 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_dynamic_resharding": "true", 2023-11-25T04:23:03.626 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2023-11-25T04:23:03.626 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_enable_gc_threads": "true", 2023-11-25T04:23:03.626 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_enable_lc_threads": "true", 2023-11-25T04:23:03.627 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_enable_ops_log": "false", 2023-11-25T04:23:03.627 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_enable_quota_threads": "true", 2023-11-25T04:23:03.627 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_enable_static_website": "false", 2023-11-25T04:23:03.627 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_enable_usage_log": "false", 2023-11-25T04:23:03.627 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_enforce_swift_acls": "true", 2023-11-25T04:23:03.627 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_exit_timeout_secs": "120", 2023-11-25T04:23:03.627 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_expose_bucket": "false", 2023-11-25T04:23:03.627 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_extended_http_attrs": "", 2023-11-25T04:23:03.627 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_filter": "none", 2023-11-25T04:23:03.627 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2023-11-25T04:23:03.627 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_frontends": "beast port=7480", 2023-11-25T04:23:03.627 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_gc_max_concurrent_io": "10", 2023-11-25T04:23:03.627 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_gc_max_deferred": "50", 2023-11-25T04:23:03.628 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2023-11-25T04:23:03.628 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_gc_max_objs": "32", 2023-11-25T04:23:03.628 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_gc_max_queue_size": "134213632", 2023-11-25T04:23:03.628 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_gc_max_trim_chunk": "16", 2023-11-25T04:23:03.628 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_gc_obj_min_wait": "7200", 2023-11-25T04:23:03.628 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_gc_processor_max_time": "3600", 2023-11-25T04:23:03.628 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_gc_processor_period": "3600", 2023-11-25T04:23:03.628 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_get_obj_max_req_size": "4194304", 2023-11-25T04:23:03.628 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_get_obj_window_size": "16777216", 2023-11-25T04:23:03.628 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_healthcheck_disabling_path": "", 2023-11-25T04:23:03.628 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_ignore_get_invalid_range": "false", 2023-11-25T04:23:03.628 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_init_timeout": "300", 2023-11-25T04:23:03.629 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2023-11-25T04:23:03.629 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2023-11-25T04:23:03.629 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_keystone_accepted_admin_roles": "", 2023-11-25T04:23:03.629 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_keystone_accepted_reader_roles": "", 2023-11-25T04:23:03.629 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2023-11-25T04:23:03.629 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_keystone_admin_domain": "", 2023-11-25T04:23:03.629 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_keystone_admin_password": "", 2023-11-25T04:23:03.629 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_keystone_admin_password_path": "", 2023-11-25T04:23:03.629 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_keystone_admin_project": "", 2023-11-25T04:23:03.629 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_keystone_admin_tenant": "", 2023-11-25T04:23:03.629 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_keystone_admin_token": "", 2023-11-25T04:23:03.629 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_keystone_admin_token_path": "", 2023-11-25T04:23:03.629 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_keystone_admin_user": "", 2023-11-25T04:23:03.630 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_keystone_api_version": "2", 2023-11-25T04:23:03.630 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_keystone_barbican_domain": "", 2023-11-25T04:23:03.630 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_keystone_barbican_password": "", 2023-11-25T04:23:03.630 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_keystone_barbican_project": "", 2023-11-25T04:23:03.630 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_keystone_barbican_tenant": "", 2023-11-25T04:23:03.630 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_keystone_barbican_user": "", 2023-11-25T04:23:03.630 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2023-11-25T04:23:03.630 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_keystone_implicit_tenants": "false", 2023-11-25T04:23:03.630 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2023-11-25T04:23:03.630 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_keystone_service_token_enabled": "false", 2023-11-25T04:23:03.630 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_keystone_token_cache_size": "10000", 2023-11-25T04:23:03.630 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_keystone_url": "", 2023-11-25T04:23:03.630 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_keystone_verify_ssl": "true", 2023-11-25T04:23:03.631 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_lc_debug_interval": "-1", 2023-11-25T04:23:03.631 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_lc_lock_max_time": "90", 2023-11-25T04:23:03.631 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_lc_max_objs": "32", 2023-11-25T04:23:03.631 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_lc_max_rules": "1000", 2023-11-25T04:23:03.631 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_lc_max_worker": "3", 2023-11-25T04:23:03.631 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_lc_max_wp_worker": "3", 2023-11-25T04:23:03.631 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_lc_thread_delay": "0", 2023-11-25T04:23:03.631 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2023-11-25T04:23:03.631 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_ldap_dnattr": "uid", 2023-11-25T04:23:03.631 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2023-11-25T04:23:03.631 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_ldap_searchfilter": "", 2023-11-25T04:23:03.631 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2023-11-25T04:23:03.631 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_ldap_uri": "ldaps://", 2023-11-25T04:23:03.632 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2023-11-25T04:23:03.632 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_list_bucket_min_readahead": "1000", 2023-11-25T04:23:03.632 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_list_buckets_max_chunk": "1000", 2023-11-25T04:23:03.632 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_log_http_headers": "", 2023-11-25T04:23:03.632 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_log_nonexistent_bucket": "false", 2023-11-25T04:23:03.632 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2023-11-25T04:23:03.632 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_log_object_name_utc": "false", 2023-11-25T04:23:03.632 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_lua_max_memory_per_state": "128000", 2023-11-25T04:23:03.632 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2023-11-25T04:23:03.632 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_max_attr_name_len": "0", 2023-11-25T04:23:03.632 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_max_attr_size": "0", 2023-11-25T04:23:03.632 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_max_attrs_num_in_req": "0", 2023-11-25T04:23:03.633 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_max_chunk_size": "4194304", 2023-11-25T04:23:03.633 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_max_concurrent_requests": "1024", 2023-11-25T04:23:03.633 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2023-11-25T04:23:03.633 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_max_dynamic_shards": "1999", 2023-11-25T04:23:03.633 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_max_listing_results": "1000", 2023-11-25T04:23:03.633 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_max_notify_retries": "10", 2023-11-25T04:23:03.633 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_max_objs_per_shard": "100000", 2023-11-25T04:23:03.633 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_max_put_param_size": "1048576", 2023-11-25T04:23:03.633 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_max_put_size": "5368709120", 2023-11-25T04:23:03.633 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_max_slo_entries": "1000", 2023-11-25T04:23:03.633 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_md_log_max_shards": "64", 2023-11-25T04:23:03.633 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_md_notify_interval_msec": "200", 2023-11-25T04:23:03.633 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_meta_sync_poll_interval": "20", 2023-11-25T04:23:03.634 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_meta_sync_spawn_window": "20", 2023-11-25T04:23:03.634 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_mime_types_file": "/etc/mime.types", 2023-11-25T04:23:03.634 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_mp_lock_max_time": "600", 2023-11-25T04:23:03.634 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_multi_obj_del_max_aio": "16", 2023-11-25T04:23:03.634 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_multipart_min_part_size": "5242880", 2023-11-25T04:23:03.634 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_multipart_part_upload_limit": "10000", 2023-11-25T04:23:03.634 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_nfs_fhcache_partitions": "3", 2023-11-25T04:23:03.634 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_nfs_fhcache_size": "2017", 2023-11-25T04:23:03.634 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_nfs_frontends": "rgw-nfs", 2023-11-25T04:23:03.634 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2023-11-25T04:23:03.634 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_nfs_lru_lanes": "5", 2023-11-25T04:23:03.634 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_nfs_max_gc": "300", 2023-11-25T04:23:03.635 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_nfs_namespace_expire_secs": "300", 2023-11-25T04:23:03.635 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_nfs_run_gc_threads": "false", 2023-11-25T04:23:03.635 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_nfs_run_lc_threads": "false", 2023-11-25T04:23:03.635 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_nfs_run_quota_threads": "false", 2023-11-25T04:23:03.635 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_nfs_run_sync_thread": "false", 2023-11-25T04:23:03.635 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_nfs_s3_fast_attrs": "false", 2023-11-25T04:23:03.635 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_nfs_write_completion_interval_s": "10", 2023-11-25T04:23:03.635 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_num_async_rados_threads": "32", 2023-11-25T04:23:03.635 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_num_control_oids": "8", 2023-11-25T04:23:03.635 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_numa_node": "-1", 2023-11-25T04:23:03.635 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_obj_stripe_size": "4194304", 2023-11-25T04:23:03.635 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_obj_tombstone_cache_size": "1000", 2023-11-25T04:23:03.636 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_objexp_chunk_size": "100", 2023-11-25T04:23:03.636 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_objexp_gc_interval": "600", 2023-11-25T04:23:03.636 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_objexp_hints_num_shards": "127", 2023-11-25T04:23:03.636 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_olh_pending_timeout_sec": "3600", 2023-11-25T04:23:03.636 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_op_thread_suicide_timeout": "0", 2023-11-25T04:23:03.636 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_op_thread_timeout": "600", 2023-11-25T04:23:03.636 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_op_tracing": "false", 2023-11-25T04:23:03.636 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_opa_token": "", 2023-11-25T04:23:03.636 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_opa_url": "", 2023-11-25T04:23:03.636 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_opa_verify_ssl": "true", 2023-11-25T04:23:03.636 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_ops_log_data_backlog": "5242880", 2023-11-25T04:23:03.636 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2023-11-25T04:23:03.636 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_ops_log_rados": "false", 2023-11-25T04:23:03.637 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_ops_log_socket_path": "", 2023-11-25T04:23:03.637 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_override_bucket_index_max_shards": "0", 2023-11-25T04:23:03.637 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2023-11-25T04:23:03.637 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2023-11-25T04:23:03.637 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_period_push_interval": "2.000000", 2023-11-25T04:23:03.637 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_period_push_interval_max": "30.000000", 2023-11-25T04:23:03.637 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_period_root_pool": ".rgw.root", 2023-11-25T04:23:03.637 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_policy_reject_invalid_principals": "true", 2023-11-25T04:23:03.637 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2023-11-25T04:23:03.637 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_posix_cache_lanes": "3", 2023-11-25T04:23:03.637 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_posix_cache_lmdb_count": "3", 2023-11-25T04:23:03.637 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_posix_cache_max_buckets": "100", 2023-11-25T04:23:03.638 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_posix_cache_partitions": "3", 2023-11-25T04:23:03.638 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2023-11-25T04:23:03.638 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_print_continue": "true", 2023-11-25T04:23:03.638 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_print_prohibited_content_length": "false", 2023-11-25T04:23:03.638 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_put_obj_max_window_size": "67108864", 2023-11-25T04:23:03.638 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_put_obj_min_window_size": "16777216", 2023-11-25T04:23:03.638 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2023-11-25T04:23:03.638 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_rados_pool_recovery_priority": "5", 2023-11-25T04:23:03.638 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_rados_tracing": "false", 2023-11-25T04:23:03.638 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_realm": "", 2023-11-25T04:23:03.638 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_realm_id": "", 2023-11-25T04:23:03.638 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_realm_root_pool": ".rgw.root", 2023-11-25T04:23:03.639 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_region": "", 2023-11-25T04:23:03.639 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_region_root_pool": ".rgw.root", 2023-11-25T04:23:03.639 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_relaxed_region_enforcement": "false", 2023-11-25T04:23:03.639 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_relaxed_s3_bucket_names": "false", 2023-11-25T04:23:03.639 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2023-11-25T04:23:03.639 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_request_uri": "", 2023-11-25T04:23:03.639 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_reshard_batch_size": "64", 2023-11-25T04:23:03.639 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_reshard_bucket_lock_duration": "360", 2023-11-25T04:23:03.639 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_reshard_max_aio": "128", 2023-11-25T04:23:03.639 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_reshard_num_logs": "16", 2023-11-25T04:23:03.639 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_reshard_thread_interval": "600", 2023-11-25T04:23:03.639 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_resolve_cname": "false", 2023-11-25T04:23:03.639 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_rest_getusage_op_compat": "false", 2023-11-25T04:23:03.640 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_run_sync_thread": "true", 2023-11-25T04:23:03.640 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_s3_auth_order": "sts, external, local", 2023-11-25T04:23:03.640 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_s3_auth_use_keystone": "false", 2023-11-25T04:23:03.640 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_s3_auth_use_ldap": "false", 2023-11-25T04:23:03.640 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_s3_auth_use_rados": "true", 2023-11-25T04:23:03.640 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_s3_auth_use_sts": "false", 2023-11-25T04:23:03.640 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_s3_client_max_sig_ver": "-1", 2023-11-25T04:23:03.640 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_s3_success_create_obj_status": "0", 2023-11-25T04:23:03.640 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_safe_max_objects_per_shard": "102400", 2023-11-25T04:23:03.640 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_scheduler_type": "throttler", 2023-11-25T04:23:03.640 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_script_uri": "", 2023-11-25T04:23:03.640 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_service_provider_name": "", 2023-11-25T04:23:03.641 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_shard_warning_threshold": "90.000000", 2023-11-25T04:23:03.641 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_sts_client_id": "", 2023-11-25T04:23:03.641 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_sts_client_secret": "", 2023-11-25T04:23:03.641 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_sts_entry": "sts", 2023-11-25T04:23:03.641 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_sts_key": "sts", 2023-11-25T04:23:03.641 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_sts_max_session_duration": "43200", 2023-11-25T04:23:03.641 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_sts_min_session_duration": "900", 2023-11-25T04:23:03.641 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_sts_token_introspection_url": "", 2023-11-25T04:23:03.641 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_swift_account_in_url": "false", 2023-11-25T04:23:03.641 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_swift_auth_entry": "auth", 2023-11-25T04:23:03.641 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_swift_auth_url": "", 2023-11-25T04:23:03.641 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_swift_custom_header": "", 2023-11-25T04:23:03.642 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_swift_enforce_content_length": "false", 2023-11-25T04:23:03.642 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_swift_need_stats": "true", 2023-11-25T04:23:03.642 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_swift_tenant_name": "", 2023-11-25T04:23:03.642 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_swift_token_expiration": "86400", 2023-11-25T04:23:03.642 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_swift_url": "", 2023-11-25T04:23:03.642 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_swift_url_prefix": "swift", 2023-11-25T04:23:03.642 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_swift_versioning_enabled": "false", 2023-11-25T04:23:03.642 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2023-11-25T04:23:03.642 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2023-11-25T04:23:03.642 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_sync_lease_period": "120", 2023-11-25T04:23:03.642 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2023-11-25T04:23:03.643 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_sync_log_trim_interval": "1200", 2023-11-25T04:23:03.643 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_sync_log_trim_max_buckets": "16", 2023-11-25T04:23:03.643 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2023-11-25T04:23:03.643 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2023-11-25T04:23:03.643 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_sync_obj_etag_verify": "false", 2023-11-25T04:23:03.643 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_sync_trace_history_size": "4096", 2023-11-25T04:23:03.643 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_sync_trace_per_node_log_size": "32", 2023-11-25T04:23:03.643 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2023-11-25T04:23:03.643 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_thread_pool_size": "512", 2023-11-25T04:23:03.643 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_topic_persistency_max_retries": "0", 2023-11-25T04:23:03.643 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_topic_persistency_sleep_duration": "0", 2023-11-25T04:23:03.643 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_topic_persistency_time_to_live": "0", 2023-11-25T04:23:03.643 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_torrent_comment": "", 2023-11-25T04:23:03.644 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_torrent_createby": "", 2023-11-25T04:23:03.644 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_torrent_encoding": "", 2023-11-25T04:23:03.644 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_torrent_flag": "false", 2023-11-25T04:23:03.644 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_torrent_max_size": "5368709120", 2023-11-25T04:23:03.644 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_torrent_origin": "", 2023-11-25T04:23:03.644 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_torrent_sha_unit": "524288", 2023-11-25T04:23:03.644 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_torrent_tracker": "", 2023-11-25T04:23:03.644 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_trust_forwarded_https": "false", 2023-11-25T04:23:03.644 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_usage_log_flush_threshold": "1024", 2023-11-25T04:23:03.644 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_usage_log_tick_interval": "30", 2023-11-25T04:23:03.644 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_usage_max_shards": "32", 2023-11-25T04:23:03.644 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_usage_max_user_shards": "1", 2023-11-25T04:23:03.645 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_use_opa_authz": "false", 2023-11-25T04:23:03.645 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_user_counters_cache": "false", 2023-11-25T04:23:03.645 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_user_counters_cache_size": "10000", 2023-11-25T04:23:03.645 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_user_default_quota_max_objects": "-1", 2023-11-25T04:23:03.645 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_user_default_quota_max_size": "-1", 2023-11-25T04:23:03.645 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_user_max_buckets": "1000", 2023-11-25T04:23:03.645 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_user_policies_max_num": "100", 2023-11-25T04:23:03.645 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2023-11-25T04:23:03.645 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_user_quota_sync_idle_users": "false", 2023-11-25T04:23:03.645 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_user_quota_sync_interval": "86400", 2023-11-25T04:23:03.645 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_user_quota_sync_wait_time": "86400", 2023-11-25T04:23:03.645 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_user_unique_email": "true", 2023-11-25T04:23:03.646 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_verify_ssl": "true", 2023-11-25T04:23:03.646 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_website_routing_rules_max_num": "50", 2023-11-25T04:23:03.646 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_zone": "", 2023-11-25T04:23:03.646 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_zone_id": "", 2023-11-25T04:23:03.646 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_zone_root_pool": ".rgw.root", 2023-11-25T04:23:03.646 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_zonegroup": "", 2023-11-25T04:23:03.646 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_zonegroup_id": "", 2023-11-25T04:23:03.646 INFO:teuthology.orchestra.run.smithi067.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2023-11-25T04:23:03.646 INFO:teuthology.orchestra.run.smithi067.stdout: "rgwlc_auto_session_clear": "true", 2023-11-25T04:23:03.646 INFO:teuthology.orchestra.run.smithi067.stdout: "rgwlc_skip_bucket_step": "false", 2023-11-25T04:23:03.646 INFO:teuthology.orchestra.run.smithi067.stdout: "rocksdb_block_size": "4096", 2023-11-25T04:23:03.646 INFO:teuthology.orchestra.run.smithi067.stdout: "rocksdb_bloom_bits_per_key": "20", 2023-11-25T04:23:03.647 INFO:teuthology.orchestra.run.smithi067.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2023-11-25T04:23:03.647 INFO:teuthology.orchestra.run.smithi067.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2023-11-25T04:23:03.647 INFO:teuthology.orchestra.run.smithi067.stdout: "rocksdb_cache_row_ratio": "0.000000", 2023-11-25T04:23:03.647 INFO:teuthology.orchestra.run.smithi067.stdout: "rocksdb_cache_shard_bits": "4", 2023-11-25T04:23:03.647 INFO:teuthology.orchestra.run.smithi067.stdout: "rocksdb_cache_size": "536870912", 2023-11-25T04:23:03.647 INFO:teuthology.orchestra.run.smithi067.stdout: "rocksdb_cache_type": "binned_lru", 2023-11-25T04:23:03.647 INFO:teuthology.orchestra.run.smithi067.stdout: "rocksdb_cf_compact_on_deletion": "true", 2023-11-25T04:23:03.647 INFO:teuthology.orchestra.run.smithi067.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2023-11-25T04:23:03.647 INFO:teuthology.orchestra.run.smithi067.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2023-11-25T04:23:03.647 INFO:teuthology.orchestra.run.smithi067.stdout: "rocksdb_collect_compaction_stats": "false", 2023-11-25T04:23:03.647 INFO:teuthology.orchestra.run.smithi067.stdout: "rocksdb_collect_extended_stats": "false", 2023-11-25T04:23:03.647 INFO:teuthology.orchestra.run.smithi067.stdout: "rocksdb_collect_memory_stats": "false", 2023-11-25T04:23:03.647 INFO:teuthology.orchestra.run.smithi067.stdout: "rocksdb_delete_range_threshold": "1048576", 2023-11-25T04:23:03.648 INFO:teuthology.orchestra.run.smithi067.stdout: "rocksdb_index_type": "binary_search", 2023-11-25T04:23:03.648 INFO:teuthology.orchestra.run.smithi067.stdout: "rocksdb_log_to_ceph_log": "true", 2023-11-25T04:23:03.648 INFO:teuthology.orchestra.run.smithi067.stdout: "rocksdb_metadata_block_size": "4096", 2023-11-25T04:23:03.648 INFO:teuthology.orchestra.run.smithi067.stdout: "rocksdb_partition_filters": "false", 2023-11-25T04:23:03.648 INFO:teuthology.orchestra.run.smithi067.stdout: "rocksdb_perf": "false", 2023-11-25T04:23:03.648 INFO:teuthology.orchestra.run.smithi067.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2023-11-25T04:23:03.648 INFO:teuthology.orchestra.run.smithi067.stdout: "rotating_keys_bootstrap_timeout": "30", 2023-11-25T04:23:03.648 INFO:teuthology.orchestra.run.smithi067.stdout: "rotating_keys_renewal_timeout": "10", 2023-11-25T04:23:03.648 INFO:teuthology.orchestra.run.smithi067.stdout: "run_dir": "/var/run/ceph", 2023-11-25T04:23:03.648 INFO:teuthology.orchestra.run.smithi067.stdout: "seastore_block_create": "true", 2023-11-25T04:23:03.648 INFO:teuthology.orchestra.run.smithi067.stdout: "seastore_cache_lru_size": "67108864", 2023-11-25T04:23:03.648 INFO:teuthology.orchestra.run.smithi067.stdout: "seastore_cbjournal_size": "5368709120", 2023-11-25T04:23:03.649 INFO:teuthology.orchestra.run.smithi067.stdout: "seastore_data_delta_based_overwrite": "0", 2023-11-25T04:23:03.649 INFO:teuthology.orchestra.run.smithi067.stdout: "seastore_default_max_object_size": "16777216", 2023-11-25T04:23:03.649 INFO:teuthology.orchestra.run.smithi067.stdout: "seastore_default_object_metadata_reservation": "16777216", 2023-11-25T04:23:03.649 INFO:teuthology.orchestra.run.smithi067.stdout: "seastore_device_size": "53687091200", 2023-11-25T04:23:03.649 INFO:teuthology.orchestra.run.smithi067.stdout: "seastore_journal_batch_capacity": "16", 2023-11-25T04:23:03.649 INFO:teuthology.orchestra.run.smithi067.stdout: "seastore_journal_batch_flush_size": "16777216", 2023-11-25T04:23:03.649 INFO:teuthology.orchestra.run.smithi067.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2023-11-25T04:23:03.649 INFO:teuthology.orchestra.run.smithi067.stdout: "seastore_journal_iodepth_limit": "5", 2023-11-25T04:23:03.649 INFO:teuthology.orchestra.run.smithi067.stdout: "seastore_main_device_type": "SSD", 2023-11-25T04:23:03.649 INFO:teuthology.orchestra.run.smithi067.stdout: "seastore_max_concurrent_transactions": "8", 2023-11-25T04:23:03.649 INFO:teuthology.orchestra.run.smithi067.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2023-11-25T04:23:03.649 INFO:teuthology.orchestra.run.smithi067.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2023-11-25T04:23:03.649 INFO:teuthology.orchestra.run.smithi067.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2023-11-25T04:23:03.650 INFO:teuthology.orchestra.run.smithi067.stdout: "seastore_obj_data_write_amplification": "1.250000", 2023-11-25T04:23:03.650 INFO:teuthology.orchestra.run.smithi067.stdout: "seastore_segment_size": "67108864", 2023-11-25T04:23:03.650 INFO:teuthology.orchestra.run.smithi067.stdout: "set_keepcaps": "false", 2023-11-25T04:23:03.650 INFO:teuthology.orchestra.run.smithi067.stdout: "setgroup": "", 2023-11-25T04:23:03.650 INFO:teuthology.orchestra.run.smithi067.stdout: "setuser": "", 2023-11-25T04:23:03.650 INFO:teuthology.orchestra.run.smithi067.stdout: "setuser_match_path": "", 2023-11-25T04:23:03.650 INFO:teuthology.orchestra.run.smithi067.stdout: "target_max_misplaced_ratio": "0.050000", 2023-11-25T04:23:03.650 INFO:teuthology.orchestra.run.smithi067.stdout: "thp": "false", 2023-11-25T04:23:03.650 INFO:teuthology.orchestra.run.smithi067.stdout: "threadpool_default_timeout": "60", 2023-11-25T04:23:03.650 INFO:teuthology.orchestra.run.smithi067.stdout: "threadpool_empty_queue_max_wait": "2", 2023-11-25T04:23:03.650 INFO:teuthology.orchestra.run.smithi067.stdout: "throttler_perf_counter": "true" 2023-11-25T04:23:03.650 INFO:teuthology.orchestra.run.smithi067.stdout:} 2023-11-25T04:23:03.653 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2023-11-25T04:23:03.773 INFO:teuthology.orchestra.run.smithi067.stdout:mon.a: {} 2023-11-25T04:23:03.773 INFO:teuthology.orchestra.run.smithi067.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2023-11-25T04:23:03.782 INFO:teuthology.orchestra.run.smithi067.stdout:mon.b: {} 2023-11-25T04:23:03.782 INFO:teuthology.orchestra.run.smithi067.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2023-11-25T04:23:03.791 INFO:teuthology.orchestra.run.smithi067.stdout:mon.c: {} 2023-11-25T04:23:03.791 INFO:teuthology.orchestra.run.smithi067.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2023-11-25T04:23:03.802 INFO:tasks.thrashosds.thrasher:Unable to test ceph-objectstore-tool, powercycle testing 2023-11-25T04:23:03.803 INFO:teuthology.run_tasks:Running task ceph-fuse... 2023-11-25T04:23:03.828 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2023-11-25T04:23:03.828 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2023-11-25T04:23:03.829 INFO:tasks.ceph_fuse:client.0 config is {} 2023-11-25T04:23:03.829 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-25T04:23:03.829 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-25T04:23:03.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-25T04:23:03.829 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list 2023-11-25T04:23:03.830 INFO:tasks.thrashosds.thrasher:starting do_thrash 2023-11-25T04:23:03.830 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2] 2023-11-25T04:23:03.830 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 1.00 2023-11-25T04:23:03.830 INFO:tasks.thrashosds.thrasher:Killing osd 0, live_osds are [0, 1, 2] 2023-11-25T04:23:03.830 INFO:tasks.ceph.ceph_manager.ceph:kill_osd on osd.0 doing powercycle of ubuntu@smithi076.front.sepia.ceph.com 2023-11-25T04:23:03.830 INFO:teuthology.orchestra.console.smithi076:Power off 2023-11-25T04:23:03.830 DEBUG:teuthology.orchestra.console.smithi076:pexpect command: ipmitool -H smithi076.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-25T04:23:03.839 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2023-11-25T04:23:03.840 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2023-11-25T04:23:03.840 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-11-25T04:23:03.840 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2023-11-25T04:23:03.841 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:23:03.841 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2023-11-25T04:23:03.841 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2023-11-25T04:23:03.843 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:03.839+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:03.854 DEBUG:teuthology.orchestra.console.smithi076:power off output: Chassis Power Control: Down/Off 2023-11-25T04:23:03.854 DEBUG:teuthology.orchestra.console.smithi076:pexpect command: ipmitool -H smithi076.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-25T04:23:03.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-25T04:23:03.864 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link delete ceph-brx 2023-11-25T04:23:03.878 DEBUG:teuthology.orchestra.console.smithi076:check power output: Chassis Power is on 2023-11-25T04:23:03.932 INFO:teuthology.orchestra.run.smithi067.stderr:Cannot find device "ceph-brx" 2023-11-25T04:23:03.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-25T04:23:03.934 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2023-11-25T04:23:03.935 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-25T04:23:03.941 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:03.940+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:03.959 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:03.959 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:23:03.959 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2023-11-25T04:23:03.959 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:03.968 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:23:03.977 INFO:teuthology.orchestra.run.smithi067.stdout:osd.0: {} 2023-11-25T04:23:03.979 INFO:teuthology.orchestra.run.smithi067.stderr:osd.0: osd_enable_op_tracker = '' 2023-11-25T04:23:03.995 INFO:teuthology.orchestra.run.smithi067.stdout:osd.1: {} 2023-11-25T04:23:03.995 INFO:teuthology.orchestra.run.smithi067.stderr:osd.1: osd_enable_op_tracker = '' 2023-11-25T04:23:04.013 INFO:teuthology.orchestra.run.smithi067.stdout:osd.2: {} 2023-11-25T04:23:04.014 INFO:teuthology.orchestra.run.smithi067.stderr:osd.2: osd_enable_op_tracker = '' 2023-11-25T04:23:04.042 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:04.041+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:04.075 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:04.075 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:04.084 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:23:04.143 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:04.142+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:04.189 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:04.189 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:04.198 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:23:04.244 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:04.243+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:04.310 INFO:teuthology.orchestra.run.smithi067.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-25T04:23:04.314 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:04.315 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:04.323 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-25T04:23:04.323 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-25T04:23:04.323 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi067.front.sepia.ceph.com 2023-11-25T04:23:04.323 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-25T04:23:04.323 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-25T04:23:04.323 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-25T04:23:04.324 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-25T04:23:04.324 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-25T04:23:04.324 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-25T04:23:04.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-25T04:23:04.324 DEBUG:teuthology.orchestra.run.smithi067:> ip addr 2023-11-25T04:23:04.327 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:23:04.342 INFO:teuthology.orchestra.run.smithi067.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-25T04:23:04.343 INFO:teuthology.orchestra.run.smithi067.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-25T04:23:04.343 INFO:teuthology.orchestra.run.smithi067.stdout: inet 127.0.0.1/8 scope host lo 2023-11-25T04:23:04.343 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2023-11-25T04:23:04.343 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 ::1/128 scope host 2023-11-25T04:23:04.343 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2023-11-25T04:23:04.343 INFO:teuthology.orchestra.run.smithi067.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-25T04:23:04.343 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7e brd ff:ff:ff:ff:ff:ff 2023-11-25T04:23:04.343 INFO:teuthology.orchestra.run.smithi067.stdout: altname enp7s0f0 2023-11-25T04:23:04.343 INFO:teuthology.orchestra.run.smithi067.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-25T04:23:04.343 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7f brd ff:ff:ff:ff:ff:ff 2023-11-25T04:23:04.343 INFO:teuthology.orchestra.run.smithi067.stdout: altname enp7s0f1 2023-11-25T04:23:04.343 INFO:teuthology.orchestra.run.smithi067.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-25T04:23:04.344 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0a brd ff:ff:ff:ff:ff:ff 2023-11-25T04:23:04.344 INFO:teuthology.orchestra.run.smithi067.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-25T04:23:04.344 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0b brd ff:ff:ff:ff:ff:ff 2023-11-25T04:23:04.344 INFO:teuthology.orchestra.run.smithi067.stdout: inet 172.21.15.67/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-25T04:23:04.344 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft 42646sec preferred_lft 42646sec 2023-11-25T04:23:04.344 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::ec4:7aff:feb6:fc0b/64 scope link 2023-11-25T04:23:04.344 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2023-11-25T04:23:04.345 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-25T04:23:04.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-25T04:23:04.345 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec bash -c ' 2023-11-25T04:23:04.345 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2023-11-25T04:23:04.345 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link add name ceph-brx type bridge 2023-11-25T04:23:04.345 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip addr flush dev ceph-brx 2023-11-25T04:23:04.345 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set ceph-brx up 2023-11-25T04:23:04.345 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-25T04:23:04.345 DEBUG:teuthology.orchestra.run.smithi067:> ') 2023-11-25T04:23:04.347 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:04.345+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:04.432 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:04.432 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:04.441 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:23:04.485 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:04.484+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:04.546 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-25T04:23:04.546 DEBUG:teuthology.orchestra.run.smithi067:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-25T04:23:04.547 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:04.547 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:04.548 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:04.546+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:04.555 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:23:04.577 INFO:teuthology.orchestra.run.smithi067.stdout:1 2023-11-25T04:23:04.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-25T04:23:04.579 DEBUG:teuthology.orchestra.run.smithi067:> route 2023-11-25T04:23:04.648 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:04.647+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:04.652 INFO:teuthology.orchestra.run.smithi067.stdout:Kernel IP routing table 2023-11-25T04:23:04.652 INFO:teuthology.orchestra.run.smithi067.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-25T04:23:04.652 INFO:teuthology.orchestra.run.smithi067.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-25T04:23:04.652 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-25T04:23:04.652 INFO:teuthology.orchestra.run.smithi067.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-25T04:23:04.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-25T04:23:04.653 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec bash -c ' 2023-11-25T04:23:04.653 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2023-11-25T04:23:04.653 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-25T04:23:04.653 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-25T04:23:04.653 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-25T04:23:04.653 DEBUG:teuthology.orchestra.run.smithi067:> ') 2023-11-25T04:23:04.660 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:04.660 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:04.669 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:23:04.750 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:04.748+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:04.814 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:04.814 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:04.823 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:23:04.866 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:04.865+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:04.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-25T04:23:04.921 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list 2023-11-25T04:23:04.928 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:04.928 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:04.937 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:23:04.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-25T04:23:04.938 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list-id 2023-11-25T04:23:04.981 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:04.980+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:04.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-25T04:23:04.993 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec bash -c ' 2023-11-25T04:23:04.993 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2023-11-25T04:23:04.993 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-25T04:23:04.993 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-11-25T04:23:04.993 DEBUG:teuthology.orchestra.run.smithi067:> ') 2023-11-25T04:23:05.046 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:05.046 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:05.052 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:05.051+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:05.056 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:23:05.082 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-11-25T04:23:05.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-25T04:23:05.083 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec bash -c ' 2023-11-25T04:23:05.083 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2023-11-25T04:23:05.083 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-11-25T04:23:05.083 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-25T04:23:05.083 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-11-25T04:23:05.083 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-11-25T04:23:05.083 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-11-25T04:23:05.083 DEBUG:teuthology.orchestra.run.smithi067:> ') 2023-11-25T04:23:05.098 INFO:teuthology.orchestra.run.smithi067.stderr:noscrub is set 2023-11-25T04:23:05.153 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:05.152+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:05.201 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:05.201 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:05.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-25T04:23:05.202 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec bash -c ' 2023-11-25T04:23:05.202 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2023-11-25T04:23:05.202 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set brx.0 up 2023-11-25T04:23:05.202 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set dev brx.0 master ceph-brx 2023-11-25T04:23:05.203 DEBUG:teuthology.orchestra.run.smithi067:> ') 2023-11-25T04:23:05.210 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:23:05.252 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-25T04:23:05.252 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-25T04:23:05.252 DEBUG:teuthology.orchestra.run.smithi067:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-25T04:23:05.254 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:05.252+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:05.267 INFO:teuthology.orchestra.run.smithi067.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-25T04:23:05.267 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-25T04:23:05.267 DEBUG:teuthology.orchestra.run.smithi067:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-25T04:23:05.315 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:05.315 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:05.322 DEBUG:teuthology.orchestra.run.smithi067:> sudo modprobe fuse 2023-11-25T04:23:05.324 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:23:05.368 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:05.367+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:05.393 DEBUG:teuthology.orchestra.run.smithi067:> cat /proc/self/mounts | awk '{print $2}' 2023-11-25T04:23:05.430 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:05.431 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:05.440 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:23:05.449 INFO:teuthology.orchestra.run.smithi067.stdout:/proc 2023-11-25T04:23:05.449 INFO:teuthology.orchestra.run.smithi067.stdout:/sys 2023-11-25T04:23:05.449 INFO:teuthology.orchestra.run.smithi067.stdout:/dev 2023-11-25T04:23:05.449 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/security 2023-11-25T04:23:05.449 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/shm 2023-11-25T04:23:05.449 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/pts 2023-11-25T04:23:05.449 INFO:teuthology.orchestra.run.smithi067.stdout:/run 2023-11-25T04:23:05.449 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/cgroup 2023-11-25T04:23:05.449 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/pstore 2023-11-25T04:23:05.450 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/bpf 2023-11-25T04:23:05.450 INFO:teuthology.orchestra.run.smithi067.stdout:/ 2023-11-25T04:23:05.450 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/selinux 2023-11-25T04:23:05.450 INFO:teuthology.orchestra.run.smithi067.stdout:/proc/sys/fs/binfmt_misc 2023-11-25T04:23:05.450 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/hugepages 2023-11-25T04:23:05.450 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/mqueue 2023-11-25T04:23:05.450 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/debug 2023-11-25T04:23:05.450 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/tracing 2023-11-25T04:23:05.450 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-25T04:23:05.450 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/fuse/connections 2023-11-25T04:23:05.450 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/config 2023-11-25T04:23:05.450 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-sysctl.service 2023-11-25T04:23:05.450 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-25T04:23:05.451 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/nfs/rpc_pipefs 2023-11-25T04:23:05.451 INFO:teuthology.orchestra.run.smithi067.stdout:/run/user/1000 2023-11-25T04:23:05.451 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph 2023-11-25T04:23:05.451 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns 2023-11-25T04:23:05.451 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-25T04:23:05.451 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-25T04:23:05.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-25T04:23:05.451 DEBUG:teuthology.orchestra.run.smithi067:> ls /sys/fs/fuse/connections 2023-11-25T04:23:05.481 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:05.480+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:05.505 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-25T04:23:05.505 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-25T04:23:05.547 DEBUG:teuthology.orchestra.run.smithi067:> sudo modprobe fuse 2023-11-25T04:23:05.557 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:05.555+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:05.578 DEBUG:teuthology.orchestra.run.smithi067:> cat /proc/self/mounts | awk '{print $2}' 2023-11-25T04:23:05.585 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:05.585 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:05.594 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:23:05.635 INFO:teuthology.orchestra.run.smithi067.stdout:/proc 2023-11-25T04:23:05.635 INFO:teuthology.orchestra.run.smithi067.stdout:/sys 2023-11-25T04:23:05.635 INFO:teuthology.orchestra.run.smithi067.stdout:/dev 2023-11-25T04:23:05.635 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/security 2023-11-25T04:23:05.635 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/shm 2023-11-25T04:23:05.636 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/pts 2023-11-25T04:23:05.636 INFO:teuthology.orchestra.run.smithi067.stdout:/run 2023-11-25T04:23:05.636 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/cgroup 2023-11-25T04:23:05.636 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/pstore 2023-11-25T04:23:05.636 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/bpf 2023-11-25T04:23:05.636 INFO:teuthology.orchestra.run.smithi067.stdout:/ 2023-11-25T04:23:05.636 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/selinux 2023-11-25T04:23:05.636 INFO:teuthology.orchestra.run.smithi067.stdout:/proc/sys/fs/binfmt_misc 2023-11-25T04:23:05.636 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/hugepages 2023-11-25T04:23:05.636 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/mqueue 2023-11-25T04:23:05.636 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/debug 2023-11-25T04:23:05.637 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/tracing 2023-11-25T04:23:05.637 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-25T04:23:05.637 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/fuse/connections 2023-11-25T04:23:05.637 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/config 2023-11-25T04:23:05.637 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-sysctl.service 2023-11-25T04:23:05.637 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-25T04:23:05.637 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/nfs/rpc_pipefs 2023-11-25T04:23:05.637 INFO:teuthology.orchestra.run.smithi067.stdout:/run/user/1000 2023-11-25T04:23:05.637 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph 2023-11-25T04:23:05.637 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns 2023-11-25T04:23:05.637 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-25T04:23:05.638 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-25T04:23:05.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-25T04:23:05.638 DEBUG:teuthology.orchestra.run.smithi067:> ls /sys/fs/fuse/connections 2023-11-25T04:23:05.657 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:05.656+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:05.660 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi067.stderr:2023-11-25T04:23:05.658+0000 7f70ad237480 -1 init, newargv = 0x56268b42dd10 newargc=15 2023-11-25T04:23:05.660 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi067.stderr:ceph-fuse[85907]: starting ceph client 2023-11-25T04:23:05.664 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi067.stderr:ceph-fuse[85907]: starting fuse 2023-11-25T04:23:05.675 INFO:teuthology.orchestra.run.smithi067.stdout:42 2023-11-25T04:23:05.675 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2023-11-25T04:23:05.676 DEBUG:teuthology.orchestra.run.smithi067:> sudo stdin-killer -- python3 -c ' 2023-11-25T04:23:05.676 DEBUG:teuthology.orchestra.run.smithi067:> import glob 2023-11-25T04:23:05.676 DEBUG:teuthology.orchestra.run.smithi067:> import re 2023-11-25T04:23:05.676 DEBUG:teuthology.orchestra.run.smithi067:> import os 2023-11-25T04:23:05.676 DEBUG:teuthology.orchestra.run.smithi067:> import subprocess 2023-11-25T04:23:05.676 DEBUG:teuthology.orchestra.run.smithi067:> 2023-11-25T04:23:05.676 DEBUG:teuthology.orchestra.run.smithi067:> def _find_admin_socket(client_name): 2023-11-25T04:23:05.676 DEBUG:teuthology.orchestra.run.smithi067:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-25T04:23:05.676 DEBUG:teuthology.orchestra.run.smithi067:> files = glob.glob(asok_path) 2023-11-25T04:23:05.676 DEBUG:teuthology.orchestra.run.smithi067:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-25T04:23:05.677 DEBUG:teuthology.orchestra.run.smithi067:> 2023-11-25T04:23:05.677 DEBUG:teuthology.orchestra.run.smithi067:> # Given a non-glob path, it better be there 2023-11-25T04:23:05.677 DEBUG:teuthology.orchestra.run.smithi067:> if "*" not in asok_path: 2023-11-25T04:23:05.677 DEBUG:teuthology.orchestra.run.smithi067:> assert(len(files) == 1) 2023-11-25T04:23:05.677 DEBUG:teuthology.orchestra.run.smithi067:> return files[0] 2023-11-25T04:23:05.677 DEBUG:teuthology.orchestra.run.smithi067:> 2023-11-25T04:23:05.677 DEBUG:teuthology.orchestra.run.smithi067:> for f in files: 2023-11-25T04:23:05.677 DEBUG:teuthology.orchestra.run.smithi067:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-25T04:23:05.677 DEBUG:teuthology.orchestra.run.smithi067:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-25T04:23:05.677 DEBUG:teuthology.orchestra.run.smithi067:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-25T04:23:05.677 DEBUG:teuthology.orchestra.run.smithi067:> contents = proc_f.read() 2023-11-25T04:23:05.677 DEBUG:teuthology.orchestra.run.smithi067:> if mountpoint in contents: 2023-11-25T04:23:05.677 DEBUG:teuthology.orchestra.run.smithi067:> return f 2023-11-25T04:23:05.678 DEBUG:teuthology.orchestra.run.smithi067:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-25T04:23:05.678 DEBUG:teuthology.orchestra.run.smithi067:> 2023-11-25T04:23:05.678 DEBUG:teuthology.orchestra.run.smithi067:> print(_find_admin_socket("client.0")) 2023-11-25T04:23:05.678 DEBUG:teuthology.orchestra.run.smithi067:> ' 2023-11-25T04:23:05.699 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:05.699 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:05.708 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:23:05.758 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:05.756+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:05.801 INFO:teuthology.orchestra.run.smithi067.stdout:/var/run/ceph/ceph-client.0.85907.asok 2023-11-25T04:23:05.804 INFO:teuthology.orchestra.run.smithi067.stderr:2023-11-25T04:23:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-25T04:23:05.812 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.85907.asok 2023-11-25T04:23:05.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-25T04:23:05.812 DEBUG:teuthology.orchestra.run.smithi067:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.85907.asok status 2023-11-25T04:23:05.813 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:05.813 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:05.822 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:23:05.863 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:05.862+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:05.954 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2023-11-25T04:23:05.954 INFO:teuthology.orchestra.run.smithi067.stdout: "metadata": { 2023-11-25T04:23:05.954 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_sha1": "ec928368edcc5fc71884a56fd9fd821b148f4e30", 2023-11-25T04:23:05.955 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_version": "ceph version 18.0.0-7506-gec928368 (ec928368edcc5fc71884a56fd9fd821b148f4e30) reef (dev)", 2023-11-25T04:23:05.955 INFO:teuthology.orchestra.run.smithi067.stdout: "entity_id": "0", 2023-11-25T04:23:05.955 INFO:teuthology.orchestra.run.smithi067.stdout: "hostname": "smithi067", 2023-11-25T04:23:05.955 INFO:teuthology.orchestra.run.smithi067.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-25T04:23:05.955 INFO:teuthology.orchestra.run.smithi067.stdout: "pid": "85907", 2023-11-25T04:23:05.955 INFO:teuthology.orchestra.run.smithi067.stdout: "root": "/" 2023-11-25T04:23:05.955 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2023-11-25T04:23:05.955 INFO:teuthology.orchestra.run.smithi067.stdout: "dentry_count": 0, 2023-11-25T04:23:05.955 INFO:teuthology.orchestra.run.smithi067.stdout: "dentry_pinned_count": 0, 2023-11-25T04:23:05.955 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 4561, 2023-11-25T04:23:05.956 INFO:teuthology.orchestra.run.smithi067.stdout: "inst": { 2023-11-25T04:23:05.956 INFO:teuthology.orchestra.run.smithi067.stdout: "name": { 2023-11-25T04:23:05.956 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "client", 2023-11-25T04:23:05.956 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 4561 2023-11-25T04:23:05.956 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2023-11-25T04:23:05.956 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": { 2023-11-25T04:23:05.957 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2023-11-25T04:23:05.957 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "192.168.0.1:0", 2023-11-25T04:23:05.957 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 1268434739 2023-11-25T04:23:05.957 INFO:teuthology.orchestra.run.smithi067.stdout: } 2023-11-25T04:23:05.957 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2023-11-25T04:23:05.957 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": { 2023-11-25T04:23:05.958 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2023-11-25T04:23:05.958 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "192.168.0.1:0", 2023-11-25T04:23:05.959 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 1268434739 2023-11-25T04:23:05.959 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2023-11-25T04:23:05.960 INFO:teuthology.orchestra.run.smithi067.stdout: "inst_str": "client.4561 192.168.0.1:0/1268434739", 2023-11-25T04:23:05.960 INFO:teuthology.orchestra.run.smithi067.stdout: "addr_str": "192.168.0.1:0/1268434739", 2023-11-25T04:23:05.962 INFO:teuthology.orchestra.run.smithi067.stdout: "inode_count": 1, 2023-11-25T04:23:05.962 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_epoch": 5, 2023-11-25T04:23:05.962 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_epoch": 20, 2023-11-25T04:23:05.962 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_epoch_barrier": 0, 2023-11-25T04:23:05.962 INFO:teuthology.orchestra.run.smithi067.stdout: "blocklisted": false, 2023-11-25T04:23:05.962 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs" 2023-11-25T04:23:05.962 INFO:teuthology.orchestra.run.smithi067.stdout:} 2023-11-25T04:23:05.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-25T04:23:05.964 DEBUG:teuthology.orchestra.run.smithi067:> stat --file-system '--printf=%T 2023-11-25T04:23:05.964 DEBUG:teuthology.orchestra.run.smithi067:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-25T04:23:05.964 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:05.962+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:05.967 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:05.967 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:05.976 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:23:05.981 INFO:teuthology.orchestra.run.smithi067.stdout:fuseblk 2023-11-25T04:23:05.981 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-25T04:23:05.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-25T04:23:05.982 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-25T04:23:06.025 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-11-25T04:23:06.053 INFO:teuthology.run_tasks:Running task workunit... 2023-11-25T04:23:06.069 INFO:tasks.workunit:Pulling workunits from ref ec928368edcc5fc71884a56fd9fd821b148f4e30 2023-11-25T04:23:06.069 INFO:tasks.workunit:Making a separate scratch dir for every client... 2023-11-25T04:23:06.069 INFO:tasks.workunit:timeout=6h 2023-11-25T04:23:06.069 INFO:tasks.workunit:cleanup=True 2023-11-25T04:23:06.069 DEBUG:teuthology.orchestra.run.smithi067:> stat -- /home/ubuntu/cephtest/mnt.0 2023-11-25T04:23:06.071 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:06.069+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:06.082 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:06.082 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:06.091 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:23:06.110 INFO:teuthology.orchestra.run.smithi067.stdout: File: /home/ubuntu/cephtest/mnt.0 2023-11-25T04:23:06.110 INFO:teuthology.orchestra.run.smithi067.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2023-11-25T04:23:06.110 INFO:teuthology.orchestra.run.smithi067.stdout:Device: 2ah/42d Inode: 1 Links: 2 2023-11-25T04:23:06.110 INFO:teuthology.orchestra.run.smithi067.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-25T04:23:06.110 INFO:teuthology.orchestra.run.smithi067.stdout:Context: system_u:object_r:fusefs_t:s0 2023-11-25T04:23:06.110 INFO:teuthology.orchestra.run.smithi067.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2023-11-25T04:23:06.111 INFO:teuthology.orchestra.run.smithi067.stdout:Modify: 2023-11-25 04:22:47.024106678 +0000 2023-11-25T04:23:06.111 INFO:teuthology.orchestra.run.smithi067.stdout:Change: 2023-11-25 04:23:06.050908369 +0000 2023-11-25T04:23:06.111 INFO:teuthology.orchestra.run.smithi067.stdout: Birth: - 2023-11-25T04:23:06.111 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2023-11-25T04:23:06.111 DEBUG:teuthology.orchestra.run.smithi067:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2023-11-25T04:23:06.173 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:06.172+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:06.185 DEBUG:teuthology.orchestra.run.smithi067:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout ec928368edcc5fc71884a56fd9fd821b148f4e30 2023-11-25T04:23:06.198 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:06.198 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:06.207 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:23:06.223 INFO:teuthology.orchestra.run.smithi067.stdout:osd.0: {} 2023-11-25T04:23:06.224 INFO:teuthology.orchestra.run.smithi067.stderr:osd.0: osd_enable_op_tracker = '' 2023-11-25T04:23:06.239 INFO:tasks.workunit.client.0.smithi067.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2023-11-25T04:23:06.242 INFO:teuthology.orchestra.run.smithi067.stdout:osd.1: {} 2023-11-25T04:23:06.242 INFO:teuthology.orchestra.run.smithi067.stderr:osd.1: osd_enable_op_tracker = '' 2023-11-25T04:23:06.259 INFO:teuthology.orchestra.run.smithi067.stdout:osd.2: {} 2023-11-25T04:23:06.260 INFO:teuthology.orchestra.run.smithi067.stderr:osd.2: osd_enable_op_tracker = '' 2023-11-25T04:23:06.291 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:06.290+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:06.352 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:06.352 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:23:06.353 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2023-11-25T04:23:06.353 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2023-11-25T04:23:06.353 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:06.361 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:23:06.377 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:06.373+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:06.475 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:06.474+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:06.509 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:06.509 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2023-11-25T04:23:06.509 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2023-11-25T04:23:06.509 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2023-11-25T04:23:06.509 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.509 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:155 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'109, mlcod=11'105)", 2023-11-25T04:23:06.509 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:06.510 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.721691424999999, 2023-11-25T04:23:06.510 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061719019999999999, 2023-11-25T04:23:06.510 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:06.510 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:06.510 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:06.510 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.510 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:06.510 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:06.510 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:06.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.510 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.510 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:06.511 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:06.511 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:06.511 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.511 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.511 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:06.511 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784884+0000", 2023-11-25T04:23:06.511 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3099999999999999e-07 2023-11-25T04:23:06.511 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.511 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.511 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:06.511 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784942+0000", 2023-11-25T04:23:06.511 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8570000000000003e-05 2023-11-25T04:23:06.511 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.512 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.512 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:06.512 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784949+0000", 2023-11-25T04:23:06.512 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0310000000000001e-06 2023-11-25T04:23:06.513 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.513 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.513 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:06.513 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784953+0000", 2023-11-25T04:23:06.513 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6320000000000001e-06 2023-11-25T04:23:06.513 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.513 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.513 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:06.513 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784993+0000", 2023-11-25T04:23:06.513 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9737000000000003e-05 2023-11-25T04:23:06.513 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.514 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.514 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:06.514 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785011+0000", 2023-11-25T04:23:06.514 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8349999999999999e-05 2023-11-25T04:23:06.514 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.514 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.514 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:06.514 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791024+0000", 2023-11-25T04:23:06.514 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006012997 2023-11-25T04:23:06.514 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.514 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.514 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:06.515 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791055+0000", 2023-11-25T04:23:06.515 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1254000000000003e-05 2023-11-25T04:23:06.515 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:06.515 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:06.515 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:06.515 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.515 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.515 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:156 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'110, mlcod=11'105)", 2023-11-25T04:23:06.515 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:06.515 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.721105376000001, 2023-11-25T04:23:06.515 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005616116, 2023-11-25T04:23:06.515 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:06.515 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:06.516 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:06.516 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.516 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:06.516 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:06.516 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:06.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.516 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.516 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:06.516 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:06.516 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:06.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.516 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.517 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:06.517 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785470+0000", 2023-11-25T04:23:06.517 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0699999999999998e-07 2023-11-25T04:23:06.517 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.517 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.517 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:06.517 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785474+0000", 2023-11-25T04:23:06.517 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.048e-06 2023-11-25T04:23:06.517 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.517 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.518 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:06.518 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785480+0000", 2023-11-25T04:23:06.518 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7729999999999996e-06 2023-11-25T04:23:06.518 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.518 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.518 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:06.518 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785483+0000", 2023-11-25T04:23:06.518 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1970000000000001e-06 2023-11-25T04:23:06.518 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.518 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.518 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:06.518 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787936+0000", 2023-11-25T04:23:06.519 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024529040000000001 2023-11-25T04:23:06.519 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.519 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.519 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:06.519 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787947+0000", 2023-11-25T04:23:06.519 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1669e-05 2023-11-25T04:23:06.519 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.519 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.519 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:06.519 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791057+0000", 2023-11-25T04:23:06.519 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031095200000000002 2023-11-25T04:23:06.519 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.519 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.520 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:06.520 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791086+0000", 2023-11-25T04:23:06.520 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8697999999999999e-05 2023-11-25T04:23:06.520 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:06.520 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:06.520 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:06.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.520 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.520 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:157 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'111, mlcod=11'106)", 2023-11-25T04:23:06.520 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:06.520 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.720512818, 2023-11-25T04:23:06.520 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050590280000000001, 2023-11-25T04:23:06.520 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:06.521 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:06.521 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:06.521 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.521 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:06.521 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:06.521 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:06.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.521 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.521 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:06.521 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:06.521 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:06.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.521 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.522 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:06.522 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786063+0000", 2023-11-25T04:23:06.522 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2600000000000002e-07 2023-11-25T04:23:06.522 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.522 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.522 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:06.522 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786069+0000", 2023-11-25T04:23:06.522 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8839999999999999e-06 2023-11-25T04:23:06.522 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.523 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.523 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:06.523 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786079+0000", 2023-11-25T04:23:06.523 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0124e-05 2023-11-25T04:23:06.523 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.523 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.523 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:06.523 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786085+0000", 2023-11-25T04:23:06.523 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6660000000000002e-06 2023-11-25T04:23:06.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.524 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.524 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:06.524 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788160+0000", 2023-11-25T04:23:06.524 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020747669999999999 2023-11-25T04:23:06.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.524 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.524 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:06.524 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788180+0000", 2023-11-25T04:23:06.524 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0208999999999999e-05 2023-11-25T04:23:06.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.525 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.525 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:06.525 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791087+0000", 2023-11-25T04:23:06.525 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029064630000000002 2023-11-25T04:23:06.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.525 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.525 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:06.525 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791121+0000", 2023-11-25T04:23:06.525 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4289000000000003e-05 2023-11-25T04:23:06.525 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:06.525 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:06.525 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:06.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.526 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.526 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:158 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'112, mlcod=11'106)", 2023-11-25T04:23:06.526 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:06.526 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.720096425000001, 2023-11-25T04:23:06.526 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046663030000000001, 2023-11-25T04:23:06.526 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:06.526 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:06.526 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:06.526 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.526 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:06.526 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:06.526 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:06.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.527 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.527 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:06.527 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:06.527 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:06.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.527 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.527 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:06.527 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786479+0000", 2023-11-25T04:23:06.527 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2799999999999996e-07 2023-11-25T04:23:06.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.527 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.527 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:06.528 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786498+0000", 2023-11-25T04:23:06.528 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9069e-05 2023-11-25T04:23:06.528 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.528 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.528 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:06.528 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786508+0000", 2023-11-25T04:23:06.528 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5810000000000003e-06 2023-11-25T04:23:06.528 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.528 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.528 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:06.528 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786512+0000", 2023-11-25T04:23:06.528 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4889999999999997e-06 2023-11-25T04:23:06.528 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.529 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.529 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:06.529 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788438+0000", 2023-11-25T04:23:06.529 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019257059999999999 2023-11-25T04:23:06.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.529 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.529 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:06.529 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788449+0000", 2023-11-25T04:23:06.529 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1419e-05 2023-11-25T04:23:06.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.529 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.529 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:06.529 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791122+0000", 2023-11-25T04:23:06.530 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002672822 2023-11-25T04:23:06.530 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.530 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.530 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:06.530 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791145+0000", 2023-11-25T04:23:06.530 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2589e-05 2023-11-25T04:23:06.530 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:06.530 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:06.530 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:06.530 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.530 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.530 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:206 1.0 e12/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 12'152, mlcod=12'148)", 2023-11-25T04:23:06.531 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:06.531 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.208327955000001, 2023-11-25T04:23:06.531 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046728189999999999, 2023-11-25T04:23:06.531 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:06.531 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:06.531 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:06.531 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.531 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:06.531 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:06.531 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:06.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.531 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.531 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:06.532 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:06.532 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:06.532 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.532 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.532 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:06.532 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298248+0000", 2023-11-25T04:23:06.532 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3600000000000003e-07 2023-11-25T04:23:06.532 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.532 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.532 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:06.532 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298278+0000", 2023-11-25T04:23:06.532 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0474e-05 2023-11-25T04:23:06.532 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.533 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.533 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:06.533 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298288+0000", 2023-11-25T04:23:06.533 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0156000000000001e-05 2023-11-25T04:23:06.533 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.533 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.533 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:06.533 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298293+0000", 2023-11-25T04:23:06.533 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7400000000000004e-06 2023-11-25T04:23:06.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.534 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.534 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:06.534 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298333+0000", 2023-11-25T04:23:06.534 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9666000000000003e-05 2023-11-25T04:23:06.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.534 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.534 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:06.534 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298351+0000", 2023-11-25T04:23:06.534 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8544999999999999e-05 2023-11-25T04:23:06.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.534 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.534 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:06.535 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302891+0000", 2023-11-25T04:23:06.535 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045398519999999996 2023-11-25T04:23:06.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.535 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:06.535 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302920+0000", 2023-11-25T04:23:06.535 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8750000000000001e-05 2023-11-25T04:23:06.535 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:06.535 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:06.535 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:06.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.535 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:14 3.15 3:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:06.536 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:06.536 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.473193598000002, 2023-11-25T04:23:06.536 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010990437, 2023-11-25T04:23:06.536 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:06.536 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:06.536 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:06.536 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.536 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:06.536 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:06.536 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:06.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.536 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.536 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:06.537 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:06.537 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:06.537 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.537 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.537 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:06.537 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033383+0000", 2023-11-25T04:23:06.537 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9870000000000002e-06 2023-11-25T04:23:06.537 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.537 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.537 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:06.537 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033398+0000", 2023-11-25T04:23:06.537 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4569999999999999e-05 2023-11-25T04:23:06.537 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.538 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.538 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:06.538 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033411+0000", 2023-11-25T04:23:06.538 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3543999999999999e-05 2023-11-25T04:23:06.538 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.538 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.538 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:06.538 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033423+0000", 2023-11-25T04:23:06.538 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1905999999999999e-05 2023-11-25T04:23:06.538 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.538 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.538 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:06.538 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034433+0000", 2023-11-25T04:23:06.539 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00101022 2023-11-25T04:23:06.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.539 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:06.539 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034539+0000", 2023-11-25T04:23:06.539 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010597199999999999 2023-11-25T04:23:06.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.539 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:06.539 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034667+0000", 2023-11-25T04:23:06.539 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012765500000000001 2023-11-25T04:23:06.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.540 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:06.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037803+0000", 2023-11-25T04:23:06.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031362220000000001 2023-11-25T04:23:06.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.540 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.540 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:06.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:23:06.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0064964669999999997 2023-11-25T04:23:06.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.540 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.540 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:06.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044331+0000", 2023-11-25T04:23:06.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1544999999999997e-05 2023-11-25T04:23:06.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.541 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.541 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:06.541 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044372+0000", 2023-11-25T04:23:06.541 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0349000000000003e-05 2023-11-25T04:23:06.541 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:06.541 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:06.541 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:06.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.541 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.541 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:15 3.15 3:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:06.541 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:06.541 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.472533947999999, 2023-11-25T04:23:06.541 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010464362, 2023-11-25T04:23:06.542 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:06.542 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:06.542 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:06.542 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.542 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:06.542 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:06.542 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:06.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.542 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.542 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:06.542 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:06.542 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:06.543 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.543 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.543 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:06.543 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034042+0000", 2023-11-25T04:23:06.543 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0999999999999997e-07 2023-11-25T04:23:06.543 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.543 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.543 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:06.543 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034048+0000", 2023-11-25T04:23:06.544 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0390000000000001e-06 2023-11-25T04:23:06.544 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.544 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.544 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:06.544 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034059+0000", 2023-11-25T04:23:06.544 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1182999999999999e-05 2023-11-25T04:23:06.544 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.544 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.544 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:06.544 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034067+0000", 2023-11-25T04:23:06.544 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1559999999999995e-06 2023-11-25T04:23:06.544 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.545 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.545 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:06.545 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035005+0000", 2023-11-25T04:23:06.545 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00093747399999999999 2023-11-25T04:23:06.545 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.545 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.545 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:06.545 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035077+0000", 2023-11-25T04:23:06.545 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2747e-05 2023-11-25T04:23:06.545 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.545 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.545 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:06.545 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035219+0000", 2023-11-25T04:23:06.546 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000141877 2023-11-25T04:23:06.546 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.546 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.546 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:06.546 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037810+0000", 2023-11-25T04:23:06.546 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002590752 2023-11-25T04:23:06.546 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.546 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.546 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:06.546 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044437+0000", 2023-11-25T04:23:06.546 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066270110000000004 2023-11-25T04:23:06.546 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.546 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.547 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:06.547 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044481+0000", 2023-11-25T04:23:06.547 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4106000000000001e-05 2023-11-25T04:23:06.547 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.547 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.547 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:06.547 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044505+0000", 2023-11-25T04:23:06.547 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4207e-05 2023-11-25T04:23:06.547 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:06.547 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:06.547 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:06.547 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.547 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.547 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:16 3.1d 3:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:06.548 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:06.548 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.4724921, 2023-11-25T04:23:06.548 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010413923, 2023-11-25T04:23:06.548 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:06.548 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:06.548 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:06.548 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.548 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:06.548 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:06.548 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:06.548 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.548 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.548 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:06.549 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:06.549 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:06.549 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.549 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.549 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:06.549 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:06.549 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4499999999999997e-07 2023-11-25T04:23:06.549 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.549 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.549 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:06.549 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034087+0000", 2023-11-25T04:23:06.549 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9770000000000002e-06 2023-11-25T04:23:06.549 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.550 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.550 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:06.550 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034094+0000", 2023-11-25T04:23:06.550 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8809999999999998e-06 2023-11-25T04:23:06.550 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.550 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.550 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:06.550 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034100+0000", 2023-11-25T04:23:06.550 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8440000000000003e-06 2023-11-25T04:23:06.550 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.550 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.550 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:06.550 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034993+0000", 2023-11-25T04:23:06.551 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00089336799999999996 2023-11-25T04:23:06.551 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.551 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.551 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:06.551 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035099+0000", 2023-11-25T04:23:06.551 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106037 2023-11-25T04:23:06.551 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.551 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.551 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:06.551 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035276+0000", 2023-11-25T04:23:06.551 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017631799999999999 2023-11-25T04:23:06.551 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.551 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.552 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:06.552 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037815+0000", 2023-11-25T04:23:06.552 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025395959999999999 2023-11-25T04:23:06.552 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.552 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.552 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:06.552 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044420+0000", 2023-11-25T04:23:06.552 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066045749999999997 2023-11-25T04:23:06.552 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.552 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.552 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:06.552 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044463+0000", 2023-11-25T04:23:06.552 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3205999999999999e-05 2023-11-25T04:23:06.553 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.553 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.553 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:06.553 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044497+0000", 2023-11-25T04:23:06.553 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3476000000000003e-05 2023-11-25T04:23:06.553 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:06.553 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:06.553 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:06.553 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.553 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.553 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:17 3.1d 3:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:06.553 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:06.554 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.472466891, 2023-11-25T04:23:06.554 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01127356, 2023-11-25T04:23:06.554 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:06.554 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:06.554 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:06.554 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.554 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:06.554 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:06.554 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:06.554 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.554 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.554 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:06.554 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:06.555 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:06.555 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.555 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.555 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:06.555 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:06.555 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1100000000000001e-07 2023-11-25T04:23:06.555 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.555 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.555 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:06.555 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034112+0000", 2023-11-25T04:23:06.555 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8600000000000003e-06 2023-11-25T04:23:06.555 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.555 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.556 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:06.556 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034120+0000", 2023-11-25T04:23:06.556 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7700000000000001e-06 2023-11-25T04:23:06.556 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.556 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.556 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:06.556 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034127+0000", 2023-11-25T04:23:06.556 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0999999999999998e-06 2023-11-25T04:23:06.556 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.556 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.556 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:06.556 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035601+0000", 2023-11-25T04:23:06.556 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014739410000000001 2023-11-25T04:23:06.557 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.557 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.557 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:06.557 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035685+0000", 2023-11-25T04:23:06.557 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3677999999999994e-05 2023-11-25T04:23:06.557 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.557 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.557 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:06.557 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035827+0000", 2023-11-25T04:23:06.557 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014232599999999999 2023-11-25T04:23:06.557 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.557 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.557 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:06.557 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038725+0000", 2023-11-25T04:23:06.558 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028982779999999998 2023-11-25T04:23:06.558 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.558 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.558 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:06.558 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045323+0000", 2023-11-25T04:23:06.558 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0065980780000000003 2023-11-25T04:23:06.558 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.558 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.558 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:06.558 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045354+0000", 2023-11-25T04:23:06.558 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0463e-05 2023-11-25T04:23:06.558 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.558 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.559 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:06.559 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045381+0000", 2023-11-25T04:23:06.559 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7654999999999999e-05 2023-11-25T04:23:06.559 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:06.559 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:06.559 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:06.559 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.559 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.559 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:22 3.e 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:06.559 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:06.559 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.472440477999999, 2023-11-25T04:23:06.559 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010133422, 2023-11-25T04:23:06.559 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:06.560 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:06.560 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:06.560 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.560 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:06.560 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:06.560 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:06.560 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.560 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.560 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:06.560 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:06.560 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:06.560 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.560 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.561 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:06.561 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034135+0000", 2023-11-25T04:23:06.561 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-07 2023-11-25T04:23:06.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.561 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:06.561 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034138+0000", 2023-11-25T04:23:06.561 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6009999999999999e-06 2023-11-25T04:23:06.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.561 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:06.561 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034145+0000", 2023-11-25T04:23:06.561 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1169999999999998e-06 2023-11-25T04:23:06.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.562 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.562 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:06.562 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034150+0000", 2023-11-25T04:23:06.562 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9080000000000003e-06 2023-11-25T04:23:06.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.562 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.562 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:06.562 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034209+0000", 2023-11-25T04:23:06.562 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8149000000000002e-05 2023-11-25T04:23:06.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.562 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.562 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:06.563 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034415+0000", 2023-11-25T04:23:06.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020637699999999999 2023-11-25T04:23:06.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.563 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:06.563 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034584+0000", 2023-11-25T04:23:06.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016870399999999999 2023-11-25T04:23:06.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.563 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:06.563 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.036979+0000", 2023-11-25T04:23:06.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023957509999999998 2023-11-25T04:23:06.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:06.564 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044161+0000", 2023-11-25T04:23:06.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0071811590000000003 2023-11-25T04:23:06.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:06.564 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044224+0000", 2023-11-25T04:23:06.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3454000000000006e-05 2023-11-25T04:23:06.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:06.565 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044268+0000", 2023-11-25T04:23:06.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3761999999999999e-05 2023-11-25T04:23:06.565 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:06.565 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:06.565 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:06.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.565 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:23 3.e 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:06.565 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:06.565 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.472409069000001, 2023-11-25T04:23:06.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010281676, 2023-11-25T04:23:06.565 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:06.565 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:06.566 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:06.566 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.566 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:06.566 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:06.566 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:06.566 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.566 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.566 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:06.566 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:06.566 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:06.566 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.566 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.566 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:06.566 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:06.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5799999999999999e-07 2023-11-25T04:23:06.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:06.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034169+0000", 2023-11-25T04:23:06.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1879999999999998e-06 2023-11-25T04:23:06.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:06.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034177+0000", 2023-11-25T04:23:06.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7020000000000001e-06 2023-11-25T04:23:06.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:06.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034184+0000", 2023-11-25T04:23:06.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3910000000000003e-06 2023-11-25T04:23:06.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:06.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034924+0000", 2023-11-25T04:23:06.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00074032999999999998 2023-11-25T04:23:06.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:06.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034991+0000", 2023-11-25T04:23:06.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7347000000000004e-05 2023-11-25T04:23:06.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:06.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035096+0000", 2023-11-25T04:23:06.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010488 2023-11-25T04:23:06.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:06.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037588+0000", 2023-11-25T04:23:06.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024918309999999999 2023-11-25T04:23:06.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:06.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044367+0000", 2023-11-25T04:23:06.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067787589999999997 2023-11-25T04:23:06.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.570 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:06.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044415+0000", 2023-11-25T04:23:06.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7803000000000003e-05 2023-11-25T04:23:06.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.570 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:06.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044447+0000", 2023-11-25T04:23:06.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2886999999999999e-05 2023-11-25T04:23:06.570 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:06.570 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:06.570 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:06.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.571 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:30 3.15 3:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:06.571 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:06.571 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.472350629000001, 2023-11-25T04:23:06.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010365070000000001, 2023-11-25T04:23:06.571 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:06.571 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:06.571 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:06.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:06.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:06.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:06.572 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.572 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.572 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:06.572 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:06.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:06.572 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.572 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.572 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:06.572 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034225+0000", 2023-11-25T04:23:06.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7400000000000002e-07 2023-11-25T04:23:06.572 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.572 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.573 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:06.573 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034229+0000", 2023-11-25T04:23:06.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3100000000000002e-06 2023-11-25T04:23:06.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.573 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:06.573 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034239+0000", 2023-11-25T04:23:06.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9140000000000003e-06 2023-11-25T04:23:06.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.573 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:06.573 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034246+0000", 2023-11-25T04:23:06.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0149999999999998e-06 2023-11-25T04:23:06.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:06.574 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035572+0000", 2023-11-25T04:23:06.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001325655 2023-11-25T04:23:06.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:06.574 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035730+0000", 2023-11-25T04:23:06.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015827899999999999 2023-11-25T04:23:06.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:06.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035857+0000", 2023-11-25T04:23:06.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012644699999999999 2023-11-25T04:23:06.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:06.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038737+0000", 2023-11-25T04:23:06.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002880077 2023-11-25T04:23:06.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:06.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044540+0000", 2023-11-25T04:23:06.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058034239999999997 2023-11-25T04:23:06.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.576 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:06.576 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044569+0000", 2023-11-25T04:23:06.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8388999999999999e-05 2023-11-25T04:23:06.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.576 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:06.576 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044589+0000", 2023-11-25T04:23:06.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0686000000000002e-05 2023-11-25T04:23:06.576 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:06.576 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:06.576 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:06.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.577 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:33 3.1b 3:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:06.577 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:06.577 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.472305130999999, 2023-11-25T04:23:06.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099754049999999997, 2023-11-25T04:23:06.577 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:06.577 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:23:06.577 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:06.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.577 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:06.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:06.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:06.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.578 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:06.578 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:06.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:06.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.578 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:06.578 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:06.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4799999999999998e-07 2023-11-25T04:23:06.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.578 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:06.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034273+0000", 2023-11-25T04:23:06.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7819999999999998e-06 2023-11-25T04:23:06.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:06.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034285+0000", 2023-11-25T04:23:06.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2058e-05 2023-11-25T04:23:06.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:06.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034291+0000", 2023-11-25T04:23:06.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0680000000000002e-06 2023-11-25T04:23:06.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.581 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:06.581 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034370+0000", 2023-11-25T04:23:06.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8452e-05 2023-11-25T04:23:06.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.581 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:06.581 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034477+0000", 2023-11-25T04:23:06.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107471 2023-11-25T04:23:06.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:06.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044245+0000", 2023-11-25T04:23:06.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0097679259999999997 2023-11-25T04:23:06.582 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:06.582 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:06.582 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:06.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.582 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:36 3.1a 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:06.582 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:06.582 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.472258457999999, 2023-11-25T04:23:06.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058179099999999999, 2023-11-25T04:23:06.583 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:06.583 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:23:06.583 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:06.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.583 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:06.583 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:06.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:06.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.583 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:06.583 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:06.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:06.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.584 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:06.584 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034317+0000", 2023-11-25T04:23:06.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1999999999999999e-07 2023-11-25T04:23:06.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.584 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:06.584 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034321+0000", 2023-11-25T04:23:06.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.233e-06 2023-11-25T04:23:06.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:06.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034337+0000", 2023-11-25T04:23:06.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5308999999999999e-05 2023-11-25T04:23:06.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:06.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034354+0000", 2023-11-25T04:23:06.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7411999999999999e-05 2023-11-25T04:23:06.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:06.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034465+0000", 2023-11-25T04:23:06.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011154699999999999 2023-11-25T04:23:06.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:06.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034560+0000", 2023-11-25T04:23:06.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4186999999999998e-05 2023-11-25T04:23:06.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:06.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.040134+0000", 2023-11-25T04:23:06.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055746019999999997 2023-11-25T04:23:06.586 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:06.586 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:06.587 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:06.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.587 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:9 3.10 e18/16 3:09401a81:::601.00000000:head v 18'4, mlcod=0'0)", 2023-11-25T04:23:06.587 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:06.587 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.46336415, 2023-11-25T04:23:06.587 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061641020000000003, 2023-11-25T04:23:06.587 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:06.587 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:06.587 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:06.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.587 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:06.588 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:06.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:06.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.588 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:06.588 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:06.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:06.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.588 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:06.588 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:06.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5300000000000001e-07 2023-11-25T04:23:06.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:06.589 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043222+0000", 2023-11-25T04:23:06.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.082e-05 2023-11-25T04:23:06.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:06.589 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043228+0000", 2023-11-25T04:23:06.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6699999999999997e-06 2023-11-25T04:23:06.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:06.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043236+0000", 2023-11-25T04:23:06.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8189999999999992e-06 2023-11-25T04:23:06.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:06.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043507+0000", 2023-11-25T04:23:06.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000271012 2023-11-25T04:23:06.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:06.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043530+0000", 2023-11-25T04:23:06.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2659000000000001e-05 2023-11-25T04:23:06.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.591 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:06.591 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049321+0000", 2023-11-25T04:23:06.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057915129999999999 2023-11-25T04:23:06.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.591 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:06.591 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049375+0000", 2023-11-25T04:23:06.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3455999999999998e-05 2023-11-25T04:23:06.591 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:06.591 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:06.591 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:06.592 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.592 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.592 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:1 3.14 e18/16 3:292cf221:::200.00000000:head v 18'3, mlcod=0'0)", 2023-11-25T04:23:06.592 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:06.592 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.463327437, 2023-11-25T04:23:06.592 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0060983599999999997, 2023-11-25T04:23:06.592 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:06.592 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:06.592 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:06.592 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.592 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:06.592 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:06.592 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:06.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:06.593 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:06.593 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:06.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:06.593 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043248+0000", 2023-11-25T04:23:06.593 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9399999999999995e-07 2023-11-25T04:23:06.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:06.594 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043252+0000", 2023-11-25T04:23:06.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1799999999999998e-06 2023-11-25T04:23:06.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.594 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:06.594 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043259+0000", 2023-11-25T04:23:06.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0049999999999998e-06 2023-11-25T04:23:06.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.594 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:06.594 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043265+0000", 2023-11-25T04:23:06.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0859999999999998e-06 2023-11-25T04:23:06.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:06.595 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043311+0000", 2023-11-25T04:23:06.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6082999999999999e-05 2023-11-25T04:23:06.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:06.595 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043330+0000", 2023-11-25T04:23:06.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8556e-05 2023-11-25T04:23:06.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.596 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:06.596 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049303+0000", 2023-11-25T04:23:06.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059733470000000004 2023-11-25T04:23:06.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.596 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.596 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:06.596 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049346+0000", 2023-11-25T04:23:06.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2608999999999998e-05 2023-11-25T04:23:06.596 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:06.596 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:06.596 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:06.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.596 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.597 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:3 3.14 e18/16 3:287034c4:::1.00000000.inode:head v 18'4, mlcod=0'0)", 2023-11-25T04:23:06.597 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:06.597 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.462810931, 2023-11-25T04:23:06.597 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057040090000000003, 2023-11-25T04:23:06.597 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:06.597 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:06.597 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:06.597 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.597 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:06.597 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:06.597 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:06.597 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:06.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:06.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:06.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:06.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:06.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9799999999999999e-07 2023-11-25T04:23:06.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:06.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043785+0000", 2023-11-25T04:23:06.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0721999999999999e-05 2023-11-25T04:23:06.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:06.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043793+0000", 2023-11-25T04:23:06.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5839999999999995e-06 2023-11-25T04:23:06.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:06.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043803+0000", 2023-11-25T04:23:06.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0729999999999994e-06 2023-11-25T04:23:06.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.600 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:06.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043843+0000", 2023-11-25T04:23:06.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0747999999999997e-05 2023-11-25T04:23:06.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.600 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:06.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043856+0000", 2023-11-25T04:23:06.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2697000000000001e-05 2023-11-25T04:23:06.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.600 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:06.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049426+0000", 2023-11-25T04:23:06.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055704409999999998 2023-11-25T04:23:06.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:06.601 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049468+0000", 2023-11-25T04:23:06.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1446000000000003e-05 2023-11-25T04:23:06.601 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:06.601 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:06.601 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:06.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.601 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "pg_update_log_missing(3.14 epoch 18/16 rep_tid 171 entries 18'5 (0'0) error 3:2e2fa760:::200.00000001:head by mds.0.3:32 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:23:06.601 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:06.602 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.462649968000001, 2023-11-25T04:23:06.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005556938, 2023-11-25T04:23:06.602 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:06.602 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2023-11-25T04:23:06.602 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:06.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:06.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:06.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:06.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:06.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:06.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:06.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:06.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:06.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.03e-07 2023-11-25T04:23:06.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:06.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043931+0000", 2023-11-25T04:23:06.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1079999999999998e-06 2023-11-25T04:23:06.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:06.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043944+0000", 2023-11-25T04:23:06.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2852e-05 2023-11-25T04:23:06.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:06.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043955+0000", 2023-11-25T04:23:06.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1037e-05 2023-11-25T04:23:06.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:06.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044088+0000", 2023-11-25T04:23:06.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013269599999999999 2023-11-25T04:23:06.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.605 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:06.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049482+0000", 2023-11-25T04:23:06.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053940419999999999 2023-11-25T04:23:06.605 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:06.605 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:06.605 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:06.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.605 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:10 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'5, mlcod=0'0)", 2023-11-25T04:23:06.606 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:06.606 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.462559939999998, 2023-11-25T04:23:06.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054493099999999997, 2023-11-25T04:23:06.606 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:06.606 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:06.606 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:06.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:06.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:06.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:06.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:06.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:06.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:06.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:06.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:06.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6100000000000002e-07 2023-11-25T04:23:06.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:06.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044019+0000", 2023-11-25T04:23:06.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.839e-06 2023-11-25T04:23:06.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:06.608 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044026+0000", 2023-11-25T04:23:06.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5869999999999998e-06 2023-11-25T04:23:06.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:06.608 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044031+0000", 2023-11-25T04:23:06.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4909999999999996e-06 2023-11-25T04:23:06.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:06.609 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044070+0000", 2023-11-25T04:23:06.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8829999999999999e-05 2023-11-25T04:23:06.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:06.609 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044081+0000", 2023-11-25T04:23:06.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1447999999999999e-05 2023-11-25T04:23:06.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:06.609 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049406+0000", 2023-11-25T04:23:06.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053246480000000004 2023-11-25T04:23:06.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.610 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:06.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049464+0000", 2023-11-25T04:23:06.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8205999999999998e-05 2023-11-25T04:23:06.610 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:06.610 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:06.610 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:06.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.610 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:11 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'6, mlcod=0'0)", 2023-11-25T04:23:06.611 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:06.611 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 19.462358676000001, 2023-11-25T04:23:06.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052673499999999996, 2023-11-25T04:23:06.611 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:06.611 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:06.611 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:06.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:06.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:06.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:06.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:06.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:06.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:06.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:06.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:06.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0199999999999998e-07 2023-11-25T04:23:06.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:06.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044220+0000", 2023-11-25T04:23:06.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7720000000000001e-06 2023-11-25T04:23:06.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.613 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:06.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044226+0000", 2023-11-25T04:23:06.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8579999999999996e-06 2023-11-25T04:23:06.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.613 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:06.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044232+0000", 2023-11-25T04:23:06.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1700000000000002e-06 2023-11-25T04:23:06.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.613 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:06.614 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044274+0000", 2023-11-25T04:23:06.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1254999999999997e-05 2023-11-25T04:23:06.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:06.614 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:23:06.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6880999999999999e-05 2023-11-25T04:23:06.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:06.614 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049466+0000", 2023-11-25T04:23:06.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051659219999999999 2023-11-25T04:23:06.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:06.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:06.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:06.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049483+0000", 2023-11-25T04:23:06.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.719e-05 2023-11-25T04:23:06.615 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:06.615 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:06.615 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:06.615 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:06.615 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:06.615 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:06.617 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:23:06.617 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:06.615+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:06.717 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:06.716+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:06.726 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:06.726 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:23:06.726 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2023-11-25T04:23:06.726 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:06.736 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:23:06.820 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:06.819+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:06.881 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:06.882 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:23:06.882 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2023-11-25T04:23:06.882 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2023-11-25T04:23:06.882 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:06.890 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:23:06.919 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:06.918+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:07.021 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:07.019+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:07.038 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:07.038 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2023-11-25T04:23:07.038 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2023-11-25T04:23:07.038 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2023-11-25T04:23:07.038 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.038 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:07.039 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:07.039 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 20.004042066, 2023-11-25T04:23:07.039 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01072943, 2023-11-25T04:23:07.039 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:07.039 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:07.039 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:07.039 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.039 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:07.039 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:07.039 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:07.039 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.039 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.039 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:07.040 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:07.040 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:07.040 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.040 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.040 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:07.040 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031829+0000", 2023-11-25T04:23:07.040 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.666e-06 2023-11-25T04:23:07.040 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.040 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.040 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:07.040 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031839+0000", 2023-11-25T04:23:07.040 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9490000000000008e-06 2023-11-25T04:23:07.041 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.041 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.041 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:07.041 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031857+0000", 2023-11-25T04:23:07.041 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8093000000000001e-05 2023-11-25T04:23:07.041 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.041 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.042 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:07.042 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031882+0000", 2023-11-25T04:23:07.042 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4294000000000002e-05 2023-11-25T04:23:07.042 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.042 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.042 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:07.042 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033130+0000", 2023-11-25T04:23:07.042 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001248264 2023-11-25T04:23:07.042 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.042 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.042 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:07.042 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033426+0000", 2023-11-25T04:23:07.043 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029620999999999998 2023-11-25T04:23:07.043 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.043 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.043 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:07.043 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033814+0000", 2023-11-25T04:23:07.043 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000387742 2023-11-25T04:23:07.043 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.043 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.043 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:07.043 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036793+0000", 2023-11-25T04:23:07.044 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002979598 2023-11-25T04:23:07.044 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.044 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.044 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:07.044 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042455+0000", 2023-11-25T04:23:07.044 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0056619260000000003 2023-11-25T04:23:07.044 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.044 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.044 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:07.045 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042502+0000", 2023-11-25T04:23:07.045 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7139999999999999e-05 2023-11-25T04:23:07.045 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.045 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.045 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:07.046 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042557+0000", 2023-11-25T04:23:07.046 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4548e-05 2023-11-25T04:23:07.046 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:07.046 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:07.046 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:07.046 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.046 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.046 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:07.046 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:07.046 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 20.002843531, 2023-11-25T04:23:07.046 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011505202000000001, 2023-11-25T04:23:07.046 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:07.047 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:07.047 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:07.047 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.047 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:07.047 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:07.047 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:07.047 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.047 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.047 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:07.047 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:07.047 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:07.047 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.047 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.048 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:07.048 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033027+0000", 2023-11-25T04:23:07.048 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0699999999999999e-06 2023-11-25T04:23:07.048 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.048 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.048 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:07.048 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033035+0000", 2023-11-25T04:23:07.048 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0830000000000004e-06 2023-11-25T04:23:07.048 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.048 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.048 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:07.048 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033059+0000", 2023-11-25T04:23:07.049 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3975e-05 2023-11-25T04:23:07.049 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.049 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.049 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:07.049 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033482+0000", 2023-11-25T04:23:07.049 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042253700000000001 2023-11-25T04:23:07.049 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.049 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.049 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:07.049 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034520+0000", 2023-11-25T04:23:07.049 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001037991 2023-11-25T04:23:07.049 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.050 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.050 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:07.050 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034611+0000", 2023-11-25T04:23:07.050 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1113999999999995e-05 2023-11-25T04:23:07.050 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.050 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.050 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:07.050 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034769+0000", 2023-11-25T04:23:07.050 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000157641 2023-11-25T04:23:07.050 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.050 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.050 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:07.050 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038292+0000", 2023-11-25T04:23:07.051 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035231160000000002 2023-11-25T04:23:07.051 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.051 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.051 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:07.051 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044440+0000", 2023-11-25T04:23:07.051 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0061486980000000002 2023-11-25T04:23:07.051 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.051 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.051 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:07.051 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044495+0000", 2023-11-25T04:23:07.051 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4339000000000003e-05 2023-11-25T04:23:07.051 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.052 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.052 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:07.052 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044531+0000", 2023-11-25T04:23:07.052 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6637999999999997e-05 2023-11-25T04:23:07.052 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:07.052 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:07.052 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:07.052 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.052 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.052 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:07.052 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:07.052 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 20.002369176999999, 2023-11-25T04:23:07.052 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0057634039999999997, 2023-11-25T04:23:07.053 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:07.053 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:07.053 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:07.053 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.053 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:07.053 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:07.053 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:07.053 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.053 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.053 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:07.053 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:07.053 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:07.053 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.054 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.054 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:07.054 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:07.054 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7900000000000007e-07 2023-11-25T04:23:07.054 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.054 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.054 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:07.054 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033507+0000", 2023-11-25T04:23:07.054 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8900000000000004e-06 2023-11-25T04:23:07.054 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.054 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.055 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:07.055 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033519+0000", 2023-11-25T04:23:07.055 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1756000000000001e-05 2023-11-25T04:23:07.055 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.055 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.055 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:07.055 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033527+0000", 2023-11-25T04:23:07.055 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3610000000000001e-06 2023-11-25T04:23:07.055 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.055 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.055 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:07.055 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034451+0000", 2023-11-25T04:23:07.055 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00092372400000000001 2023-11-25T04:23:07.056 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.056 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.056 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:07.056 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034809+0000", 2023-11-25T04:23:07.056 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00035757699999999998 2023-11-25T04:23:07.056 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.056 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.056 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:23:07.056 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034965+0000", 2023-11-25T04:23:07.056 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015638699999999999 2023-11-25T04:23:07.056 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.056 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.057 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:07.057 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038299+0000", 2023-11-25T04:23:07.057 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0033336559999999999 2023-11-25T04:23:07.057 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.057 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.057 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:07.057 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039169+0000", 2023-11-25T04:23:07.057 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00087066400000000005 2023-11-25T04:23:07.057 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.057 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.057 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:07.057 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039226+0000", 2023-11-25T04:23:07.057 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6511999999999997e-05 2023-11-25T04:23:07.058 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.058 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.058 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:07.058 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039264+0000", 2023-11-25T04:23:07.058 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7898000000000002e-05 2023-11-25T04:23:07.058 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:07.058 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:07.058 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:07.058 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.058 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.058 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:25 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:07.058 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:07.059 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 20.002290593000001, 2023-11-25T04:23:07.059 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096656169999999996, 2023-11-25T04:23:07.059 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:07.059 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:07.059 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:07.059 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.059 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:07.059 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:07.059 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:07.059 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.059 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.059 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:07.059 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:07.060 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:07.060 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.060 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.060 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:07.060 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033580+0000", 2023-11-25T04:23:07.060 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8199999999999999e-07 2023-11-25T04:23:07.060 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.060 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.060 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:07.060 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033585+0000", 2023-11-25T04:23:07.060 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3680000000000001e-06 2023-11-25T04:23:07.060 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.060 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.061 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:07.061 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033600+0000", 2023-11-25T04:23:07.061 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4331000000000001e-05 2023-11-25T04:23:07.061 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.061 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.061 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:07.061 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033606+0000", 2023-11-25T04:23:07.061 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5780000000000004e-06 2023-11-25T04:23:07.061 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.061 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.061 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:07.061 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035747+0000", 2023-11-25T04:23:07.061 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0021406530000000002 2023-11-25T04:23:07.062 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.062 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.062 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:07.062 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035901+0000", 2023-11-25T04:23:07.062 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000153931 2023-11-25T04:23:07.062 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.062 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.062 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:23:07.062 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036193+0000", 2023-11-25T04:23:07.063 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029211299999999999 2023-11-25T04:23:07.063 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.063 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.063 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:07.063 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038655+0000", 2023-11-25T04:23:07.063 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0024621640000000002 2023-11-25T04:23:07.063 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.063 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.063 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:07.063 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043130+0000", 2023-11-25T04:23:07.063 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.004474698 2023-11-25T04:23:07.063 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.063 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.064 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:07.064 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043204+0000", 2023-11-25T04:23:07.064 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4091999999999995e-05 2023-11-25T04:23:07.064 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.064 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.064 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:07.064 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043245+0000", 2023-11-25T04:23:07.064 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0907e-05 2023-11-25T04:23:07.064 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:07.064 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:07.064 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:07.064 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.064 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.065 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:26 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:07.065 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:07.065 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 20.002254769, 2023-11-25T04:23:07.065 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011086945000000001, 2023-11-25T04:23:07.065 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:07.065 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:07.065 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:07.065 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.065 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:07.065 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:07.065 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:07.065 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.065 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.066 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:07.066 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:07.066 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:07.066 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.066 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.066 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:07.066 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:07.066 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1699999999999998e-07 2023-11-25T04:23:07.067 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.067 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.067 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:07.067 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033627+0000", 2023-11-25T04:23:07.067 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.163e-05 2023-11-25T04:23:07.067 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.067 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.067 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:07.067 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033636+0000", 2023-11-25T04:23:07.067 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6859999999999995e-06 2023-11-25T04:23:07.067 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.067 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.067 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:07.068 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033642+0000", 2023-11-25T04:23:07.068 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3149999999999997e-06 2023-11-25T04:23:07.068 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.068 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.068 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:07.068 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035414+0000", 2023-11-25T04:23:07.068 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0017716380000000001 2023-11-25T04:23:07.068 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.068 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.068 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:07.068 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035539+0000", 2023-11-25T04:23:07.068 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000125477 2023-11-25T04:23:07.068 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.069 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.069 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:07.069 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035672+0000", 2023-11-25T04:23:07.069 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000133316 2023-11-25T04:23:07.069 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.069 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.069 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:07.069 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038649+0000", 2023-11-25T04:23:07.069 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029764679999999999 2023-11-25T04:23:07.069 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.069 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.069 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:07.070 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044578+0000", 2023-11-25T04:23:07.070 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0059293510000000002 2023-11-25T04:23:07.070 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.070 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.070 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:07.070 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044651+0000", 2023-11-25T04:23:07.070 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2513000000000002e-05 2023-11-25T04:23:07.070 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.070 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.070 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:07.070 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044702+0000", 2023-11-25T04:23:07.070 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1034e-05 2023-11-25T04:23:07.070 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:07.071 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:07.071 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:07.071 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.071 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.071 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:29 3.b 3:d0630e4c:::mds0_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:07.071 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:07.071 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 20.002216849, 2023-11-25T04:23:07.071 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010980227, 2023-11-25T04:23:07.071 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:07.071 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:07.072 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:07.072 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.072 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:07.072 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:07.072 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:07.072 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.072 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.072 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:07.072 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:07.072 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:07.072 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.072 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.073 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:07.073 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:07.073 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3499999999999996e-07 2023-11-25T04:23:07.073 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.073 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.073 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:07.073 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033658+0000", 2023-11-25T04:23:07.073 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1280000000000001e-06 2023-11-25T04:23:07.073 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.073 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.073 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:07.073 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033667+0000", 2023-11-25T04:23:07.073 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1430000000000004e-06 2023-11-25T04:23:07.074 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.074 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.074 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:07.074 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033678+0000", 2023-11-25T04:23:07.074 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0906e-05 2023-11-25T04:23:07.074 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.074 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.074 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:07.074 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034456+0000", 2023-11-25T04:23:07.074 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00077833899999999996 2023-11-25T04:23:07.074 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.074 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.075 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:07.075 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034575+0000", 2023-11-25T04:23:07.075 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011926000000000001 2023-11-25T04:23:07.075 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.075 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.075 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:07.075 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035001+0000", 2023-11-25T04:23:07.075 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042550999999999998 2023-11-25T04:23:07.075 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.075 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.075 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:07.075 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.037923+0000", 2023-11-25T04:23:07.075 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029219670000000001 2023-11-25T04:23:07.076 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.076 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.076 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:07.076 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044538+0000", 2023-11-25T04:23:07.076 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0066152349999999997 2023-11-25T04:23:07.076 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.076 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.076 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:07.076 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044594+0000", 2023-11-25T04:23:07.076 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5593000000000001e-05 2023-11-25T04:23:07.076 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.076 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.076 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:07.077 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044633+0000", 2023-11-25T04:23:07.077 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9511000000000002e-05 2023-11-25T04:23:07.077 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:07.077 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:07.077 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:07.077 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.077 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.077 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:34 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:07.077 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:07.077 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 20.001966736, 2023-11-25T04:23:07.077 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010920445000000001, 2023-11-25T04:23:07.077 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:07.077 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:07.078 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:07.078 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.078 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:07.078 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:07.078 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:07.078 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.078 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.078 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:07.078 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:07.078 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:07.078 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.078 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.078 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:07.079 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033904+0000", 2023-11-25T04:23:07.079 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6599999999999994e-07 2023-11-25T04:23:07.079 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.079 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.079 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:07.079 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033909+0000", 2023-11-25T04:23:07.079 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5639999999999996e-06 2023-11-25T04:23:07.079 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.079 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.079 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:07.080 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033923+0000", 2023-11-25T04:23:07.080 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3028e-05 2023-11-25T04:23:07.080 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.080 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.080 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:07.080 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034043+0000", 2023-11-25T04:23:07.080 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000120328 2023-11-25T04:23:07.080 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.080 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.080 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:07.080 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035576+0000", 2023-11-25T04:23:07.080 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015333720000000001 2023-11-25T04:23:07.080 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.081 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.081 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:07.081 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035699+0000", 2023-11-25T04:23:07.081 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000123001 2023-11-25T04:23:07.081 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.081 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.081 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:07.081 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044823+0000", 2023-11-25T04:23:07.081 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0091241859999999994 2023-11-25T04:23:07.081 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:07.081 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:07.081 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:07.081 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.082 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.082 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:35 3.1f 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:07.082 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:07.082 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 20.000662531, 2023-11-25T04:23:07.082 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096164779999999995, 2023-11-25T04:23:07.082 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:07.082 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:07.082 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:07.082 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.082 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:07.082 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:07.082 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:07.082 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.082 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.083 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:07.083 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:07.083 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:07.083 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.083 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.083 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:07.083 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035208+0000", 2023-11-25T04:23:07.083 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2699999999999998e-07 2023-11-25T04:23:07.083 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.083 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.083 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:07.083 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035213+0000", 2023-11-25T04:23:07.083 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.6070000000000001e-06 2023-11-25T04:23:07.084 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.084 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.084 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:07.084 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035350+0000", 2023-11-25T04:23:07.084 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013693599999999999 2023-11-25T04:23:07.084 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.084 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.084 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:07.084 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035358+0000", 2023-11-25T04:23:07.084 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0099999999999995e-06 2023-11-25T04:23:07.084 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.084 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.084 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:07.084 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036260+0000", 2023-11-25T04:23:07.085 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00090240100000000005 2023-11-25T04:23:07.085 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.085 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.085 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:07.085 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036517+0000", 2023-11-25T04:23:07.085 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00025736299999999998 2023-11-25T04:23:07.085 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.085 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.085 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:07.085 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044824+0000", 2023-11-25T04:23:07.085 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0083063340000000003 2023-11-25T04:23:07.085 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:07.085 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:07.085 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:07.086 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.086 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.086 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:37 3.1e 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:07.086 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:07.086 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 20.000495845, 2023-11-25T04:23:07.086 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0078322640000000002, 2023-11-25T04:23:07.086 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:07.086 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:07.086 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:07.086 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.086 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:07.086 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:07.087 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:07.087 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.087 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.087 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:07.087 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:07.087 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:07.087 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.087 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.087 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:07.087 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035375+0000", 2023-11-25T04:23:07.087 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9100000000000003e-07 2023-11-25T04:23:07.087 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.087 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.087 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:07.088 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035379+0000", 2023-11-25T04:23:07.088 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9430000000000002e-06 2023-11-25T04:23:07.088 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.088 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.088 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:07.088 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035388+0000", 2023-11-25T04:23:07.088 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2760000000000006e-06 2023-11-25T04:23:07.088 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.088 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.088 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:07.088 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035396+0000", 2023-11-25T04:23:07.088 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8480000000000001e-06 2023-11-25T04:23:07.088 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.088 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.089 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:07.089 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035452+0000", 2023-11-25T04:23:07.089 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5831000000000001e-05 2023-11-25T04:23:07.089 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.089 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.089 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:07.089 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035934+0000", 2023-11-25T04:23:07.089 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00048265599999999998 2023-11-25T04:23:07.089 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.089 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.089 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:07.089 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043206+0000", 2023-11-25T04:23:07.089 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.007271919 2023-11-25T04:23:07.089 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:07.090 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:07.090 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:07.090 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.090 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.090 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing(3.1a epoch 18/16 rep_tid 3 entries 18'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:36 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:23:07.090 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:07.090 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 19.999220259000001, 2023-11-25T04:23:07.090 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002073414, 2023-11-25T04:23:07.090 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:07.090 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:07.090 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:07.090 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.090 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:07.091 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:07.091 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:07.091 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.091 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.091 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:07.091 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:07.091 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:07.091 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.091 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.091 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:07.091 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036650+0000", 2023-11-25T04:23:07.091 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7300000000000002e-07 2023-11-25T04:23:07.091 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.091 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.092 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:07.092 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036659+0000", 2023-11-25T04:23:07.092 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.7499999999999992e-06 2023-11-25T04:23:07.092 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.092 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.092 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:07.092 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036673+0000", 2023-11-25T04:23:07.092 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4917e-05 2023-11-25T04:23:07.092 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.092 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.092 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:07.092 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036687+0000", 2023-11-25T04:23:07.092 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3883000000000001e-05 2023-11-25T04:23:07.092 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.093 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.093 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:07.093 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036781+0000", 2023-11-25T04:23:07.093 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3219000000000003e-05 2023-11-25T04:23:07.093 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.093 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.093 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:07.093 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038723+0000", 2023-11-25T04:23:07.093 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019422720000000001 2023-11-25T04:23:07.093 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:07.093 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:07.093 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:07.093 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.093 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.094 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:28 3.13 e18/16 3:cd5a64a3:::100.00000000:head v 18'1, mlcod=0'0)", 2023-11-25T04:23:07.094 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:07.094 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 19.995559101000001, 2023-11-25T04:23:07.094 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001850546, 2023-11-25T04:23:07.094 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:07.094 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:07.094 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:07.094 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.094 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:07.094 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:07.094 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:07.094 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.094 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.095 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:07.095 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:07.095 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:07.095 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.095 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.095 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:07.095 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:07.095 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7800000000000002e-07 2023-11-25T04:23:07.095 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.095 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.095 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:07.095 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040331+0000", 2023-11-25T04:23:07.095 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0449999999999999e-05 2023-11-25T04:23:07.095 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.096 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.096 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:07.096 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040339+0000", 2023-11-25T04:23:07.096 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0709999999999995e-06 2023-11-25T04:23:07.096 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.096 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.096 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:07.096 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040347+0000", 2023-11-25T04:23:07.096 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7079999999999997e-06 2023-11-25T04:23:07.096 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.096 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.096 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:07.096 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040410+0000", 2023-11-25T04:23:07.096 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2453000000000007e-05 2023-11-25T04:23:07.097 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.097 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.097 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:07.097 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040422+0000", 2023-11-25T04:23:07.097 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2429000000000001e-05 2023-11-25T04:23:07.097 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.097 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.097 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:07.097 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042129+0000", 2023-11-25T04:23:07.097 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001707266 2023-11-25T04:23:07.097 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.097 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.097 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:07.097 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042161+0000", 2023-11-25T04:23:07.098 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1791000000000001e-05 2023-11-25T04:23:07.098 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:07.098 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:07.098 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:07.098 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.098 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.098 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:27 3.13 e18/16 3:c91d4a1d:::mds0_inotable:head v 18'2, mlcod=0'0)", 2023-11-25T04:23:07.098 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:07.098 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 19.995355578000002, 2023-11-25T04:23:07.098 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0016886430000000001, 2023-11-25T04:23:07.098 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:07.098 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:07.098 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:07.098 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.099 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:07.099 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:07.099 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:07.099 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.099 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.099 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:07.099 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:07.099 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:07.099 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.099 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.099 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:07.099 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:07.099 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3999999999999997e-07 2023-11-25T04:23:07.099 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.100 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.100 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:07.100 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040520+0000", 2023-11-25T04:23:07.100 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2989999999999996e-06 2023-11-25T04:23:07.100 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.100 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.100 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:07.100 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040527+0000", 2023-11-25T04:23:07.100 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5730000000000002e-06 2023-11-25T04:23:07.100 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.100 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.100 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:07.100 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040535+0000", 2023-11-25T04:23:07.100 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8630000000000008e-06 2023-11-25T04:23:07.101 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.101 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.101 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:07.101 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040683+0000", 2023-11-25T04:23:07.101 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000148002 2023-11-25T04:23:07.101 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.101 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.101 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:07.101 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040700+0000", 2023-11-25T04:23:07.101 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7065e-05 2023-11-25T04:23:07.101 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.101 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.101 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:07.102 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042163+0000", 2023-11-25T04:23:07.102 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001462767 2023-11-25T04:23:07.102 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.102 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.102 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:07.102 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042203+0000", 2023-11-25T04:23:07.102 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9734000000000003e-05 2023-11-25T04:23:07.102 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:07.102 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:07.102 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:07.102 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.102 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.102 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:20 3.1c e18/16 3:3ed09add:::607.00000000:head v 18'1, mlcod=0'0)", 2023-11-25T04:23:07.102 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:07.103 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 19.995320537000001, 2023-11-25T04:23:07.103 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015271, 2023-11-25T04:23:07.103 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:07.103 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:07.103 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:07.103 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.103 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:07.103 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:07.103 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:07.103 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.103 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.103 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:07.103 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:07.103 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:07.104 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.104 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.104 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:07.104 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:07.104 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4799999999999999e-07 2023-11-25T04:23:07.104 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.104 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.104 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:07.104 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040561+0000", 2023-11-25T04:23:07.104 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1800000000000001e-05 2023-11-25T04:23:07.104 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.104 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.104 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:07.104 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040568+0000", 2023-11-25T04:23:07.105 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7730000000000002e-06 2023-11-25T04:23:07.105 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.105 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.105 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:07.105 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040587+0000", 2023-11-25T04:23:07.105 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8691000000000001e-05 2023-11-25T04:23:07.105 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.105 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.105 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:07.105 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040638+0000", 2023-11-25T04:23:07.105 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1068e-05 2023-11-25T04:23:07.105 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.105 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.105 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:07.106 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040670+0000", 2023-11-25T04:23:07.106 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2125999999999997e-05 2023-11-25T04:23:07.106 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.106 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.106 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:07.106 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042032+0000", 2023-11-25T04:23:07.106 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001361821 2023-11-25T04:23:07.106 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.106 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.106 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:07.106 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042076+0000", 2023-11-25T04:23:07.106 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4472999999999999e-05 2023-11-25T04:23:07.106 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:07.106 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:07.107 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:07.107 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.107 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.107 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:21 3.1c e18/16 3:3ed09add:::607.00000000:head v 18'2, mlcod=0'0)", 2023-11-25T04:23:07.107 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:07.107 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 19.995085811999999, 2023-11-25T04:23:07.107 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0013228160000000001, 2023-11-25T04:23:07.107 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:07.107 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:07.107 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:07.107 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.107 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:07.107 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:07.107 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:07.108 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.108 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.108 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:07.108 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:07.108 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:07.108 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.108 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.108 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:07.108 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:07.108 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9299999999999999e-07 2023-11-25T04:23:07.108 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.108 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.108 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:07.108 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040790+0000", 2023-11-25T04:23:07.109 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5770000000000001e-06 2023-11-25T04:23:07.109 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.109 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.109 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:07.109 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040799+0000", 2023-11-25T04:23:07.109 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6439999999999995e-06 2023-11-25T04:23:07.109 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.109 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.109 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:07.109 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040806+0000", 2023-11-25T04:23:07.109 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0180000000000004e-06 2023-11-25T04:23:07.109 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.109 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.109 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:07.110 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040907+0000", 2023-11-25T04:23:07.110 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000100096 2023-11-25T04:23:07.110 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.110 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.110 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:07.110 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040925+0000", 2023-11-25T04:23:07.110 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8536999999999999e-05 2023-11-25T04:23:07.110 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.110 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.110 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:07.110 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042079+0000", 2023-11-25T04:23:07.110 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00115358 2023-11-25T04:23:07.110 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.110 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.111 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:07.111 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042107+0000", 2023-11-25T04:23:07.111 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7971e-05 2023-11-25T04:23:07.111 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:07.111 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:07.111 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:07.111 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.111 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.111 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop_reply(mds.0.3:4 3.1f e18/16 ondisk, result = 0)", 2023-11-25T04:23:07.111 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:07.111 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 19.993746422000001, 2023-11-25T04:23:07.111 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00044092900000000001, 2023-11-25T04:23:07.111 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:07.111 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:07.112 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:07.112 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.112 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:07.112 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:07.112 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:07.112 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.112 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.112 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:07.112 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:07.112 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:07.112 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.112 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.112 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:07.112 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042124+0000", 2023-11-25T04:23:07.113 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1800000000000005e-07 2023-11-25T04:23:07.113 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.113 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.113 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:07.113 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042128+0000", 2023-11-25T04:23:07.113 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2289999999999997e-06 2023-11-25T04:23:07.113 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.113 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.113 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:07.113 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042138+0000", 2023-11-25T04:23:07.113 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9729999999999992e-06 2023-11-25T04:23:07.113 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.113 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.114 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:07.114 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042145+0000", 2023-11-25T04:23:07.114 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.336e-06 2023-11-25T04:23:07.114 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.114 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.114 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:07.114 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042441+0000", 2023-11-25T04:23:07.114 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029650500000000002 2023-11-25T04:23:07.114 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.114 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.114 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:07.114 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042448+0000", 2023-11-25T04:23:07.115 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4100000000000002e-06 2023-11-25T04:23:07.115 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.115 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.115 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:07.115 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042564+0000", 2023-11-25T04:23:07.115 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000115758 2023-11-25T04:23:07.115 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:07.115 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:07.115 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:07.115 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.115 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.115 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop_reply(mds.0.3:26 3.1f e18/16 ondisk, result = 0)", 2023-11-25T04:23:07.115 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.044422+0000", 2023-11-25T04:23:07.116 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 19.991448058, 2023-11-25T04:23:07.116 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00028785799999999999, 2023-11-25T04:23:07.116 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:07.116 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:07.116 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:07.116 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.116 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:07.116 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044422+0000", 2023-11-25T04:23:07.116 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:07.116 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.116 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.116 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:07.116 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044422+0000", 2023-11-25T04:23:07.117 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:07.117 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.117 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.117 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:07.117 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044422+0000", 2023-11-25T04:23:07.117 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9000000000000001e-07 2023-11-25T04:23:07.117 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.117 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.117 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:07.117 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044423+0000", 2023-11-25T04:23:07.117 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.669e-06 2023-11-25T04:23:07.117 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.118 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.118 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:07.118 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044427+0000", 2023-11-25T04:23:07.118 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9330000000000003e-06 2023-11-25T04:23:07.118 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.118 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.118 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:07.118 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044442+0000", 2023-11-25T04:23:07.118 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4525999999999999e-05 2023-11-25T04:23:07.118 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.118 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.118 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:07.118 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044566+0000", 2023-11-25T04:23:07.119 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012453099999999999 2023-11-25T04:23:07.119 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.119 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.119 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:07.119 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044573+0000", 2023-11-25T04:23:07.119 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4409999999999998e-06 2023-11-25T04:23:07.119 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.119 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.119 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:07.120 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044709+0000", 2023-11-25T04:23:07.120 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000136568 2023-11-25T04:23:07.120 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:07.120 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:07.120 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:07.120 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.120 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.120 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing_reply(3.1f epoch 18/16 rep_tid 8 lcod 18'4)", 2023-11-25T04:23:07.120 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:07.120 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 19.99142101, 2023-11-25T04:23:07.120 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00038154699999999998, 2023-11-25T04:23:07.120 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:07.120 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:07.121 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:07.121 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.121 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:07.121 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:07.121 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:07.121 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.121 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.121 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:07.121 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:07.121 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:07.121 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.121 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.122 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:07.122 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:07.122 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2700000000000001e-07 2023-11-25T04:23:07.122 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.122 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.122 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:07.122 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044451+0000", 2023-11-25T04:23:07.122 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7460000000000001e-06 2023-11-25T04:23:07.122 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.122 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.122 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:07.122 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044455+0000", 2023-11-25T04:23:07.122 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5809999999999998e-06 2023-11-25T04:23:07.123 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.123 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.123 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:07.123 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044458+0000", 2023-11-25T04:23:07.123 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0170000000000001e-06 2023-11-25T04:23:07.123 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.123 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.123 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:07.123 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044734+0000", 2023-11-25T04:23:07.123 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00027582799999999999 2023-11-25T04:23:07.123 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.123 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.123 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:07.124 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044830+0000", 2023-11-25T04:23:07.124 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6248000000000004e-05 2023-11-25T04:23:07.124 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:07.124 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:07.124 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:07.124 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.124 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.124 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing_reply(3.b epoch 18/16 rep_tid 5 lcod 18'2)", 2023-11-25T04:23:07.124 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:07.124 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 19.991384349, 2023-11-25T04:23:07.124 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000344488, 2023-11-25T04:23:07.124 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:07.125 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:07.125 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:07.125 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.125 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:07.125 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:07.125 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:07.125 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.125 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.125 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:07.125 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:07.125 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:07.125 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.125 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.126 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:07.126 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:07.126 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.55e-07 2023-11-25T04:23:07.126 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.126 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.126 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:07.126 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044487+0000", 2023-11-25T04:23:07.126 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2470000000000001e-06 2023-11-25T04:23:07.126 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.126 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.126 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:07.126 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044491+0000", 2023-11-25T04:23:07.126 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9659999999999998e-06 2023-11-25T04:23:07.127 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.127 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.127 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:07.127 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044494+0000", 2023-11-25T04:23:07.127 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2310000000000001e-06 2023-11-25T04:23:07.127 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.127 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.127 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:07.127 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044756+0000", 2023-11-25T04:23:07.127 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00026172800000000003 2023-11-25T04:23:07.127 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.127 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.127 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:07.128 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044830+0000", 2023-11-25T04:23:07.128 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4160999999999997e-05 2023-11-25T04:23:07.128 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:07.128 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:07.128 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:07.128 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.128 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.128 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:39 3.6 3:654134d2:::mds0_openfiles.0:head [omap-clear,omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2023-11-25T04:23:07.128 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:07.128 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 14.931558307, 2023-11-25T04:23:07.128 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001788658, 2023-11-25T04:23:07.128 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:07.128 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:07.129 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:07.129 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.129 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:07.129 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:07.129 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:07.129 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.129 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.129 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:07.129 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:07.129 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:07.129 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.129 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.129 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:07.130 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104314+0000", 2023-11-25T04:23:07.130 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3099999999999999e-06 2023-11-25T04:23:07.130 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.130 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.130 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:07.130 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104325+0000", 2023-11-25T04:23:07.130 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1511999999999999e-05 2023-11-25T04:23:07.130 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.130 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.130 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:07.130 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104362+0000", 2023-11-25T04:23:07.130 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6903000000000002e-05 2023-11-25T04:23:07.131 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.131 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.131 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:07.131 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104387+0000", 2023-11-25T04:23:07.132 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4766e-05 2023-11-25T04:23:07.132 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.132 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.132 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:07.132 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104465+0000", 2023-11-25T04:23:07.132 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8119000000000002e-05 2023-11-25T04:23:07.132 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.132 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.132 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:07.132 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104557+0000", 2023-11-25T04:23:07.132 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2496000000000003e-05 2023-11-25T04:23:07.132 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.133 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.133 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:07.133 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104646+0000", 2023-11-25T04:23:07.133 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8818000000000002e-05 2023-11-25T04:23:07.133 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.133 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.133 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:07.133 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105201+0000", 2023-11-25T04:23:07.133 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00055506000000000002 2023-11-25T04:23:07.133 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.133 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.133 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:07.133 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105994+0000", 2023-11-25T04:23:07.134 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00079294700000000001 2023-11-25T04:23:07.134 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.134 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.134 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:07.134 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.106055+0000", 2023-11-25T04:23:07.134 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0566000000000003e-05 2023-11-25T04:23:07.134 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.134 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.134 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:07.134 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.106100+0000", 2023-11-25T04:23:07.134 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5160999999999997e-05 2023-11-25T04:23:07.134 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:07.135 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:07.135 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:07.135 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.135 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.135 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop_reply(mds.0.3:39 3.6 e19/16 ondisk, result = 0)", 2023-11-25T04:23:07.135 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:52.105910+0000", 2023-11-25T04:23:07.135 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 14.929959354999999, 2023-11-25T04:23:07.135 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00019886599999999999, 2023-11-25T04:23:07.135 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:07.135 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:07.135 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:07.135 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.135 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:07.136 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105910+0000", 2023-11-25T04:23:07.136 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:07.136 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.136 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.136 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:07.136 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105910+0000", 2023-11-25T04:23:07.136 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:07.136 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.136 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.136 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:07.136 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105911+0000", 2023-11-25T04:23:07.136 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9999999999999995e-07 2023-11-25T04:23:07.136 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.137 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.137 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:07.137 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105916+0000", 2023-11-25T04:23:07.137 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4769999999999997e-06 2023-11-25T04:23:07.137 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.137 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.137 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:07.137 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105929+0000", 2023-11-25T04:23:07.137 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2729e-05 2023-11-25T04:23:07.137 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.137 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.137 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:07.137 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105935+0000", 2023-11-25T04:23:07.138 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1249999999999998e-06 2023-11-25T04:23:07.138 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.138 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.138 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:07.138 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105979+0000", 2023-11-25T04:23:07.138 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4184999999999998e-05 2023-11-25T04:23:07.138 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.138 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.138 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:07.138 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105987+0000", 2023-11-25T04:23:07.138 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5949999999999999e-06 2023-11-25T04:23:07.138 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:07.139 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:07.139 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:07.139 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.106109+0000", 2023-11-25T04:23:07.139 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000121755 2023-11-25T04:23:07.139 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:07.139 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:07.139 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:07.139 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:07.139 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:07.139 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:07.139 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2023-11-25T04:23:07.141 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:07.139+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:07.141 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:23:07.241 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:07.240+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:07.255 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:07.255 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:23:07.255 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2023-11-25T04:23:07.255 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:07.264 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:23:07.348 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:07.347+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:07.410 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:07.411 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:23:07.411 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2023-11-25T04:23:07.411 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2023-11-25T04:23:07.411 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:07.419 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:23:07.460 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:07.459+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:07.545 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:07.544+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:07.570 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:07.570 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2023-11-25T04:23:07.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2023-11-25T04:23:07.570 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2023-11-25T04:23:07.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.570 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:155 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'109, mlcod=11'105)", 2023-11-25T04:23:07.571 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:07.571 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 29.782398604000001, 2023-11-25T04:23:07.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061719019999999999, 2023-11-25T04:23:07.571 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:07.571 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:07.571 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:07.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:07.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:07.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:07.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.572 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:07.572 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:07.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:07.572 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.572 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.572 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:07.572 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784884+0000", 2023-11-25T04:23:07.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3099999999999999e-07 2023-11-25T04:23:07.572 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.572 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.572 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:07.572 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784942+0000", 2023-11-25T04:23:07.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8570000000000003e-05 2023-11-25T04:23:07.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.573 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:07.573 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784949+0000", 2023-11-25T04:23:07.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0310000000000001e-06 2023-11-25T04:23:07.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.573 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:07.573 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784953+0000", 2023-11-25T04:23:07.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6320000000000001e-06 2023-11-25T04:23:07.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:07.574 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784993+0000", 2023-11-25T04:23:07.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9737000000000003e-05 2023-11-25T04:23:07.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:07.574 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785011+0000", 2023-11-25T04:23:07.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8349999999999999e-05 2023-11-25T04:23:07.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:07.574 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791024+0000", 2023-11-25T04:23:07.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006012997 2023-11-25T04:23:07.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:07.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791055+0000", 2023-11-25T04:23:07.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1254000000000003e-05 2023-11-25T04:23:07.575 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:07.575 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:07.575 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:07.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.575 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:156 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'110, mlcod=11'105)", 2023-11-25T04:23:07.575 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:07.576 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 29.781812554999998, 2023-11-25T04:23:07.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005616116, 2023-11-25T04:23:07.576 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:07.576 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:07.576 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:07.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.576 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:07.576 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:07.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:07.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.577 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:07.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:07.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:07.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.577 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:07.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785470+0000", 2023-11-25T04:23:07.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0699999999999998e-07 2023-11-25T04:23:07.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.577 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:07.578 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785474+0000", 2023-11-25T04:23:07.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.048e-06 2023-11-25T04:23:07.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.578 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:07.578 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785480+0000", 2023-11-25T04:23:07.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7729999999999996e-06 2023-11-25T04:23:07.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:07.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785483+0000", 2023-11-25T04:23:07.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1970000000000001e-06 2023-11-25T04:23:07.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:07.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787936+0000", 2023-11-25T04:23:07.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024529040000000001 2023-11-25T04:23:07.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:07.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787947+0000", 2023-11-25T04:23:07.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1669e-05 2023-11-25T04:23:07.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:07.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791057+0000", 2023-11-25T04:23:07.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031095200000000002 2023-11-25T04:23:07.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.581 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:07.581 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791086+0000", 2023-11-25T04:23:07.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8697999999999999e-05 2023-11-25T04:23:07.581 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:07.581 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:07.581 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:07.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.581 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:157 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'111, mlcod=11'106)", 2023-11-25T04:23:07.581 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:07.581 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 29.781219997000001, 2023-11-25T04:23:07.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050590280000000001, 2023-11-25T04:23:07.581 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:07.582 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:07.582 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:07.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:07.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:07.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:07.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:07.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:07.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:07.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.583 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:07.583 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786063+0000", 2023-11-25T04:23:07.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2600000000000002e-07 2023-11-25T04:23:07.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.583 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:07.584 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786069+0000", 2023-11-25T04:23:07.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8839999999999999e-06 2023-11-25T04:23:07.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.584 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:07.584 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786079+0000", 2023-11-25T04:23:07.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0124e-05 2023-11-25T04:23:07.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:07.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786085+0000", 2023-11-25T04:23:07.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6660000000000002e-06 2023-11-25T04:23:07.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:07.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788160+0000", 2023-11-25T04:23:07.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020747669999999999 2023-11-25T04:23:07.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:07.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788180+0000", 2023-11-25T04:23:07.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0208999999999999e-05 2023-11-25T04:23:07.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:07.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791087+0000", 2023-11-25T04:23:07.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029064630000000002 2023-11-25T04:23:07.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:07.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791121+0000", 2023-11-25T04:23:07.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4289000000000003e-05 2023-11-25T04:23:07.587 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:07.587 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:07.587 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:07.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.587 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:158 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'112, mlcod=11'106)", 2023-11-25T04:23:07.587 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:07.587 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 29.780803603999999, 2023-11-25T04:23:07.587 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046663030000000001, 2023-11-25T04:23:07.587 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:07.587 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:07.588 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:07.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.588 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:07.588 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:07.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:07.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.588 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:07.588 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:07.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:07.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.588 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:07.589 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786479+0000", 2023-11-25T04:23:07.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2799999999999996e-07 2023-11-25T04:23:07.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:07.589 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786498+0000", 2023-11-25T04:23:07.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9069e-05 2023-11-25T04:23:07.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:07.589 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786508+0000", 2023-11-25T04:23:07.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5810000000000003e-06 2023-11-25T04:23:07.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:07.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786512+0000", 2023-11-25T04:23:07.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4889999999999997e-06 2023-11-25T04:23:07.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:07.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788438+0000", 2023-11-25T04:23:07.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019257059999999999 2023-11-25T04:23:07.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.591 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:07.591 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788449+0000", 2023-11-25T04:23:07.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1419e-05 2023-11-25T04:23:07.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.591 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:07.591 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791122+0000", 2023-11-25T04:23:07.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002672822 2023-11-25T04:23:07.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.592 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.592 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:07.592 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791145+0000", 2023-11-25T04:23:07.592 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2589e-05 2023-11-25T04:23:07.592 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:07.592 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:07.592 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:07.592 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.592 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.592 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:206 1.0 e12/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 12'152, mlcod=12'148)", 2023-11-25T04:23:07.592 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:07.592 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 29.269035133999999, 2023-11-25T04:23:07.592 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046728189999999999, 2023-11-25T04:23:07.593 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:07.593 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:07.593 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:07.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:07.593 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:07.593 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:07.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:07.593 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:07.593 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:07.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.594 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:07.594 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298248+0000", 2023-11-25T04:23:07.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3600000000000003e-07 2023-11-25T04:23:07.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.594 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:07.594 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298278+0000", 2023-11-25T04:23:07.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0474e-05 2023-11-25T04:23:07.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.594 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:07.595 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298288+0000", 2023-11-25T04:23:07.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0156000000000001e-05 2023-11-25T04:23:07.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:07.595 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298293+0000", 2023-11-25T04:23:07.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7400000000000004e-06 2023-11-25T04:23:07.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:07.595 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298333+0000", 2023-11-25T04:23:07.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9666000000000003e-05 2023-11-25T04:23:07.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.596 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.596 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:07.596 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298351+0000", 2023-11-25T04:23:07.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8544999999999999e-05 2023-11-25T04:23:07.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.596 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.596 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:07.596 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302891+0000", 2023-11-25T04:23:07.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045398519999999996 2023-11-25T04:23:07.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.597 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.597 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:07.597 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302920+0000", 2023-11-25T04:23:07.597 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8750000000000001e-05 2023-11-25T04:23:07.597 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:07.597 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:07.597 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:07.597 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.597 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.597 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:14 3.15 3:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:07.597 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:07.597 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.533900776999999, 2023-11-25T04:23:07.597 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010990437, 2023-11-25T04:23:07.598 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:07.598 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:07.598 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:07.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:07.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:07.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:07.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:07.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:07.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:07.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:07.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033383+0000", 2023-11-25T04:23:07.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9870000000000002e-06 2023-11-25T04:23:07.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:07.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033398+0000", 2023-11-25T04:23:07.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4569999999999999e-05 2023-11-25T04:23:07.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:07.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033411+0000", 2023-11-25T04:23:07.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3543999999999999e-05 2023-11-25T04:23:07.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.600 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:07.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033423+0000", 2023-11-25T04:23:07.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1905999999999999e-05 2023-11-25T04:23:07.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.600 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:07.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034433+0000", 2023-11-25T04:23:07.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00101022 2023-11-25T04:23:07.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:07.601 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034539+0000", 2023-11-25T04:23:07.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010597199999999999 2023-11-25T04:23:07.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:07.601 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034667+0000", 2023-11-25T04:23:07.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012765500000000001 2023-11-25T04:23:07.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:07.601 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037803+0000", 2023-11-25T04:23:07.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031362220000000001 2023-11-25T04:23:07.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:07.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:23:07.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0064964669999999997 2023-11-25T04:23:07.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:07.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044331+0000", 2023-11-25T04:23:07.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1544999999999997e-05 2023-11-25T04:23:07.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:07.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044372+0000", 2023-11-25T04:23:07.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0349000000000003e-05 2023-11-25T04:23:07.603 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:07.603 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:07.603 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:07.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.605 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:15 3.15 3:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:07.605 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:07.605 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.533241127, 2023-11-25T04:23:07.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010464362, 2023-11-25T04:23:07.605 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:07.605 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:07.605 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:07.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.605 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:07.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:07.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:07.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:07.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:07.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:07.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:07.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034042+0000", 2023-11-25T04:23:07.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0999999999999997e-07 2023-11-25T04:23:07.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:07.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034048+0000", 2023-11-25T04:23:07.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0390000000000001e-06 2023-11-25T04:23:07.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:07.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034059+0000", 2023-11-25T04:23:07.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1182999999999999e-05 2023-11-25T04:23:07.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:07.608 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034067+0000", 2023-11-25T04:23:07.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1559999999999995e-06 2023-11-25T04:23:07.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:07.608 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035005+0000", 2023-11-25T04:23:07.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00093747399999999999 2023-11-25T04:23:07.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:07.608 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035077+0000", 2023-11-25T04:23:07.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2747e-05 2023-11-25T04:23:07.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:07.609 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035219+0000", 2023-11-25T04:23:07.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000141877 2023-11-25T04:23:07.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:07.609 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037810+0000", 2023-11-25T04:23:07.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002590752 2023-11-25T04:23:07.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.610 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:07.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044437+0000", 2023-11-25T04:23:07.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066270110000000004 2023-11-25T04:23:07.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.610 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:07.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044481+0000", 2023-11-25T04:23:07.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4106000000000001e-05 2023-11-25T04:23:07.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.610 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:07.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044505+0000", 2023-11-25T04:23:07.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4207e-05 2023-11-25T04:23:07.611 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:07.611 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:07.611 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:07.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.611 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:16 3.1d 3:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:07.611 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:07.611 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.533199279000002, 2023-11-25T04:23:07.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010413923, 2023-11-25T04:23:07.611 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:07.611 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:07.611 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:07.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:07.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:07.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:07.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:07.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:07.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:07.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:07.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:07.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4499999999999997e-07 2023-11-25T04:23:07.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.613 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:07.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034087+0000", 2023-11-25T04:23:07.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9770000000000002e-06 2023-11-25T04:23:07.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.613 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:07.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034094+0000", 2023-11-25T04:23:07.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8809999999999998e-06 2023-11-25T04:23:07.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:07.614 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034100+0000", 2023-11-25T04:23:07.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8440000000000003e-06 2023-11-25T04:23:07.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:07.614 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034993+0000", 2023-11-25T04:23:07.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00089336799999999996 2023-11-25T04:23:07.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:07.614 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035099+0000", 2023-11-25T04:23:07.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106037 2023-11-25T04:23:07.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:07.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035276+0000", 2023-11-25T04:23:07.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017631799999999999 2023-11-25T04:23:07.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:07.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037815+0000", 2023-11-25T04:23:07.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025395959999999999 2023-11-25T04:23:07.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:07.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044420+0000", 2023-11-25T04:23:07.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066045749999999997 2023-11-25T04:23:07.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:07.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044463+0000", 2023-11-25T04:23:07.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3205999999999999e-05 2023-11-25T04:23:07.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:07.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044497+0000", 2023-11-25T04:23:07.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3476000000000003e-05 2023-11-25T04:23:07.617 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:07.617 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:07.617 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:07.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.617 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:17 3.1d 3:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:07.617 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:07.617 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.533174070000001, 2023-11-25T04:23:07.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01127356, 2023-11-25T04:23:07.617 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:07.617 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:07.617 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:07.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.617 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:07.618 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:07.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:07.618 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.618 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:07.618 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:07.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:07.618 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.618 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:07.618 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:07.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1100000000000001e-07 2023-11-25T04:23:07.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:07.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034112+0000", 2023-11-25T04:23:07.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8600000000000003e-06 2023-11-25T04:23:07.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:07.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034120+0000", 2023-11-25T04:23:07.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7700000000000001e-06 2023-11-25T04:23:07.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:07.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034127+0000", 2023-11-25T04:23:07.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0999999999999998e-06 2023-11-25T04:23:07.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:07.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035601+0000", 2023-11-25T04:23:07.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014739410000000001 2023-11-25T04:23:07.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:07.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035685+0000", 2023-11-25T04:23:07.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3677999999999994e-05 2023-11-25T04:23:07.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:07.621 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035827+0000", 2023-11-25T04:23:07.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014232599999999999 2023-11-25T04:23:07.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:07.621 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038725+0000", 2023-11-25T04:23:07.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028982779999999998 2023-11-25T04:23:07.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:07.622 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045323+0000", 2023-11-25T04:23:07.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0065980780000000003 2023-11-25T04:23:07.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.622 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:07.622 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045354+0000", 2023-11-25T04:23:07.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0463e-05 2023-11-25T04:23:07.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.622 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:07.622 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045381+0000", 2023-11-25T04:23:07.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7654999999999999e-05 2023-11-25T04:23:07.623 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:07.623 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:07.623 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:07.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.623 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:22 3.e 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:07.623 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:07.623 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.533147657000001, 2023-11-25T04:23:07.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010133422, 2023-11-25T04:23:07.623 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:07.623 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:07.623 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:07.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:07.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:07.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:07.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:07.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:07.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:07.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:07.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034135+0000", 2023-11-25T04:23:07.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-07 2023-11-25T04:23:07.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:07.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034138+0000", 2023-11-25T04:23:07.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6009999999999999e-06 2023-11-25T04:23:07.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:07.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034145+0000", 2023-11-25T04:23:07.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1169999999999998e-06 2023-11-25T04:23:07.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:07.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034150+0000", 2023-11-25T04:23:07.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9080000000000003e-06 2023-11-25T04:23:07.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:07.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034209+0000", 2023-11-25T04:23:07.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8149000000000002e-05 2023-11-25T04:23:07.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:07.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034415+0000", 2023-11-25T04:23:07.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020637699999999999 2023-11-25T04:23:07.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.627 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:07.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034584+0000", 2023-11-25T04:23:07.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016870399999999999 2023-11-25T04:23:07.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.627 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:07.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.036979+0000", 2023-11-25T04:23:07.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023957509999999998 2023-11-25T04:23:07.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.627 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:07.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044161+0000", 2023-11-25T04:23:07.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0071811590000000003 2023-11-25T04:23:07.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:07.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044224+0000", 2023-11-25T04:23:07.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3454000000000006e-05 2023-11-25T04:23:07.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:07.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044268+0000", 2023-11-25T04:23:07.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3761999999999999e-05 2023-11-25T04:23:07.628 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:07.629 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:07.629 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:07.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.629 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:23 3.e 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:07.629 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:07.629 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.533116247999999, 2023-11-25T04:23:07.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010281676, 2023-11-25T04:23:07.629 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:07.629 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:07.629 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:07.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:07.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:07.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:07.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:07.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:07.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:07.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:07.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:07.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5799999999999999e-07 2023-11-25T04:23:07.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.631 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:07.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034169+0000", 2023-11-25T04:23:07.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1879999999999998e-06 2023-11-25T04:23:07.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.631 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:07.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034177+0000", 2023-11-25T04:23:07.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7020000000000001e-06 2023-11-25T04:23:07.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:07.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034184+0000", 2023-11-25T04:23:07.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3910000000000003e-06 2023-11-25T04:23:07.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:07.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034924+0000", 2023-11-25T04:23:07.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00074032999999999998 2023-11-25T04:23:07.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:07.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034991+0000", 2023-11-25T04:23:07.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7347000000000004e-05 2023-11-25T04:23:07.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:07.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035096+0000", 2023-11-25T04:23:07.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010488 2023-11-25T04:23:07.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:07.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037588+0000", 2023-11-25T04:23:07.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024918309999999999 2023-11-25T04:23:07.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:07.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044367+0000", 2023-11-25T04:23:07.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067787589999999997 2023-11-25T04:23:07.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:07.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044415+0000", 2023-11-25T04:23:07.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7803000000000003e-05 2023-11-25T04:23:07.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:07.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044447+0000", 2023-11-25T04:23:07.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2886999999999999e-05 2023-11-25T04:23:07.635 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:07.635 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:07.635 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:07.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.635 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:30 3.15 3:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:07.635 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:07.635 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.533057807999999, 2023-11-25T04:23:07.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010365070000000001, 2023-11-25T04:23:07.635 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:07.636 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:07.636 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:07.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:07.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:07.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:07.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:07.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:07.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:07.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:07.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034225+0000", 2023-11-25T04:23:07.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7400000000000002e-07 2023-11-25T04:23:07.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:07.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034229+0000", 2023-11-25T04:23:07.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3100000000000002e-06 2023-11-25T04:23:07.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:07.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034239+0000", 2023-11-25T04:23:07.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9140000000000003e-06 2023-11-25T04:23:07.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:07.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034246+0000", 2023-11-25T04:23:07.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0149999999999998e-06 2023-11-25T04:23:07.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:07.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035572+0000", 2023-11-25T04:23:07.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001325655 2023-11-25T04:23:07.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:07.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035730+0000", 2023-11-25T04:23:07.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015827899999999999 2023-11-25T04:23:07.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:07.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035857+0000", 2023-11-25T04:23:07.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012644699999999999 2023-11-25T04:23:07.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:07.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038737+0000", 2023-11-25T04:23:07.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002880077 2023-11-25T04:23:07.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:07.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044540+0000", 2023-11-25T04:23:07.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058034239999999997 2023-11-25T04:23:07.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:07.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044569+0000", 2023-11-25T04:23:07.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8388999999999999e-05 2023-11-25T04:23:07.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:07.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044589+0000", 2023-11-25T04:23:07.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0686000000000002e-05 2023-11-25T04:23:07.642 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:07.642 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:07.642 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:07.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.642 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:33 3.1b 3:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:07.642 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:07.642 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.53301231, 2023-11-25T04:23:07.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099754049999999997, 2023-11-25T04:23:07.643 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:07.643 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:23:07.643 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:07.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:07.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:07.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:07.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:07.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:07.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:07.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.644 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:07.644 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:07.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4799999999999998e-07 2023-11-25T04:23:07.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.644 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:07.644 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034273+0000", 2023-11-25T04:23:07.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7819999999999998e-06 2023-11-25T04:23:07.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:07.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034285+0000", 2023-11-25T04:23:07.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2058e-05 2023-11-25T04:23:07.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:07.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034291+0000", 2023-11-25T04:23:07.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0680000000000002e-06 2023-11-25T04:23:07.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:07.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034370+0000", 2023-11-25T04:23:07.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8452e-05 2023-11-25T04:23:07.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:07.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034477+0000", 2023-11-25T04:23:07.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107471 2023-11-25T04:23:07.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:07.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044245+0000", 2023-11-25T04:23:07.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0097679259999999997 2023-11-25T04:23:07.646 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:07.647 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:07.647 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:07.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.647 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:36 3.1a 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:07.647 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:07.647 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.532965637, 2023-11-25T04:23:07.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058179099999999999, 2023-11-25T04:23:07.647 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:07.647 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:23:07.647 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:07.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.648 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:07.648 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:07.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:07.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.648 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:07.648 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:07.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:07.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.649 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:07.649 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034317+0000", 2023-11-25T04:23:07.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1999999999999999e-07 2023-11-25T04:23:07.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.649 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:07.649 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034321+0000", 2023-11-25T04:23:07.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.233e-06 2023-11-25T04:23:07.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.649 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:07.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034337+0000", 2023-11-25T04:23:07.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5308999999999999e-05 2023-11-25T04:23:07.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:07.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034354+0000", 2023-11-25T04:23:07.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7411999999999999e-05 2023-11-25T04:23:07.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:07.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034465+0000", 2023-11-25T04:23:07.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011154699999999999 2023-11-25T04:23:07.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:07.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034560+0000", 2023-11-25T04:23:07.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4186999999999998e-05 2023-11-25T04:23:07.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:07.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.040134+0000", 2023-11-25T04:23:07.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055746019999999997 2023-11-25T04:23:07.651 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:07.652 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:07.652 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:07.652 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.652 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.653 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:9 3.10 e18/16 3:09401a81:::601.00000000:head v 18'4, mlcod=0'0)", 2023-11-25T04:23:07.653 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:07.653 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.524071329000002, 2023-11-25T04:23:07.653 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061641020000000003, 2023-11-25T04:23:07.653 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:07.653 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:07.653 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:07.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.653 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:07.653 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:07.653 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:07.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:07.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:07.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:07.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:07.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:07.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5300000000000001e-07 2023-11-25T04:23:07.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:07.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043222+0000", 2023-11-25T04:23:07.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.082e-05 2023-11-25T04:23:07.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:07.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043228+0000", 2023-11-25T04:23:07.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6699999999999997e-06 2023-11-25T04:23:07.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:07.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043236+0000", 2023-11-25T04:23:07.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8189999999999992e-06 2023-11-25T04:23:07.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.656 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:07.656 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043507+0000", 2023-11-25T04:23:07.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000271012 2023-11-25T04:23:07.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.656 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:07.656 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043530+0000", 2023-11-25T04:23:07.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2659000000000001e-05 2023-11-25T04:23:07.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.656 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:07.656 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049321+0000", 2023-11-25T04:23:07.657 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057915129999999999 2023-11-25T04:23:07.657 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.657 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.657 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:07.657 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049375+0000", 2023-11-25T04:23:07.657 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3455999999999998e-05 2023-11-25T04:23:07.657 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:07.657 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:07.657 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:07.657 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.657 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.657 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:1 3.14 e18/16 3:292cf221:::200.00000000:head v 18'3, mlcod=0'0)", 2023-11-25T04:23:07.657 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:07.657 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.524034616000002, 2023-11-25T04:23:07.658 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0060983599999999997, 2023-11-25T04:23:07.658 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:07.658 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:07.658 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:07.658 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.658 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:07.658 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:07.658 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:07.658 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.658 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.658 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:07.659 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:07.659 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:07.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.659 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.659 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:07.659 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043248+0000", 2023-11-25T04:23:07.659 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9399999999999995e-07 2023-11-25T04:23:07.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.659 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.660 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:07.660 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043252+0000", 2023-11-25T04:23:07.660 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1799999999999998e-06 2023-11-25T04:23:07.660 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.660 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.660 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:07.660 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043259+0000", 2023-11-25T04:23:07.660 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0049999999999998e-06 2023-11-25T04:23:07.660 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.660 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.660 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:07.660 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043265+0000", 2023-11-25T04:23:07.661 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0859999999999998e-06 2023-11-25T04:23:07.661 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.661 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.661 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:07.661 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043311+0000", 2023-11-25T04:23:07.661 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6082999999999999e-05 2023-11-25T04:23:07.661 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.661 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.661 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:07.661 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043330+0000", 2023-11-25T04:23:07.661 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8556e-05 2023-11-25T04:23:07.661 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.661 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.661 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:07.662 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049303+0000", 2023-11-25T04:23:07.662 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059733470000000004 2023-11-25T04:23:07.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.662 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.662 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:07.662 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049346+0000", 2023-11-25T04:23:07.662 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2608999999999998e-05 2023-11-25T04:23:07.662 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:07.662 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:07.662 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:07.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.662 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.662 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:3 3.14 e18/16 3:287034c4:::1.00000000.inode:head v 18'4, mlcod=0'0)", 2023-11-25T04:23:07.663 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:07.663 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.523518110000001, 2023-11-25T04:23:07.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057040090000000003, 2023-11-25T04:23:07.663 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:07.663 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:07.663 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:07.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:07.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:07.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:07.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:07.664 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:07.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:07.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:07.664 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:07.665 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9799999999999999e-07 2023-11-25T04:23:07.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.665 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.665 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:07.665 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043785+0000", 2023-11-25T04:23:07.665 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0721999999999999e-05 2023-11-25T04:23:07.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.666 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.666 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:07.666 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043793+0000", 2023-11-25T04:23:07.666 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5839999999999995e-06 2023-11-25T04:23:07.666 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.666 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.666 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:07.666 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043803+0000", 2023-11-25T04:23:07.666 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0729999999999994e-06 2023-11-25T04:23:07.666 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.666 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.666 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:07.667 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043843+0000", 2023-11-25T04:23:07.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0747999999999997e-05 2023-11-25T04:23:07.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.667 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:07.667 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043856+0000", 2023-11-25T04:23:07.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2697000000000001e-05 2023-11-25T04:23:07.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.667 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:07.667 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049426+0000", 2023-11-25T04:23:07.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055704409999999998 2023-11-25T04:23:07.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.668 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:07.668 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049468+0000", 2023-11-25T04:23:07.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1446000000000003e-05 2023-11-25T04:23:07.668 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:07.668 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:07.668 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:07.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.668 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "pg_update_log_missing(3.14 epoch 18/16 rep_tid 171 entries 18'5 (0'0) error 3:2e2fa760:::200.00000001:head by mds.0.3:32 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:23:07.669 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:07.669 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.523357146999999, 2023-11-25T04:23:07.669 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005556938, 2023-11-25T04:23:07.669 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:07.669 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2023-11-25T04:23:07.669 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:07.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.669 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:07.670 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:07.670 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:07.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.670 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:07.670 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:07.670 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:07.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.670 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:07.670 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:07.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.03e-07 2023-11-25T04:23:07.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:07.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043931+0000", 2023-11-25T04:23:07.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1079999999999998e-06 2023-11-25T04:23:07.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:07.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043944+0000", 2023-11-25T04:23:07.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2852e-05 2023-11-25T04:23:07.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.672 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:07.672 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043955+0000", 2023-11-25T04:23:07.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1037e-05 2023-11-25T04:23:07.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.672 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:07.672 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044088+0000", 2023-11-25T04:23:07.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013269599999999999 2023-11-25T04:23:07.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.672 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:07.672 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049482+0000", 2023-11-25T04:23:07.673 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053940419999999999 2023-11-25T04:23:07.673 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:07.673 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:07.673 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:07.673 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.673 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.673 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:10 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'5, mlcod=0'0)", 2023-11-25T04:23:07.673 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:07.673 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.523267119, 2023-11-25T04:23:07.673 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054493099999999997, 2023-11-25T04:23:07.673 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:07.673 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:07.674 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:07.674 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.674 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:07.674 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:07.674 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:07.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.674 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.674 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:07.674 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:07.674 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:07.675 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.675 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.675 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:07.675 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:07.675 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6100000000000002e-07 2023-11-25T04:23:07.675 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.675 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.675 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:07.675 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044019+0000", 2023-11-25T04:23:07.675 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.839e-06 2023-11-25T04:23:07.675 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.676 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:07.676 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044026+0000", 2023-11-25T04:23:07.676 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5869999999999998e-06 2023-11-25T04:23:07.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.676 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:07.676 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044031+0000", 2023-11-25T04:23:07.676 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4909999999999996e-06 2023-11-25T04:23:07.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.677 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:07.677 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044070+0000", 2023-11-25T04:23:07.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8829999999999999e-05 2023-11-25T04:23:07.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.677 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:07.677 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044081+0000", 2023-11-25T04:23:07.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1447999999999999e-05 2023-11-25T04:23:07.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.677 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:07.678 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049406+0000", 2023-11-25T04:23:07.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053246480000000004 2023-11-25T04:23:07.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.678 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.678 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:07.678 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049464+0000", 2023-11-25T04:23:07.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8205999999999998e-05 2023-11-25T04:23:07.678 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:07.678 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:07.678 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:07.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.679 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.679 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:11 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'6, mlcod=0'0)", 2023-11-25T04:23:07.679 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:07.679 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 20.523065854999999, 2023-11-25T04:23:07.679 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052673499999999996, 2023-11-25T04:23:07.679 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:07.679 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:07.679 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:07.680 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.680 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:07.680 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:07.680 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:07.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.680 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.680 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:07.680 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:07.680 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:07.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.681 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:07.681 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:07.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0199999999999998e-07 2023-11-25T04:23:07.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.681 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:07.681 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044220+0000", 2023-11-25T04:23:07.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7720000000000001e-06 2023-11-25T04:23:07.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:07.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044226+0000", 2023-11-25T04:23:07.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8579999999999996e-06 2023-11-25T04:23:07.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:07.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044232+0000", 2023-11-25T04:23:07.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1700000000000002e-06 2023-11-25T04:23:07.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:07.683 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044274+0000", 2023-11-25T04:23:07.683 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1254999999999997e-05 2023-11-25T04:23:07.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.683 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:07.683 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:23:07.683 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6880999999999999e-05 2023-11-25T04:23:07.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.683 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:07.684 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049466+0000", 2023-11-25T04:23:07.684 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051659219999999999 2023-11-25T04:23:07.684 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:07.684 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:07.684 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:07.684 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049483+0000", 2023-11-25T04:23:07.684 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.719e-05 2023-11-25T04:23:07.684 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:07.684 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:07.684 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:07.684 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:07.684 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:07.685 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:07.686 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:23:07.687 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:07.684+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:07.787 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:07.785+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:07.799 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:07.799 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:23:07.799 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2023-11-25T04:23:07.799 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:07.810 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:23:07.879 DEBUG:teuthology.orchestra.console.smithi076:pexpect command: ipmitool -H smithi076.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-25T04:23:07.889 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:07.887+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:07.955 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:07.955 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:23:07.955 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2023-11-25T04:23:07.955 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2023-11-25T04:23:07.955 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:07.964 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:23:07.991 DEBUG:teuthology.orchestra.console.smithi076:check power output: Chassis Power is on 2023-11-25T04:23:08.007 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:08.006+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:08.071 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:08.071 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2023-11-25T04:23:08.071 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2023-11-25T04:23:08.071 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2023-11-25T04:23:08.071 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.071 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:08.071 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:08.071 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 21.036582277000001, 2023-11-25T04:23:08.071 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01072943, 2023-11-25T04:23:08.071 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:08.072 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:08.072 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:08.072 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.072 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:08.072 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:08.072 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:08.072 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.072 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.072 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:08.072 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:08.072 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:08.072 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.073 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.073 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:08.073 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031829+0000", 2023-11-25T04:23:08.073 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.666e-06 2023-11-25T04:23:08.073 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.073 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.073 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:08.073 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031839+0000", 2023-11-25T04:23:08.073 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9490000000000008e-06 2023-11-25T04:23:08.073 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.073 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.073 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:08.073 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031857+0000", 2023-11-25T04:23:08.074 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8093000000000001e-05 2023-11-25T04:23:08.074 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.074 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.074 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:08.074 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031882+0000", 2023-11-25T04:23:08.074 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4294000000000002e-05 2023-11-25T04:23:08.074 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.074 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.074 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:08.074 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033130+0000", 2023-11-25T04:23:08.075 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001248264 2023-11-25T04:23:08.075 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.075 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.075 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:08.075 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033426+0000", 2023-11-25T04:23:08.075 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029620999999999998 2023-11-25T04:23:08.075 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.075 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.075 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:08.075 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033814+0000", 2023-11-25T04:23:08.075 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000387742 2023-11-25T04:23:08.075 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.076 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.076 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:08.076 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036793+0000", 2023-11-25T04:23:08.076 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002979598 2023-11-25T04:23:08.076 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.076 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.076 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:08.076 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042455+0000", 2023-11-25T04:23:08.076 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0056619260000000003 2023-11-25T04:23:08.076 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.076 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.076 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:08.076 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042502+0000", 2023-11-25T04:23:08.077 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7139999999999999e-05 2023-11-25T04:23:08.077 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.077 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.077 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:08.077 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042557+0000", 2023-11-25T04:23:08.077 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4548e-05 2023-11-25T04:23:08.077 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:08.077 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:08.077 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:08.077 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.077 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.078 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:08.078 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:08.078 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 21.035383742, 2023-11-25T04:23:08.078 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011505202000000001, 2023-11-25T04:23:08.078 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:08.078 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:08.078 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:08.078 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.078 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:08.078 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:08.078 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:08.078 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.079 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.079 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:08.079 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:08.079 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:08.079 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.079 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.079 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:08.079 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033027+0000", 2023-11-25T04:23:08.079 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0699999999999999e-06 2023-11-25T04:23:08.079 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.079 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.079 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:08.080 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033035+0000", 2023-11-25T04:23:08.080 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0830000000000004e-06 2023-11-25T04:23:08.080 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.080 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.080 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:08.080 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033059+0000", 2023-11-25T04:23:08.080 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3975e-05 2023-11-25T04:23:08.080 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.080 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.080 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:08.080 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033482+0000", 2023-11-25T04:23:08.080 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042253700000000001 2023-11-25T04:23:08.080 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.081 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.081 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:08.081 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034520+0000", 2023-11-25T04:23:08.081 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001037991 2023-11-25T04:23:08.081 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.081 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.081 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:08.081 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034611+0000", 2023-11-25T04:23:08.081 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1113999999999995e-05 2023-11-25T04:23:08.081 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.081 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.081 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:08.081 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034769+0000", 2023-11-25T04:23:08.081 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000157641 2023-11-25T04:23:08.082 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.082 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.082 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:08.082 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038292+0000", 2023-11-25T04:23:08.082 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035231160000000002 2023-11-25T04:23:08.082 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.082 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.082 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:08.082 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044440+0000", 2023-11-25T04:23:08.082 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0061486980000000002 2023-11-25T04:23:08.082 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.082 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.082 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:08.083 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044495+0000", 2023-11-25T04:23:08.083 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4339000000000003e-05 2023-11-25T04:23:08.083 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.083 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.083 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:08.083 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044531+0000", 2023-11-25T04:23:08.083 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6637999999999997e-05 2023-11-25T04:23:08.083 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:08.083 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:08.083 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:08.083 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.083 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.083 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:08.084 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:08.084 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 21.034909387999999, 2023-11-25T04:23:08.084 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0057634039999999997, 2023-11-25T04:23:08.084 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:08.084 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:08.084 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:08.084 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.084 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:08.084 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:08.084 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:08.084 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.084 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.085 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:08.085 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:08.085 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:08.085 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.085 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.085 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:08.085 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:08.085 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7900000000000007e-07 2023-11-25T04:23:08.085 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.085 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.085 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:08.085 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033507+0000", 2023-11-25T04:23:08.085 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8900000000000004e-06 2023-11-25T04:23:08.085 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.086 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.086 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:08.086 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033519+0000", 2023-11-25T04:23:08.086 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1756000000000001e-05 2023-11-25T04:23:08.086 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.086 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.086 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:08.086 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033527+0000", 2023-11-25T04:23:08.086 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3610000000000001e-06 2023-11-25T04:23:08.086 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.086 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.087 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:08.087 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034451+0000", 2023-11-25T04:23:08.087 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00092372400000000001 2023-11-25T04:23:08.087 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.087 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.087 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:08.087 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034809+0000", 2023-11-25T04:23:08.087 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00035757699999999998 2023-11-25T04:23:08.087 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.087 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.087 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:23:08.087 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034965+0000", 2023-11-25T04:23:08.087 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015638699999999999 2023-11-25T04:23:08.087 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.088 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.088 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:08.088 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038299+0000", 2023-11-25T04:23:08.088 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0033336559999999999 2023-11-25T04:23:08.088 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.088 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.088 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:08.088 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039169+0000", 2023-11-25T04:23:08.088 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00087066400000000005 2023-11-25T04:23:08.088 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.088 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.088 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:08.088 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039226+0000", 2023-11-25T04:23:08.089 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6511999999999997e-05 2023-11-25T04:23:08.089 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.089 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.089 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:08.089 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039264+0000", 2023-11-25T04:23:08.089 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7898000000000002e-05 2023-11-25T04:23:08.089 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:08.089 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:08.089 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:08.089 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.089 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.089 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:25 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:08.089 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:08.090 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 21.034830803999998, 2023-11-25T04:23:08.090 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096656169999999996, 2023-11-25T04:23:08.090 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:08.090 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:08.090 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:08.090 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.090 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:08.090 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:08.090 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:08.090 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.090 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.090 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:08.090 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:08.091 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:08.091 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.091 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.091 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:08.091 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033580+0000", 2023-11-25T04:23:08.091 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8199999999999999e-07 2023-11-25T04:23:08.091 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.091 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.091 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:08.091 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033585+0000", 2023-11-25T04:23:08.091 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3680000000000001e-06 2023-11-25T04:23:08.091 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.091 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.092 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:08.092 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033600+0000", 2023-11-25T04:23:08.092 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4331000000000001e-05 2023-11-25T04:23:08.092 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.092 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.092 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:08.092 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033606+0000", 2023-11-25T04:23:08.092 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5780000000000004e-06 2023-11-25T04:23:08.092 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.092 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.092 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:08.092 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035747+0000", 2023-11-25T04:23:08.092 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0021406530000000002 2023-11-25T04:23:08.093 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.093 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.093 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:08.093 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035901+0000", 2023-11-25T04:23:08.093 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000153931 2023-11-25T04:23:08.093 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.093 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.093 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:23:08.093 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036193+0000", 2023-11-25T04:23:08.093 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029211299999999999 2023-11-25T04:23:08.093 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.093 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.093 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:08.094 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038655+0000", 2023-11-25T04:23:08.094 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0024621640000000002 2023-11-25T04:23:08.094 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.094 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.094 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:08.094 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043130+0000", 2023-11-25T04:23:08.094 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.004474698 2023-11-25T04:23:08.094 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.094 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.095 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:08.095 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043204+0000", 2023-11-25T04:23:08.095 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4091999999999995e-05 2023-11-25T04:23:08.095 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.095 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.095 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:08.095 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043245+0000", 2023-11-25T04:23:08.095 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0907e-05 2023-11-25T04:23:08.095 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:08.095 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:08.095 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:08.095 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.096 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.096 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:26 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:08.096 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:08.096 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 21.034794980000001, 2023-11-25T04:23:08.096 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011086945000000001, 2023-11-25T04:23:08.096 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:08.096 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:08.096 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:08.096 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.096 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:08.096 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:08.096 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:08.097 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.097 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.097 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:08.097 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:08.097 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:08.097 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.097 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.097 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:08.097 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:08.097 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1699999999999998e-07 2023-11-25T04:23:08.097 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.097 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.098 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:08.098 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033627+0000", 2023-11-25T04:23:08.098 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.163e-05 2023-11-25T04:23:08.098 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.098 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.098 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:08.098 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033636+0000", 2023-11-25T04:23:08.098 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6859999999999995e-06 2023-11-25T04:23:08.098 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.098 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.098 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:08.099 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033642+0000", 2023-11-25T04:23:08.099 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3149999999999997e-06 2023-11-25T04:23:08.099 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.099 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.099 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:08.099 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035414+0000", 2023-11-25T04:23:08.099 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0017716380000000001 2023-11-25T04:23:08.099 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.099 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.099 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:08.099 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035539+0000", 2023-11-25T04:23:08.099 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000125477 2023-11-25T04:23:08.100 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.100 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.100 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:08.100 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035672+0000", 2023-11-25T04:23:08.100 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000133316 2023-11-25T04:23:08.100 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.100 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.100 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:08.100 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038649+0000", 2023-11-25T04:23:08.100 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029764679999999999 2023-11-25T04:23:08.100 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.100 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.100 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:08.101 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044578+0000", 2023-11-25T04:23:08.101 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0059293510000000002 2023-11-25T04:23:08.101 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.101 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.101 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:08.101 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044651+0000", 2023-11-25T04:23:08.101 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2513000000000002e-05 2023-11-25T04:23:08.101 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.101 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.101 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:08.101 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044702+0000", 2023-11-25T04:23:08.101 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1034e-05 2023-11-25T04:23:08.102 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:08.102 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:08.102 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:08.102 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.102 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.102 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:29 3.b 3:d0630e4c:::mds0_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:08.102 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:08.102 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 21.03475706, 2023-11-25T04:23:08.102 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010980227, 2023-11-25T04:23:08.102 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:08.102 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:08.103 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:08.103 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.103 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:08.103 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:08.103 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:08.103 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.103 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.103 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:08.103 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:08.103 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:08.103 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.103 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.103 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:08.104 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:08.104 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3499999999999996e-07 2023-11-25T04:23:08.104 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.104 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.104 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:08.104 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033658+0000", 2023-11-25T04:23:08.104 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1280000000000001e-06 2023-11-25T04:23:08.104 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.104 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.104 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:08.104 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033667+0000", 2023-11-25T04:23:08.104 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1430000000000004e-06 2023-11-25T04:23:08.104 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.105 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.105 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:08.105 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033678+0000", 2023-11-25T04:23:08.105 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0906e-05 2023-11-25T04:23:08.105 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.105 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.105 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:08.105 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034456+0000", 2023-11-25T04:23:08.105 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00077833899999999996 2023-11-25T04:23:08.105 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.105 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.106 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:08.106 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034575+0000", 2023-11-25T04:23:08.106 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011926000000000001 2023-11-25T04:23:08.106 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.106 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.106 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:08.106 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035001+0000", 2023-11-25T04:23:08.106 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042550999999999998 2023-11-25T04:23:08.106 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.106 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.106 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:08.106 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.037923+0000", 2023-11-25T04:23:08.106 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029219670000000001 2023-11-25T04:23:08.107 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.107 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.107 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:08.107 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044538+0000", 2023-11-25T04:23:08.107 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0066152349999999997 2023-11-25T04:23:08.107 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.107 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.107 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:08.107 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044594+0000", 2023-11-25T04:23:08.107 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5593000000000001e-05 2023-11-25T04:23:08.107 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.107 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.108 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:08.108 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044633+0000", 2023-11-25T04:23:08.108 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9511000000000002e-05 2023-11-25T04:23:08.108 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:08.108 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:08.108 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:08.108 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.108 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.108 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:34 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:08.108 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:08.108 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 21.034506947000001, 2023-11-25T04:23:08.108 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010920445000000001, 2023-11-25T04:23:08.108 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:08.109 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:08.109 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:08.109 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.109 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:08.109 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:08.109 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:08.109 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.109 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.109 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:08.109 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:08.109 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:08.109 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.110 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.110 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:08.110 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033904+0000", 2023-11-25T04:23:08.110 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6599999999999994e-07 2023-11-25T04:23:08.110 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.110 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.110 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:08.110 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033909+0000", 2023-11-25T04:23:08.110 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5639999999999996e-06 2023-11-25T04:23:08.110 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.110 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.110 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:08.110 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033923+0000", 2023-11-25T04:23:08.111 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3028e-05 2023-11-25T04:23:08.111 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.111 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.111 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:08.111 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034043+0000", 2023-11-25T04:23:08.111 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000120328 2023-11-25T04:23:08.111 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.111 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.111 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:08.111 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035576+0000", 2023-11-25T04:23:08.111 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015333720000000001 2023-11-25T04:23:08.112 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.112 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.112 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:08.112 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035699+0000", 2023-11-25T04:23:08.112 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000123001 2023-11-25T04:23:08.112 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.112 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.112 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:08.112 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044823+0000", 2023-11-25T04:23:08.112 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0091241859999999994 2023-11-25T04:23:08.112 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:08.112 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:08.113 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:08.113 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.113 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.113 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:35 3.1f 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:08.113 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:08.113 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 21.033202742, 2023-11-25T04:23:08.113 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096164779999999995, 2023-11-25T04:23:08.113 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:08.113 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:08.113 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:08.113 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.113 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:08.113 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:08.114 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:08.114 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.114 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.114 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:08.114 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:08.114 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:08.114 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.114 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.114 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:08.114 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035208+0000", 2023-11-25T04:23:08.114 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2699999999999998e-07 2023-11-25T04:23:08.114 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.114 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.115 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:08.115 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035213+0000", 2023-11-25T04:23:08.115 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.6070000000000001e-06 2023-11-25T04:23:08.115 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.115 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.115 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:08.115 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035350+0000", 2023-11-25T04:23:08.115 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013693599999999999 2023-11-25T04:23:08.115 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.115 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.115 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:08.115 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035358+0000", 2023-11-25T04:23:08.116 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0099999999999995e-06 2023-11-25T04:23:08.116 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.116 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.116 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:08.116 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036260+0000", 2023-11-25T04:23:08.116 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00090240100000000005 2023-11-25T04:23:08.116 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.116 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.116 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:08.116 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036517+0000", 2023-11-25T04:23:08.116 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00025736299999999998 2023-11-25T04:23:08.116 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.116 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.117 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:08.117 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044824+0000", 2023-11-25T04:23:08.117 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0083063340000000003 2023-11-25T04:23:08.117 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:08.117 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:08.117 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:08.117 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.117 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.117 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:37 3.1e 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:08.117 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:08.117 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 21.033036056, 2023-11-25T04:23:08.118 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0078322640000000002, 2023-11-25T04:23:08.118 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:08.118 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:08.118 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:08.118 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.118 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:08.118 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:08.118 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:08.118 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.118 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.118 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:08.118 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:08.119 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:08.119 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.119 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.119 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:08.119 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035375+0000", 2023-11-25T04:23:08.119 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9100000000000003e-07 2023-11-25T04:23:08.119 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.119 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.119 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:08.119 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035379+0000", 2023-11-25T04:23:08.119 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9430000000000002e-06 2023-11-25T04:23:08.119 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.120 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.120 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:08.120 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035388+0000", 2023-11-25T04:23:08.120 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2760000000000006e-06 2023-11-25T04:23:08.120 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.120 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.120 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:08.120 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035396+0000", 2023-11-25T04:23:08.120 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8480000000000001e-06 2023-11-25T04:23:08.120 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.120 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.120 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:08.120 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035452+0000", 2023-11-25T04:23:08.121 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5831000000000001e-05 2023-11-25T04:23:08.121 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.121 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.121 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:08.121 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035934+0000", 2023-11-25T04:23:08.121 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00048265599999999998 2023-11-25T04:23:08.121 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.121 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.121 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:08.121 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043206+0000", 2023-11-25T04:23:08.121 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.007271919 2023-11-25T04:23:08.121 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:08.121 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:08.122 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:08.122 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.122 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.122 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing(3.1a epoch 18/16 rep_tid 3 entries 18'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:36 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:23:08.122 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:08.122 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 21.031760469999998, 2023-11-25T04:23:08.122 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002073414, 2023-11-25T04:23:08.122 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:08.122 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:08.122 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:08.122 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.122 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:08.123 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:08.123 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:08.123 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.123 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.123 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:08.123 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:08.123 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:08.123 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.123 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.123 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:08.123 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036650+0000", 2023-11-25T04:23:08.123 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7300000000000002e-07 2023-11-25T04:23:08.123 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.124 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.124 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:08.124 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036659+0000", 2023-11-25T04:23:08.124 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.7499999999999992e-06 2023-11-25T04:23:08.124 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.124 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.124 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:08.124 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036673+0000", 2023-11-25T04:23:08.124 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4917e-05 2023-11-25T04:23:08.124 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.124 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.124 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:08.125 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036687+0000", 2023-11-25T04:23:08.125 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3883000000000001e-05 2023-11-25T04:23:08.125 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.125 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.125 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:08.125 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036781+0000", 2023-11-25T04:23:08.125 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3219000000000003e-05 2023-11-25T04:23:08.125 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.125 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.125 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:08.125 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038723+0000", 2023-11-25T04:23:08.125 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019422720000000001 2023-11-25T04:23:08.125 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:08.126 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:08.126 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:08.126 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.126 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.126 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:28 3.13 e18/16 3:cd5a64a3:::100.00000000:head v 18'1, mlcod=0'0)", 2023-11-25T04:23:08.126 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:08.126 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 21.028099311999998, 2023-11-25T04:23:08.126 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001850546, 2023-11-25T04:23:08.126 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:08.126 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:08.126 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:08.126 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.127 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:08.127 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:08.127 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:08.127 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.127 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.127 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:08.127 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:08.127 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:08.127 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.127 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.127 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:08.127 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:08.127 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7800000000000002e-07 2023-11-25T04:23:08.128 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.128 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.128 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:08.128 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040331+0000", 2023-11-25T04:23:08.128 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0449999999999999e-05 2023-11-25T04:23:08.128 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.128 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.128 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:08.128 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040339+0000", 2023-11-25T04:23:08.128 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0709999999999995e-06 2023-11-25T04:23:08.128 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.129 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.129 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:08.129 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040347+0000", 2023-11-25T04:23:08.129 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7079999999999997e-06 2023-11-25T04:23:08.129 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.129 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.129 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:08.129 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040410+0000", 2023-11-25T04:23:08.129 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2453000000000007e-05 2023-11-25T04:23:08.129 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.129 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.129 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:08.129 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040422+0000", 2023-11-25T04:23:08.130 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2429000000000001e-05 2023-11-25T04:23:08.130 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.130 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.130 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:08.130 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042129+0000", 2023-11-25T04:23:08.130 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001707266 2023-11-25T04:23:08.130 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.130 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.130 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:08.130 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042161+0000", 2023-11-25T04:23:08.130 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1791000000000001e-05 2023-11-25T04:23:08.130 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:08.130 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:08.131 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:08.131 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.131 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.131 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:27 3.13 e18/16 3:c91d4a1d:::mds0_inotable:head v 18'2, mlcod=0'0)", 2023-11-25T04:23:08.131 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:08.131 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 21.027895788999999, 2023-11-25T04:23:08.131 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0016886430000000001, 2023-11-25T04:23:08.131 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:08.131 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:08.131 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:08.131 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.131 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:08.132 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:08.132 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:08.132 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.132 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.132 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:08.132 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:08.132 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:08.132 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.132 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.132 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:08.132 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:08.132 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3999999999999997e-07 2023-11-25T04:23:08.132 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.133 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.133 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:08.133 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040520+0000", 2023-11-25T04:23:08.133 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2989999999999996e-06 2023-11-25T04:23:08.133 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.133 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.133 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:08.133 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040527+0000", 2023-11-25T04:23:08.133 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5730000000000002e-06 2023-11-25T04:23:08.133 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.133 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.133 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:08.134 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040535+0000", 2023-11-25T04:23:08.134 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8630000000000008e-06 2023-11-25T04:23:08.134 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.134 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.134 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:08.134 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040683+0000", 2023-11-25T04:23:08.134 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000148002 2023-11-25T04:23:08.134 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.134 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.134 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:08.134 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040700+0000", 2023-11-25T04:23:08.134 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7065e-05 2023-11-25T04:23:08.135 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.135 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.135 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:08.135 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042163+0000", 2023-11-25T04:23:08.135 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001462767 2023-11-25T04:23:08.135 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.135 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.135 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:08.135 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042203+0000", 2023-11-25T04:23:08.135 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9734000000000003e-05 2023-11-25T04:23:08.135 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:08.135 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:08.136 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:08.136 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.136 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.136 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:20 3.1c e18/16 3:3ed09add:::607.00000000:head v 18'1, mlcod=0'0)", 2023-11-25T04:23:08.136 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:08.136 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 21.027860747999998, 2023-11-25T04:23:08.136 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015271, 2023-11-25T04:23:08.136 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:08.136 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:08.136 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:08.136 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.136 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:08.137 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:08.137 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:08.137 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.137 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.137 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:08.137 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:08.137 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:08.137 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.137 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.137 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:08.137 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:08.137 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4799999999999999e-07 2023-11-25T04:23:08.137 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.138 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.138 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:08.138 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040561+0000", 2023-11-25T04:23:08.138 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1800000000000001e-05 2023-11-25T04:23:08.138 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.138 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.138 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:08.138 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040568+0000", 2023-11-25T04:23:08.138 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7730000000000002e-06 2023-11-25T04:23:08.138 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.138 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.138 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:08.139 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040587+0000", 2023-11-25T04:23:08.139 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8691000000000001e-05 2023-11-25T04:23:08.139 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.139 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.139 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:08.139 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040638+0000", 2023-11-25T04:23:08.139 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1068e-05 2023-11-25T04:23:08.139 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.139 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.139 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:08.139 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040670+0000", 2023-11-25T04:23:08.140 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2125999999999997e-05 2023-11-25T04:23:08.140 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.140 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.140 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:08.140 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042032+0000", 2023-11-25T04:23:08.140 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001361821 2023-11-25T04:23:08.140 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.140 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.140 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:08.140 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042076+0000", 2023-11-25T04:23:08.140 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4472999999999999e-05 2023-11-25T04:23:08.140 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:08.140 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:08.141 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:08.141 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.141 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.141 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:21 3.1c e18/16 3:3ed09add:::607.00000000:head v 18'2, mlcod=0'0)", 2023-11-25T04:23:08.141 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:08.141 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 21.027626023, 2023-11-25T04:23:08.141 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0013228160000000001, 2023-11-25T04:23:08.141 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:08.141 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:08.141 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:08.141 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.141 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:08.142 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:08.142 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:08.142 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.142 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.142 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:08.142 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:08.142 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:08.142 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.142 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.142 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:08.142 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:08.143 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9299999999999999e-07 2023-11-25T04:23:08.143 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.143 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.143 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:08.143 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040790+0000", 2023-11-25T04:23:08.143 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5770000000000001e-06 2023-11-25T04:23:08.143 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.143 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.143 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:08.143 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040799+0000", 2023-11-25T04:23:08.143 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6439999999999995e-06 2023-11-25T04:23:08.143 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.144 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.144 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:08.144 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040806+0000", 2023-11-25T04:23:08.144 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0180000000000004e-06 2023-11-25T04:23:08.144 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.144 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.144 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:08.144 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040907+0000", 2023-11-25T04:23:08.144 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000100096 2023-11-25T04:23:08.144 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.144 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.144 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:08.144 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040925+0000", 2023-11-25T04:23:08.145 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8536999999999999e-05 2023-11-25T04:23:08.145 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.145 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.145 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:08.145 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042079+0000", 2023-11-25T04:23:08.145 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00115358 2023-11-25T04:23:08.145 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.145 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.145 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:08.145 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042107+0000", 2023-11-25T04:23:08.145 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7971e-05 2023-11-25T04:23:08.145 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:08.146 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:08.146 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:08.146 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.146 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.146 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop_reply(mds.0.3:4 3.1f e18/16 ondisk, result = 0)", 2023-11-25T04:23:08.146 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:08.146 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 21.026286633000002, 2023-11-25T04:23:08.146 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00044092900000000001, 2023-11-25T04:23:08.146 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:08.146 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:08.146 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:08.147 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.147 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:08.147 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:08.147 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:08.147 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.147 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.147 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:08.147 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:08.147 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:08.147 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.147 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.147 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:08.148 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042124+0000", 2023-11-25T04:23:08.148 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1800000000000005e-07 2023-11-25T04:23:08.148 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.148 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.148 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:08.148 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042128+0000", 2023-11-25T04:23:08.148 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2289999999999997e-06 2023-11-25T04:23:08.148 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.148 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.148 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:08.148 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042138+0000", 2023-11-25T04:23:08.149 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9729999999999992e-06 2023-11-25T04:23:08.149 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.149 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.149 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:08.149 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042145+0000", 2023-11-25T04:23:08.149 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.336e-06 2023-11-25T04:23:08.149 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.149 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.149 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:08.149 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042441+0000", 2023-11-25T04:23:08.149 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029650500000000002 2023-11-25T04:23:08.150 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.150 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.150 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:08.150 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042448+0000", 2023-11-25T04:23:08.150 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4100000000000002e-06 2023-11-25T04:23:08.150 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.150 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.150 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:08.150 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042564+0000", 2023-11-25T04:23:08.150 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000115758 2023-11-25T04:23:08.150 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:08.151 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:08.151 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:08.151 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.151 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.151 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop_reply(mds.0.3:26 3.1f e18/16 ondisk, result = 0)", 2023-11-25T04:23:08.151 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.044422+0000", 2023-11-25T04:23:08.151 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 21.023988269, 2023-11-25T04:23:08.151 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00028785799999999999, 2023-11-25T04:23:08.151 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:08.151 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:08.151 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:08.151 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.152 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:08.152 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044422+0000", 2023-11-25T04:23:08.152 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:08.152 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.152 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.152 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:08.152 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044422+0000", 2023-11-25T04:23:08.152 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:08.152 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.152 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.152 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:08.153 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044422+0000", 2023-11-25T04:23:08.153 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9000000000000001e-07 2023-11-25T04:23:08.153 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.153 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.153 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:08.153 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044423+0000", 2023-11-25T04:23:08.153 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.669e-06 2023-11-25T04:23:08.153 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.153 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.153 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:08.153 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044427+0000", 2023-11-25T04:23:08.153 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9330000000000003e-06 2023-11-25T04:23:08.153 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.154 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.154 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:08.154 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044442+0000", 2023-11-25T04:23:08.154 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4525999999999999e-05 2023-11-25T04:23:08.154 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.154 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.154 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:08.154 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044566+0000", 2023-11-25T04:23:08.154 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012453099999999999 2023-11-25T04:23:08.154 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.154 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.154 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:08.155 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044573+0000", 2023-11-25T04:23:08.155 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4409999999999998e-06 2023-11-25T04:23:08.155 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.155 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.155 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:08.155 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044709+0000", 2023-11-25T04:23:08.155 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000136568 2023-11-25T04:23:08.155 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:08.155 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:08.155 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:08.155 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.155 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.155 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing_reply(3.1f epoch 18/16 rep_tid 8 lcod 18'4)", 2023-11-25T04:23:08.156 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:08.156 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 21.023961221, 2023-11-25T04:23:08.156 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00038154699999999998, 2023-11-25T04:23:08.156 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:08.156 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:08.156 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:08.156 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.156 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:08.156 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:08.156 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:08.156 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.157 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.157 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:08.157 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:08.157 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:08.157 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.157 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.157 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:08.157 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:08.157 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2700000000000001e-07 2023-11-25T04:23:08.157 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.157 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.157 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:08.157 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044451+0000", 2023-11-25T04:23:08.158 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7460000000000001e-06 2023-11-25T04:23:08.158 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.158 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.158 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:08.158 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044455+0000", 2023-11-25T04:23:08.158 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5809999999999998e-06 2023-11-25T04:23:08.158 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.158 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.158 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:08.158 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044458+0000", 2023-11-25T04:23:08.158 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0170000000000001e-06 2023-11-25T04:23:08.158 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.159 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.159 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:08.159 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044734+0000", 2023-11-25T04:23:08.159 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00027582799999999999 2023-11-25T04:23:08.159 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.159 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.159 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:08.159 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044830+0000", 2023-11-25T04:23:08.159 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6248000000000004e-05 2023-11-25T04:23:08.159 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:08.159 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:08.159 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:08.160 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.160 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.160 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing_reply(3.b epoch 18/16 rep_tid 5 lcod 18'2)", 2023-11-25T04:23:08.160 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:08.160 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 21.023924560000001, 2023-11-25T04:23:08.160 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000344488, 2023-11-25T04:23:08.160 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:08.160 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:08.160 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:08.160 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.160 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:08.160 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:08.161 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:08.161 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.161 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.161 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:08.161 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:08.161 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:08.161 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.161 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.161 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:08.161 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:08.161 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.55e-07 2023-11-25T04:23:08.161 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.162 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.162 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:08.162 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044487+0000", 2023-11-25T04:23:08.162 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2470000000000001e-06 2023-11-25T04:23:08.162 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.162 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.162 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:08.162 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044491+0000", 2023-11-25T04:23:08.162 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9659999999999998e-06 2023-11-25T04:23:08.162 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.162 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.162 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:08.163 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044494+0000", 2023-11-25T04:23:08.163 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2310000000000001e-06 2023-11-25T04:23:08.163 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.163 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.163 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:08.163 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044756+0000", 2023-11-25T04:23:08.163 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00026172800000000003 2023-11-25T04:23:08.163 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.163 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.163 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:08.163 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044830+0000", 2023-11-25T04:23:08.163 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4160999999999997e-05 2023-11-25T04:23:08.164 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:08.164 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:08.164 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:08.164 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.164 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.164 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:39 3.6 3:654134d2:::mds0_openfiles.0:head [omap-clear,omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2023-11-25T04:23:08.164 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:08.164 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 15.964098518, 2023-11-25T04:23:08.164 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001788658, 2023-11-25T04:23:08.164 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:08.164 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:08.164 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:08.165 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.165 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:08.165 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:08.165 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:08.165 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.165 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.165 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:08.165 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:08.165 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:08.165 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.165 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.165 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:08.165 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104314+0000", 2023-11-25T04:23:08.165 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3099999999999999e-06 2023-11-25T04:23:08.166 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.166 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.166 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:08.166 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104325+0000", 2023-11-25T04:23:08.166 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1511999999999999e-05 2023-11-25T04:23:08.166 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.166 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.166 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:08.166 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104362+0000", 2023-11-25T04:23:08.166 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6903000000000002e-05 2023-11-25T04:23:08.166 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.167 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.167 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:08.167 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104387+0000", 2023-11-25T04:23:08.167 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4766e-05 2023-11-25T04:23:08.167 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.167 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.167 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:08.167 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104465+0000", 2023-11-25T04:23:08.167 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8119000000000002e-05 2023-11-25T04:23:08.167 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.167 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.167 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:08.168 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104557+0000", 2023-11-25T04:23:08.168 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2496000000000003e-05 2023-11-25T04:23:08.168 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.168 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.168 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:08.168 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104646+0000", 2023-11-25T04:23:08.168 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8818000000000002e-05 2023-11-25T04:23:08.168 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.168 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.168 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:08.168 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105201+0000", 2023-11-25T04:23:08.168 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00055506000000000002 2023-11-25T04:23:08.169 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.169 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.169 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:08.169 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105994+0000", 2023-11-25T04:23:08.169 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00079294700000000001 2023-11-25T04:23:08.169 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.169 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.169 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:08.169 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.106055+0000", 2023-11-25T04:23:08.169 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0566000000000003e-05 2023-11-25T04:23:08.169 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.169 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.169 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:08.170 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.106100+0000", 2023-11-25T04:23:08.170 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5160999999999997e-05 2023-11-25T04:23:08.170 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:08.170 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:08.170 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:08.170 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.170 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.170 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:43 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2023-11-25T04:23:08.170 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:08.170 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 0.96387352199999998, 2023-11-25T04:23:08.170 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001751166, 2023-11-25T04:23:08.170 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:08.171 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:08.171 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:08.171 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.171 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:08.171 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:08.171 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:08.171 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.171 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.171 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:08.171 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:08.171 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:08.171 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.172 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.172 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:08.172 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104539+0000", 2023-11-25T04:23:08.172 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9529999999999999e-06 2023-11-25T04:23:08.172 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.172 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.172 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:08.172 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104557+0000", 2023-11-25T04:23:08.172 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7464000000000001e-05 2023-11-25T04:23:08.172 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.172 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.172 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:08.172 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104590+0000", 2023-11-25T04:23:08.173 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2966000000000003e-05 2023-11-25T04:23:08.173 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.173 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.173 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:08.173 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104608+0000", 2023-11-25T04:23:08.173 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8193e-05 2023-11-25T04:23:08.173 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.173 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.173 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:08.173 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104767+0000", 2023-11-25T04:23:08.173 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015882 2023-11-25T04:23:08.173 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.173 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.174 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:08.174 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104909+0000", 2023-11-25T04:23:08.174 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014244999999999999 2023-11-25T04:23:08.174 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.174 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.174 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:08.174 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.105059+0000", 2023-11-25T04:23:08.174 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014964399999999999 2023-11-25T04:23:08.174 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.174 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.174 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:08.174 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.105701+0000", 2023-11-25T04:23:08.175 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00064201099999999995 2023-11-25T04:23:08.175 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.175 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.175 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:08.175 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106233+0000", 2023-11-25T04:23:08.175 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00053178500000000001 2023-11-25T04:23:08.175 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.175 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.175 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:08.175 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106265+0000", 2023-11-25T04:23:08.175 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1986000000000002e-05 2023-11-25T04:23:08.176 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:08.176 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:08.176 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:08.176 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106288+0000", 2023-11-25T04:23:08.176 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2894000000000001e-05 2023-11-25T04:23:08.176 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:08.176 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:08.176 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:08.176 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:08.176 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:08.176 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:08.178 INFO:teuthology.orchestra.run.smithi067.stderr:nodeep-scrub is set 2023-11-25T04:23:08.179 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:08.176+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:08.179 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:23:08.271 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-11-25T04:23:08.279 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:08.278+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:08.286 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:08.286 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:23:08.286 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2023-11-25T04:23:08.286 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:08.295 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:23:08.380 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:08.379+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:08.391 INFO:teuthology.orchestra.run.smithi067.stderr:Error ENXIO: problem getting command descriptions from osd.0 2023-11-25T04:23:08.411 INFO:teuthology.orchestra.run.smithi067.stdout:osd.1: {} 2023-11-25T04:23:08.411 INFO:teuthology.orchestra.run.smithi067.stderr:osd.1: osd_enable_op_tracker = '' 2023-11-25T04:23:08.430 INFO:teuthology.orchestra.run.smithi067.stdout:osd.2: {} 2023-11-25T04:23:08.430 INFO:teuthology.orchestra.run.smithi067.stderr:osd.2: osd_enable_op_tracker = '' 2023-11-25T04:23:08.432 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-11-25T04:23:08.433 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-11-25T04:23:08.442 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:08.442 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:08.451 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:23:08.495 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:08.494+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:08.558 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:08.559 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:08.567 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:23:08.608 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:08.607+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:08.673 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:08.673 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:08.683 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:23:08.684 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:08.681+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:08.784 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:08.783+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:08.829 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:08.829 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:08.838 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:23:08.884 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:08.883+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:08.943 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:08.943 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:08.952 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:23:08.985 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2023-11-25T04:23:08.994 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:08.992+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:09.058 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:09.058 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:09.067 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:23:09.110 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:09.109+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:09.174 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:09.174 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:09.183 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:23:09.188 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:09.186+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:09.289 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:09.288+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:09.330 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:09.330 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:09.339 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:23:09.389 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:09.388+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:09.448 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:09.448 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:09.458 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:23:09.490 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:09.489+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:09.561 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:09.562 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:09.570 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:23:09.591 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:09.590+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:09.674 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:09.674 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:09.683 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:23:09.727 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:09.726+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:09.791 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:09.791 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:09.793 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:09.792+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:09.799 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:23:09.894 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:09.893+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:09.945 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:09.945 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:09.954 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:23:09.998 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:09.997+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:10.061 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:10.061 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:10.068 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:23:10.109 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:10.108+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:10.175 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:10.175 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:10.179 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2023-11-25T04:23:10.183 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:23:10.196 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:10.195+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:10.288 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:10.289 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:10.297 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:23:10.297 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:10.295+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:10.397 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:10.396+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:10.434 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-11-25T04:23:10.443 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:10.443 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:10.452 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:23:10.499 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:10.497+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:10.546 INFO:teuthology.orchestra.run.smithi067.stderr:Error ENXIO: problem getting command descriptions from osd.0 2023-11-25T04:23:10.558 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:10.559 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:10.563 INFO:teuthology.orchestra.run.smithi067.stdout:osd.1: {} 2023-11-25T04:23:10.564 INFO:teuthology.orchestra.run.smithi067.stderr:osd.1: osd_enable_op_tracker = '' 2023-11-25T04:23:10.567 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:23:10.582 INFO:teuthology.orchestra.run.smithi067.stdout:osd.2: {} 2023-11-25T04:23:10.583 INFO:teuthology.orchestra.run.smithi067.stderr:osd.2: osd_enable_op_tracker = '' 2023-11-25T04:23:10.586 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-11-25T04:23:10.586 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-11-25T04:23:10.608 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:10.607+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:10.701 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:10.699+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:10.711 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:10.711 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:23:10.711 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2023-11-25T04:23:10.711 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2023-11-25T04:23:10.712 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:10.720 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:23:10.801 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:10.800+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:10.867 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:10.867 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2023-11-25T04:23:10.867 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2023-11-25T04:23:10.867 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2023-11-25T04:23:10.867 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.868 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:155 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'109, mlcod=11'105)", 2023-11-25T04:23:10.868 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:10.868 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 33.079863531999997, 2023-11-25T04:23:10.868 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061719019999999999, 2023-11-25T04:23:10.868 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:10.868 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:10.868 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:10.868 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.869 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:10.869 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:10.869 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:10.869 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.869 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.869 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:10.869 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:10.870 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:10.870 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.870 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.870 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:10.870 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784884+0000", 2023-11-25T04:23:10.870 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3099999999999999e-07 2023-11-25T04:23:10.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.871 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.871 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:10.871 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784942+0000", 2023-11-25T04:23:10.871 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8570000000000003e-05 2023-11-25T04:23:10.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.872 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.872 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:10.872 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784949+0000", 2023-11-25T04:23:10.872 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0310000000000001e-06 2023-11-25T04:23:10.872 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.872 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.872 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:10.873 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784953+0000", 2023-11-25T04:23:10.873 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6320000000000001e-06 2023-11-25T04:23:10.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.873 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.873 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:10.873 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784993+0000", 2023-11-25T04:23:10.874 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9737000000000003e-05 2023-11-25T04:23:10.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.874 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:10.874 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785011+0000", 2023-11-25T04:23:10.875 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8349999999999999e-05 2023-11-25T04:23:10.875 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.875 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.875 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:10.875 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791024+0000", 2023-11-25T04:23:10.875 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006012997 2023-11-25T04:23:10.875 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.876 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.876 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:10.876 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791055+0000", 2023-11-25T04:23:10.876 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1254000000000003e-05 2023-11-25T04:23:10.876 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:10.876 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:10.876 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:10.877 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.877 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.877 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:156 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'110, mlcod=11'105)", 2023-11-25T04:23:10.877 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:10.877 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 33.079277482999998, 2023-11-25T04:23:10.877 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005616116, 2023-11-25T04:23:10.877 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:10.877 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:10.878 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:10.878 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.878 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:10.878 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:10.878 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:10.878 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.878 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.879 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:10.879 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:10.879 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:10.879 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.879 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.879 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:10.879 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785470+0000", 2023-11-25T04:23:10.879 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0699999999999998e-07 2023-11-25T04:23:10.880 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.883 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.883 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:10.883 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785474+0000", 2023-11-25T04:23:10.883 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.048e-06 2023-11-25T04:23:10.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.883 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.883 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:10.883 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785480+0000", 2023-11-25T04:23:10.884 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7729999999999996e-06 2023-11-25T04:23:10.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.884 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.884 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:10.884 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785483+0000", 2023-11-25T04:23:10.884 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1970000000000001e-06 2023-11-25T04:23:10.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.884 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.884 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:10.884 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787936+0000", 2023-11-25T04:23:10.884 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024529040000000001 2023-11-25T04:23:10.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.885 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.885 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:10.885 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787947+0000", 2023-11-25T04:23:10.885 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1669e-05 2023-11-25T04:23:10.885 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.885 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.885 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:10.885 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791057+0000", 2023-11-25T04:23:10.885 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031095200000000002 2023-11-25T04:23:10.885 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.885 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.885 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:10.886 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791086+0000", 2023-11-25T04:23:10.886 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8697999999999999e-05 2023-11-25T04:23:10.886 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:10.886 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:10.886 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:10.886 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.886 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.886 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:157 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'111, mlcod=11'106)", 2023-11-25T04:23:10.886 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:10.886 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 33.078684924999997, 2023-11-25T04:23:10.886 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050590280000000001, 2023-11-25T04:23:10.886 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:10.887 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:10.887 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:10.887 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.887 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:10.887 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:10.887 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:10.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.887 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.887 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:10.887 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:10.887 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:10.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.887 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.887 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:10.888 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786063+0000", 2023-11-25T04:23:10.888 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2600000000000002e-07 2023-11-25T04:23:10.888 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.888 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.888 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:10.888 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786069+0000", 2023-11-25T04:23:10.888 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8839999999999999e-06 2023-11-25T04:23:10.888 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.888 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.888 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:10.888 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786079+0000", 2023-11-25T04:23:10.888 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0124e-05 2023-11-25T04:23:10.888 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.889 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.889 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:10.889 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786085+0000", 2023-11-25T04:23:10.889 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6660000000000002e-06 2023-11-25T04:23:10.889 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.889 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.889 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:10.889 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788160+0000", 2023-11-25T04:23:10.889 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020747669999999999 2023-11-25T04:23:10.889 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.889 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.889 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:10.890 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788180+0000", 2023-11-25T04:23:10.890 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0208999999999999e-05 2023-11-25T04:23:10.890 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.890 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.890 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:10.890 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791087+0000", 2023-11-25T04:23:10.890 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029064630000000002 2023-11-25T04:23:10.890 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.890 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.890 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:10.890 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791121+0000", 2023-11-25T04:23:10.890 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4289000000000003e-05 2023-11-25T04:23:10.890 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:10.891 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:10.891 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:10.891 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.891 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.891 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:158 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'112, mlcod=11'106)", 2023-11-25T04:23:10.891 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:10.891 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 33.078268532000003, 2023-11-25T04:23:10.891 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046663030000000001, 2023-11-25T04:23:10.891 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:10.891 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:10.891 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:10.892 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.892 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:10.892 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:10.892 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:10.892 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.892 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.892 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:10.892 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:10.892 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:10.892 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.892 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.892 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:10.893 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786479+0000", 2023-11-25T04:23:10.893 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2799999999999996e-07 2023-11-25T04:23:10.893 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.893 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.893 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:10.893 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786498+0000", 2023-11-25T04:23:10.893 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9069e-05 2023-11-25T04:23:10.894 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.894 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.894 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:10.894 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786508+0000", 2023-11-25T04:23:10.894 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5810000000000003e-06 2023-11-25T04:23:10.894 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.894 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.894 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:10.894 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786512+0000", 2023-11-25T04:23:10.894 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4889999999999997e-06 2023-11-25T04:23:10.894 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.894 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:10.895 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788438+0000", 2023-11-25T04:23:10.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019257059999999999 2023-11-25T04:23:10.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:10.895 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788449+0000", 2023-11-25T04:23:10.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1419e-05 2023-11-25T04:23:10.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.896 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:10.896 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791122+0000", 2023-11-25T04:23:10.896 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002672822 2023-11-25T04:23:10.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.896 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:10.896 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791145+0000", 2023-11-25T04:23:10.896 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2589e-05 2023-11-25T04:23:10.896 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:10.896 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:10.896 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:10.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.897 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.897 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:206 1.0 e12/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 12'152, mlcod=12'148)", 2023-11-25T04:23:10.897 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:10.897 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.566500062000003, 2023-11-25T04:23:10.897 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046728189999999999, 2023-11-25T04:23:10.897 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:10.897 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:10.897 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:10.897 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.897 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:10.897 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:10.897 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:10.897 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.898 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:10.898 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:10.898 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:10.898 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.898 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:10.898 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298248+0000", 2023-11-25T04:23:10.898 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3600000000000003e-07 2023-11-25T04:23:10.898 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.898 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:10.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298278+0000", 2023-11-25T04:23:10.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0474e-05 2023-11-25T04:23:10.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.899 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:10.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298288+0000", 2023-11-25T04:23:10.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0156000000000001e-05 2023-11-25T04:23:10.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.899 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:10.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298293+0000", 2023-11-25T04:23:10.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7400000000000004e-06 2023-11-25T04:23:10.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.900 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:10.900 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298333+0000", 2023-11-25T04:23:10.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9666000000000003e-05 2023-11-25T04:23:10.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.900 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:10.900 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298351+0000", 2023-11-25T04:23:10.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8544999999999999e-05 2023-11-25T04:23:10.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.901 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:10.901 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302891+0000", 2023-11-25T04:23:10.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045398519999999996 2023-11-25T04:23:10.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.901 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:10.901 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302920+0000", 2023-11-25T04:23:10.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8750000000000001e-05 2023-11-25T04:23:10.901 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:10.901 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:10.901 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:10.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.902 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:14 3.15 3:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:10.902 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:10.902 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 23.831365705, 2023-11-25T04:23:10.902 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010990437, 2023-11-25T04:23:10.902 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:10.902 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:10.902 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:10.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.902 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:10.902 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:10.902 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:10.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.903 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.903 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:10.903 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:10.903 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:10.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.903 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.903 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:10.903 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033383+0000", 2023-11-25T04:23:10.903 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9870000000000002e-06 2023-11-25T04:23:10.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.903 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.904 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:10.904 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033398+0000", 2023-11-25T04:23:10.904 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4569999999999999e-05 2023-11-25T04:23:10.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.904 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:10.904 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033411+0000", 2023-11-25T04:23:10.904 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3543999999999999e-05 2023-11-25T04:23:10.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.904 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:10.904 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033423+0000", 2023-11-25T04:23:10.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1905999999999999e-05 2023-11-25T04:23:10.905 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.905 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.905 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:10.905 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034433+0000", 2023-11-25T04:23:10.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00101022 2023-11-25T04:23:10.905 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.905 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.905 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:10.905 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034539+0000", 2023-11-25T04:23:10.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010597199999999999 2023-11-25T04:23:10.905 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.906 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.906 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:10.906 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034667+0000", 2023-11-25T04:23:10.906 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012765500000000001 2023-11-25T04:23:10.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.906 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.906 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:10.906 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037803+0000", 2023-11-25T04:23:10.906 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031362220000000001 2023-11-25T04:23:10.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.906 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.906 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:10.907 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:23:10.907 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0064964669999999997 2023-11-25T04:23:10.907 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.907 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.907 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:10.907 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044331+0000", 2023-11-25T04:23:10.907 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1544999999999997e-05 2023-11-25T04:23:10.907 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.907 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.907 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:10.907 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044372+0000", 2023-11-25T04:23:10.907 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0349000000000003e-05 2023-11-25T04:23:10.908 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:10.908 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:10.908 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:10.908 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.908 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.908 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:15 3.15 3:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:10.908 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:10.908 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 23.830706055, 2023-11-25T04:23:10.908 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010464362, 2023-11-25T04:23:10.908 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:10.908 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:10.908 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:10.909 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.909 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:10.909 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:10.909 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:10.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.909 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.909 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:10.909 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:10.909 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:10.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.909 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.910 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:10.910 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034042+0000", 2023-11-25T04:23:10.910 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0999999999999997e-07 2023-11-25T04:23:10.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.910 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.910 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:10.910 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034048+0000", 2023-11-25T04:23:10.910 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0390000000000001e-06 2023-11-25T04:23:10.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.910 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.910 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:10.910 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034059+0000", 2023-11-25T04:23:10.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1182999999999999e-05 2023-11-25T04:23:10.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.911 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:10.911 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034067+0000", 2023-11-25T04:23:10.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1559999999999995e-06 2023-11-25T04:23:10.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.911 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:10.911 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035005+0000", 2023-11-25T04:23:10.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00093747399999999999 2023-11-25T04:23:10.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.912 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:10.912 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035077+0000", 2023-11-25T04:23:10.912 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2747e-05 2023-11-25T04:23:10.912 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.912 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:10.912 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035219+0000", 2023-11-25T04:23:10.912 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000141877 2023-11-25T04:23:10.912 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.912 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:10.913 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037810+0000", 2023-11-25T04:23:10.913 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002590752 2023-11-25T04:23:10.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.913 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.913 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:10.913 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044437+0000", 2023-11-25T04:23:10.913 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066270110000000004 2023-11-25T04:23:10.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.913 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.913 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:10.913 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044481+0000", 2023-11-25T04:23:10.913 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4106000000000001e-05 2023-11-25T04:23:10.914 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.914 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.914 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:10.914 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044505+0000", 2023-11-25T04:23:10.914 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4207e-05 2023-11-25T04:23:10.914 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:10.914 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:10.914 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:10.914 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.914 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.914 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:16 3.1d 3:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:10.914 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:10.915 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 23.830664207000002, 2023-11-25T04:23:10.915 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010413923, 2023-11-25T04:23:10.915 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:10.915 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:10.915 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:10.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.915 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:10.915 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:10.915 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:10.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.915 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:10.916 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:10.916 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:10.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.916 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:10.916 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:10.916 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4499999999999997e-07 2023-11-25T04:23:10.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.916 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:10.916 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034087+0000", 2023-11-25T04:23:10.916 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9770000000000002e-06 2023-11-25T04:23:10.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.917 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:10.917 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034094+0000", 2023-11-25T04:23:10.917 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8809999999999998e-06 2023-11-25T04:23:10.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.917 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:10.917 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034100+0000", 2023-11-25T04:23:10.917 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8440000000000003e-06 2023-11-25T04:23:10.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.918 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:10.918 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034993+0000", 2023-11-25T04:23:10.918 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00089336799999999996 2023-11-25T04:23:10.918 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.918 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.918 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:10.918 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035099+0000", 2023-11-25T04:23:10.918 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106037 2023-11-25T04:23:10.918 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.918 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.918 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:10.918 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035276+0000", 2023-11-25T04:23:10.919 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017631799999999999 2023-11-25T04:23:10.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.919 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.919 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:10.919 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037815+0000", 2023-11-25T04:23:10.919 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025395959999999999 2023-11-25T04:23:10.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.919 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.919 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:10.919 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044420+0000", 2023-11-25T04:23:10.919 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066045749999999997 2023-11-25T04:23:10.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.920 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.920 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:10.920 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044463+0000", 2023-11-25T04:23:10.920 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3205999999999999e-05 2023-11-25T04:23:10.920 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.920 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.920 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:10.920 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044497+0000", 2023-11-25T04:23:10.920 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3476000000000003e-05 2023-11-25T04:23:10.920 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:10.920 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:10.921 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:10.921 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.921 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.921 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:17 3.1d 3:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:10.921 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:10.921 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 23.830638998000001, 2023-11-25T04:23:10.921 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01127356, 2023-11-25T04:23:10.921 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:10.921 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:10.921 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:10.921 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.921 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:10.921 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:10.922 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:10.922 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.922 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.922 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:10.923 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:10.923 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:10.923 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.923 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.923 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:10.923 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:10.923 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1100000000000001e-07 2023-11-25T04:23:10.923 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.923 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.923 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:10.923 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034112+0000", 2023-11-25T04:23:10.924 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8600000000000003e-06 2023-11-25T04:23:10.924 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.924 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.924 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:10.924 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034120+0000", 2023-11-25T04:23:10.924 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7700000000000001e-06 2023-11-25T04:23:10.924 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.924 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.924 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:10.924 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034127+0000", 2023-11-25T04:23:10.924 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0999999999999998e-06 2023-11-25T04:23:10.924 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.924 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.925 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:10.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035601+0000", 2023-11-25T04:23:10.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014739410000000001 2023-11-25T04:23:10.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.925 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:10.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035685+0000", 2023-11-25T04:23:10.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3677999999999994e-05 2023-11-25T04:23:10.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.925 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:10.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035827+0000", 2023-11-25T04:23:10.926 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014232599999999999 2023-11-25T04:23:10.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.926 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.926 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:10.926 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038725+0000", 2023-11-25T04:23:10.926 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028982779999999998 2023-11-25T04:23:10.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.926 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.926 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:10.926 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045323+0000", 2023-11-25T04:23:10.926 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0065980780000000003 2023-11-25T04:23:10.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.927 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.927 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:10.927 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045354+0000", 2023-11-25T04:23:10.927 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0463e-05 2023-11-25T04:23:10.927 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.927 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.927 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:10.927 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045381+0000", 2023-11-25T04:23:10.927 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7654999999999999e-05 2023-11-25T04:23:10.927 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:10.927 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:10.927 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:10.928 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.928 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.928 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:22 3.e 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:10.928 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:10.928 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 23.830612585000001, 2023-11-25T04:23:10.928 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010133422, 2023-11-25T04:23:10.928 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:10.928 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:10.928 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:10.928 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.928 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:10.928 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:10.928 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:10.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.929 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:10.929 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:10.929 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:10.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.929 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:10.929 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034135+0000", 2023-11-25T04:23:10.929 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-07 2023-11-25T04:23:10.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.929 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:10.930 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034138+0000", 2023-11-25T04:23:10.930 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6009999999999999e-06 2023-11-25T04:23:10.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.930 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.930 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:10.930 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034145+0000", 2023-11-25T04:23:10.930 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1169999999999998e-06 2023-11-25T04:23:10.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.930 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.930 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:10.930 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034150+0000", 2023-11-25T04:23:10.930 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9080000000000003e-06 2023-11-25T04:23:10.931 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.931 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.931 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:10.931 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034209+0000", 2023-11-25T04:23:10.931 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8149000000000002e-05 2023-11-25T04:23:10.931 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.931 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.931 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:10.931 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034415+0000", 2023-11-25T04:23:10.931 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020637699999999999 2023-11-25T04:23:10.931 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.931 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.932 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:10.932 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034584+0000", 2023-11-25T04:23:10.932 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016870399999999999 2023-11-25T04:23:10.932 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.932 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.932 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:10.932 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.036979+0000", 2023-11-25T04:23:10.932 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023957509999999998 2023-11-25T04:23:10.932 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.932 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.932 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:10.932 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044161+0000", 2023-11-25T04:23:10.933 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0071811590000000003 2023-11-25T04:23:10.933 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.933 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.933 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:10.933 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044224+0000", 2023-11-25T04:23:10.933 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3454000000000006e-05 2023-11-25T04:23:10.933 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.933 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.933 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:10.933 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044268+0000", 2023-11-25T04:23:10.933 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3761999999999999e-05 2023-11-25T04:23:10.933 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:10.933 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:10.934 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:10.934 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.934 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.934 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:23 3.e 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:10.934 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:10.934 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 23.830581175999999, 2023-11-25T04:23:10.934 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010281676, 2023-11-25T04:23:10.934 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:10.934 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:10.934 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:10.934 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.935 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:10.935 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:10.935 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:10.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.935 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.935 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:10.935 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:10.935 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:10.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.935 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.935 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:10.935 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:10.935 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5799999999999999e-07 2023-11-25T04:23:10.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.936 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:10.936 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034169+0000", 2023-11-25T04:23:10.936 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1879999999999998e-06 2023-11-25T04:23:10.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.936 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:10.936 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034177+0000", 2023-11-25T04:23:10.936 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7020000000000001e-06 2023-11-25T04:23:10.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.937 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:10.937 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034184+0000", 2023-11-25T04:23:10.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3910000000000003e-06 2023-11-25T04:23:10.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.937 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.937 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:10.937 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034924+0000", 2023-11-25T04:23:10.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00074032999999999998 2023-11-25T04:23:10.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.937 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.937 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:10.937 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034991+0000", 2023-11-25T04:23:10.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7347000000000004e-05 2023-11-25T04:23:10.938 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.938 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.938 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:10.938 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035096+0000", 2023-11-25T04:23:10.938 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010488 2023-11-25T04:23:10.938 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.938 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.938 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:10.938 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037588+0000", 2023-11-25T04:23:10.938 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024918309999999999 2023-11-25T04:23:10.938 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.938 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.939 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:10.939 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044367+0000", 2023-11-25T04:23:10.939 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067787589999999997 2023-11-25T04:23:10.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.939 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:10.939 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044415+0000", 2023-11-25T04:23:10.939 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7803000000000003e-05 2023-11-25T04:23:10.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.939 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:10.939 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044447+0000", 2023-11-25T04:23:10.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2886999999999999e-05 2023-11-25T04:23:10.940 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:10.940 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:10.940 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:10.940 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.940 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.940 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:30 3.15 3:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:10.940 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:10.940 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 23.830522735999999, 2023-11-25T04:23:10.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010365070000000001, 2023-11-25T04:23:10.940 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:10.940 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:10.940 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:10.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.941 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:10.941 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:10.941 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:10.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.941 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:10.941 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:10.941 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:10.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.941 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:10.942 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034225+0000", 2023-11-25T04:23:10.942 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7400000000000002e-07 2023-11-25T04:23:10.942 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.942 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.942 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:10.942 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034229+0000", 2023-11-25T04:23:10.942 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3100000000000002e-06 2023-11-25T04:23:10.942 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.942 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.942 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:10.942 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034239+0000", 2023-11-25T04:23:10.942 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9140000000000003e-06 2023-11-25T04:23:10.942 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.943 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:10.943 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034246+0000", 2023-11-25T04:23:10.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0149999999999998e-06 2023-11-25T04:23:10.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.943 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:10.943 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035572+0000", 2023-11-25T04:23:10.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001325655 2023-11-25T04:23:10.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.943 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:10.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035730+0000", 2023-11-25T04:23:10.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015827899999999999 2023-11-25T04:23:10.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:10.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035857+0000", 2023-11-25T04:23:10.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012644699999999999 2023-11-25T04:23:10.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:10.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038737+0000", 2023-11-25T04:23:10.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002880077 2023-11-25T04:23:10.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:10.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044540+0000", 2023-11-25T04:23:10.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058034239999999997 2023-11-25T04:23:10.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:10.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044569+0000", 2023-11-25T04:23:10.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8388999999999999e-05 2023-11-25T04:23:10.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:10.946 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044589+0000", 2023-11-25T04:23:10.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0686000000000002e-05 2023-11-25T04:23:10.946 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:10.946 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:10.946 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:10.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.946 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:33 3.1b 3:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:10.946 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:10.946 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 23.830477238, 2023-11-25T04:23:10.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099754049999999997, 2023-11-25T04:23:10.946 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:10.947 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:23:10.947 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:10.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.947 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:10.947 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:10.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:10.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.947 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:10.947 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:10.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:10.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.948 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:10.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:10.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4799999999999998e-07 2023-11-25T04:23:10.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.948 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:10.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034273+0000", 2023-11-25T04:23:10.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7819999999999998e-06 2023-11-25T04:23:10.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.948 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:10.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034285+0000", 2023-11-25T04:23:10.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2058e-05 2023-11-25T04:23:10.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:10.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034291+0000", 2023-11-25T04:23:10.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0680000000000002e-06 2023-11-25T04:23:10.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:10.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034370+0000", 2023-11-25T04:23:10.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8452e-05 2023-11-25T04:23:10.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:10.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034477+0000", 2023-11-25T04:23:10.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107471 2023-11-25T04:23:10.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:10.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044245+0000", 2023-11-25T04:23:10.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0097679259999999997 2023-11-25T04:23:10.950 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:10.950 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:10.950 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:10.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.951 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:36 3.1a 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:10.951 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:10.951 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 23.830430565, 2023-11-25T04:23:10.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058179099999999999, 2023-11-25T04:23:10.951 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:10.951 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:23:10.951 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:10.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:10.951 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:10.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:10.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.952 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:10.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:10.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:10.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.952 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:10.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034317+0000", 2023-11-25T04:23:10.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1999999999999999e-07 2023-11-25T04:23:10.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:10.953 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034321+0000", 2023-11-25T04:23:10.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.233e-06 2023-11-25T04:23:10.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:10.953 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034337+0000", 2023-11-25T04:23:10.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5308999999999999e-05 2023-11-25T04:23:10.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:10.953 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034354+0000", 2023-11-25T04:23:10.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7411999999999999e-05 2023-11-25T04:23:10.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.954 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:10.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034465+0000", 2023-11-25T04:23:10.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011154699999999999 2023-11-25T04:23:10.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.954 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:10.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034560+0000", 2023-11-25T04:23:10.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4186999999999998e-05 2023-11-25T04:23:10.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:10.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.040134+0000", 2023-11-25T04:23:10.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055746019999999997 2023-11-25T04:23:10.955 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:10.955 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:10.955 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:10.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.955 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:9 3.10 e18/16 3:09401a81:::601.00000000:head v 18'4, mlcod=0'0)", 2023-11-25T04:23:10.955 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:10.955 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 23.821536256999998, 2023-11-25T04:23:10.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061641020000000003, 2023-11-25T04:23:10.956 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:10.956 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:10.956 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:10.956 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.956 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:10.956 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:10.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:10.956 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.956 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.956 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:10.956 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:10.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:10.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.957 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:10.957 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:10.957 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5300000000000001e-07 2023-11-25T04:23:10.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.957 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:10.957 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043222+0000", 2023-11-25T04:23:10.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.082e-05 2023-11-25T04:23:10.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:10.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043228+0000", 2023-11-25T04:23:10.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6699999999999997e-06 2023-11-25T04:23:10.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.959 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:10.959 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043236+0000", 2023-11-25T04:23:10.959 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8189999999999992e-06 2023-11-25T04:23:10.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.959 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:10.959 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043507+0000", 2023-11-25T04:23:10.960 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000271012 2023-11-25T04:23:10.960 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.960 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.960 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:10.960 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043530+0000", 2023-11-25T04:23:10.960 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2659000000000001e-05 2023-11-25T04:23:10.960 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.961 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:10.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049321+0000", 2023-11-25T04:23:10.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057915129999999999 2023-11-25T04:23:10.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.961 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:10.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049375+0000", 2023-11-25T04:23:10.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3455999999999998e-05 2023-11-25T04:23:10.962 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:10.962 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:10.962 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:10.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.963 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:1 3.14 e18/16 3:292cf221:::200.00000000:head v 18'3, mlcod=0'0)", 2023-11-25T04:23:10.963 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:10.963 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 23.821499544000002, 2023-11-25T04:23:10.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0060983599999999997, 2023-11-25T04:23:10.963 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:10.963 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:10.963 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:10.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.964 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:10.964 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:10.964 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:10.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.964 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.964 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:10.964 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:10.965 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:10.965 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.965 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.965 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:10.965 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043248+0000", 2023-11-25T04:23:10.965 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9399999999999995e-07 2023-11-25T04:23:10.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:10.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043252+0000", 2023-11-25T04:23:10.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1799999999999998e-06 2023-11-25T04:23:10.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.967 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:10.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043259+0000", 2023-11-25T04:23:10.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0049999999999998e-06 2023-11-25T04:23:10.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.967 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:10.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043265+0000", 2023-11-25T04:23:10.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0859999999999998e-06 2023-11-25T04:23:10.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.968 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:10.968 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043311+0000", 2023-11-25T04:23:10.968 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6082999999999999e-05 2023-11-25T04:23:10.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.968 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:10.969 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043330+0000", 2023-11-25T04:23:10.969 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8556e-05 2023-11-25T04:23:10.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.969 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:10.969 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049303+0000", 2023-11-25T04:23:10.969 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059733470000000004 2023-11-25T04:23:10.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.970 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:10.970 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049346+0000", 2023-11-25T04:23:10.970 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2608999999999998e-05 2023-11-25T04:23:10.970 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:10.970 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:10.970 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:10.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.971 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:3 3.14 e18/16 3:287034c4:::1.00000000.inode:head v 18'4, mlcod=0'0)", 2023-11-25T04:23:10.971 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:10.971 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 23.820983038000001, 2023-11-25T04:23:10.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057040090000000003, 2023-11-25T04:23:10.971 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:10.971 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:10.971 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:10.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:10.972 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:10.972 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:10.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:10.973 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:10.973 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:10.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.973 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.973 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:10.973 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:10.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9799999999999999e-07 2023-11-25T04:23:10.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.974 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:10.974 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043785+0000", 2023-11-25T04:23:10.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0721999999999999e-05 2023-11-25T04:23:10.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:10.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043793+0000", 2023-11-25T04:23:10.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5839999999999995e-06 2023-11-25T04:23:10.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:10.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043803+0000", 2023-11-25T04:23:10.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0729999999999994e-06 2023-11-25T04:23:10.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.976 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:10.976 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043843+0000", 2023-11-25T04:23:10.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0747999999999997e-05 2023-11-25T04:23:10.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.977 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:10.977 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043856+0000", 2023-11-25T04:23:10.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2697000000000001e-05 2023-11-25T04:23:10.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.977 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:10.977 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049426+0000", 2023-11-25T04:23:10.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055704409999999998 2023-11-25T04:23:10.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.978 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.978 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:10.978 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049468+0000", 2023-11-25T04:23:10.978 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1446000000000003e-05 2023-11-25T04:23:10.978 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:10.978 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:10.979 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:10.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.979 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "pg_update_log_missing(3.14 epoch 18/16 rep_tid 171 entries 18'5 (0'0) error 3:2e2fa760:::200.00000001:head by mds.0.3:32 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:23:10.979 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:10.979 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 23.820822074999999, 2023-11-25T04:23:10.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005556938, 2023-11-25T04:23:10.979 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:10.980 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2023-11-25T04:23:10.980 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:10.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:10.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:10.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:10.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:10.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:10.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:10.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.982 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:10.982 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:10.982 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.03e-07 2023-11-25T04:23:10.982 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.982 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.982 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:10.982 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043931+0000", 2023-11-25T04:23:10.983 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1079999999999998e-06 2023-11-25T04:23:10.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.983 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:10.983 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043944+0000", 2023-11-25T04:23:10.983 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2852e-05 2023-11-25T04:23:10.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.983 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:10.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043955+0000", 2023-11-25T04:23:10.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1037e-05 2023-11-25T04:23:10.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.984 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:10.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044088+0000", 2023-11-25T04:23:10.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013269599999999999 2023-11-25T04:23:10.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:10.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049482+0000", 2023-11-25T04:23:10.985 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053940419999999999 2023-11-25T04:23:10.985 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:10.985 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:10.986 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:10.986 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.986 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.986 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:10 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'5, mlcod=0'0)", 2023-11-25T04:23:10.986 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:10.986 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 23.820732047, 2023-11-25T04:23:10.987 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054493099999999997, 2023-11-25T04:23:10.987 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:10.987 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:10.987 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:10.987 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.987 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:10.987 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:10.987 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:10.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.988 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:10.988 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:10.988 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:10.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.989 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:10.989 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:10.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6100000000000002e-07 2023-11-25T04:23:10.989 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.989 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.989 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:10.989 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044019+0000", 2023-11-25T04:23:10.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.839e-06 2023-11-25T04:23:10.990 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.990 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.990 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:10.990 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044026+0000", 2023-11-25T04:23:10.990 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5869999999999998e-06 2023-11-25T04:23:10.990 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.991 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.991 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:10.991 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044031+0000", 2023-11-25T04:23:10.991 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4909999999999996e-06 2023-11-25T04:23:10.991 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.991 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.991 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:10.991 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044070+0000", 2023-11-25T04:23:10.992 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8829999999999999e-05 2023-11-25T04:23:10.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.992 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:10.992 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044081+0000", 2023-11-25T04:23:10.992 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1447999999999999e-05 2023-11-25T04:23:10.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.993 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:10.993 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049406+0000", 2023-11-25T04:23:10.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053246480000000004 2023-11-25T04:23:10.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.993 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:10.993 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049464+0000", 2023-11-25T04:23:10.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8205999999999998e-05 2023-11-25T04:23:10.994 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:10.994 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:10.994 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:10.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.994 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:11 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'6, mlcod=0'0)", 2023-11-25T04:23:10.995 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:10.995 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 23.820530782999999, 2023-11-25T04:23:10.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052673499999999996, 2023-11-25T04:23:10.995 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:10.995 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:10.995 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:10.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.996 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:10.996 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:10.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:10.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.996 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:10.996 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:10.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:10.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.997 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:10.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:10.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0199999999999998e-07 2023-11-25T04:23:10.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:10.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044220+0000", 2023-11-25T04:23:10.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7720000000000001e-06 2023-11-25T04:23:10.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:10.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044226+0000", 2023-11-25T04:23:10.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8579999999999996e-06 2023-11-25T04:23:10.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:10.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:10.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:11.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044232+0000", 2023-11-25T04:23:11.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1700000000000002e-06 2023-11-25T04:23:11.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:11.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044274+0000", 2023-11-25T04:23:11.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1254999999999997e-05 2023-11-25T04:23:11.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.001 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:11.001 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:23:11.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6880999999999999e-05 2023-11-25T04:23:11.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.001 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:11.001 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049466+0000", 2023-11-25T04:23:11.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051659219999999999 2023-11-25T04:23:11.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:11.002 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049483+0000", 2023-11-25T04:23:11.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.719e-05 2023-11-25T04:23:11.002 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:11.003 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:11.003 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:11.003 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:11.003 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:11.003 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:11.006 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:23:11.006 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:11.003+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:11.105 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:11.104+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:11.114 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:11.114 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:23:11.114 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2023-11-25T04:23:11.114 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:11.123 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:23:11.124 INFO:teuthology.orchestra.run.smithi067.stderr:noscrub is unset 2023-11-25T04:23:11.207 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:11.206+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:11.229 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:11.229 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:23:11.229 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2023-11-25T04:23:11.229 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2023-11-25T04:23:11.229 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:11.237 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:23:11.308 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:11.307+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:11.345 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:11.345 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2023-11-25T04:23:11.345 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2023-11-25T04:23:11.345 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2023-11-25T04:23:11.345 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.346 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:11.346 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:11.346 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 24.311070136000001, 2023-11-25T04:23:11.346 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01072943, 2023-11-25T04:23:11.346 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:11.346 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:11.346 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:11.346 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.346 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:11.346 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:11.347 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:11.347 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.347 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.347 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:11.347 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:11.347 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:11.347 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.347 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.347 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:11.347 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031829+0000", 2023-11-25T04:23:11.347 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.666e-06 2023-11-25T04:23:11.348 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.348 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.348 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:11.348 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031839+0000", 2023-11-25T04:23:11.348 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9490000000000008e-06 2023-11-25T04:23:11.348 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.348 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.348 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:11.348 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031857+0000", 2023-11-25T04:23:11.348 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8093000000000001e-05 2023-11-25T04:23:11.348 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.349 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.349 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:11.349 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031882+0000", 2023-11-25T04:23:11.349 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4294000000000002e-05 2023-11-25T04:23:11.349 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.349 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.349 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:11.349 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033130+0000", 2023-11-25T04:23:11.349 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001248264 2023-11-25T04:23:11.349 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.349 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.349 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:11.350 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033426+0000", 2023-11-25T04:23:11.350 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029620999999999998 2023-11-25T04:23:11.350 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.350 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.350 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:11.350 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033814+0000", 2023-11-25T04:23:11.350 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000387742 2023-11-25T04:23:11.350 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.350 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.350 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:11.350 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036793+0000", 2023-11-25T04:23:11.350 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002979598 2023-11-25T04:23:11.351 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.351 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.351 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:11.351 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042455+0000", 2023-11-25T04:23:11.351 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0056619260000000003 2023-11-25T04:23:11.351 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.351 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.351 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:11.351 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042502+0000", 2023-11-25T04:23:11.351 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7139999999999999e-05 2023-11-25T04:23:11.351 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.351 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.351 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:11.352 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042557+0000", 2023-11-25T04:23:11.352 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4548e-05 2023-11-25T04:23:11.352 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:11.352 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:11.352 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:11.352 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.352 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.352 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:11.352 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:11.352 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 24.309871601000001, 2023-11-25T04:23:11.352 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011505202000000001, 2023-11-25T04:23:11.352 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:11.353 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:11.353 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:11.353 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.353 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:11.353 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:11.353 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:11.353 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.353 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.353 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:11.353 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:11.353 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:11.353 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.354 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.354 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:11.354 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033027+0000", 2023-11-25T04:23:11.354 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0699999999999999e-06 2023-11-25T04:23:11.354 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.354 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.354 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:11.354 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033035+0000", 2023-11-25T04:23:11.354 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0830000000000004e-06 2023-11-25T04:23:11.354 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.354 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.355 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:11.355 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033059+0000", 2023-11-25T04:23:11.355 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3975e-05 2023-11-25T04:23:11.355 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.355 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.355 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:11.355 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033482+0000", 2023-11-25T04:23:11.355 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042253700000000001 2023-11-25T04:23:11.355 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.355 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.355 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:11.355 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034520+0000", 2023-11-25T04:23:11.355 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001037991 2023-11-25T04:23:11.356 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.356 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.356 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:11.356 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034611+0000", 2023-11-25T04:23:11.356 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1113999999999995e-05 2023-11-25T04:23:11.356 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.356 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.356 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:11.356 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034769+0000", 2023-11-25T04:23:11.356 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000157641 2023-11-25T04:23:11.356 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.357 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.357 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:11.357 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038292+0000", 2023-11-25T04:23:11.357 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035231160000000002 2023-11-25T04:23:11.357 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.357 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.357 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:11.357 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044440+0000", 2023-11-25T04:23:11.357 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0061486980000000002 2023-11-25T04:23:11.357 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.357 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.357 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:11.358 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044495+0000", 2023-11-25T04:23:11.358 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4339000000000003e-05 2023-11-25T04:23:11.358 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.358 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.358 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:11.358 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044531+0000", 2023-11-25T04:23:11.358 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6637999999999997e-05 2023-11-25T04:23:11.358 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:11.358 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:11.358 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:11.359 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.359 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.359 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:11.359 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:11.359 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 24.309397247, 2023-11-25T04:23:11.359 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0057634039999999997, 2023-11-25T04:23:11.359 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:11.359 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:11.359 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:11.359 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.359 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:11.360 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:11.360 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:11.360 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.360 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.360 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:11.360 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:11.360 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:11.360 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.360 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.360 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:11.360 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:11.360 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7900000000000007e-07 2023-11-25T04:23:11.361 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.361 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.361 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:11.361 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033507+0000", 2023-11-25T04:23:11.361 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8900000000000004e-06 2023-11-25T04:23:11.361 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.361 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.361 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:11.361 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033519+0000", 2023-11-25T04:23:11.361 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1756000000000001e-05 2023-11-25T04:23:11.361 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.361 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.362 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:11.362 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033527+0000", 2023-11-25T04:23:11.362 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3610000000000001e-06 2023-11-25T04:23:11.362 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.362 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.362 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:11.362 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034451+0000", 2023-11-25T04:23:11.362 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00092372400000000001 2023-11-25T04:23:11.362 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.362 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.362 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:11.363 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034809+0000", 2023-11-25T04:23:11.363 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00035757699999999998 2023-11-25T04:23:11.363 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.363 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.363 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:23:11.363 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034965+0000", 2023-11-25T04:23:11.363 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015638699999999999 2023-11-25T04:23:11.363 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.363 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.363 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:11.363 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038299+0000", 2023-11-25T04:23:11.363 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0033336559999999999 2023-11-25T04:23:11.364 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.364 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.364 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:11.364 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039169+0000", 2023-11-25T04:23:11.364 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00087066400000000005 2023-11-25T04:23:11.364 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.364 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.364 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:11.364 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039226+0000", 2023-11-25T04:23:11.364 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6511999999999997e-05 2023-11-25T04:23:11.364 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.364 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.365 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:11.365 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039264+0000", 2023-11-25T04:23:11.365 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7898000000000002e-05 2023-11-25T04:23:11.365 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:11.365 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:11.365 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:11.365 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.365 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.365 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:25 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:11.365 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:11.365 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 24.309318662999999, 2023-11-25T04:23:11.366 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096656169999999996, 2023-11-25T04:23:11.366 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:11.366 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:11.366 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:11.366 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.366 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:11.366 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:11.366 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:11.366 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.366 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.366 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:11.367 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:11.367 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:11.367 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.367 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.367 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:11.367 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033580+0000", 2023-11-25T04:23:11.367 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8199999999999999e-07 2023-11-25T04:23:11.367 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.367 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.367 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:11.367 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033585+0000", 2023-11-25T04:23:11.367 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3680000000000001e-06 2023-11-25T04:23:11.368 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.368 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.368 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:11.368 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033600+0000", 2023-11-25T04:23:11.368 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4331000000000001e-05 2023-11-25T04:23:11.368 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.368 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.368 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:11.368 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033606+0000", 2023-11-25T04:23:11.368 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5780000000000004e-06 2023-11-25T04:23:11.368 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.368 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.369 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:11.369 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035747+0000", 2023-11-25T04:23:11.369 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0021406530000000002 2023-11-25T04:23:11.369 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.369 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.369 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:11.369 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035901+0000", 2023-11-25T04:23:11.369 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000153931 2023-11-25T04:23:11.369 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.369 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.369 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:23:11.370 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036193+0000", 2023-11-25T04:23:11.370 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029211299999999999 2023-11-25T04:23:11.370 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.370 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.370 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:11.370 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038655+0000", 2023-11-25T04:23:11.370 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0024621640000000002 2023-11-25T04:23:11.370 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.370 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.370 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:11.370 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043130+0000", 2023-11-25T04:23:11.371 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.004474698 2023-11-25T04:23:11.371 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.371 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.371 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:11.371 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043204+0000", 2023-11-25T04:23:11.371 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4091999999999995e-05 2023-11-25T04:23:11.371 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.371 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.371 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:11.371 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043245+0000", 2023-11-25T04:23:11.371 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0907e-05 2023-11-25T04:23:11.372 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:11.372 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:11.372 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:11.372 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.372 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.372 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:26 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:11.372 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:11.372 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 24.309282839000002, 2023-11-25T04:23:11.372 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011086945000000001, 2023-11-25T04:23:11.372 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:11.372 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:11.372 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:11.373 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.373 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:11.373 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:11.373 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:11.373 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.373 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.373 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:11.373 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:11.373 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:11.373 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.373 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.373 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:11.374 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:11.374 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1699999999999998e-07 2023-11-25T04:23:11.374 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.374 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.374 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:11.374 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033627+0000", 2023-11-25T04:23:11.374 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.163e-05 2023-11-25T04:23:11.374 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.374 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.374 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:11.375 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033636+0000", 2023-11-25T04:23:11.375 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6859999999999995e-06 2023-11-25T04:23:11.375 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.375 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.375 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:11.375 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033642+0000", 2023-11-25T04:23:11.375 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3149999999999997e-06 2023-11-25T04:23:11.375 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.375 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.375 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:11.375 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035414+0000", 2023-11-25T04:23:11.375 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0017716380000000001 2023-11-25T04:23:11.376 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.376 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.376 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:11.376 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035539+0000", 2023-11-25T04:23:11.376 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000125477 2023-11-25T04:23:11.376 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.376 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.376 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:11.376 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035672+0000", 2023-11-25T04:23:11.376 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000133316 2023-11-25T04:23:11.376 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.376 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.377 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:11.377 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038649+0000", 2023-11-25T04:23:11.377 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029764679999999999 2023-11-25T04:23:11.377 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.377 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.377 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:11.377 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044578+0000", 2023-11-25T04:23:11.377 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0059293510000000002 2023-11-25T04:23:11.377 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.377 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.377 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:11.377 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044651+0000", 2023-11-25T04:23:11.377 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2513000000000002e-05 2023-11-25T04:23:11.378 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.378 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.378 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:11.378 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044702+0000", 2023-11-25T04:23:11.378 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1034e-05 2023-11-25T04:23:11.378 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:11.379 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:11.379 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:11.379 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.379 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.379 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:29 3.b 3:d0630e4c:::mds0_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:11.379 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:11.379 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 24.309244919000001, 2023-11-25T04:23:11.379 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010980227, 2023-11-25T04:23:11.379 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:11.379 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:11.380 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:11.380 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.380 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:11.380 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:11.380 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:11.380 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.380 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.380 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:11.380 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:11.380 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:11.380 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.381 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.381 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:11.381 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:11.381 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3499999999999996e-07 2023-11-25T04:23:11.381 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.381 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.381 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:11.381 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033658+0000", 2023-11-25T04:23:11.381 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1280000000000001e-06 2023-11-25T04:23:11.381 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.381 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.381 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:11.382 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033667+0000", 2023-11-25T04:23:11.382 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1430000000000004e-06 2023-11-25T04:23:11.382 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.382 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.382 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:11.382 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033678+0000", 2023-11-25T04:23:11.382 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0906e-05 2023-11-25T04:23:11.382 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.382 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.382 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:11.382 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034456+0000", 2023-11-25T04:23:11.383 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00077833899999999996 2023-11-25T04:23:11.383 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.383 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.383 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:11.383 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034575+0000", 2023-11-25T04:23:11.383 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011926000000000001 2023-11-25T04:23:11.383 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.383 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.383 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:11.383 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035001+0000", 2023-11-25T04:23:11.383 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042550999999999998 2023-11-25T04:23:11.383 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.384 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.384 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:11.384 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.037923+0000", 2023-11-25T04:23:11.384 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029219670000000001 2023-11-25T04:23:11.384 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.384 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.384 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:11.384 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044538+0000", 2023-11-25T04:23:11.384 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0066152349999999997 2023-11-25T04:23:11.384 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.384 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.384 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:11.385 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044594+0000", 2023-11-25T04:23:11.385 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5593000000000001e-05 2023-11-25T04:23:11.385 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.385 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.385 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:11.385 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044633+0000", 2023-11-25T04:23:11.385 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9511000000000002e-05 2023-11-25T04:23:11.385 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:11.385 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:11.385 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:11.385 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.385 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.385 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:34 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:11.386 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:11.386 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 24.308994806000001, 2023-11-25T04:23:11.386 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010920445000000001, 2023-11-25T04:23:11.386 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:11.386 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:11.386 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:11.386 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.386 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:11.386 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:11.386 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:11.386 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.386 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.387 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:11.387 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:11.387 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:11.387 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.387 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.387 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:11.387 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033904+0000", 2023-11-25T04:23:11.387 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6599999999999994e-07 2023-11-25T04:23:11.387 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.387 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.387 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:11.388 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033909+0000", 2023-11-25T04:23:11.388 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5639999999999996e-06 2023-11-25T04:23:11.388 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.388 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.388 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:11.388 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033923+0000", 2023-11-25T04:23:11.388 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3028e-05 2023-11-25T04:23:11.388 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.388 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.388 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:11.388 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034043+0000", 2023-11-25T04:23:11.389 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000120328 2023-11-25T04:23:11.389 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.389 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.389 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:11.389 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035576+0000", 2023-11-25T04:23:11.389 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015333720000000001 2023-11-25T04:23:11.389 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.389 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.389 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:11.389 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035699+0000", 2023-11-25T04:23:11.389 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000123001 2023-11-25T04:23:11.390 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.390 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.390 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:11.390 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044823+0000", 2023-11-25T04:23:11.390 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0091241859999999994 2023-11-25T04:23:11.390 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:11.390 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:11.390 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:11.390 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.390 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.390 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:35 3.1f 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:11.391 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:11.391 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 24.307690601000001, 2023-11-25T04:23:11.391 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096164779999999995, 2023-11-25T04:23:11.391 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:11.391 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:11.391 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:11.391 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.391 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:11.391 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:11.391 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:11.391 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.392 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.392 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:11.392 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:11.392 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:11.392 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.392 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.392 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:11.392 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035208+0000", 2023-11-25T04:23:11.392 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2699999999999998e-07 2023-11-25T04:23:11.392 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.392 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.392 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:11.392 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035213+0000", 2023-11-25T04:23:11.393 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.6070000000000001e-06 2023-11-25T04:23:11.393 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.393 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.393 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:11.393 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035350+0000", 2023-11-25T04:23:11.393 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013693599999999999 2023-11-25T04:23:11.393 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.393 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.393 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:11.393 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035358+0000", 2023-11-25T04:23:11.393 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0099999999999995e-06 2023-11-25T04:23:11.393 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.394 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.394 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:11.394 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036260+0000", 2023-11-25T04:23:11.394 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00090240100000000005 2023-11-25T04:23:11.394 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.394 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.394 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:11.394 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036517+0000", 2023-11-25T04:23:11.394 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00025736299999999998 2023-11-25T04:23:11.394 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.394 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.395 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:11.395 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044824+0000", 2023-11-25T04:23:11.395 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0083063340000000003 2023-11-25T04:23:11.395 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:11.395 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:11.395 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:11.395 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.395 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.395 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:37 3.1e 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:11.395 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:11.395 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 24.307523915000001, 2023-11-25T04:23:11.396 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0078322640000000002, 2023-11-25T04:23:11.396 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:11.396 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:11.396 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:11.396 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.396 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:11.396 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:11.396 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:11.396 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.396 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.396 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:11.396 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:11.397 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:11.397 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.397 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.397 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:11.397 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035375+0000", 2023-11-25T04:23:11.397 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9100000000000003e-07 2023-11-25T04:23:11.397 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.397 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.397 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:11.397 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035379+0000", 2023-11-25T04:23:11.397 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9430000000000002e-06 2023-11-25T04:23:11.398 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.398 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.398 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:11.398 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035388+0000", 2023-11-25T04:23:11.398 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2760000000000006e-06 2023-11-25T04:23:11.398 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.398 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.398 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:11.398 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035396+0000", 2023-11-25T04:23:11.398 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8480000000000001e-06 2023-11-25T04:23:11.398 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.399 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.399 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:11.399 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035452+0000", 2023-11-25T04:23:11.399 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5831000000000001e-05 2023-11-25T04:23:11.399 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.399 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.399 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:11.399 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035934+0000", 2023-11-25T04:23:11.399 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00048265599999999998 2023-11-25T04:23:11.399 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.399 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.399 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:11.400 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043206+0000", 2023-11-25T04:23:11.400 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.007271919 2023-11-25T04:23:11.400 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:11.400 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:11.400 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:11.400 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.400 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.400 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing(3.1a epoch 18/16 rep_tid 3 entries 18'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:36 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:23:11.400 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:11.400 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 24.306248328999999, 2023-11-25T04:23:11.400 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002073414, 2023-11-25T04:23:11.401 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:11.401 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:11.401 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:11.401 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.401 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:11.401 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:11.401 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:11.401 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.401 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.401 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:11.401 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:11.401 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:11.402 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.402 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.402 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:11.402 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036650+0000", 2023-11-25T04:23:11.402 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7300000000000002e-07 2023-11-25T04:23:11.402 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.402 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.402 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:11.402 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036659+0000", 2023-11-25T04:23:11.402 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.7499999999999992e-06 2023-11-25T04:23:11.402 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.403 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.403 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:11.403 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036673+0000", 2023-11-25T04:23:11.403 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4917e-05 2023-11-25T04:23:11.403 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.403 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.403 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:11.403 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036687+0000", 2023-11-25T04:23:11.403 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3883000000000001e-05 2023-11-25T04:23:11.403 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.403 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.403 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:11.404 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036781+0000", 2023-11-25T04:23:11.404 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3219000000000003e-05 2023-11-25T04:23:11.404 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.404 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.404 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:11.404 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038723+0000", 2023-11-25T04:23:11.404 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019422720000000001 2023-11-25T04:23:11.404 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:11.404 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:11.404 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:11.404 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.405 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.405 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:28 3.13 e18/16 3:cd5a64a3:::100.00000000:head v 18'1, mlcod=0'0)", 2023-11-25T04:23:11.405 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:11.405 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 24.302587170999999, 2023-11-25T04:23:11.405 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001850546, 2023-11-25T04:23:11.405 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:11.405 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:11.405 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:11.405 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.405 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:11.405 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:11.405 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:11.406 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.406 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.406 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:11.406 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:11.406 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:11.406 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.406 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.406 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:11.406 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:11.406 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7800000000000002e-07 2023-11-25T04:23:11.406 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.407 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.407 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:11.407 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040331+0000", 2023-11-25T04:23:11.407 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0449999999999999e-05 2023-11-25T04:23:11.407 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.407 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.407 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:11.407 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040339+0000", 2023-11-25T04:23:11.407 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0709999999999995e-06 2023-11-25T04:23:11.407 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.407 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.407 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:11.408 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040347+0000", 2023-11-25T04:23:11.408 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7079999999999997e-06 2023-11-25T04:23:11.408 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.408 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.408 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:11.408 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040410+0000", 2023-11-25T04:23:11.408 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2453000000000007e-05 2023-11-25T04:23:11.408 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.408 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.408 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:11.408 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040422+0000", 2023-11-25T04:23:11.408 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2429000000000001e-05 2023-11-25T04:23:11.409 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.409 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.409 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:11.409 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042129+0000", 2023-11-25T04:23:11.409 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001707266 2023-11-25T04:23:11.409 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.409 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.409 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:11.409 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042161+0000", 2023-11-25T04:23:11.409 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1791000000000001e-05 2023-11-25T04:23:11.409 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:11.410 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:11.410 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:11.410 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.410 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.410 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:27 3.13 e18/16 3:c91d4a1d:::mds0_inotable:head v 18'2, mlcod=0'0)", 2023-11-25T04:23:11.410 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:11.410 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 24.302383647999999, 2023-11-25T04:23:11.410 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0016886430000000001, 2023-11-25T04:23:11.410 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:11.410 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:11.410 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:11.411 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.411 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:11.411 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:11.411 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:11.411 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.411 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.411 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:11.411 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:11.411 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:11.411 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.411 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.411 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:11.412 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:11.412 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3999999999999997e-07 2023-11-25T04:23:11.412 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.412 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.412 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:11.412 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040520+0000", 2023-11-25T04:23:11.412 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2989999999999996e-06 2023-11-25T04:23:11.412 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.412 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.412 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:11.412 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040527+0000", 2023-11-25T04:23:11.413 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5730000000000002e-06 2023-11-25T04:23:11.413 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.413 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.413 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:11.413 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040535+0000", 2023-11-25T04:23:11.413 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8630000000000008e-06 2023-11-25T04:23:11.413 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.413 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.413 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:11.413 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040683+0000", 2023-11-25T04:23:11.413 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000148002 2023-11-25T04:23:11.413 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.414 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.414 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:11.414 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040700+0000", 2023-11-25T04:23:11.414 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7065e-05 2023-11-25T04:23:11.414 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.414 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.414 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:11.414 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042163+0000", 2023-11-25T04:23:11.414 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001462767 2023-11-25T04:23:11.414 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.414 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.414 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:11.415 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042203+0000", 2023-11-25T04:23:11.415 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9734000000000003e-05 2023-11-25T04:23:11.415 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:11.415 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:11.415 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:11.415 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.415 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.415 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:20 3.1c e18/16 3:3ed09add:::607.00000000:head v 18'1, mlcod=0'0)", 2023-11-25T04:23:11.415 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:11.415 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 24.302348606999999, 2023-11-25T04:23:11.415 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015271, 2023-11-25T04:23:11.416 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:11.416 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:11.416 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:11.416 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.416 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:11.416 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:11.416 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:11.416 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.416 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.416 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:11.416 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:11.416 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:11.417 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.417 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.417 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:11.417 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:11.417 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4799999999999999e-07 2023-11-25T04:23:11.417 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.417 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.417 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:11.417 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040561+0000", 2023-11-25T04:23:11.417 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1800000000000001e-05 2023-11-25T04:23:11.417 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.417 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.418 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:11.418 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040568+0000", 2023-11-25T04:23:11.418 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7730000000000002e-06 2023-11-25T04:23:11.418 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.418 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.418 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:11.418 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040587+0000", 2023-11-25T04:23:11.418 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8691000000000001e-05 2023-11-25T04:23:11.418 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.418 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.418 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:11.418 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040638+0000", 2023-11-25T04:23:11.419 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1068e-05 2023-11-25T04:23:11.419 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.419 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.419 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:11.419 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040670+0000", 2023-11-25T04:23:11.419 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2125999999999997e-05 2023-11-25T04:23:11.419 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.419 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.419 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:11.419 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042032+0000", 2023-11-25T04:23:11.419 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001361821 2023-11-25T04:23:11.420 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.420 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.420 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:11.420 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042076+0000", 2023-11-25T04:23:11.420 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4472999999999999e-05 2023-11-25T04:23:11.420 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:11.420 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:11.420 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:11.420 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.420 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.420 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:21 3.1c e18/16 3:3ed09add:::607.00000000:head v 18'2, mlcod=0'0)", 2023-11-25T04:23:11.420 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:11.421 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 24.302113882, 2023-11-25T04:23:11.421 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0013228160000000001, 2023-11-25T04:23:11.421 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:11.421 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:11.421 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:11.421 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.421 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:11.421 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:11.421 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:11.421 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.421 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.421 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:11.422 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:11.422 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:11.422 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.422 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.422 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:11.422 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:11.422 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9299999999999999e-07 2023-11-25T04:23:11.422 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.422 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.422 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:11.422 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040790+0000", 2023-11-25T04:23:11.422 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5770000000000001e-06 2023-11-25T04:23:11.423 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.423 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.423 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:11.423 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040799+0000", 2023-11-25T04:23:11.423 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6439999999999995e-06 2023-11-25T04:23:11.423 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.423 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.423 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:11.423 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040806+0000", 2023-11-25T04:23:11.423 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0180000000000004e-06 2023-11-25T04:23:11.423 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.423 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.424 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:11.424 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040907+0000", 2023-11-25T04:23:11.424 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000100096 2023-11-25T04:23:11.424 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.424 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.424 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:11.424 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040925+0000", 2023-11-25T04:23:11.424 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8536999999999999e-05 2023-11-25T04:23:11.424 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.424 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.424 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:11.425 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042079+0000", 2023-11-25T04:23:11.425 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00115358 2023-11-25T04:23:11.425 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.425 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.425 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:11.425 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042107+0000", 2023-11-25T04:23:11.425 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7971e-05 2023-11-25T04:23:11.425 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:11.425 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:11.425 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:11.425 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.426 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.426 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop_reply(mds.0.3:4 3.1f e18/16 ondisk, result = 0)", 2023-11-25T04:23:11.426 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:11.429 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 24.300774491999999, 2023-11-25T04:23:11.429 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00044092900000000001, 2023-11-25T04:23:11.429 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:11.430 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:11.430 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:11.430 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.430 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:11.430 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:11.430 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:11.430 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.430 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.430 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:11.430 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:11.431 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:11.431 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.431 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.431 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:11.431 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042124+0000", 2023-11-25T04:23:11.431 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1800000000000005e-07 2023-11-25T04:23:11.431 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.431 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.431 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:11.431 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042128+0000", 2023-11-25T04:23:11.431 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2289999999999997e-06 2023-11-25T04:23:11.432 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.432 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.432 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:11.432 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042138+0000", 2023-11-25T04:23:11.432 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9729999999999992e-06 2023-11-25T04:23:11.432 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.432 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.432 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:11.432 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042145+0000", 2023-11-25T04:23:11.432 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.336e-06 2023-11-25T04:23:11.432 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.433 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.433 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:11.433 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042441+0000", 2023-11-25T04:23:11.433 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029650500000000002 2023-11-25T04:23:11.433 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.433 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.433 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:11.433 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042448+0000", 2023-11-25T04:23:11.433 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4100000000000002e-06 2023-11-25T04:23:11.433 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.433 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.433 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:11.434 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042564+0000", 2023-11-25T04:23:11.434 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000115758 2023-11-25T04:23:11.434 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:11.434 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:11.434 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:11.434 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.434 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.434 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop_reply(mds.0.3:26 3.1f e18/16 ondisk, result = 0)", 2023-11-25T04:23:11.434 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.044422+0000", 2023-11-25T04:23:11.434 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 24.298476128000001, 2023-11-25T04:23:11.434 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00028785799999999999, 2023-11-25T04:23:11.435 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:11.435 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:11.435 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:11.435 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.435 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:11.435 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044422+0000", 2023-11-25T04:23:11.435 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:11.435 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.435 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.435 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:11.435 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044422+0000", 2023-11-25T04:23:11.436 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:11.436 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.436 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.436 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:11.436 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044422+0000", 2023-11-25T04:23:11.436 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9000000000000001e-07 2023-11-25T04:23:11.436 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.436 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.436 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:11.436 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044423+0000", 2023-11-25T04:23:11.436 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.669e-06 2023-11-25T04:23:11.437 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.437 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.437 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:11.437 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044427+0000", 2023-11-25T04:23:11.437 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9330000000000003e-06 2023-11-25T04:23:11.437 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.437 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.437 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:11.437 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044442+0000", 2023-11-25T04:23:11.437 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4525999999999999e-05 2023-11-25T04:23:11.437 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.438 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.438 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:11.438 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044566+0000", 2023-11-25T04:23:11.438 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012453099999999999 2023-11-25T04:23:11.438 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.438 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.438 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:11.438 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044573+0000", 2023-11-25T04:23:11.438 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4409999999999998e-06 2023-11-25T04:23:11.438 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.438 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.439 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:11.439 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044709+0000", 2023-11-25T04:23:11.439 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000136568 2023-11-25T04:23:11.439 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:11.439 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:11.439 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:11.439 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.439 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.439 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing_reply(3.1f epoch 18/16 rep_tid 8 lcod 18'4)", 2023-11-25T04:23:11.439 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:11.439 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 24.298449080000001, 2023-11-25T04:23:11.440 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00038154699999999998, 2023-11-25T04:23:11.440 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:11.440 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:11.440 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:11.440 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.440 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:11.440 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:11.440 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:11.440 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.440 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.441 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:11.441 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:11.441 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:11.441 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.441 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.441 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:11.441 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:11.441 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2700000000000001e-07 2023-11-25T04:23:11.441 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.441 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.441 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:11.441 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044451+0000", 2023-11-25T04:23:11.442 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7460000000000001e-06 2023-11-25T04:23:11.442 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.442 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.442 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:11.442 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044455+0000", 2023-11-25T04:23:11.442 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5809999999999998e-06 2023-11-25T04:23:11.442 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.442 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.442 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:11.442 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044458+0000", 2023-11-25T04:23:11.443 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0170000000000001e-06 2023-11-25T04:23:11.443 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.443 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.443 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:11.443 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044734+0000", 2023-11-25T04:23:11.443 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00027582799999999999 2023-11-25T04:23:11.443 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.443 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.443 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:11.443 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044830+0000", 2023-11-25T04:23:11.443 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6248000000000004e-05 2023-11-25T04:23:11.444 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:11.444 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:11.444 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:11.444 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.444 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.444 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing_reply(3.b epoch 18/16 rep_tid 5 lcod 18'2)", 2023-11-25T04:23:11.444 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:11.444 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 24.298412419000002, 2023-11-25T04:23:11.444 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000344488, 2023-11-25T04:23:11.444 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:11.444 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:11.445 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:11.445 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.445 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:11.445 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:11.445 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:11.445 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.445 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.445 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:11.445 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:11.445 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:11.445 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.446 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.446 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:11.446 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:11.446 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.55e-07 2023-11-25T04:23:11.446 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.446 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.446 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:11.446 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044487+0000", 2023-11-25T04:23:11.446 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2470000000000001e-06 2023-11-25T04:23:11.446 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.446 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.447 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:11.447 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044491+0000", 2023-11-25T04:23:11.447 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9659999999999998e-06 2023-11-25T04:23:11.447 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.447 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.447 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:11.447 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044494+0000", 2023-11-25T04:23:11.447 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2310000000000001e-06 2023-11-25T04:23:11.447 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.447 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.447 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:11.447 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044756+0000", 2023-11-25T04:23:11.448 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00026172800000000003 2023-11-25T04:23:11.448 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.448 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.448 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:11.448 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044830+0000", 2023-11-25T04:23:11.448 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4160999999999997e-05 2023-11-25T04:23:11.448 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:11.448 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:11.448 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:11.448 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.448 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.448 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:39 3.6 3:654134d2:::mds0_openfiles.0:head [omap-clear,omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2023-11-25T04:23:11.449 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:11.449 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 19.238586377000001, 2023-11-25T04:23:11.449 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001788658, 2023-11-25T04:23:11.449 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:11.449 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:11.449 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:11.449 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.449 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:11.449 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:11.449 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:11.449 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.449 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.450 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:11.450 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:11.450 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:11.450 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.450 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.450 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:11.450 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104314+0000", 2023-11-25T04:23:11.450 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3099999999999999e-06 2023-11-25T04:23:11.450 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.450 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.451 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:11.451 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104325+0000", 2023-11-25T04:23:11.451 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1511999999999999e-05 2023-11-25T04:23:11.451 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.451 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.451 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:11.451 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104362+0000", 2023-11-25T04:23:11.451 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6903000000000002e-05 2023-11-25T04:23:11.451 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.451 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.451 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:11.452 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104387+0000", 2023-11-25T04:23:11.452 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4766e-05 2023-11-25T04:23:11.452 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.452 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.452 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:11.452 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104465+0000", 2023-11-25T04:23:11.452 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8119000000000002e-05 2023-11-25T04:23:11.452 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.452 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.452 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:11.452 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104557+0000", 2023-11-25T04:23:11.453 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2496000000000003e-05 2023-11-25T04:23:11.453 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.453 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.453 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:11.453 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104646+0000", 2023-11-25T04:23:11.453 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8818000000000002e-05 2023-11-25T04:23:11.453 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.453 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.453 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:11.453 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105201+0000", 2023-11-25T04:23:11.453 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00055506000000000002 2023-11-25T04:23:11.453 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.454 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.454 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:11.454 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105994+0000", 2023-11-25T04:23:11.454 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00079294700000000001 2023-11-25T04:23:11.454 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.454 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.454 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:11.454 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.106055+0000", 2023-11-25T04:23:11.454 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0566000000000003e-05 2023-11-25T04:23:11.454 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.454 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.454 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:11.455 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.106100+0000", 2023-11-25T04:23:11.455 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5160999999999997e-05 2023-11-25T04:23:11.455 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:11.455 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:11.455 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:11.455 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.455 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.455 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:43 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2023-11-25T04:23:11.455 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:11.455 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.2383613809999998, 2023-11-25T04:23:11.455 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001751166, 2023-11-25T04:23:11.455 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:11.456 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:11.456 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:11.456 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.456 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:11.456 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:11.456 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:11.456 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.456 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.456 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:11.456 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:11.456 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:11.457 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.457 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.457 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:11.457 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104539+0000", 2023-11-25T04:23:11.457 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9529999999999999e-06 2023-11-25T04:23:11.457 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.457 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.457 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:11.457 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104557+0000", 2023-11-25T04:23:11.457 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7464000000000001e-05 2023-11-25T04:23:11.457 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.458 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.458 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:11.458 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104590+0000", 2023-11-25T04:23:11.458 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2966000000000003e-05 2023-11-25T04:23:11.458 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.458 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.458 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:11.458 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104608+0000", 2023-11-25T04:23:11.458 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8193e-05 2023-11-25T04:23:11.458 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.458 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.458 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:11.459 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104767+0000", 2023-11-25T04:23:11.459 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015882 2023-11-25T04:23:11.459 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.459 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.459 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:11.459 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104909+0000", 2023-11-25T04:23:11.459 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014244999999999999 2023-11-25T04:23:11.459 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.459 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.459 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:11.459 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.105059+0000", 2023-11-25T04:23:11.459 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014964399999999999 2023-11-25T04:23:11.460 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.460 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.460 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:11.460 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.105701+0000", 2023-11-25T04:23:11.460 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00064201099999999995 2023-11-25T04:23:11.460 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.460 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.460 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:11.460 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106233+0000", 2023-11-25T04:23:11.460 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00053178500000000001 2023-11-25T04:23:11.460 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.461 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.461 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:11.461 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106265+0000", 2023-11-25T04:23:11.461 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1986000000000002e-05 2023-11-25T04:23:11.461 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:11.461 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:11.461 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:11.461 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106288+0000", 2023-11-25T04:23:11.461 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2894000000000001e-05 2023-11-25T04:23:11.461 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:11.461 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:11.461 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:11.462 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:11.462 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:11.462 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:11.463 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:11.462+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:11.464 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:23:11.564 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:11.563+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:11.570 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:11.570 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:23:11.570 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2023-11-25T04:23:11.570 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:11.579 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:23:11.666 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:11.664+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:11.724 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:11.724 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:23:11.724 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2023-11-25T04:23:11.724 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2023-11-25T04:23:11.724 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:11.733 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:23:11.767 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:11.765+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:11.868 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:11.867+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:11.879 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:11.879 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2023-11-25T04:23:11.880 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2023-11-25T04:23:11.880 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2023-11-25T04:23:11.880 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.880 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:155 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'109, mlcod=11'105)", 2023-11-25T04:23:11.880 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:11.880 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 34.092450978000002, 2023-11-25T04:23:11.880 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061719019999999999, 2023-11-25T04:23:11.880 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:11.880 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:11.880 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:11.881 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.881 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:11.881 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:11.881 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:11.881 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.881 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.881 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:11.881 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:11.881 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:11.881 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.881 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.881 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:11.881 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784884+0000", 2023-11-25T04:23:11.882 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3099999999999999e-07 2023-11-25T04:23:11.882 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.882 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.882 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:11.882 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784942+0000", 2023-11-25T04:23:11.882 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8570000000000003e-05 2023-11-25T04:23:11.882 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.882 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.882 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:11.882 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784949+0000", 2023-11-25T04:23:11.883 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0310000000000001e-06 2023-11-25T04:23:11.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.883 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.883 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:11.883 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784953+0000", 2023-11-25T04:23:11.883 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6320000000000001e-06 2023-11-25T04:23:11.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.883 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.883 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:11.883 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784993+0000", 2023-11-25T04:23:11.883 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9737000000000003e-05 2023-11-25T04:23:11.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.884 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.884 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:11.884 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785011+0000", 2023-11-25T04:23:11.884 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8349999999999999e-05 2023-11-25T04:23:11.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.884 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.884 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:11.884 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791024+0000", 2023-11-25T04:23:11.884 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006012997 2023-11-25T04:23:11.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.884 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.884 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:11.885 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791055+0000", 2023-11-25T04:23:11.885 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1254000000000003e-05 2023-11-25T04:23:11.885 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:11.885 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:11.885 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:11.885 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.885 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.885 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:156 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'110, mlcod=11'105)", 2023-11-25T04:23:11.885 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:11.885 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 34.091864929000003, 2023-11-25T04:23:11.885 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005616116, 2023-11-25T04:23:11.885 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:11.886 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:11.886 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:11.886 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.886 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:11.886 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:11.886 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:11.886 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.886 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.886 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:11.886 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:11.886 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:11.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.887 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.887 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:11.887 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785470+0000", 2023-11-25T04:23:11.887 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0699999999999998e-07 2023-11-25T04:23:11.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.887 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.887 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:11.887 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785474+0000", 2023-11-25T04:23:11.887 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.048e-06 2023-11-25T04:23:11.888 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.888 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.888 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:11.888 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785480+0000", 2023-11-25T04:23:11.888 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7729999999999996e-06 2023-11-25T04:23:11.888 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.888 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.888 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:11.888 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785483+0000", 2023-11-25T04:23:11.888 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1970000000000001e-06 2023-11-25T04:23:11.888 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.889 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.889 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:11.889 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787936+0000", 2023-11-25T04:23:11.889 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024529040000000001 2023-11-25T04:23:11.889 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.889 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.889 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:11.889 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787947+0000", 2023-11-25T04:23:11.889 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1669e-05 2023-11-25T04:23:11.889 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.889 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.889 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:11.890 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791057+0000", 2023-11-25T04:23:11.890 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031095200000000002 2023-11-25T04:23:11.890 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.890 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.890 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:11.890 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791086+0000", 2023-11-25T04:23:11.890 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8697999999999999e-05 2023-11-25T04:23:11.890 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:11.890 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:11.890 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:11.890 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.890 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.891 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:157 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'111, mlcod=11'106)", 2023-11-25T04:23:11.891 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:11.891 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 34.091272371000002, 2023-11-25T04:23:11.891 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050590280000000001, 2023-11-25T04:23:11.891 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:11.891 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:11.891 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:11.891 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.891 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:11.892 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:11.892 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:11.892 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.892 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.892 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:11.892 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:11.892 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:11.892 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.892 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.892 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:11.892 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786063+0000", 2023-11-25T04:23:11.892 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2600000000000002e-07 2023-11-25T04:23:11.893 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.893 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.893 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:11.893 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786069+0000", 2023-11-25T04:23:11.893 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8839999999999999e-06 2023-11-25T04:23:11.893 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.893 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.893 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:11.893 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786079+0000", 2023-11-25T04:23:11.893 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0124e-05 2023-11-25T04:23:11.893 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.893 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.894 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:11.894 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786085+0000", 2023-11-25T04:23:11.894 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6660000000000002e-06 2023-11-25T04:23:11.894 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.894 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.894 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:11.894 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788160+0000", 2023-11-25T04:23:11.894 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020747669999999999 2023-11-25T04:23:11.894 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:11.895 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788180+0000", 2023-11-25T04:23:11.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0208999999999999e-05 2023-11-25T04:23:11.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:11.895 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791087+0000", 2023-11-25T04:23:11.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029064630000000002 2023-11-25T04:23:11.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.896 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:11.896 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791121+0000", 2023-11-25T04:23:11.896 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4289000000000003e-05 2023-11-25T04:23:11.896 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:11.896 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:11.896 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:11.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.896 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:158 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'112, mlcod=11'106)", 2023-11-25T04:23:11.896 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:11.896 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 34.090855978, 2023-11-25T04:23:11.896 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046663030000000001, 2023-11-25T04:23:11.897 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:11.897 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:11.897 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:11.897 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.897 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:11.897 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:11.897 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:11.897 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.897 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.897 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:11.897 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:11.897 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:11.898 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.898 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:11.898 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786479+0000", 2023-11-25T04:23:11.898 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2799999999999996e-07 2023-11-25T04:23:11.898 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.898 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:11.898 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786498+0000", 2023-11-25T04:23:11.898 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9069e-05 2023-11-25T04:23:11.898 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.899 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:11.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786508+0000", 2023-11-25T04:23:11.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5810000000000003e-06 2023-11-25T04:23:11.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.899 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:11.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786512+0000", 2023-11-25T04:23:11.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4889999999999997e-06 2023-11-25T04:23:11.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.899 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:11.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788438+0000", 2023-11-25T04:23:11.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019257059999999999 2023-11-25T04:23:11.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.900 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:11.900 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788449+0000", 2023-11-25T04:23:11.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1419e-05 2023-11-25T04:23:11.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.900 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:11.900 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791122+0000", 2023-11-25T04:23:11.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002672822 2023-11-25T04:23:11.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.901 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:11.901 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791145+0000", 2023-11-25T04:23:11.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2589e-05 2023-11-25T04:23:11.901 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:11.901 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:11.901 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:11.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.901 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:206 1.0 e12/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 12'152, mlcod=12'148)", 2023-11-25T04:23:11.901 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:11.901 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 33.579087508000001, 2023-11-25T04:23:11.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046728189999999999, 2023-11-25T04:23:11.902 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:11.902 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:11.902 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:11.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.902 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:11.902 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:11.902 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:11.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.902 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:11.902 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:11.902 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:11.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.903 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.903 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:11.903 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298248+0000", 2023-11-25T04:23:11.903 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3600000000000003e-07 2023-11-25T04:23:11.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.903 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.903 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:11.903 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298278+0000", 2023-11-25T04:23:11.903 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0474e-05 2023-11-25T04:23:11.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.903 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.904 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:11.904 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298288+0000", 2023-11-25T04:23:11.904 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0156000000000001e-05 2023-11-25T04:23:11.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.904 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:11.904 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298293+0000", 2023-11-25T04:23:11.904 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7400000000000004e-06 2023-11-25T04:23:11.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.904 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:11.904 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298333+0000", 2023-11-25T04:23:11.904 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9666000000000003e-05 2023-11-25T04:23:11.905 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.905 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.905 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:11.905 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298351+0000", 2023-11-25T04:23:11.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8544999999999999e-05 2023-11-25T04:23:11.905 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.905 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.905 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:11.905 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302891+0000", 2023-11-25T04:23:11.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045398519999999996 2023-11-25T04:23:11.905 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.905 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.906 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:11.906 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302920+0000", 2023-11-25T04:23:11.906 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8750000000000001e-05 2023-11-25T04:23:11.906 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:11.906 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:11.906 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:11.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.906 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.906 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:14 3.15 3:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:11.906 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:11.906 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 24.843953151000001, 2023-11-25T04:23:11.906 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010990437, 2023-11-25T04:23:11.907 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:11.907 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:11.907 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:11.907 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.907 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:11.907 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:11.907 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:11.907 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.909 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.909 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:11.909 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:11.909 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:11.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.909 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.909 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:11.909 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033383+0000", 2023-11-25T04:23:11.909 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9870000000000002e-06 2023-11-25T04:23:11.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.910 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.910 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:11.910 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033398+0000", 2023-11-25T04:23:11.910 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4569999999999999e-05 2023-11-25T04:23:11.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.910 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.910 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:11.910 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033411+0000", 2023-11-25T04:23:11.910 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3543999999999999e-05 2023-11-25T04:23:11.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.910 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.911 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:11.911 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033423+0000", 2023-11-25T04:23:11.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1905999999999999e-05 2023-11-25T04:23:11.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.911 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:11.911 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034433+0000", 2023-11-25T04:23:11.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00101022 2023-11-25T04:23:11.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.911 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:11.911 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034539+0000", 2023-11-25T04:23:11.912 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010597199999999999 2023-11-25T04:23:11.912 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.912 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:11.912 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034667+0000", 2023-11-25T04:23:11.912 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012765500000000001 2023-11-25T04:23:11.912 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.912 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:11.912 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037803+0000", 2023-11-25T04:23:11.912 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031362220000000001 2023-11-25T04:23:11.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.913 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.913 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:11.913 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:23:11.913 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0064964669999999997 2023-11-25T04:23:11.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.913 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.913 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:11.913 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044331+0000", 2023-11-25T04:23:11.913 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1544999999999997e-05 2023-11-25T04:23:11.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.913 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.913 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:11.914 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044372+0000", 2023-11-25T04:23:11.914 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0349000000000003e-05 2023-11-25T04:23:11.914 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:11.914 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:11.914 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:11.914 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.914 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.914 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:15 3.15 3:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:11.914 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:11.914 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 24.843293501000002, 2023-11-25T04:23:11.914 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010464362, 2023-11-25T04:23:11.915 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:11.915 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:11.915 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:11.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.915 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:11.915 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:11.915 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:11.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.915 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:11.915 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:11.915 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:11.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.916 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:11.916 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034042+0000", 2023-11-25T04:23:11.916 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0999999999999997e-07 2023-11-25T04:23:11.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.916 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:11.916 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034048+0000", 2023-11-25T04:23:11.916 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0390000000000001e-06 2023-11-25T04:23:11.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.917 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:11.917 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034059+0000", 2023-11-25T04:23:11.917 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1182999999999999e-05 2023-11-25T04:23:11.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.917 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:11.917 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034067+0000", 2023-11-25T04:23:11.917 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1559999999999995e-06 2023-11-25T04:23:11.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.917 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:11.917 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035005+0000", 2023-11-25T04:23:11.918 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00093747399999999999 2023-11-25T04:23:11.918 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.918 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.918 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:11.918 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035077+0000", 2023-11-25T04:23:11.918 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2747e-05 2023-11-25T04:23:11.918 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.918 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.918 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:11.918 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035219+0000", 2023-11-25T04:23:11.918 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000141877 2023-11-25T04:23:11.918 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.919 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.919 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:11.919 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037810+0000", 2023-11-25T04:23:11.919 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002590752 2023-11-25T04:23:11.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.919 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.919 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:11.919 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044437+0000", 2023-11-25T04:23:11.919 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066270110000000004 2023-11-25T04:23:11.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.919 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.919 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:11.920 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044481+0000", 2023-11-25T04:23:11.920 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4106000000000001e-05 2023-11-25T04:23:11.920 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.920 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.920 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:11.920 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044505+0000", 2023-11-25T04:23:11.920 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4207e-05 2023-11-25T04:23:11.920 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:11.920 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:11.920 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:11.920 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.920 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.921 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:16 3.1d 3:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:11.921 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:11.921 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 24.843251652999999, 2023-11-25T04:23:11.921 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010413923, 2023-11-25T04:23:11.921 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:11.921 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:11.921 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:11.921 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.921 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:11.921 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:11.921 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:11.921 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.921 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.922 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:11.922 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:11.922 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:11.922 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.922 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.922 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:11.922 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:11.922 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4499999999999997e-07 2023-11-25T04:23:11.922 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.922 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.922 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:11.923 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034087+0000", 2023-11-25T04:23:11.923 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9770000000000002e-06 2023-11-25T04:23:11.923 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.923 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.923 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:11.923 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034094+0000", 2023-11-25T04:23:11.923 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8809999999999998e-06 2023-11-25T04:23:11.923 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.923 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.923 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:11.923 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034100+0000", 2023-11-25T04:23:11.923 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8440000000000003e-06 2023-11-25T04:23:11.924 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.924 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.924 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:11.924 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034993+0000", 2023-11-25T04:23:11.924 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00089336799999999996 2023-11-25T04:23:11.924 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.924 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.924 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:11.924 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035099+0000", 2023-11-25T04:23:11.924 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106037 2023-11-25T04:23:11.924 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.924 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.925 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:11.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035276+0000", 2023-11-25T04:23:11.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017631799999999999 2023-11-25T04:23:11.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.925 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:11.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037815+0000", 2023-11-25T04:23:11.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025395959999999999 2023-11-25T04:23:11.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.925 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:11.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044420+0000", 2023-11-25T04:23:11.926 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066045749999999997 2023-11-25T04:23:11.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.926 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.926 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:11.926 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044463+0000", 2023-11-25T04:23:11.926 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3205999999999999e-05 2023-11-25T04:23:11.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.926 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.926 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:11.926 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044497+0000", 2023-11-25T04:23:11.926 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3476000000000003e-05 2023-11-25T04:23:11.926 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:11.927 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:11.927 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:11.927 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.927 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.927 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:17 3.1d 3:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:11.927 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:11.927 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 24.843226443999999, 2023-11-25T04:23:11.927 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01127356, 2023-11-25T04:23:11.927 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:11.927 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:11.927 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:11.927 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.927 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:11.928 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:11.928 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:11.928 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.928 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.928 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:11.928 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:11.928 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:11.928 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.928 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.928 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:11.928 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:11.928 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1100000000000001e-07 2023-11-25T04:23:11.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.929 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:11.929 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034112+0000", 2023-11-25T04:23:11.929 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8600000000000003e-06 2023-11-25T04:23:11.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.929 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:11.929 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034120+0000", 2023-11-25T04:23:11.929 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7700000000000001e-06 2023-11-25T04:23:11.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.930 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.930 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:11.930 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034127+0000", 2023-11-25T04:23:11.930 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0999999999999998e-06 2023-11-25T04:23:11.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.930 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.930 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:11.930 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035601+0000", 2023-11-25T04:23:11.930 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014739410000000001 2023-11-25T04:23:11.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.931 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.931 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:11.931 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035685+0000", 2023-11-25T04:23:11.931 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3677999999999994e-05 2023-11-25T04:23:11.931 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.931 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.931 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:11.931 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035827+0000", 2023-11-25T04:23:11.931 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014232599999999999 2023-11-25T04:23:11.931 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.931 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.931 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:11.931 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038725+0000", 2023-11-25T04:23:11.932 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028982779999999998 2023-11-25T04:23:11.932 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.932 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.932 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:11.932 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045323+0000", 2023-11-25T04:23:11.932 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0065980780000000003 2023-11-25T04:23:11.932 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.932 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.932 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:11.932 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045354+0000", 2023-11-25T04:23:11.932 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0463e-05 2023-11-25T04:23:11.932 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.933 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.933 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:11.933 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045381+0000", 2023-11-25T04:23:11.933 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7654999999999999e-05 2023-11-25T04:23:11.933 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:11.933 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:11.933 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:11.933 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.933 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.933 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:22 3.e 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:11.933 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:11.933 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 24.843200030999999, 2023-11-25T04:23:11.934 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010133422, 2023-11-25T04:23:11.934 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:11.934 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:11.934 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:11.934 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.934 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:11.934 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:11.934 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:11.934 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.934 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.934 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:11.935 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:11.935 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:11.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.935 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.935 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:11.935 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034135+0000", 2023-11-25T04:23:11.935 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-07 2023-11-25T04:23:11.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.935 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.935 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:11.935 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034138+0000", 2023-11-25T04:23:11.935 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6009999999999999e-06 2023-11-25T04:23:11.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.936 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:11.936 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034145+0000", 2023-11-25T04:23:11.936 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1169999999999998e-06 2023-11-25T04:23:11.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.936 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:11.936 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034150+0000", 2023-11-25T04:23:11.936 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9080000000000003e-06 2023-11-25T04:23:11.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.936 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:11.937 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034209+0000", 2023-11-25T04:23:11.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8149000000000002e-05 2023-11-25T04:23:11.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.937 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.937 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:11.937 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034415+0000", 2023-11-25T04:23:11.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020637699999999999 2023-11-25T04:23:11.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.937 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.937 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:11.937 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034584+0000", 2023-11-25T04:23:11.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016870399999999999 2023-11-25T04:23:11.938 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.938 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.938 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:11.938 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.036979+0000", 2023-11-25T04:23:11.938 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023957509999999998 2023-11-25T04:23:11.938 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.938 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.938 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:11.938 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044161+0000", 2023-11-25T04:23:11.938 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0071811590000000003 2023-11-25T04:23:11.938 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.938 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.938 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:11.939 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044224+0000", 2023-11-25T04:23:11.939 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3454000000000006e-05 2023-11-25T04:23:11.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.939 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:11.939 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044268+0000", 2023-11-25T04:23:11.939 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3761999999999999e-05 2023-11-25T04:23:11.939 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:11.939 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:11.939 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:11.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.939 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:23 3.e 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:11.940 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:11.940 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 24.843168622, 2023-11-25T04:23:11.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010281676, 2023-11-25T04:23:11.940 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:11.940 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:11.940 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:11.940 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.940 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:11.940 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:11.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:11.940 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.940 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.940 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:11.941 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:11.941 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:11.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.941 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:11.941 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:11.941 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5799999999999999e-07 2023-11-25T04:23:11.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.941 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:11.941 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034169+0000", 2023-11-25T04:23:11.941 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1879999999999998e-06 2023-11-25T04:23:11.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.942 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.942 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:11.942 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034177+0000", 2023-11-25T04:23:11.942 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7020000000000001e-06 2023-11-25T04:23:11.942 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.942 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.942 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:11.942 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034184+0000", 2023-11-25T04:23:11.942 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3910000000000003e-06 2023-11-25T04:23:11.942 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.942 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.942 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:11.942 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034924+0000", 2023-11-25T04:23:11.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00074032999999999998 2023-11-25T04:23:11.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.943 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:11.943 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034991+0000", 2023-11-25T04:23:11.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7347000000000004e-05 2023-11-25T04:23:11.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.943 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:11.943 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035096+0000", 2023-11-25T04:23:11.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010488 2023-11-25T04:23:11.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:11.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037588+0000", 2023-11-25T04:23:11.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024918309999999999 2023-11-25T04:23:11.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:11.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044367+0000", 2023-11-25T04:23:11.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067787589999999997 2023-11-25T04:23:11.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:11.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044415+0000", 2023-11-25T04:23:11.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7803000000000003e-05 2023-11-25T04:23:11.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:11.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044447+0000", 2023-11-25T04:23:11.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2886999999999999e-05 2023-11-25T04:23:11.945 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:11.945 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:11.945 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:11.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.945 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:30 3.15 3:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:11.946 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:11.946 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 24.843110182, 2023-11-25T04:23:11.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010365070000000001, 2023-11-25T04:23:11.946 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:11.946 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:11.946 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:11.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.946 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:11.946 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:11.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:11.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.947 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:11.947 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:11.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:11.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.947 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:11.947 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034225+0000", 2023-11-25T04:23:11.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7400000000000002e-07 2023-11-25T04:23:11.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.947 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:11.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034229+0000", 2023-11-25T04:23:11.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3100000000000002e-06 2023-11-25T04:23:11.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.948 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:11.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034239+0000", 2023-11-25T04:23:11.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9140000000000003e-06 2023-11-25T04:23:11.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.948 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:11.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034246+0000", 2023-11-25T04:23:11.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0149999999999998e-06 2023-11-25T04:23:11.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:11.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035572+0000", 2023-11-25T04:23:11.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001325655 2023-11-25T04:23:11.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:11.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035730+0000", 2023-11-25T04:23:11.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015827899999999999 2023-11-25T04:23:11.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:11.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035857+0000", 2023-11-25T04:23:11.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012644699999999999 2023-11-25T04:23:11.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:11.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038737+0000", 2023-11-25T04:23:11.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002880077 2023-11-25T04:23:11.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:11.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044540+0000", 2023-11-25T04:23:11.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058034239999999997 2023-11-25T04:23:11.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:11.951 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044569+0000", 2023-11-25T04:23:11.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8388999999999999e-05 2023-11-25T04:23:11.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:11.951 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044589+0000", 2023-11-25T04:23:11.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0686000000000002e-05 2023-11-25T04:23:11.951 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:11.951 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:11.951 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:11.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.952 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:33 3.1b 3:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:11.952 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:11.952 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 24.843064684000002, 2023-11-25T04:23:11.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099754049999999997, 2023-11-25T04:23:11.952 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:11.952 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:23:11.952 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:11.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.952 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:11.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:11.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:11.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:11.953 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:11.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:11.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:11.953 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:11.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4799999999999998e-07 2023-11-25T04:23:11.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:11.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034273+0000", 2023-11-25T04:23:11.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7819999999999998e-06 2023-11-25T04:23:11.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.954 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:11.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034285+0000", 2023-11-25T04:23:11.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2058e-05 2023-11-25T04:23:11.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.954 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:11.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034291+0000", 2023-11-25T04:23:11.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0680000000000002e-06 2023-11-25T04:23:11.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:11.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034370+0000", 2023-11-25T04:23:11.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8452e-05 2023-11-25T04:23:11.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:11.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034477+0000", 2023-11-25T04:23:11.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107471 2023-11-25T04:23:11.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:11.956 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044245+0000", 2023-11-25T04:23:11.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0097679259999999997 2023-11-25T04:23:11.956 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:11.956 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:11.956 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:11.956 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.956 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.956 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:36 3.1a 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:11.956 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:11.956 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 24.843018011000002, 2023-11-25T04:23:11.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058179099999999999, 2023-11-25T04:23:11.956 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:11.956 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:23:11.957 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:11.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.957 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:11.957 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:11.957 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:11.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.957 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:11.957 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:11.957 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:11.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:11.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034317+0000", 2023-11-25T04:23:11.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1999999999999999e-07 2023-11-25T04:23:11.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:11.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034321+0000", 2023-11-25T04:23:11.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.233e-06 2023-11-25T04:23:11.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:11.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034337+0000", 2023-11-25T04:23:11.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5308999999999999e-05 2023-11-25T04:23:11.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.959 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:11.959 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034354+0000", 2023-11-25T04:23:11.959 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7411999999999999e-05 2023-11-25T04:23:11.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.959 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:11.959 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034465+0000", 2023-11-25T04:23:11.959 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011154699999999999 2023-11-25T04:23:11.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.959 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:11.960 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034560+0000", 2023-11-25T04:23:11.960 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4186999999999998e-05 2023-11-25T04:23:11.960 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.960 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.960 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:11.960 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.040134+0000", 2023-11-25T04:23:11.960 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055746019999999997 2023-11-25T04:23:11.960 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:11.960 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:11.960 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:11.960 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.961 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:9 3.10 e18/16 3:09401a81:::601.00000000:head v 18'4, mlcod=0'0)", 2023-11-25T04:23:11.961 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:11.961 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 24.834123702999999, 2023-11-25T04:23:11.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061641020000000003, 2023-11-25T04:23:11.961 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:11.961 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:11.961 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:11.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.961 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:11.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:11.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:11.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:11.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:11.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:11.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:11.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:11.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5300000000000001e-07 2023-11-25T04:23:11.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:11.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043222+0000", 2023-11-25T04:23:11.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.082e-05 2023-11-25T04:23:11.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:11.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043228+0000", 2023-11-25T04:23:11.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6699999999999997e-06 2023-11-25T04:23:11.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:11.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043236+0000", 2023-11-25T04:23:11.964 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8189999999999992e-06 2023-11-25T04:23:11.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.964 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.964 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:11.964 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043507+0000", 2023-11-25T04:23:11.964 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000271012 2023-11-25T04:23:11.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.964 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.964 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:11.964 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043530+0000", 2023-11-25T04:23:11.964 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2659000000000001e-05 2023-11-25T04:23:11.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.964 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.965 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:11.965 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049321+0000", 2023-11-25T04:23:11.965 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057915129999999999 2023-11-25T04:23:11.965 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.965 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.965 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:11.965 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049375+0000", 2023-11-25T04:23:11.965 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3455999999999998e-05 2023-11-25T04:23:11.965 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:11.965 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:11.965 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:11.965 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.966 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:1 3.14 e18/16 3:292cf221:::200.00000000:head v 18'3, mlcod=0'0)", 2023-11-25T04:23:11.966 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:11.966 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 24.834086989999999, 2023-11-25T04:23:11.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0060983599999999997, 2023-11-25T04:23:11.966 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:11.966 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:11.966 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:11.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:11.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:11.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:11.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.967 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:11.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:11.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:11.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.967 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:11.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043248+0000", 2023-11-25T04:23:11.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9399999999999995e-07 2023-11-25T04:23:11.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.967 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:11.968 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043252+0000", 2023-11-25T04:23:11.968 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1799999999999998e-06 2023-11-25T04:23:11.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.968 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:11.968 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043259+0000", 2023-11-25T04:23:11.968 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0049999999999998e-06 2023-11-25T04:23:11.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.968 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:11.968 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043265+0000", 2023-11-25T04:23:11.968 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0859999999999998e-06 2023-11-25T04:23:11.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.969 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:11.969 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043311+0000", 2023-11-25T04:23:11.969 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6082999999999999e-05 2023-11-25T04:23:11.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.969 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:11.969 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043330+0000", 2023-11-25T04:23:11.969 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8556e-05 2023-11-25T04:23:11.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.969 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:11.969 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049303+0000", 2023-11-25T04:23:11.970 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059733470000000004 2023-11-25T04:23:11.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.970 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:11.970 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049346+0000", 2023-11-25T04:23:11.970 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2608999999999998e-05 2023-11-25T04:23:11.970 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:11.970 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:11.970 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:11.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.970 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:3 3.14 e18/16 3:287034c4:::1.00000000.inode:head v 18'4, mlcod=0'0)", 2023-11-25T04:23:11.971 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:11.971 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 24.833570483999999, 2023-11-25T04:23:11.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057040090000000003, 2023-11-25T04:23:11.971 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:11.971 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:11.971 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:11.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.971 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:11.971 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:11.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:11.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.971 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:11.972 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:11.972 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:11.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:11.972 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:11.972 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9799999999999999e-07 2023-11-25T04:23:11.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:11.972 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043785+0000", 2023-11-25T04:23:11.972 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0721999999999999e-05 2023-11-25T04:23:11.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.973 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.973 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:11.973 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043793+0000", 2023-11-25T04:23:11.973 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5839999999999995e-06 2023-11-25T04:23:11.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.973 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.973 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:11.973 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043803+0000", 2023-11-25T04:23:11.973 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0729999999999994e-06 2023-11-25T04:23:11.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.973 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.973 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:11.973 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043843+0000", 2023-11-25T04:23:11.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0747999999999997e-05 2023-11-25T04:23:11.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.974 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:11.974 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043856+0000", 2023-11-25T04:23:11.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2697000000000001e-05 2023-11-25T04:23:11.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.974 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:11.974 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049426+0000", 2023-11-25T04:23:11.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055704409999999998 2023-11-25T04:23:11.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:11.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049468+0000", 2023-11-25T04:23:11.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1446000000000003e-05 2023-11-25T04:23:11.975 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:11.975 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:11.975 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:11.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.975 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "pg_update_log_missing(3.14 epoch 18/16 rep_tid 171 entries 18'5 (0'0) error 3:2e2fa760:::200.00000001:head by mds.0.3:32 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:23:11.975 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:11.975 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 24.833409521, 2023-11-25T04:23:11.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005556938, 2023-11-25T04:23:11.975 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:11.976 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2023-11-25T04:23:11.976 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:11.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.976 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:11.976 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:11.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:11.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.976 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:11.976 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:11.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:11.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.977 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:11.977 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:11.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.03e-07 2023-11-25T04:23:11.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.977 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:11.977 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043931+0000", 2023-11-25T04:23:11.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1079999999999998e-06 2023-11-25T04:23:11.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.977 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:11.977 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043944+0000", 2023-11-25T04:23:11.978 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2852e-05 2023-11-25T04:23:11.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.978 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.978 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:11.978 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043955+0000", 2023-11-25T04:23:11.978 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1037e-05 2023-11-25T04:23:11.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.978 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.978 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:11.978 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044088+0000", 2023-11-25T04:23:11.978 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013269599999999999 2023-11-25T04:23:11.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:11.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049482+0000", 2023-11-25T04:23:11.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053940419999999999 2023-11-25T04:23:11.979 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:11.979 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:11.979 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:11.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.979 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:10 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'5, mlcod=0'0)", 2023-11-25T04:23:11.979 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:11.979 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 24.833319493000001, 2023-11-25T04:23:11.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054493099999999997, 2023-11-25T04:23:11.980 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:11.980 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:11.980 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:11.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:11.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:11.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:11.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:11.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:11.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:11.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:11.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:11.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6100000000000002e-07 2023-11-25T04:23:11.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:11.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044019+0000", 2023-11-25T04:23:11.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.839e-06 2023-11-25T04:23:11.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.982 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.982 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:11.982 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044026+0000", 2023-11-25T04:23:11.982 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5869999999999998e-06 2023-11-25T04:23:11.982 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.982 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.982 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:11.982 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044031+0000", 2023-11-25T04:23:11.982 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4909999999999996e-06 2023-11-25T04:23:11.982 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.982 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.982 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:11.982 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044070+0000", 2023-11-25T04:23:11.983 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8829999999999999e-05 2023-11-25T04:23:11.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.983 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:11.983 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044081+0000", 2023-11-25T04:23:11.983 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1447999999999999e-05 2023-11-25T04:23:11.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.983 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:11.983 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049406+0000", 2023-11-25T04:23:11.983 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053246480000000004 2023-11-25T04:23:11.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.984 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:11.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049464+0000", 2023-11-25T04:23:11.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8205999999999998e-05 2023-11-25T04:23:11.984 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:11.984 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:11.984 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:11.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.984 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:11 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'6, mlcod=0'0)", 2023-11-25T04:23:11.984 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:11.984 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 24.833118229, 2023-11-25T04:23:11.985 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052673499999999996, 2023-11-25T04:23:11.985 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:11.985 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:11.985 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:11.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:11.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:11.985 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:11.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:11.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:11.986 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:11.986 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.986 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.986 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:11.986 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:11.986 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0199999999999998e-07 2023-11-25T04:23:11.986 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.986 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.986 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:11.986 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044220+0000", 2023-11-25T04:23:11.986 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7720000000000001e-06 2023-11-25T04:23:11.987 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.987 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.987 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:11.987 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044226+0000", 2023-11-25T04:23:11.987 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8579999999999996e-06 2023-11-25T04:23:11.987 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.987 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.987 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:11.987 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044232+0000", 2023-11-25T04:23:11.987 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1700000000000002e-06 2023-11-25T04:23:11.987 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.987 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.988 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:11.988 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044274+0000", 2023-11-25T04:23:11.988 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1254999999999997e-05 2023-11-25T04:23:11.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.988 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:11.988 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:23:11.988 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6880999999999999e-05 2023-11-25T04:23:11.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.988 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:11.988 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049466+0000", 2023-11-25T04:23:11.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051659219999999999 2023-11-25T04:23:11.989 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:11.989 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:11.989 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:11.989 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049483+0000", 2023-11-25T04:23:11.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.719e-05 2023-11-25T04:23:11.989 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:11.989 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:11.989 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:11.989 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:11.989 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:11.989 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:11.991 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:23:11.991 DEBUG:teuthology.orchestra.console.smithi076:pexpect command: ipmitool -H smithi076.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-25T04:23:12.002 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:11.989+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:12.091 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:12.090+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:12.103 DEBUG:teuthology.orchestra.console.smithi076:check power output: Chassis Power is on 2023-11-25T04:23:12.111 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:12.111 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:23:12.111 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2023-11-25T04:23:12.111 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:12.120 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:23:12.193 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:12.191+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:12.225 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:12.225 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:23:12.226 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2023-11-25T04:23:12.226 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2023-11-25T04:23:12.226 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:12.234 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:23:12.294 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:12.293+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:12.340 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:12.340 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2023-11-25T04:23:12.341 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2023-11-25T04:23:12.341 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2023-11-25T04:23:12.341 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.341 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:12.341 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:12.341 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 25.30641439, 2023-11-25T04:23:12.342 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01072943, 2023-11-25T04:23:12.342 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:12.342 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:12.342 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:12.342 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.343 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:12.343 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:12.343 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:12.343 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.343 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.343 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:12.343 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:12.343 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:12.343 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.344 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.344 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:12.344 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031829+0000", 2023-11-25T04:23:12.344 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.666e-06 2023-11-25T04:23:12.344 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.344 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.345 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:12.345 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031839+0000", 2023-11-25T04:23:12.345 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9490000000000008e-06 2023-11-25T04:23:12.345 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.345 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.345 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:12.345 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031857+0000", 2023-11-25T04:23:12.346 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8093000000000001e-05 2023-11-25T04:23:12.346 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.346 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.346 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:12.346 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031882+0000", 2023-11-25T04:23:12.346 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4294000000000002e-05 2023-11-25T04:23:12.346 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.347 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.347 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:12.347 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033130+0000", 2023-11-25T04:23:12.347 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001248264 2023-11-25T04:23:12.347 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.348 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.348 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:12.348 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033426+0000", 2023-11-25T04:23:12.348 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029620999999999998 2023-11-25T04:23:12.348 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.348 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.349 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:12.349 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033814+0000", 2023-11-25T04:23:12.349 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000387742 2023-11-25T04:23:12.349 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.349 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.350 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:12.350 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036793+0000", 2023-11-25T04:23:12.350 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002979598 2023-11-25T04:23:12.350 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.350 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.350 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:12.351 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042455+0000", 2023-11-25T04:23:12.351 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0056619260000000003 2023-11-25T04:23:12.351 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.351 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.351 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:12.352 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042502+0000", 2023-11-25T04:23:12.352 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7139999999999999e-05 2023-11-25T04:23:12.352 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.353 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.353 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:12.353 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042557+0000", 2023-11-25T04:23:12.353 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4548e-05 2023-11-25T04:23:12.353 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:12.354 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:12.354 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:12.354 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.354 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.354 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:12.354 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:12.354 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 25.305215855, 2023-11-25T04:23:12.354 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011505202000000001, 2023-11-25T04:23:12.355 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:12.355 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:12.355 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:12.355 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.355 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:12.355 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:12.355 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:12.356 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.356 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.356 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:12.356 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:12.356 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:12.356 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.356 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.357 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:12.357 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033027+0000", 2023-11-25T04:23:12.357 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0699999999999999e-06 2023-11-25T04:23:12.357 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.357 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.357 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:12.357 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033035+0000", 2023-11-25T04:23:12.357 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0830000000000004e-06 2023-11-25T04:23:12.358 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.358 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.358 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:12.358 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033059+0000", 2023-11-25T04:23:12.358 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3975e-05 2023-11-25T04:23:12.358 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.359 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.359 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:12.359 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033482+0000", 2023-11-25T04:23:12.359 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042253700000000001 2023-11-25T04:23:12.359 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.359 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.359 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:12.359 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034520+0000", 2023-11-25T04:23:12.359 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001037991 2023-11-25T04:23:12.360 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.360 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.360 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:12.360 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034611+0000", 2023-11-25T04:23:12.360 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1113999999999995e-05 2023-11-25T04:23:12.360 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.360 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.360 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:12.360 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034769+0000", 2023-11-25T04:23:12.361 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000157641 2023-11-25T04:23:12.361 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.361 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.361 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:12.361 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038292+0000", 2023-11-25T04:23:12.361 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035231160000000002 2023-11-25T04:23:12.361 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.361 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.361 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:12.361 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044440+0000", 2023-11-25T04:23:12.361 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0061486980000000002 2023-11-25T04:23:12.362 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.362 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.362 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:12.362 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044495+0000", 2023-11-25T04:23:12.362 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4339000000000003e-05 2023-11-25T04:23:12.362 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.362 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.363 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:12.363 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044531+0000", 2023-11-25T04:23:12.363 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6637999999999997e-05 2023-11-25T04:23:12.363 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:12.363 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:12.363 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:12.364 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.364 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.364 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:12.364 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:12.364 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 25.304741500999999, 2023-11-25T04:23:12.364 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0057634039999999997, 2023-11-25T04:23:12.364 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:12.364 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:12.364 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:12.365 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.365 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:12.365 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:12.365 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:12.365 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.365 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.365 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:12.365 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:12.365 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:12.365 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.365 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.366 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:12.366 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:12.366 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7900000000000007e-07 2023-11-25T04:23:12.366 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.366 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.366 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:12.366 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033507+0000", 2023-11-25T04:23:12.366 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8900000000000004e-06 2023-11-25T04:23:12.367 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.367 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.367 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:12.367 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033519+0000", 2023-11-25T04:23:12.367 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1756000000000001e-05 2023-11-25T04:23:12.367 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.367 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.367 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:12.367 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033527+0000", 2023-11-25T04:23:12.367 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3610000000000001e-06 2023-11-25T04:23:12.367 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.367 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.368 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:12.368 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034451+0000", 2023-11-25T04:23:12.368 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00092372400000000001 2023-11-25T04:23:12.368 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.368 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.368 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:12.369 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034809+0000", 2023-11-25T04:23:12.369 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00035757699999999998 2023-11-25T04:23:12.369 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.369 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.369 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:23:12.369 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034965+0000", 2023-11-25T04:23:12.370 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015638699999999999 2023-11-25T04:23:12.370 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.370 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.370 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:12.370 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038299+0000", 2023-11-25T04:23:12.370 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0033336559999999999 2023-11-25T04:23:12.370 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.370 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.370 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:12.370 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039169+0000", 2023-11-25T04:23:12.370 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00087066400000000005 2023-11-25T04:23:12.370 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.370 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.371 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:12.371 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039226+0000", 2023-11-25T04:23:12.371 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6511999999999997e-05 2023-11-25T04:23:12.371 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.371 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.371 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:12.371 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039264+0000", 2023-11-25T04:23:12.371 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7898000000000002e-05 2023-11-25T04:23:12.371 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:12.371 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:12.371 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:12.371 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.372 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.372 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:25 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:12.372 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:12.372 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 25.304662917000002, 2023-11-25T04:23:12.372 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096656169999999996, 2023-11-25T04:23:12.372 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:12.372 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:12.372 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:12.372 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.372 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:12.372 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:12.372 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:12.372 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.373 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.373 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:12.373 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:12.373 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:12.373 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.373 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.373 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:12.373 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033580+0000", 2023-11-25T04:23:12.373 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8199999999999999e-07 2023-11-25T04:23:12.373 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.373 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.373 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:12.374 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033585+0000", 2023-11-25T04:23:12.374 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3680000000000001e-06 2023-11-25T04:23:12.374 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.374 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.374 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:12.374 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033600+0000", 2023-11-25T04:23:12.374 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4331000000000001e-05 2023-11-25T04:23:12.374 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.374 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.374 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:12.374 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033606+0000", 2023-11-25T04:23:12.374 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5780000000000004e-06 2023-11-25T04:23:12.375 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.375 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.375 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:12.375 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035747+0000", 2023-11-25T04:23:12.375 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0021406530000000002 2023-11-25T04:23:12.375 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.375 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.375 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:12.375 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035901+0000", 2023-11-25T04:23:12.375 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000153931 2023-11-25T04:23:12.375 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.375 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.376 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:23:12.376 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036193+0000", 2023-11-25T04:23:12.376 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029211299999999999 2023-11-25T04:23:12.376 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.376 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.376 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:12.376 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038655+0000", 2023-11-25T04:23:12.376 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0024621640000000002 2023-11-25T04:23:12.376 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.376 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.376 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:12.376 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043130+0000", 2023-11-25T04:23:12.377 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.004474698 2023-11-25T04:23:12.377 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.377 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.377 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:12.377 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043204+0000", 2023-11-25T04:23:12.377 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4091999999999995e-05 2023-11-25T04:23:12.377 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.377 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.377 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:12.377 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043245+0000", 2023-11-25T04:23:12.377 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0907e-05 2023-11-25T04:23:12.377 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:12.378 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:12.378 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:12.378 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.378 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.378 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:26 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:12.378 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:12.378 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 25.304627093000001, 2023-11-25T04:23:12.378 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011086945000000001, 2023-11-25T04:23:12.378 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:12.378 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:12.378 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:12.378 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.379 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:12.379 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:12.379 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:12.379 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.379 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.379 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:12.379 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:12.379 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:12.379 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.379 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.379 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:12.379 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:12.380 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1699999999999998e-07 2023-11-25T04:23:12.380 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.380 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.380 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:12.380 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033627+0000", 2023-11-25T04:23:12.380 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.163e-05 2023-11-25T04:23:12.380 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.380 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.380 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:12.380 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033636+0000", 2023-11-25T04:23:12.381 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6859999999999995e-06 2023-11-25T04:23:12.381 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.381 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.381 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:12.381 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033642+0000", 2023-11-25T04:23:12.382 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3149999999999997e-06 2023-11-25T04:23:12.382 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.382 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.382 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:12.382 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035414+0000", 2023-11-25T04:23:12.383 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0017716380000000001 2023-11-25T04:23:12.383 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.384 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.384 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:12.385 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035539+0000", 2023-11-25T04:23:12.385 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000125477 2023-11-25T04:23:12.386 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.387 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.388 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:12.388 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035672+0000", 2023-11-25T04:23:12.388 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000133316 2023-11-25T04:23:12.388 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.388 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.388 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:12.388 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038649+0000", 2023-11-25T04:23:12.388 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029764679999999999 2023-11-25T04:23:12.388 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.388 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.388 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:12.388 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044578+0000", 2023-11-25T04:23:12.388 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0059293510000000002 2023-11-25T04:23:12.389 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.389 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.389 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:12.389 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044651+0000", 2023-11-25T04:23:12.389 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2513000000000002e-05 2023-11-25T04:23:12.389 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.389 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.389 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:12.389 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044702+0000", 2023-11-25T04:23:12.389 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1034e-05 2023-11-25T04:23:12.389 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:12.389 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:12.390 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:12.390 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.390 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.390 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:29 3.b 3:d0630e4c:::mds0_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:12.390 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:12.390 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 25.304589173, 2023-11-25T04:23:12.390 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010980227, 2023-11-25T04:23:12.390 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:12.390 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:12.390 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:12.390 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.391 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:12.391 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:12.391 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:12.391 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.391 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.391 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:12.391 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:12.391 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:12.391 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.391 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.391 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:12.391 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:12.392 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3499999999999996e-07 2023-11-25T04:23:12.392 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.392 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.392 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:12.392 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033658+0000", 2023-11-25T04:23:12.392 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1280000000000001e-06 2023-11-25T04:23:12.392 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.392 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.392 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:12.392 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033667+0000", 2023-11-25T04:23:12.392 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1430000000000004e-06 2023-11-25T04:23:12.392 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.392 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.393 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:12.393 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033678+0000", 2023-11-25T04:23:12.393 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0906e-05 2023-11-25T04:23:12.393 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.393 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.393 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:12.393 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034456+0000", 2023-11-25T04:23:12.393 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00077833899999999996 2023-11-25T04:23:12.393 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.393 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.393 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:12.393 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034575+0000", 2023-11-25T04:23:12.393 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011926000000000001 2023-11-25T04:23:12.394 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.394 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.394 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:12.394 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035001+0000", 2023-11-25T04:23:12.394 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042550999999999998 2023-11-25T04:23:12.394 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.394 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.394 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:12.394 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.037923+0000", 2023-11-25T04:23:12.394 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029219670000000001 2023-11-25T04:23:12.394 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.394 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.394 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:12.395 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044538+0000", 2023-11-25T04:23:12.395 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0066152349999999997 2023-11-25T04:23:12.395 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.395 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.395 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:12.395 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044594+0000", 2023-11-25T04:23:12.395 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5593000000000001e-05 2023-11-25T04:23:12.395 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.395 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.395 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:12.395 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044633+0000", 2023-11-25T04:23:12.395 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9511000000000002e-05 2023-11-25T04:23:12.396 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:12.396 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:12.396 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:12.397 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.397 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.398 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:34 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:12.399 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:12.399 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 25.30433906, 2023-11-25T04:23:12.400 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010920445000000001, 2023-11-25T04:23:12.400 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:12.400 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:12.401 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:12.401 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.402 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:12.402 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:12.402 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:12.402 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.403 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.403 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:12.403 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:12.403 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:12.403 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.403 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.403 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:12.403 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033904+0000", 2023-11-25T04:23:12.403 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6599999999999994e-07 2023-11-25T04:23:12.403 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.404 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.404 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:12.404 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033909+0000", 2023-11-25T04:23:12.404 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5639999999999996e-06 2023-11-25T04:23:12.404 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.404 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.404 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:12.404 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033923+0000", 2023-11-25T04:23:12.404 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3028e-05 2023-11-25T04:23:12.404 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.404 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.405 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:12.405 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034043+0000", 2023-11-25T04:23:12.405 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000120328 2023-11-25T04:23:12.405 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.405 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.405 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:12.405 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035576+0000", 2023-11-25T04:23:12.405 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015333720000000001 2023-11-25T04:23:12.406 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.406 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.406 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:12.406 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035699+0000", 2023-11-25T04:23:12.406 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000123001 2023-11-25T04:23:12.406 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.406 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.406 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:12.406 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044823+0000", 2023-11-25T04:23:12.406 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0091241859999999994 2023-11-25T04:23:12.406 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:12.407 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:12.407 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:12.407 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.407 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.407 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:35 3.1f 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:12.407 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:12.407 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 25.303034855, 2023-11-25T04:23:12.407 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096164779999999995, 2023-11-25T04:23:12.407 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:12.407 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:12.407 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:12.407 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.408 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:12.408 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:12.408 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:12.408 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.408 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.408 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:12.408 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:12.408 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:12.408 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.408 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.408 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:12.409 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035208+0000", 2023-11-25T04:23:12.409 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2699999999999998e-07 2023-11-25T04:23:12.409 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.409 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.409 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:12.409 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035213+0000", 2023-11-25T04:23:12.409 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.6070000000000001e-06 2023-11-25T04:23:12.409 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.409 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.410 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:12.410 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035350+0000", 2023-11-25T04:23:12.410 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013693599999999999 2023-11-25T04:23:12.410 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.410 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.410 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:12.410 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035358+0000", 2023-11-25T04:23:12.410 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0099999999999995e-06 2023-11-25T04:23:12.410 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.410 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.410 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:12.411 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036260+0000", 2023-11-25T04:23:12.411 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00090240100000000005 2023-11-25T04:23:12.411 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.411 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.411 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:12.411 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036517+0000", 2023-11-25T04:23:12.411 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00025736299999999998 2023-11-25T04:23:12.411 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.411 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.411 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:12.412 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044824+0000", 2023-11-25T04:23:12.412 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0083063340000000003 2023-11-25T04:23:12.412 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:12.412 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:12.412 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:12.412 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.412 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.412 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:37 3.1e 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:12.412 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:12.412 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 25.302868169, 2023-11-25T04:23:12.412 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0078322640000000002, 2023-11-25T04:23:12.413 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:12.413 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:12.413 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:12.413 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.413 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:12.413 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:12.413 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:12.413 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.413 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.413 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:12.413 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:12.413 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:12.414 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.414 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.414 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:12.414 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035375+0000", 2023-11-25T04:23:12.414 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9100000000000003e-07 2023-11-25T04:23:12.414 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.414 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.414 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:12.414 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035379+0000", 2023-11-25T04:23:12.414 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9430000000000002e-06 2023-11-25T04:23:12.414 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.414 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.415 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:12.415 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035388+0000", 2023-11-25T04:23:12.415 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2760000000000006e-06 2023-11-25T04:23:12.415 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.415 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.415 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:12.415 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035396+0000", 2023-11-25T04:23:12.415 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8480000000000001e-06 2023-11-25T04:23:12.415 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.415 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.415 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:12.415 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035452+0000", 2023-11-25T04:23:12.416 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5831000000000001e-05 2023-11-25T04:23:12.416 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.416 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.416 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:12.416 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035934+0000", 2023-11-25T04:23:12.416 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00048265599999999998 2023-11-25T04:23:12.416 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.416 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.416 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:12.416 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043206+0000", 2023-11-25T04:23:12.416 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.007271919 2023-11-25T04:23:12.417 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:12.417 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:12.417 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:12.417 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.417 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.417 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing(3.1a epoch 18/16 rep_tid 3 entries 18'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:36 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:23:12.417 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:12.417 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 25.301592583000001, 2023-11-25T04:23:12.418 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002073414, 2023-11-25T04:23:12.418 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:12.418 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:12.418 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:12.418 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.418 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:12.418 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:12.418 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:12.418 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.418 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.418 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:12.419 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:12.420 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:12.420 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.420 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.420 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:12.420 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036650+0000", 2023-11-25T04:23:12.420 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7300000000000002e-07 2023-11-25T04:23:12.420 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.420 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.420 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:12.420 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036659+0000", 2023-11-25T04:23:12.421 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.7499999999999992e-06 2023-11-25T04:23:12.421 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.421 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.421 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:12.421 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036673+0000", 2023-11-25T04:23:12.421 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4917e-05 2023-11-25T04:23:12.421 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.421 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.421 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:12.421 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036687+0000", 2023-11-25T04:23:12.422 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3883000000000001e-05 2023-11-25T04:23:12.422 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.422 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.422 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:12.422 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036781+0000", 2023-11-25T04:23:12.422 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3219000000000003e-05 2023-11-25T04:23:12.422 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.422 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.422 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:12.423 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038723+0000", 2023-11-25T04:23:12.423 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019422720000000001 2023-11-25T04:23:12.423 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:12.423 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:12.423 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:12.423 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.423 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.423 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:28 3.13 e18/16 3:cd5a64a3:::100.00000000:head v 18'1, mlcod=0'0)", 2023-11-25T04:23:12.423 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:12.423 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 25.297931425000002, 2023-11-25T04:23:12.423 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001850546, 2023-11-25T04:23:12.424 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:12.424 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:12.424 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:12.424 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.424 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:12.424 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:12.424 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:12.424 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.424 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.424 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:12.424 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:12.424 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:12.425 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.425 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.425 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:12.425 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:12.425 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7800000000000002e-07 2023-11-25T04:23:12.425 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.425 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.425 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:12.425 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040331+0000", 2023-11-25T04:23:12.425 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0449999999999999e-05 2023-11-25T04:23:12.426 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.426 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.426 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:12.426 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040339+0000", 2023-11-25T04:23:12.426 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0709999999999995e-06 2023-11-25T04:23:12.426 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.426 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.426 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:12.426 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040347+0000", 2023-11-25T04:23:12.426 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7079999999999997e-06 2023-11-25T04:23:12.426 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.427 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.427 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:12.427 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040410+0000", 2023-11-25T04:23:12.427 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2453000000000007e-05 2023-11-25T04:23:12.427 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.427 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.427 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:12.427 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040422+0000", 2023-11-25T04:23:12.427 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2429000000000001e-05 2023-11-25T04:23:12.427 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.428 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.428 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:12.428 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042129+0000", 2023-11-25T04:23:12.428 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001707266 2023-11-25T04:23:12.428 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.428 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.428 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:12.428 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042161+0000", 2023-11-25T04:23:12.428 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1791000000000001e-05 2023-11-25T04:23:12.428 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:12.428 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:12.428 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:12.428 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.429 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.429 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:27 3.13 e18/16 3:c91d4a1d:::mds0_inotable:head v 18'2, mlcod=0'0)", 2023-11-25T04:23:12.429 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:12.429 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 25.297727901999998, 2023-11-25T04:23:12.429 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0016886430000000001, 2023-11-25T04:23:12.429 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:12.429 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:12.429 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:12.429 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.429 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:12.429 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:12.429 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:12.429 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.429 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.430 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:12.430 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:12.430 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:12.430 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.430 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.430 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:12.430 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:12.430 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3999999999999997e-07 2023-11-25T04:23:12.430 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.430 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.430 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:12.430 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040520+0000", 2023-11-25T04:23:12.430 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2989999999999996e-06 2023-11-25T04:23:12.431 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.431 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.431 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:12.431 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040527+0000", 2023-11-25T04:23:12.431 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5730000000000002e-06 2023-11-25T04:23:12.431 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.431 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.431 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:12.431 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040535+0000", 2023-11-25T04:23:12.431 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8630000000000008e-06 2023-11-25T04:23:12.431 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.431 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.431 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:12.431 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040683+0000", 2023-11-25T04:23:12.432 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000148002 2023-11-25T04:23:12.432 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.432 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.432 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:12.432 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040700+0000", 2023-11-25T04:23:12.432 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7065e-05 2023-11-25T04:23:12.432 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.432 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.432 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:12.432 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042163+0000", 2023-11-25T04:23:12.432 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001462767 2023-11-25T04:23:12.432 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.432 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.432 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:12.433 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042203+0000", 2023-11-25T04:23:12.433 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9734000000000003e-05 2023-11-25T04:23:12.433 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:12.433 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:12.433 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:12.433 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.433 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.433 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:20 3.1c e18/16 3:3ed09add:::607.00000000:head v 18'1, mlcod=0'0)", 2023-11-25T04:23:12.433 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:12.433 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 25.297692861000002, 2023-11-25T04:23:12.433 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015271, 2023-11-25T04:23:12.433 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:12.433 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:12.433 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:12.434 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.434 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:12.434 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:12.434 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:12.434 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.434 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.434 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:12.434 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:12.434 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:12.434 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.434 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.434 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:12.434 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:12.434 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4799999999999999e-07 2023-11-25T04:23:12.435 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.435 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.435 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:12.435 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040561+0000", 2023-11-25T04:23:12.435 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1800000000000001e-05 2023-11-25T04:23:12.435 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.435 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.435 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:12.435 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040568+0000", 2023-11-25T04:23:12.435 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7730000000000002e-06 2023-11-25T04:23:12.435 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.435 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.435 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:12.435 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040587+0000", 2023-11-25T04:23:12.436 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8691000000000001e-05 2023-11-25T04:23:12.436 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.436 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.436 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:12.436 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040638+0000", 2023-11-25T04:23:12.436 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1068e-05 2023-11-25T04:23:12.436 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.436 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.436 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:12.436 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040670+0000", 2023-11-25T04:23:12.436 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2125999999999997e-05 2023-11-25T04:23:12.436 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.436 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.436 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:12.437 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042032+0000", 2023-11-25T04:23:12.437 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001361821 2023-11-25T04:23:12.437 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.437 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.437 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:12.437 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042076+0000", 2023-11-25T04:23:12.437 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4472999999999999e-05 2023-11-25T04:23:12.437 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:12.437 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:12.437 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:12.437 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.437 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.437 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:21 3.1c e18/16 3:3ed09add:::607.00000000:head v 18'2, mlcod=0'0)", 2023-11-25T04:23:12.437 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:12.438 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 25.297458135999999, 2023-11-25T04:23:12.438 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0013228160000000001, 2023-11-25T04:23:12.438 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:12.438 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:12.438 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:12.438 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.438 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:12.438 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:12.438 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:12.438 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.438 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.438 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:12.439 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:12.439 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:12.439 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.439 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.439 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:12.439 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:12.439 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9299999999999999e-07 2023-11-25T04:23:12.439 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.439 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.439 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:12.439 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040790+0000", 2023-11-25T04:23:12.440 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5770000000000001e-06 2023-11-25T04:23:12.440 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.440 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.440 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:12.440 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040799+0000", 2023-11-25T04:23:12.440 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6439999999999995e-06 2023-11-25T04:23:12.440 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.440 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.440 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:12.440 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040806+0000", 2023-11-25T04:23:12.440 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0180000000000004e-06 2023-11-25T04:23:12.440 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.440 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.440 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:12.441 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040907+0000", 2023-11-25T04:23:12.441 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000100096 2023-11-25T04:23:12.441 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.441 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.441 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:12.441 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040925+0000", 2023-11-25T04:23:12.441 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8536999999999999e-05 2023-11-25T04:23:12.441 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.441 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.441 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:12.441 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042079+0000", 2023-11-25T04:23:12.441 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00115358 2023-11-25T04:23:12.441 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.441 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.442 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:12.442 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042107+0000", 2023-11-25T04:23:12.442 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7971e-05 2023-11-25T04:23:12.442 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:12.442 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:12.442 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:12.442 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.442 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.442 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop_reply(mds.0.3:4 3.1f e18/16 ondisk, result = 0)", 2023-11-25T04:23:12.442 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:12.442 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 25.296118746000001, 2023-11-25T04:23:12.442 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00044092900000000001, 2023-11-25T04:23:12.442 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:12.442 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:12.443 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:12.443 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.443 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:12.443 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:12.443 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:12.443 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.443 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.443 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:12.443 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:12.443 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:12.443 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.443 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.443 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:12.443 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042124+0000", 2023-11-25T04:23:12.443 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1800000000000005e-07 2023-11-25T04:23:12.444 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.444 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.444 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:12.444 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042128+0000", 2023-11-25T04:23:12.444 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2289999999999997e-06 2023-11-25T04:23:12.444 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.444 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.444 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:12.444 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042138+0000", 2023-11-25T04:23:12.444 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9729999999999992e-06 2023-11-25T04:23:12.444 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.444 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.444 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:12.444 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042145+0000", 2023-11-25T04:23:12.445 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.336e-06 2023-11-25T04:23:12.445 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.445 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.445 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:12.445 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042441+0000", 2023-11-25T04:23:12.445 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029650500000000002 2023-11-25T04:23:12.445 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.445 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.445 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:12.445 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042448+0000", 2023-11-25T04:23:12.445 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4100000000000002e-06 2023-11-25T04:23:12.445 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.445 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.446 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:12.446 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042564+0000", 2023-11-25T04:23:12.446 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000115758 2023-11-25T04:23:12.446 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:12.446 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:12.446 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:12.446 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.446 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.446 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing_reply(3.1f epoch 18/16 rep_tid 8 lcod 18'4)", 2023-11-25T04:23:12.446 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:12.446 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 25.293793334, 2023-11-25T04:23:12.446 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00038154699999999998, 2023-11-25T04:23:12.446 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:12.447 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:12.447 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:12.447 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.447 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:12.447 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:12.447 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:12.447 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.447 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.447 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:12.447 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:12.447 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:12.447 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.447 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.448 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:12.448 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:12.448 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2700000000000001e-07 2023-11-25T04:23:12.448 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.448 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.448 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:12.448 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044451+0000", 2023-11-25T04:23:12.448 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7460000000000001e-06 2023-11-25T04:23:12.448 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.448 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.448 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:12.448 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044455+0000", 2023-11-25T04:23:12.448 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5809999999999998e-06 2023-11-25T04:23:12.449 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.449 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.449 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:12.449 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044458+0000", 2023-11-25T04:23:12.449 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0170000000000001e-06 2023-11-25T04:23:12.449 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.449 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.449 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:12.449 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044734+0000", 2023-11-25T04:23:12.449 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00027582799999999999 2023-11-25T04:23:12.449 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.449 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.449 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:12.450 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044830+0000", 2023-11-25T04:23:12.450 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6248000000000004e-05 2023-11-25T04:23:12.450 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:12.450 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:12.450 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:12.450 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.450 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.450 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing_reply(3.b epoch 18/16 rep_tid 5 lcod 18'2)", 2023-11-25T04:23:12.450 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:12.450 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 25.293756673000001, 2023-11-25T04:23:12.450 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000344488, 2023-11-25T04:23:12.450 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:12.450 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:12.451 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:12.451 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.451 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:12.451 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:12.451 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:12.451 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.451 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.451 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:12.451 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:12.451 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:12.451 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.451 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.451 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:12.452 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:12.452 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.55e-07 2023-11-25T04:23:12.452 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.452 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.452 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:12.452 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044487+0000", 2023-11-25T04:23:12.452 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2470000000000001e-06 2023-11-25T04:23:12.452 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.452 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.452 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:12.452 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044491+0000", 2023-11-25T04:23:12.452 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9659999999999998e-06 2023-11-25T04:23:12.452 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.453 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.453 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:12.453 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044494+0000", 2023-11-25T04:23:12.453 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2310000000000001e-06 2023-11-25T04:23:12.453 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.453 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.453 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:12.453 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044756+0000", 2023-11-25T04:23:12.453 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00026172800000000003 2023-11-25T04:23:12.453 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.453 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.453 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:12.453 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044830+0000", 2023-11-25T04:23:12.454 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4160999999999997e-05 2023-11-25T04:23:12.454 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:12.454 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:12.454 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:12.454 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.454 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.454 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:39 3.6 3:654134d2:::mds0_openfiles.0:head [omap-clear,omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2023-11-25T04:23:12.454 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:12.454 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 20.233930631, 2023-11-25T04:23:12.454 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001788658, 2023-11-25T04:23:12.454 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:12.454 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:12.454 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:12.455 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.455 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:12.455 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:12.455 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:12.455 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.455 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.455 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:12.455 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:12.455 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:12.455 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.455 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.455 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:12.455 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104314+0000", 2023-11-25T04:23:12.455 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3099999999999999e-06 2023-11-25T04:23:12.456 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.456 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.456 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:12.456 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104325+0000", 2023-11-25T04:23:12.456 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1511999999999999e-05 2023-11-25T04:23:12.456 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.456 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.456 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:12.456 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104362+0000", 2023-11-25T04:23:12.456 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6903000000000002e-05 2023-11-25T04:23:12.456 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.456 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.456 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:12.456 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104387+0000", 2023-11-25T04:23:12.457 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4766e-05 2023-11-25T04:23:12.457 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.457 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.457 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:12.457 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104465+0000", 2023-11-25T04:23:12.457 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8119000000000002e-05 2023-11-25T04:23:12.457 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.457 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.457 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:12.457 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104557+0000", 2023-11-25T04:23:12.457 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2496000000000003e-05 2023-11-25T04:23:12.457 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.457 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.458 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:12.458 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104646+0000", 2023-11-25T04:23:12.458 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8818000000000002e-05 2023-11-25T04:23:12.458 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.458 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.458 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:12.458 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105201+0000", 2023-11-25T04:23:12.458 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00055506000000000002 2023-11-25T04:23:12.458 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.458 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.458 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:12.458 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105994+0000", 2023-11-25T04:23:12.459 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00079294700000000001 2023-11-25T04:23:12.459 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.459 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.459 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:12.459 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.106055+0000", 2023-11-25T04:23:12.459 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0566000000000003e-05 2023-11-25T04:23:12.459 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.459 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.459 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:12.459 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.106100+0000", 2023-11-25T04:23:12.459 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5160999999999997e-05 2023-11-25T04:23:12.459 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:12.459 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:12.459 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:12.460 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.460 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.460 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:43 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2023-11-25T04:23:12.460 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:12.460 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.2337056349999997, 2023-11-25T04:23:12.460 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001751166, 2023-11-25T04:23:12.460 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:12.460 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:12.460 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:12.460 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.460 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:12.460 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:12.461 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:12.461 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.461 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.461 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:12.461 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:12.461 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:12.461 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.461 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.461 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:12.461 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104539+0000", 2023-11-25T04:23:12.461 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9529999999999999e-06 2023-11-25T04:23:12.461 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.461 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.462 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:12.462 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104557+0000", 2023-11-25T04:23:12.462 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7464000000000001e-05 2023-11-25T04:23:12.462 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.462 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.462 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:12.462 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104590+0000", 2023-11-25T04:23:12.462 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2966000000000003e-05 2023-11-25T04:23:12.462 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.462 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.462 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:12.462 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104608+0000", 2023-11-25T04:23:12.462 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8193e-05 2023-11-25T04:23:12.463 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.463 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.463 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:12.463 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104767+0000", 2023-11-25T04:23:12.463 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015882 2023-11-25T04:23:12.463 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.463 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.463 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:12.463 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104909+0000", 2023-11-25T04:23:12.463 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014244999999999999 2023-11-25T04:23:12.463 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.463 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.463 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:12.464 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.105059+0000", 2023-11-25T04:23:12.464 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014964399999999999 2023-11-25T04:23:12.464 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.464 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.464 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:12.464 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.105701+0000", 2023-11-25T04:23:12.464 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00064201099999999995 2023-11-25T04:23:12.464 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.464 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.464 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:12.464 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106233+0000", 2023-11-25T04:23:12.464 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00053178500000000001 2023-11-25T04:23:12.464 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.465 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.465 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:12.465 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106265+0000", 2023-11-25T04:23:12.465 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1986000000000002e-05 2023-11-25T04:23:12.465 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.465 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.465 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:12.465 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106288+0000", 2023-11-25T04:23:12.465 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2894000000000001e-05 2023-11-25T04:23:12.465 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:12.465 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:12.465 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:12.465 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.466 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.466 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:45 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e23)", 2023-11-25T04:23:12.466 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:23:12.466 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 0.23350759500000001, 2023-11-25T04:23:12.466 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00098423299999999994, 2023-11-25T04:23:12.466 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:12.466 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:12.466 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:12.466 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.466 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:12.466 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:23:12.466 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:12.466 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.466 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.467 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:12.467 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:23:12.467 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:12.467 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.467 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.467 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:12.467 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104737+0000", 2023-11-25T04:23:12.467 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3309999999999998e-06 2023-11-25T04:23:12.467 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.467 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.467 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:12.467 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104748+0000", 2023-11-25T04:23:12.467 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1080999999999999e-05 2023-11-25T04:23:12.468 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.468 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.468 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:12.468 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104771+0000", 2023-11-25T04:23:12.468 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2707000000000001e-05 2023-11-25T04:23:12.468 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.468 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.468 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:12.468 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104800+0000", 2023-11-25T04:23:12.468 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9595e-05 2023-11-25T04:23:12.468 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.468 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.468 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:12.469 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104890+0000", 2023-11-25T04:23:12.469 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.9808999999999993e-05 2023-11-25T04:23:12.469 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.469 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.469 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:12.469 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105004+0000", 2023-11-25T04:23:12.469 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000114508 2023-11-25T04:23:12.469 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.469 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.469 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:12.469 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105634+0000", 2023-11-25T04:23:12.469 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00062985799999999996 2023-11-25T04:23:12.469 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.470 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.470 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:12.470 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105671+0000", 2023-11-25T04:23:12.470 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6640000000000002e-05 2023-11-25T04:23:12.470 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:12.470 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:12.470 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:12.470 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105719+0000", 2023-11-25T04:23:12.470 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7704000000000003e-05 2023-11-25T04:23:12.470 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:12.470 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:12.470 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:12.470 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:12.471 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:12.471 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:12.472 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:12.470+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:12.472 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:23:12.573 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:12.572+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:12.579 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:12.579 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:23:12.580 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2023-11-25T04:23:12.580 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:12.587 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-11-25T04:23:12.589 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:23:12.674 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:12.673+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:12.692 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:12.692 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:23:12.693 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2023-11-25T04:23:12.693 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2023-11-25T04:23:12.693 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:12.701 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:23:12.712 INFO:teuthology.orchestra.run.smithi067.stderr:Error ENXIO: problem getting command descriptions from osd.0 2023-11-25T04:23:12.730 INFO:teuthology.orchestra.run.smithi067.stdout:osd.1: {} 2023-11-25T04:23:12.731 INFO:teuthology.orchestra.run.smithi067.stderr:osd.1: osd_enable_op_tracker = '' 2023-11-25T04:23:12.750 INFO:teuthology.orchestra.run.smithi067.stdout:osd.2: {} 2023-11-25T04:23:12.750 INFO:teuthology.orchestra.run.smithi067.stderr:osd.2: osd_enable_op_tracker = '' 2023-11-25T04:23:12.753 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-11-25T04:23:12.753 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-11-25T04:23:12.775 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:12.774+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:12.806 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:12.806 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:12.815 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:23:12.876 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:12.875+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:12.922 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:12.922 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:12.931 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:23:12.977 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:12.975+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:13.036 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:13.036 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:13.045 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:23:13.078 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:13.076+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:13.141 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2023-11-25T04:23:13.179 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:13.177+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:13.191 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:13.191 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:13.200 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:23:13.280 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:13.279+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:13.306 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:13.306 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:13.316 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:23:13.381 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:13.379+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:13.420 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:13.421 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:13.429 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:23:13.481 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:13.480+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:13.533 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:13.533 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:13.541 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:23:13.584 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:13.583+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:13.647 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:13.648 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:13.656 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:23:13.684 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:13.682+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:13.784 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:13.783+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:13.805 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:13.805 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:13.815 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:23:13.885 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:13.884+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:13.920 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:13.921 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:13.930 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:23:13.986 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:13.985+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:14.038 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:14.038 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:14.047 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:23:14.087 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:14.085+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:14.137 INFO:teuthology.orchestra.run.smithi067.stderr:nodeep-scrub is unset 2023-11-25T04:23:14.188 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:14.187+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:14.193 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:14.193 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:14.202 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:23:14.289 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:14.288+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:14.348 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:14.348 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:14.356 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:23:14.390 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:14.389+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:14.461 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:14.462 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:14.470 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:23:14.490 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:14.489+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:14.575 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:14.575 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:14.585 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:23:14.629 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:14.628+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:14.689 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:14.690 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:14.692 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:14.691+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:14.698 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:23:14.754 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-11-25T04:23:14.792 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~6s 2023-11-25T04:23:14.793 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:14.792+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:14.804 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:14.805 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:14.813 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:23:14.867 INFO:teuthology.orchestra.run.smithi067.stderr:Error ENXIO: problem getting command descriptions from osd.0 2023-11-25T04:23:14.889 INFO:teuthology.orchestra.run.smithi067.stdout:osd.1: {} 2023-11-25T04:23:14.890 INFO:teuthology.orchestra.run.smithi067.stderr:osd.1: osd_enable_op_tracker = '' 2023-11-25T04:23:14.897 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:14.896+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:14.908 INFO:teuthology.orchestra.run.smithi067.stdout:osd.2: {} 2023-11-25T04:23:14.908 INFO:teuthology.orchestra.run.smithi067.stderr:osd.2: osd_enable_op_tracker = '' 2023-11-25T04:23:14.911 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-11-25T04:23:14.911 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-11-25T04:23:14.959 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:14.959 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:23:14.959 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2023-11-25T04:23:14.959 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2023-11-25T04:23:14.959 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:14.967 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:23:14.995 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:14.994+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:15.096 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:15.095+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:15.114 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:15.114 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2023-11-25T04:23:15.114 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2023-11-25T04:23:15.114 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2023-11-25T04:23:15.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.115 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:155 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'109, mlcod=11'105)", 2023-11-25T04:23:15.115 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:15.115 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.327214040999998, 2023-11-25T04:23:15.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061719019999999999, 2023-11-25T04:23:15.115 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:15.115 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:15.115 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:15.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.115 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:15.115 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:15.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:15.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.116 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:15.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:15.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:15.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.116 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:15.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784884+0000", 2023-11-25T04:23:15.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3099999999999999e-07 2023-11-25T04:23:15.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.116 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:15.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784942+0000", 2023-11-25T04:23:15.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8570000000000003e-05 2023-11-25T04:23:15.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.117 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:15.117 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784949+0000", 2023-11-25T04:23:15.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0310000000000001e-06 2023-11-25T04:23:15.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.117 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:15.117 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784953+0000", 2023-11-25T04:23:15.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6320000000000001e-06 2023-11-25T04:23:15.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.118 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:15.118 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784993+0000", 2023-11-25T04:23:15.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9737000000000003e-05 2023-11-25T04:23:15.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.118 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:15.118 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785011+0000", 2023-11-25T04:23:15.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8349999999999999e-05 2023-11-25T04:23:15.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.118 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:15.118 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791024+0000", 2023-11-25T04:23:15.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006012997 2023-11-25T04:23:15.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:15.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791055+0000", 2023-11-25T04:23:15.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1254000000000003e-05 2023-11-25T04:23:15.119 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:15.119 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:15.119 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:15.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.119 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:156 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'110, mlcod=11'105)", 2023-11-25T04:23:15.119 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:15.119 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.326627991999999, 2023-11-25T04:23:15.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005616116, 2023-11-25T04:23:15.120 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:15.120 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:15.120 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:15.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.120 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:15.120 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:15.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:15.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.120 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:15.120 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:15.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:15.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.121 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:15.121 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785470+0000", 2023-11-25T04:23:15.121 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0699999999999998e-07 2023-11-25T04:23:15.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.121 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:15.121 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785474+0000", 2023-11-25T04:23:15.121 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.048e-06 2023-11-25T04:23:15.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.121 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:15.122 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785480+0000", 2023-11-25T04:23:15.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7729999999999996e-06 2023-11-25T04:23:15.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.122 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:15.122 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785483+0000", 2023-11-25T04:23:15.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1970000000000001e-06 2023-11-25T04:23:15.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.122 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:15.122 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787936+0000", 2023-11-25T04:23:15.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024529040000000001 2023-11-25T04:23:15.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.123 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:15.123 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787947+0000", 2023-11-25T04:23:15.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1669e-05 2023-11-25T04:23:15.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.123 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:15.123 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791057+0000", 2023-11-25T04:23:15.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031095200000000002 2023-11-25T04:23:15.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.123 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:15.124 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791086+0000", 2023-11-25T04:23:15.124 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8697999999999999e-05 2023-11-25T04:23:15.124 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:15.124 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:15.124 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:15.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.124 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:157 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'111, mlcod=11'106)", 2023-11-25T04:23:15.124 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:15.124 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.326035433999998, 2023-11-25T04:23:15.124 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050590280000000001, 2023-11-25T04:23:15.124 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:15.124 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:15.125 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:15.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.125 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:15.125 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:15.125 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:15.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.125 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:15.125 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:15.125 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:15.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.125 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:15.126 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786063+0000", 2023-11-25T04:23:15.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2600000000000002e-07 2023-11-25T04:23:15.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.126 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:15.126 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786069+0000", 2023-11-25T04:23:15.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8839999999999999e-06 2023-11-25T04:23:15.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.126 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:15.126 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786079+0000", 2023-11-25T04:23:15.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0124e-05 2023-11-25T04:23:15.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.127 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:15.127 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786085+0000", 2023-11-25T04:23:15.127 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6660000000000002e-06 2023-11-25T04:23:15.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.127 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:15.127 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788160+0000", 2023-11-25T04:23:15.127 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020747669999999999 2023-11-25T04:23:15.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.127 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:15.128 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788180+0000", 2023-11-25T04:23:15.128 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0208999999999999e-05 2023-11-25T04:23:15.128 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.128 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.128 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:15.128 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791087+0000", 2023-11-25T04:23:15.128 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029064630000000002 2023-11-25T04:23:15.128 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.128 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.128 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:15.128 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791121+0000", 2023-11-25T04:23:15.128 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4289000000000003e-05 2023-11-25T04:23:15.128 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:15.128 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:15.129 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:15.129 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.129 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.129 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:158 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'112, mlcod=11'106)", 2023-11-25T04:23:15.129 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:15.129 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.325619041000003, 2023-11-25T04:23:15.129 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046663030000000001, 2023-11-25T04:23:15.129 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:15.129 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:15.129 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:15.129 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.129 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:15.130 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:15.130 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:15.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.130 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.130 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:15.130 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:15.130 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:15.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.130 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.130 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:15.130 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786479+0000", 2023-11-25T04:23:15.130 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2799999999999996e-07 2023-11-25T04:23:15.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:15.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786498+0000", 2023-11-25T04:23:15.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9069e-05 2023-11-25T04:23:15.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:15.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786508+0000", 2023-11-25T04:23:15.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5810000000000003e-06 2023-11-25T04:23:15.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.132 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:15.132 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786512+0000", 2023-11-25T04:23:15.132 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4889999999999997e-06 2023-11-25T04:23:15.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.132 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:15.132 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788438+0000", 2023-11-25T04:23:15.132 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019257059999999999 2023-11-25T04:23:15.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.133 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:15.133 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788449+0000", 2023-11-25T04:23:15.133 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1419e-05 2023-11-25T04:23:15.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.133 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.133 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:15.133 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791122+0000", 2023-11-25T04:23:15.133 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002672822 2023-11-25T04:23:15.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.133 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.133 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:15.133 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791145+0000", 2023-11-25T04:23:15.133 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2589e-05 2023-11-25T04:23:15.134 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:15.134 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:15.134 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:15.134 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.134 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.134 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:206 1.0 e12/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 12'152, mlcod=12'148)", 2023-11-25T04:23:15.134 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:15.134 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.813850571000003, 2023-11-25T04:23:15.134 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046728189999999999, 2023-11-25T04:23:15.134 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:15.135 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:15.135 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:15.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.135 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:15.135 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:15.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:15.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.135 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:15.135 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:15.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:15.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:15.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298248+0000", 2023-11-25T04:23:15.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3600000000000003e-07 2023-11-25T04:23:15.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:15.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298278+0000", 2023-11-25T04:23:15.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0474e-05 2023-11-25T04:23:15.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:15.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298288+0000", 2023-11-25T04:23:15.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0156000000000001e-05 2023-11-25T04:23:15.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:15.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298293+0000", 2023-11-25T04:23:15.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7400000000000004e-06 2023-11-25T04:23:15.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:15.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298333+0000", 2023-11-25T04:23:15.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9666000000000003e-05 2023-11-25T04:23:15.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:15.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298351+0000", 2023-11-25T04:23:15.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8544999999999999e-05 2023-11-25T04:23:15.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.138 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:15.138 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302891+0000", 2023-11-25T04:23:15.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045398519999999996 2023-11-25T04:23:15.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.138 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:15.138 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302920+0000", 2023-11-25T04:23:15.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8750000000000001e-05 2023-11-25T04:23:15.138 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:15.139 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:15.139 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:15.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.139 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:14 3.15 3:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:15.139 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:15.139 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.078716214, 2023-11-25T04:23:15.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010990437, 2023-11-25T04:23:15.139 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:15.139 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:15.139 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:15.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.139 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:15.140 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:15.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:15.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.140 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:15.140 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:15.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:15.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.140 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:15.140 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033383+0000", 2023-11-25T04:23:15.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9870000000000002e-06 2023-11-25T04:23:15.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.141 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.141 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:15.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033398+0000", 2023-11-25T04:23:15.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4569999999999999e-05 2023-11-25T04:23:15.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.141 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.141 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:15.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033411+0000", 2023-11-25T04:23:15.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3543999999999999e-05 2023-11-25T04:23:15.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.141 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.141 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:15.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033423+0000", 2023-11-25T04:23:15.142 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1905999999999999e-05 2023-11-25T04:23:15.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.142 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:15.142 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034433+0000", 2023-11-25T04:23:15.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00101022 2023-11-25T04:23:15.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:15.143 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034539+0000", 2023-11-25T04:23:15.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010597199999999999 2023-11-25T04:23:15.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:15.143 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034667+0000", 2023-11-25T04:23:15.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012765500000000001 2023-11-25T04:23:15.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:15.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037803+0000", 2023-11-25T04:23:15.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031362220000000001 2023-11-25T04:23:15.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:15.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:23:15.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0064964669999999997 2023-11-25T04:23:15.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:15.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044331+0000", 2023-11-25T04:23:15.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1544999999999997e-05 2023-11-25T04:23:15.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:15.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044372+0000", 2023-11-25T04:23:15.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0349000000000003e-05 2023-11-25T04:23:15.145 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:15.145 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:15.145 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:15.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.146 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:15 3.15 3:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:15.146 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:15.146 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.078056564000001, 2023-11-25T04:23:15.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010464362, 2023-11-25T04:23:15.146 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:15.146 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:15.146 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:15.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.146 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:15.146 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:15.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:15.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.147 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:15.147 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:15.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:15.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.147 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:15.147 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034042+0000", 2023-11-25T04:23:15.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0999999999999997e-07 2023-11-25T04:23:15.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.147 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:15.148 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034048+0000", 2023-11-25T04:23:15.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0390000000000001e-06 2023-11-25T04:23:15.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.148 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:15.148 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034059+0000", 2023-11-25T04:23:15.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1182999999999999e-05 2023-11-25T04:23:15.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.148 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:15.148 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034067+0000", 2023-11-25T04:23:15.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1559999999999995e-06 2023-11-25T04:23:15.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:15.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035005+0000", 2023-11-25T04:23:15.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00093747399999999999 2023-11-25T04:23:15.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:15.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035077+0000", 2023-11-25T04:23:15.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2747e-05 2023-11-25T04:23:15.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:15.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035219+0000", 2023-11-25T04:23:15.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000141877 2023-11-25T04:23:15.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:15.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037810+0000", 2023-11-25T04:23:15.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002590752 2023-11-25T04:23:15.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:15.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044437+0000", 2023-11-25T04:23:15.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066270110000000004 2023-11-25T04:23:15.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.151 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:15.151 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044481+0000", 2023-11-25T04:23:15.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4106000000000001e-05 2023-11-25T04:23:15.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.151 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:15.151 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044505+0000", 2023-11-25T04:23:15.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4207e-05 2023-11-25T04:23:15.151 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:15.151 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:15.151 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:15.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.152 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:16 3.1d 3:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:15.152 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:15.152 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.078014715999998, 2023-11-25T04:23:15.152 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010413923, 2023-11-25T04:23:15.152 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:15.152 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:15.152 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:15.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.152 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:15.152 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:15.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:15.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:15.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:15.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:15.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:15.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:15.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4499999999999997e-07 2023-11-25T04:23:15.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:15.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034087+0000", 2023-11-25T04:23:15.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9770000000000002e-06 2023-11-25T04:23:15.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:15.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034094+0000", 2023-11-25T04:23:15.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8809999999999998e-06 2023-11-25T04:23:15.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:15.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034100+0000", 2023-11-25T04:23:15.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8440000000000003e-06 2023-11-25T04:23:15.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.155 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:15.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034993+0000", 2023-11-25T04:23:15.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00089336799999999996 2023-11-25T04:23:15.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.155 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:15.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035099+0000", 2023-11-25T04:23:15.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106037 2023-11-25T04:23:15.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.156 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:15.156 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035276+0000", 2023-11-25T04:23:15.156 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017631799999999999 2023-11-25T04:23:15.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.156 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:15.156 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037815+0000", 2023-11-25T04:23:15.156 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025395959999999999 2023-11-25T04:23:15.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.156 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:15.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044420+0000", 2023-11-25T04:23:15.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066045749999999997 2023-11-25T04:23:15.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.157 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:15.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044463+0000", 2023-11-25T04:23:15.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3205999999999999e-05 2023-11-25T04:23:15.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.157 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:15.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044497+0000", 2023-11-25T04:23:15.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3476000000000003e-05 2023-11-25T04:23:15.157 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:15.158 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:15.158 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:15.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.158 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:17 3.1d 3:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:15.158 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:15.158 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.077989507000002, 2023-11-25T04:23:15.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01127356, 2023-11-25T04:23:15.158 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:15.158 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:15.158 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:15.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:15.159 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:15.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:15.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.159 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:15.159 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:15.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:15.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.159 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:15.159 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:15.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1100000000000001e-07 2023-11-25T04:23:15.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.160 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.160 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:15.160 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034112+0000", 2023-11-25T04:23:15.160 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8600000000000003e-06 2023-11-25T04:23:15.160 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.160 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.160 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:15.160 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034120+0000", 2023-11-25T04:23:15.160 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7700000000000001e-06 2023-11-25T04:23:15.160 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.160 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.160 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:15.161 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034127+0000", 2023-11-25T04:23:15.161 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0999999999999998e-06 2023-11-25T04:23:15.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.161 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.161 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:15.161 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035601+0000", 2023-11-25T04:23:15.161 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014739410000000001 2023-11-25T04:23:15.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.161 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.161 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:15.161 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035685+0000", 2023-11-25T04:23:15.161 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3677999999999994e-05 2023-11-25T04:23:15.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.162 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:15.162 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035827+0000", 2023-11-25T04:23:15.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014232599999999999 2023-11-25T04:23:15.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.162 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:15.162 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038725+0000", 2023-11-25T04:23:15.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028982779999999998 2023-11-25T04:23:15.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.162 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:15.163 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045323+0000", 2023-11-25T04:23:15.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0065980780000000003 2023-11-25T04:23:15.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.163 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.163 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:15.163 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045354+0000", 2023-11-25T04:23:15.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0463e-05 2023-11-25T04:23:15.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.163 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.163 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:15.163 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045381+0000", 2023-11-25T04:23:15.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7654999999999999e-05 2023-11-25T04:23:15.164 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:15.164 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:15.164 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:15.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.164 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.164 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:22 3.e 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:15.164 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:15.164 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.077963094000001, 2023-11-25T04:23:15.164 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010133422, 2023-11-25T04:23:15.164 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:15.164 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:15.164 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:15.164 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.165 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:15.165 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:15.165 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:15.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.165 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.165 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:15.165 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:15.165 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:15.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.165 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.165 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:15.165 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034135+0000", 2023-11-25T04:23:15.165 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-07 2023-11-25T04:23:15.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.166 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:15.166 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034138+0000", 2023-11-25T04:23:15.166 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6009999999999999e-06 2023-11-25T04:23:15.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.166 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:15.166 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034145+0000", 2023-11-25T04:23:15.166 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1169999999999998e-06 2023-11-25T04:23:15.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.166 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:15.167 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034150+0000", 2023-11-25T04:23:15.167 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9080000000000003e-06 2023-11-25T04:23:15.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.167 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:15.167 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034209+0000", 2023-11-25T04:23:15.167 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8149000000000002e-05 2023-11-25T04:23:15.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.168 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:15.168 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034415+0000", 2023-11-25T04:23:15.168 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020637699999999999 2023-11-25T04:23:15.168 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.168 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.168 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:15.168 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034584+0000", 2023-11-25T04:23:15.168 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016870399999999999 2023-11-25T04:23:15.168 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.168 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.168 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:15.168 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.036979+0000", 2023-11-25T04:23:15.168 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023957509999999998 2023-11-25T04:23:15.169 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.169 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.169 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:15.169 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044161+0000", 2023-11-25T04:23:15.169 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0071811590000000003 2023-11-25T04:23:15.169 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.169 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.169 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:15.169 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044224+0000", 2023-11-25T04:23:15.169 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3454000000000006e-05 2023-11-25T04:23:15.169 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.170 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:15.170 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044268+0000", 2023-11-25T04:23:15.170 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3761999999999999e-05 2023-11-25T04:23:15.170 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:15.170 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:15.170 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:15.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.170 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:23 3.e 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:15.170 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:15.170 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.077931684999999, 2023-11-25T04:23:15.170 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010281676, 2023-11-25T04:23:15.171 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:15.171 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:15.171 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:15.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.171 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:15.171 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:15.171 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:15.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.171 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:15.171 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:15.171 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:15.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.172 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:15.172 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:15.172 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5799999999999999e-07 2023-11-25T04:23:15.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.172 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:15.172 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034169+0000", 2023-11-25T04:23:15.172 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1879999999999998e-06 2023-11-25T04:23:15.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.172 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:15.173 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034177+0000", 2023-11-25T04:23:15.173 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7020000000000001e-06 2023-11-25T04:23:15.173 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.173 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.173 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:15.173 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034184+0000", 2023-11-25T04:23:15.173 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3910000000000003e-06 2023-11-25T04:23:15.173 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.173 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.173 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:15.173 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034924+0000", 2023-11-25T04:23:15.173 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00074032999999999998 2023-11-25T04:23:15.173 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.173 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.174 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:15.174 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034991+0000", 2023-11-25T04:23:15.174 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7347000000000004e-05 2023-11-25T04:23:15.174 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.174 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.174 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:15.174 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035096+0000", 2023-11-25T04:23:15.174 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010488 2023-11-25T04:23:15.174 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.175 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.175 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:15.175 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037588+0000", 2023-11-25T04:23:15.175 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024918309999999999 2023-11-25T04:23:15.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.175 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.175 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:15.175 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044367+0000", 2023-11-25T04:23:15.175 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067787589999999997 2023-11-25T04:23:15.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.175 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.175 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:15.175 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044415+0000", 2023-11-25T04:23:15.176 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7803000000000003e-05 2023-11-25T04:23:15.176 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.176 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.176 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:15.176 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044447+0000", 2023-11-25T04:23:15.176 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2886999999999999e-05 2023-11-25T04:23:15.176 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:15.176 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:15.176 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:15.176 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.176 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.176 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:30 3.15 3:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:15.177 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:15.177 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.077873244999999, 2023-11-25T04:23:15.177 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010365070000000001, 2023-11-25T04:23:15.177 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:15.177 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:15.177 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:15.177 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.177 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:15.177 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:15.177 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:15.177 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.178 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.178 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:15.178 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:15.178 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:15.178 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.178 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.178 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:15.178 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034225+0000", 2023-11-25T04:23:15.178 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7400000000000002e-07 2023-11-25T04:23:15.178 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.178 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.179 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:15.179 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034229+0000", 2023-11-25T04:23:15.179 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3100000000000002e-06 2023-11-25T04:23:15.179 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.179 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.179 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:15.179 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034239+0000", 2023-11-25T04:23:15.179 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9140000000000003e-06 2023-11-25T04:23:15.179 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.179 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.179 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:15.179 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034246+0000", 2023-11-25T04:23:15.180 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0149999999999998e-06 2023-11-25T04:23:15.180 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.180 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.180 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:15.180 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035572+0000", 2023-11-25T04:23:15.180 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001325655 2023-11-25T04:23:15.180 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.180 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.180 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:15.180 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035730+0000", 2023-11-25T04:23:15.180 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015827899999999999 2023-11-25T04:23:15.180 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.180 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.181 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:15.181 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035857+0000", 2023-11-25T04:23:15.181 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012644699999999999 2023-11-25T04:23:15.181 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.181 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.181 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:15.181 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038737+0000", 2023-11-25T04:23:15.181 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002880077 2023-11-25T04:23:15.181 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.181 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.181 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:15.181 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044540+0000", 2023-11-25T04:23:15.182 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058034239999999997 2023-11-25T04:23:15.182 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.182 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.182 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:15.182 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044569+0000", 2023-11-25T04:23:15.182 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8388999999999999e-05 2023-11-25T04:23:15.182 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.182 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.182 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:15.182 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044589+0000", 2023-11-25T04:23:15.182 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0686000000000002e-05 2023-11-25T04:23:15.182 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:15.183 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:15.183 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:15.183 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.183 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.183 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:33 3.1b 3:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:15.183 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:15.183 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.077827747000001, 2023-11-25T04:23:15.183 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099754049999999997, 2023-11-25T04:23:15.183 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:15.183 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:23:15.183 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:15.183 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.184 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:15.184 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:15.184 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:15.185 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.185 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.185 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:15.185 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:15.185 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:15.185 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.185 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.185 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:15.185 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:15.185 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4799999999999998e-07 2023-11-25T04:23:15.185 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.185 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.185 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:15.186 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034273+0000", 2023-11-25T04:23:15.186 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7819999999999998e-06 2023-11-25T04:23:15.186 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.186 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.186 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:15.186 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034285+0000", 2023-11-25T04:23:15.186 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2058e-05 2023-11-25T04:23:15.186 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.186 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.186 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:15.186 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034291+0000", 2023-11-25T04:23:15.186 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0680000000000002e-06 2023-11-25T04:23:15.187 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.187 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.187 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:15.187 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034370+0000", 2023-11-25T04:23:15.187 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8452e-05 2023-11-25T04:23:15.187 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.187 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.187 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:15.187 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034477+0000", 2023-11-25T04:23:15.187 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107471 2023-11-25T04:23:15.187 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.187 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.187 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:15.188 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044245+0000", 2023-11-25T04:23:15.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0097679259999999997 2023-11-25T04:23:15.188 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:15.188 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:15.188 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:15.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.188 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:36 3.1a 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:15.188 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:15.188 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.077781074000001, 2023-11-25T04:23:15.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058179099999999999, 2023-11-25T04:23:15.188 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:15.189 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:23:15.189 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:15.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:15.189 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:15.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:15.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:15.189 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:15.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:15.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.190 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:15.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034317+0000", 2023-11-25T04:23:15.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1999999999999999e-07 2023-11-25T04:23:15.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.190 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:15.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034321+0000", 2023-11-25T04:23:15.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.233e-06 2023-11-25T04:23:15.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.190 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:15.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034337+0000", 2023-11-25T04:23:15.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5308999999999999e-05 2023-11-25T04:23:15.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.191 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:15.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034354+0000", 2023-11-25T04:23:15.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7411999999999999e-05 2023-11-25T04:23:15.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.191 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:15.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034465+0000", 2023-11-25T04:23:15.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011154699999999999 2023-11-25T04:23:15.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.192 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:15.192 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034560+0000", 2023-11-25T04:23:15.192 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4186999999999998e-05 2023-11-25T04:23:15.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.192 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:15.192 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.040134+0000", 2023-11-25T04:23:15.192 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055746019999999997 2023-11-25T04:23:15.192 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:15.192 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:15.192 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:15.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.193 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:9 3.10 e18/16 3:09401a81:::601.00000000:head v 18'4, mlcod=0'0)", 2023-11-25T04:23:15.193 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:15.193 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.068886765999999, 2023-11-25T04:23:15.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061641020000000003, 2023-11-25T04:23:15.193 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:15.193 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:15.193 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:15.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.193 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:15.193 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:15.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:15.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:15.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:15.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:15.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:15.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:15.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5300000000000001e-07 2023-11-25T04:23:15.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:15.195 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043222+0000", 2023-11-25T04:23:15.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.082e-05 2023-11-25T04:23:15.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.195 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:15.195 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043228+0000", 2023-11-25T04:23:15.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6699999999999997e-06 2023-11-25T04:23:15.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.195 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:15.195 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043236+0000", 2023-11-25T04:23:15.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8189999999999992e-06 2023-11-25T04:23:15.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.196 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:15.196 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043507+0000", 2023-11-25T04:23:15.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000271012 2023-11-25T04:23:15.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.196 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:15.196 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043530+0000", 2023-11-25T04:23:15.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2659000000000001e-05 2023-11-25T04:23:15.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.196 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:15.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049321+0000", 2023-11-25T04:23:15.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057915129999999999 2023-11-25T04:23:15.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:15.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049375+0000", 2023-11-25T04:23:15.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3455999999999998e-05 2023-11-25T04:23:15.197 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:15.197 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:15.197 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:15.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.197 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:1 3.14 e18/16 3:292cf221:::200.00000000:head v 18'3, mlcod=0'0)", 2023-11-25T04:23:15.198 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:15.198 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.068850052999998, 2023-11-25T04:23:15.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0060983599999999997, 2023-11-25T04:23:15.198 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:15.198 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:15.198 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:15.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:15.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:15.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:15.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:15.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:15.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:15.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:15.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043248+0000", 2023-11-25T04:23:15.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9399999999999995e-07 2023-11-25T04:23:15.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:15.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043252+0000", 2023-11-25T04:23:15.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1799999999999998e-06 2023-11-25T04:23:15.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.200 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:15.200 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043259+0000", 2023-11-25T04:23:15.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0049999999999998e-06 2023-11-25T04:23:15.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.200 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:15.200 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043265+0000", 2023-11-25T04:23:15.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0859999999999998e-06 2023-11-25T04:23:15.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:15.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043311+0000", 2023-11-25T04:23:15.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6082999999999999e-05 2023-11-25T04:23:15.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:15.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043330+0000", 2023-11-25T04:23:15.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8556e-05 2023-11-25T04:23:15.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:15.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049303+0000", 2023-11-25T04:23:15.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059733470000000004 2023-11-25T04:23:15.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.202 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:15.202 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049346+0000", 2023-11-25T04:23:15.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2608999999999998e-05 2023-11-25T04:23:15.202 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:15.202 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:15.202 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:15.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.202 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:3 3.14 e18/16 3:287034c4:::1.00000000.inode:head v 18'4, mlcod=0'0)", 2023-11-25T04:23:15.202 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:15.203 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.068333547000002, 2023-11-25T04:23:15.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057040090000000003, 2023-11-25T04:23:15.203 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:15.203 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:15.203 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:15.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:15.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:15.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:15.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:15.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:15.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:15.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.204 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:15.204 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:15.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9799999999999999e-07 2023-11-25T04:23:15.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.204 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:15.204 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043785+0000", 2023-11-25T04:23:15.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0721999999999999e-05 2023-11-25T04:23:15.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.205 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:15.205 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043793+0000", 2023-11-25T04:23:15.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5839999999999995e-06 2023-11-25T04:23:15.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.205 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:15.205 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043803+0000", 2023-11-25T04:23:15.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0729999999999994e-06 2023-11-25T04:23:15.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.205 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:15.205 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043843+0000", 2023-11-25T04:23:15.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0747999999999997e-05 2023-11-25T04:23:15.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:15.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043856+0000", 2023-11-25T04:23:15.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2697000000000001e-05 2023-11-25T04:23:15.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:15.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049426+0000", 2023-11-25T04:23:15.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055704409999999998 2023-11-25T04:23:15.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.207 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:15.207 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049468+0000", 2023-11-25T04:23:15.207 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1446000000000003e-05 2023-11-25T04:23:15.207 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:15.207 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:15.207 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:15.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.207 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "pg_update_log_missing(3.14 epoch 18/16 rep_tid 171 entries 18'5 (0'0) error 3:2e2fa760:::200.00000001:head by mds.0.3:32 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:23:15.207 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:15.207 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.068172583999999, 2023-11-25T04:23:15.207 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005556938, 2023-11-25T04:23:15.207 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:15.207 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2023-11-25T04:23:15.208 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:15.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.208 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:15.208 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:15.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:15.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.208 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:15.208 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:15.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:15.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.209 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.209 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:15.209 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:15.209 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.03e-07 2023-11-25T04:23:15.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.209 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.209 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:15.209 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043931+0000", 2023-11-25T04:23:15.209 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1079999999999998e-06 2023-11-25T04:23:15.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.209 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.210 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:15.210 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043944+0000", 2023-11-25T04:23:15.210 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2852e-05 2023-11-25T04:23:15.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.210 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:15.210 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043955+0000", 2023-11-25T04:23:15.210 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1037e-05 2023-11-25T04:23:15.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.210 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:15.210 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044088+0000", 2023-11-25T04:23:15.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013269599999999999 2023-11-25T04:23:15.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.211 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:15.211 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049482+0000", 2023-11-25T04:23:15.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053940419999999999 2023-11-25T04:23:15.211 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:15.211 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:15.211 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:15.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.211 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:10 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'5, mlcod=0'0)", 2023-11-25T04:23:15.211 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:15.212 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.068082556, 2023-11-25T04:23:15.212 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054493099999999997, 2023-11-25T04:23:15.212 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:15.212 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:15.212 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:15.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.212 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:15.212 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:15.212 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:15.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.212 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:15.212 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:15.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:15.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.213 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:15.213 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:15.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6100000000000002e-07 2023-11-25T04:23:15.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.213 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:15.213 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044019+0000", 2023-11-25T04:23:15.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.839e-06 2023-11-25T04:23:15.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.213 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:15.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044026+0000", 2023-11-25T04:23:15.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5869999999999998e-06 2023-11-25T04:23:15.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.214 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.214 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:15.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044031+0000", 2023-11-25T04:23:15.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4909999999999996e-06 2023-11-25T04:23:15.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.214 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.214 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:15.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044070+0000", 2023-11-25T04:23:15.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8829999999999999e-05 2023-11-25T04:23:15.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:15.215 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044081+0000", 2023-11-25T04:23:15.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1447999999999999e-05 2023-11-25T04:23:15.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:15.215 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049406+0000", 2023-11-25T04:23:15.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053246480000000004 2023-11-25T04:23:15.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:15.215 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049464+0000", 2023-11-25T04:23:15.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8205999999999998e-05 2023-11-25T04:23:15.216 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:15.216 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:15.216 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:15.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.216 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:11 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'6, mlcod=0'0)", 2023-11-25T04:23:15.216 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:15.216 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 28.067881291999999, 2023-11-25T04:23:15.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052673499999999996, 2023-11-25T04:23:15.216 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:15.216 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:15.217 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:15.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.217 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:15.217 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:15.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:15.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.217 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:15.217 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:15.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:15.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.217 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:15.218 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:15.218 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0199999999999998e-07 2023-11-25T04:23:15.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.218 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:15.218 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044220+0000", 2023-11-25T04:23:15.218 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7720000000000001e-06 2023-11-25T04:23:15.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.218 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:15.218 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044226+0000", 2023-11-25T04:23:15.218 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8579999999999996e-06 2023-11-25T04:23:15.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.219 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:15.219 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044232+0000", 2023-11-25T04:23:15.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1700000000000002e-06 2023-11-25T04:23:15.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.219 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:15.219 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044274+0000", 2023-11-25T04:23:15.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1254999999999997e-05 2023-11-25T04:23:15.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.219 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:15.220 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:23:15.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6880999999999999e-05 2023-11-25T04:23:15.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:15.220 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049466+0000", 2023-11-25T04:23:15.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051659219999999999 2023-11-25T04:23:15.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:15.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:15.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:15.220 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049483+0000", 2023-11-25T04:23:15.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.719e-05 2023-11-25T04:23:15.220 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:15.221 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:15.221 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:15.221 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:15.221 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:15.221 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:15.222 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:23:15.223 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:15.221+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:15.323 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:15.321+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:15.329 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:15.329 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:23:15.329 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2023-11-25T04:23:15.329 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:15.338 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:23:15.424 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:15.423+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:15.483 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:15.483 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:23:15.483 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2023-11-25T04:23:15.483 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2023-11-25T04:23:15.483 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:15.492 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:23:15.532 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:15.531+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:15.627 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:15.625+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:15.638 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:15.638 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2023-11-25T04:23:15.638 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2023-11-25T04:23:15.639 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2023-11-25T04:23:15.639 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.639 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:15.639 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:15.639 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 28.604670694999999, 2023-11-25T04:23:15.639 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01072943, 2023-11-25T04:23:15.639 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:15.639 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:15.639 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:15.639 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.639 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:15.639 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:15.640 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:15.640 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.640 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.640 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:15.640 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:15.640 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:15.640 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.640 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.640 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:15.640 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031829+0000", 2023-11-25T04:23:15.640 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.666e-06 2023-11-25T04:23:15.640 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.640 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.641 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:15.641 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031839+0000", 2023-11-25T04:23:15.641 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9490000000000008e-06 2023-11-25T04:23:15.641 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.641 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.641 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:15.641 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031857+0000", 2023-11-25T04:23:15.641 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8093000000000001e-05 2023-11-25T04:23:15.641 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.641 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.641 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:15.641 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031882+0000", 2023-11-25T04:23:15.641 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4294000000000002e-05 2023-11-25T04:23:15.642 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.642 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.642 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:15.642 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033130+0000", 2023-11-25T04:23:15.642 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001248264 2023-11-25T04:23:15.642 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.642 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.642 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:15.642 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033426+0000", 2023-11-25T04:23:15.642 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029620999999999998 2023-11-25T04:23:15.642 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.643 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.643 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:15.643 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033814+0000", 2023-11-25T04:23:15.643 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000387742 2023-11-25T04:23:15.643 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.643 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.643 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:15.643 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036793+0000", 2023-11-25T04:23:15.643 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002979598 2023-11-25T04:23:15.643 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.643 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.643 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:15.643 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042455+0000", 2023-11-25T04:23:15.643 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0056619260000000003 2023-11-25T04:23:15.644 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.644 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.644 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:15.644 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042502+0000", 2023-11-25T04:23:15.644 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7139999999999999e-05 2023-11-25T04:23:15.644 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.644 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.644 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:15.644 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042557+0000", 2023-11-25T04:23:15.645 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4548e-05 2023-11-25T04:23:15.645 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:15.645 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:15.645 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:15.645 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.645 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.645 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:15.645 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:15.645 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 28.603472159999999, 2023-11-25T04:23:15.645 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011505202000000001, 2023-11-25T04:23:15.645 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:15.645 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:15.645 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:15.646 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.646 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:15.646 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:15.646 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:15.646 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.646 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.646 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:15.646 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:15.646 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:15.646 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.646 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.647 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:15.647 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033027+0000", 2023-11-25T04:23:15.647 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0699999999999999e-06 2023-11-25T04:23:15.647 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.647 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.647 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:15.647 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033035+0000", 2023-11-25T04:23:15.647 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0830000000000004e-06 2023-11-25T04:23:15.647 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.647 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.647 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:15.647 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033059+0000", 2023-11-25T04:23:15.648 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3975e-05 2023-11-25T04:23:15.648 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.648 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.648 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:15.648 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033482+0000", 2023-11-25T04:23:15.648 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042253700000000001 2023-11-25T04:23:15.648 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.648 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.648 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:15.648 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034520+0000", 2023-11-25T04:23:15.649 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001037991 2023-11-25T04:23:15.649 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.649 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.649 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:15.649 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034611+0000", 2023-11-25T04:23:15.649 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1113999999999995e-05 2023-11-25T04:23:15.649 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.649 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.649 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:15.650 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034769+0000", 2023-11-25T04:23:15.650 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000157641 2023-11-25T04:23:15.650 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.650 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.650 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:15.650 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038292+0000", 2023-11-25T04:23:15.650 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035231160000000002 2023-11-25T04:23:15.650 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.650 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.650 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:15.651 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044440+0000", 2023-11-25T04:23:15.651 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0061486980000000002 2023-11-25T04:23:15.651 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.651 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.651 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:15.651 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044495+0000", 2023-11-25T04:23:15.651 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4339000000000003e-05 2023-11-25T04:23:15.651 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.651 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.651 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:15.651 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044531+0000", 2023-11-25T04:23:15.652 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6637999999999997e-05 2023-11-25T04:23:15.652 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:15.652 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:15.652 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:15.652 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.652 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.652 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:15.652 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:15.652 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 28.602997806000001, 2023-11-25T04:23:15.652 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0057634039999999997, 2023-11-25T04:23:15.652 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:15.652 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:15.653 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:15.653 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.653 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:15.653 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:15.653 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:15.653 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.653 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.653 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:15.653 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:15.653 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:15.653 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.653 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.654 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:15.654 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:15.654 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7900000000000007e-07 2023-11-25T04:23:15.654 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.654 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.654 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:15.654 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033507+0000", 2023-11-25T04:23:15.654 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8900000000000004e-06 2023-11-25T04:23:15.654 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.654 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.654 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:15.655 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033519+0000", 2023-11-25T04:23:15.655 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1756000000000001e-05 2023-11-25T04:23:15.655 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.655 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.655 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:15.655 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033527+0000", 2023-11-25T04:23:15.655 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3610000000000001e-06 2023-11-25T04:23:15.655 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.655 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.655 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:15.655 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034451+0000", 2023-11-25T04:23:15.655 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00092372400000000001 2023-11-25T04:23:15.656 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.656 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.656 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:15.656 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034809+0000", 2023-11-25T04:23:15.656 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00035757699999999998 2023-11-25T04:23:15.656 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.656 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.656 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:23:15.656 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034965+0000", 2023-11-25T04:23:15.656 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015638699999999999 2023-11-25T04:23:15.656 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.656 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.657 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:15.657 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038299+0000", 2023-11-25T04:23:15.657 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0033336559999999999 2023-11-25T04:23:15.657 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.657 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.657 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:15.657 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039169+0000", 2023-11-25T04:23:15.657 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00087066400000000005 2023-11-25T04:23:15.657 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.657 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.657 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:15.657 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039226+0000", 2023-11-25T04:23:15.657 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6511999999999997e-05 2023-11-25T04:23:15.658 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.658 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.660 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:15.660 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039264+0000", 2023-11-25T04:23:15.660 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7898000000000002e-05 2023-11-25T04:23:15.661 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:15.661 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:15.661 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:15.661 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.661 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.661 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:25 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:15.661 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:15.661 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 28.602919222000001, 2023-11-25T04:23:15.661 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096656169999999996, 2023-11-25T04:23:15.661 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:15.661 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:15.661 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:15.661 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.662 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:15.662 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:15.662 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:15.662 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.662 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.662 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:15.662 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:15.662 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:15.662 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.662 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.662 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:15.662 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033580+0000", 2023-11-25T04:23:15.662 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8199999999999999e-07 2023-11-25T04:23:15.663 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.663 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.663 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:15.663 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033585+0000", 2023-11-25T04:23:15.663 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3680000000000001e-06 2023-11-25T04:23:15.663 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.663 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.663 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:15.663 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033600+0000", 2023-11-25T04:23:15.663 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4331000000000001e-05 2023-11-25T04:23:15.663 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.663 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.663 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:15.664 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033606+0000", 2023-11-25T04:23:15.664 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5780000000000004e-06 2023-11-25T04:23:15.664 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.664 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.664 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:15.664 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035747+0000", 2023-11-25T04:23:15.664 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0021406530000000002 2023-11-25T04:23:15.664 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.664 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.664 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:15.664 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035901+0000", 2023-11-25T04:23:15.664 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000153931 2023-11-25T04:23:15.664 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.665 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.665 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:23:15.665 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036193+0000", 2023-11-25T04:23:15.665 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029211299999999999 2023-11-25T04:23:15.665 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.665 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.665 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:15.665 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038655+0000", 2023-11-25T04:23:15.665 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0024621640000000002 2023-11-25T04:23:15.665 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.665 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.665 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:15.665 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043130+0000", 2023-11-25T04:23:15.666 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.004474698 2023-11-25T04:23:15.666 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.666 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.666 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:15.666 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043204+0000", 2023-11-25T04:23:15.666 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4091999999999995e-05 2023-11-25T04:23:15.666 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.666 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.666 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:15.666 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043245+0000", 2023-11-25T04:23:15.666 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0907e-05 2023-11-25T04:23:15.666 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:15.666 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:15.667 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:15.667 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.667 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.667 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:26 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:15.667 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:15.667 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 28.602883397999999, 2023-11-25T04:23:15.667 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011086945000000001, 2023-11-25T04:23:15.667 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:15.667 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:15.667 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:15.667 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.667 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:15.667 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:15.667 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:15.668 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.668 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.668 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:15.668 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:15.668 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:15.668 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.668 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.668 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:15.668 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:15.668 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1699999999999998e-07 2023-11-25T04:23:15.668 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.668 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.668 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:15.669 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033627+0000", 2023-11-25T04:23:15.669 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.163e-05 2023-11-25T04:23:15.669 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.669 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.669 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:15.669 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033636+0000", 2023-11-25T04:23:15.669 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6859999999999995e-06 2023-11-25T04:23:15.669 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.669 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.669 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:15.669 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033642+0000", 2023-11-25T04:23:15.669 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3149999999999997e-06 2023-11-25T04:23:15.669 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.670 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.670 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:15.670 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035414+0000", 2023-11-25T04:23:15.670 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0017716380000000001 2023-11-25T04:23:15.670 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.670 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.670 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:15.670 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035539+0000", 2023-11-25T04:23:15.670 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000125477 2023-11-25T04:23:15.670 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.670 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.670 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:15.670 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035672+0000", 2023-11-25T04:23:15.671 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000133316 2023-11-25T04:23:15.671 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.671 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.671 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:15.671 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038649+0000", 2023-11-25T04:23:15.671 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029764679999999999 2023-11-25T04:23:15.671 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.671 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.671 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:15.671 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044578+0000", 2023-11-25T04:23:15.671 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0059293510000000002 2023-11-25T04:23:15.671 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.671 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.672 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:15.672 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044651+0000", 2023-11-25T04:23:15.672 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2513000000000002e-05 2023-11-25T04:23:15.672 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.672 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.672 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:15.672 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044702+0000", 2023-11-25T04:23:15.672 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1034e-05 2023-11-25T04:23:15.672 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:15.672 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:15.672 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:15.672 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.672 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.673 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:29 3.b 3:d0630e4c:::mds0_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:15.673 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:15.673 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 28.602845477999999, 2023-11-25T04:23:15.673 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010980227, 2023-11-25T04:23:15.673 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:15.673 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:15.673 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:15.673 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.673 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:15.673 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:15.673 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:15.673 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.673 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.673 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:15.674 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:15.674 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:15.674 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.674 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.674 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:15.674 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:15.674 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3499999999999996e-07 2023-11-25T04:23:15.674 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.674 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.674 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:15.674 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033658+0000", 2023-11-25T04:23:15.674 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1280000000000001e-06 2023-11-25T04:23:15.675 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.675 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.675 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:15.675 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033667+0000", 2023-11-25T04:23:15.675 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1430000000000004e-06 2023-11-25T04:23:15.675 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.675 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.675 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:15.675 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033678+0000", 2023-11-25T04:23:15.675 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0906e-05 2023-11-25T04:23:15.675 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.675 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.675 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:15.676 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034456+0000", 2023-11-25T04:23:15.676 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00077833899999999996 2023-11-25T04:23:15.676 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.676 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.676 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:15.676 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034575+0000", 2023-11-25T04:23:15.676 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011926000000000001 2023-11-25T04:23:15.676 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.676 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.676 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:15.676 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035001+0000", 2023-11-25T04:23:15.676 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042550999999999998 2023-11-25T04:23:15.676 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.677 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.677 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:15.677 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.037923+0000", 2023-11-25T04:23:15.677 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029219670000000001 2023-11-25T04:23:15.677 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.677 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.677 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:15.677 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044538+0000", 2023-11-25T04:23:15.677 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0066152349999999997 2023-11-25T04:23:15.677 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.677 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.677 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:15.677 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044594+0000", 2023-11-25T04:23:15.678 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5593000000000001e-05 2023-11-25T04:23:15.678 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.678 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.678 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:15.678 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044633+0000", 2023-11-25T04:23:15.678 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9511000000000002e-05 2023-11-25T04:23:15.678 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:15.678 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:15.678 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:15.678 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.678 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.678 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:34 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:15.678 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:15.679 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 28.602595364999999, 2023-11-25T04:23:15.679 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010920445000000001, 2023-11-25T04:23:15.679 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:15.679 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:15.679 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:15.679 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.679 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:15.679 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:15.679 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:15.679 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.679 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.679 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:15.679 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:15.680 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:15.680 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.680 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.680 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:15.680 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033904+0000", 2023-11-25T04:23:15.680 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6599999999999994e-07 2023-11-25T04:23:15.680 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.680 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.680 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:15.680 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033909+0000", 2023-11-25T04:23:15.680 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5639999999999996e-06 2023-11-25T04:23:15.680 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.680 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.680 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:15.681 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033923+0000", 2023-11-25T04:23:15.681 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3028e-05 2023-11-25T04:23:15.681 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.681 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.681 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:15.681 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034043+0000", 2023-11-25T04:23:15.681 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000120328 2023-11-25T04:23:15.681 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.681 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.681 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:15.681 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035576+0000", 2023-11-25T04:23:15.681 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015333720000000001 2023-11-25T04:23:15.681 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.682 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.682 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:15.682 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035699+0000", 2023-11-25T04:23:15.682 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000123001 2023-11-25T04:23:15.682 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.682 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.682 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:15.682 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044823+0000", 2023-11-25T04:23:15.682 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0091241859999999994 2023-11-25T04:23:15.682 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:15.682 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:15.682 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:15.682 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.683 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.683 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:35 3.1f 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:15.683 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:15.683 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 28.601291159999999, 2023-11-25T04:23:15.683 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096164779999999995, 2023-11-25T04:23:15.683 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:15.683 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:15.683 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:15.683 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.683 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:15.683 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:15.683 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:15.683 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.684 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.684 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:15.684 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:15.684 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:15.684 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.684 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.684 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:15.684 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035208+0000", 2023-11-25T04:23:15.684 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2699999999999998e-07 2023-11-25T04:23:15.684 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.684 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.684 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:15.684 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035213+0000", 2023-11-25T04:23:15.684 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.6070000000000001e-06 2023-11-25T04:23:15.685 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.685 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.685 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:15.685 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035350+0000", 2023-11-25T04:23:15.685 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013693599999999999 2023-11-25T04:23:15.685 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.685 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.685 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:15.685 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035358+0000", 2023-11-25T04:23:15.685 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0099999999999995e-06 2023-11-25T04:23:15.685 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.685 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.685 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:15.686 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036260+0000", 2023-11-25T04:23:15.686 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00090240100000000005 2023-11-25T04:23:15.686 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.686 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.686 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:15.686 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036517+0000", 2023-11-25T04:23:15.686 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00025736299999999998 2023-11-25T04:23:15.686 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.686 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.686 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:15.686 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044824+0000", 2023-11-25T04:23:15.686 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0083063340000000003 2023-11-25T04:23:15.686 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:15.687 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:15.687 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:15.687 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.687 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.687 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:37 3.1e 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:15.687 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:15.687 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 28.601124473999999, 2023-11-25T04:23:15.687 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0078322640000000002, 2023-11-25T04:23:15.687 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:15.687 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:15.687 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:15.687 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.688 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:15.688 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:15.688 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:15.688 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.688 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.688 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:15.688 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:15.688 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:15.688 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.688 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.688 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:15.688 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035375+0000", 2023-11-25T04:23:15.688 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9100000000000003e-07 2023-11-25T04:23:15.689 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.689 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.689 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:15.689 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035379+0000", 2023-11-25T04:23:15.689 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9430000000000002e-06 2023-11-25T04:23:15.689 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.689 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.689 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:15.689 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035388+0000", 2023-11-25T04:23:15.689 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2760000000000006e-06 2023-11-25T04:23:15.689 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.689 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.689 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:15.690 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035396+0000", 2023-11-25T04:23:15.690 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8480000000000001e-06 2023-11-25T04:23:15.690 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.690 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.690 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:15.690 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035452+0000", 2023-11-25T04:23:15.690 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5831000000000001e-05 2023-11-25T04:23:15.690 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.690 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.690 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:15.690 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035934+0000", 2023-11-25T04:23:15.690 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00048265599999999998 2023-11-25T04:23:15.690 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.691 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.691 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:15.691 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043206+0000", 2023-11-25T04:23:15.691 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.007271919 2023-11-25T04:23:15.691 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:15.691 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:15.691 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:15.691 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.691 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.691 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing(3.1a epoch 18/16 rep_tid 3 entries 18'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:36 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:23:15.691 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:15.691 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 28.599848888, 2023-11-25T04:23:15.691 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002073414, 2023-11-25T04:23:15.692 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:15.692 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:15.692 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:15.692 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.692 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:15.692 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:15.692 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:15.692 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.692 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.692 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:15.693 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:15.693 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:15.693 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.693 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.693 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:15.693 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036650+0000", 2023-11-25T04:23:15.693 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7300000000000002e-07 2023-11-25T04:23:15.693 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.693 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.693 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:15.693 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036659+0000", 2023-11-25T04:23:15.693 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.7499999999999992e-06 2023-11-25T04:23:15.693 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.693 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.694 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:15.694 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036673+0000", 2023-11-25T04:23:15.694 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4917e-05 2023-11-25T04:23:15.694 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.694 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.694 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:15.694 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036687+0000", 2023-11-25T04:23:15.694 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3883000000000001e-05 2023-11-25T04:23:15.694 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.694 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.694 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:15.695 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036781+0000", 2023-11-25T04:23:15.695 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3219000000000003e-05 2023-11-25T04:23:15.695 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.695 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.695 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:15.695 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038723+0000", 2023-11-25T04:23:15.695 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019422720000000001 2023-11-25T04:23:15.695 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:15.695 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:15.695 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:15.695 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.695 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.695 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:28 3.13 e18/16 3:cd5a64a3:::100.00000000:head v 18'1, mlcod=0'0)", 2023-11-25T04:23:15.696 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:15.696 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 28.59618773, 2023-11-25T04:23:15.696 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001850546, 2023-11-25T04:23:15.696 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:15.696 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:15.696 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:15.696 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.696 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:15.696 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:15.696 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:15.696 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.696 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.696 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:15.697 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:15.697 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:15.697 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.697 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.697 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:15.697 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:15.697 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7800000000000002e-07 2023-11-25T04:23:15.697 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.697 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.697 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:15.697 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040331+0000", 2023-11-25T04:23:15.697 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0449999999999999e-05 2023-11-25T04:23:15.697 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.698 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.698 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:15.698 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040339+0000", 2023-11-25T04:23:15.698 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0709999999999995e-06 2023-11-25T04:23:15.698 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.699 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.699 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:15.699 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040347+0000", 2023-11-25T04:23:15.699 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7079999999999997e-06 2023-11-25T04:23:15.699 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.699 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.699 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:15.699 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040410+0000", 2023-11-25T04:23:15.699 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2453000000000007e-05 2023-11-25T04:23:15.700 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.700 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.700 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:15.700 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040422+0000", 2023-11-25T04:23:15.700 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2429000000000001e-05 2023-11-25T04:23:15.700 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.700 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.700 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:15.700 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042129+0000", 2023-11-25T04:23:15.700 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001707266 2023-11-25T04:23:15.700 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.700 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.700 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:15.701 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042161+0000", 2023-11-25T04:23:15.701 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1791000000000001e-05 2023-11-25T04:23:15.701 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:15.701 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:15.701 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:15.701 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.701 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.701 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:27 3.13 e18/16 3:c91d4a1d:::mds0_inotable:head v 18'2, mlcod=0'0)", 2023-11-25T04:23:15.701 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:15.701 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 28.595984207000001, 2023-11-25T04:23:15.701 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0016886430000000001, 2023-11-25T04:23:15.701 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:15.701 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:15.702 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:15.702 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.702 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:15.702 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:15.702 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:15.702 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.702 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.702 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:15.702 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:15.702 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:15.702 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.702 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.702 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:15.703 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:15.703 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3999999999999997e-07 2023-11-25T04:23:15.703 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.703 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.703 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:15.703 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040520+0000", 2023-11-25T04:23:15.703 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2989999999999996e-06 2023-11-25T04:23:15.703 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.703 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.703 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:15.703 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040527+0000", 2023-11-25T04:23:15.703 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5730000000000002e-06 2023-11-25T04:23:15.704 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.704 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.704 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:15.704 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040535+0000", 2023-11-25T04:23:15.704 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8630000000000008e-06 2023-11-25T04:23:15.704 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.704 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.704 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:15.704 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040683+0000", 2023-11-25T04:23:15.704 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000148002 2023-11-25T04:23:15.704 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.704 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.704 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:15.705 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040700+0000", 2023-11-25T04:23:15.705 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7065e-05 2023-11-25T04:23:15.705 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.705 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.705 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:15.705 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042163+0000", 2023-11-25T04:23:15.705 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001462767 2023-11-25T04:23:15.705 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.705 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.705 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:15.705 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042203+0000", 2023-11-25T04:23:15.705 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9734000000000003e-05 2023-11-25T04:23:15.706 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:15.706 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:15.706 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:15.706 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.706 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.706 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:20 3.1c e18/16 3:3ed09add:::607.00000000:head v 18'1, mlcod=0'0)", 2023-11-25T04:23:15.706 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:15.706 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 28.595949166, 2023-11-25T04:23:15.706 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015271, 2023-11-25T04:23:15.706 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:15.706 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:15.706 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:15.706 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.707 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:15.707 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:15.707 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:15.707 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.707 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.707 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:15.707 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:15.707 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:15.707 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.707 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.707 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:15.707 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:15.707 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4799999999999999e-07 2023-11-25T04:23:15.708 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.708 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.708 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:15.708 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040561+0000", 2023-11-25T04:23:15.708 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1800000000000001e-05 2023-11-25T04:23:15.708 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.708 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.708 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:15.708 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040568+0000", 2023-11-25T04:23:15.708 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7730000000000002e-06 2023-11-25T04:23:15.708 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.708 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.709 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:15.709 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040587+0000", 2023-11-25T04:23:15.709 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8691000000000001e-05 2023-11-25T04:23:15.709 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.709 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.709 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:15.709 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040638+0000", 2023-11-25T04:23:15.709 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1068e-05 2023-11-25T04:23:15.709 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.709 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.709 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:15.709 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040670+0000", 2023-11-25T04:23:15.709 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2125999999999997e-05 2023-11-25T04:23:15.710 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.710 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.710 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:15.710 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042032+0000", 2023-11-25T04:23:15.710 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001361821 2023-11-25T04:23:15.710 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.710 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.710 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:15.710 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042076+0000", 2023-11-25T04:23:15.710 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4472999999999999e-05 2023-11-25T04:23:15.710 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:15.710 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:15.710 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:15.711 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.711 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.711 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:21 3.1c e18/16 3:3ed09add:::607.00000000:head v 18'2, mlcod=0'0)", 2023-11-25T04:23:15.711 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:15.711 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 28.595714440999998, 2023-11-25T04:23:15.711 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0013228160000000001, 2023-11-25T04:23:15.711 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:15.711 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:15.711 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:15.711 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.711 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:15.711 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:15.711 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:15.712 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.712 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.712 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:15.712 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:15.712 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:15.712 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.712 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.712 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:15.712 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:15.712 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9299999999999999e-07 2023-11-25T04:23:15.712 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.712 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.712 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:15.713 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040790+0000", 2023-11-25T04:23:15.713 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5770000000000001e-06 2023-11-25T04:23:15.713 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.713 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.713 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:15.713 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040799+0000", 2023-11-25T04:23:15.713 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6439999999999995e-06 2023-11-25T04:23:15.713 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.713 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.713 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:15.713 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040806+0000", 2023-11-25T04:23:15.714 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0180000000000004e-06 2023-11-25T04:23:15.714 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.714 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.714 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:15.714 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040907+0000", 2023-11-25T04:23:15.714 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000100096 2023-11-25T04:23:15.714 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.714 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.714 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:15.715 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040925+0000", 2023-11-25T04:23:15.715 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8536999999999999e-05 2023-11-25T04:23:15.715 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.715 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.715 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:15.715 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042079+0000", 2023-11-25T04:23:15.715 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00115358 2023-11-25T04:23:15.715 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.715 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.715 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:15.716 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042107+0000", 2023-11-25T04:23:15.716 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7971e-05 2023-11-25T04:23:15.716 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:15.716 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:15.716 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:15.716 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.716 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.716 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop_reply(mds.0.3:4 3.1f e18/16 ondisk, result = 0)", 2023-11-25T04:23:15.716 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:15.716 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 28.594375051, 2023-11-25T04:23:15.716 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00044092900000000001, 2023-11-25T04:23:15.716 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:15.716 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:15.717 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:15.717 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.717 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:15.717 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:15.717 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:15.717 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.717 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.717 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:15.717 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:15.717 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:15.717 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.717 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.717 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:15.718 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042124+0000", 2023-11-25T04:23:15.718 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1800000000000005e-07 2023-11-25T04:23:15.718 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.718 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.718 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:15.718 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042128+0000", 2023-11-25T04:23:15.718 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2289999999999997e-06 2023-11-25T04:23:15.718 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.718 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.718 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:15.718 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042138+0000", 2023-11-25T04:23:15.718 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9729999999999992e-06 2023-11-25T04:23:15.719 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.719 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.719 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:15.719 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042145+0000", 2023-11-25T04:23:15.719 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.336e-06 2023-11-25T04:23:15.719 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.719 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.719 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:15.719 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042441+0000", 2023-11-25T04:23:15.719 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029650500000000002 2023-11-25T04:23:15.719 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.719 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.719 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:15.720 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042448+0000", 2023-11-25T04:23:15.720 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4100000000000002e-06 2023-11-25T04:23:15.720 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.720 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.720 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:15.720 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042564+0000", 2023-11-25T04:23:15.720 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000115758 2023-11-25T04:23:15.720 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:15.720 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:15.720 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:15.720 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.720 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.720 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing_reply(3.1f epoch 18/16 rep_tid 8 lcod 18'4)", 2023-11-25T04:23:15.721 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:15.721 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 28.592049638999999, 2023-11-25T04:23:15.721 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00038154699999999998, 2023-11-25T04:23:15.721 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:15.721 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:15.721 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:15.721 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.721 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:15.721 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:15.721 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:15.721 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.721 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.721 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:15.722 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:15.722 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:15.722 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.722 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.722 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:15.722 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:15.722 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2700000000000001e-07 2023-11-25T04:23:15.722 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.722 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.722 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:15.722 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044451+0000", 2023-11-25T04:23:15.722 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7460000000000001e-06 2023-11-25T04:23:15.723 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.723 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.723 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:15.723 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044455+0000", 2023-11-25T04:23:15.723 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5809999999999998e-06 2023-11-25T04:23:15.723 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.723 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.723 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:15.723 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044458+0000", 2023-11-25T04:23:15.723 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0170000000000001e-06 2023-11-25T04:23:15.723 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.723 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.723 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:15.724 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044734+0000", 2023-11-25T04:23:15.724 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00027582799999999999 2023-11-25T04:23:15.724 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.724 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.724 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:15.724 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044830+0000", 2023-11-25T04:23:15.724 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6248000000000004e-05 2023-11-25T04:23:15.724 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:15.724 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:15.724 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:15.724 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.724 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.724 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing_reply(3.b epoch 18/16 rep_tid 5 lcod 18'2)", 2023-11-25T04:23:15.725 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:15.725 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 28.592012978, 2023-11-25T04:23:15.725 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000344488, 2023-11-25T04:23:15.725 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:15.725 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:15.725 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:15.725 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.725 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:15.725 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:15.725 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:15.725 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.726 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.726 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:15.726 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:15.726 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:15.726 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.726 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.726 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:15.727 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:15.727 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.55e-07 2023-11-25T04:23:15.727 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.727 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.727 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:15.727 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044487+0000", 2023-11-25T04:23:15.727 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2470000000000001e-06 2023-11-25T04:23:15.727 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.727 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.727 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:15.727 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044491+0000", 2023-11-25T04:23:15.727 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9659999999999998e-06 2023-11-25T04:23:15.727 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.728 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.728 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:15.728 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044494+0000", 2023-11-25T04:23:15.728 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2310000000000001e-06 2023-11-25T04:23:15.728 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.728 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.728 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:15.728 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044756+0000", 2023-11-25T04:23:15.728 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00026172800000000003 2023-11-25T04:23:15.728 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.728 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.728 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:15.729 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044830+0000", 2023-11-25T04:23:15.729 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4160999999999997e-05 2023-11-25T04:23:15.729 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:15.729 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:15.729 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:15.729 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.729 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.729 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:39 3.6 3:654134d2:::mds0_openfiles.0:head [omap-clear,omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2023-11-25T04:23:15.729 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:15.729 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 23.532186935999999, 2023-11-25T04:23:15.729 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001788658, 2023-11-25T04:23:15.729 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:15.730 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:15.730 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:15.730 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.730 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:15.730 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:15.730 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:15.730 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.730 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.730 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:15.730 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:15.730 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:15.730 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.730 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.731 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:15.731 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104314+0000", 2023-11-25T04:23:15.731 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3099999999999999e-06 2023-11-25T04:23:15.731 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.731 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.731 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:15.731 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104325+0000", 2023-11-25T04:23:15.731 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1511999999999999e-05 2023-11-25T04:23:15.731 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.731 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.731 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:15.731 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104362+0000", 2023-11-25T04:23:15.731 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6903000000000002e-05 2023-11-25T04:23:15.732 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.732 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.732 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:15.732 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104387+0000", 2023-11-25T04:23:15.732 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4766e-05 2023-11-25T04:23:15.732 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.732 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.732 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:15.732 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104465+0000", 2023-11-25T04:23:15.732 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8119000000000002e-05 2023-11-25T04:23:15.732 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.732 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.732 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:15.733 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104557+0000", 2023-11-25T04:23:15.733 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2496000000000003e-05 2023-11-25T04:23:15.733 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.733 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.733 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:15.733 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104646+0000", 2023-11-25T04:23:15.733 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8818000000000002e-05 2023-11-25T04:23:15.733 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.733 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.733 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:15.733 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105201+0000", 2023-11-25T04:23:15.733 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00055506000000000002 2023-11-25T04:23:15.733 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.734 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.734 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:15.734 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105994+0000", 2023-11-25T04:23:15.734 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00079294700000000001 2023-11-25T04:23:15.734 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.734 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.734 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:15.734 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.106055+0000", 2023-11-25T04:23:15.734 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0566000000000003e-05 2023-11-25T04:23:15.734 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.734 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.734 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:15.734 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.106100+0000", 2023-11-25T04:23:15.735 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5160999999999997e-05 2023-11-25T04:23:15.735 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:15.735 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:15.735 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:15.735 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.735 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.735 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:43 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2023-11-25T04:23:15.735 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:15.735 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 8.5319619400000004, 2023-11-25T04:23:15.735 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001751166, 2023-11-25T04:23:15.735 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:15.735 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:15.735 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:15.736 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.736 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:15.736 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:15.736 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:15.736 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.736 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.736 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:15.736 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:15.736 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:15.736 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.736 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.736 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:15.737 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104539+0000", 2023-11-25T04:23:15.737 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9529999999999999e-06 2023-11-25T04:23:15.737 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.737 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.737 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:15.737 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104557+0000", 2023-11-25T04:23:15.737 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7464000000000001e-05 2023-11-25T04:23:15.737 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.737 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.737 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:15.737 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104590+0000", 2023-11-25T04:23:15.737 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2966000000000003e-05 2023-11-25T04:23:15.737 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.738 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.738 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:15.738 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104608+0000", 2023-11-25T04:23:15.738 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8193e-05 2023-11-25T04:23:15.738 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.738 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.738 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:15.738 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104767+0000", 2023-11-25T04:23:15.738 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015882 2023-11-25T04:23:15.738 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.738 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.738 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:15.739 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104909+0000", 2023-11-25T04:23:15.739 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014244999999999999 2023-11-25T04:23:15.739 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.739 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.739 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:15.739 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.105059+0000", 2023-11-25T04:23:15.739 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014964399999999999 2023-11-25T04:23:15.739 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.739 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.739 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:15.739 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.105701+0000", 2023-11-25T04:23:15.739 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00064201099999999995 2023-11-25T04:23:15.739 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.740 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.740 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:15.740 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106233+0000", 2023-11-25T04:23:15.740 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00053178500000000001 2023-11-25T04:23:15.740 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.740 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.740 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:15.740 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106265+0000", 2023-11-25T04:23:15.740 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1986000000000002e-05 2023-11-25T04:23:15.740 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.740 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.740 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:15.741 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106288+0000", 2023-11-25T04:23:15.741 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2894000000000001e-05 2023-11-25T04:23:15.741 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:15.741 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:15.741 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:15.741 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.741 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.741 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:45 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e23)", 2023-11-25T04:23:15.741 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:23:15.741 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.5317639000000001, 2023-11-25T04:23:15.741 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00098423299999999994, 2023-11-25T04:23:15.741 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:15.742 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:15.742 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:15.742 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.742 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:15.742 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:23:15.742 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:15.742 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.742 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.742 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:15.742 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:23:15.742 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:15.742 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.742 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.743 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:15.743 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104737+0000", 2023-11-25T04:23:15.743 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3309999999999998e-06 2023-11-25T04:23:15.743 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.743 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.743 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:15.743 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104748+0000", 2023-11-25T04:23:15.743 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1080999999999999e-05 2023-11-25T04:23:15.743 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.743 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.743 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:15.743 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104771+0000", 2023-11-25T04:23:15.743 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2707000000000001e-05 2023-11-25T04:23:15.744 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.744 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.744 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:15.744 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104800+0000", 2023-11-25T04:23:15.744 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9595e-05 2023-11-25T04:23:15.744 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.744 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.744 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:15.744 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104890+0000", 2023-11-25T04:23:15.744 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.9808999999999993e-05 2023-11-25T04:23:15.744 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.744 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.744 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:15.745 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105004+0000", 2023-11-25T04:23:15.745 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000114508 2023-11-25T04:23:15.745 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.745 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.745 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:15.745 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105634+0000", 2023-11-25T04:23:15.745 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00062985799999999996 2023-11-25T04:23:15.745 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.745 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.745 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:15.745 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105671+0000", 2023-11-25T04:23:15.745 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6640000000000002e-05 2023-11-25T04:23:15.746 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:15.746 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:15.746 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:15.746 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105719+0000", 2023-11-25T04:23:15.746 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7704000000000003e-05 2023-11-25T04:23:15.746 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:15.746 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:15.746 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:15.746 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:15.746 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:15.746 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:15.748 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:23:15.748 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:15.746+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:15.848 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:15.847+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:15.858 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:15.858 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:23:15.858 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2023-11-25T04:23:15.858 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:15.867 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:23:15.949 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:15.948+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:15.971 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:15.971 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:23:15.972 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2023-11-25T04:23:15.972 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2023-11-25T04:23:15.972 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:15.980 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:23:16.064 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:16.063+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:16.105 DEBUG:teuthology.orchestra.console.smithi076:pexpect command: ipmitool -H smithi076.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-25T04:23:16.127 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:16.128 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2023-11-25T04:23:16.128 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2023-11-25T04:23:16.128 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2023-11-25T04:23:16.128 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.128 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:155 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'109, mlcod=11'105)", 2023-11-25T04:23:16.128 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:16.128 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 38.339666655999999, 2023-11-25T04:23:16.128 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061719019999999999, 2023-11-25T04:23:16.128 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:16.128 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:16.128 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:16.128 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.129 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:16.129 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:16.129 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:16.129 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.130 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.130 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:16.130 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:16.130 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:16.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.130 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.130 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:16.130 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784884+0000", 2023-11-25T04:23:16.130 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3099999999999999e-07 2023-11-25T04:23:16.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:16.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784942+0000", 2023-11-25T04:23:16.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8570000000000003e-05 2023-11-25T04:23:16.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:16.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784949+0000", 2023-11-25T04:23:16.132 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0310000000000001e-06 2023-11-25T04:23:16.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.132 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:16.132 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784953+0000", 2023-11-25T04:23:16.132 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6320000000000001e-06 2023-11-25T04:23:16.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.133 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:16.133 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784993+0000", 2023-11-25T04:23:16.133 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9737000000000003e-05 2023-11-25T04:23:16.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.133 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.133 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:16.133 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785011+0000", 2023-11-25T04:23:16.133 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8349999999999999e-05 2023-11-25T04:23:16.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.133 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.133 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:16.133 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791024+0000", 2023-11-25T04:23:16.134 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006012997 2023-11-25T04:23:16.134 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.134 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.134 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:16.134 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791055+0000", 2023-11-25T04:23:16.134 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1254000000000003e-05 2023-11-25T04:23:16.134 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:16.134 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:16.134 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:16.134 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.134 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.134 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:156 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'110, mlcod=11'105)", 2023-11-25T04:23:16.134 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:16.135 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 38.339080607, 2023-11-25T04:23:16.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005616116, 2023-11-25T04:23:16.135 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:16.135 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:16.135 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:16.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.135 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:16.135 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:16.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:16.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.135 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:16.135 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:16.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:16.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:16.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785470+0000", 2023-11-25T04:23:16.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0699999999999998e-07 2023-11-25T04:23:16.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:16.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785474+0000", 2023-11-25T04:23:16.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.048e-06 2023-11-25T04:23:16.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:16.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785480+0000", 2023-11-25T04:23:16.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7729999999999996e-06 2023-11-25T04:23:16.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:16.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785483+0000", 2023-11-25T04:23:16.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1970000000000001e-06 2023-11-25T04:23:16.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:16.138 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787936+0000", 2023-11-25T04:23:16.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024529040000000001 2023-11-25T04:23:16.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.138 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:16.138 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787947+0000", 2023-11-25T04:23:16.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1669e-05 2023-11-25T04:23:16.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.138 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:16.138 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791057+0000", 2023-11-25T04:23:16.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031095200000000002 2023-11-25T04:23:16.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.139 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:16.139 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791086+0000", 2023-11-25T04:23:16.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8697999999999999e-05 2023-11-25T04:23:16.139 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:16.139 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:16.139 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:16.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.139 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:157 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'111, mlcod=11'106)", 2023-11-25T04:23:16.139 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:16.140 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 38.338488048999999, 2023-11-25T04:23:16.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050590280000000001, 2023-11-25T04:23:16.140 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:16.140 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:16.140 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:16.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.140 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:16.142 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:16.142 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:16.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.142 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:16.142 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:16.142 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:16.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.142 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:16.143 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786063+0000", 2023-11-25T04:23:16.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2600000000000002e-07 2023-11-25T04:23:16.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:16.143 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786069+0000", 2023-11-25T04:23:16.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8839999999999999e-06 2023-11-25T04:23:16.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:16.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786079+0000", 2023-11-25T04:23:16.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0124e-05 2023-11-25T04:23:16.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:16.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786085+0000", 2023-11-25T04:23:16.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6660000000000002e-06 2023-11-25T04:23:16.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:16.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788160+0000", 2023-11-25T04:23:16.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020747669999999999 2023-11-25T04:23:16.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:16.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788180+0000", 2023-11-25T04:23:16.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0208999999999999e-05 2023-11-25T04:23:16.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:16.146 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791087+0000", 2023-11-25T04:23:16.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029064630000000002 2023-11-25T04:23:16.146 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.146 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:16.146 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791121+0000", 2023-11-25T04:23:16.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4289000000000003e-05 2023-11-25T04:23:16.146 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:16.146 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:16.146 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:16.146 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.147 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:158 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'112, mlcod=11'106)", 2023-11-25T04:23:16.147 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:16.147 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 38.338071655999997, 2023-11-25T04:23:16.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046663030000000001, 2023-11-25T04:23:16.147 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:16.147 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:16.147 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:16.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.147 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:16.147 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:16.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:16.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.148 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:16.148 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:16.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:16.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.148 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:16.148 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786479+0000", 2023-11-25T04:23:16.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2799999999999996e-07 2023-11-25T04:23:16.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:16.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786498+0000", 2023-11-25T04:23:16.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9069e-05 2023-11-25T04:23:16.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:16.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786508+0000", 2023-11-25T04:23:16.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5810000000000003e-06 2023-11-25T04:23:16.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:16.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786512+0000", 2023-11-25T04:23:16.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4889999999999997e-06 2023-11-25T04:23:16.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:16.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788438+0000", 2023-11-25T04:23:16.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019257059999999999 2023-11-25T04:23:16.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:16.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788449+0000", 2023-11-25T04:23:16.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1419e-05 2023-11-25T04:23:16.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.151 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:16.151 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791122+0000", 2023-11-25T04:23:16.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002672822 2023-11-25T04:23:16.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.151 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:16.151 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791145+0000", 2023-11-25T04:23:16.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2589e-05 2023-11-25T04:23:16.151 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:16.152 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:16.152 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:16.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.152 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:206 1.0 e12/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 12'152, mlcod=12'148)", 2023-11-25T04:23:16.152 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:16.152 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.826303185999997, 2023-11-25T04:23:16.152 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046728189999999999, 2023-11-25T04:23:16.152 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:16.152 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:16.152 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:16.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:16.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:16.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:16.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:16.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:16.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:16.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:16.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298248+0000", 2023-11-25T04:23:16.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3600000000000003e-07 2023-11-25T04:23:16.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:16.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298278+0000", 2023-11-25T04:23:16.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0474e-05 2023-11-25T04:23:16.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:16.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298288+0000", 2023-11-25T04:23:16.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0156000000000001e-05 2023-11-25T04:23:16.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.155 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:16.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298293+0000", 2023-11-25T04:23:16.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7400000000000004e-06 2023-11-25T04:23:16.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.155 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:16.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298333+0000", 2023-11-25T04:23:16.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9666000000000003e-05 2023-11-25T04:23:16.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.156 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:16.156 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298351+0000", 2023-11-25T04:23:16.156 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8544999999999999e-05 2023-11-25T04:23:16.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.156 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:16.156 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302891+0000", 2023-11-25T04:23:16.156 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045398519999999996 2023-11-25T04:23:16.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.156 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:16.156 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302920+0000", 2023-11-25T04:23:16.156 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8750000000000001e-05 2023-11-25T04:23:16.157 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:16.157 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:16.157 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:16.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.157 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:14 3.15 3:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:16.157 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:16.157 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 29.091168829000001, 2023-11-25T04:23:16.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010990437, 2023-11-25T04:23:16.157 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:16.157 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:16.158 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:16.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:16.158 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:16.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:16.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:16.158 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:16.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:16.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.159 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:16.159 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033383+0000", 2023-11-25T04:23:16.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9870000000000002e-06 2023-11-25T04:23:16.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.159 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:16.159 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033398+0000", 2023-11-25T04:23:16.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4569999999999999e-05 2023-11-25T04:23:16.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.160 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:16.160 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033411+0000", 2023-11-25T04:23:16.160 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3543999999999999e-05 2023-11-25T04:23:16.160 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.160 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.160 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:16.160 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033423+0000", 2023-11-25T04:23:16.160 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1905999999999999e-05 2023-11-25T04:23:16.160 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.160 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.160 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:16.160 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034433+0000", 2023-11-25T04:23:16.161 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00101022 2023-11-25T04:23:16.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.161 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.161 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:16.161 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034539+0000", 2023-11-25T04:23:16.161 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010597199999999999 2023-11-25T04:23:16.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.161 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.162 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:16.162 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034667+0000", 2023-11-25T04:23:16.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012765500000000001 2023-11-25T04:23:16.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.162 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:16.162 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037803+0000", 2023-11-25T04:23:16.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031362220000000001 2023-11-25T04:23:16.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.162 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:16.163 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:23:16.164 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0064964669999999997 2023-11-25T04:23:16.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.164 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.164 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:16.164 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044331+0000", 2023-11-25T04:23:16.164 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1544999999999997e-05 2023-11-25T04:23:16.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.165 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.165 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:16.165 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044372+0000", 2023-11-25T04:23:16.165 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0349000000000003e-05 2023-11-25T04:23:16.165 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:16.165 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:16.165 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:16.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.165 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.165 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:15 3.15 3:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:16.166 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:16.166 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 29.090509179000001, 2023-11-25T04:23:16.166 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010464362, 2023-11-25T04:23:16.166 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:16.166 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:16.166 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:16.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.167 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:16.167 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:16.167 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:16.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.167 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:16.168 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:16.168 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:16.168 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.168 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.168 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:16.168 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034042+0000", 2023-11-25T04:23:16.168 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0999999999999997e-07 2023-11-25T04:23:16.168 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.168 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.168 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:16.168 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034048+0000", 2023-11-25T04:23:16.168 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0390000000000001e-06 2023-11-25T04:23:16.169 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.169 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.169 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:16.169 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034059+0000", 2023-11-25T04:23:16.169 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1182999999999999e-05 2023-11-25T04:23:16.169 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.169 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.169 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:16.169 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034067+0000", 2023-11-25T04:23:16.169 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1559999999999995e-06 2023-11-25T04:23:16.169 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.169 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.170 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:16.170 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035005+0000", 2023-11-25T04:23:16.170 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00093747399999999999 2023-11-25T04:23:16.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.170 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:16.170 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035077+0000", 2023-11-25T04:23:16.170 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2747e-05 2023-11-25T04:23:16.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.170 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:16.170 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035219+0000", 2023-11-25T04:23:16.171 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000141877 2023-11-25T04:23:16.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.171 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:16.171 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037810+0000", 2023-11-25T04:23:16.171 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002590752 2023-11-25T04:23:16.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.171 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:16.171 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044437+0000", 2023-11-25T04:23:16.171 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066270110000000004 2023-11-25T04:23:16.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.172 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:16.172 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044481+0000", 2023-11-25T04:23:16.172 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4106000000000001e-05 2023-11-25T04:23:16.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.172 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:16.172 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044505+0000", 2023-11-25T04:23:16.172 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4207e-05 2023-11-25T04:23:16.172 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:16.172 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:16.172 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:16.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.173 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.173 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:16 3.1d 3:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:16.173 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:16.173 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 29.090467330999999, 2023-11-25T04:23:16.173 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010413923, 2023-11-25T04:23:16.173 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:16.173 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:16.173 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:16.173 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.173 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:16.173 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:16.173 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:16.174 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.174 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.174 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:16.174 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:16.174 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:16.174 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.174 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.174 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:16.174 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:16.174 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4499999999999997e-07 2023-11-25T04:23:16.174 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.174 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.175 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:16.175 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034087+0000", 2023-11-25T04:23:16.175 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9770000000000002e-06 2023-11-25T04:23:16.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.175 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.175 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:16.175 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034094+0000", 2023-11-25T04:23:16.175 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8809999999999998e-06 2023-11-25T04:23:16.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.175 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.175 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:16.175 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034100+0000", 2023-11-25T04:23:16.176 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8440000000000003e-06 2023-11-25T04:23:16.176 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.176 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.176 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:16.176 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034993+0000", 2023-11-25T04:23:16.176 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00089336799999999996 2023-11-25T04:23:16.176 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.176 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.176 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:16.176 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035099+0000", 2023-11-25T04:23:16.176 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106037 2023-11-25T04:23:16.177 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.177 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.177 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:16.177 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035276+0000", 2023-11-25T04:23:16.177 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017631799999999999 2023-11-25T04:23:16.177 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.177 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.177 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:16.177 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037815+0000", 2023-11-25T04:23:16.177 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025395959999999999 2023-11-25T04:23:16.177 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.177 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.178 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:16.178 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044420+0000", 2023-11-25T04:23:16.178 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066045749999999997 2023-11-25T04:23:16.178 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.178 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.178 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:16.178 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044463+0000", 2023-11-25T04:23:16.178 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3205999999999999e-05 2023-11-25T04:23:16.178 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.178 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.178 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:16.178 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044497+0000", 2023-11-25T04:23:16.179 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3476000000000003e-05 2023-11-25T04:23:16.179 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:16.179 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:16.179 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:16.179 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.179 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.179 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:17 3.1d 3:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:16.179 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:16.179 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 29.090442121999999, 2023-11-25T04:23:16.179 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01127356, 2023-11-25T04:23:16.179 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:16.179 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:16.179 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:16.180 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.180 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:16.180 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:16.180 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:16.180 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.180 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.180 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:16.180 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:16.180 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:16.181 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.181 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.181 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:16.181 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:16.181 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1100000000000001e-07 2023-11-25T04:23:16.181 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.181 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.181 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:16.181 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034112+0000", 2023-11-25T04:23:16.181 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8600000000000003e-06 2023-11-25T04:23:16.181 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.182 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.182 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:16.182 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034120+0000", 2023-11-25T04:23:16.182 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7700000000000001e-06 2023-11-25T04:23:16.182 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.182 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.182 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:16.182 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034127+0000", 2023-11-25T04:23:16.182 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0999999999999998e-06 2023-11-25T04:23:16.182 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.182 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.183 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:16.183 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035601+0000", 2023-11-25T04:23:16.183 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014739410000000001 2023-11-25T04:23:16.183 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.183 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.183 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:16.183 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035685+0000", 2023-11-25T04:23:16.183 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3677999999999994e-05 2023-11-25T04:23:16.183 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.183 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.184 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:16.184 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035827+0000", 2023-11-25T04:23:16.184 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014232599999999999 2023-11-25T04:23:16.184 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.184 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.184 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:16.184 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038725+0000", 2023-11-25T04:23:16.184 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028982779999999998 2023-11-25T04:23:16.185 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.185 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.185 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:16.185 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045323+0000", 2023-11-25T04:23:16.185 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0065980780000000003 2023-11-25T04:23:16.185 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.185 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.185 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:16.186 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045354+0000", 2023-11-25T04:23:16.186 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0463e-05 2023-11-25T04:23:16.186 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.186 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.186 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:16.186 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045381+0000", 2023-11-25T04:23:16.186 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7654999999999999e-05 2023-11-25T04:23:16.186 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:16.186 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:16.186 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:16.187 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.187 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.187 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:22 3.e 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:16.187 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:16.187 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 29.090415708999998, 2023-11-25T04:23:16.187 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010133422, 2023-11-25T04:23:16.187 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:16.187 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:16.187 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:16.187 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.187 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:16.187 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:16.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:16.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.188 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:16.188 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:16.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:16.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.188 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:16.188 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034135+0000", 2023-11-25T04:23:16.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-07 2023-11-25T04:23:16.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:16.189 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034138+0000", 2023-11-25T04:23:16.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6009999999999999e-06 2023-11-25T04:23:16.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:16.189 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034145+0000", 2023-11-25T04:23:16.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1169999999999998e-06 2023-11-25T04:23:16.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:16.189 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034150+0000", 2023-11-25T04:23:16.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9080000000000003e-06 2023-11-25T04:23:16.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.190 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:16.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034209+0000", 2023-11-25T04:23:16.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8149000000000002e-05 2023-11-25T04:23:16.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.190 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:16.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034415+0000", 2023-11-25T04:23:16.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020637699999999999 2023-11-25T04:23:16.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.191 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:16.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034584+0000", 2023-11-25T04:23:16.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016870399999999999 2023-11-25T04:23:16.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.191 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:16.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.036979+0000", 2023-11-25T04:23:16.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023957509999999998 2023-11-25T04:23:16.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.191 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:16.192 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044161+0000", 2023-11-25T04:23:16.192 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0071811590000000003 2023-11-25T04:23:16.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.192 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:16.192 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044224+0000", 2023-11-25T04:23:16.192 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3454000000000006e-05 2023-11-25T04:23:16.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.192 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:16.192 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044268+0000", 2023-11-25T04:23:16.192 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3761999999999999e-05 2023-11-25T04:23:16.193 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:16.193 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:16.193 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:16.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.193 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:23 3.e 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:16.193 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:16.193 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 29.0903843, 2023-11-25T04:23:16.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010281676, 2023-11-25T04:23:16.193 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:16.193 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:16.193 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:16.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:16.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:16.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:16.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:16.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:16.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:16.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:16.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:16.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5799999999999999e-07 2023-11-25T04:23:16.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.195 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:16.195 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034169+0000", 2023-11-25T04:23:16.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1879999999999998e-06 2023-11-25T04:23:16.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.195 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:16.195 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034177+0000", 2023-11-25T04:23:16.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7020000000000001e-06 2023-11-25T04:23:16.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.196 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:16.196 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034184+0000", 2023-11-25T04:23:16.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3910000000000003e-06 2023-11-25T04:23:16.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.196 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:16.196 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034924+0000", 2023-11-25T04:23:16.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00074032999999999998 2023-11-25T04:23:16.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.196 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:16.196 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034991+0000", 2023-11-25T04:23:16.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7347000000000004e-05 2023-11-25T04:23:16.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:16.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035096+0000", 2023-11-25T04:23:16.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010488 2023-11-25T04:23:16.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:16.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037588+0000", 2023-11-25T04:23:16.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024918309999999999 2023-11-25T04:23:16.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:16.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044367+0000", 2023-11-25T04:23:16.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067787589999999997 2023-11-25T04:23:16.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:16.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044415+0000", 2023-11-25T04:23:16.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7803000000000003e-05 2023-11-25T04:23:16.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:16.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044447+0000", 2023-11-25T04:23:16.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2886999999999999e-05 2023-11-25T04:23:16.199 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:16.199 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:16.199 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:16.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.199 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:30 3.15 3:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:16.199 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:16.199 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 29.09032586, 2023-11-25T04:23:16.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010365070000000001, 2023-11-25T04:23:16.200 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:16.200 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:16.200 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:16.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.200 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:16.200 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:16.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:16.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.200 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:16.200 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:16.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:16.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:16.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034225+0000", 2023-11-25T04:23:16.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7400000000000002e-07 2023-11-25T04:23:16.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:16.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034229+0000", 2023-11-25T04:23:16.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3100000000000002e-06 2023-11-25T04:23:16.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:16.202 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034239+0000", 2023-11-25T04:23:16.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9140000000000003e-06 2023-11-25T04:23:16.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.202 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:16.202 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034246+0000", 2023-11-25T04:23:16.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0149999999999998e-06 2023-11-25T04:23:16.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.202 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:16.202 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035572+0000", 2023-11-25T04:23:16.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001325655 2023-11-25T04:23:16.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:16.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035730+0000", 2023-11-25T04:23:16.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015827899999999999 2023-11-25T04:23:16.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:16.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035857+0000", 2023-11-25T04:23:16.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012644699999999999 2023-11-25T04:23:16.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:16.204 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038737+0000", 2023-11-25T04:23:16.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002880077 2023-11-25T04:23:16.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.204 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:16.204 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044540+0000", 2023-11-25T04:23:16.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058034239999999997 2023-11-25T04:23:16.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.204 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:16.204 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044569+0000", 2023-11-25T04:23:16.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8388999999999999e-05 2023-11-25T04:23:16.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.205 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:16.205 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044589+0000", 2023-11-25T04:23:16.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0686000000000002e-05 2023-11-25T04:23:16.205 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:16.205 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:16.205 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:16.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.205 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:33 3.1b 3:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:16.205 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:16.205 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 29.090280362000001, 2023-11-25T04:23:16.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099754049999999997, 2023-11-25T04:23:16.206 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:16.206 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:23:16.206 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:16.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:16.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:16.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:16.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:16.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:16.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:16.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.207 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:16.207 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:16.207 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4799999999999998e-07 2023-11-25T04:23:16.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.207 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:16.207 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034273+0000", 2023-11-25T04:23:16.207 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7819999999999998e-06 2023-11-25T04:23:16.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.207 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:16.208 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034285+0000", 2023-11-25T04:23:16.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2058e-05 2023-11-25T04:23:16.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.208 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:16.208 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034291+0000", 2023-11-25T04:23:16.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0680000000000002e-06 2023-11-25T04:23:16.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.208 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:16.208 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034370+0000", 2023-11-25T04:23:16.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8452e-05 2023-11-25T04:23:16.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.209 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.209 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:16.209 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034477+0000", 2023-11-25T04:23:16.209 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107471 2023-11-25T04:23:16.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.209 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.209 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:16.209 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044245+0000", 2023-11-25T04:23:16.209 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0097679259999999997 2023-11-25T04:23:16.209 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:16.209 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:16.209 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:16.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.210 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:36 3.1a 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:16.210 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:16.210 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 29.090233689000002, 2023-11-25T04:23:16.210 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058179099999999999, 2023-11-25T04:23:16.210 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:16.210 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:23:16.210 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:16.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.210 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:16.210 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:16.210 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:16.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.211 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:16.211 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:16.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:16.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.211 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:16.211 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034317+0000", 2023-11-25T04:23:16.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1999999999999999e-07 2023-11-25T04:23:16.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.211 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:16.212 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034321+0000", 2023-11-25T04:23:16.212 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.233e-06 2023-11-25T04:23:16.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.212 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:16.212 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034337+0000", 2023-11-25T04:23:16.212 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5308999999999999e-05 2023-11-25T04:23:16.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.212 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:16.212 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034354+0000", 2023-11-25T04:23:16.212 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7411999999999999e-05 2023-11-25T04:23:16.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.213 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:16.213 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034465+0000", 2023-11-25T04:23:16.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011154699999999999 2023-11-25T04:23:16.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.213 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:16.213 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034560+0000", 2023-11-25T04:23:16.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4186999999999998e-05 2023-11-25T04:23:16.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.214 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:16.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.040134+0000", 2023-11-25T04:23:16.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055746019999999997 2023-11-25T04:23:16.214 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:16.214 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:16.214 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:16.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.214 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.214 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:9 3.10 e18/16 3:09401a81:::601.00000000:head v 18'4, mlcod=0'0)", 2023-11-25T04:23:16.214 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:16.214 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 29.081339380999999, 2023-11-25T04:23:16.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061641020000000003, 2023-11-25T04:23:16.215 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:16.215 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:16.215 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:16.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:16.215 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:16.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:16.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:16.215 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:16.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:16.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:16.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:16.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5300000000000001e-07 2023-11-25T04:23:16.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:16.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043222+0000", 2023-11-25T04:23:16.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.082e-05 2023-11-25T04:23:16.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:16.217 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043228+0000", 2023-11-25T04:23:16.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6699999999999997e-06 2023-11-25T04:23:16.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.217 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:16.217 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043236+0000", 2023-11-25T04:23:16.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8189999999999992e-06 2023-11-25T04:23:16.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.217 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:16.217 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043507+0000", 2023-11-25T04:23:16.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000271012 2023-11-25T04:23:16.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.218 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:16.218 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043530+0000", 2023-11-25T04:23:16.218 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2659000000000001e-05 2023-11-25T04:23:16.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.218 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:16.218 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049321+0000", 2023-11-25T04:23:16.218 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057915129999999999 2023-11-25T04:23:16.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.218 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:16.218 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049375+0000", 2023-11-25T04:23:16.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3455999999999998e-05 2023-11-25T04:23:16.219 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:16.219 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:16.219 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:16.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.219 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:1 3.14 e18/16 3:292cf221:::200.00000000:head v 18'3, mlcod=0'0)", 2023-11-25T04:23:16.219 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:16.219 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 29.081302667999999, 2023-11-25T04:23:16.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0060983599999999997, 2023-11-25T04:23:16.219 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:16.219 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:16.220 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:16.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:16.220 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:16.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:16.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:16.220 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:16.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:16.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:16.221 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043248+0000", 2023-11-25T04:23:16.221 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9399999999999995e-07 2023-11-25T04:23:16.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.221 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.221 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:16.221 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043252+0000", 2023-11-25T04:23:16.221 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1799999999999998e-06 2023-11-25T04:23:16.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.221 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.221 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:16.221 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043259+0000", 2023-11-25T04:23:16.221 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0049999999999998e-06 2023-11-25T04:23:16.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.222 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:16.222 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043265+0000", 2023-11-25T04:23:16.222 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0859999999999998e-06 2023-11-25T04:23:16.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.222 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:16.222 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043311+0000", 2023-11-25T04:23:16.222 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6082999999999999e-05 2023-11-25T04:23:16.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.222 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:16.222 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043330+0000", 2023-11-25T04:23:16.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8556e-05 2023-11-25T04:23:16.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.223 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.223 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:16.223 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049303+0000", 2023-11-25T04:23:16.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059733470000000004 2023-11-25T04:23:16.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.223 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.223 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:16.223 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049346+0000", 2023-11-25T04:23:16.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2608999999999998e-05 2023-11-25T04:23:16.223 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:16.223 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:16.224 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:16.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.224 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:3 3.14 e18/16 3:287034c4:::1.00000000.inode:head v 18'4, mlcod=0'0)", 2023-11-25T04:23:16.224 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:16.224 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 29.080786161999999, 2023-11-25T04:23:16.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057040090000000003, 2023-11-25T04:23:16.224 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:16.224 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:16.224 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:16.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:16.224 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:16.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:16.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.225 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:16.225 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:16.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:16.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.225 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:16.225 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:16.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9799999999999999e-07 2023-11-25T04:23:16.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.226 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:16.226 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043785+0000", 2023-11-25T04:23:16.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0721999999999999e-05 2023-11-25T04:23:16.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.226 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:16.226 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043793+0000", 2023-11-25T04:23:16.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5839999999999995e-06 2023-11-25T04:23:16.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.226 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:16.226 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043803+0000", 2023-11-25T04:23:16.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0729999999999994e-06 2023-11-25T04:23:16.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:16.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043843+0000", 2023-11-25T04:23:16.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0747999999999997e-05 2023-11-25T04:23:16.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:16.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043856+0000", 2023-11-25T04:23:16.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2697000000000001e-05 2023-11-25T04:23:16.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:16.228 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049426+0000", 2023-11-25T04:23:16.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055704409999999998 2023-11-25T04:23:16.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.228 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:16.228 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049468+0000", 2023-11-25T04:23:16.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1446000000000003e-05 2023-11-25T04:23:16.228 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:16.228 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:16.228 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:16.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.228 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "pg_update_log_missing(3.14 epoch 18/16 rep_tid 171 entries 18'5 (0'0) error 3:2e2fa760:::200.00000001:head by mds.0.3:32 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:23:16.229 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:16.229 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 29.080625199, 2023-11-25T04:23:16.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005556938, 2023-11-25T04:23:16.229 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:16.229 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2023-11-25T04:23:16.229 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:16.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.229 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:16.229 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:16.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:16.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.229 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:16.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:16.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:16.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:16.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:16.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.03e-07 2023-11-25T04:23:16.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:16.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043931+0000", 2023-11-25T04:23:16.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1079999999999998e-06 2023-11-25T04:23:16.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:16.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043944+0000", 2023-11-25T04:23:16.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2852e-05 2023-11-25T04:23:16.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:16.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043955+0000", 2023-11-25T04:23:16.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1037e-05 2023-11-25T04:23:16.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:16.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044088+0000", 2023-11-25T04:23:16.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013269599999999999 2023-11-25T04:23:16.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:16.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049482+0000", 2023-11-25T04:23:16.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053940419999999999 2023-11-25T04:23:16.232 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:16.232 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:16.232 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:16.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.232 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:10 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'5, mlcod=0'0)", 2023-11-25T04:23:16.232 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:16.232 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 29.080535171000001, 2023-11-25T04:23:16.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054493099999999997, 2023-11-25T04:23:16.233 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:16.233 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:16.233 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:16.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.233 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:16.233 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:16.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:16.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.233 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:16.233 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:16.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:16.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:16.234 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:16.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6100000000000002e-07 2023-11-25T04:23:16.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:16.234 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044019+0000", 2023-11-25T04:23:16.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.839e-06 2023-11-25T04:23:16.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:16.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044026+0000", 2023-11-25T04:23:16.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5869999999999998e-06 2023-11-25T04:23:16.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.235 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:16.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044031+0000", 2023-11-25T04:23:16.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4909999999999996e-06 2023-11-25T04:23:16.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.235 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:16.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044070+0000", 2023-11-25T04:23:16.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8829999999999999e-05 2023-11-25T04:23:16.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:16.236 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044081+0000", 2023-11-25T04:23:16.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1447999999999999e-05 2023-11-25T04:23:16.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:16.236 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049406+0000", 2023-11-25T04:23:16.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053246480000000004 2023-11-25T04:23:16.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:16.236 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049464+0000", 2023-11-25T04:23:16.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8205999999999998e-05 2023-11-25T04:23:16.237 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:16.237 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:16.237 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:16.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.237 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:11 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'6, mlcod=0'0)", 2023-11-25T04:23:16.237 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:16.237 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 29.080333907, 2023-11-25T04:23:16.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052673499999999996, 2023-11-25T04:23:16.237 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:16.237 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:16.237 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:16.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:16.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:16.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:16.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:16.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:16.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:16.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:16.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:16.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0199999999999998e-07 2023-11-25T04:23:16.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.239 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:16.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044220+0000", 2023-11-25T04:23:16.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7720000000000001e-06 2023-11-25T04:23:16.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.239 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:16.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044226+0000", 2023-11-25T04:23:16.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8579999999999996e-06 2023-11-25T04:23:16.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:16.240 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044232+0000", 2023-11-25T04:23:16.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1700000000000002e-06 2023-11-25T04:23:16.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:16.240 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044274+0000", 2023-11-25T04:23:16.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1254999999999997e-05 2023-11-25T04:23:16.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:16.240 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:23:16.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6880999999999999e-05 2023-11-25T04:23:16.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:16.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049466+0000", 2023-11-25T04:23:16.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051659219999999999 2023-11-25T04:23:16.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:16.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:16.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:16.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049483+0000", 2023-11-25T04:23:16.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.719e-05 2023-11-25T04:23:16.241 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:16.241 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:16.241 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:16.242 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:16.242 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:16.242 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:16.242 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2023-11-25T04:23:16.245 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:23:16.245 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:16.242+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:16.245 DEBUG:teuthology.orchestra.console.smithi076:check power output: Chassis Power is off 2023-11-25T04:23:16.344 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:16.343+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:16.346 INFO:teuthology.orchestra.console.smithi076:Power off completed 2023-11-25T04:23:16.357 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:16.357 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:23:16.357 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2023-11-25T04:23:16.357 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:16.366 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:23:16.446 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:16.445+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:16.511 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:16.511 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:23:16.511 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2023-11-25T04:23:16.511 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2023-11-25T04:23:16.511 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:16.520 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:23:16.546 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:16.545+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:16.627 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:16.627 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2023-11-25T04:23:16.627 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2023-11-25T04:23:16.627 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2023-11-25T04:23:16.627 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.627 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:16.627 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:16.627 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 29.592880949000001, 2023-11-25T04:23:16.627 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01072943, 2023-11-25T04:23:16.628 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:16.628 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:16.628 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:16.628 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.628 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:16.628 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:16.628 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:16.628 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.628 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.628 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:16.628 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:16.628 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:16.629 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.629 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.629 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:16.629 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031829+0000", 2023-11-25T04:23:16.629 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.666e-06 2023-11-25T04:23:16.629 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.629 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.629 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:16.629 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031839+0000", 2023-11-25T04:23:16.629 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9490000000000008e-06 2023-11-25T04:23:16.629 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.629 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.629 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:16.630 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031857+0000", 2023-11-25T04:23:16.630 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8093000000000001e-05 2023-11-25T04:23:16.630 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.630 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.630 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:16.630 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031882+0000", 2023-11-25T04:23:16.630 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4294000000000002e-05 2023-11-25T04:23:16.630 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.630 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.630 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:16.630 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033130+0000", 2023-11-25T04:23:16.630 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001248264 2023-11-25T04:23:16.631 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.631 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.631 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:16.631 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033426+0000", 2023-11-25T04:23:16.631 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029620999999999998 2023-11-25T04:23:16.631 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.631 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.631 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:16.631 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033814+0000", 2023-11-25T04:23:16.631 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000387742 2023-11-25T04:23:16.631 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.631 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.631 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:16.632 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036793+0000", 2023-11-25T04:23:16.632 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002979598 2023-11-25T04:23:16.632 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.632 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.632 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:16.632 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042455+0000", 2023-11-25T04:23:16.632 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0056619260000000003 2023-11-25T04:23:16.632 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.632 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.632 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:16.632 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042502+0000", 2023-11-25T04:23:16.632 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7139999999999999e-05 2023-11-25T04:23:16.633 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.633 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.633 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:16.633 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042557+0000", 2023-11-25T04:23:16.633 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4548e-05 2023-11-25T04:23:16.633 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:16.633 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:16.633 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:16.633 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.633 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.633 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:16.633 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:16.634 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 29.591682414000001, 2023-11-25T04:23:16.634 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011505202000000001, 2023-11-25T04:23:16.634 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:16.634 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:16.634 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:16.634 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.634 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:16.634 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:16.635 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:16.635 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.635 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.635 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:16.635 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:16.635 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:16.635 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.635 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.635 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:16.635 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033027+0000", 2023-11-25T04:23:16.635 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0699999999999999e-06 2023-11-25T04:23:16.635 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.636 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.636 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:16.636 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033035+0000", 2023-11-25T04:23:16.636 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0830000000000004e-06 2023-11-25T04:23:16.636 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.636 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.636 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:16.636 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033059+0000", 2023-11-25T04:23:16.636 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3975e-05 2023-11-25T04:23:16.636 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.636 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.636 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:16.636 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033482+0000", 2023-11-25T04:23:16.637 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042253700000000001 2023-11-25T04:23:16.637 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.637 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.637 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:16.637 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034520+0000", 2023-11-25T04:23:16.637 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001037991 2023-11-25T04:23:16.637 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.637 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.637 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:16.637 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034611+0000", 2023-11-25T04:23:16.637 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1113999999999995e-05 2023-11-25T04:23:16.637 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.637 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.638 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:16.638 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034769+0000", 2023-11-25T04:23:16.638 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000157641 2023-11-25T04:23:16.638 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.638 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.638 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:16.638 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038292+0000", 2023-11-25T04:23:16.638 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035231160000000002 2023-11-25T04:23:16.638 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.638 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.638 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:16.638 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044440+0000", 2023-11-25T04:23:16.639 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0061486980000000002 2023-11-25T04:23:16.639 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.639 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.639 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:16.639 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044495+0000", 2023-11-25T04:23:16.639 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4339000000000003e-05 2023-11-25T04:23:16.639 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.639 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.639 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:16.639 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044531+0000", 2023-11-25T04:23:16.639 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6637999999999997e-05 2023-11-25T04:23:16.639 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:16.639 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:16.640 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:16.640 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.640 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.640 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:16.640 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:16.640 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 29.59120806, 2023-11-25T04:23:16.640 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0057634039999999997, 2023-11-25T04:23:16.640 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:16.640 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:16.640 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:16.640 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.640 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:16.641 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:16.641 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:16.641 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.641 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.641 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:16.641 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:16.641 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:16.641 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.641 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.641 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:16.641 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:16.641 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7900000000000007e-07 2023-11-25T04:23:16.641 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.642 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.642 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:16.642 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033507+0000", 2023-11-25T04:23:16.642 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8900000000000004e-06 2023-11-25T04:23:16.642 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.642 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.642 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:16.642 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033519+0000", 2023-11-25T04:23:16.642 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1756000000000001e-05 2023-11-25T04:23:16.642 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.642 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.642 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:16.643 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033527+0000", 2023-11-25T04:23:16.643 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3610000000000001e-06 2023-11-25T04:23:16.643 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.643 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.643 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:16.643 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034451+0000", 2023-11-25T04:23:16.643 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00092372400000000001 2023-11-25T04:23:16.643 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.643 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.643 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:16.643 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034809+0000", 2023-11-25T04:23:16.643 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00035757699999999998 2023-11-25T04:23:16.644 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.644 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.644 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:23:16.644 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034965+0000", 2023-11-25T04:23:16.644 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015638699999999999 2023-11-25T04:23:16.644 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.644 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.644 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:16.645 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038299+0000", 2023-11-25T04:23:16.645 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0033336559999999999 2023-11-25T04:23:16.645 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.645 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.645 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:16.645 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039169+0000", 2023-11-25T04:23:16.645 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00087066400000000005 2023-11-25T04:23:16.645 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.645 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.645 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:16.645 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039226+0000", 2023-11-25T04:23:16.645 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6511999999999997e-05 2023-11-25T04:23:16.646 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.646 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.646 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:16.646 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039264+0000", 2023-11-25T04:23:16.646 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7898000000000002e-05 2023-11-25T04:23:16.646 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:16.646 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:16.646 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:16.646 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.646 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.646 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:25 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:16.646 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:16.647 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 29.591129475999999, 2023-11-25T04:23:16.647 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096656169999999996, 2023-11-25T04:23:16.647 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:16.647 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:16.647 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:16.647 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.647 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:16.647 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:16.647 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:16.647 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.647 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.647 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:16.647 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:16.648 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:16.648 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.648 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.648 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:16.648 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033580+0000", 2023-11-25T04:23:16.648 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8199999999999999e-07 2023-11-25T04:23:16.648 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.648 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.648 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:16.648 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033585+0000", 2023-11-25T04:23:16.648 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3680000000000001e-06 2023-11-25T04:23:16.648 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.648 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.649 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:16.649 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033600+0000", 2023-11-25T04:23:16.649 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4331000000000001e-05 2023-11-25T04:23:16.649 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.649 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.649 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:16.649 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033606+0000", 2023-11-25T04:23:16.649 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5780000000000004e-06 2023-11-25T04:23:16.649 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.649 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.649 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:16.649 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035747+0000", 2023-11-25T04:23:16.650 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0021406530000000002 2023-11-25T04:23:16.650 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.650 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.650 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:16.650 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035901+0000", 2023-11-25T04:23:16.650 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000153931 2023-11-25T04:23:16.650 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.650 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.650 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:23:16.650 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036193+0000", 2023-11-25T04:23:16.650 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029211299999999999 2023-11-25T04:23:16.650 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.651 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.651 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:16.651 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038655+0000", 2023-11-25T04:23:16.651 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0024621640000000002 2023-11-25T04:23:16.651 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.651 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.651 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:16.651 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043130+0000", 2023-11-25T04:23:16.651 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.004474698 2023-11-25T04:23:16.651 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.651 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.651 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:16.651 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043204+0000", 2023-11-25T04:23:16.652 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4091999999999995e-05 2023-11-25T04:23:16.652 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.652 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.652 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:16.652 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043245+0000", 2023-11-25T04:23:16.652 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0907e-05 2023-11-25T04:23:16.652 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:16.652 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:16.652 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:16.652 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.652 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.652 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:26 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:16.652 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:16.653 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 29.591093652000001, 2023-11-25T04:23:16.653 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011086945000000001, 2023-11-25T04:23:16.653 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:16.653 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:16.653 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:16.653 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.653 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:16.653 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:16.653 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:16.653 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.653 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.653 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:16.653 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:16.654 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:16.654 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.654 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.654 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:16.654 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:16.654 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1699999999999998e-07 2023-11-25T04:23:16.654 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.654 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.655 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:16.655 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033627+0000", 2023-11-25T04:23:16.655 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.163e-05 2023-11-25T04:23:16.655 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.655 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.655 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:16.655 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033636+0000", 2023-11-25T04:23:16.655 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6859999999999995e-06 2023-11-25T04:23:16.655 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.655 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.656 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:16.656 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033642+0000", 2023-11-25T04:23:16.656 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3149999999999997e-06 2023-11-25T04:23:16.656 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.656 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.656 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:16.656 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035414+0000", 2023-11-25T04:23:16.656 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0017716380000000001 2023-11-25T04:23:16.656 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.656 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.656 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:16.656 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035539+0000", 2023-11-25T04:23:16.656 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000125477 2023-11-25T04:23:16.657 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.657 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.657 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:16.657 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035672+0000", 2023-11-25T04:23:16.657 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000133316 2023-11-25T04:23:16.657 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.657 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.657 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:16.657 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038649+0000", 2023-11-25T04:23:16.657 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029764679999999999 2023-11-25T04:23:16.657 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.657 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.657 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:16.658 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044578+0000", 2023-11-25T04:23:16.658 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0059293510000000002 2023-11-25T04:23:16.658 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.658 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.658 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:16.658 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044651+0000", 2023-11-25T04:23:16.658 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2513000000000002e-05 2023-11-25T04:23:16.658 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.658 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.658 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:16.658 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044702+0000", 2023-11-25T04:23:16.658 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1034e-05 2023-11-25T04:23:16.659 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:16.659 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:16.659 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:16.659 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.659 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.659 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:29 3.b 3:d0630e4c:::mds0_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:16.659 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:16.659 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 29.591055732000001, 2023-11-25T04:23:16.659 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010980227, 2023-11-25T04:23:16.659 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:16.659 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:16.659 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:16.659 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.660 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:16.660 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:16.660 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:16.660 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.660 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.660 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:16.660 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:16.660 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:16.660 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.660 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.660 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:16.660 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:16.660 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3499999999999996e-07 2023-11-25T04:23:16.661 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.661 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.661 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:16.661 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033658+0000", 2023-11-25T04:23:16.661 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1280000000000001e-06 2023-11-25T04:23:16.661 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.661 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.661 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:16.661 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033667+0000", 2023-11-25T04:23:16.661 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1430000000000004e-06 2023-11-25T04:23:16.661 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.661 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.662 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:16.662 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033678+0000", 2023-11-25T04:23:16.662 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0906e-05 2023-11-25T04:23:16.662 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.662 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.662 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:16.662 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034456+0000", 2023-11-25T04:23:16.662 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00077833899999999996 2023-11-25T04:23:16.662 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.662 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.662 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:16.662 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034575+0000", 2023-11-25T04:23:16.663 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011926000000000001 2023-11-25T04:23:16.663 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.663 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.663 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:16.663 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035001+0000", 2023-11-25T04:23:16.663 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042550999999999998 2023-11-25T04:23:16.663 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.663 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.663 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:16.663 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.037923+0000", 2023-11-25T04:23:16.663 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029219670000000001 2023-11-25T04:23:16.663 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.663 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.664 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:16.664 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044538+0000", 2023-11-25T04:23:16.664 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0066152349999999997 2023-11-25T04:23:16.664 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.664 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.664 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:16.664 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044594+0000", 2023-11-25T04:23:16.664 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5593000000000001e-05 2023-11-25T04:23:16.664 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.664 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.664 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:16.664 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044633+0000", 2023-11-25T04:23:16.664 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9511000000000002e-05 2023-11-25T04:23:16.665 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:16.665 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:16.665 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:16.665 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.665 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.665 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:34 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:16.665 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:16.665 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 29.590805619000001, 2023-11-25T04:23:16.665 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010920445000000001, 2023-11-25T04:23:16.665 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:16.665 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:16.665 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:16.666 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.666 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:16.666 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:16.666 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:16.666 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.666 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.666 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:16.666 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:16.666 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:16.666 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.666 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.666 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:16.666 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033904+0000", 2023-11-25T04:23:16.667 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6599999999999994e-07 2023-11-25T04:23:16.667 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.667 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.667 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:16.667 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033909+0000", 2023-11-25T04:23:16.667 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5639999999999996e-06 2023-11-25T04:23:16.667 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.667 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.667 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:16.667 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033923+0000", 2023-11-25T04:23:16.667 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3028e-05 2023-11-25T04:23:16.667 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.667 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.668 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:16.668 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034043+0000", 2023-11-25T04:23:16.668 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000120328 2023-11-25T04:23:16.668 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.668 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.668 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:16.668 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035576+0000", 2023-11-25T04:23:16.668 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015333720000000001 2023-11-25T04:23:16.668 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.668 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.668 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:16.668 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035699+0000", 2023-11-25T04:23:16.668 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000123001 2023-11-25T04:23:16.669 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.669 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.669 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:16.669 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044823+0000", 2023-11-25T04:23:16.669 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0091241859999999994 2023-11-25T04:23:16.669 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:16.669 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:16.669 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:16.669 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.669 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.669 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:35 3.1f 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:16.669 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:16.670 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 29.589501414000001, 2023-11-25T04:23:16.670 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096164779999999995, 2023-11-25T04:23:16.670 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:16.670 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:16.670 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:16.670 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.670 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:16.670 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:16.670 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:16.670 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.670 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.670 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:16.670 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:16.670 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:16.671 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.671 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.671 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:16.671 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035208+0000", 2023-11-25T04:23:16.671 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2699999999999998e-07 2023-11-25T04:23:16.671 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.671 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.671 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:16.671 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035213+0000", 2023-11-25T04:23:16.671 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.6070000000000001e-06 2023-11-25T04:23:16.671 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.671 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.671 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:16.672 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035350+0000", 2023-11-25T04:23:16.672 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013693599999999999 2023-11-25T04:23:16.672 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.672 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.672 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:16.672 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035358+0000", 2023-11-25T04:23:16.672 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0099999999999995e-06 2023-11-25T04:23:16.672 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.672 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.672 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:16.672 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036260+0000", 2023-11-25T04:23:16.672 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00090240100000000005 2023-11-25T04:23:16.673 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.673 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.673 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:16.673 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036517+0000", 2023-11-25T04:23:16.673 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00025736299999999998 2023-11-25T04:23:16.673 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.673 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.673 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:16.673 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044824+0000", 2023-11-25T04:23:16.673 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0083063340000000003 2023-11-25T04:23:16.673 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:16.673 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:16.674 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:16.674 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.674 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.674 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:37 3.1e 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:16.674 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:16.674 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 29.589334728000001, 2023-11-25T04:23:16.674 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0078322640000000002, 2023-11-25T04:23:16.674 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:16.674 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:16.674 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:16.674 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.674 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:16.675 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:16.675 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:16.675 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.675 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.675 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:16.675 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:16.675 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:16.675 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.675 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.675 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:16.675 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035375+0000", 2023-11-25T04:23:16.675 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9100000000000003e-07 2023-11-25T04:23:16.676 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.676 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.676 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:16.676 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035379+0000", 2023-11-25T04:23:16.676 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9430000000000002e-06 2023-11-25T04:23:16.676 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.676 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.676 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:16.676 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035388+0000", 2023-11-25T04:23:16.676 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2760000000000006e-06 2023-11-25T04:23:16.676 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.676 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.677 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:16.677 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035396+0000", 2023-11-25T04:23:16.677 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8480000000000001e-06 2023-11-25T04:23:16.677 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.677 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.677 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:16.677 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035452+0000", 2023-11-25T04:23:16.677 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5831000000000001e-05 2023-11-25T04:23:16.677 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.677 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.677 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:16.677 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035934+0000", 2023-11-25T04:23:16.677 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00048265599999999998 2023-11-25T04:23:16.678 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.678 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.678 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:16.678 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043206+0000", 2023-11-25T04:23:16.678 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.007271919 2023-11-25T04:23:16.678 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:16.678 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:16.678 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:16.678 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.678 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.678 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing(3.1a epoch 18/16 rep_tid 3 entries 18'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:36 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:23:16.678 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:16.679 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 29.588059141999999, 2023-11-25T04:23:16.679 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002073414, 2023-11-25T04:23:16.679 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:16.679 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:16.679 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:16.679 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.679 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:16.679 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:16.679 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:16.679 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.679 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.679 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:16.680 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:16.680 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:16.680 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.680 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.680 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:16.680 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036650+0000", 2023-11-25T04:23:16.680 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7300000000000002e-07 2023-11-25T04:23:16.680 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.680 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.680 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:16.680 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036659+0000", 2023-11-25T04:23:16.680 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.7499999999999992e-06 2023-11-25T04:23:16.681 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.681 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.681 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:16.681 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036673+0000", 2023-11-25T04:23:16.681 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4917e-05 2023-11-25T04:23:16.681 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.681 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.681 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:16.681 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036687+0000", 2023-11-25T04:23:16.681 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3883000000000001e-05 2023-11-25T04:23:16.681 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.681 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.681 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:16.682 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036781+0000", 2023-11-25T04:23:16.682 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3219000000000003e-05 2023-11-25T04:23:16.682 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.682 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.682 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:16.682 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038723+0000", 2023-11-25T04:23:16.682 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019422720000000001 2023-11-25T04:23:16.682 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:16.682 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:16.682 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:16.682 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.682 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.683 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:28 3.13 e18/16 3:cd5a64a3:::100.00000000:head v 18'1, mlcod=0'0)", 2023-11-25T04:23:16.683 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:16.683 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 29.584397983999999, 2023-11-25T04:23:16.683 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001850546, 2023-11-25T04:23:16.683 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:16.683 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:16.683 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:16.683 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.683 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:16.683 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:16.683 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:16.683 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.684 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.684 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:16.684 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:16.684 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:16.684 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.684 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.684 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:16.684 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:16.684 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7800000000000002e-07 2023-11-25T04:23:16.684 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.684 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.684 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:16.684 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040331+0000", 2023-11-25T04:23:16.685 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0449999999999999e-05 2023-11-25T04:23:16.685 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.685 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.685 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:16.685 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040339+0000", 2023-11-25T04:23:16.685 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0709999999999995e-06 2023-11-25T04:23:16.685 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.685 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.685 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:16.685 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040347+0000", 2023-11-25T04:23:16.685 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7079999999999997e-06 2023-11-25T04:23:16.685 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.686 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.686 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:16.686 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040410+0000", 2023-11-25T04:23:16.686 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2453000000000007e-05 2023-11-25T04:23:16.686 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.686 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.686 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:16.686 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040422+0000", 2023-11-25T04:23:16.686 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2429000000000001e-05 2023-11-25T04:23:16.686 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.686 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.686 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:16.687 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042129+0000", 2023-11-25T04:23:16.687 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001707266 2023-11-25T04:23:16.687 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.687 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.687 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:16.687 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042161+0000", 2023-11-25T04:23:16.687 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1791000000000001e-05 2023-11-25T04:23:16.687 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:16.687 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:16.687 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:16.687 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.687 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.687 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:27 3.13 e18/16 3:c91d4a1d:::mds0_inotable:head v 18'2, mlcod=0'0)", 2023-11-25T04:23:16.688 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:16.688 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 29.584194460999999, 2023-11-25T04:23:16.688 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0016886430000000001, 2023-11-25T04:23:16.688 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:16.688 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:16.688 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:16.688 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.688 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:16.688 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:16.688 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:16.688 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.688 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.689 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:16.689 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:16.689 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:16.689 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.689 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.689 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:16.689 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:16.689 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3999999999999997e-07 2023-11-25T04:23:16.689 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.689 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.689 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:16.689 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040520+0000", 2023-11-25T04:23:16.690 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2989999999999996e-06 2023-11-25T04:23:16.690 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.690 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.690 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:16.690 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040527+0000", 2023-11-25T04:23:16.690 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5730000000000002e-06 2023-11-25T04:23:16.690 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.690 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.690 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:16.690 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040535+0000", 2023-11-25T04:23:16.690 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8630000000000008e-06 2023-11-25T04:23:16.690 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.691 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.691 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:16.691 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040683+0000", 2023-11-25T04:23:16.691 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000148002 2023-11-25T04:23:16.691 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.691 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.691 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:16.691 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040700+0000", 2023-11-25T04:23:16.691 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7065e-05 2023-11-25T04:23:16.691 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.691 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.691 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:16.692 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042163+0000", 2023-11-25T04:23:16.692 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001462767 2023-11-25T04:23:16.692 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.692 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.692 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:16.692 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042203+0000", 2023-11-25T04:23:16.692 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9734000000000003e-05 2023-11-25T04:23:16.692 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:16.692 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:16.692 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:16.692 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.692 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.693 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:20 3.1c e18/16 3:3ed09add:::607.00000000:head v 18'1, mlcod=0'0)", 2023-11-25T04:23:16.693 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:16.693 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 29.584159419999999, 2023-11-25T04:23:16.693 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015271, 2023-11-25T04:23:16.693 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:16.693 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:16.693 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:16.693 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.693 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:16.693 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:16.693 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:16.693 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.694 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.694 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:16.694 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:16.694 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:16.694 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.694 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.694 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:16.694 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:16.694 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4799999999999999e-07 2023-11-25T04:23:16.694 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.694 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.694 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:16.695 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040561+0000", 2023-11-25T04:23:16.695 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1800000000000001e-05 2023-11-25T04:23:16.695 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.695 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.695 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:16.695 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040568+0000", 2023-11-25T04:23:16.695 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7730000000000002e-06 2023-11-25T04:23:16.695 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.695 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.695 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:16.695 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040587+0000", 2023-11-25T04:23:16.695 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8691000000000001e-05 2023-11-25T04:23:16.696 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.696 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.696 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:16.696 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040638+0000", 2023-11-25T04:23:16.696 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1068e-05 2023-11-25T04:23:16.696 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.696 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.696 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:16.696 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040670+0000", 2023-11-25T04:23:16.696 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2125999999999997e-05 2023-11-25T04:23:16.696 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.696 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.697 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:16.697 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042032+0000", 2023-11-25T04:23:16.697 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001361821 2023-11-25T04:23:16.697 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.697 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.697 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:16.697 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042076+0000", 2023-11-25T04:23:16.697 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4472999999999999e-05 2023-11-25T04:23:16.697 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:16.697 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:16.697 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:16.697 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.698 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.698 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:21 3.1c e18/16 3:3ed09add:::607.00000000:head v 18'2, mlcod=0'0)", 2023-11-25T04:23:16.698 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:16.698 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 29.583924695, 2023-11-25T04:23:16.698 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0013228160000000001, 2023-11-25T04:23:16.698 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:16.698 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:16.698 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:16.698 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.698 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:16.698 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:16.698 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:16.699 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.699 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.699 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:16.699 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:16.699 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:16.699 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.699 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.699 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:16.699 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:16.699 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9299999999999999e-07 2023-11-25T04:23:16.699 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.699 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.699 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:16.700 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040790+0000", 2023-11-25T04:23:16.700 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5770000000000001e-06 2023-11-25T04:23:16.700 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.700 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.700 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:16.700 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040799+0000", 2023-11-25T04:23:16.700 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6439999999999995e-06 2023-11-25T04:23:16.700 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.700 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.700 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:16.700 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040806+0000", 2023-11-25T04:23:16.700 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0180000000000004e-06 2023-11-25T04:23:16.701 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.701 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.701 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:16.701 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040907+0000", 2023-11-25T04:23:16.701 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000100096 2023-11-25T04:23:16.701 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.701 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.701 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:16.701 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040925+0000", 2023-11-25T04:23:16.701 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8536999999999999e-05 2023-11-25T04:23:16.701 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.701 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.702 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:16.702 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042079+0000", 2023-11-25T04:23:16.702 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00115358 2023-11-25T04:23:16.702 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.702 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.702 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:16.702 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042107+0000", 2023-11-25T04:23:16.702 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7971e-05 2023-11-25T04:23:16.702 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:16.702 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:16.702 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:16.702 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.703 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.703 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop_reply(mds.0.3:4 3.1f e18/16 ondisk, result = 0)", 2023-11-25T04:23:16.703 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:16.703 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 29.582585304999998, 2023-11-25T04:23:16.703 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00044092900000000001, 2023-11-25T04:23:16.703 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:16.703 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:16.703 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:16.703 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.703 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:16.703 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:16.703 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:16.704 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.704 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.704 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:16.704 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:16.704 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:16.704 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.704 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.704 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:16.704 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042124+0000", 2023-11-25T04:23:16.704 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1800000000000005e-07 2023-11-25T04:23:16.704 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.704 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.705 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:16.705 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042128+0000", 2023-11-25T04:23:16.705 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2289999999999997e-06 2023-11-25T04:23:16.705 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.705 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.705 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:16.705 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042138+0000", 2023-11-25T04:23:16.705 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9729999999999992e-06 2023-11-25T04:23:16.705 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.705 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.705 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:16.705 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042145+0000", 2023-11-25T04:23:16.705 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.336e-06 2023-11-25T04:23:16.706 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.706 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.706 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:16.706 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042441+0000", 2023-11-25T04:23:16.706 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029650500000000002 2023-11-25T04:23:16.706 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.706 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.706 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:16.706 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042448+0000", 2023-11-25T04:23:16.706 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4100000000000002e-06 2023-11-25T04:23:16.706 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.706 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.707 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:16.707 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042564+0000", 2023-11-25T04:23:16.707 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000115758 2023-11-25T04:23:16.707 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:16.707 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:16.707 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:16.707 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.707 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.707 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing_reply(3.1f epoch 18/16 rep_tid 8 lcod 18'4)", 2023-11-25T04:23:16.707 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:16.707 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 29.580259893000001, 2023-11-25T04:23:16.707 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00038154699999999998, 2023-11-25T04:23:16.707 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:16.708 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:16.708 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:16.708 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.708 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:16.708 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:16.708 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:16.708 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.708 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.708 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:16.708 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:16.708 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:16.708 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.709 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.709 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:16.709 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:16.709 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2700000000000001e-07 2023-11-25T04:23:16.709 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.709 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.709 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:16.709 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044451+0000", 2023-11-25T04:23:16.709 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7460000000000001e-06 2023-11-25T04:23:16.709 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.709 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.709 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:16.709 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044455+0000", 2023-11-25T04:23:16.710 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5809999999999998e-06 2023-11-25T04:23:16.710 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.710 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.710 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:16.710 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044458+0000", 2023-11-25T04:23:16.710 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0170000000000001e-06 2023-11-25T04:23:16.710 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.710 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.710 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:16.710 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044734+0000", 2023-11-25T04:23:16.710 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00027582799999999999 2023-11-25T04:23:16.710 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.711 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.711 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:16.711 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044830+0000", 2023-11-25T04:23:16.711 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6248000000000004e-05 2023-11-25T04:23:16.711 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:16.711 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:16.711 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:16.711 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.711 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.711 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing_reply(3.b epoch 18/16 rep_tid 5 lcod 18'2)", 2023-11-25T04:23:16.711 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:16.711 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 29.580223232000002, 2023-11-25T04:23:16.711 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000344488, 2023-11-25T04:23:16.712 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:16.712 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:16.712 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:16.712 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.712 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:16.712 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:16.712 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:16.712 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.712 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.712 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:16.712 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:16.712 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:16.713 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.713 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.713 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:16.713 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:16.713 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.55e-07 2023-11-25T04:23:16.713 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.713 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.713 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:16.713 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044487+0000", 2023-11-25T04:23:16.713 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2470000000000001e-06 2023-11-25T04:23:16.713 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.714 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.714 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:16.714 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044491+0000", 2023-11-25T04:23:16.714 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9659999999999998e-06 2023-11-25T04:23:16.714 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.714 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.714 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:16.714 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044494+0000", 2023-11-25T04:23:16.714 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2310000000000001e-06 2023-11-25T04:23:16.714 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.714 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.714 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:16.714 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044756+0000", 2023-11-25T04:23:16.715 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00026172800000000003 2023-11-25T04:23:16.715 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.715 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.715 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:16.715 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044830+0000", 2023-11-25T04:23:16.715 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4160999999999997e-05 2023-11-25T04:23:16.715 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:16.715 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:16.715 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:16.715 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.715 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.715 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:39 3.6 3:654134d2:::mds0_openfiles.0:head [omap-clear,omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2023-11-25T04:23:16.715 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:16.716 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 24.520397190000001, 2023-11-25T04:23:16.716 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001788658, 2023-11-25T04:23:16.716 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:16.716 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:16.716 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:16.716 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.716 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:16.716 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:16.716 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:16.716 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.716 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.716 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:16.717 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:16.717 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:16.717 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.717 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.717 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:16.717 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104314+0000", 2023-11-25T04:23:16.717 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3099999999999999e-06 2023-11-25T04:23:16.717 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.717 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.717 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:16.717 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104325+0000", 2023-11-25T04:23:16.717 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1511999999999999e-05 2023-11-25T04:23:16.717 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.718 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.718 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:16.718 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104362+0000", 2023-11-25T04:23:16.718 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6903000000000002e-05 2023-11-25T04:23:16.718 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.718 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.718 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:16.718 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104387+0000", 2023-11-25T04:23:16.718 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4766e-05 2023-11-25T04:23:16.718 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.718 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.718 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:16.718 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104465+0000", 2023-11-25T04:23:16.719 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8119000000000002e-05 2023-11-25T04:23:16.719 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.719 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.719 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:16.719 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104557+0000", 2023-11-25T04:23:16.719 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2496000000000003e-05 2023-11-25T04:23:16.719 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.719 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.719 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:16.719 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104646+0000", 2023-11-25T04:23:16.719 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8818000000000002e-05 2023-11-25T04:23:16.719 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.719 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.720 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:16.720 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105201+0000", 2023-11-25T04:23:16.720 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00055506000000000002 2023-11-25T04:23:16.720 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.720 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.720 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:16.720 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105994+0000", 2023-11-25T04:23:16.720 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00079294700000000001 2023-11-25T04:23:16.720 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.720 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.720 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:16.720 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.106055+0000", 2023-11-25T04:23:16.721 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0566000000000003e-05 2023-11-25T04:23:16.721 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.721 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.721 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:16.721 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.106100+0000", 2023-11-25T04:23:16.721 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5160999999999997e-05 2023-11-25T04:23:16.721 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:16.721 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:16.721 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:16.721 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.721 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.721 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:43 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2023-11-25T04:23:16.722 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:16.722 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 9.5201721940000006, 2023-11-25T04:23:16.722 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001751166, 2023-11-25T04:23:16.722 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:16.722 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:16.722 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:16.722 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.722 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:16.722 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:16.722 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:16.722 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.722 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.722 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:16.723 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:16.723 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:16.723 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.723 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.723 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:16.723 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104539+0000", 2023-11-25T04:23:16.723 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9529999999999999e-06 2023-11-25T04:23:16.723 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.723 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.723 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:16.723 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104557+0000", 2023-11-25T04:23:16.723 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7464000000000001e-05 2023-11-25T04:23:16.724 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.724 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.724 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:16.724 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104590+0000", 2023-11-25T04:23:16.724 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2966000000000003e-05 2023-11-25T04:23:16.724 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.724 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.724 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:16.724 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104608+0000", 2023-11-25T04:23:16.724 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8193e-05 2023-11-25T04:23:16.724 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.724 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.724 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:16.725 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104767+0000", 2023-11-25T04:23:16.725 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015882 2023-11-25T04:23:16.725 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.725 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.725 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:16.725 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104909+0000", 2023-11-25T04:23:16.725 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014244999999999999 2023-11-25T04:23:16.725 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.725 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.725 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:16.725 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.105059+0000", 2023-11-25T04:23:16.725 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014964399999999999 2023-11-25T04:23:16.726 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.726 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.726 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:16.726 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.105701+0000", 2023-11-25T04:23:16.726 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00064201099999999995 2023-11-25T04:23:16.726 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.726 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.726 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:16.726 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106233+0000", 2023-11-25T04:23:16.726 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00053178500000000001 2023-11-25T04:23:16.726 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.726 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.727 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:16.727 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106265+0000", 2023-11-25T04:23:16.727 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1986000000000002e-05 2023-11-25T04:23:16.727 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.727 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.727 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:16.727 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106288+0000", 2023-11-25T04:23:16.727 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2894000000000001e-05 2023-11-25T04:23:16.727 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:16.727 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:16.727 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:16.727 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.727 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.728 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:45 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e23)", 2023-11-25T04:23:16.728 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:23:16.728 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.5199741539999998, 2023-11-25T04:23:16.728 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00098423299999999994, 2023-11-25T04:23:16.728 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:16.728 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:16.728 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:16.728 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.728 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:16.728 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:23:16.728 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:16.728 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.729 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.729 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:16.729 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:23:16.729 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:16.729 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.729 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.729 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:16.729 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104737+0000", 2023-11-25T04:23:16.729 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3309999999999998e-06 2023-11-25T04:23:16.729 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.729 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.729 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:16.729 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104748+0000", 2023-11-25T04:23:16.730 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1080999999999999e-05 2023-11-25T04:23:16.730 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.730 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.730 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:16.730 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104771+0000", 2023-11-25T04:23:16.730 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2707000000000001e-05 2023-11-25T04:23:16.730 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.730 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.730 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:16.730 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104800+0000", 2023-11-25T04:23:16.730 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9595e-05 2023-11-25T04:23:16.730 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.731 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.731 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:16.731 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104890+0000", 2023-11-25T04:23:16.731 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.9808999999999993e-05 2023-11-25T04:23:16.731 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.731 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.731 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:16.731 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105004+0000", 2023-11-25T04:23:16.731 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000114508 2023-11-25T04:23:16.731 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.731 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.731 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:16.732 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105634+0000", 2023-11-25T04:23:16.732 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00062985799999999996 2023-11-25T04:23:16.732 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.732 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.732 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:16.732 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105671+0000", 2023-11-25T04:23:16.732 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6640000000000002e-05 2023-11-25T04:23:16.732 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:16.732 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:16.732 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:16.732 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105719+0000", 2023-11-25T04:23:16.732 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7704000000000003e-05 2023-11-25T04:23:16.732 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:16.733 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:16.733 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:16.733 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:16.733 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:16.733 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:16.734 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:16.733+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:16.735 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:23:16.835 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:16.834+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:16.844 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:16.844 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:23:16.845 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2023-11-25T04:23:16.845 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:16.853 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:23:16.912 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-11-25T04:23:16.936 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:16.935+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:16.998 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:16.998 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:23:16.998 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2023-11-25T04:23:16.998 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2023-11-25T04:23:16.998 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:17.006 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:23:17.037 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:17.036+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:17.063 INFO:teuthology.orchestra.run.smithi067.stderr:Error ENXIO: problem getting command descriptions from osd.0 2023-11-25T04:23:17.081 INFO:teuthology.orchestra.run.smithi067.stdout:osd.1: {} 2023-11-25T04:23:17.081 INFO:teuthology.orchestra.run.smithi067.stderr:osd.1: osd_enable_op_tracker = '' 2023-11-25T04:23:17.101 INFO:teuthology.orchestra.run.smithi067.stdout:osd.2: {} 2023-11-25T04:23:17.101 INFO:teuthology.orchestra.run.smithi067.stderr:osd.2: osd_enable_op_tracker = '' 2023-11-25T04:23:17.102 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-11-25T04:23:17.103 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-11-25T04:23:17.112 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:17.112 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:17.121 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:23:17.139 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:17.137+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:17.197 INFO:teuthology.orchestra.run.smithi067.stderr:noscrub is set 2023-11-25T04:23:17.232 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:17.232 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:17.240 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:17.238+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:17.241 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:23:17.340 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:17.339+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:17.346 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:17.346 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:17.355 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:23:17.441 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:17.440+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:17.501 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:17.501 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:17.510 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:23:17.550 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:17.549+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:17.618 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:17.619 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:17.628 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:23:17.671 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:17.670+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:17.733 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:17.734 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:17.742 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:23:17.783 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:17.782+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:17.845 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:17.844+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:17.887 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:17.887 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:17.896 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:23:17.946 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:17.944+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:18.005 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:18.006 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:18.014 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:23:18.056 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:18.054+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:18.148 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:18.147+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:18.161 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:18.162 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:18.171 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:23:18.249 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:18.247+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:18.278 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:18.278 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:18.287 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:23:18.350 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:18.348+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:18.393 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:18.393 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:18.401 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:23:18.450 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:18.449+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:18.507 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:18.507 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:18.516 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:23:18.560 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:18.559+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:18.621 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:18.621 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:18.631 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:23:18.672 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:18.671+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:18.738 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:18.739 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:18.747 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:23:18.791 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:18.790+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:18.851 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:18.851 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:18.853 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:18.853+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:18.861 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:23:18.955 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:18.953+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:19.006 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:19.006 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:19.015 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:23:19.056 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:19.054+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:19.104 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-11-25T04:23:19.121 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:19.121 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:19.129 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:23:19.173 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:19.172+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:19.212 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2023-11-25T04:23:19.219 INFO:teuthology.orchestra.run.smithi067.stderr:Error ENXIO: problem getting command descriptions from osd.0 2023-11-25T04:23:19.234 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:19.234 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:19.243 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:23:19.254 INFO:teuthology.orchestra.run.smithi067.stdout:osd.1: {} 2023-11-25T04:23:19.254 INFO:teuthology.orchestra.run.smithi067.stderr:osd.1: osd_enable_op_tracker = '' 2023-11-25T04:23:19.285 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:19.283+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:19.289 INFO:teuthology.orchestra.run.smithi067.stdout:osd.2: {} 2023-11-25T04:23:19.289 INFO:teuthology.orchestra.run.smithi067.stderr:osd.2: osd_enable_op_tracker = '' 2023-11-25T04:23:19.293 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-11-25T04:23:19.294 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-11-25T04:23:19.359 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:19.358+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:19.391 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:19.391 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2023-11-25T04:23:19.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2023-11-25T04:23:19.391 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2023-11-25T04:23:19.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.391 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:155 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'109, mlcod=11'105)", 2023-11-25T04:23:19.392 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:19.392 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 41.603823994999999, 2023-11-25T04:23:19.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061719019999999999, 2023-11-25T04:23:19.392 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:19.392 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:19.392 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:19.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:19.392 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:19.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:19.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:19.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:19.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:19.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:19.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784884+0000", 2023-11-25T04:23:19.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3099999999999999e-07 2023-11-25T04:23:19.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:19.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784942+0000", 2023-11-25T04:23:19.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8570000000000003e-05 2023-11-25T04:23:19.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:19.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784949+0000", 2023-11-25T04:23:19.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0310000000000001e-06 2023-11-25T04:23:19.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:19.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784953+0000", 2023-11-25T04:23:19.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6320000000000001e-06 2023-11-25T04:23:19.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:19.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784993+0000", 2023-11-25T04:23:19.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9737000000000003e-05 2023-11-25T04:23:19.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:19.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785011+0000", 2023-11-25T04:23:19.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8349999999999999e-05 2023-11-25T04:23:19.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:19.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791024+0000", 2023-11-25T04:23:19.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006012997 2023-11-25T04:23:19.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:19.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791055+0000", 2023-11-25T04:23:19.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1254000000000003e-05 2023-11-25T04:23:19.396 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:19.396 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:19.396 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:19.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.396 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:156 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'110, mlcod=11'105)", 2023-11-25T04:23:19.396 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:19.397 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 41.603237946, 2023-11-25T04:23:19.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005616116, 2023-11-25T04:23:19.397 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:19.397 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:19.397 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:19.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:19.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:19.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:19.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:19.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:19.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:19.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:19.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785470+0000", 2023-11-25T04:23:19.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0699999999999998e-07 2023-11-25T04:23:19.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:19.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785474+0000", 2023-11-25T04:23:19.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.048e-06 2023-11-25T04:23:19.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:19.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785480+0000", 2023-11-25T04:23:19.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7729999999999996e-06 2023-11-25T04:23:19.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:19.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785483+0000", 2023-11-25T04:23:19.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1970000000000001e-06 2023-11-25T04:23:19.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:19.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787936+0000", 2023-11-25T04:23:19.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024529040000000001 2023-11-25T04:23:19.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:19.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787947+0000", 2023-11-25T04:23:19.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1669e-05 2023-11-25T04:23:19.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:19.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791057+0000", 2023-11-25T04:23:19.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031095200000000002 2023-11-25T04:23:19.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:19.401 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791086+0000", 2023-11-25T04:23:19.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8697999999999999e-05 2023-11-25T04:23:19.401 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:19.401 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:19.401 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:19.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.401 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:157 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'111, mlcod=11'106)", 2023-11-25T04:23:19.402 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:19.402 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 41.602645387999999, 2023-11-25T04:23:19.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050590280000000001, 2023-11-25T04:23:19.402 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:19.402 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:19.402 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:19.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:19.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:19.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:19.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:19.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:19.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:19.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:19.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786063+0000", 2023-11-25T04:23:19.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2600000000000002e-07 2023-11-25T04:23:19.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:19.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786069+0000", 2023-11-25T04:23:19.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8839999999999999e-06 2023-11-25T04:23:19.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:19.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786079+0000", 2023-11-25T04:23:19.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0124e-05 2023-11-25T04:23:19.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:19.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786085+0000", 2023-11-25T04:23:19.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6660000000000002e-06 2023-11-25T04:23:19.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:19.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788160+0000", 2023-11-25T04:23:19.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020747669999999999 2023-11-25T04:23:19.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:19.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788180+0000", 2023-11-25T04:23:19.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0208999999999999e-05 2023-11-25T04:23:19.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:19.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791087+0000", 2023-11-25T04:23:19.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029064630000000002 2023-11-25T04:23:19.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:19.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791121+0000", 2023-11-25T04:23:19.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4289000000000003e-05 2023-11-25T04:23:19.407 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:19.407 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:19.407 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:19.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.407 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:158 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'112, mlcod=11'106)", 2023-11-25T04:23:19.407 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:19.407 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 41.602228994999997, 2023-11-25T04:23:19.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046663030000000001, 2023-11-25T04:23:19.407 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:19.407 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:19.408 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:19.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:19.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:19.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:19.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:19.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:19.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:19.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:19.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786479+0000", 2023-11-25T04:23:19.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2799999999999996e-07 2023-11-25T04:23:19.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:19.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786498+0000", 2023-11-25T04:23:19.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9069e-05 2023-11-25T04:23:19.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:19.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786508+0000", 2023-11-25T04:23:19.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5810000000000003e-06 2023-11-25T04:23:19.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:19.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786512+0000", 2023-11-25T04:23:19.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4889999999999997e-06 2023-11-25T04:23:19.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:19.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788438+0000", 2023-11-25T04:23:19.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019257059999999999 2023-11-25T04:23:19.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.411 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:19.411 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788449+0000", 2023-11-25T04:23:19.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1419e-05 2023-11-25T04:23:19.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.411 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:19.411 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791122+0000", 2023-11-25T04:23:19.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002672822 2023-11-25T04:23:19.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.412 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:19.412 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791145+0000", 2023-11-25T04:23:19.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2589e-05 2023-11-25T04:23:19.412 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:19.412 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:19.412 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:19.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.412 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:206 1.0 e12/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 12'152, mlcod=12'148)", 2023-11-25T04:23:19.412 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:19.412 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 41.090460524999997, 2023-11-25T04:23:19.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046728189999999999, 2023-11-25T04:23:19.412 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:19.412 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:19.413 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:19.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:19.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:19.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:19.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:19.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:19.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:19.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:19.414 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298248+0000", 2023-11-25T04:23:19.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3600000000000003e-07 2023-11-25T04:23:19.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.414 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:19.414 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298278+0000", 2023-11-25T04:23:19.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0474e-05 2023-11-25T04:23:19.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.414 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:19.414 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298288+0000", 2023-11-25T04:23:19.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0156000000000001e-05 2023-11-25T04:23:19.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:19.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298293+0000", 2023-11-25T04:23:19.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7400000000000004e-06 2023-11-25T04:23:19.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:19.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298333+0000", 2023-11-25T04:23:19.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9666000000000003e-05 2023-11-25T04:23:19.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:19.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298351+0000", 2023-11-25T04:23:19.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8544999999999999e-05 2023-11-25T04:23:19.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:19.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302891+0000", 2023-11-25T04:23:19.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045398519999999996 2023-11-25T04:23:19.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:19.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302920+0000", 2023-11-25T04:23:19.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8750000000000001e-05 2023-11-25T04:23:19.417 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:19.417 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:19.417 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:19.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.417 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:14 3.15 3:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:19.417 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:19.417 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.355326167999998, 2023-11-25T04:23:19.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010990437, 2023-11-25T04:23:19.417 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:19.417 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:19.418 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:19.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:19.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:19.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:19.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:19.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:19.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:19.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:19.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033383+0000", 2023-11-25T04:23:19.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9870000000000002e-06 2023-11-25T04:23:19.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:19.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033398+0000", 2023-11-25T04:23:19.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4569999999999999e-05 2023-11-25T04:23:19.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:19.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033411+0000", 2023-11-25T04:23:19.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3543999999999999e-05 2023-11-25T04:23:19.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:19.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033423+0000", 2023-11-25T04:23:19.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1905999999999999e-05 2023-11-25T04:23:19.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:19.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034433+0000", 2023-11-25T04:23:19.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00101022 2023-11-25T04:23:19.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:19.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034539+0000", 2023-11-25T04:23:19.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010597199999999999 2023-11-25T04:23:19.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:19.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034667+0000", 2023-11-25T04:23:19.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012765500000000001 2023-11-25T04:23:19.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:19.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037803+0000", 2023-11-25T04:23:19.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031362220000000001 2023-11-25T04:23:19.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:19.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:23:19.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0064964669999999997 2023-11-25T04:23:19.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:19.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044331+0000", 2023-11-25T04:23:19.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1544999999999997e-05 2023-11-25T04:23:19.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:19.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044372+0000", 2023-11-25T04:23:19.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0349000000000003e-05 2023-11-25T04:23:19.422 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:19.422 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:19.422 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:19.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.423 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:15 3.15 3:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:19.423 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:19.423 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.354666518000002, 2023-11-25T04:23:19.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010464362, 2023-11-25T04:23:19.423 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:19.423 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:19.423 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:19.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:19.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:19.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:19.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:19.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:19.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:19.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:19.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034042+0000", 2023-11-25T04:23:19.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0999999999999997e-07 2023-11-25T04:23:19.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:19.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034048+0000", 2023-11-25T04:23:19.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0390000000000001e-06 2023-11-25T04:23:19.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:19.425 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034059+0000", 2023-11-25T04:23:19.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1182999999999999e-05 2023-11-25T04:23:19.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:19.425 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034067+0000", 2023-11-25T04:23:19.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1559999999999995e-06 2023-11-25T04:23:19.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:19.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035005+0000", 2023-11-25T04:23:19.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00093747399999999999 2023-11-25T04:23:19.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:19.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035077+0000", 2023-11-25T04:23:19.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2747e-05 2023-11-25T04:23:19.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:19.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035219+0000", 2023-11-25T04:23:19.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000141877 2023-11-25T04:23:19.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:19.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037810+0000", 2023-11-25T04:23:19.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002590752 2023-11-25T04:23:19.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:19.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044437+0000", 2023-11-25T04:23:19.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066270110000000004 2023-11-25T04:23:19.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:19.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044481+0000", 2023-11-25T04:23:19.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4106000000000001e-05 2023-11-25T04:23:19.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:19.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044505+0000", 2023-11-25T04:23:19.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4207e-05 2023-11-25T04:23:19.428 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:19.428 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:19.428 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:19.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.428 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:16 3.1d 3:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:19.428 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:19.428 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.35462467, 2023-11-25T04:23:19.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010413923, 2023-11-25T04:23:19.429 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:19.429 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:19.429 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:19.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:19.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:19.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:19.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:19.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:19.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:19.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.430 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:19.430 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:19.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4499999999999997e-07 2023-11-25T04:23:19.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.430 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:19.430 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034087+0000", 2023-11-25T04:23:19.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9770000000000002e-06 2023-11-25T04:23:19.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.430 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:19.430 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034094+0000", 2023-11-25T04:23:19.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8809999999999998e-06 2023-11-25T04:23:19.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:19.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034100+0000", 2023-11-25T04:23:19.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8440000000000003e-06 2023-11-25T04:23:19.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:19.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034993+0000", 2023-11-25T04:23:19.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00089336799999999996 2023-11-25T04:23:19.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:19.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035099+0000", 2023-11-25T04:23:19.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106037 2023-11-25T04:23:19.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:19.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035276+0000", 2023-11-25T04:23:19.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017631799999999999 2023-11-25T04:23:19.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:19.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037815+0000", 2023-11-25T04:23:19.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025395959999999999 2023-11-25T04:23:19.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:19.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044420+0000", 2023-11-25T04:23:19.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066045749999999997 2023-11-25T04:23:19.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:19.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044463+0000", 2023-11-25T04:23:19.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3205999999999999e-05 2023-11-25T04:23:19.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:19.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044497+0000", 2023-11-25T04:23:19.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3476000000000003e-05 2023-11-25T04:23:19.433 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:19.434 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:19.434 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:19.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.434 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:17 3.1d 3:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:19.434 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:19.434 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.354599460999999, 2023-11-25T04:23:19.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01127356, 2023-11-25T04:23:19.434 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:19.434 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:19.434 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:19.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.434 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:19.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:19.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:19.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:19.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:19.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:19.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:19.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:19.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1100000000000001e-07 2023-11-25T04:23:19.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:19.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034112+0000", 2023-11-25T04:23:19.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8600000000000003e-06 2023-11-25T04:23:19.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:19.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034120+0000", 2023-11-25T04:23:19.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7700000000000001e-06 2023-11-25T04:23:19.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:19.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034127+0000", 2023-11-25T04:23:19.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0999999999999998e-06 2023-11-25T04:23:19.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:19.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035601+0000", 2023-11-25T04:23:19.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014739410000000001 2023-11-25T04:23:19.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:19.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035685+0000", 2023-11-25T04:23:19.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3677999999999994e-05 2023-11-25T04:23:19.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:19.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035827+0000", 2023-11-25T04:23:19.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014232599999999999 2023-11-25T04:23:19.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.438 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:19.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038725+0000", 2023-11-25T04:23:19.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028982779999999998 2023-11-25T04:23:19.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.438 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:19.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045323+0000", 2023-11-25T04:23:19.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0065980780000000003 2023-11-25T04:23:19.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.439 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:19.439 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045354+0000", 2023-11-25T04:23:19.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0463e-05 2023-11-25T04:23:19.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.439 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:19.439 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045381+0000", 2023-11-25T04:23:19.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7654999999999999e-05 2023-11-25T04:23:19.439 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:19.439 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:19.439 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:19.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.440 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:22 3.e 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:19.440 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:19.440 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.354573047999999, 2023-11-25T04:23:19.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010133422, 2023-11-25T04:23:19.440 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:19.440 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:19.440 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:19.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:19.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:19.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:19.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:19.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:19.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:19.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:19.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034135+0000", 2023-11-25T04:23:19.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-07 2023-11-25T04:23:19.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:19.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034138+0000", 2023-11-25T04:23:19.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6009999999999999e-06 2023-11-25T04:23:19.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:19.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034145+0000", 2023-11-25T04:23:19.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1169999999999998e-06 2023-11-25T04:23:19.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:19.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034150+0000", 2023-11-25T04:23:19.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9080000000000003e-06 2023-11-25T04:23:19.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:19.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034209+0000", 2023-11-25T04:23:19.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8149000000000002e-05 2023-11-25T04:23:19.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:19.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034415+0000", 2023-11-25T04:23:19.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020637699999999999 2023-11-25T04:23:19.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:19.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034584+0000", 2023-11-25T04:23:19.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016870399999999999 2023-11-25T04:23:19.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:19.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.036979+0000", 2023-11-25T04:23:19.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023957509999999998 2023-11-25T04:23:19.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:19.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044161+0000", 2023-11-25T04:23:19.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0071811590000000003 2023-11-25T04:23:19.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:19.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044224+0000", 2023-11-25T04:23:19.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3454000000000006e-05 2023-11-25T04:23:19.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:19.445 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044268+0000", 2023-11-25T04:23:19.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3761999999999999e-05 2023-11-25T04:23:19.445 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:19.445 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:19.445 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:19.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.445 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:23 3.e 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:19.445 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:19.445 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.354541638999997, 2023-11-25T04:23:19.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010281676, 2023-11-25T04:23:19.445 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:19.445 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:19.445 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:19.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:19.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:19.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:19.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:19.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:19.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:19.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:19.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:19.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5799999999999999e-07 2023-11-25T04:23:19.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.447 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:19.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034169+0000", 2023-11-25T04:23:19.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1879999999999998e-06 2023-11-25T04:23:19.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.447 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:19.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034177+0000", 2023-11-25T04:23:19.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7020000000000001e-06 2023-11-25T04:23:19.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:19.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034184+0000", 2023-11-25T04:23:19.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3910000000000003e-06 2023-11-25T04:23:19.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:19.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034924+0000", 2023-11-25T04:23:19.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00074032999999999998 2023-11-25T04:23:19.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:19.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034991+0000", 2023-11-25T04:23:19.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7347000000000004e-05 2023-11-25T04:23:19.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.449 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:19.449 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035096+0000", 2023-11-25T04:23:19.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010488 2023-11-25T04:23:19.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.449 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:19.449 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037588+0000", 2023-11-25T04:23:19.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024918309999999999 2023-11-25T04:23:19.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.449 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:19.449 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044367+0000", 2023-11-25T04:23:19.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067787589999999997 2023-11-25T04:23:19.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.450 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:19.450 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044415+0000", 2023-11-25T04:23:19.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7803000000000003e-05 2023-11-25T04:23:19.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.450 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:19.450 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044447+0000", 2023-11-25T04:23:19.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2886999999999999e-05 2023-11-25T04:23:19.450 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:19.450 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:19.450 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:19.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.451 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:30 3.15 3:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:19.451 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:19.451 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.354483199000001, 2023-11-25T04:23:19.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010365070000000001, 2023-11-25T04:23:19.451 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:19.451 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:19.451 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:19.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.451 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:19.451 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:19.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:19.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:19.452 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:19.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:19.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:19.452 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034225+0000", 2023-11-25T04:23:19.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7400000000000002e-07 2023-11-25T04:23:19.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:19.453 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034229+0000", 2023-11-25T04:23:19.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3100000000000002e-06 2023-11-25T04:23:19.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.453 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:19.453 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034239+0000", 2023-11-25T04:23:19.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9140000000000003e-06 2023-11-25T04:23:19.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.453 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:19.453 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034246+0000", 2023-11-25T04:23:19.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0149999999999998e-06 2023-11-25T04:23:19.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.454 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:19.454 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035572+0000", 2023-11-25T04:23:19.454 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001325655 2023-11-25T04:23:19.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.454 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:19.454 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035730+0000", 2023-11-25T04:23:19.454 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015827899999999999 2023-11-25T04:23:19.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.454 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:19.454 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035857+0000", 2023-11-25T04:23:19.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012644699999999999 2023-11-25T04:23:19.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:19.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038737+0000", 2023-11-25T04:23:19.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002880077 2023-11-25T04:23:19.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:19.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044540+0000", 2023-11-25T04:23:19.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058034239999999997 2023-11-25T04:23:19.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:19.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044569+0000", 2023-11-25T04:23:19.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8388999999999999e-05 2023-11-25T04:23:19.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:19.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044589+0000", 2023-11-25T04:23:19.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0686000000000002e-05 2023-11-25T04:23:19.456 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:19.456 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:19.456 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:19.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.456 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:33 3.1b 3:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:19.456 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:19.457 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.354437701000002, 2023-11-25T04:23:19.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099754049999999997, 2023-11-25T04:23:19.457 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:19.457 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:23:19.457 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:19.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:19.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:19.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:19.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:19.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:19.458 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:19.458 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.458 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.458 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:19.458 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:19.458 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4799999999999998e-07 2023-11-25T04:23:19.458 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.458 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.458 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:19.458 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034273+0000", 2023-11-25T04:23:19.458 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7819999999999998e-06 2023-11-25T04:23:19.458 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.458 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.459 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:19.459 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034285+0000", 2023-11-25T04:23:19.459 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2058e-05 2023-11-25T04:23:19.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.459 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:19.459 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034291+0000", 2023-11-25T04:23:19.459 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0680000000000002e-06 2023-11-25T04:23:19.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.459 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:19.459 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034370+0000", 2023-11-25T04:23:19.459 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8452e-05 2023-11-25T04:23:19.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:19.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034477+0000", 2023-11-25T04:23:19.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107471 2023-11-25T04:23:19.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:19.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044245+0000", 2023-11-25T04:23:19.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0097679259999999997 2023-11-25T04:23:19.460 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:19.460 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:19.460 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:19.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.461 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:36 3.1a 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:19.461 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:19.461 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.354391028000002, 2023-11-25T04:23:19.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058179099999999999, 2023-11-25T04:23:19.461 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:19.461 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:23:19.461 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:19.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:19.461 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:19.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:19.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.462 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:19.462 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:19.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:19.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.462 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:19.462 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034317+0000", 2023-11-25T04:23:19.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1999999999999999e-07 2023-11-25T04:23:19.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.462 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:19.462 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034321+0000", 2023-11-25T04:23:19.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.233e-06 2023-11-25T04:23:19.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.463 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.463 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:19.463 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034337+0000", 2023-11-25T04:23:19.463 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5308999999999999e-05 2023-11-25T04:23:19.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.463 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.463 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:19.463 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034354+0000", 2023-11-25T04:23:19.463 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7411999999999999e-05 2023-11-25T04:23:19.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.463 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.463 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:19.463 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034465+0000", 2023-11-25T04:23:19.463 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011154699999999999 2023-11-25T04:23:19.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.464 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:19.464 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034560+0000", 2023-11-25T04:23:19.464 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4186999999999998e-05 2023-11-25T04:23:19.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.464 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:19.464 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.040134+0000", 2023-11-25T04:23:19.464 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055746019999999997 2023-11-25T04:23:19.464 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:19.464 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:19.464 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:19.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.465 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:9 3.10 e18/16 3:09401a81:::601.00000000:head v 18'4, mlcod=0'0)", 2023-11-25T04:23:19.465 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:19.465 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.34549672, 2023-11-25T04:23:19.465 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061641020000000003, 2023-11-25T04:23:19.465 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:19.465 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:19.465 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:19.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.465 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:19.465 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:19.465 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:19.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:19.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:19.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:19.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:19.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:19.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5300000000000001e-07 2023-11-25T04:23:19.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:19.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043222+0000", 2023-11-25T04:23:19.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.082e-05 2023-11-25T04:23:19.467 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.467 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.467 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:19.467 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043228+0000", 2023-11-25T04:23:19.467 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6699999999999997e-06 2023-11-25T04:23:19.467 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.467 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.467 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:19.467 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043236+0000", 2023-11-25T04:23:19.467 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8189999999999992e-06 2023-11-25T04:23:19.467 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.467 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.467 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:19.467 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043507+0000", 2023-11-25T04:23:19.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000271012 2023-11-25T04:23:19.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.468 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:19.468 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043530+0000", 2023-11-25T04:23:19.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2659000000000001e-05 2023-11-25T04:23:19.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.468 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:19.468 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049321+0000", 2023-11-25T04:23:19.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057915129999999999 2023-11-25T04:23:19.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.468 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:19.469 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049375+0000", 2023-11-25T04:23:19.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3455999999999998e-05 2023-11-25T04:23:19.469 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:19.469 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:19.469 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:19.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.469 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.469 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:1 3.14 e18/16 3:292cf221:::200.00000000:head v 18'3, mlcod=0'0)", 2023-11-25T04:23:19.469 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:19.469 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.345460007, 2023-11-25T04:23:19.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0060983599999999997, 2023-11-25T04:23:19.469 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:19.469 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:19.469 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:19.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.470 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:19.470 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:19.470 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:19.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.470 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:19.470 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:19.470 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:19.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.470 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:19.470 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043248+0000", 2023-11-25T04:23:19.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9399999999999995e-07 2023-11-25T04:23:19.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.471 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:19.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043252+0000", 2023-11-25T04:23:19.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1799999999999998e-06 2023-11-25T04:23:19.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.471 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:19.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043259+0000", 2023-11-25T04:23:19.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0049999999999998e-06 2023-11-25T04:23:19.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.471 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:19.472 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043265+0000", 2023-11-25T04:23:19.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0859999999999998e-06 2023-11-25T04:23:19.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.472 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:19.472 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043311+0000", 2023-11-25T04:23:19.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6082999999999999e-05 2023-11-25T04:23:19.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.472 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:19.472 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043330+0000", 2023-11-25T04:23:19.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8556e-05 2023-11-25T04:23:19.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.473 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:19.473 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049303+0000", 2023-11-25T04:23:19.473 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059733470000000004 2023-11-25T04:23:19.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.473 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:19.473 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049346+0000", 2023-11-25T04:23:19.473 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2608999999999998e-05 2023-11-25T04:23:19.473 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:19.473 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:19.473 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:19.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.473 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:3 3.14 e18/16 3:287034c4:::1.00000000.inode:head v 18'4, mlcod=0'0)", 2023-11-25T04:23:19.474 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:19.474 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.344943501000003, 2023-11-25T04:23:19.474 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057040090000000003, 2023-11-25T04:23:19.474 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:19.474 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:19.474 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:19.474 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.474 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:19.474 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:19.474 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:19.474 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.474 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.474 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:19.475 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:19.475 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:19.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.475 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.475 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:19.475 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:19.475 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9799999999999999e-07 2023-11-25T04:23:19.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.475 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.475 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:19.475 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043785+0000", 2023-11-25T04:23:19.475 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0721999999999999e-05 2023-11-25T04:23:19.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.475 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:19.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043793+0000", 2023-11-25T04:23:19.476 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5839999999999995e-06 2023-11-25T04:23:19.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.476 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:19.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043803+0000", 2023-11-25T04:23:19.476 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0729999999999994e-06 2023-11-25T04:23:19.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.476 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:19.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043843+0000", 2023-11-25T04:23:19.476 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0747999999999997e-05 2023-11-25T04:23:19.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.477 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:19.477 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043856+0000", 2023-11-25T04:23:19.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2697000000000001e-05 2023-11-25T04:23:19.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.477 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:19.477 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049426+0000", 2023-11-25T04:23:19.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055704409999999998 2023-11-25T04:23:19.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.477 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:19.477 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049468+0000", 2023-11-25T04:23:19.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1446000000000003e-05 2023-11-25T04:23:19.478 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:19.478 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:19.478 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:19.478 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.478 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.478 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "pg_update_log_missing(3.14 epoch 18/16 rep_tid 171 entries 18'5 (0'0) error 3:2e2fa760:::200.00000001:head by mds.0.3:32 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:23:19.478 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:19.478 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.344782537999997, 2023-11-25T04:23:19.478 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005556938, 2023-11-25T04:23:19.478 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:19.478 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2023-11-25T04:23:19.479 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:19.479 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.479 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:19.479 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:19.479 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:19.479 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.479 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.479 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:19.479 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:19.479 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:19.479 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.479 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.479 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:19.480 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:19.480 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.03e-07 2023-11-25T04:23:19.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.480 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.480 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:19.480 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043931+0000", 2023-11-25T04:23:19.480 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1079999999999998e-06 2023-11-25T04:23:19.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.480 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.480 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:19.480 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043944+0000", 2023-11-25T04:23:19.480 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2852e-05 2023-11-25T04:23:19.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.480 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.480 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:19.481 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043955+0000", 2023-11-25T04:23:19.481 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1037e-05 2023-11-25T04:23:19.481 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.481 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:19.481 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044088+0000", 2023-11-25T04:23:19.481 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013269599999999999 2023-11-25T04:23:19.481 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.481 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:19.481 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049482+0000", 2023-11-25T04:23:19.481 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053940419999999999 2023-11-25T04:23:19.481 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:19.481 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:19.482 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:19.482 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.482 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.482 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:10 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'5, mlcod=0'0)", 2023-11-25T04:23:19.482 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:19.482 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.344692510000002, 2023-11-25T04:23:19.482 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054493099999999997, 2023-11-25T04:23:19.482 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:19.482 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:19.482 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:19.482 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.482 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:19.482 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:19.483 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:19.483 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.483 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.483 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:19.483 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:19.483 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:19.483 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.483 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.483 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:19.483 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:19.483 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6100000000000002e-07 2023-11-25T04:23:19.483 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.483 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.483 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:19.484 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044019+0000", 2023-11-25T04:23:19.484 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.839e-06 2023-11-25T04:23:19.484 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.484 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.484 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:19.484 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044026+0000", 2023-11-25T04:23:19.484 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5869999999999998e-06 2023-11-25T04:23:19.484 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.484 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.484 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:19.484 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044031+0000", 2023-11-25T04:23:19.484 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4909999999999996e-06 2023-11-25T04:23:19.484 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.484 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.485 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:19.485 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044070+0000", 2023-11-25T04:23:19.485 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8829999999999999e-05 2023-11-25T04:23:19.485 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.485 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.485 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:19.485 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044081+0000", 2023-11-25T04:23:19.485 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1447999999999999e-05 2023-11-25T04:23:19.485 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.485 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.485 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:19.485 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049406+0000", 2023-11-25T04:23:19.485 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053246480000000004 2023-11-25T04:23:19.485 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.486 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.486 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:19.486 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049464+0000", 2023-11-25T04:23:19.486 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8205999999999998e-05 2023-11-25T04:23:19.486 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:19.486 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:19.486 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:19.486 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.486 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.486 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:11 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'6, mlcod=0'0)", 2023-11-25T04:23:19.486 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:19.486 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 32.344491245999997, 2023-11-25T04:23:19.487 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052673499999999996, 2023-11-25T04:23:19.487 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:19.487 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:19.487 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:19.487 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.487 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:19.487 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:19.487 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:19.487 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.487 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.487 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:19.487 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:19.487 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:19.488 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.488 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.488 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:19.488 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:19.488 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0199999999999998e-07 2023-11-25T04:23:19.488 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.488 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.488 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:19.488 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044220+0000", 2023-11-25T04:23:19.488 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7720000000000001e-06 2023-11-25T04:23:19.488 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.488 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.488 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:19.488 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044226+0000", 2023-11-25T04:23:19.489 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8579999999999996e-06 2023-11-25T04:23:19.489 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.489 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.489 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:19.489 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044232+0000", 2023-11-25T04:23:19.489 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1700000000000002e-06 2023-11-25T04:23:19.489 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.489 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.489 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:19.489 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044274+0000", 2023-11-25T04:23:19.489 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1254999999999997e-05 2023-11-25T04:23:19.489 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.489 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.489 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:19.490 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:23:19.490 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6880999999999999e-05 2023-11-25T04:23:19.490 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.490 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.490 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:19.490 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049466+0000", 2023-11-25T04:23:19.490 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051659219999999999 2023-11-25T04:23:19.490 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:19.490 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:19.490 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:19.490 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049483+0000", 2023-11-25T04:23:19.493 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.719e-05 2023-11-25T04:23:19.493 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:19.493 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:19.493 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:19.493 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:19.493 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:19.493 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:19.495 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:23:19.495 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:19.493+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:19.595 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:19.594+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:19.603 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:19.603 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:23:19.603 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2023-11-25T04:23:19.603 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:19.613 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:23:19.696 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:19.695+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:19.758 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:19.758 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:23:19.758 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2023-11-25T04:23:19.758 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2023-11-25T04:23:19.758 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:19.767 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:23:19.807 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:19.806+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:19.898 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:19.897+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:19.913 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:19.913 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2023-11-25T04:23:19.913 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2023-11-25T04:23:19.913 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2023-11-25T04:23:19.914 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.914 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:19.914 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:19.914 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 32.878569278000001, 2023-11-25T04:23:19.914 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01072943, 2023-11-25T04:23:19.914 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:19.914 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:19.914 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:19.914 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.914 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:19.914 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:19.914 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:19.914 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.915 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.915 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:19.915 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:19.915 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:19.915 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.915 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.915 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:19.915 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031829+0000", 2023-11-25T04:23:19.915 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.666e-06 2023-11-25T04:23:19.915 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.915 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.915 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:19.916 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031839+0000", 2023-11-25T04:23:19.916 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9490000000000008e-06 2023-11-25T04:23:19.916 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.916 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.916 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:19.916 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031857+0000", 2023-11-25T04:23:19.916 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8093000000000001e-05 2023-11-25T04:23:19.916 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.916 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.916 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:19.916 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031882+0000", 2023-11-25T04:23:19.916 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4294000000000002e-05 2023-11-25T04:23:19.917 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.917 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.917 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:19.917 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033130+0000", 2023-11-25T04:23:19.917 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001248264 2023-11-25T04:23:19.917 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.917 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.917 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:19.917 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033426+0000", 2023-11-25T04:23:19.917 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029620999999999998 2023-11-25T04:23:19.917 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.917 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.917 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:19.918 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033814+0000", 2023-11-25T04:23:19.918 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000387742 2023-11-25T04:23:19.918 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.918 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.918 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:19.918 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036793+0000", 2023-11-25T04:23:19.918 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002979598 2023-11-25T04:23:19.918 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.918 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.918 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:19.918 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042455+0000", 2023-11-25T04:23:19.919 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0056619260000000003 2023-11-25T04:23:19.919 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.919 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.919 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:19.919 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042502+0000", 2023-11-25T04:23:19.919 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7139999999999999e-05 2023-11-25T04:23:19.919 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.919 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.919 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:19.920 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042557+0000", 2023-11-25T04:23:19.920 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4548e-05 2023-11-25T04:23:19.920 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:19.920 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:19.920 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:19.920 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.920 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.920 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:19.920 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:19.920 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 32.877370743, 2023-11-25T04:23:19.920 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011505202000000001, 2023-11-25T04:23:19.920 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:19.921 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:19.921 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:19.921 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.921 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:19.921 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:19.921 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:19.921 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.921 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.921 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:19.921 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:19.921 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:19.921 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.922 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.922 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:19.922 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033027+0000", 2023-11-25T04:23:19.922 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0699999999999999e-06 2023-11-25T04:23:19.922 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.922 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.922 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:19.922 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033035+0000", 2023-11-25T04:23:19.922 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0830000000000004e-06 2023-11-25T04:23:19.922 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.922 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.922 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:19.922 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033059+0000", 2023-11-25T04:23:19.923 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3975e-05 2023-11-25T04:23:19.923 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.923 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.923 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:19.923 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033482+0000", 2023-11-25T04:23:19.923 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042253700000000001 2023-11-25T04:23:19.923 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.923 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.923 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:19.923 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034520+0000", 2023-11-25T04:23:19.923 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001037991 2023-11-25T04:23:19.923 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.924 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.924 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:19.924 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034611+0000", 2023-11-25T04:23:19.924 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1113999999999995e-05 2023-11-25T04:23:19.924 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.924 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.924 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:19.924 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034769+0000", 2023-11-25T04:23:19.924 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000157641 2023-11-25T04:23:19.924 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.924 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.924 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:19.924 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038292+0000", 2023-11-25T04:23:19.925 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035231160000000002 2023-11-25T04:23:19.925 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.925 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.925 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:19.925 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044440+0000", 2023-11-25T04:23:19.925 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0061486980000000002 2023-11-25T04:23:19.925 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.925 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.925 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:19.925 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044495+0000", 2023-11-25T04:23:19.925 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4339000000000003e-05 2023-11-25T04:23:19.925 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.925 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.926 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:19.926 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044531+0000", 2023-11-25T04:23:19.926 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6637999999999997e-05 2023-11-25T04:23:19.926 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:19.926 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:19.926 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:19.926 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.926 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.926 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:19.926 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:19.926 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 32.876896389000002, 2023-11-25T04:23:19.926 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0057634039999999997, 2023-11-25T04:23:19.927 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:19.927 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:19.927 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:19.927 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.927 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:19.927 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:19.927 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:19.927 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.927 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.927 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:19.927 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:19.928 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:19.928 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.928 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.928 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:19.928 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:19.928 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7900000000000007e-07 2023-11-25T04:23:19.928 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.928 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.928 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:19.929 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033507+0000", 2023-11-25T04:23:19.929 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8900000000000004e-06 2023-11-25T04:23:19.929 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.929 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.929 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:19.929 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033519+0000", 2023-11-25T04:23:19.929 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1756000000000001e-05 2023-11-25T04:23:19.929 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.929 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.929 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:19.929 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033527+0000", 2023-11-25T04:23:19.929 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3610000000000001e-06 2023-11-25T04:23:19.930 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.930 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.930 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:19.930 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034451+0000", 2023-11-25T04:23:19.930 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00092372400000000001 2023-11-25T04:23:19.930 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.930 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.930 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:19.930 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034809+0000", 2023-11-25T04:23:19.930 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00035757699999999998 2023-11-25T04:23:19.930 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.931 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.931 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:23:19.931 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034965+0000", 2023-11-25T04:23:19.931 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015638699999999999 2023-11-25T04:23:19.931 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.931 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.931 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:19.931 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038299+0000", 2023-11-25T04:23:19.931 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0033336559999999999 2023-11-25T04:23:19.931 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.931 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.931 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:19.931 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039169+0000", 2023-11-25T04:23:19.932 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00087066400000000005 2023-11-25T04:23:19.932 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.932 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.932 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:19.932 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039226+0000", 2023-11-25T04:23:19.932 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6511999999999997e-05 2023-11-25T04:23:19.932 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.932 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.932 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:19.932 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039264+0000", 2023-11-25T04:23:19.933 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7898000000000002e-05 2023-11-25T04:23:19.933 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:19.933 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:19.933 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:19.933 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.933 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.933 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:25 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:19.933 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:19.933 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 32.876817805000002, 2023-11-25T04:23:19.933 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096656169999999996, 2023-11-25T04:23:19.933 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:19.933 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:19.934 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:19.934 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.934 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:19.934 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:19.934 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:19.934 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.934 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.934 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:19.934 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:19.934 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:19.934 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.934 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.935 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:19.935 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033580+0000", 2023-11-25T04:23:19.935 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8199999999999999e-07 2023-11-25T04:23:19.935 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.935 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.935 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:19.935 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033585+0000", 2023-11-25T04:23:19.935 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3680000000000001e-06 2023-11-25T04:23:19.935 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.935 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.935 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:19.935 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033600+0000", 2023-11-25T04:23:19.935 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4331000000000001e-05 2023-11-25T04:23:19.936 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.936 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.936 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:19.936 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033606+0000", 2023-11-25T04:23:19.936 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5780000000000004e-06 2023-11-25T04:23:19.936 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.936 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.936 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:19.936 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035747+0000", 2023-11-25T04:23:19.936 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0021406530000000002 2023-11-25T04:23:19.936 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.936 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.937 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:19.937 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035901+0000", 2023-11-25T04:23:19.937 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000153931 2023-11-25T04:23:19.937 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.937 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.937 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:23:19.937 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036193+0000", 2023-11-25T04:23:19.937 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029211299999999999 2023-11-25T04:23:19.937 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.937 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.937 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:19.937 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038655+0000", 2023-11-25T04:23:19.937 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0024621640000000002 2023-11-25T04:23:19.938 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.938 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.938 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:19.938 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043130+0000", 2023-11-25T04:23:19.938 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.004474698 2023-11-25T04:23:19.938 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.938 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.938 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:19.938 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043204+0000", 2023-11-25T04:23:19.938 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4091999999999995e-05 2023-11-25T04:23:19.938 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.938 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.938 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:19.939 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043245+0000", 2023-11-25T04:23:19.939 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0907e-05 2023-11-25T04:23:19.939 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:19.939 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:19.939 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:19.939 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.939 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.939 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:26 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:19.939 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:19.939 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 32.876781981000001, 2023-11-25T04:23:19.939 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011086945000000001, 2023-11-25T04:23:19.939 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:19.940 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:19.940 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:19.940 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.940 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:19.940 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:19.940 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:19.940 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.940 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.940 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:19.940 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:19.940 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:19.940 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.940 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.941 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:19.941 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:19.941 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1699999999999998e-07 2023-11-25T04:23:19.941 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.941 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.941 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:19.941 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033627+0000", 2023-11-25T04:23:19.941 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.163e-05 2023-11-25T04:23:19.941 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.941 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.941 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:19.941 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033636+0000", 2023-11-25T04:23:19.941 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6859999999999995e-06 2023-11-25T04:23:19.942 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.942 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.942 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:19.942 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033642+0000", 2023-11-25T04:23:19.942 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3149999999999997e-06 2023-11-25T04:23:19.942 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.942 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.942 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:19.942 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035414+0000", 2023-11-25T04:23:19.942 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0017716380000000001 2023-11-25T04:23:19.942 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.943 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.943 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:19.943 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035539+0000", 2023-11-25T04:23:19.943 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000125477 2023-11-25T04:23:19.943 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.943 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.943 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:19.943 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035672+0000", 2023-11-25T04:23:19.943 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000133316 2023-11-25T04:23:19.943 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.943 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.943 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:19.943 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038649+0000", 2023-11-25T04:23:19.944 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029764679999999999 2023-11-25T04:23:19.944 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.944 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.944 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:19.944 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044578+0000", 2023-11-25T04:23:19.944 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0059293510000000002 2023-11-25T04:23:19.944 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.944 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.944 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:19.944 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044651+0000", 2023-11-25T04:23:19.944 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2513000000000002e-05 2023-11-25T04:23:19.944 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.945 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.945 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:19.945 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044702+0000", 2023-11-25T04:23:19.945 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1034e-05 2023-11-25T04:23:19.945 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:19.945 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:19.945 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:19.945 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.945 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.945 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:29 3.b 3:d0630e4c:::mds0_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:19.945 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:19.945 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 32.876744060999997, 2023-11-25T04:23:19.946 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010980227, 2023-11-25T04:23:19.946 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:19.946 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:19.946 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:19.946 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.946 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:19.946 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:19.946 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:19.946 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.946 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.946 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:19.946 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:19.947 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:19.947 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.947 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.947 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:19.947 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:19.947 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3499999999999996e-07 2023-11-25T04:23:19.947 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.947 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.947 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:19.947 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033658+0000", 2023-11-25T04:23:19.947 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1280000000000001e-06 2023-11-25T04:23:19.947 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.947 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.948 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:19.948 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033667+0000", 2023-11-25T04:23:19.948 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1430000000000004e-06 2023-11-25T04:23:19.948 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.948 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.948 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:19.948 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033678+0000", 2023-11-25T04:23:19.948 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0906e-05 2023-11-25T04:23:19.948 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.948 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.948 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:19.948 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034456+0000", 2023-11-25T04:23:19.949 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00077833899999999996 2023-11-25T04:23:19.949 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.949 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.949 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:19.949 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034575+0000", 2023-11-25T04:23:19.949 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011926000000000001 2023-11-25T04:23:19.949 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.949 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.949 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:19.949 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035001+0000", 2023-11-25T04:23:19.949 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042550999999999998 2023-11-25T04:23:19.949 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.949 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.950 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:19.950 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.037923+0000", 2023-11-25T04:23:19.950 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029219670000000001 2023-11-25T04:23:19.950 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.950 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.950 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:19.950 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044538+0000", 2023-11-25T04:23:19.950 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0066152349999999997 2023-11-25T04:23:19.950 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.950 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.950 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:19.950 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044594+0000", 2023-11-25T04:23:19.951 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5593000000000001e-05 2023-11-25T04:23:19.951 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.951 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.951 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:19.951 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044633+0000", 2023-11-25T04:23:19.951 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9511000000000002e-05 2023-11-25T04:23:19.951 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:19.951 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:19.951 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:19.951 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.951 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.951 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:34 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:19.952 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:19.952 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 32.876493947999997, 2023-11-25T04:23:19.952 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010920445000000001, 2023-11-25T04:23:19.952 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:19.952 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:19.952 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:19.952 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.952 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:19.952 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:19.952 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:19.952 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.952 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.952 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:19.953 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:19.953 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:19.953 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.953 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.953 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:19.953 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033904+0000", 2023-11-25T04:23:19.953 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6599999999999994e-07 2023-11-25T04:23:19.953 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.953 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.953 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:19.953 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033909+0000", 2023-11-25T04:23:19.953 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5639999999999996e-06 2023-11-25T04:23:19.953 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.954 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.954 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:19.954 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033923+0000", 2023-11-25T04:23:19.954 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3028e-05 2023-11-25T04:23:19.954 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.954 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.954 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:19.954 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034043+0000", 2023-11-25T04:23:19.954 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000120328 2023-11-25T04:23:19.954 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.954 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.954 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:19.955 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035576+0000", 2023-11-25T04:23:19.955 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015333720000000001 2023-11-25T04:23:19.955 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.955 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.955 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:19.955 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035699+0000", 2023-11-25T04:23:19.955 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000123001 2023-11-25T04:23:19.955 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.955 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.955 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:19.955 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044823+0000", 2023-11-25T04:23:19.956 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0091241859999999994 2023-11-25T04:23:19.956 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:19.956 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:19.956 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:19.956 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.956 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.956 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:35 3.1f 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:19.956 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:19.956 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 32.875189743, 2023-11-25T04:23:19.956 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096164779999999995, 2023-11-25T04:23:19.956 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:19.956 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:19.957 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:19.957 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.957 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:19.957 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:19.957 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:19.957 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.957 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.957 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:19.957 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:19.957 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:19.957 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.957 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.957 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:19.958 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035208+0000", 2023-11-25T04:23:19.958 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2699999999999998e-07 2023-11-25T04:23:19.958 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.958 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.958 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:19.958 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035213+0000", 2023-11-25T04:23:19.958 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.6070000000000001e-06 2023-11-25T04:23:19.958 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.958 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.958 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:19.958 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035350+0000", 2023-11-25T04:23:19.958 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013693599999999999 2023-11-25T04:23:19.959 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.959 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.959 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:19.959 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035358+0000", 2023-11-25T04:23:19.959 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0099999999999995e-06 2023-11-25T04:23:19.959 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.959 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.959 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:19.959 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036260+0000", 2023-11-25T04:23:19.959 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00090240100000000005 2023-11-25T04:23:19.959 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.959 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.960 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:19.960 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036517+0000", 2023-11-25T04:23:19.960 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00025736299999999998 2023-11-25T04:23:19.960 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.960 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.960 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:19.960 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044824+0000", 2023-11-25T04:23:19.960 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0083063340000000003 2023-11-25T04:23:19.960 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:19.960 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:19.960 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:19.960 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.960 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.961 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:37 3.1e 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:19.961 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:19.961 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 32.875023057, 2023-11-25T04:23:19.961 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0078322640000000002, 2023-11-25T04:23:19.961 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:19.961 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:19.961 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:19.961 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.961 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:19.961 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:19.961 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:19.961 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.961 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.962 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:19.962 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:19.962 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:19.962 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.962 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.962 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:19.962 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035375+0000", 2023-11-25T04:23:19.962 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9100000000000003e-07 2023-11-25T04:23:19.962 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.962 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.962 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:19.962 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035379+0000", 2023-11-25T04:23:19.963 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9430000000000002e-06 2023-11-25T04:23:19.963 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.963 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.963 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:19.963 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035388+0000", 2023-11-25T04:23:19.963 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2760000000000006e-06 2023-11-25T04:23:19.963 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.963 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.963 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:19.963 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035396+0000", 2023-11-25T04:23:19.963 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8480000000000001e-06 2023-11-25T04:23:19.963 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.964 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.964 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:19.964 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035452+0000", 2023-11-25T04:23:19.964 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5831000000000001e-05 2023-11-25T04:23:19.964 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.964 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.964 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:19.964 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035934+0000", 2023-11-25T04:23:19.964 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00048265599999999998 2023-11-25T04:23:19.964 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.964 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.964 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:19.965 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043206+0000", 2023-11-25T04:23:19.965 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.007271919 2023-11-25T04:23:19.965 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:19.965 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:19.965 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:19.965 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.965 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.965 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing(3.1a epoch 18/16 rep_tid 3 entries 18'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:36 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:23:19.965 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:19.965 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 32.873747471000001, 2023-11-25T04:23:19.965 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002073414, 2023-11-25T04:23:19.965 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:19.966 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:19.966 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:19.966 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.966 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:19.966 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:19.966 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:19.966 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.966 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.966 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:19.966 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:19.966 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:19.966 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.966 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.967 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:19.967 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036650+0000", 2023-11-25T04:23:19.967 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7300000000000002e-07 2023-11-25T04:23:19.967 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.967 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.967 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:19.967 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036659+0000", 2023-11-25T04:23:19.967 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.7499999999999992e-06 2023-11-25T04:23:19.967 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.967 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.967 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:19.967 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036673+0000", 2023-11-25T04:23:19.968 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4917e-05 2023-11-25T04:23:19.968 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.968 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.968 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:19.968 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036687+0000", 2023-11-25T04:23:19.968 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3883000000000001e-05 2023-11-25T04:23:19.968 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.968 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.968 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:19.968 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036781+0000", 2023-11-25T04:23:19.968 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3219000000000003e-05 2023-11-25T04:23:19.968 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.968 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.969 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:19.969 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038723+0000", 2023-11-25T04:23:19.969 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019422720000000001 2023-11-25T04:23:19.969 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:19.969 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:19.969 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:19.969 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.969 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.969 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:28 3.13 e18/16 3:cd5a64a3:::100.00000000:head v 18'1, mlcod=0'0)", 2023-11-25T04:23:19.969 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:19.969 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 32.870086313000002, 2023-11-25T04:23:19.969 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001850546, 2023-11-25T04:23:19.969 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:19.970 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:19.970 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:19.970 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.970 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:19.970 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:19.970 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:19.970 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.970 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.970 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:19.970 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:19.970 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:19.970 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.971 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.971 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:19.971 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:19.971 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7800000000000002e-07 2023-11-25T04:23:19.971 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.971 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.971 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:19.971 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040331+0000", 2023-11-25T04:23:19.971 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0449999999999999e-05 2023-11-25T04:23:19.971 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.971 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.971 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:19.971 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040339+0000", 2023-11-25T04:23:19.972 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0709999999999995e-06 2023-11-25T04:23:19.972 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.972 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.972 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:19.972 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040347+0000", 2023-11-25T04:23:19.972 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7079999999999997e-06 2023-11-25T04:23:19.972 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.972 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.972 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:19.972 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040410+0000", 2023-11-25T04:23:19.972 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2453000000000007e-05 2023-11-25T04:23:19.972 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.973 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.973 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:19.973 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040422+0000", 2023-11-25T04:23:19.973 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2429000000000001e-05 2023-11-25T04:23:19.973 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.973 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.973 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:19.973 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042129+0000", 2023-11-25T04:23:19.973 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001707266 2023-11-25T04:23:19.973 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.973 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.973 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:19.973 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042161+0000", 2023-11-25T04:23:19.974 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1791000000000001e-05 2023-11-25T04:23:19.974 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:19.974 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:19.974 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:19.974 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.974 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.974 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:27 3.13 e18/16 3:c91d4a1d:::mds0_inotable:head v 18'2, mlcod=0'0)", 2023-11-25T04:23:19.974 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:19.974 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 32.869882789999998, 2023-11-25T04:23:19.974 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0016886430000000001, 2023-11-25T04:23:19.974 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:19.974 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:19.975 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:19.975 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.975 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:19.975 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:19.975 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:19.975 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.975 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.975 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:19.975 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:19.975 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:19.975 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.975 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.976 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:19.976 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:19.976 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3999999999999997e-07 2023-11-25T04:23:19.976 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.976 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.976 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:19.976 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040520+0000", 2023-11-25T04:23:19.976 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2989999999999996e-06 2023-11-25T04:23:19.976 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.976 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.976 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:19.976 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040527+0000", 2023-11-25T04:23:19.976 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5730000000000002e-06 2023-11-25T04:23:19.977 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.977 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.977 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:19.977 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040535+0000", 2023-11-25T04:23:19.977 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8630000000000008e-06 2023-11-25T04:23:19.977 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.977 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.977 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:19.977 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040683+0000", 2023-11-25T04:23:19.978 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000148002 2023-11-25T04:23:19.978 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.978 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.978 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:19.978 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040700+0000", 2023-11-25T04:23:19.978 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7065e-05 2023-11-25T04:23:19.978 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.978 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.978 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:19.978 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042163+0000", 2023-11-25T04:23:19.978 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001462767 2023-11-25T04:23:19.978 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.979 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.979 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:19.979 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042203+0000", 2023-11-25T04:23:19.979 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9734000000000003e-05 2023-11-25T04:23:19.979 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:19.979 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:19.979 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:19.979 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.979 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.979 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:20 3.1c e18/16 3:3ed09add:::607.00000000:head v 18'1, mlcod=0'0)", 2023-11-25T04:23:19.979 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:19.979 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 32.869847749000002, 2023-11-25T04:23:19.979 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015271, 2023-11-25T04:23:19.980 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:19.980 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:19.980 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:19.980 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.980 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:19.980 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:19.980 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:19.980 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.980 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.980 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:19.980 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:19.980 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:19.980 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.981 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.981 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:19.981 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:19.981 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4799999999999999e-07 2023-11-25T04:23:19.981 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.981 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.981 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:19.981 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040561+0000", 2023-11-25T04:23:19.981 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1800000000000001e-05 2023-11-25T04:23:19.981 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.981 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.981 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:19.981 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040568+0000", 2023-11-25T04:23:19.982 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7730000000000002e-06 2023-11-25T04:23:19.982 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.982 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.982 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:19.982 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040587+0000", 2023-11-25T04:23:19.982 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8691000000000001e-05 2023-11-25T04:23:19.982 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.982 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.982 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:19.982 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040638+0000", 2023-11-25T04:23:19.982 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1068e-05 2023-11-25T04:23:19.982 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.982 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.983 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:19.983 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040670+0000", 2023-11-25T04:23:19.983 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2125999999999997e-05 2023-11-25T04:23:19.983 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.983 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.983 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:19.983 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042032+0000", 2023-11-25T04:23:19.983 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001361821 2023-11-25T04:23:19.983 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.983 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.983 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:19.983 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042076+0000", 2023-11-25T04:23:19.984 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4472999999999999e-05 2023-11-25T04:23:19.984 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:19.984 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:19.984 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:19.984 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.984 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.984 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:21 3.1c e18/16 3:3ed09add:::607.00000000:head v 18'2, mlcod=0'0)", 2023-11-25T04:23:19.984 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:19.984 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 32.869613024000003, 2023-11-25T04:23:19.984 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0013228160000000001, 2023-11-25T04:23:19.984 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:19.984 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:19.984 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:19.985 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.985 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:19.985 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:19.985 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:19.985 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.985 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.985 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:19.985 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:19.985 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:19.985 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.985 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.985 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:19.985 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:19.986 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9299999999999999e-07 2023-11-25T04:23:19.986 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.986 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.986 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:19.986 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040790+0000", 2023-11-25T04:23:19.986 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5770000000000001e-06 2023-11-25T04:23:19.986 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.986 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.986 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:19.986 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040799+0000", 2023-11-25T04:23:19.986 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6439999999999995e-06 2023-11-25T04:23:19.986 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.986 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.987 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:19.987 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040806+0000", 2023-11-25T04:23:19.987 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0180000000000004e-06 2023-11-25T04:23:19.987 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.987 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.987 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:19.987 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040907+0000", 2023-11-25T04:23:19.987 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000100096 2023-11-25T04:23:19.987 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.987 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.987 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:19.987 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040925+0000", 2023-11-25T04:23:19.988 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8536999999999999e-05 2023-11-25T04:23:19.988 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.988 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.988 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:19.988 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042079+0000", 2023-11-25T04:23:19.988 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00115358 2023-11-25T04:23:19.988 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.988 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.988 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:19.988 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042107+0000", 2023-11-25T04:23:19.988 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7971e-05 2023-11-25T04:23:19.988 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:19.988 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:19.989 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:19.989 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.989 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.989 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop_reply(mds.0.3:4 3.1f e18/16 ondisk, result = 0)", 2023-11-25T04:23:19.989 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:19.989 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 32.868273633999998, 2023-11-25T04:23:19.989 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00044092900000000001, 2023-11-25T04:23:19.989 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:19.989 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:19.989 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:19.989 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.989 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:19.989 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:19.990 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:19.990 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.990 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.990 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:19.990 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:19.990 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:19.990 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.990 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.990 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:19.990 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042124+0000", 2023-11-25T04:23:19.990 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1800000000000005e-07 2023-11-25T04:23:19.991 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.991 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.991 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:19.991 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042128+0000", 2023-11-25T04:23:19.991 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2289999999999997e-06 2023-11-25T04:23:19.991 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.991 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.991 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:19.991 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042138+0000", 2023-11-25T04:23:19.991 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9729999999999992e-06 2023-11-25T04:23:19.991 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.991 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.991 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:19.992 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042145+0000", 2023-11-25T04:23:19.992 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.336e-06 2023-11-25T04:23:19.992 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.992 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.992 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:19.992 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042441+0000", 2023-11-25T04:23:19.992 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029650500000000002 2023-11-25T04:23:19.992 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.992 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.992 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:19.992 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042448+0000", 2023-11-25T04:23:19.992 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4100000000000002e-06 2023-11-25T04:23:19.993 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.993 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.993 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:19.993 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042564+0000", 2023-11-25T04:23:19.993 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000115758 2023-11-25T04:23:19.993 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:19.993 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:19.993 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:19.993 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.993 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.993 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing_reply(3.1f epoch 18/16 rep_tid 8 lcod 18'4)", 2023-11-25T04:23:19.993 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:19.993 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 32.865948222, 2023-11-25T04:23:19.994 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00038154699999999998, 2023-11-25T04:23:19.994 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:19.994 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:19.994 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:19.994 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.994 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:19.994 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:19.994 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:19.994 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.994 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.994 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:19.994 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:19.995 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:19.995 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.995 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.995 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:19.995 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:19.995 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2700000000000001e-07 2023-11-25T04:23:19.995 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.995 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.995 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:19.995 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044451+0000", 2023-11-25T04:23:19.995 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7460000000000001e-06 2023-11-25T04:23:19.995 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.996 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.996 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:19.996 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044455+0000", 2023-11-25T04:23:19.996 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5809999999999998e-06 2023-11-25T04:23:19.996 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.996 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.996 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:19.996 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044458+0000", 2023-11-25T04:23:19.996 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0170000000000001e-06 2023-11-25T04:23:19.996 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.996 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.996 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:19.996 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044734+0000", 2023-11-25T04:23:19.997 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00027582799999999999 2023-11-25T04:23:19.997 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.997 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.997 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:19.997 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044830+0000", 2023-11-25T04:23:19.997 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6248000000000004e-05 2023-11-25T04:23:19.997 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:19.997 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:19.997 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:19.997 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.997 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.997 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing_reply(3.b epoch 18/16 rep_tid 5 lcod 18'2)", 2023-11-25T04:23:19.997 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:19.998 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 32.865911560999997, 2023-11-25T04:23:19.998 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000344488, 2023-11-25T04:23:19.998 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:19.998 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:19.998 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:19.998 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.998 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:19.998 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:19.998 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:19.998 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.998 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.998 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:19.999 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:19.999 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:19.999 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.999 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.999 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:19.999 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:19.999 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.55e-07 2023-11-25T04:23:19.999 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:19.999 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:19.999 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:19.999 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044487+0000", 2023-11-25T04:23:19.999 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2470000000000001e-06 2023-11-25T04:23:19.999 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.000 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.000 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:20.000 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044491+0000", 2023-11-25T04:23:20.000 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9659999999999998e-06 2023-11-25T04:23:20.000 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.000 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.000 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:20.000 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044494+0000", 2023-11-25T04:23:20.000 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2310000000000001e-06 2023-11-25T04:23:20.000 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.000 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.000 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:20.001 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044756+0000", 2023-11-25T04:23:20.001 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00026172800000000003 2023-11-25T04:23:20.001 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.001 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.001 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:20.001 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044830+0000", 2023-11-25T04:23:20.001 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4160999999999997e-05 2023-11-25T04:23:20.001 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:20.001 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:20.001 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:20.001 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.001 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.001 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:39 3.6 3:654134d2:::mds0_openfiles.0:head [omap-clear,omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2023-11-25T04:23:20.002 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:20.002 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 27.806085519, 2023-11-25T04:23:20.002 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001788658, 2023-11-25T04:23:20.002 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:20.002 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:20.002 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:20.002 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.002 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:20.002 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:20.002 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:20.002 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.002 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.002 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:20.003 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:20.003 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:20.003 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.003 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.003 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:20.003 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104314+0000", 2023-11-25T04:23:20.003 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3099999999999999e-06 2023-11-25T04:23:20.003 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.003 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.003 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:20.003 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104325+0000", 2023-11-25T04:23:20.003 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1511999999999999e-05 2023-11-25T04:23:20.004 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.004 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.004 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:20.004 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104362+0000", 2023-11-25T04:23:20.004 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6903000000000002e-05 2023-11-25T04:23:20.004 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.004 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.004 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:20.004 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104387+0000", 2023-11-25T04:23:20.004 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4766e-05 2023-11-25T04:23:20.004 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.004 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.004 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:20.005 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104465+0000", 2023-11-25T04:23:20.005 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8119000000000002e-05 2023-11-25T04:23:20.005 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.005 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.005 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:20.005 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104557+0000", 2023-11-25T04:23:20.005 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2496000000000003e-05 2023-11-25T04:23:20.005 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.005 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.005 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:20.005 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104646+0000", 2023-11-25T04:23:20.005 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8818000000000002e-05 2023-11-25T04:23:20.005 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.006 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.006 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:20.006 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105201+0000", 2023-11-25T04:23:20.006 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00055506000000000002 2023-11-25T04:23:20.006 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.006 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.006 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:20.006 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105994+0000", 2023-11-25T04:23:20.006 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00079294700000000001 2023-11-25T04:23:20.006 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.006 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.007 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:20.007 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.106055+0000", 2023-11-25T04:23:20.007 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0566000000000003e-05 2023-11-25T04:23:20.007 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.007 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.007 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:20.007 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.106100+0000", 2023-11-25T04:23:20.007 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5160999999999997e-05 2023-11-25T04:23:20.007 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:20.007 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:20.007 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:20.007 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.008 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.008 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:43 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2023-11-25T04:23:20.008 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:20.008 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 12.805860523, 2023-11-25T04:23:20.008 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001751166, 2023-11-25T04:23:20.008 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:20.008 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:20.008 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:20.008 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.008 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:20.008 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:20.008 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:20.009 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.009 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.009 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:20.009 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:20.009 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:20.009 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.009 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.009 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:20.009 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104539+0000", 2023-11-25T04:23:20.009 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9529999999999999e-06 2023-11-25T04:23:20.009 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.009 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.010 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:20.010 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104557+0000", 2023-11-25T04:23:20.010 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7464000000000001e-05 2023-11-25T04:23:20.010 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.010 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.010 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:20.010 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104590+0000", 2023-11-25T04:23:20.010 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2966000000000003e-05 2023-11-25T04:23:20.010 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.010 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.010 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:20.010 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104608+0000", 2023-11-25T04:23:20.011 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8193e-05 2023-11-25T04:23:20.011 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.011 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.011 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:20.011 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104767+0000", 2023-11-25T04:23:20.011 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015882 2023-11-25T04:23:20.011 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.011 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.011 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:20.011 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104909+0000", 2023-11-25T04:23:20.011 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014244999999999999 2023-11-25T04:23:20.011 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.012 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.012 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:20.012 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.105059+0000", 2023-11-25T04:23:20.012 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014964399999999999 2023-11-25T04:23:20.012 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.012 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.012 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:20.012 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.105701+0000", 2023-11-25T04:23:20.012 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00064201099999999995 2023-11-25T04:23:20.012 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.012 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.012 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:20.013 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106233+0000", 2023-11-25T04:23:20.013 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00053178500000000001 2023-11-25T04:23:20.013 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.013 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.013 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:20.013 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106265+0000", 2023-11-25T04:23:20.013 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1986000000000002e-05 2023-11-25T04:23:20.013 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.013 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.013 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:20.013 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106288+0000", 2023-11-25T04:23:20.013 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2894000000000001e-05 2023-11-25T04:23:20.014 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:20.014 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:20.014 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:20.014 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.014 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.014 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:45 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e23)", 2023-11-25T04:23:20.014 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:23:20.014 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 7.8056624829999999, 2023-11-25T04:23:20.014 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00098423299999999994, 2023-11-25T04:23:20.014 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:20.014 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:20.014 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:20.014 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.015 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:20.015 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:23:20.015 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:20.015 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.015 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.015 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:20.015 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:23:20.015 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:20.015 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.015 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.015 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:20.015 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104737+0000", 2023-11-25T04:23:20.016 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3309999999999998e-06 2023-11-25T04:23:20.016 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.016 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.016 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:20.016 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104748+0000", 2023-11-25T04:23:20.016 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1080999999999999e-05 2023-11-25T04:23:20.016 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.016 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.016 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:20.016 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104771+0000", 2023-11-25T04:23:20.016 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2707000000000001e-05 2023-11-25T04:23:20.016 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.017 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.017 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:20.017 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104800+0000", 2023-11-25T04:23:20.017 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9595e-05 2023-11-25T04:23:20.017 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.017 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.017 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:20.017 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104890+0000", 2023-11-25T04:23:20.017 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.9808999999999993e-05 2023-11-25T04:23:20.017 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.017 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.017 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:20.017 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105004+0000", 2023-11-25T04:23:20.018 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000114508 2023-11-25T04:23:20.018 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.018 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.018 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:20.018 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105634+0000", 2023-11-25T04:23:20.018 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00062985799999999996 2023-11-25T04:23:20.018 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.018 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.018 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:20.018 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105671+0000", 2023-11-25T04:23:20.018 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6640000000000002e-05 2023-11-25T04:23:20.018 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.018 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.019 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:20.019 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105719+0000", 2023-11-25T04:23:20.019 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7704000000000003e-05 2023-11-25T04:23:20.019 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:20.019 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:20.019 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:20.019 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:20.019 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:20.019 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:20.021 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:20.019+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:20.021 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:23:20.121 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:20.120+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:20.130 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:20.130 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:23:20.130 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2023-11-25T04:23:20.130 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:20.139 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:23:20.223 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:20.221+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:20.244 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:20.244 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:23:20.245 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2023-11-25T04:23:20.245 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2023-11-25T04:23:20.245 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:20.253 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:23:20.279 INFO:teuthology.orchestra.run.smithi067.stderr:nodeep-scrub is set 2023-11-25T04:23:20.323 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:20.322+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:20.361 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:20.361 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2023-11-25T04:23:20.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2023-11-25T04:23:20.361 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2023-11-25T04:23:20.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.361 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:155 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'109, mlcod=11'105)", 2023-11-25T04:23:20.362 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:20.362 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.573696237999997, 2023-11-25T04:23:20.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061719019999999999, 2023-11-25T04:23:20.362 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:20.362 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:20.362 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:20.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.362 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:20.362 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:20.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:20.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:20.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:20.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:20.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:20.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784884+0000", 2023-11-25T04:23:20.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3099999999999999e-07 2023-11-25T04:23:20.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:20.364 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784942+0000", 2023-11-25T04:23:20.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8570000000000003e-05 2023-11-25T04:23:20.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.364 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:20.364 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784949+0000", 2023-11-25T04:23:20.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0310000000000001e-06 2023-11-25T04:23:20.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.364 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:20.364 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784953+0000", 2023-11-25T04:23:20.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6320000000000001e-06 2023-11-25T04:23:20.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.365 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:20.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784993+0000", 2023-11-25T04:23:20.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9737000000000003e-05 2023-11-25T04:23:20.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.365 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:20.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785011+0000", 2023-11-25T04:23:20.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8349999999999999e-05 2023-11-25T04:23:20.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:20.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791024+0000", 2023-11-25T04:23:20.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006012997 2023-11-25T04:23:20.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:20.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791055+0000", 2023-11-25T04:23:20.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1254000000000003e-05 2023-11-25T04:23:20.368 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:20.368 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:20.368 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:20.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.368 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:156 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'110, mlcod=11'105)", 2023-11-25T04:23:20.368 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:20.369 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.573110188999998, 2023-11-25T04:23:20.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005616116, 2023-11-25T04:23:20.369 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:20.369 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:20.369 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:20.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:20.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:20.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:20.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:20.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:20.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:20.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:20.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785470+0000", 2023-11-25T04:23:20.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0699999999999998e-07 2023-11-25T04:23:20.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:20.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785474+0000", 2023-11-25T04:23:20.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.048e-06 2023-11-25T04:23:20.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:20.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785480+0000", 2023-11-25T04:23:20.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7729999999999996e-06 2023-11-25T04:23:20.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:20.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785483+0000", 2023-11-25T04:23:20.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1970000000000001e-06 2023-11-25T04:23:20.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:20.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787936+0000", 2023-11-25T04:23:20.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024529040000000001 2023-11-25T04:23:20.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:20.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787947+0000", 2023-11-25T04:23:20.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1669e-05 2023-11-25T04:23:20.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:20.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791057+0000", 2023-11-25T04:23:20.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031095200000000002 2023-11-25T04:23:20.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:20.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791086+0000", 2023-11-25T04:23:20.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8697999999999999e-05 2023-11-25T04:23:20.374 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:20.374 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:20.374 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:20.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.374 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:157 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'111, mlcod=11'106)", 2023-11-25T04:23:20.374 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:20.374 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.572517630999997, 2023-11-25T04:23:20.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050590280000000001, 2023-11-25T04:23:20.374 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:20.374 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:20.375 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:20.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:20.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:20.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:20.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:20.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:20.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:20.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:20.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786063+0000", 2023-11-25T04:23:20.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2600000000000002e-07 2023-11-25T04:23:20.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:20.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786069+0000", 2023-11-25T04:23:20.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8839999999999999e-06 2023-11-25T04:23:20.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:20.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786079+0000", 2023-11-25T04:23:20.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0124e-05 2023-11-25T04:23:20.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.377 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:20.377 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786085+0000", 2023-11-25T04:23:20.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6660000000000002e-06 2023-11-25T04:23:20.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.377 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:20.377 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788160+0000", 2023-11-25T04:23:20.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020747669999999999 2023-11-25T04:23:20.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.378 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:20.378 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788180+0000", 2023-11-25T04:23:20.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0208999999999999e-05 2023-11-25T04:23:20.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.378 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:20.378 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791087+0000", 2023-11-25T04:23:20.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029064630000000002 2023-11-25T04:23:20.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:20.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791121+0000", 2023-11-25T04:23:20.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4289000000000003e-05 2023-11-25T04:23:20.379 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:20.379 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:20.379 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:20.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.379 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:158 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'112, mlcod=11'106)", 2023-11-25T04:23:20.379 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:20.379 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.572101238000002, 2023-11-25T04:23:20.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046663030000000001, 2023-11-25T04:23:20.380 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:20.380 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:20.380 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:20.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:20.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:20.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:20.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:20.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:20.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:20.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:20.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786479+0000", 2023-11-25T04:23:20.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2799999999999996e-07 2023-11-25T04:23:20.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:20.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786498+0000", 2023-11-25T04:23:20.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9069e-05 2023-11-25T04:23:20.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:20.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786508+0000", 2023-11-25T04:23:20.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5810000000000003e-06 2023-11-25T04:23:20.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.382 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:20.382 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786512+0000", 2023-11-25T04:23:20.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4889999999999997e-06 2023-11-25T04:23:20.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.382 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:20.382 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788438+0000", 2023-11-25T04:23:20.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019257059999999999 2023-11-25T04:23:20.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:20.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788449+0000", 2023-11-25T04:23:20.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1419e-05 2023-11-25T04:23:20.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:20.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791122+0000", 2023-11-25T04:23:20.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002672822 2023-11-25T04:23:20.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:20.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791145+0000", 2023-11-25T04:23:20.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2589e-05 2023-11-25T04:23:20.384 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:20.384 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:20.385 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:20.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.385 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:206 1.0 e12/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 12'152, mlcod=12'148)", 2023-11-25T04:23:20.385 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:20.385 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.060332768000002, 2023-11-25T04:23:20.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046728189999999999, 2023-11-25T04:23:20.385 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:20.385 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:20.386 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:20.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:20.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:20.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:20.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:20.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:20.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:20.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:20.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298248+0000", 2023-11-25T04:23:20.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3600000000000003e-07 2023-11-25T04:23:20.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:20.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298278+0000", 2023-11-25T04:23:20.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0474e-05 2023-11-25T04:23:20.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.388 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:20.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298288+0000", 2023-11-25T04:23:20.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0156000000000001e-05 2023-11-25T04:23:20.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.388 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:20.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298293+0000", 2023-11-25T04:23:20.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7400000000000004e-06 2023-11-25T04:23:20.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:20.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298333+0000", 2023-11-25T04:23:20.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9666000000000003e-05 2023-11-25T04:23:20.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:20.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298351+0000", 2023-11-25T04:23:20.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8544999999999999e-05 2023-11-25T04:23:20.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:20.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302891+0000", 2023-11-25T04:23:20.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045398519999999996 2023-11-25T04:23:20.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:20.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302920+0000", 2023-11-25T04:23:20.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8750000000000001e-05 2023-11-25T04:23:20.390 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:20.390 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:20.390 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:20.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.391 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:14 3.15 3:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:20.391 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:20.391 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 33.325198411000002, 2023-11-25T04:23:20.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010990437, 2023-11-25T04:23:20.391 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:20.391 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:20.391 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:20.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:20.391 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:20.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:20.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:20.392 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:20.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:20.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:20.392 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033383+0000", 2023-11-25T04:23:20.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9870000000000002e-06 2023-11-25T04:23:20.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:20.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033398+0000", 2023-11-25T04:23:20.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4569999999999999e-05 2023-11-25T04:23:20.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:20.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033411+0000", 2023-11-25T04:23:20.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3543999999999999e-05 2023-11-25T04:23:20.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:20.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033423+0000", 2023-11-25T04:23:20.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1905999999999999e-05 2023-11-25T04:23:20.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:20.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034433+0000", 2023-11-25T04:23:20.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00101022 2023-11-25T04:23:20.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:20.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034539+0000", 2023-11-25T04:23:20.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010597199999999999 2023-11-25T04:23:20.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:20.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034667+0000", 2023-11-25T04:23:20.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012765500000000001 2023-11-25T04:23:20.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:20.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037803+0000", 2023-11-25T04:23:20.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031362220000000001 2023-11-25T04:23:20.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:20.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:23:20.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0064964669999999997 2023-11-25T04:23:20.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:20.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044331+0000", 2023-11-25T04:23:20.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1544999999999997e-05 2023-11-25T04:23:20.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:20.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044372+0000", 2023-11-25T04:23:20.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0349000000000003e-05 2023-11-25T04:23:20.397 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:20.397 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:20.397 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:20.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.397 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:15 3.15 3:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:20.397 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:20.397 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 33.324538760999999, 2023-11-25T04:23:20.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010464362, 2023-11-25T04:23:20.397 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:20.397 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:20.398 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:20.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:20.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:20.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:20.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:20.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:20.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:20.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:20.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034042+0000", 2023-11-25T04:23:20.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0999999999999997e-07 2023-11-25T04:23:20.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:20.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034048+0000", 2023-11-25T04:23:20.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0390000000000001e-06 2023-11-25T04:23:20.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:20.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034059+0000", 2023-11-25T04:23:20.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1182999999999999e-05 2023-11-25T04:23:20.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:20.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034067+0000", 2023-11-25T04:23:20.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1559999999999995e-06 2023-11-25T04:23:20.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:20.401 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035005+0000", 2023-11-25T04:23:20.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00093747399999999999 2023-11-25T04:23:20.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:20.401 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035077+0000", 2023-11-25T04:23:20.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2747e-05 2023-11-25T04:23:20.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:20.401 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035219+0000", 2023-11-25T04:23:20.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000141877 2023-11-25T04:23:20.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:20.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037810+0000", 2023-11-25T04:23:20.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002590752 2023-11-25T04:23:20.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:20.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044437+0000", 2023-11-25T04:23:20.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066270110000000004 2023-11-25T04:23:20.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:20.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044481+0000", 2023-11-25T04:23:20.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4106000000000001e-05 2023-11-25T04:23:20.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:20.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044505+0000", 2023-11-25T04:23:20.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4207e-05 2023-11-25T04:23:20.403 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:20.403 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:20.403 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:20.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.404 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:16 3.1d 3:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:20.404 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:20.404 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 33.324496912999997, 2023-11-25T04:23:20.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010413923, 2023-11-25T04:23:20.404 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:20.404 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:20.404 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:20.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:20.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:20.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:20.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:20.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:20.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:20.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:20.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:20.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4499999999999997e-07 2023-11-25T04:23:20.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:20.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034087+0000", 2023-11-25T04:23:20.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9770000000000002e-06 2023-11-25T04:23:20.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:20.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034094+0000", 2023-11-25T04:23:20.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8809999999999998e-06 2023-11-25T04:23:20.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:20.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034100+0000", 2023-11-25T04:23:20.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8440000000000003e-06 2023-11-25T04:23:20.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:20.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034993+0000", 2023-11-25T04:23:20.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00089336799999999996 2023-11-25T04:23:20.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:20.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035099+0000", 2023-11-25T04:23:20.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106037 2023-11-25T04:23:20.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:20.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035276+0000", 2023-11-25T04:23:20.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017631799999999999 2023-11-25T04:23:20.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:20.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037815+0000", 2023-11-25T04:23:20.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025395959999999999 2023-11-25T04:23:20.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:20.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044420+0000", 2023-11-25T04:23:20.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066045749999999997 2023-11-25T04:23:20.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:20.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044463+0000", 2023-11-25T04:23:20.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3205999999999999e-05 2023-11-25T04:23:20.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:20.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044497+0000", 2023-11-25T04:23:20.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3476000000000003e-05 2023-11-25T04:23:20.410 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:20.410 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:20.410 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:20.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.410 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:17 3.1d 3:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:20.410 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:20.411 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 33.324471703999997, 2023-11-25T04:23:20.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01127356, 2023-11-25T04:23:20.411 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:20.411 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:20.411 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:20.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.411 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:20.411 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:20.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:20.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.412 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:20.412 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:20.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:20.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.412 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:20.412 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:20.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1100000000000001e-07 2023-11-25T04:23:20.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.412 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:20.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034112+0000", 2023-11-25T04:23:20.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8600000000000003e-06 2023-11-25T04:23:20.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:20.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034120+0000", 2023-11-25T04:23:20.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7700000000000001e-06 2023-11-25T04:23:20.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:20.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034127+0000", 2023-11-25T04:23:20.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0999999999999998e-06 2023-11-25T04:23:20.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.414 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:20.414 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035601+0000", 2023-11-25T04:23:20.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014739410000000001 2023-11-25T04:23:20.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.414 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:20.414 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035685+0000", 2023-11-25T04:23:20.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3677999999999994e-05 2023-11-25T04:23:20.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:20.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035827+0000", 2023-11-25T04:23:20.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014232599999999999 2023-11-25T04:23:20.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:20.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038725+0000", 2023-11-25T04:23:20.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028982779999999998 2023-11-25T04:23:20.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:20.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045323+0000", 2023-11-25T04:23:20.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0065980780000000003 2023-11-25T04:23:20.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:20.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045354+0000", 2023-11-25T04:23:20.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0463e-05 2023-11-25T04:23:20.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:20.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045381+0000", 2023-11-25T04:23:20.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7654999999999999e-05 2023-11-25T04:23:20.417 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:20.417 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:20.417 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:20.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.417 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:22 3.e 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:20.417 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:20.417 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 33.324445291000004, 2023-11-25T04:23:20.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010133422, 2023-11-25T04:23:20.417 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:20.418 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:20.418 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:20.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:20.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:20.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:20.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:20.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:20.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:20.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:20.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034135+0000", 2023-11-25T04:23:20.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-07 2023-11-25T04:23:20.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:20.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034138+0000", 2023-11-25T04:23:20.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6009999999999999e-06 2023-11-25T04:23:20.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:20.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034145+0000", 2023-11-25T04:23:20.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1169999999999998e-06 2023-11-25T04:23:20.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:20.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034150+0000", 2023-11-25T04:23:20.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9080000000000003e-06 2023-11-25T04:23:20.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:20.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034209+0000", 2023-11-25T04:23:20.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8149000000000002e-05 2023-11-25T04:23:20.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:20.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034415+0000", 2023-11-25T04:23:20.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020637699999999999 2023-11-25T04:23:20.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:20.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034584+0000", 2023-11-25T04:23:20.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016870399999999999 2023-11-25T04:23:20.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:20.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.036979+0000", 2023-11-25T04:23:20.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023957509999999998 2023-11-25T04:23:20.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:20.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044161+0000", 2023-11-25T04:23:20.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0071811590000000003 2023-11-25T04:23:20.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:20.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044224+0000", 2023-11-25T04:23:20.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3454000000000006e-05 2023-11-25T04:23:20.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:20.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044268+0000", 2023-11-25T04:23:20.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3761999999999999e-05 2023-11-25T04:23:20.424 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:20.424 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:20.424 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:20.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.425 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:23 3.e 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:20.425 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:20.425 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 33.324413882000002, 2023-11-25T04:23:20.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010281676, 2023-11-25T04:23:20.425 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:20.425 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:20.425 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:20.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:20.425 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:20.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:20.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:20.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:20.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:20.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:20.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:20.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5799999999999999e-07 2023-11-25T04:23:20.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:20.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034169+0000", 2023-11-25T04:23:20.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1879999999999998e-06 2023-11-25T04:23:20.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:20.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034177+0000", 2023-11-25T04:23:20.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7020000000000001e-06 2023-11-25T04:23:20.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:20.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034184+0000", 2023-11-25T04:23:20.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3910000000000003e-06 2023-11-25T04:23:20.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:20.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034924+0000", 2023-11-25T04:23:20.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00074032999999999998 2023-11-25T04:23:20.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:20.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034991+0000", 2023-11-25T04:23:20.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7347000000000004e-05 2023-11-25T04:23:20.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.430 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:20.430 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035096+0000", 2023-11-25T04:23:20.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010488 2023-11-25T04:23:20.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:20.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037588+0000", 2023-11-25T04:23:20.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024918309999999999 2023-11-25T04:23:20.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:20.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044367+0000", 2023-11-25T04:23:20.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067787589999999997 2023-11-25T04:23:20.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:20.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044415+0000", 2023-11-25T04:23:20.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7803000000000003e-05 2023-11-25T04:23:20.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:20.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044447+0000", 2023-11-25T04:23:20.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2886999999999999e-05 2023-11-25T04:23:20.432 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:20.432 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:20.433 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:20.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.433 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:30 3.15 3:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:20.433 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:20.433 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 33.324355441999998, 2023-11-25T04:23:20.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010365070000000001, 2023-11-25T04:23:20.433 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:20.433 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:20.433 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:20.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.434 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:20.434 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:20.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:20.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.434 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:20.434 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:20.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:20.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.434 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:20.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034225+0000", 2023-11-25T04:23:20.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7400000000000002e-07 2023-11-25T04:23:20.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:20.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034229+0000", 2023-11-25T04:23:20.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3100000000000002e-06 2023-11-25T04:23:20.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:20.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034239+0000", 2023-11-25T04:23:20.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9140000000000003e-06 2023-11-25T04:23:20.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:20.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034246+0000", 2023-11-25T04:23:20.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0149999999999998e-06 2023-11-25T04:23:20.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:20.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035572+0000", 2023-11-25T04:23:20.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001325655 2023-11-25T04:23:20.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:20.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035730+0000", 2023-11-25T04:23:20.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015827899999999999 2023-11-25T04:23:20.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:20.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035857+0000", 2023-11-25T04:23:20.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012644699999999999 2023-11-25T04:23:20.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:20.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038737+0000", 2023-11-25T04:23:20.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002880077 2023-11-25T04:23:20.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.438 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:20.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044540+0000", 2023-11-25T04:23:20.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058034239999999997 2023-11-25T04:23:20.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.438 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:20.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044569+0000", 2023-11-25T04:23:20.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8388999999999999e-05 2023-11-25T04:23:20.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.439 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:20.439 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044589+0000", 2023-11-25T04:23:20.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0686000000000002e-05 2023-11-25T04:23:20.439 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:20.439 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:20.439 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:20.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.439 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:33 3.1b 3:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:20.439 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:20.440 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 33.324309943999999, 2023-11-25T04:23:20.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099754049999999997, 2023-11-25T04:23:20.440 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:20.440 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:23:20.440 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:20.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:20.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:20.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:20.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:20.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:20.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:20.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:20.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:20.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4799999999999998e-07 2023-11-25T04:23:20.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:20.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034273+0000", 2023-11-25T04:23:20.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7819999999999998e-06 2023-11-25T04:23:20.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:20.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034285+0000", 2023-11-25T04:23:20.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2058e-05 2023-11-25T04:23:20.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:20.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034291+0000", 2023-11-25T04:23:20.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0680000000000002e-06 2023-11-25T04:23:20.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:20.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034370+0000", 2023-11-25T04:23:20.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8452e-05 2023-11-25T04:23:20.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:20.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034477+0000", 2023-11-25T04:23:20.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107471 2023-11-25T04:23:20.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:20.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044245+0000", 2023-11-25T04:23:20.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0097679259999999997 2023-11-25T04:23:20.444 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:20.444 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:20.444 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:20.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.444 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:36 3.1a 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:20.444 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:20.444 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 33.324263271, 2023-11-25T04:23:20.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058179099999999999, 2023-11-25T04:23:20.444 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:20.445 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:23:20.445 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:20.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.445 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:20.445 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:20.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:20.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.445 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:20.445 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:20.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:20.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:20.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034317+0000", 2023-11-25T04:23:20.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1999999999999999e-07 2023-11-25T04:23:20.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:20.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034321+0000", 2023-11-25T04:23:20.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.233e-06 2023-11-25T04:23:20.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.447 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:20.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034337+0000", 2023-11-25T04:23:20.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5308999999999999e-05 2023-11-25T04:23:20.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.447 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:20.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034354+0000", 2023-11-25T04:23:20.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7411999999999999e-05 2023-11-25T04:23:20.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:20.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034465+0000", 2023-11-25T04:23:20.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011154699999999999 2023-11-25T04:23:20.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:20.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034560+0000", 2023-11-25T04:23:20.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4186999999999998e-05 2023-11-25T04:23:20.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:20.449 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.040134+0000", 2023-11-25T04:23:20.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055746019999999997 2023-11-25T04:23:20.449 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:20.449 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:20.449 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:20.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.449 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:9 3.10 e18/16 3:09401a81:::601.00000000:head v 18'4, mlcod=0'0)", 2023-11-25T04:23:20.449 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:20.449 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 33.315368962999997, 2023-11-25T04:23:20.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061641020000000003, 2023-11-25T04:23:20.450 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:20.450 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:20.450 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:20.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.450 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:20.450 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:20.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:20.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.451 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:20.451 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:20.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:20.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.451 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:20.451 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:20.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5300000000000001e-07 2023-11-25T04:23:20.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.451 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:20.452 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043222+0000", 2023-11-25T04:23:20.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.082e-05 2023-11-25T04:23:20.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:20.452 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043228+0000", 2023-11-25T04:23:20.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6699999999999997e-06 2023-11-25T04:23:20.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:20.452 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043236+0000", 2023-11-25T04:23:20.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8189999999999992e-06 2023-11-25T04:23:20.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.453 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:20.453 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043507+0000", 2023-11-25T04:23:20.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000271012 2023-11-25T04:23:20.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.453 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:20.453 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043530+0000", 2023-11-25T04:23:20.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2659000000000001e-05 2023-11-25T04:23:20.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.454 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:20.454 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049321+0000", 2023-11-25T04:23:20.454 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057915129999999999 2023-11-25T04:23:20.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.454 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:20.454 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049375+0000", 2023-11-25T04:23:20.454 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3455999999999998e-05 2023-11-25T04:23:20.454 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:20.455 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:20.455 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:20.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.455 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:1 3.14 e18/16 3:292cf221:::200.00000000:head v 18'3, mlcod=0'0)", 2023-11-25T04:23:20.456 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:20.456 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 33.315332249999997, 2023-11-25T04:23:20.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0060983599999999997, 2023-11-25T04:23:20.456 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:20.456 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:20.456 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:20.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:20.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:20.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:20.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:20.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:20.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:20.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:20.458 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043248+0000", 2023-11-25T04:23:20.458 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9399999999999995e-07 2023-11-25T04:23:20.458 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.458 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.458 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:20.458 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043252+0000", 2023-11-25T04:23:20.459 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1799999999999998e-06 2023-11-25T04:23:20.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.459 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:20.459 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043259+0000", 2023-11-25T04:23:20.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0049999999999998e-06 2023-11-25T04:23:20.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:20.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043265+0000", 2023-11-25T04:23:20.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0859999999999998e-06 2023-11-25T04:23:20.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:20.461 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043311+0000", 2023-11-25T04:23:20.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6082999999999999e-05 2023-11-25T04:23:20.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.462 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:20.462 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043330+0000", 2023-11-25T04:23:20.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8556e-05 2023-11-25T04:23:20.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.463 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.463 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:20.463 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049303+0000", 2023-11-25T04:23:20.463 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059733470000000004 2023-11-25T04:23:20.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.464 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:20.464 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049346+0000", 2023-11-25T04:23:20.464 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2608999999999998e-05 2023-11-25T04:23:20.464 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:20.465 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:20.465 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:20.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.465 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:3 3.14 e18/16 3:287034c4:::1.00000000.inode:head v 18'4, mlcod=0'0)", 2023-11-25T04:23:20.465 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:20.465 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 33.314815744000001, 2023-11-25T04:23:20.465 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057040090000000003, 2023-11-25T04:23:20.465 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:20.465 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:20.465 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:20.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:20.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:20.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:20.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:20.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:20.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:20.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.467 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:20.467 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:20.467 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9799999999999999e-07 2023-11-25T04:23:20.467 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.467 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.467 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:20.467 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043785+0000", 2023-11-25T04:23:20.467 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0721999999999999e-05 2023-11-25T04:23:20.467 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.467 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.468 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:20.468 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043793+0000", 2023-11-25T04:23:20.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5839999999999995e-06 2023-11-25T04:23:20.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.468 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:20.468 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043803+0000", 2023-11-25T04:23:20.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0729999999999994e-06 2023-11-25T04:23:20.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.468 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:20.469 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043843+0000", 2023-11-25T04:23:20.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0747999999999997e-05 2023-11-25T04:23:20.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.469 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.469 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:20.469 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043856+0000", 2023-11-25T04:23:20.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2697000000000001e-05 2023-11-25T04:23:20.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.469 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.469 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:20.469 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049426+0000", 2023-11-25T04:23:20.470 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055704409999999998 2023-11-25T04:23:20.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.470 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:20.470 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049468+0000", 2023-11-25T04:23:20.470 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1446000000000003e-05 2023-11-25T04:23:20.470 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:20.470 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:20.470 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:20.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.471 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "pg_update_log_missing(3.14 epoch 18/16 rep_tid 171 entries 18'5 (0'0) error 3:2e2fa760:::200.00000001:head by mds.0.3:32 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:23:20.471 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:20.471 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 33.314654781000002, 2023-11-25T04:23:20.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005556938, 2023-11-25T04:23:20.471 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:20.471 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2023-11-25T04:23:20.471 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:20.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.471 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:20.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:20.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:20.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.472 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:20.472 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:20.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:20.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.472 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:20.472 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:20.473 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.03e-07 2023-11-25T04:23:20.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.473 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:20.473 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043931+0000", 2023-11-25T04:23:20.473 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1079999999999998e-06 2023-11-25T04:23:20.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.473 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:20.473 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043944+0000", 2023-11-25T04:23:20.473 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2852e-05 2023-11-25T04:23:20.474 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.474 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.474 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:20.474 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043955+0000", 2023-11-25T04:23:20.474 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1037e-05 2023-11-25T04:23:20.474 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.474 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.474 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:20.474 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044088+0000", 2023-11-25T04:23:20.474 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013269599999999999 2023-11-25T04:23:20.474 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.474 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.475 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:20.475 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049482+0000", 2023-11-25T04:23:20.475 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053940419999999999 2023-11-25T04:23:20.475 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:20.475 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:20.475 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:20.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.475 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.475 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:10 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'5, mlcod=0'0)", 2023-11-25T04:23:20.475 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:20.475 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 33.314564752999999, 2023-11-25T04:23:20.476 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054493099999999997, 2023-11-25T04:23:20.476 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:20.476 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:20.476 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:20.476 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:20.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:20.476 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:20.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.476 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:20.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:20.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:20.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.477 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:20.477 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:20.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6100000000000002e-07 2023-11-25T04:23:20.478 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.478 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.478 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:20.478 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044019+0000", 2023-11-25T04:23:20.478 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.839e-06 2023-11-25T04:23:20.478 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.478 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.478 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:20.478 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044026+0000", 2023-11-25T04:23:20.478 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5869999999999998e-06 2023-11-25T04:23:20.478 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.479 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.479 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:20.479 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044031+0000", 2023-11-25T04:23:20.479 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4909999999999996e-06 2023-11-25T04:23:20.479 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.479 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.479 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:20.479 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044070+0000", 2023-11-25T04:23:20.479 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8829999999999999e-05 2023-11-25T04:23:20.479 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.480 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.480 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:20.480 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044081+0000", 2023-11-25T04:23:20.480 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1447999999999999e-05 2023-11-25T04:23:20.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.480 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.480 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:20.480 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049406+0000", 2023-11-25T04:23:20.480 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053246480000000004 2023-11-25T04:23:20.481 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.481 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:20.481 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049464+0000", 2023-11-25T04:23:20.481 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8205999999999998e-05 2023-11-25T04:23:20.481 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:20.481 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:20.481 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:20.481 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.481 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:11 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'6, mlcod=0'0)", 2023-11-25T04:23:20.482 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:20.482 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 33.314363489000002, 2023-11-25T04:23:20.482 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052673499999999996, 2023-11-25T04:23:20.482 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:20.482 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:20.482 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:20.482 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.482 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:20.482 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:20.483 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:20.483 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.483 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.483 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:20.483 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:20.483 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:20.483 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.483 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.483 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:20.483 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:20.483 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0199999999999998e-07 2023-11-25T04:23:20.484 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.484 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.484 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:20.484 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044220+0000", 2023-11-25T04:23:20.484 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7720000000000001e-06 2023-11-25T04:23:20.485 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.485 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.485 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:20.485 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044226+0000", 2023-11-25T04:23:20.485 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8579999999999996e-06 2023-11-25T04:23:20.485 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.486 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.486 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:20.486 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044232+0000", 2023-11-25T04:23:20.486 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1700000000000002e-06 2023-11-25T04:23:20.486 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.486 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.486 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:20.486 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044274+0000", 2023-11-25T04:23:20.486 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1254999999999997e-05 2023-11-25T04:23:20.487 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.487 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.487 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:20.487 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:23:20.487 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6880999999999999e-05 2023-11-25T04:23:20.487 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.487 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.487 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:20.487 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049466+0000", 2023-11-25T04:23:20.487 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051659219999999999 2023-11-25T04:23:20.487 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:20.487 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:20.488 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:20.488 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049483+0000", 2023-11-25T04:23:20.488 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.719e-05 2023-11-25T04:23:20.488 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:20.488 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:20.488 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:20.488 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:20.488 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:20.488 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:20.489 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:20.488+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:20.490 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:23:20.591 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:20.589+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:20.599 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:20.599 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:23:20.599 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2023-11-25T04:23:20.599 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:20.608 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:23:20.691 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~12s 2023-11-25T04:23:20.692 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:20.691+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:20.752 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:20.752 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:23:20.753 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2023-11-25T04:23:20.753 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2023-11-25T04:23:20.753 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:20.761 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:23:20.805 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:20.804+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:20.868 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:20.868 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2023-11-25T04:23:20.868 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2023-11-25T04:23:20.868 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2023-11-25T04:23:20.868 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.868 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:20.868 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:20.868 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 33.833923689999999, 2023-11-25T04:23:20.868 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01072943, 2023-11-25T04:23:20.869 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:20.869 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:20.869 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:20.869 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.869 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:20.869 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:20.869 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:20.869 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.869 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.869 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:20.869 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:20.869 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:20.870 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.870 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.870 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:20.870 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031829+0000", 2023-11-25T04:23:20.870 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.666e-06 2023-11-25T04:23:20.870 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.870 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.870 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:20.870 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031839+0000", 2023-11-25T04:23:20.870 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9490000000000008e-06 2023-11-25T04:23:20.870 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.870 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.870 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:20.870 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031857+0000", 2023-11-25T04:23:20.871 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8093000000000001e-05 2023-11-25T04:23:20.871 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.871 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.871 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:20.871 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031882+0000", 2023-11-25T04:23:20.871 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4294000000000002e-05 2023-11-25T04:23:20.871 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.871 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.871 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:20.871 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033130+0000", 2023-11-25T04:23:20.871 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001248264 2023-11-25T04:23:20.871 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.871 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.872 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:20.872 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033426+0000", 2023-11-25T04:23:20.872 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029620999999999998 2023-11-25T04:23:20.872 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.872 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.872 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:20.872 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033814+0000", 2023-11-25T04:23:20.872 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000387742 2023-11-25T04:23:20.872 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.872 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.872 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:20.872 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036793+0000", 2023-11-25T04:23:20.873 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002979598 2023-11-25T04:23:20.873 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.873 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.873 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:20.873 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042455+0000", 2023-11-25T04:23:20.873 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0056619260000000003 2023-11-25T04:23:20.873 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.873 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.873 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:20.873 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042502+0000", 2023-11-25T04:23:20.873 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7139999999999999e-05 2023-11-25T04:23:20.873 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.873 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.873 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:20.874 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042557+0000", 2023-11-25T04:23:20.874 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4548e-05 2023-11-25T04:23:20.874 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:20.874 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:20.874 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:20.874 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.874 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.874 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:20.874 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:20.874 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 33.832725154999999, 2023-11-25T04:23:20.874 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011505202000000001, 2023-11-25T04:23:20.874 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:20.874 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:20.875 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:20.875 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.875 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:20.875 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:20.875 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:20.875 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.875 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.875 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:20.875 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:20.875 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:20.875 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.875 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.875 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:20.875 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033027+0000", 2023-11-25T04:23:20.876 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0699999999999999e-06 2023-11-25T04:23:20.876 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.876 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.876 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:20.876 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033035+0000", 2023-11-25T04:23:20.876 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0830000000000004e-06 2023-11-25T04:23:20.876 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.876 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.876 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:20.876 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033059+0000", 2023-11-25T04:23:20.876 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3975e-05 2023-11-25T04:23:20.876 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.877 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.877 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:20.877 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033482+0000", 2023-11-25T04:23:20.877 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042253700000000001 2023-11-25T04:23:20.877 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.877 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.877 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:20.877 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034520+0000", 2023-11-25T04:23:20.877 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001037991 2023-11-25T04:23:20.877 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.877 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.877 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:20.877 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034611+0000", 2023-11-25T04:23:20.877 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1113999999999995e-05 2023-11-25T04:23:20.878 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.878 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.878 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:20.878 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034769+0000", 2023-11-25T04:23:20.878 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000157641 2023-11-25T04:23:20.878 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.878 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.878 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:20.878 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038292+0000", 2023-11-25T04:23:20.878 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035231160000000002 2023-11-25T04:23:20.878 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.878 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.878 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:20.878 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044440+0000", 2023-11-25T04:23:20.879 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0061486980000000002 2023-11-25T04:23:20.879 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.879 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.879 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:20.879 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044495+0000", 2023-11-25T04:23:20.879 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4339000000000003e-05 2023-11-25T04:23:20.879 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.879 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.879 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:20.879 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044531+0000", 2023-11-25T04:23:20.879 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6637999999999997e-05 2023-11-25T04:23:20.879 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:20.879 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:20.880 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:20.880 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.880 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.880 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:20.880 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:20.880 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 33.832250801000001, 2023-11-25T04:23:20.880 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0057634039999999997, 2023-11-25T04:23:20.880 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:20.880 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:20.880 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:20.880 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.880 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:20.881 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:20.881 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:20.881 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.881 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.881 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:20.881 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:20.881 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:20.881 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.881 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.881 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:20.881 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:20.882 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7900000000000007e-07 2023-11-25T04:23:20.882 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.882 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.882 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:20.882 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033507+0000", 2023-11-25T04:23:20.882 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8900000000000004e-06 2023-11-25T04:23:20.882 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.882 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.882 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:20.882 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033519+0000", 2023-11-25T04:23:20.882 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1756000000000001e-05 2023-11-25T04:23:20.882 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.882 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.883 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:20.883 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033527+0000", 2023-11-25T04:23:20.883 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3610000000000001e-06 2023-11-25T04:23:20.883 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.883 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.883 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:20.883 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034451+0000", 2023-11-25T04:23:20.883 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00092372400000000001 2023-11-25T04:23:20.883 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.883 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.883 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:20.883 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034809+0000", 2023-11-25T04:23:20.883 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00035757699999999998 2023-11-25T04:23:20.884 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.884 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.884 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:23:20.884 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034965+0000", 2023-11-25T04:23:20.884 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015638699999999999 2023-11-25T04:23:20.884 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.884 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.884 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:20.884 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038299+0000", 2023-11-25T04:23:20.884 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0033336559999999999 2023-11-25T04:23:20.884 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.884 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.884 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:20.884 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039169+0000", 2023-11-25T04:23:20.885 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00087066400000000005 2023-11-25T04:23:20.885 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.885 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.885 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:20.885 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039226+0000", 2023-11-25T04:23:20.885 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6511999999999997e-05 2023-11-25T04:23:20.885 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.885 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.885 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:20.885 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039264+0000", 2023-11-25T04:23:20.885 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7898000000000002e-05 2023-11-25T04:23:20.885 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:20.885 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:20.886 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:20.886 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.886 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.886 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:25 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:20.886 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:20.886 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 33.832172217, 2023-11-25T04:23:20.886 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096656169999999996, 2023-11-25T04:23:20.886 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:20.886 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:20.886 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:20.886 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.886 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:20.886 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:20.887 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:20.887 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.887 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.887 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:20.887 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:20.887 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:20.887 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.887 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.887 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:20.887 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033580+0000", 2023-11-25T04:23:20.887 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8199999999999999e-07 2023-11-25T04:23:20.887 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.888 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.888 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:20.888 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033585+0000", 2023-11-25T04:23:20.888 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3680000000000001e-06 2023-11-25T04:23:20.888 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.888 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.888 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:20.888 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033600+0000", 2023-11-25T04:23:20.888 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4331000000000001e-05 2023-11-25T04:23:20.888 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.888 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.888 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:20.888 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033606+0000", 2023-11-25T04:23:20.888 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5780000000000004e-06 2023-11-25T04:23:20.889 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.889 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.889 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:20.889 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035747+0000", 2023-11-25T04:23:20.889 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0021406530000000002 2023-11-25T04:23:20.889 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.889 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.889 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:20.889 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035901+0000", 2023-11-25T04:23:20.889 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000153931 2023-11-25T04:23:20.889 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.889 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.890 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:23:20.890 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036193+0000", 2023-11-25T04:23:20.890 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029211299999999999 2023-11-25T04:23:20.890 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.890 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.890 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:20.890 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038655+0000", 2023-11-25T04:23:20.890 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0024621640000000002 2023-11-25T04:23:20.890 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.890 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.890 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:20.890 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043130+0000", 2023-11-25T04:23:20.890 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.004474698 2023-11-25T04:23:20.891 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.891 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.891 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:20.891 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043204+0000", 2023-11-25T04:23:20.891 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4091999999999995e-05 2023-11-25T04:23:20.891 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.891 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.891 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:20.891 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043245+0000", 2023-11-25T04:23:20.891 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0907e-05 2023-11-25T04:23:20.891 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:20.891 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:20.891 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:20.891 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.892 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.892 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:26 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:20.892 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:20.892 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 33.832136392999999, 2023-11-25T04:23:20.892 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011086945000000001, 2023-11-25T04:23:20.892 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:20.892 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:20.892 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:20.892 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.892 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:20.892 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:20.892 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:20.892 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.893 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.893 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:20.893 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:20.893 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:20.893 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.893 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.893 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:20.893 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:20.893 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1699999999999998e-07 2023-11-25T04:23:20.894 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.894 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.894 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:20.894 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033627+0000", 2023-11-25T04:23:20.894 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.163e-05 2023-11-25T04:23:20.894 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.895 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.895 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:20.895 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033636+0000", 2023-11-25T04:23:20.895 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6859999999999995e-06 2023-11-25T04:23:20.895 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.895 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.895 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:20.895 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033642+0000", 2023-11-25T04:23:20.895 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3149999999999997e-06 2023-11-25T04:23:20.895 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.895 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.895 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:20.896 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035414+0000", 2023-11-25T04:23:20.896 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0017716380000000001 2023-11-25T04:23:20.896 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.896 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.896 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:20.896 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035539+0000", 2023-11-25T04:23:20.896 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000125477 2023-11-25T04:23:20.896 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.896 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.896 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:20.896 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035672+0000", 2023-11-25T04:23:20.896 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000133316 2023-11-25T04:23:20.896 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.897 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.897 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:20.897 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038649+0000", 2023-11-25T04:23:20.897 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029764679999999999 2023-11-25T04:23:20.897 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.897 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.897 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:20.897 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044578+0000", 2023-11-25T04:23:20.897 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0059293510000000002 2023-11-25T04:23:20.897 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.897 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.897 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:20.897 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044651+0000", 2023-11-25T04:23:20.898 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2513000000000002e-05 2023-11-25T04:23:20.898 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.898 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.898 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:20.898 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044702+0000", 2023-11-25T04:23:20.898 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1034e-05 2023-11-25T04:23:20.898 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:20.898 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:20.898 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:20.898 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.898 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.898 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:29 3.b 3:d0630e4c:::mds0_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:20.899 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:20.899 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 33.832098473000002, 2023-11-25T04:23:20.899 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010980227, 2023-11-25T04:23:20.899 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:20.899 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:20.899 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:20.899 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.899 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:20.899 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:20.899 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:20.899 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.899 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.899 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:20.900 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:20.900 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:20.900 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.900 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.900 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:20.900 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:20.900 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3499999999999996e-07 2023-11-25T04:23:20.900 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.900 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.900 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:20.900 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033658+0000", 2023-11-25T04:23:20.900 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1280000000000001e-06 2023-11-25T04:23:20.900 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.901 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.901 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:20.901 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033667+0000", 2023-11-25T04:23:20.901 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1430000000000004e-06 2023-11-25T04:23:20.901 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.901 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.901 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:20.901 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033678+0000", 2023-11-25T04:23:20.901 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0906e-05 2023-11-25T04:23:20.901 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.901 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.901 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:20.901 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034456+0000", 2023-11-25T04:23:20.902 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00077833899999999996 2023-11-25T04:23:20.902 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.902 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.902 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:20.902 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034575+0000", 2023-11-25T04:23:20.902 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011926000000000001 2023-11-25T04:23:20.902 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.902 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.902 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:20.902 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035001+0000", 2023-11-25T04:23:20.902 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042550999999999998 2023-11-25T04:23:20.902 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.903 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.903 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:20.903 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.037923+0000", 2023-11-25T04:23:20.903 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029219670000000001 2023-11-25T04:23:20.903 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.903 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.903 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:20.903 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044538+0000", 2023-11-25T04:23:20.903 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0066152349999999997 2023-11-25T04:23:20.903 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.903 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.903 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:20.903 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044594+0000", 2023-11-25T04:23:20.904 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5593000000000001e-05 2023-11-25T04:23:20.904 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.904 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.904 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:20.904 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044633+0000", 2023-11-25T04:23:20.904 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9511000000000002e-05 2023-11-25T04:23:20.904 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:20.904 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:20.904 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:20.904 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.904 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.904 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:34 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:20.904 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:20.905 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 33.831848360000002, 2023-11-25T04:23:20.905 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010920445000000001, 2023-11-25T04:23:20.905 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:20.905 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:20.905 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:20.905 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.905 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:20.905 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:20.905 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:20.905 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.905 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.905 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:20.905 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:20.906 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:20.906 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.906 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.906 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:20.906 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033904+0000", 2023-11-25T04:23:20.906 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6599999999999994e-07 2023-11-25T04:23:20.906 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.906 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.906 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:20.906 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033909+0000", 2023-11-25T04:23:20.906 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5639999999999996e-06 2023-11-25T04:23:20.906 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.907 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.907 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:20.907 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033923+0000", 2023-11-25T04:23:20.907 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3028e-05 2023-11-25T04:23:20.907 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.907 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.907 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:20.907 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034043+0000", 2023-11-25T04:23:20.907 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000120328 2023-11-25T04:23:20.907 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.907 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.907 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:20.907 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035576+0000", 2023-11-25T04:23:20.908 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015333720000000001 2023-11-25T04:23:20.908 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.908 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.908 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:20.908 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035699+0000", 2023-11-25T04:23:20.908 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000123001 2023-11-25T04:23:20.908 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.908 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.908 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:20.908 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044823+0000", 2023-11-25T04:23:20.908 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0091241859999999994 2023-11-25T04:23:20.908 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:20.908 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:20.909 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:20.909 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.909 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.909 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:35 3.1f 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:20.909 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:20.909 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 33.830544154999998, 2023-11-25T04:23:20.909 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096164779999999995, 2023-11-25T04:23:20.909 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:20.909 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:20.909 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:20.909 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.909 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:20.909 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:20.909 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:20.910 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.910 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.910 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:20.910 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:20.910 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:20.910 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.910 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.910 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:20.910 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035208+0000", 2023-11-25T04:23:20.910 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2699999999999998e-07 2023-11-25T04:23:20.910 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.910 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.911 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:20.911 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035213+0000", 2023-11-25T04:23:20.911 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.6070000000000001e-06 2023-11-25T04:23:20.911 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.911 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.911 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:20.911 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035350+0000", 2023-11-25T04:23:20.911 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013693599999999999 2023-11-25T04:23:20.911 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.911 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.911 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:20.911 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035358+0000", 2023-11-25T04:23:20.911 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0099999999999995e-06 2023-11-25T04:23:20.912 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.912 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.912 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:20.912 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036260+0000", 2023-11-25T04:23:20.912 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00090240100000000005 2023-11-25T04:23:20.912 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.912 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.912 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:20.912 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036517+0000", 2023-11-25T04:23:20.912 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00025736299999999998 2023-11-25T04:23:20.912 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.912 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.912 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:20.913 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044824+0000", 2023-11-25T04:23:20.913 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0083063340000000003 2023-11-25T04:23:20.913 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:20.913 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:20.913 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:20.913 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.913 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.913 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:37 3.1e 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:20.913 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:20.913 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 33.830377468999998, 2023-11-25T04:23:20.913 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0078322640000000002, 2023-11-25T04:23:20.913 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:20.914 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:20.914 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:20.914 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.914 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:20.914 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:20.914 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:20.914 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.914 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.914 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:20.914 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:20.914 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:20.914 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.915 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.915 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:20.915 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035375+0000", 2023-11-25T04:23:20.915 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9100000000000003e-07 2023-11-25T04:23:20.915 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.915 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.915 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:20.915 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035379+0000", 2023-11-25T04:23:20.915 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9430000000000002e-06 2023-11-25T04:23:20.915 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.915 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.915 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:20.915 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035388+0000", 2023-11-25T04:23:20.916 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2760000000000006e-06 2023-11-25T04:23:20.916 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.916 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.916 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:20.916 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035396+0000", 2023-11-25T04:23:20.916 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8480000000000001e-06 2023-11-25T04:23:20.916 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.916 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.916 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:20.916 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035452+0000", 2023-11-25T04:23:20.916 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5831000000000001e-05 2023-11-25T04:23:20.916 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.917 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.917 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:20.917 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035934+0000", 2023-11-25T04:23:20.917 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00048265599999999998 2023-11-25T04:23:20.917 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.917 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.917 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:20.917 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043206+0000", 2023-11-25T04:23:20.917 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.007271919 2023-11-25T04:23:20.917 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:20.917 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:20.917 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:20.917 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.918 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.918 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing(3.1a epoch 18/16 rep_tid 3 entries 18'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:36 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:23:20.918 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:20.918 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 33.829101883, 2023-11-25T04:23:20.918 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002073414, 2023-11-25T04:23:20.918 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:20.918 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:20.918 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:20.918 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.918 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:20.918 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:20.918 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:20.919 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.919 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.919 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:20.919 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:20.919 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:20.919 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.919 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.919 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:20.919 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036650+0000", 2023-11-25T04:23:20.919 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7300000000000002e-07 2023-11-25T04:23:20.919 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.921 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.921 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:20.921 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036659+0000", 2023-11-25T04:23:20.921 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.7499999999999992e-06 2023-11-25T04:23:20.921 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.921 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.921 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:20.921 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036673+0000", 2023-11-25T04:23:20.922 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4917e-05 2023-11-25T04:23:20.922 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.922 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.922 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:20.922 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036687+0000", 2023-11-25T04:23:20.922 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3883000000000001e-05 2023-11-25T04:23:20.922 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.922 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.922 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:20.922 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036781+0000", 2023-11-25T04:23:20.922 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3219000000000003e-05 2023-11-25T04:23:20.922 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.923 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.923 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:20.923 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038723+0000", 2023-11-25T04:23:20.923 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019422720000000001 2023-11-25T04:23:20.923 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:20.923 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:20.923 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:20.923 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.923 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.923 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:28 3.13 e18/16 3:cd5a64a3:::100.00000000:head v 18'1, mlcod=0'0)", 2023-11-25T04:23:20.923 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:20.924 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 33.825440725, 2023-11-25T04:23:20.924 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001850546, 2023-11-25T04:23:20.924 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:20.924 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:20.924 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:20.924 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.924 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:20.924 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:20.924 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:20.924 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.924 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.924 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:20.924 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:20.925 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:20.925 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.925 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.925 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:20.925 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:20.925 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7800000000000002e-07 2023-11-25T04:23:20.925 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.925 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.925 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:20.925 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040331+0000", 2023-11-25T04:23:20.925 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0449999999999999e-05 2023-11-25T04:23:20.925 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.925 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.926 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:20.926 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040339+0000", 2023-11-25T04:23:20.926 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0709999999999995e-06 2023-11-25T04:23:20.926 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.926 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.926 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:20.926 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040347+0000", 2023-11-25T04:23:20.926 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7079999999999997e-06 2023-11-25T04:23:20.926 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.926 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.926 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:20.926 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040410+0000", 2023-11-25T04:23:20.927 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2453000000000007e-05 2023-11-25T04:23:20.927 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.927 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.927 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:20.927 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040422+0000", 2023-11-25T04:23:20.927 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2429000000000001e-05 2023-11-25T04:23:20.927 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.927 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.927 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:20.927 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042129+0000", 2023-11-25T04:23:20.927 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001707266 2023-11-25T04:23:20.927 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.927 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.928 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:20.928 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042161+0000", 2023-11-25T04:23:20.928 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1791000000000001e-05 2023-11-25T04:23:20.928 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:20.928 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:20.928 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:20.928 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.928 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.928 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:27 3.13 e18/16 3:c91d4a1d:::mds0_inotable:head v 18'2, mlcod=0'0)", 2023-11-25T04:23:20.928 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:20.928 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 33.825237201999997, 2023-11-25T04:23:20.928 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0016886430000000001, 2023-11-25T04:23:20.928 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:20.929 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:20.929 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:20.929 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.929 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:20.929 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:20.929 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:20.929 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.929 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.929 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:20.929 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:20.929 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:20.929 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.929 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.930 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:20.930 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:20.930 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3999999999999997e-07 2023-11-25T04:23:20.930 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.930 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.930 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:20.930 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040520+0000", 2023-11-25T04:23:20.930 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2989999999999996e-06 2023-11-25T04:23:20.930 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.930 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.930 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:20.931 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040527+0000", 2023-11-25T04:23:20.931 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5730000000000002e-06 2023-11-25T04:23:20.931 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.931 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.931 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:20.931 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040535+0000", 2023-11-25T04:23:20.931 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8630000000000008e-06 2023-11-25T04:23:20.931 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.931 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.931 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:20.931 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040683+0000", 2023-11-25T04:23:20.931 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000148002 2023-11-25T04:23:20.931 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.931 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.932 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:20.932 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040700+0000", 2023-11-25T04:23:20.932 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7065e-05 2023-11-25T04:23:20.932 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.932 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.932 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:20.932 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042163+0000", 2023-11-25T04:23:20.932 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001462767 2023-11-25T04:23:20.932 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.932 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.932 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:20.932 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042203+0000", 2023-11-25T04:23:20.932 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9734000000000003e-05 2023-11-25T04:23:20.933 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:20.933 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:20.933 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:20.933 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.933 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.933 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:20 3.1c e18/16 3:3ed09add:::607.00000000:head v 18'1, mlcod=0'0)", 2023-11-25T04:23:20.933 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:20.933 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 33.825202161, 2023-11-25T04:23:20.933 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015271, 2023-11-25T04:23:20.933 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:20.933 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:20.933 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:20.934 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.934 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:20.934 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:20.934 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:20.934 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.934 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.934 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:20.934 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:20.934 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:20.934 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.934 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.934 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:20.935 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:20.935 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4799999999999999e-07 2023-11-25T04:23:20.935 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.935 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.935 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:20.935 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040561+0000", 2023-11-25T04:23:20.935 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1800000000000001e-05 2023-11-25T04:23:20.935 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.935 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.935 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:20.935 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040568+0000", 2023-11-25T04:23:20.935 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7730000000000002e-06 2023-11-25T04:23:20.935 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.936 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.936 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:20.936 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040587+0000", 2023-11-25T04:23:20.936 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8691000000000001e-05 2023-11-25T04:23:20.936 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.936 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.936 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:20.936 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040638+0000", 2023-11-25T04:23:20.936 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1068e-05 2023-11-25T04:23:20.936 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.936 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.936 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:20.937 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040670+0000", 2023-11-25T04:23:20.937 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2125999999999997e-05 2023-11-25T04:23:20.937 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.937 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.937 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:20.937 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042032+0000", 2023-11-25T04:23:20.937 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001361821 2023-11-25T04:23:20.937 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.937 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.937 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:20.937 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042076+0000", 2023-11-25T04:23:20.937 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4472999999999999e-05 2023-11-25T04:23:20.937 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:20.938 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:20.938 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:20.938 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.938 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.938 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:21 3.1c e18/16 3:3ed09add:::607.00000000:head v 18'2, mlcod=0'0)", 2023-11-25T04:23:20.938 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:20.938 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 33.824967436000001, 2023-11-25T04:23:20.938 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0013228160000000001, 2023-11-25T04:23:20.938 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:20.938 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:20.938 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:20.938 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.939 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:20.939 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:20.939 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:20.939 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.939 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.939 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:20.939 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:20.939 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:20.939 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.939 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.939 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:20.939 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:20.939 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9299999999999999e-07 2023-11-25T04:23:20.940 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.940 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.940 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:20.940 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040790+0000", 2023-11-25T04:23:20.940 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5770000000000001e-06 2023-11-25T04:23:20.940 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.940 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.940 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:20.940 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040799+0000", 2023-11-25T04:23:20.940 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6439999999999995e-06 2023-11-25T04:23:20.940 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.940 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.940 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:20.941 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040806+0000", 2023-11-25T04:23:20.941 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0180000000000004e-06 2023-11-25T04:23:20.941 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.941 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.941 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:20.941 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040907+0000", 2023-11-25T04:23:20.941 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000100096 2023-11-25T04:23:20.941 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.941 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.941 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:20.941 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040925+0000", 2023-11-25T04:23:20.941 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8536999999999999e-05 2023-11-25T04:23:20.942 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.942 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.942 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:20.942 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042079+0000", 2023-11-25T04:23:20.942 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00115358 2023-11-25T04:23:20.942 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.942 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.942 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:20.942 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042107+0000", 2023-11-25T04:23:20.942 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7971e-05 2023-11-25T04:23:20.942 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:20.942 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:20.943 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:20.943 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.943 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.943 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop_reply(mds.0.3:4 3.1f e18/16 ondisk, result = 0)", 2023-11-25T04:23:20.943 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:20.943 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 33.823628046000003, 2023-11-25T04:23:20.943 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00044092900000000001, 2023-11-25T04:23:20.943 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:20.943 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:20.943 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:20.943 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.944 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:20.944 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:20.944 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:20.944 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.944 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.944 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:20.944 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:20.944 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:20.944 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.944 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.944 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:20.944 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042124+0000", 2023-11-25T04:23:20.945 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1800000000000005e-07 2023-11-25T04:23:20.945 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.945 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.945 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:20.945 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042128+0000", 2023-11-25T04:23:20.945 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2289999999999997e-06 2023-11-25T04:23:20.945 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.945 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.945 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:20.945 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042138+0000", 2023-11-25T04:23:20.945 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9729999999999992e-06 2023-11-25T04:23:20.945 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.946 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.946 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:20.946 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042145+0000", 2023-11-25T04:23:20.946 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.336e-06 2023-11-25T04:23:20.946 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.946 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.946 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:20.946 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042441+0000", 2023-11-25T04:23:20.946 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029650500000000002 2023-11-25T04:23:20.946 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.946 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.946 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:20.947 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042448+0000", 2023-11-25T04:23:20.947 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4100000000000002e-06 2023-11-25T04:23:20.947 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.947 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.947 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:20.947 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042564+0000", 2023-11-25T04:23:20.947 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000115758 2023-11-25T04:23:20.947 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:20.947 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:20.947 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:20.947 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.947 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.948 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing_reply(3.1f epoch 18/16 rep_tid 8 lcod 18'4)", 2023-11-25T04:23:20.948 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:20.948 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 33.821302633999998, 2023-11-25T04:23:20.948 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00038154699999999998, 2023-11-25T04:23:20.948 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:20.948 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:20.948 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:20.948 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.948 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:20.948 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:20.948 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:20.948 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.949 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.949 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:20.949 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:20.949 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:20.949 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.949 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.949 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:20.949 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:20.949 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2700000000000001e-07 2023-11-25T04:23:20.949 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.949 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.950 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:20.950 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044451+0000", 2023-11-25T04:23:20.950 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7460000000000001e-06 2023-11-25T04:23:20.950 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.950 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.950 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:20.950 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044455+0000", 2023-11-25T04:23:20.950 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5809999999999998e-06 2023-11-25T04:23:20.950 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.950 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.950 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:20.950 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044458+0000", 2023-11-25T04:23:20.951 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0170000000000001e-06 2023-11-25T04:23:20.951 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.951 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.951 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:20.951 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044734+0000", 2023-11-25T04:23:20.951 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00027582799999999999 2023-11-25T04:23:20.951 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.951 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.951 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:20.951 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044830+0000", 2023-11-25T04:23:20.951 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6248000000000004e-05 2023-11-25T04:23:20.951 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:20.951 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:20.952 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:20.952 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.952 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.952 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing_reply(3.b epoch 18/16 rep_tid 5 lcod 18'2)", 2023-11-25T04:23:20.952 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:20.952 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 33.821265973000003, 2023-11-25T04:23:20.952 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000344488, 2023-11-25T04:23:20.952 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:20.952 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:20.952 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:20.952 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.952 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:20.953 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:20.953 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:20.953 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.953 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.953 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:20.953 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:20.953 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:20.953 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.953 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.953 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:20.953 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:20.953 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.55e-07 2023-11-25T04:23:20.953 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.954 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.954 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:20.954 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044487+0000", 2023-11-25T04:23:20.954 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2470000000000001e-06 2023-11-25T04:23:20.954 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.954 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.954 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:20.954 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044491+0000", 2023-11-25T04:23:20.954 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9659999999999998e-06 2023-11-25T04:23:20.954 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.954 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.954 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:20.955 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044494+0000", 2023-11-25T04:23:20.955 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2310000000000001e-06 2023-11-25T04:23:20.955 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.955 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.955 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:20.955 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044756+0000", 2023-11-25T04:23:20.955 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00026172800000000003 2023-11-25T04:23:20.955 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.955 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.955 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:20.955 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044830+0000", 2023-11-25T04:23:20.956 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4160999999999997e-05 2023-11-25T04:23:20.956 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:20.956 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:20.956 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:20.956 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.956 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.956 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:39 3.6 3:654134d2:::mds0_openfiles.0:head [omap-clear,omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2023-11-25T04:23:20.956 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:20.956 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 28.761439930999998, 2023-11-25T04:23:20.956 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001788658, 2023-11-25T04:23:20.956 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:20.956 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:20.956 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:20.957 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.957 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:20.957 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:20.957 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:20.957 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.957 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.957 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:20.957 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:20.957 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:20.957 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.957 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.957 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:20.957 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104314+0000", 2023-11-25T04:23:20.958 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3099999999999999e-06 2023-11-25T04:23:20.958 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.958 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.958 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:20.958 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104325+0000", 2023-11-25T04:23:20.958 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1511999999999999e-05 2023-11-25T04:23:20.958 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.958 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.958 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:20.958 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104362+0000", 2023-11-25T04:23:20.958 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6903000000000002e-05 2023-11-25T04:23:20.958 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.959 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.959 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:20.959 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104387+0000", 2023-11-25T04:23:20.959 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4766e-05 2023-11-25T04:23:20.959 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.959 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.959 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:20.959 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104465+0000", 2023-11-25T04:23:20.959 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8119000000000002e-05 2023-11-25T04:23:20.959 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.959 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.959 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:20.960 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104557+0000", 2023-11-25T04:23:20.960 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2496000000000003e-05 2023-11-25T04:23:20.960 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.960 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.960 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:20.960 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104646+0000", 2023-11-25T04:23:20.960 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8818000000000002e-05 2023-11-25T04:23:20.960 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.960 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.960 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:20.960 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105201+0000", 2023-11-25T04:23:20.961 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00055506000000000002 2023-11-25T04:23:20.961 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.961 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.961 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:20.961 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105994+0000", 2023-11-25T04:23:20.961 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00079294700000000001 2023-11-25T04:23:20.961 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.961 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.961 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:20.961 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.106055+0000", 2023-11-25T04:23:20.961 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0566000000000003e-05 2023-11-25T04:23:20.961 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.962 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.962 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:20.962 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.106100+0000", 2023-11-25T04:23:20.962 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5160999999999997e-05 2023-11-25T04:23:20.962 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:20.962 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:20.962 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:20.962 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.962 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.962 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:43 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2023-11-25T04:23:20.962 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:20.963 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 13.761214935, 2023-11-25T04:23:20.963 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001751166, 2023-11-25T04:23:20.963 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:20.963 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:20.963 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:20.963 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.963 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:20.963 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:20.963 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:20.963 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.963 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.963 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:20.964 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:20.964 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:20.964 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.964 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.964 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:20.964 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104539+0000", 2023-11-25T04:23:20.964 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9529999999999999e-06 2023-11-25T04:23:20.964 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.964 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.964 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:20.964 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104557+0000", 2023-11-25T04:23:20.964 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7464000000000001e-05 2023-11-25T04:23:20.964 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.965 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.965 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:20.965 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104590+0000", 2023-11-25T04:23:20.965 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2966000000000003e-05 2023-11-25T04:23:20.965 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.965 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.965 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:20.965 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104608+0000", 2023-11-25T04:23:20.965 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8193e-05 2023-11-25T04:23:20.965 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.965 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.965 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:20.966 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104767+0000", 2023-11-25T04:23:20.966 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015882 2023-11-25T04:23:20.966 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.966 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.966 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:20.966 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104909+0000", 2023-11-25T04:23:20.966 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014244999999999999 2023-11-25T04:23:20.966 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.966 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.966 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:20.966 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.105059+0000", 2023-11-25T04:23:20.966 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014964399999999999 2023-11-25T04:23:20.967 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.967 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.967 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:20.967 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.105701+0000", 2023-11-25T04:23:20.967 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00064201099999999995 2023-11-25T04:23:20.967 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.967 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.967 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:20.967 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106233+0000", 2023-11-25T04:23:20.967 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00053178500000000001 2023-11-25T04:23:20.967 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.967 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.968 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:20.968 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106265+0000", 2023-11-25T04:23:20.968 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1986000000000002e-05 2023-11-25T04:23:20.968 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.968 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.968 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:20.968 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106288+0000", 2023-11-25T04:23:20.968 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2894000000000001e-05 2023-11-25T04:23:20.968 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:20.968 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:20.968 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:20.968 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.969 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.969 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:45 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e23)", 2023-11-25T04:23:20.969 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:23:20.969 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 8.7610168950000009, 2023-11-25T04:23:20.969 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00098423299999999994, 2023-11-25T04:23:20.969 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:20.969 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:20.969 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:20.969 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.969 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:20.969 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:23:20.969 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:20.969 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.970 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.970 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:20.970 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:23:20.970 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:20.970 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.970 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.970 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:20.970 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104737+0000", 2023-11-25T04:23:20.970 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3309999999999998e-06 2023-11-25T04:23:20.970 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.970 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.970 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:20.970 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104748+0000", 2023-11-25T04:23:20.971 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1080999999999999e-05 2023-11-25T04:23:20.971 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.971 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.971 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:20.971 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104771+0000", 2023-11-25T04:23:20.971 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2707000000000001e-05 2023-11-25T04:23:20.971 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.971 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.971 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:20.971 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104800+0000", 2023-11-25T04:23:20.971 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9595e-05 2023-11-25T04:23:20.971 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.972 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.972 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:20.972 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104890+0000", 2023-11-25T04:23:20.972 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.9808999999999993e-05 2023-11-25T04:23:20.972 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.972 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.972 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:20.972 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105004+0000", 2023-11-25T04:23:20.972 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000114508 2023-11-25T04:23:20.972 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.972 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.972 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:20.973 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105634+0000", 2023-11-25T04:23:20.973 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00062985799999999996 2023-11-25T04:23:20.973 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.973 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.973 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:20.973 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105671+0000", 2023-11-25T04:23:20.973 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6640000000000002e-05 2023-11-25T04:23:20.973 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:20.973 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:20.973 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:20.973 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105719+0000", 2023-11-25T04:23:20.973 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7704000000000003e-05 2023-11-25T04:23:20.973 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:20.974 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:20.974 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:20.974 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:20.974 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:20.974 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:20.975 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:20.974+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:20.976 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:23:21.076 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:21.075+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:21.083 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:21.084 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:23:21.084 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2023-11-25T04:23:21.084 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:21.094 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:23:21.177 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:21.176+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:21.199 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:21.199 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:23:21.199 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2023-11-25T04:23:21.199 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2023-11-25T04:23:21.199 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:21.207 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:23:21.292 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:21.291+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:21.295 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-11-25T04:23:21.355 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:21.355 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2023-11-25T04:23:21.357 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2023-11-25T04:23:21.358 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2023-11-25T04:23:21.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.358 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:155 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'109, mlcod=11'105)", 2023-11-25T04:23:21.358 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:21.358 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 43.567926597000003, 2023-11-25T04:23:21.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061719019999999999, 2023-11-25T04:23:21.358 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:21.359 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:21.359 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:21.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:21.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:21.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:21.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:21.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:21.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:21.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.360 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:21.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784884+0000", 2023-11-25T04:23:21.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3099999999999999e-07 2023-11-25T04:23:21.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.360 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:21.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784942+0000", 2023-11-25T04:23:21.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8570000000000003e-05 2023-11-25T04:23:21.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:21.361 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784949+0000", 2023-11-25T04:23:21.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0310000000000001e-06 2023-11-25T04:23:21.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:21.361 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784953+0000", 2023-11-25T04:23:21.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6320000000000001e-06 2023-11-25T04:23:21.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:21.362 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784993+0000", 2023-11-25T04:23:21.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9737000000000003e-05 2023-11-25T04:23:21.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.362 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:21.362 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785011+0000", 2023-11-25T04:23:21.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8349999999999999e-05 2023-11-25T04:23:21.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.362 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:21.362 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791024+0000", 2023-11-25T04:23:21.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006012997 2023-11-25T04:23:21.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:21.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791055+0000", 2023-11-25T04:23:21.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1254000000000003e-05 2023-11-25T04:23:21.363 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:21.363 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:21.363 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:21.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.364 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:156 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'110, mlcod=11'105)", 2023-11-25T04:23:21.364 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:21.364 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 43.567340547999997, 2023-11-25T04:23:21.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005616116, 2023-11-25T04:23:21.364 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:21.364 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:21.364 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:21.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.364 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:21.364 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:21.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:21.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.365 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:21.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:21.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:21.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.365 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:21.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785470+0000", 2023-11-25T04:23:21.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0699999999999998e-07 2023-11-25T04:23:21.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:21.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785474+0000", 2023-11-25T04:23:21.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.048e-06 2023-11-25T04:23:21.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:21.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785480+0000", 2023-11-25T04:23:21.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7729999999999996e-06 2023-11-25T04:23:21.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:21.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785483+0000", 2023-11-25T04:23:21.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1970000000000001e-06 2023-11-25T04:23:21.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:21.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787936+0000", 2023-11-25T04:23:21.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024529040000000001 2023-11-25T04:23:21.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:21.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787947+0000", 2023-11-25T04:23:21.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1669e-05 2023-11-25T04:23:21.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:21.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791057+0000", 2023-11-25T04:23:21.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031095200000000002 2023-11-25T04:23:21.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:21.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791086+0000", 2023-11-25T04:23:21.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8697999999999999e-05 2023-11-25T04:23:21.369 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:21.369 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:21.369 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:21.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.369 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:157 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'111, mlcod=11'106)", 2023-11-25T04:23:21.369 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:21.369 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 43.566747990000003, 2023-11-25T04:23:21.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050590280000000001, 2023-11-25T04:23:21.369 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:21.370 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:21.370 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:21.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:21.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:21.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:21.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:21.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:21.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:21.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:21.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786063+0000", 2023-11-25T04:23:21.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2600000000000002e-07 2023-11-25T04:23:21.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:21.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786069+0000", 2023-11-25T04:23:21.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8839999999999999e-06 2023-11-25T04:23:21.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:21.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786079+0000", 2023-11-25T04:23:21.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0124e-05 2023-11-25T04:23:21.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:21.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786085+0000", 2023-11-25T04:23:21.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6660000000000002e-06 2023-11-25T04:23:21.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:21.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788160+0000", 2023-11-25T04:23:21.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020747669999999999 2023-11-25T04:23:21.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:21.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788180+0000", 2023-11-25T04:23:21.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0208999999999999e-05 2023-11-25T04:23:21.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:21.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791087+0000", 2023-11-25T04:23:21.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029064630000000002 2023-11-25T04:23:21.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:21.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791121+0000", 2023-11-25T04:23:21.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4289000000000003e-05 2023-11-25T04:23:21.374 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:21.375 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:21.375 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:21.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.375 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:158 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'112, mlcod=11'106)", 2023-11-25T04:23:21.375 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:21.375 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 43.566331597000001, 2023-11-25T04:23:21.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046663030000000001, 2023-11-25T04:23:21.375 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:21.375 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:21.375 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:21.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:21.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:21.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:21.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:21.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:21.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:21.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:21.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786479+0000", 2023-11-25T04:23:21.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2799999999999996e-07 2023-11-25T04:23:21.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.377 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:21.377 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786498+0000", 2023-11-25T04:23:21.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9069e-05 2023-11-25T04:23:21.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.377 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:21.377 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786508+0000", 2023-11-25T04:23:21.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5810000000000003e-06 2023-11-25T04:23:21.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.378 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:21.378 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786512+0000", 2023-11-25T04:23:21.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4889999999999997e-06 2023-11-25T04:23:21.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.378 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:21.378 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788438+0000", 2023-11-25T04:23:21.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019257059999999999 2023-11-25T04:23:21.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:21.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788449+0000", 2023-11-25T04:23:21.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1419e-05 2023-11-25T04:23:21.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:21.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791122+0000", 2023-11-25T04:23:21.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002672822 2023-11-25T04:23:21.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:21.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791145+0000", 2023-11-25T04:23:21.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2589e-05 2023-11-25T04:23:21.380 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:21.380 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:21.380 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:21.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.380 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:206 1.0 e12/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 12'152, mlcod=12'148)", 2023-11-25T04:23:21.380 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:21.380 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 43.054563127000002, 2023-11-25T04:23:21.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046728189999999999, 2023-11-25T04:23:21.380 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:21.381 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:21.381 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:21.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:21.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:21.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:21.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:21.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:21.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:21.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.382 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:21.382 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298248+0000", 2023-11-25T04:23:21.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3600000000000003e-07 2023-11-25T04:23:21.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.382 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:21.382 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298278+0000", 2023-11-25T04:23:21.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0474e-05 2023-11-25T04:23:21.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:21.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298288+0000", 2023-11-25T04:23:21.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0156000000000001e-05 2023-11-25T04:23:21.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:21.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298293+0000", 2023-11-25T04:23:21.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7400000000000004e-06 2023-11-25T04:23:21.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:21.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298333+0000", 2023-11-25T04:23:21.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9666000000000003e-05 2023-11-25T04:23:21.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:21.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298351+0000", 2023-11-25T04:23:21.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8544999999999999e-05 2023-11-25T04:23:21.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:21.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302891+0000", 2023-11-25T04:23:21.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045398519999999996 2023-11-25T04:23:21.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.385 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:21.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302920+0000", 2023-11-25T04:23:21.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8750000000000001e-05 2023-11-25T04:23:21.385 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:21.385 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:21.385 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:21.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.385 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:14 3.15 3:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:21.386 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:21.386 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 34.319428770000002, 2023-11-25T04:23:21.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010990437, 2023-11-25T04:23:21.386 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:21.386 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:21.386 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:21.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:21.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:21.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:21.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:21.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:21.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:21.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:21.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033383+0000", 2023-11-25T04:23:21.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9870000000000002e-06 2023-11-25T04:23:21.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:21.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033398+0000", 2023-11-25T04:23:21.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4569999999999999e-05 2023-11-25T04:23:21.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.388 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:21.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033411+0000", 2023-11-25T04:23:21.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3543999999999999e-05 2023-11-25T04:23:21.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.388 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:21.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033423+0000", 2023-11-25T04:23:21.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1905999999999999e-05 2023-11-25T04:23:21.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:21.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034433+0000", 2023-11-25T04:23:21.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00101022 2023-11-25T04:23:21.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:21.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034539+0000", 2023-11-25T04:23:21.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010597199999999999 2023-11-25T04:23:21.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:21.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034667+0000", 2023-11-25T04:23:21.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012765500000000001 2023-11-25T04:23:21.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:21.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037803+0000", 2023-11-25T04:23:21.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031362220000000001 2023-11-25T04:23:21.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:21.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:23:21.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0064964669999999997 2023-11-25T04:23:21.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:21.391 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044331+0000", 2023-11-25T04:23:21.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1544999999999997e-05 2023-11-25T04:23:21.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:21.391 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044372+0000", 2023-11-25T04:23:21.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0349000000000003e-05 2023-11-25T04:23:21.391 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:21.391 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:21.391 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:21.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.392 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:15 3.15 3:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:21.392 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:21.392 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 34.318769119999999, 2023-11-25T04:23:21.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010464362, 2023-11-25T04:23:21.392 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:21.392 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:21.392 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:21.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:21.392 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:21.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:21.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:21.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:21.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:21.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:21.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034042+0000", 2023-11-25T04:23:21.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0999999999999997e-07 2023-11-25T04:23:21.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:21.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034048+0000", 2023-11-25T04:23:21.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0390000000000001e-06 2023-11-25T04:23:21.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:21.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034059+0000", 2023-11-25T04:23:21.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1182999999999999e-05 2023-11-25T04:23:21.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:21.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034067+0000", 2023-11-25T04:23:21.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1559999999999995e-06 2023-11-25T04:23:21.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:21.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035005+0000", 2023-11-25T04:23:21.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00093747399999999999 2023-11-25T04:23:21.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:21.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035077+0000", 2023-11-25T04:23:21.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2747e-05 2023-11-25T04:23:21.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:21.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035219+0000", 2023-11-25T04:23:21.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000141877 2023-11-25T04:23:21.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:21.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037810+0000", 2023-11-25T04:23:21.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002590752 2023-11-25T04:23:21.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:21.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044437+0000", 2023-11-25T04:23:21.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066270110000000004 2023-11-25T04:23:21.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:21.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044481+0000", 2023-11-25T04:23:21.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4106000000000001e-05 2023-11-25T04:23:21.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:21.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044505+0000", 2023-11-25T04:23:21.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4207e-05 2023-11-25T04:23:21.397 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:21.397 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:21.397 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:21.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.398 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:16 3.1d 3:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:21.398 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:21.398 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 34.318727271999997, 2023-11-25T04:23:21.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010413923, 2023-11-25T04:23:21.398 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:21.398 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:21.398 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:21.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:21.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:21.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:21.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:21.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:21.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:21.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:21.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:21.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4499999999999997e-07 2023-11-25T04:23:21.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:21.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034087+0000", 2023-11-25T04:23:21.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9770000000000002e-06 2023-11-25T04:23:21.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:21.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034094+0000", 2023-11-25T04:23:21.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8809999999999998e-06 2023-11-25T04:23:21.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:21.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034100+0000", 2023-11-25T04:23:21.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8440000000000003e-06 2023-11-25T04:23:21.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:21.401 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034993+0000", 2023-11-25T04:23:21.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00089336799999999996 2023-11-25T04:23:21.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:21.401 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035099+0000", 2023-11-25T04:23:21.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106037 2023-11-25T04:23:21.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:21.401 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035276+0000", 2023-11-25T04:23:21.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017631799999999999 2023-11-25T04:23:21.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:21.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037815+0000", 2023-11-25T04:23:21.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025395959999999999 2023-11-25T04:23:21.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:21.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044420+0000", 2023-11-25T04:23:21.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066045749999999997 2023-11-25T04:23:21.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:21.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044463+0000", 2023-11-25T04:23:21.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3205999999999999e-05 2023-11-25T04:23:21.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:21.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044497+0000", 2023-11-25T04:23:21.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3476000000000003e-05 2023-11-25T04:23:21.403 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:21.403 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:21.403 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:21.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.404 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:17 3.1d 3:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:21.404 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:21.404 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 34.318702063000003, 2023-11-25T04:23:21.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01127356, 2023-11-25T04:23:21.404 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:21.404 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:21.404 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:21.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:21.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:21.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:21.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:21.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:21.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:21.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:21.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:21.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1100000000000001e-07 2023-11-25T04:23:21.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:21.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034112+0000", 2023-11-25T04:23:21.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8600000000000003e-06 2023-11-25T04:23:21.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:21.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034120+0000", 2023-11-25T04:23:21.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7700000000000001e-06 2023-11-25T04:23:21.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:21.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034127+0000", 2023-11-25T04:23:21.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0999999999999998e-06 2023-11-25T04:23:21.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:21.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035601+0000", 2023-11-25T04:23:21.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014739410000000001 2023-11-25T04:23:21.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:21.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035685+0000", 2023-11-25T04:23:21.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3677999999999994e-05 2023-11-25T04:23:21.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:21.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035827+0000", 2023-11-25T04:23:21.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014232599999999999 2023-11-25T04:23:21.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:21.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038725+0000", 2023-11-25T04:23:21.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028982779999999998 2023-11-25T04:23:21.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:21.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045323+0000", 2023-11-25T04:23:21.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0065980780000000003 2023-11-25T04:23:21.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:21.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045354+0000", 2023-11-25T04:23:21.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0463e-05 2023-11-25T04:23:21.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:21.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045381+0000", 2023-11-25T04:23:21.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7654999999999999e-05 2023-11-25T04:23:21.409 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:21.409 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:21.409 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:21.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.409 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:22 3.e 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:21.409 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:21.409 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 34.318675650000003, 2023-11-25T04:23:21.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010133422, 2023-11-25T04:23:21.410 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:21.410 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:21.410 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:21.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:21.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:21.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:21.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:21.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:21.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:21.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.411 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:21.411 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034135+0000", 2023-11-25T04:23:21.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-07 2023-11-25T04:23:21.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.411 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:21.411 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034138+0000", 2023-11-25T04:23:21.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6009999999999999e-06 2023-11-25T04:23:21.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.411 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:21.412 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034145+0000", 2023-11-25T04:23:21.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1169999999999998e-06 2023-11-25T04:23:21.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.412 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:21.412 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034150+0000", 2023-11-25T04:23:21.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9080000000000003e-06 2023-11-25T04:23:21.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.412 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:21.412 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034209+0000", 2023-11-25T04:23:21.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8149000000000002e-05 2023-11-25T04:23:21.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:21.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034415+0000", 2023-11-25T04:23:21.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020637699999999999 2023-11-25T04:23:21.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:21.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034584+0000", 2023-11-25T04:23:21.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016870399999999999 2023-11-25T04:23:21.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:21.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.036979+0000", 2023-11-25T04:23:21.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023957509999999998 2023-11-25T04:23:21.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.414 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:21.414 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044161+0000", 2023-11-25T04:23:21.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0071811590000000003 2023-11-25T04:23:21.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.414 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:21.414 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044224+0000", 2023-11-25T04:23:21.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3454000000000006e-05 2023-11-25T04:23:21.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:21.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044268+0000", 2023-11-25T04:23:21.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3761999999999999e-05 2023-11-25T04:23:21.415 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:21.415 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:21.415 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:21.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.415 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:23 3.e 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:21.415 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:21.416 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 34.318644241000001, 2023-11-25T04:23:21.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010281676, 2023-11-25T04:23:21.416 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:21.416 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:21.416 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:21.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:21.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:21.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:21.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:21.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:21.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:21.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:21.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:21.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5799999999999999e-07 2023-11-25T04:23:21.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:21.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034169+0000", 2023-11-25T04:23:21.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1879999999999998e-06 2023-11-25T04:23:21.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:21.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034177+0000", 2023-11-25T04:23:21.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7020000000000001e-06 2023-11-25T04:23:21.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:21.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034184+0000", 2023-11-25T04:23:21.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3910000000000003e-06 2023-11-25T04:23:21.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:21.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034924+0000", 2023-11-25T04:23:21.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00074032999999999998 2023-11-25T04:23:21.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:21.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034991+0000", 2023-11-25T04:23:21.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7347000000000004e-05 2023-11-25T04:23:21.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:21.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035096+0000", 2023-11-25T04:23:21.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010488 2023-11-25T04:23:21.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:21.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037588+0000", 2023-11-25T04:23:21.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024918309999999999 2023-11-25T04:23:21.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:21.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044367+0000", 2023-11-25T04:23:21.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067787589999999997 2023-11-25T04:23:21.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:21.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044415+0000", 2023-11-25T04:23:21.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7803000000000003e-05 2023-11-25T04:23:21.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:21.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044447+0000", 2023-11-25T04:23:21.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2886999999999999e-05 2023-11-25T04:23:21.421 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:21.422 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:21.422 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:21.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.422 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:30 3.15 3:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:21.422 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:21.422 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 34.318585800999998, 2023-11-25T04:23:21.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010365070000000001, 2023-11-25T04:23:21.422 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:21.422 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:21.422 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:21.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:21.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:21.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:21.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:21.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:21.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:21.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:21.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034225+0000", 2023-11-25T04:23:21.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7400000000000002e-07 2023-11-25T04:23:21.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:21.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034229+0000", 2023-11-25T04:23:21.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3100000000000002e-06 2023-11-25T04:23:21.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:21.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034239+0000", 2023-11-25T04:23:21.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9140000000000003e-06 2023-11-25T04:23:21.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:21.425 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034246+0000", 2023-11-25T04:23:21.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0149999999999998e-06 2023-11-25T04:23:21.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:21.425 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035572+0000", 2023-11-25T04:23:21.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001325655 2023-11-25T04:23:21.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:21.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035730+0000", 2023-11-25T04:23:21.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015827899999999999 2023-11-25T04:23:21.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:21.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035857+0000", 2023-11-25T04:23:21.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012644699999999999 2023-11-25T04:23:21.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:21.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038737+0000", 2023-11-25T04:23:21.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002880077 2023-11-25T04:23:21.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:21.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044540+0000", 2023-11-25T04:23:21.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058034239999999997 2023-11-25T04:23:21.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:21.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044569+0000", 2023-11-25T04:23:21.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8388999999999999e-05 2023-11-25T04:23:21.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:21.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044589+0000", 2023-11-25T04:23:21.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0686000000000002e-05 2023-11-25T04:23:21.428 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:21.428 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:21.428 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:21.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.428 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:33 3.1b 3:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:21.428 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:21.428 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 34.318540302999999, 2023-11-25T04:23:21.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099754049999999997, 2023-11-25T04:23:21.428 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:21.429 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:23:21.429 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:21.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:21.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:21.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:21.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:21.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:21.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:21.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.430 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:21.430 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:21.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4799999999999998e-07 2023-11-25T04:23:21.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.430 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:21.430 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034273+0000", 2023-11-25T04:23:21.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7819999999999998e-06 2023-11-25T04:23:21.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.430 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:21.430 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034285+0000", 2023-11-25T04:23:21.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2058e-05 2023-11-25T04:23:21.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:21.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034291+0000", 2023-11-25T04:23:21.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0680000000000002e-06 2023-11-25T04:23:21.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:21.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034370+0000", 2023-11-25T04:23:21.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8452e-05 2023-11-25T04:23:21.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:21.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034477+0000", 2023-11-25T04:23:21.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107471 2023-11-25T04:23:21.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:21.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044245+0000", 2023-11-25T04:23:21.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0097679259999999997 2023-11-25T04:23:21.432 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:21.432 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:21.432 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:21.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.433 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:36 3.1a 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:21.433 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:21.433 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 34.318493629999999, 2023-11-25T04:23:21.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058179099999999999, 2023-11-25T04:23:21.433 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:21.433 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:23:21.433 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:21.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:21.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:21.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:21.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.434 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:21.434 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:21.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:21.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.434 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:21.434 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034317+0000", 2023-11-25T04:23:21.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1999999999999999e-07 2023-11-25T04:23:21.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.434 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:21.434 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034321+0000", 2023-11-25T04:23:21.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.233e-06 2023-11-25T04:23:21.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:21.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034337+0000", 2023-11-25T04:23:21.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5308999999999999e-05 2023-11-25T04:23:21.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:21.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034354+0000", 2023-11-25T04:23:21.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7411999999999999e-05 2023-11-25T04:23:21.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.439 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:21.439 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034465+0000", 2023-11-25T04:23:21.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011154699999999999 2023-11-25T04:23:21.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.439 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:21.439 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034560+0000", 2023-11-25T04:23:21.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4186999999999998e-05 2023-11-25T04:23:21.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:21.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.040134+0000", 2023-11-25T04:23:21.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055746019999999997 2023-11-25T04:23:21.440 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:21.440 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:21.440 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:21.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.440 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:9 3.10 e18/16 3:09401a81:::601.00000000:head v 18'4, mlcod=0'0)", 2023-11-25T04:23:21.440 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:21.440 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 34.309599321999997, 2023-11-25T04:23:21.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061641020000000003, 2023-11-25T04:23:21.441 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:21.441 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:21.441 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:21.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:21.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:21.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:21.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:21.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:21.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:21.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:21.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:21.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5300000000000001e-07 2023-11-25T04:23:21.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:21.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043222+0000", 2023-11-25T04:23:21.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.082e-05 2023-11-25T04:23:21.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:21.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043228+0000", 2023-11-25T04:23:21.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6699999999999997e-06 2023-11-25T04:23:21.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:21.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043236+0000", 2023-11-25T04:23:21.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8189999999999992e-06 2023-11-25T04:23:21.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:21.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043507+0000", 2023-11-25T04:23:21.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000271012 2023-11-25T04:23:21.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:21.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043530+0000", 2023-11-25T04:23:21.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2659000000000001e-05 2023-11-25T04:23:21.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.445 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:21.445 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049321+0000", 2023-11-25T04:23:21.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057915129999999999 2023-11-25T04:23:21.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.445 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:21.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049375+0000", 2023-11-25T04:23:21.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3455999999999998e-05 2023-11-25T04:23:21.446 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:21.446 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:21.446 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:21.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.446 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:1 3.14 e18/16 3:292cf221:::200.00000000:head v 18'3, mlcod=0'0)", 2023-11-25T04:23:21.446 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:21.446 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 34.309562608999997, 2023-11-25T04:23:21.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0060983599999999997, 2023-11-25T04:23:21.447 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:21.447 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:21.447 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:21.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.447 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:21.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:21.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:21.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.447 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:21.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:21.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:21.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:21.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043248+0000", 2023-11-25T04:23:21.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9399999999999995e-07 2023-11-25T04:23:21.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:21.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043252+0000", 2023-11-25T04:23:21.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1799999999999998e-06 2023-11-25T04:23:21.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.449 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:21.449 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043259+0000", 2023-11-25T04:23:21.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0049999999999998e-06 2023-11-25T04:23:21.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.449 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:21.449 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043265+0000", 2023-11-25T04:23:21.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0859999999999998e-06 2023-11-25T04:23:21.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.449 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:21.449 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043311+0000", 2023-11-25T04:23:21.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6082999999999999e-05 2023-11-25T04:23:21.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.450 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:21.450 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043330+0000", 2023-11-25T04:23:21.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8556e-05 2023-11-25T04:23:21.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.450 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:21.450 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049303+0000", 2023-11-25T04:23:21.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059733470000000004 2023-11-25T04:23:21.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.451 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:21.451 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049346+0000", 2023-11-25T04:23:21.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2608999999999998e-05 2023-11-25T04:23:21.451 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:21.451 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:21.451 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:21.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.451 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:3 3.14 e18/16 3:287034c4:::1.00000000.inode:head v 18'4, mlcod=0'0)", 2023-11-25T04:23:21.451 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:21.451 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 34.309046103, 2023-11-25T04:23:21.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057040090000000003, 2023-11-25T04:23:21.451 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:21.451 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:21.452 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:21.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:21.452 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:21.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:21.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:21.452 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:21.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:21.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.453 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:21.453 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:21.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9799999999999999e-07 2023-11-25T04:23:21.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.453 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:21.453 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043785+0000", 2023-11-25T04:23:21.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0721999999999999e-05 2023-11-25T04:23:21.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.453 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:21.453 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043793+0000", 2023-11-25T04:23:21.454 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5839999999999995e-06 2023-11-25T04:23:21.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.454 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:21.454 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043803+0000", 2023-11-25T04:23:21.454 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0729999999999994e-06 2023-11-25T04:23:21.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.454 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:21.454 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043843+0000", 2023-11-25T04:23:21.454 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0747999999999997e-05 2023-11-25T04:23:21.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:21.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043856+0000", 2023-11-25T04:23:21.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2697000000000001e-05 2023-11-25T04:23:21.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:21.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049426+0000", 2023-11-25T04:23:21.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055704409999999998 2023-11-25T04:23:21.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:21.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049468+0000", 2023-11-25T04:23:21.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1446000000000003e-05 2023-11-25T04:23:21.455 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:21.456 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:21.456 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:21.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.456 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "pg_update_log_missing(3.14 epoch 18/16 rep_tid 171 entries 18'5 (0'0) error 3:2e2fa760:::200.00000001:head by mds.0.3:32 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:23:21.456 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:21.456 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 34.308885140000001, 2023-11-25T04:23:21.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005556938, 2023-11-25T04:23:21.456 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:21.456 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2023-11-25T04:23:21.456 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:21.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:21.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:21.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:21.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:21.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:21.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:21.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:21.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:21.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.03e-07 2023-11-25T04:23:21.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.458 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:21.458 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043931+0000", 2023-11-25T04:23:21.458 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1079999999999998e-06 2023-11-25T04:23:21.458 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.458 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.458 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:21.458 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043944+0000", 2023-11-25T04:23:21.458 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2852e-05 2023-11-25T04:23:21.458 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.458 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.458 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:21.458 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043955+0000", 2023-11-25T04:23:21.458 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1037e-05 2023-11-25T04:23:21.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.459 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:21.459 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044088+0000", 2023-11-25T04:23:21.459 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013269599999999999 2023-11-25T04:23:21.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.459 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:21.459 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049482+0000", 2023-11-25T04:23:21.459 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053940419999999999 2023-11-25T04:23:21.459 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:21.459 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:21.459 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:21.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.460 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:10 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'5, mlcod=0'0)", 2023-11-25T04:23:21.460 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:21.460 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 34.308795111999999, 2023-11-25T04:23:21.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054493099999999997, 2023-11-25T04:23:21.460 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:21.460 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:21.460 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:21.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:21.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:21.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:21.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:21.461 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:21.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:21.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:21.461 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:21.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6100000000000002e-07 2023-11-25T04:23:21.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:21.461 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044019+0000", 2023-11-25T04:23:21.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.839e-06 2023-11-25T04:23:21.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.462 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:21.462 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044026+0000", 2023-11-25T04:23:21.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5869999999999998e-06 2023-11-25T04:23:21.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.462 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:21.462 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044031+0000", 2023-11-25T04:23:21.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4909999999999996e-06 2023-11-25T04:23:21.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.462 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:21.463 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044070+0000", 2023-11-25T04:23:21.463 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8829999999999999e-05 2023-11-25T04:23:21.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.463 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.463 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:21.463 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044081+0000", 2023-11-25T04:23:21.463 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1447999999999999e-05 2023-11-25T04:23:21.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.463 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.463 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:21.463 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049406+0000", 2023-11-25T04:23:21.464 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053246480000000004 2023-11-25T04:23:21.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.464 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:21.464 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049464+0000", 2023-11-25T04:23:21.464 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8205999999999998e-05 2023-11-25T04:23:21.464 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:21.464 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:21.464 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:21.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.464 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:11 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'6, mlcod=0'0)", 2023-11-25T04:23:21.465 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:21.465 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 34.308593848000001, 2023-11-25T04:23:21.465 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052673499999999996, 2023-11-25T04:23:21.465 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:21.465 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:21.465 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:21.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.465 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:21.465 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:21.465 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:21.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:21.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:21.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:21.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:21.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:21.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0199999999999998e-07 2023-11-25T04:23:21.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:21.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044220+0000", 2023-11-25T04:23:21.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7720000000000001e-06 2023-11-25T04:23:21.467 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.467 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.467 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:21.467 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044226+0000", 2023-11-25T04:23:21.467 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8579999999999996e-06 2023-11-25T04:23:21.467 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.467 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.467 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:21.467 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044232+0000", 2023-11-25T04:23:21.467 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1700000000000002e-06 2023-11-25T04:23:21.467 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.467 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.467 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:21.467 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044274+0000", 2023-11-25T04:23:21.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1254999999999997e-05 2023-11-25T04:23:21.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.468 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:21.468 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:23:21.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6880999999999999e-05 2023-11-25T04:23:21.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.468 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:21.468 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049466+0000", 2023-11-25T04:23:21.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051659219999999999 2023-11-25T04:23:21.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:21.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:21.469 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:21.469 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049483+0000", 2023-11-25T04:23:21.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.719e-05 2023-11-25T04:23:21.469 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:21.469 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:21.469 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:21.469 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:21.469 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:21.469 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:21.470 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2] out_osds: [] dead_osds: [0] live_osds: [1, 2] 2023-11-25T04:23:21.470 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 1.00 2023-11-25T04:23:21.470 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-25T04:23:21.471 INFO:teuthology.orchestra.run.smithi067.stderr:Error ENXIO: problem getting command descriptions from osd.0 2023-11-25T04:23:21.472 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:21.469+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:21.472 INFO:teuthology.orchestra.run.smithi067.stdout:osd.1: {} 2023-11-25T04:23:21.472 INFO:teuthology.orchestra.run.smithi067.stderr:osd.1: osd_enable_op_tracker = '' 2023-11-25T04:23:21.473 INFO:teuthology.orchestra.run.smithi067.stdout:osd.2: {} 2023-11-25T04:23:21.473 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:23:21.474 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-11-25T04:23:21.475 INFO:teuthology.orchestra.run.smithi067.stderr:osd.2: osd_enable_op_tracker = '' 2023-11-25T04:23:21.475 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-11-25T04:23:21.571 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:21.570+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:21.583 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:21.583 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:21.593 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:23:21.672 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:21.670+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:21.700 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:21.701 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:21.708 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:23:21.772 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:21.771+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:21.823 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:23:21.825 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2023-11-25T04:23:21.834 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":48,"stamp":"2023-11-25T04:23:21.500401+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597718,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":256,"num_write_kb":3483,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":202,"ondisk_log_size":202,"up":139,"acting":139,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":99504,"kb_used_data":1804,"kb_used_omap":4,"kb_used_meta":97659,"kb_avail":283016016,"statfs":{"total":289910292480,"available":289808400384,"internally_reserved":0,"allocated":1847296,"data_stored":1417297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":100003151},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.359783"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-25T04:23:14.143482+0000","last_change":"2023-11-25T04:22:47.039847+0000","last_active":"2023-11-25T04:23:14.143482+0000","last_peered":"2023-11-25T04:23:14.143482+0000","last_clean":"2023-11-25T04:23:14.143482+0000","last_became_active":"2023-11-25T04:22:47.039490+0000","last_became_peered":"2023-11-25T04:22:47.039490+0000","last_unstale":"2023-11-25T04:23:14.143482+0000","last_undegraded":"2023-11-25T04:23:14.143482+0000","last_fullsized":"2023-11-25T04:23:14.143482+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:25:06.132517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-25T04:23:14.143304+0000","last_change":"2023-11-25T04:22:47.040836+0000","last_active":"2023-11-25T04:23:14.143304+0000","last_peered":"2023-11-25T04:23:14.143304+0000","last_clean":"2023-11-25T04:23:14.143304+0000","last_became_active":"2023-11-25T04:22:47.040511+0000","last_became_peered":"2023-11-25T04:22:47.040511+0000","last_unstale":"2023-11-25T04:23:14.143304+0000","last_undegraded":"2023-11-25T04:23:14.143304+0000","last_fullsized":"2023-11-25T04:23:14.143304+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:15:39.511043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+undersized","last_fresh":"2023-11-25T04:23:14.142869+0000","last_change":"2023-11-25T04:23:09.124654+0000","last_active":"2023-11-25T04:23:14.142869+0000","last_peered":"2023-11-25T04:23:14.142869+0000","last_clean":"2023-11-25T04:22:46.014230+0000","last_became_active":"2023-11-25T04:23:09.124654+0000","last_became_peered":"2023-11-25T04:23:09.124654+0000","last_unstale":"2023-11-25T04:23:14.142869+0000","last_undegraded":"2023-11-25T04:23:14.142869+0000","last_fullsized":"2023-11-25T04:23:09.118297+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:48:07.052032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+undersized","last_fresh":"2023-11-25T04:23:14.139495+0000","last_change":"2023-11-25T04:23:09.118393+0000","last_active":"2023-11-25T04:23:14.139495+0000","last_peered":"2023-11-25T04:23:14.139495+0000","last_clean":"2023-11-25T04:22:46.014230+0000","last_became_active":"2023-11-25T04:23:09.118393+0000","last_became_peered":"2023-11-25T04:23:09.118393+0000","last_unstale":"2023-11-25T04:23:14.139495+0000","last_undegraded":"2023-11-25T04:23:14.139495+0000","last_fullsized":"2023-11-25T04:23:09.115286+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:11:07.574205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-25T04:23:14.142873+0000","last_change":"2023-11-25T04:22:47.039499+0000","last_active":"2023-11-25T04:23:14.142873+0000","last_peered":"2023-11-25T04:23:14.142873+0000","last_clean":"2023-11-25T04:23:14.142873+0000","last_became_active":"2023-11-25T04:22:47.039165+0000","last_became_peered":"2023-11-25T04:22:47.039165+0000","last_unstale":"2023-11-25T04:23:14.142873+0000","last_undegraded":"2023-11-25T04:23:14.142873+0000","last_fullsized":"2023-11-25T04:23:14.142873+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:57:28.855151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+undersized","last_fresh":"2023-11-25T04:23:14.139251+0000","last_change":"2023-11-25T04:23:09.118087+0000","last_active":"2023-11-25T04:23:14.139251+0000","last_peered":"2023-11-25T04:23:14.139251+0000","last_clean":"2023-11-25T04:22:46.014230+0000","last_became_active":"2023-11-25T04:23:09.118087+0000","last_became_peered":"2023-11-25T04:23:09.118087+0000","last_unstale":"2023-11-25T04:23:14.139251+0000","last_undegraded":"2023-11-25T04:23:14.139251+0000","last_fullsized":"2023-11-25T04:23:09.115029+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:18:56.845916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+undersized","last_fresh":"2023-11-25T04:23:14.139134+0000","last_change":"2023-11-25T04:23:09.118263+0000","last_active":"2023-11-25T04:23:14.139134+0000","last_peered":"2023-11-25T04:23:14.139134+0000","last_clean":"2023-11-25T04:22:46.014230+0000","last_became_active":"2023-11-25T04:23:09.118263+0000","last_became_peered":"2023-11-25T04:23:09.118263+0000","last_unstale":"2023-11-25T04:23:14.139134+0000","last_undegraded":"2023-11-25T04:23:14.139134+0000","last_fullsized":"2023-11-25T04:23:09.114937+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:23:15.914817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+undersized","last_fresh":"2023-11-25T04:23:14.139079+0000","last_change":"2023-11-25T04:23:09.118077+0000","last_active":"2023-11-25T04:23:14.139079+0000","last_peered":"2023-11-25T04:23:14.139079+0000","last_clean":"2023-11-25T04:22:46.014230+0000","last_became_active":"2023-11-25T04:23:09.118077+0000","last_became_peered":"2023-11-25T04:23:09.118077+0000","last_unstale":"2023-11-25T04:23:14.139079+0000","last_undegraded":"2023-11-25T04:23:14.139079+0000","last_fullsized":"2023-11-25T04:23:09.114891+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:00:26.187878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-25T04:23:14.139766+0000","last_change":"2023-11-25T04:22:47.038397+0000","last_active":"2023-11-25T04:23:14.139766+0000","last_peered":"2023-11-25T04:23:14.139766+0000","last_clean":"2023-11-25T04:23:14.139766+0000","last_became_active":"2023-11-25T04:22:47.037328+0000","last_became_peered":"2023-11-25T04:22:47.037328+0000","last_unstale":"2023-11-25T04:23:14.139766+0000","last_undegraded":"2023-11-25T04:23:14.139766+0000","last_fullsized":"2023-11-25T04:23:14.139766+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:56:59.689578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-25T04:23:14.143079+0000","last_change":"2023-11-25T04:22:47.040182+0000","last_active":"2023-11-25T04:23:14.143079+0000","last_peered":"2023-11-25T04:23:14.143079+0000","last_clean":"2023-11-25T04:23:14.143079+0000","last_became_active":"2023-11-25T04:22:47.039892+0000","last_became_peered":"2023-11-25T04:22:47.039892+0000","last_unstale":"2023-11-25T04:23:14.143079+0000","last_undegraded":"2023-11-25T04:23:14.143079+0000","last_fullsized":"2023-11-25T04:23:14.143079+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:38:09.176582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+undersized","last_fresh":"2023-11-25T04:23:14.139659+0000","last_change":"2023-11-25T04:23:09.118425+0000","last_active":"2023-11-25T04:23:14.139659+0000","last_peered":"2023-11-25T04:23:14.139659+0000","last_clean":"2023-11-25T04:22:46.014230+0000","last_became_active":"2023-11-25T04:23:09.118425+0000","last_became_peered":"2023-11-25T04:23:09.118425+0000","last_unstale":"2023-11-25T04:23:14.139659+0000","last_undegraded":"2023-11-25T04:23:14.139659+0000","last_fullsized":"2023-11-25T04:23:09.115337+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:47:55.336105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+undersized","last_fresh":"2023-11-25T04:23:14.142991+0000","last_change":"2023-11-25T04:23:09.124597+0000","last_active":"2023-11-25T04:23:14.142991+0000","last_peered":"2023-11-25T04:23:14.142991+0000","last_clean":"2023-11-25T04:23:05.103294+0000","last_became_active":"2023-11-25T04:23:09.124597+0000","last_became_peered":"2023-11-25T04:23:09.124597+0000","last_unstale":"2023-11-25T04:23:14.142991+0000","last_undegraded":"2023-11-25T04:23:14.142991+0000","last_fullsized":"2023-11-25T04:23:09.118225+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:11:07.574205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-25T04:23:14.142563+0000","last_change":"2023-11-25T04:22:47.040388+0000","last_active":"2023-11-25T04:23:14.142563+0000","last_peered":"2023-11-25T04:23:14.142563+0000","last_clean":"2023-11-25T04:23:14.142563+0000","last_became_active":"2023-11-25T04:22:47.040129+0000","last_became_peered":"2023-11-25T04:22:47.040129+0000","last_unstale":"2023-11-25T04:23:14.142563+0000","last_undegraded":"2023-11-25T04:23:14.142563+0000","last_fullsized":"2023-11-25T04:23:14.142563+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:31:54.857270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+undersized","last_fresh":"2023-11-25T04:23:14.142332+0000","last_change":"2023-11-25T04:23:09.121039+0000","last_active":"2023-11-25T04:23:14.142332+0000","last_peered":"2023-11-25T04:23:14.142332+0000","last_clean":"2023-11-25T04:22:46.014230+0000","last_became_active":"2023-11-25T04:23:09.121039+0000","last_became_peered":"2023-11-25T04:23:09.121039+0000","last_unstale":"2023-11-25T04:23:14.142332+0000","last_undegraded":"2023-11-25T04:23:14.142332+0000","last_fullsized":"2023-11-25T04:23:09.117797+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:18:56.845916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+undersized","last_fresh":"2023-11-25T04:23:14.142259+0000","last_change":"2023-11-25T04:23:09.124599+0000","last_active":"2023-11-25T04:23:14.142259+0000","last_peered":"2023-11-25T04:23:14.142259+0000","last_clean":"2023-11-25T04:22:46.014230+0000","last_became_active":"2023-11-25T04:23:09.124599+0000","last_became_peered":"2023-11-25T04:23:09.124599+0000","last_unstale":"2023-11-25T04:23:14.142259+0000","last_undegraded":"2023-11-25T04:23:14.142259+0000","last_fullsized":"2023-11-25T04:23:09.117743+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:13:35.199538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+undersized","last_fresh":"2023-11-25T04:23:14.139102+0000","last_change":"2023-11-25T04:23:09.118779+0000","last_active":"2023-11-25T04:23:14.139102+0000","last_peered":"2023-11-25T04:23:14.139102+0000","last_clean":"2023-11-25T04:22:46.014230+0000","last_became_active":"2023-11-25T04:23:09.118779+0000","last_became_peered":"2023-11-25T04:23:09.118779+0000","last_unstale":"2023-11-25T04:23:14.139102+0000","last_undegraded":"2023-11-25T04:23:14.139102+0000","last_fullsized":"2023-11-25T04:23:09.114892+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:10:59.482775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-25T04:23:14.140464+0000","last_change":"2023-11-25T04:22:47.039389+0000","last_active":"2023-11-25T04:23:14.140464+0000","last_peered":"2023-11-25T04:23:14.140464+0000","last_clean":"2023-11-25T04:23:14.140464+0000","last_became_active":"2023-11-25T04:22:47.039123+0000","last_became_peered":"2023-11-25T04:22:47.039123+0000","last_unstale":"2023-11-25T04:23:14.140464+0000","last_undegraded":"2023-11-25T04:23:14.140464+0000","last_fullsized":"2023-11-25T04:23:14.140464+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:09:11.570561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-25T04:23:14.143249+0000","last_change":"2023-11-25T04:22:47.036946+0000","last_active":"2023-11-25T04:23:14.143249+0000","last_peered":"2023-11-25T04:23:14.143249+0000","last_clean":"2023-11-25T04:23:14.143249+0000","last_became_active":"2023-11-25T04:22:47.036411+0000","last_became_peered":"2023-11-25T04:22:47.036411+0000","last_unstale":"2023-11-25T04:23:14.143249+0000","last_undegraded":"2023-11-25T04:23:14.143249+0000","last_fullsized":"2023-11-25T04:23:14.143249+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:16:29.085050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-25T04:23:14.139839+0000","last_change":"2023-11-25T04:22:47.038983+0000","last_active":"2023-11-25T04:23:14.139839+0000","last_peered":"2023-11-25T04:23:14.139839+0000","last_clean":"2023-11-25T04:23:14.139839+0000","last_became_active":"2023-11-25T04:22:47.038477+0000","last_became_peered":"2023-11-25T04:22:47.038477+0000","last_unstale":"2023-11-25T04:23:14.139839+0000","last_undegraded":"2023-11-25T04:23:14.139839+0000","last_fullsized":"2023-11-25T04:23:14.139839+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:30:04.568790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+undersized","last_fresh":"2023-11-25T04:23:14.139932+0000","last_change":"2023-11-25T04:23:09.118943+0000","last_active":"2023-11-25T04:23:14.139932+0000","last_peered":"2023-11-25T04:23:14.139932+0000","last_clean":"2023-11-25T04:22:46.014230+0000","last_became_active":"2023-11-25T04:23:09.118943+0000","last_became_peered":"2023-11-25T04:23:09.118943+0000","last_unstale":"2023-11-25T04:23:14.139932+0000","last_undegraded":"2023-11-25T04:23:14.139932+0000","last_fullsized":"2023-11-25T04:23:09.115616+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T16:22:28.651339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+undersized","last_fresh":"2023-11-25T04:23:14.140334+0000","last_change":"2023-11-25T04:23:09.119982+0000","last_active":"2023-11-25T04:23:14.140334+0000","last_peered":"2023-11-25T04:23:14.140334+0000","last_clean":"2023-11-25T04:22:46.014230+0000","last_became_active":"2023-11-25T04:23:09.119982+0000","last_became_peered":"2023-11-25T04:23:09.119982+0000","last_unstale":"2023-11-25T04:23:14.140334+0000","last_undegraded":"2023-11-25T04:23:14.140334+0000","last_fullsized":"2023-11-25T04:23:09.116370+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:26:25.521711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+undersized","last_fresh":"2023-11-25T04:23:14.142627+0000","last_change":"2023-11-25T04:23:09.125801+0000","last_active":"2023-11-25T04:23:14.142627+0000","last_peered":"2023-11-25T04:23:14.142627+0000","last_clean":"2023-11-25T04:23:05.103147+0000","last_became_active":"2023-11-25T04:23:09.125801+0000","last_became_peered":"2023-11-25T04:23:09.125801+0000","last_unstale":"2023-11-25T04:23:14.142627+0000","last_undegraded":"2023-11-25T04:23:14.142627+0000","last_fullsized":"2023-11-25T04:23:09.118939+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:52:57.375894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+undersized","last_fresh":"2023-11-25T04:23:14.139172+0000","last_change":"2023-11-25T04:23:09.117328+0000","last_active":"2023-11-25T04:23:14.139172+0000","last_peered":"2023-11-25T04:23:14.139172+0000","last_clean":"2023-11-25T04:23:05.102482+0000","last_became_active":"2023-11-25T04:23:09.117328+0000","last_became_peered":"2023-11-25T04:23:09.117328+0000","last_unstale":"2023-11-25T04:23:14.139172+0000","last_undegraded":"2023-11-25T04:23:14.139172+0000","last_fullsized":"2023-11-25T04:23:09.114954+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:13:35.199538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-25T04:23:14.142208+0000","last_change":"2023-11-25T04:22:47.036805+0000","last_active":"2023-11-25T04:23:14.142208+0000","last_peered":"2023-11-25T04:23:14.142208+0000","last_clean":"2023-11-25T04:23:14.142208+0000","last_became_active":"2023-11-25T04:22:47.036135+0000","last_became_peered":"2023-11-25T04:22:47.036135+0000","last_unstale":"2023-11-25T04:23:14.142208+0000","last_undegraded":"2023-11-25T04:23:14.142208+0000","last_fullsized":"2023-11-25T04:23:14.142208+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:25:21.730992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+undersized","last_fresh":"2023-11-25T04:23:14.143538+0000","last_change":"2023-11-25T04:23:09.125505+0000","last_active":"2023-11-25T04:23:14.143538+0000","last_peered":"2023-11-25T04:23:14.143538+0000","last_clean":"2023-11-25T04:23:05.103763+0000","last_became_active":"2023-11-25T04:23:09.125505+0000","last_became_peered":"2023-11-25T04:23:09.125505+0000","last_unstale":"2023-11-25T04:23:14.143538+0000","last_undegraded":"2023-11-25T04:23:14.143538+0000","last_fullsized":"2023-11-25T04:23:09.119531+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:11:02.936931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+undersized","last_fresh":"2023-11-25T04:23:14.139872+0000","last_change":"2023-11-25T04:23:09.120144+0000","last_active":"2023-11-25T04:23:14.139872+0000","last_peered":"2023-11-25T04:23:14.139872+0000","last_clean":"2023-11-25T04:23:05.103232+0000","last_became_active":"2023-11-25T04:23:09.120144+0000","last_became_peered":"2023-11-25T04:23:09.120144+0000","last_unstale":"2023-11-25T04:23:14.139872+0000","last_undegraded":"2023-11-25T04:23:14.139872+0000","last_fullsized":"2023-11-25T04:23:09.116908+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:11:32.837518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+undersized","last_fresh":"2023-11-25T04:23:14.143479+0000","last_change":"2023-11-25T04:23:09.126172+0000","last_active":"2023-11-25T04:23:14.143479+0000","last_peered":"2023-11-25T04:23:14.143479+0000","last_clean":"2023-11-25T04:22:46.014230+0000","last_became_active":"2023-11-25T04:23:09.126172+0000","last_became_peered":"2023-11-25T04:23:09.126172+0000","last_unstale":"2023-11-25T04:23:14.143479+0000","last_undegraded":"2023-11-25T04:23:14.143479+0000","last_fullsized":"2023-11-25T04:23:09.119666+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:06:15.996381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+undersized","last_fresh":"2023-11-25T04:23:14.143312+0000","last_change":"2023-11-25T04:23:09.125249+0000","last_active":"2023-11-25T04:23:14.143312+0000","last_peered":"2023-11-25T04:23:14.143312+0000","last_clean":"2023-11-25T04:23:05.103465+0000","last_became_active":"2023-11-25T04:23:09.125249+0000","last_became_peered":"2023-11-25T04:23:09.125249+0000","last_unstale":"2023-11-25T04:23:14.143312+0000","last_undegraded":"2023-11-25T04:23:14.143312+0000","last_fullsized":"2023-11-25T04:23:09.118889+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:34:12.483240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-25T04:23:14.139573+0000","last_change":"2023-11-25T04:22:47.037668+0000","last_active":"2023-11-25T04:23:14.139573+0000","last_peered":"2023-11-25T04:23:14.139573+0000","last_clean":"2023-11-25T04:23:14.139573+0000","last_became_active":"2023-11-25T04:22:47.036823+0000","last_became_peered":"2023-11-25T04:22:47.036823+0000","last_unstale":"2023-11-25T04:23:14.139573+0000","last_undegraded":"2023-11-25T04:23:14.139573+0000","last_fullsized":"2023-11-25T04:23:14.139573+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:02:07.088241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+undersized","last_fresh":"2023-11-25T04:23:14.142861+0000","last_change":"2023-11-25T04:23:09.126173+0000","last_active":"2023-11-25T04:23:14.142861+0000","last_peered":"2023-11-25T04:23:14.142861+0000","last_clean":"2023-11-25T04:23:05.103283+0000","last_became_active":"2023-11-25T04:23:09.126173+0000","last_became_peered":"2023-11-25T04:23:09.126173+0000","last_unstale":"2023-11-25T04:23:14.142861+0000","last_undegraded":"2023-11-25T04:23:14.142861+0000","last_fullsized":"2023-11-25T04:23:09.119301+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:26:25.521711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-25T04:23:14.139302+0000","last_change":"2023-11-25T04:22:47.037786+0000","last_active":"2023-11-25T04:23:14.139302+0000","last_peered":"2023-11-25T04:23:14.139302+0000","last_clean":"2023-11-25T04:23:14.139302+0000","last_became_active":"2023-11-25T04:22:47.037206+0000","last_became_peered":"2023-11-25T04:22:47.037206+0000","last_unstale":"2023-11-25T04:23:14.139302+0000","last_undegraded":"2023-11-25T04:23:14.139302+0000","last_fullsized":"2023-11-25T04:23:14.139302+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:11:40.532140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+undersized","last_fresh":"2023-11-25T04:23:14.139231+0000","last_change":"2023-11-25T04:23:09.119905+0000","last_active":"2023-11-25T04:23:14.139231+0000","last_peered":"2023-11-25T04:23:14.139231+0000","last_clean":"2023-11-25T04:22:46.014230+0000","last_became_active":"2023-11-25T04:23:09.119905+0000","last_became_peered":"2023-11-25T04:23:09.119905+0000","last_unstale":"2023-11-25T04:23:14.139231+0000","last_undegraded":"2023-11-25T04:23:14.139231+0000","last_fullsized":"2023-11-25T04:23:09.115875+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:37:34.213542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-25T04:23:14.143204+0000","last_change":"2023-11-25T04:22:47.036689+0000","last_active":"2023-11-25T04:23:14.143204+0000","last_peered":"2023-11-25T04:23:14.143204+0000","last_clean":"2023-11-25T04:23:14.143204+0000","last_became_active":"2023-11-25T04:22:47.036107+0000","last_became_peered":"2023-11-25T04:22:47.036107+0000","last_unstale":"2023-11-25T04:23:14.143204+0000","last_undegraded":"2023-11-25T04:23:14.143204+0000","last_fullsized":"2023-11-25T04:23:14.143204+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:02:07.088241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"18'2","reported_seq":20,"reported_epoch":24,"state":"active+undersized+degraded","last_fresh":"2023-11-25T04:23:14.139886+0000","last_change":"2023-11-25T04:23:09.119745+0000","last_active":"2023-11-25T04:23:14.139886+0000","last_peered":"2023-11-25T04:23:14.139886+0000","last_clean":"2023-11-25T04:22:47.031253+0000","last_became_active":"2023-11-25T04:23:09.119745+0000","last_became_peered":"2023-11-25T04:23:09.119745+0000","last_unstale":"2023-11-25T04:23:14.139886+0000","last_undegraded":"2023-11-25T04:23:09.116349+0000","last_fullsized":"2023-11-25T04:23:09.116152+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:50:47.336152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":2}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-25T04:23:14.143111+0000","last_change":"2023-11-25T04:22:47.037888+0000","last_active":"2023-11-25T04:23:14.143111+0000","last_peered":"2023-11-25T04:23:14.143111+0000","last_clean":"2023-11-25T04:23:14.143111+0000","last_became_active":"2023-11-25T04:22:47.037158+0000","last_became_peered":"2023-11-25T04:22:47.037158+0000","last_unstale":"2023-11-25T04:23:14.143111+0000","last_undegraded":"2023-11-25T04:23:14.143111+0000","last_fullsized":"2023-11-25T04:23:14.143111+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:56:59.689578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+undersized","last_fresh":"2023-11-25T04:23:14.139517+0000","last_change":"2023-11-25T04:23:09.119897+0000","last_active":"2023-11-25T04:23:14.139517+0000","last_peered":"2023-11-25T04:23:14.139517+0000","last_clean":"2023-11-25T04:23:05.102825+0000","last_became_active":"2023-11-25T04:23:09.119897+0000","last_became_peered":"2023-11-25T04:23:09.119897+0000","last_unstale":"2023-11-25T04:23:14.139517+0000","last_undegraded":"2023-11-25T04:23:14.139517+0000","last_fullsized":"2023-11-25T04:23:09.115928+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:46:13.177177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+undersized","last_fresh":"2023-11-25T04:23:14.140259+0000","last_change":"2023-11-25T04:23:09.120649+0000","last_active":"2023-11-25T04:23:14.140259+0000","last_peered":"2023-11-25T04:23:14.140259+0000","last_clean":"2023-11-25T04:23:05.103367+0000","last_became_active":"2023-11-25T04:23:09.120649+0000","last_became_peered":"2023-11-25T04:23:09.120649+0000","last_unstale":"2023-11-25T04:23:14.140259+0000","last_undegraded":"2023-11-25T04:23:14.140259+0000","last_fullsized":"2023-11-25T04:23:09.117329+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:27:17.001116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+undersized","last_fresh":"2023-11-25T04:23:14.142428+0000","last_change":"2023-11-25T04:23:09.124985+0000","last_active":"2023-11-25T04:23:14.142428+0000","last_peered":"2023-11-25T04:23:14.142428+0000","last_clean":"2023-11-25T04:23:05.103220+0000","last_became_active":"2023-11-25T04:23:09.124985+0000","last_became_peered":"2023-11-25T04:23:09.124985+0000","last_unstale":"2023-11-25T04:23:14.142428+0000","last_undegraded":"2023-11-25T04:23:14.142428+0000","last_fullsized":"2023-11-25T04:23:09.118677+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:18:18.648065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+undersized","last_fresh":"2023-11-25T04:23:14.140348+0000","last_change":"2023-11-25T04:23:09.118988+0000","last_active":"2023-11-25T04:23:14.140348+0000","last_peered":"2023-11-25T04:23:14.140348+0000","last_clean":"2023-11-25T04:23:05.103081+0000","last_became_active":"2023-11-25T04:23:09.118988+0000","last_became_peered":"2023-11-25T04:23:09.118988+0000","last_unstale":"2023-11-25T04:23:14.140348+0000","last_undegraded":"2023-11-25T04:23:14.140348+0000","last_fullsized":"2023-11-25T04:23:09.115918+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:52:57.375894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.10","version":"18'8","reported_seq":22,"reported_epoch":24,"state":"active+undersized+degraded","last_fresh":"2023-11-25T04:23:14.142223+0000","last_change":"2023-11-25T04:23:09.124549+0000","last_active":"2023-11-25T04:23:14.142223+0000","last_peered":"2023-11-25T04:23:14.142223+0000","last_clean":"2023-11-25T04:22:47.044383+0000","last_became_active":"2023-11-25T04:23:09.124549+0000","last_became_peered":"2023-11-25T04:23:09.124549+0000","last_unstale":"2023-11-25T04:23:14.142223+0000","last_undegraded":"2023-11-25T04:23:09.117830+0000","last_fullsized":"2023-11-25T04:23:09.117698+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:23:14.905353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":4}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.f","version":"18'1","reported_seq":20,"reported_epoch":24,"state":"active+undersized+degraded","last_fresh":"2023-11-25T04:23:14.143451+0000","last_change":"2023-11-25T04:23:09.125254+0000","last_active":"2023-11-25T04:23:14.143451+0000","last_peered":"2023-11-25T04:23:14.143451+0000","last_clean":"2023-11-25T04:22:47.029844+0000","last_became_active":"2023-11-25T04:23:09.125254+0000","last_became_peered":"2023-11-25T04:23:09.125254+0000","last_unstale":"2023-11-25T04:23:14.143451+0000","last_undegraded":"2023-11-25T04:23:09.118862+0000","last_fullsized":"2023-11-25T04:23:09.118666+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:37:33.204078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+undersized","last_fresh":"2023-11-25T04:23:14.142285+0000","last_change":"2023-11-25T04:23:09.125162+0000","last_active":"2023-11-25T04:23:14.142285+0000","last_peered":"2023-11-25T04:23:14.142285+0000","last_clean":"2023-11-25T04:23:05.103041+0000","last_became_active":"2023-11-25T04:23:09.125162+0000","last_became_peered":"2023-11-25T04:23:09.125162+0000","last_unstale":"2023-11-25T04:23:14.142285+0000","last_undegraded":"2023-11-25T04:23:14.142285+0000","last_fullsized":"2023-11-25T04:23:09.117813+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:22:13.056315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"18'2","reported_seq":24,"reported_epoch":24,"state":"active+undersized+degraded","last_fresh":"2023-11-25T04:23:14.143433+0000","last_change":"2023-11-25T04:23:09.125430+0000","last_active":"2023-11-25T04:23:14.143433+0000","last_peered":"2023-11-25T04:23:14.143433+0000","last_clean":"2023-11-25T04:23:05.103746+0000","last_became_active":"2023-11-25T04:23:09.125430+0000","last_became_peered":"2023-11-25T04:23:09.125430+0000","last_unstale":"2023-11-25T04:23:14.143433+0000","last_undegraded":"2023-11-25T04:23:09.119463+0000","last_fullsized":"2023-11-25T04:23:09.119370+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:46:13.177177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-25T04:23:14.142322+0000","last_change":"2023-11-25T04:22:47.039685+0000","last_active":"2023-11-25T04:23:14.142322+0000","last_peered":"2023-11-25T04:23:14.142322+0000","last_clean":"2023-11-25T04:23:14.142322+0000","last_became_active":"2023-11-25T04:22:47.039296+0000","last_became_peered":"2023-11-25T04:22:47.039296+0000","last_unstale":"2023-11-25T04:23:14.142322+0000","last_undegraded":"2023-11-25T04:23:14.142322+0000","last_fullsized":"2023-11-25T04:23:14.142322+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:11:40.532140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"18'1","reported_seq":20,"reported_epoch":24,"state":"active+undersized+degraded","last_fresh":"2023-11-25T04:23:14.143006+0000","last_change":"2023-11-25T04:23:09.125688+0000","last_active":"2023-11-25T04:23:14.143006+0000","last_peered":"2023-11-25T04:23:14.143006+0000","last_clean":"2023-11-25T04:22:47.032733+0000","last_became_active":"2023-11-25T04:23:09.125688+0000","last_became_peered":"2023-11-25T04:23:09.125688+0000","last_unstale":"2023-11-25T04:23:14.143006+0000","last_undegraded":"2023-11-25T04:23:09.119113+0000","last_fullsized":"2023-11-25T04:23:09.118834+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:27:54.544576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+undersized","last_fresh":"2023-11-25T04:23:14.140363+0000","last_change":"2023-11-25T04:23:09.119916+0000","last_active":"2023-11-25T04:23:14.140363+0000","last_peered":"2023-11-25T04:23:14.140363+0000","last_clean":"2023-11-25T04:22:46.014230+0000","last_became_active":"2023-11-25T04:23:09.119916+0000","last_became_peered":"2023-11-25T04:23:09.119916+0000","last_unstale":"2023-11-25T04:23:14.140363+0000","last_undegraded":"2023-11-25T04:23:14.140363+0000","last_fullsized":"2023-11-25T04:23:09.116313+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:53:08.463903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+undersized","last_fresh":"2023-11-25T04:23:14.140079+0000","last_change":"2023-11-25T04:23:09.119787+0000","last_active":"2023-11-25T04:23:14.140079+0000","last_peered":"2023-11-25T04:23:14.140079+0000","last_clean":"2023-11-25T04:22:45.004766+0000","last_became_active":"2023-11-25T04:23:09.119787+0000","last_became_peered":"2023-11-25T04:23:09.119787+0000","last_unstale":"2023-11-25T04:23:14.140079+0000","last_undegraded":"2023-11-25T04:23:14.140079+0000","last_fullsized":"2023-11-25T04:23:09.115982+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:36:02.391215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+undersized","last_fresh":"2023-11-25T04:23:14.139965+0000","last_change":"2023-11-25T04:23:09.119219+0000","last_active":"2023-11-25T04:23:14.139965+0000","last_peered":"2023-11-25T04:23:14.139965+0000","last_clean":"2023-11-25T04:23:05.103027+0000","last_became_active":"2023-11-25T04:23:09.119219+0000","last_became_peered":"2023-11-25T04:23:09.119219+0000","last_unstale":"2023-11-25T04:23:14.139965+0000","last_undegraded":"2023-11-25T04:23:14.139965+0000","last_fullsized":"2023-11-25T04:23:09.115853+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:18:19.657529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"18'2","reported_seq":24,"reported_epoch":24,"state":"active+undersized+degraded","last_fresh":"2023-11-25T04:23:14.140124+0000","last_change":"2023-11-25T04:23:09.119814+0000","last_active":"2023-11-25T04:23:14.140124+0000","last_peered":"2023-11-25T04:23:14.140124+0000","last_clean":"2023-11-25T04:23:05.103312+0000","last_became_active":"2023-11-25T04:23:09.119814+0000","last_became_peered":"2023-11-25T04:23:09.119814+0000","last_unstale":"2023-11-25T04:23:14.140124+0000","last_undegraded":"2023-11-25T04:23:09.117077+0000","last_fullsized":"2023-11-25T04:23:09.116963+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:06:14.986918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+undersized","last_fresh":"2023-11-25T04:23:14.139893+0000","last_change":"2023-11-25T04:23:09.118127+0000","last_active":"2023-11-25T04:23:14.139893+0000","last_peered":"2023-11-25T04:23:14.139893+0000","last_clean":"2023-11-25T04:22:46.014230+0000","last_became_active":"2023-11-25T04:23:09.118127+0000","last_became_peered":"2023-11-25T04:23:09.118127+0000","last_unstale":"2023-11-25T04:23:14.139893+0000","last_undegraded":"2023-11-25T04:23:14.139893+0000","last_fullsized":"2023-11-25T04:23:09.115225+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:45:13.331708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+undersized","last_fresh":"2023-11-25T04:23:14.139262+0000","last_change":"2023-11-25T04:23:09.118421+0000","last_active":"2023-11-25T04:23:14.139262+0000","last_peered":"2023-11-25T04:23:14.139262+0000","last_clean":"2023-11-25T04:22:45.004766+0000","last_became_active":"2023-11-25T04:23:09.118421+0000","last_became_peered":"2023-11-25T04:23:09.118421+0000","last_unstale":"2023-11-25T04:23:14.139262+0000","last_undegraded":"2023-11-25T04:23:14.139262+0000","last_fullsized":"2023-11-25T04:23:09.115042+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:22:12.046851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+undersized","last_fresh":"2023-11-25T04:23:14.143339+0000","last_change":"2023-11-25T04:23:09.125984+0000","last_active":"2023-11-25T04:23:14.143339+0000","last_peered":"2023-11-25T04:23:14.143339+0000","last_clean":"2023-11-25T04:22:46.014230+0000","last_became_active":"2023-11-25T04:23:09.125984+0000","last_became_peered":"2023-11-25T04:23:09.125984+0000","last_unstale":"2023-11-25T04:23:14.143339+0000","last_undegraded":"2023-11-25T04:23:14.143339+0000","last_fullsized":"2023-11-25T04:23:09.119402+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:28:35.822713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+undersized","last_fresh":"2023-11-25T04:23:14.142514+0000","last_change":"2023-11-25T04:23:09.124974+0000","last_active":"2023-11-25T04:23:14.142514+0000","last_peered":"2023-11-25T04:23:14.142514+0000","last_clean":"2023-11-25T04:23:05.103360+0000","last_became_active":"2023-11-25T04:23:09.124974+0000","last_became_peered":"2023-11-25T04:23:09.124974+0000","last_unstale":"2023-11-25T04:23:14.142514+0000","last_undegraded":"2023-11-25T04:23:14.142514+0000","last_fullsized":"2023-11-25T04:23:09.118269+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:35:18.298463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+undersized","last_fresh":"2023-11-25T04:23:14.142960+0000","last_change":"2023-11-25T04:23:09.125090+0000","last_active":"2023-11-25T04:23:14.142960+0000","last_peered":"2023-11-25T04:23:14.142960+0000","last_clean":"2023-11-25T04:22:46.014230+0000","last_became_active":"2023-11-25T04:23:09.125090+0000","last_became_peered":"2023-11-25T04:23:09.125090+0000","last_unstale":"2023-11-25T04:23:14.142960+0000","last_undegraded":"2023-11-25T04:23:14.142960+0000","last_fullsized":"2023-11-25T04:23:09.118350+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:09:57.875827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-25T04:23:14.142359+0000","last_change":"2023-11-25T04:22:46.034509+0000","last_active":"2023-11-25T04:23:14.142359+0000","last_peered":"2023-11-25T04:23:14.142359+0000","last_clean":"2023-11-25T04:23:14.142359+0000","last_became_active":"2023-11-25T04:22:46.034158+0000","last_became_peered":"2023-11-25T04:22:46.034158+0000","last_unstale":"2023-11-25T04:23:14.142359+0000","last_undegraded":"2023-11-25T04:23:14.142359+0000","last_fullsized":"2023-11-25T04:23:14.142359+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:26:25.338117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-25T04:23:14.139972+0000","last_change":"2023-11-25T04:22:47.038800+0000","last_active":"2023-11-25T04:23:14.139972+0000","last_peered":"2023-11-25T04:23:14.139972+0000","last_clean":"2023-11-25T04:23:14.139972+0000","last_became_active":"2023-11-25T04:22:47.038081+0000","last_became_peered":"2023-11-25T04:22:47.038081+0000","last_unstale":"2023-11-25T04:23:14.139972+0000","last_undegraded":"2023-11-25T04:23:14.139972+0000","last_fullsized":"2023-11-25T04:23:14.139972+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:45:40.065146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"12'157","reported_seq":215,"reported_epoch":24,"state":"active+undersized+degraded","last_fresh":"2023-11-25T04:23:14.142741+0000","last_change":"2023-11-25T04:23:09.125979+0000","last_active":"2023-11-25T04:23:14.142741+0000","last_peered":"2023-11-25T04:23:14.142741+0000","last_clean":"2023-11-25T04:22:38.308371+0000","last_became_active":"2023-11-25T04:23:09.125979+0000","last_became_peered":"2023-11-25T04:23:09.125979+0000","last_unstale":"2023-11-25T04:23:14.142741+0000","last_undegraded":"2023-11-25T04:23:09.118707+0000","last_fullsized":"2023-11-25T04:23:09.118610+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:35.919850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:35.919850+0000","last_clean_scrub_stamp":"2023-11-25T04:22:35.919850+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:44:30.622368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+undersized","last_fresh":"2023-11-25T04:23:14.140393+0000","last_change":"2023-11-25T04:23:09.120680+0000","last_active":"2023-11-25T04:23:14.140393+0000","last_peered":"2023-11-25T04:23:14.140393+0000","last_clean":"2023-11-25T04:22:40.987592+0000","last_became_active":"2023-11-25T04:23:09.120680+0000","last_became_peered":"2023-11-25T04:23:09.120680+0000","last_unstale":"2023-11-25T04:23:14.140393+0000","last_undegraded":"2023-11-25T04:23:14.140393+0000","last_fullsized":"2023-11-25T04:23:09.117272+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:02:21.801430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-25T04:23:14.139812+0000","last_change":"2023-11-25T04:22:46.037044+0000","last_active":"2023-11-25T04:23:14.139812+0000","last_peered":"2023-11-25T04:23:14.139812+0000","last_clean":"2023-11-25T04:23:14.139812+0000","last_became_active":"2023-11-25T04:22:46.035916+0000","last_became_peered":"2023-11-25T04:22:46.035916+0000","last_unstale":"2023-11-25T04:23:14.139812+0000","last_undegraded":"2023-11-25T04:23:14.139812+0000","last_fullsized":"2023-11-25T04:23:14.139812+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:38:20.501102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+undersized","last_fresh":"2023-11-25T04:23:14.142944+0000","last_change":"2023-11-25T04:23:09.125196+0000","last_active":"2023-11-25T04:23:14.142944+0000","last_peered":"2023-11-25T04:23:14.142944+0000","last_clean":"2023-11-25T04:22:46.014230+0000","last_became_active":"2023-11-25T04:23:09.125196+0000","last_became_peered":"2023-11-25T04:23:09.125196+0000","last_unstale":"2023-11-25T04:23:14.142944+0000","last_undegraded":"2023-11-25T04:23:14.142944+0000","last_fullsized":"2023-11-25T04:23:09.118556+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T16:22:28.651339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+undersized","last_fresh":"2023-11-25T04:23:14.142460+0000","last_change":"2023-11-25T04:23:09.125555+0000","last_active":"2023-11-25T04:23:14.142460+0000","last_peered":"2023-11-25T04:23:14.142460+0000","last_clean":"2023-11-25T04:22:40.987592+0000","last_became_active":"2023-11-25T04:23:09.125555+0000","last_became_peered":"2023-11-25T04:23:09.125555+0000","last_unstale":"2023-11-25T04:23:14.142460+0000","last_undegraded":"2023-11-25T04:23:14.142460+0000","last_fullsized":"2023-11-25T04:23:09.118210+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:45:08.305070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+undersized","last_fresh":"2023-11-25T04:23:14.139456+0000","last_change":"2023-11-25T04:23:09.119629+0000","last_active":"2023-11-25T04:23:14.139456+0000","last_peered":"2023-11-25T04:23:14.139456+0000","last_clean":"2023-11-25T04:23:05.102781+0000","last_became_active":"2023-11-25T04:23:09.119629+0000","last_became_peered":"2023-11-25T04:23:09.119629+0000","last_unstale":"2023-11-25T04:23:14.139456+0000","last_undegraded":"2023-11-25T04:23:14.139456+0000","last_fullsized":"2023-11-25T04:23:09.116426+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:32:12.858964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-25T04:23:14.143168+0000","last_change":"2023-11-25T04:22:47.039583+0000","last_active":"2023-11-25T04:23:14.143168+0000","last_peered":"2023-11-25T04:23:14.143168+0000","last_clean":"2023-11-25T04:23:14.143168+0000","last_became_active":"2023-11-25T04:22:47.039197+0000","last_became_peered":"2023-11-25T04:22:47.039197+0000","last_unstale":"2023-11-25T04:23:14.143168+0000","last_undegraded":"2023-11-25T04:23:14.143168+0000","last_fullsized":"2023-11-25T04:23:14.143168+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:37:30.959147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-25T04:23:14.139626+0000","last_change":"2023-11-25T04:22:42.000382+0000","last_active":"2023-11-25T04:23:14.139626+0000","last_peered":"2023-11-25T04:23:14.139626+0000","last_clean":"2023-11-25T04:23:14.139626+0000","last_became_active":"2023-11-25T04:22:42.000041+0000","last_became_peered":"2023-11-25T04:22:42.000041+0000","last_unstale":"2023-11-25T04:23:14.139626+0000","last_undegraded":"2023-11-25T04:23:14.139626+0000","last_fullsized":"2023-11-25T04:23:14.139626+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:08:19.655864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+undersized","last_fresh":"2023-11-25T04:23:14.142412+0000","last_change":"2023-11-25T04:23:09.126296+0000","last_active":"2023-11-25T04:23:14.142412+0000","last_peered":"2023-11-25T04:23:14.142412+0000","last_clean":"2023-11-25T04:23:05.103139+0000","last_became_active":"2023-11-25T04:23:09.126296+0000","last_became_peered":"2023-11-25T04:23:09.126296+0000","last_unstale":"2023-11-25T04:23:14.142412+0000","last_undegraded":"2023-11-25T04:23:14.142412+0000","last_fullsized":"2023-11-25T04:23:09.118067+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:36:02.391215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-25T04:23:14.140084+0000","last_change":"2023-11-25T04:22:47.038454+0000","last_active":"2023-11-25T04:23:14.140084+0000","last_peered":"2023-11-25T04:23:14.140084+0000","last_clean":"2023-11-25T04:23:14.140084+0000","last_became_active":"2023-11-25T04:22:47.038046+0000","last_became_peered":"2023-11-25T04:22:47.038046+0000","last_unstale":"2023-11-25T04:23:14.140084+0000","last_undegraded":"2023-11-25T04:23:14.140084+0000","last_fullsized":"2023-11-25T04:23:14.140084+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:38:38.839941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+undersized","last_fresh":"2023-11-25T04:23:14.140112+0000","last_change":"2023-11-25T04:23:09.120595+0000","last_active":"2023-11-25T04:23:14.140112+0000","last_peered":"2023-11-25T04:23:14.140112+0000","last_clean":"2023-11-25T04:22:40.987592+0000","last_became_active":"2023-11-25T04:23:09.120595+0000","last_became_peered":"2023-11-25T04:23:09.120595+0000","last_unstale":"2023-11-25T04:23:14.140112+0000","last_undegraded":"2023-11-25T04:23:14.140112+0000","last_fullsized":"2023-11-25T04:23:09.116969+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:11:11.495643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+undersized","last_fresh":"2023-11-25T04:23:14.139464+0000","last_change":"2023-11-25T04:23:09.118751+0000","last_active":"2023-11-25T04:23:14.139464+0000","last_peered":"2023-11-25T04:23:14.139464+0000","last_clean":"2023-11-25T04:22:45.004766+0000","last_became_active":"2023-11-25T04:23:09.118751+0000","last_became_peered":"2023-11-25T04:23:09.118751+0000","last_unstale":"2023-11-25T04:23:14.139464+0000","last_undegraded":"2023-11-25T04:23:14.139464+0000","last_fullsized":"2023-11-25T04:23:09.115453+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:48:06.042569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-25T04:23:14.140192+0000","last_change":"2023-11-25T04:22:47.037355+0000","last_active":"2023-11-25T04:23:14.140192+0000","last_peered":"2023-11-25T04:23:14.140192+0000","last_clean":"2023-11-25T04:23:14.140192+0000","last_became_active":"2023-11-25T04:22:47.036924+0000","last_became_peered":"2023-11-25T04:22:47.036924+0000","last_unstale":"2023-11-25T04:23:14.140192+0000","last_undegraded":"2023-11-25T04:23:14.140192+0000","last_fullsized":"2023-11-25T04:23:14.140192+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:15:39.511043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+undersized","last_fresh":"2023-11-25T04:23:14.139971+0000","last_change":"2023-11-25T04:23:09.118854+0000","last_active":"2023-11-25T04:23:14.139971+0000","last_peered":"2023-11-25T04:23:14.139971+0000","last_clean":"2023-11-25T04:22:45.004766+0000","last_became_active":"2023-11-25T04:23:09.118854+0000","last_became_peered":"2023-11-25T04:23:09.118854+0000","last_unstale":"2023-11-25T04:23:14.139971+0000","last_undegraded":"2023-11-25T04:23:14.139971+0000","last_fullsized":"2023-11-25T04:23:09.115675+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:44:39.707285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-25T04:23:14.140405+0000","last_change":"2023-11-25T04:22:42.000204+0000","last_active":"2023-11-25T04:23:14.140405+0000","last_peered":"2023-11-25T04:23:14.140405+0000","last_clean":"2023-11-25T04:23:14.140405+0000","last_became_active":"2023-11-25T04:22:41.999915+0000","last_became_peered":"2023-11-25T04:22:41.999915+0000","last_unstale":"2023-11-25T04:23:14.140405+0000","last_undegraded":"2023-11-25T04:23:14.140405+0000","last_fullsized":"2023-11-25T04:23:14.140405+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:07:32.457905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+undersized","last_fresh":"2023-11-25T04:23:14.142566+0000","last_change":"2023-11-25T04:23:09.126156+0000","last_active":"2023-11-25T04:23:14.142566+0000","last_peered":"2023-11-25T04:23:14.142566+0000","last_clean":"2023-11-25T04:23:05.103197+0000","last_became_active":"2023-11-25T04:23:09.126156+0000","last_became_peered":"2023-11-25T04:23:09.126156+0000","last_unstale":"2023-11-25T04:23:14.142566+0000","last_undegraded":"2023-11-25T04:23:14.142566+0000","last_fullsized":"2023-11-25T04:23:09.118560+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:01:49.725330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-25T04:23:14.140269+0000","last_change":"2023-11-25T04:22:46.035885+0000","last_active":"2023-11-25T04:23:14.140269+0000","last_peered":"2023-11-25T04:23:14.140269+0000","last_clean":"2023-11-25T04:23:14.140269+0000","last_became_active":"2023-11-25T04:22:46.035405+0000","last_became_peered":"2023-11-25T04:22:46.035405+0000","last_unstale":"2023-11-25T04:23:14.140269+0000","last_undegraded":"2023-11-25T04:23:14.140269+0000","last_fullsized":"2023-11-25T04:23:14.140269+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:53:47.398746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-25T04:23:14.143533+0000","last_change":"2023-11-25T04:22:41.999067+0000","last_active":"2023-11-25T04:23:14.143533+0000","last_peered":"2023-11-25T04:23:14.143533+0000","last_clean":"2023-11-25T04:23:14.143533+0000","last_became_active":"2023-11-25T04:22:41.998798+0000","last_became_peered":"2023-11-25T04:22:41.998798+0000","last_unstale":"2023-11-25T04:23:14.143533+0000","last_undegraded":"2023-11-25T04:23:14.143533+0000","last_fullsized":"2023-11-25T04:23:14.143533+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:00:04.266051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+undersized","last_fresh":"2023-11-25T04:23:14.142663+0000","last_change":"2023-11-25T04:23:09.124621+0000","last_active":"2023-11-25T04:23:14.142663+0000","last_peered":"2023-11-25T04:23:14.142663+0000","last_clean":"2023-11-25T04:23:05.103402+0000","last_became_active":"2023-11-25T04:23:09.124621+0000","last_became_peered":"2023-11-25T04:23:09.124621+0000","last_unstale":"2023-11-25T04:23:14.142663+0000","last_undegraded":"2023-11-25T04:23:14.142663+0000","last_fullsized":"2023-11-25T04:23:09.118235+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:47:55.336105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"18'2","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-25T04:23:14.140538+0000","last_change":"2023-11-25T04:22:46.034606+0000","last_active":"2023-11-25T04:23:14.140538+0000","last_peered":"2023-11-25T04:23:14.140538+0000","last_clean":"2023-11-25T04:23:14.140538+0000","last_became_active":"2023-11-25T04:22:46.034229+0000","last_became_peered":"2023-11-25T04:22:46.034229+0000","last_unstale":"2023-11-25T04:23:14.140538+0000","last_undegraded":"2023-11-25T04:23:14.140538+0000","last_fullsized":"2023-11-25T04:23:14.140538+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:21:42.146264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-25T04:23:14.140498+0000","last_change":"2023-11-25T04:22:42.000258+0000","last_active":"2023-11-25T04:23:14.140498+0000","last_peered":"2023-11-25T04:23:14.140498+0000","last_clean":"2023-11-25T04:23:14.140498+0000","last_became_active":"2023-11-25T04:22:41.999992+0000","last_became_peered":"2023-11-25T04:22:41.999992+0000","last_unstale":"2023-11-25T04:23:14.140498+0000","last_undegraded":"2023-11-25T04:23:14.140498+0000","last_fullsized":"2023-11-25T04:23:14.140498+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:31:13.715992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+undersized","last_fresh":"2023-11-25T04:23:14.139505+0000","last_change":"2023-11-25T04:23:09.120472+0000","last_active":"2023-11-25T04:23:14.139505+0000","last_peered":"2023-11-25T04:23:14.139505+0000","last_clean":"2023-11-25T04:23:05.102580+0000","last_became_active":"2023-11-25T04:23:09.120472+0000","last_became_peered":"2023-11-25T04:23:09.120472+0000","last_unstale":"2023-11-25T04:23:14.139505+0000","last_undegraded":"2023-11-25T04:23:14.139505+0000","last_fullsized":"2023-11-25T04:23:09.116306+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:39:22.932245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"23'3","reported_seq":25,"reported_epoch":24,"state":"active+undersized+degraded","last_fresh":"2023-11-25T04:23:14.140470+0000","last_change":"2023-11-25T04:23:09.120778+0000","last_active":"2023-11-25T04:23:14.140470+0000","last_peered":"2023-11-25T04:23:14.140470+0000","last_clean":"2023-11-25T04:23:07.106266+0000","last_became_active":"2023-11-25T04:23:09.120778+0000","last_became_peered":"2023-11-25T04:23:09.120778+0000","last_unstale":"2023-11-25T04:23:14.140470+0000","last_undegraded":"2023-11-25T04:23:09.117422+0000","last_fullsized":"2023-11-25T04:23:09.117332+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:10:58.870120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+undersized","last_fresh":"2023-11-25T04:23:14.143670+0000","last_change":"2023-11-25T04:23:09.126295+0000","last_active":"2023-11-25T04:23:14.143670+0000","last_peered":"2023-11-25T04:23:14.143670+0000","last_clean":"2023-11-25T04:22:40.987592+0000","last_became_active":"2023-11-25T04:23:09.126295+0000","last_became_peered":"2023-11-25T04:23:09.126295+0000","last_unstale":"2023-11-25T04:23:14.143670+0000","last_undegraded":"2023-11-25T04:23:14.143670+0000","last_fullsized":"2023-11-25T04:23:09.119819+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:11:11.495643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+undersized","last_fresh":"2023-11-25T04:23:14.142382+0000","last_change":"2023-11-25T04:23:09.124599+0000","last_active":"2023-11-25T04:23:14.142382+0000","last_peered":"2023-11-25T04:23:14.142382+0000","last_clean":"2023-11-25T04:23:05.103105+0000","last_became_active":"2023-11-25T04:23:09.124599+0000","last_became_peered":"2023-11-25T04:23:09.124599+0000","last_unstale":"2023-11-25T04:23:14.142382+0000","last_undegraded":"2023-11-25T04:23:14.142382+0000","last_fullsized":"2023-11-25T04:23:09.117911+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:38:46.493065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+undersized","last_fresh":"2023-11-25T04:23:14.142582+0000","last_change":"2023-11-25T04:23:09.125471+0000","last_active":"2023-11-25T04:23:14.142582+0000","last_peered":"2023-11-25T04:23:14.142582+0000","last_clean":"2023-11-25T04:22:46.014230+0000","last_became_active":"2023-11-25T04:23:09.125471+0000","last_became_peered":"2023-11-25T04:23:09.125471+0000","last_unstale":"2023-11-25T04:23:14.142582+0000","last_undegraded":"2023-11-25T04:23:14.142582+0000","last_fullsized":"2023-11-25T04:23:09.118929+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:50:48.345615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"23'10","reported_seq":32,"reported_epoch":24,"state":"active+undersized+degraded","last_fresh":"2023-11-25T04:23:14.143064+0000","last_change":"2023-11-25T04:23:09.125515+0000","last_active":"2023-11-25T04:23:14.143064+0000","last_peered":"2023-11-25T04:23:14.143064+0000","last_clean":"2023-11-25T04:23:05.666669+0000","last_became_active":"2023-11-25T04:23:09.125515+0000","last_became_peered":"2023-11-25T04:23:09.125515+0000","last_unstale":"2023-11-25T04:23:14.143064+0000","last_undegraded":"2023-11-25T04:23:09.119239+0000","last_fullsized":"2023-11-25T04:23:09.119037+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:53:07.454439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6552,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":4}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-25T04:23:14.139404+0000","last_change":"2023-11-25T04:22:47.037181+0000","last_active":"2023-11-25T04:23:14.139404+0000","last_peered":"2023-11-25T04:23:14.139404+0000","last_clean":"2023-11-25T04:23:14.139404+0000","last_became_active":"2023-11-25T04:22:47.036821+0000","last_became_peered":"2023-11-25T04:22:47.036821+0000","last_unstale":"2023-11-25T04:23:14.139404+0000","last_undegraded":"2023-11-25T04:23:14.139404+0000","last_fullsized":"2023-11-25T04:23:14.139404+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:12:51.990448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"18'3","reported_seq":25,"reported_epoch":24,"state":"active+undersized+degraded","last_fresh":"2023-11-25T04:23:14.143455+0000","last_change":"2023-11-25T04:23:09.126238+0000","last_active":"2023-11-25T04:23:14.143455+0000","last_peered":"2023-11-25T04:23:14.143455+0000","last_clean":"2023-11-25T04:23:05.103791+0000","last_became_active":"2023-11-25T04:23:09.126238+0000","last_became_peered":"2023-11-25T04:23:09.126238+0000","last_unstale":"2023-11-25T04:23:14.143455+0000","last_undegraded":"2023-11-25T04:23:09.119767+0000","last_fullsized":"2023-11-25T04:23:09.119670+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:11:31.828055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+undersized","last_fresh":"2023-11-25T04:23:14.139868+0000","last_change":"2023-11-25T04:23:09.119056+0000","last_active":"2023-11-25T04:23:14.139868+0000","last_peered":"2023-11-25T04:23:14.139868+0000","last_clean":"2023-11-25T04:22:45.004766+0000","last_became_active":"2023-11-25T04:23:09.119056+0000","last_became_peered":"2023-11-25T04:23:09.119056+0000","last_unstale":"2023-11-25T04:23:14.139868+0000","last_undegraded":"2023-11-25T04:23:14.139868+0000","last_fullsized":"2023-11-25T04:23:09.116275+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:27:54.544576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+undersized","last_fresh":"2023-11-25T04:23:14.139305+0000","last_change":"2023-11-25T04:23:09.119066+0000","last_active":"2023-11-25T04:23:14.139305+0000","last_peered":"2023-11-25T04:23:14.139305+0000","last_clean":"2023-11-25T04:23:05.102580+0000","last_became_active":"2023-11-25T04:23:09.119066+0000","last_became_peered":"2023-11-25T04:23:09.119066+0000","last_unstale":"2023-11-25T04:23:14.139305+0000","last_undegraded":"2023-11-25T04:23:14.139305+0000","last_fullsized":"2023-11-25T04:23:09.115417+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:09:57.875827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+undersized","last_fresh":"2023-11-25T04:23:14.140469+0000","last_change":"2023-11-25T04:23:09.120739+0000","last_active":"2023-11-25T04:23:14.140469+0000","last_peered":"2023-11-25T04:23:14.140469+0000","last_clean":"2023-11-25T04:23:05.103304+0000","last_became_active":"2023-11-25T04:23:09.120739+0000","last_became_peered":"2023-11-25T04:23:09.120739+0000","last_unstale":"2023-11-25T04:23:14.140469+0000","last_undegraded":"2023-11-25T04:23:14.140469+0000","last_fullsized":"2023-11-25T04:23:09.116820+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:11:01.927467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-25T04:23:14.139340+0000","last_change":"2023-11-25T04:22:47.037790+0000","last_active":"2023-11-25T04:23:14.139340+0000","last_peered":"2023-11-25T04:23:14.139340+0000","last_clean":"2023-11-25T04:23:14.139340+0000","last_became_active":"2023-11-25T04:22:47.037149+0000","last_became_peered":"2023-11-25T04:22:47.037149+0000","last_unstale":"2023-11-25T04:23:14.139340+0000","last_undegraded":"2023-11-25T04:23:14.139340+0000","last_fullsized":"2023-11-25T04:23:14.139340+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:31:54.857270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+undersized","last_fresh":"2023-11-25T04:23:14.143594+0000","last_change":"2023-11-25T04:23:09.125729+0000","last_active":"2023-11-25T04:23:14.143594+0000","last_peered":"2023-11-25T04:23:14.143594+0000","last_clean":"2023-11-25T04:23:05.103821+0000","last_became_active":"2023-11-25T04:23:09.125729+0000","last_became_peered":"2023-11-25T04:23:09.125729+0000","last_unstale":"2023-11-25T04:23:14.143594+0000","last_undegraded":"2023-11-25T04:23:14.143594+0000","last_fullsized":"2023-11-25T04:23:09.119552+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:20:27.336993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-25T04:23:14.139844+0000","last_change":"2023-11-25T04:22:46.036085+0000","last_active":"2023-11-25T04:23:14.139844+0000","last_peered":"2023-11-25T04:23:14.139844+0000","last_clean":"2023-11-25T04:23:14.139844+0000","last_became_active":"2023-11-25T04:22:46.035495+0000","last_became_peered":"2023-11-25T04:22:46.035495+0000","last_unstale":"2023-11-25T04:23:14.139844+0000","last_undegraded":"2023-11-25T04:23:14.139844+0000","last_fullsized":"2023-11-25T04:23:14.139844+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:14:56.735365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+undersized","last_fresh":"2023-11-25T04:23:14.140119+0000","last_change":"2023-11-25T04:23:09.119712+0000","last_active":"2023-11-25T04:23:14.140119+0000","last_peered":"2023-11-25T04:23:14.140119+0000","last_clean":"2023-11-25T04:23:05.103237+0000","last_became_active":"2023-11-25T04:23:09.119712+0000","last_became_peered":"2023-11-25T04:23:09.119712+0000","last_unstale":"2023-11-25T04:23:14.140119+0000","last_undegraded":"2023-11-25T04:23:14.140119+0000","last_fullsized":"2023-11-25T04:23:09.116851+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:20:27.336993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+undersized","last_fresh":"2023-11-25T04:23:14.139575+0000","last_change":"2023-11-25T04:23:09.119884+0000","last_active":"2023-11-25T04:23:14.139575+0000","last_peered":"2023-11-25T04:23:14.139575+0000","last_clean":"2023-11-25T04:23:05.102683+0000","last_became_active":"2023-11-25T04:23:09.119884+0000","last_became_peered":"2023-11-25T04:23:09.119884+0000","last_unstale":"2023-11-25T04:23:14.139575+0000","last_undegraded":"2023-11-25T04:23:14.139575+0000","last_fullsized":"2023-11-25T04:23:09.116145+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:34:11.473777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+undersized","last_fresh":"2023-11-25T04:23:14.143361+0000","last_change":"2023-11-25T04:23:09.125722+0000","last_active":"2023-11-25T04:23:14.143361+0000","last_peered":"2023-11-25T04:23:14.143361+0000","last_clean":"2023-11-25T04:23:05.103652+0000","last_became_active":"2023-11-25T04:23:09.125722+0000","last_became_peered":"2023-11-25T04:23:09.125722+0000","last_unstale":"2023-11-25T04:23:14.143361+0000","last_undegraded":"2023-11-25T04:23:14.143361+0000","last_fullsized":"2023-11-25T04:23:09.119143+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:39:22.932245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1a","version":"18'1","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-25T04:23:14.142866+0000","last_change":"2023-11-25T04:22:46.034340+0000","last_active":"2023-11-25T04:23:14.142866+0000","last_peered":"2023-11-25T04:23:14.142866+0000","last_clean":"2023-11-25T04:23:14.142866+0000","last_became_active":"2023-11-25T04:22:46.033944+0000","last_became_peered":"2023-11-25T04:22:46.033944+0000","last_unstale":"2023-11-25T04:23:14.142866+0000","last_undegraded":"2023-11-25T04:23:14.142866+0000","last_fullsized":"2023-11-25T04:23:14.142866+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:59:30.291004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-25T04:23:14.143104+0000","last_change":"2023-11-25T04:22:47.039547+0000","last_active":"2023-11-25T04:23:14.143104+0000","last_peered":"2023-11-25T04:23:14.143104+0000","last_clean":"2023-11-25T04:23:14.143104+0000","last_became_active":"2023-11-25T04:22:47.039218+0000","last_became_peered":"2023-11-25T04:22:47.039218+0000","last_unstale":"2023-11-25T04:23:14.143104+0000","last_undegraded":"2023-11-25T04:23:14.143104+0000","last_fullsized":"2023-11-25T04:23:14.143104+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:44:48.553972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"18'1","reported_seq":23,"reported_epoch":24,"state":"active+undersized","last_fresh":"2023-11-25T04:23:14.143199+0000","last_change":"2023-11-25T04:23:09.125042+0000","last_active":"2023-11-25T04:23:14.143199+0000","last_peered":"2023-11-25T04:23:14.143199+0000","last_clean":"2023-11-25T04:23:05.103542+0000","last_became_active":"2023-11-25T04:23:09.125042+0000","last_became_peered":"2023-11-25T04:23:09.125042+0000","last_unstale":"2023-11-25T04:23:14.143199+0000","last_undegraded":"2023-11-25T04:23:14.143199+0000","last_fullsized":"2023-11-25T04:23:09.118589+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:55:24.597149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+undersized","last_fresh":"2023-11-25T04:23:14.139450+0000","last_change":"2023-11-25T04:23:09.118083+0000","last_active":"2023-11-25T04:23:14.139450+0000","last_peered":"2023-11-25T04:23:14.139450+0000","last_clean":"2023-11-25T04:22:46.014230+0000","last_became_active":"2023-11-25T04:23:09.118083+0000","last_became_peered":"2023-11-25T04:23:09.118083+0000","last_unstale":"2023-11-25T04:23:14.139450+0000","last_undegraded":"2023-11-25T04:23:14.139450+0000","last_fullsized":"2023-11-25T04:23:09.115133+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:38:46.493065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"18'2","reported_seq":20,"reported_epoch":24,"state":"active+undersized+degraded","last_fresh":"2023-11-25T04:23:14.140547+0000","last_change":"2023-11-25T04:23:09.119241+0000","last_active":"2023-11-25T04:23:14.140547+0000","last_peered":"2023-11-25T04:23:14.140547+0000","last_clean":"2023-11-25T04:22:47.030661+0000","last_became_active":"2023-11-25T04:23:09.119241+0000","last_became_peered":"2023-11-25T04:23:09.119241+0000","last_unstale":"2023-11-25T04:23:14.140547+0000","last_undegraded":"2023-11-25T04:23:09.115765+0000","last_fullsized":"2023-11-25T04:23:09.115629+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:55:24.597149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+undersized","last_fresh":"2023-11-25T04:23:14.139533+0000","last_change":"2023-11-25T04:23:09.118990+0000","last_active":"2023-11-25T04:23:14.139533+0000","last_peered":"2023-11-25T04:23:14.139533+0000","last_clean":"2023-11-25T04:22:46.014230+0000","last_became_active":"2023-11-25T04:23:09.118990+0000","last_became_peered":"2023-11-25T04:23:09.118990+0000","last_unstale":"2023-11-25T04:23:14.139533+0000","last_undegraded":"2023-11-25T04:23:14.139533+0000","last_fullsized":"2023-11-25T04:23:09.115673+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:01:49.725330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"18'2","reported_seq":24,"reported_epoch":24,"state":"active+undersized+degraded","last_fresh":"2023-11-25T04:23:14.143209+0000","last_change":"2023-11-25T04:23:09.125927+0000","last_active":"2023-11-25T04:23:14.143209+0000","last_peered":"2023-11-25T04:23:14.143209+0000","last_clean":"2023-11-25T04:23:05.103763+0000","last_became_active":"2023-11-25T04:23:09.125927+0000","last_became_peered":"2023-11-25T04:23:09.125927+0000","last_unstale":"2023-11-25T04:23:14.143209+0000","last_undegraded":"2023-11-25T04:23:09.119480+0000","last_fullsized":"2023-11-25T04:23:09.119373+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:32:12.858964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-25T04:23:14.142574+0000","last_change":"2023-11-25T04:22:47.036764+0000","last_active":"2023-11-25T04:23:14.142574+0000","last_peered":"2023-11-25T04:23:14.142574+0000","last_clean":"2023-11-25T04:23:14.142574+0000","last_became_active":"2023-11-25T04:22:47.036406+0000","last_became_peered":"2023-11-25T04:22:47.036406+0000","last_unstale":"2023-11-25T04:23:14.142574+0000","last_undegraded":"2023-11-25T04:23:14.142574+0000","last_fullsized":"2023-11-25T04:23:14.142574+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:09:55.357642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1e","version":"18'1","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-25T04:23:14.140262+0000","last_change":"2023-11-25T04:22:46.035659+0000","last_active":"2023-11-25T04:23:14.140262+0000","last_peered":"2023-11-25T04:23:14.140262+0000","last_clean":"2023-11-25T04:23:14.140262+0000","last_became_active":"2023-11-25T04:22:46.035113+0000","last_became_peered":"2023-11-25T04:22:46.035113+0000","last_unstale":"2023-11-25T04:23:14.140262+0000","last_undegraded":"2023-11-25T04:23:14.140262+0000","last_fullsized":"2023-11-25T04:23:14.140262+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:26:25.338117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+undersized","last_fresh":"2023-11-25T04:23:14.139224+0000","last_change":"2023-11-25T04:23:09.118740+0000","last_active":"2023-11-25T04:23:14.139224+0000","last_peered":"2023-11-25T04:23:14.139224+0000","last_clean":"2023-11-25T04:22:46.014230+0000","last_became_active":"2023-11-25T04:23:09.118740+0000","last_became_peered":"2023-11-25T04:23:09.118740+0000","last_unstale":"2023-11-25T04:23:14.139224+0000","last_undegraded":"2023-11-25T04:23:14.139224+0000","last_fullsized":"2023-11-25T04:23:09.115345+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:35:19.307926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"18'4","reported_seq":26,"reported_epoch":24,"state":"active+undersized+degraded","last_fresh":"2023-11-25T04:23:14.140189+0000","last_change":"2023-11-25T04:23:09.120709+0000","last_active":"2023-11-25T04:23:14.140189+0000","last_peered":"2023-11-25T04:23:14.140189+0000","last_clean":"2023-11-25T04:23:05.103251+0000","last_became_active":"2023-11-25T04:23:09.120709+0000","last_became_peered":"2023-11-25T04:23:09.120709+0000","last_unstale":"2023-11-25T04:23:14.140189+0000","last_undegraded":"2023-11-25T04:23:09.116883+0000","last_fullsized":"2023-11-25T04:23:09.116367+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:28:34.813249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":2}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":87,"acting":87,"num_store_stats":3},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":7350,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":9760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":45,"ondisk_log_size":45,"up":39,"acting":39,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":12,"acting":12,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":1,"acting":1,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738381,"num_pgs":72,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35556,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336284,"statfs":{"total":96636764160,"available":96600354816,"internally_reserved":0,"allocated":221184,"data_stored":85162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33100000000000002}]}]},{"osd":1,"up_from":8,"seq":34359738381,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31996,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":835584,"data_stored":681648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]}]},{"osd":0,"up_from":8,"seq":34359738377,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31952,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339888,"statfs":{"total":96636764160,"available":96604045312,"internally_reserved":0,"allocated":790528,"data_stored":650487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":6734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-25T04:23:21.840 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-backfill 3.13 3.f 3.d 1.0 3.14 3.15 3.1c 3.1d 3.1f 2023-11-25T04:23:21.852 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:21.852 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:21.861 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:23:21.873 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:21.872+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:21.967 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:21.967 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:21.974 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:21.973+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:21.976 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:23:22.076 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:22.074+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:22.123 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:22.123 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:22.132 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:23:22.177 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:22.175+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:22.181 INFO:teuthology.orchestra.run.smithi067.stderr:instructing pg(s) [1.0,3.d,3.f,3.14,3.15,3.1d] on osd.1 to force-backfill; instructing pg(s) [3.13,3.1c,3.1f] on osd.2 to force-backfill; 2023-11-25T04:23:22.181 INFO:teuthology.orchestra.run.smithi067.stderr: 2023-11-25T04:23:22.238 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:22.238 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:22.246 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:23:22.287 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:22.286+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:22.294 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2023-11-25T04:23:22.353 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:22.353 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:22.362 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:23:22.405 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:22.404+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:22.468 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:22.468 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:22.477 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:23:22.518 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:22.516+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:22.580 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:22.579+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:22.622 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:22.622 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:22.631 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:23:22.682 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:22.680+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:22.738 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:22.738 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:22.747 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:23:22.782 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:22.781+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:22.852 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:22.852 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:22.861 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:23:22.883 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:22.882+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:22.966 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:22.966 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:22.975 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:23:23.018 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:23.017+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:23.081 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:23.081 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:23.085 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:23.084+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:23.090 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:23:23.185 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:23.184+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:23.236 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:23.236 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:23.246 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:23:23.287 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:23.286+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:23.290 INFO:teuthology.orchestra.run.smithi067.stderr:noscrub is unset 2023-11-25T04:23:23.353 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:23.353 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:23.361 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:23:23.405 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:23.404+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:23.469 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:23.469 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:23.476 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-11-25T04:23:23.478 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:23:23.519 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:23.518+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:23.589 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:23.588+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:23.590 INFO:teuthology.orchestra.run.smithi067.stderr:Error ENXIO: problem getting command descriptions from osd.0 2023-11-25T04:23:23.610 INFO:teuthology.orchestra.run.smithi067.stdout:osd.1: {} 2023-11-25T04:23:23.610 INFO:teuthology.orchestra.run.smithi067.stderr:osd.1: osd_enable_op_tracker = '' 2023-11-25T04:23:23.623 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:23.623 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:23:23.623 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2023-11-25T04:23:23.624 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2023-11-25T04:23:23.624 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:23.629 INFO:teuthology.orchestra.run.smithi067.stdout:osd.2: {} 2023-11-25T04:23:23.630 INFO:teuthology.orchestra.run.smithi067.stderr:osd.2: osd_enable_op_tracker = '' 2023-11-25T04:23:23.632 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:23:23.633 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-11-25T04:23:23.633 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-11-25T04:23:23.690 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:23.689+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:23.739 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:23.739 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2023-11-25T04:23:23.739 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2023-11-25T04:23:23.739 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2023-11-25T04:23:23.739 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.739 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:155 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'109, mlcod=11'105)", 2023-11-25T04:23:23.739 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:23.739 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 45.951864843999999, 2023-11-25T04:23:23.739 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061719019999999999, 2023-11-25T04:23:23.739 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:23.740 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:23.740 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:23.740 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.740 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:23.740 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:23.740 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:23.740 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.740 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.740 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:23.740 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:23.740 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:23.740 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.740 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:23.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784884+0000", 2023-11-25T04:23:23.741 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3099999999999999e-07 2023-11-25T04:23:23.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:23.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784942+0000", 2023-11-25T04:23:23.741 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8570000000000003e-05 2023-11-25T04:23:23.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:23.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784949+0000", 2023-11-25T04:23:23.741 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0310000000000001e-06 2023-11-25T04:23:23.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.742 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:23.742 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784953+0000", 2023-11-25T04:23:23.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6320000000000001e-06 2023-11-25T04:23:23.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.742 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:23.742 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784993+0000", 2023-11-25T04:23:23.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9737000000000003e-05 2023-11-25T04:23:23.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.742 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:23.743 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785011+0000", 2023-11-25T04:23:23.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8349999999999999e-05 2023-11-25T04:23:23.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.743 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:23.743 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791024+0000", 2023-11-25T04:23:23.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006012997 2023-11-25T04:23:23.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.743 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:23.743 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791055+0000", 2023-11-25T04:23:23.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1254000000000003e-05 2023-11-25T04:23:23.743 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:23.743 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:23.744 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:23.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.744 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:156 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'110, mlcod=11'105)", 2023-11-25T04:23:23.744 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:23.744 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 45.951278795, 2023-11-25T04:23:23.744 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005616116, 2023-11-25T04:23:23.744 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:23.744 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:23.744 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:23.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.744 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:23.744 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:23.744 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:23.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:23.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:23.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:23.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:23.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785470+0000", 2023-11-25T04:23:23.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0699999999999998e-07 2023-11-25T04:23:23.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:23.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785474+0000", 2023-11-25T04:23:23.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.048e-06 2023-11-25T04:23:23.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:23.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785480+0000", 2023-11-25T04:23:23.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7729999999999996e-06 2023-11-25T04:23:23.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:23.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785483+0000", 2023-11-25T04:23:23.747 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1970000000000001e-06 2023-11-25T04:23:23.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.747 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.747 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:23.747 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787936+0000", 2023-11-25T04:23:23.747 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024529040000000001 2023-11-25T04:23:23.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.747 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.747 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:23.747 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787947+0000", 2023-11-25T04:23:23.747 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1669e-05 2023-11-25T04:23:23.748 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.748 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.748 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:23.748 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791057+0000", 2023-11-25T04:23:23.748 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031095200000000002 2023-11-25T04:23:23.748 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.748 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.748 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:23.748 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791086+0000", 2023-11-25T04:23:23.748 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8697999999999999e-05 2023-11-25T04:23:23.748 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:23.748 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:23.748 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:23.748 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.749 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.749 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:157 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'111, mlcod=11'106)", 2023-11-25T04:23:23.749 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:23.749 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 45.950686236999999, 2023-11-25T04:23:23.749 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050590280000000001, 2023-11-25T04:23:23.749 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:23.749 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:23.749 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:23.749 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.749 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:23.749 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:23.749 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:23.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:23.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:23.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:23.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:23.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786063+0000", 2023-11-25T04:23:23.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2600000000000002e-07 2023-11-25T04:23:23.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:23.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786069+0000", 2023-11-25T04:23:23.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8839999999999999e-06 2023-11-25T04:23:23.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:23.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786079+0000", 2023-11-25T04:23:23.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0124e-05 2023-11-25T04:23:23.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:23.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786085+0000", 2023-11-25T04:23:23.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6660000000000002e-06 2023-11-25T04:23:23.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.752 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:23.752 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788160+0000", 2023-11-25T04:23:23.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020747669999999999 2023-11-25T04:23:23.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.752 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:23.752 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788180+0000", 2023-11-25T04:23:23.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0208999999999999e-05 2023-11-25T04:23:23.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.752 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:23.752 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791087+0000", 2023-11-25T04:23:23.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029064630000000002 2023-11-25T04:23:23.753 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.753 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.753 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:23.753 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791121+0000", 2023-11-25T04:23:23.753 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4289000000000003e-05 2023-11-25T04:23:23.753 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:23.753 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:23.753 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:23.753 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.753 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.753 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:158 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'112, mlcod=11'106)", 2023-11-25T04:23:23.753 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:23.753 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 45.950269843999997, 2023-11-25T04:23:23.753 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046663030000000001, 2023-11-25T04:23:23.754 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:23.754 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:23.754 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:23.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.754 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:23.754 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:23.754 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:23.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.754 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:23.754 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:23.754 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:23.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:23.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786479+0000", 2023-11-25T04:23:23.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2799999999999996e-07 2023-11-25T04:23:23.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:23.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786498+0000", 2023-11-25T04:23:23.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9069e-05 2023-11-25T04:23:23.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:23.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786508+0000", 2023-11-25T04:23:23.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5810000000000003e-06 2023-11-25T04:23:23.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.756 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.756 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:23.756 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786512+0000", 2023-11-25T04:23:23.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4889999999999997e-06 2023-11-25T04:23:23.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.756 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.756 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:23.756 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788438+0000", 2023-11-25T04:23:23.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019257059999999999 2023-11-25T04:23:23.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.756 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.756 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:23.756 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788449+0000", 2023-11-25T04:23:23.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1419e-05 2023-11-25T04:23:23.757 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.757 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.757 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:23.757 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791122+0000", 2023-11-25T04:23:23.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002672822 2023-11-25T04:23:23.757 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.757 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.757 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:23.757 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791145+0000", 2023-11-25T04:23:23.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2589e-05 2023-11-25T04:23:23.757 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:23.757 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:23.758 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:23.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.758 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:206 1.0 e12/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 12'152, mlcod=12'148)", 2023-11-25T04:23:23.758 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:23.758 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 45.438501373999998, 2023-11-25T04:23:23.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046728189999999999, 2023-11-25T04:23:23.758 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:23.758 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:23.758 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:23.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:23.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:23.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:23.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:23.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:23.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:23.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:23.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298248+0000", 2023-11-25T04:23:23.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3600000000000003e-07 2023-11-25T04:23:23.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:23.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298278+0000", 2023-11-25T04:23:23.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0474e-05 2023-11-25T04:23:23.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:23.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298288+0000", 2023-11-25T04:23:23.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0156000000000001e-05 2023-11-25T04:23:23.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:23.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298293+0000", 2023-11-25T04:23:23.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7400000000000004e-06 2023-11-25T04:23:23.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:23.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298333+0000", 2023-11-25T04:23:23.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9666000000000003e-05 2023-11-25T04:23:23.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:23.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298351+0000", 2023-11-25T04:23:23.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8544999999999999e-05 2023-11-25T04:23:23.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.762 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:23.762 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302891+0000", 2023-11-25T04:23:23.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045398519999999996 2023-11-25T04:23:23.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.762 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:23.762 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302920+0000", 2023-11-25T04:23:23.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8750000000000001e-05 2023-11-25T04:23:23.762 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:23.762 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:23.762 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:23.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.763 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:14 3.15 3:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:23.763 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:23.763 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.703367016999998, 2023-11-25T04:23:23.763 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010990437, 2023-11-25T04:23:23.763 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:23.763 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:23.763 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:23.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:23.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:23.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:23.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:23.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:23.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:23.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:23.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033383+0000", 2023-11-25T04:23:23.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9870000000000002e-06 2023-11-25T04:23:23.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:23.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033398+0000", 2023-11-25T04:23:23.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4569999999999999e-05 2023-11-25T04:23:23.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:23.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033411+0000", 2023-11-25T04:23:23.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3543999999999999e-05 2023-11-25T04:23:23.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:23.766 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033423+0000", 2023-11-25T04:23:23.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1905999999999999e-05 2023-11-25T04:23:23.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.766 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:23.766 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034433+0000", 2023-11-25T04:23:23.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00101022 2023-11-25T04:23:23.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.766 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:23.766 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034539+0000", 2023-11-25T04:23:23.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010597199999999999 2023-11-25T04:23:23.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.767 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:23.767 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034667+0000", 2023-11-25T04:23:23.767 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012765500000000001 2023-11-25T04:23:23.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.767 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:23.767 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037803+0000", 2023-11-25T04:23:23.767 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031362220000000001 2023-11-25T04:23:23.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.767 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:23.767 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:23:23.767 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0064964669999999997 2023-11-25T04:23:23.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.768 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:23.768 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044331+0000", 2023-11-25T04:23:23.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1544999999999997e-05 2023-11-25T04:23:23.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.768 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:23.768 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044372+0000", 2023-11-25T04:23:23.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0349000000000003e-05 2023-11-25T04:23:23.768 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:23.768 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:23.768 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:23.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.769 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:15 3.15 3:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:23.769 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:23.769 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.702707367000002, 2023-11-25T04:23:23.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010464362, 2023-11-25T04:23:23.769 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:23.769 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:23.769 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:23.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:23.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:23.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:23.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:23.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:23.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:23.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:23.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034042+0000", 2023-11-25T04:23:23.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0999999999999997e-07 2023-11-25T04:23:23.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:23.771 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034048+0000", 2023-11-25T04:23:23.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0390000000000001e-06 2023-11-25T04:23:23.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.771 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:23.771 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034059+0000", 2023-11-25T04:23:23.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1182999999999999e-05 2023-11-25T04:23:23.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.771 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:23.771 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034067+0000", 2023-11-25T04:23:23.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1559999999999995e-06 2023-11-25T04:23:23.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.772 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.772 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:23.772 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035005+0000", 2023-11-25T04:23:23.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00093747399999999999 2023-11-25T04:23:23.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.772 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.772 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:23.772 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035077+0000", 2023-11-25T04:23:23.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2747e-05 2023-11-25T04:23:23.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.772 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.772 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:23.773 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035219+0000", 2023-11-25T04:23:23.773 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000141877 2023-11-25T04:23:23.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.773 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:23.773 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037810+0000", 2023-11-25T04:23:23.773 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002590752 2023-11-25T04:23:23.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.773 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:23.773 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044437+0000", 2023-11-25T04:23:23.773 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066270110000000004 2023-11-25T04:23:23.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:23.774 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044481+0000", 2023-11-25T04:23:23.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4106000000000001e-05 2023-11-25T04:23:23.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:23.774 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044505+0000", 2023-11-25T04:23:23.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4207e-05 2023-11-25T04:23:23.774 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:23.774 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:23.774 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:23.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.775 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:16 3.1d 3:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:23.775 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:23.775 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.702665519, 2023-11-25T04:23:23.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010413923, 2023-11-25T04:23:23.775 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:23.775 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:23.775 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:23.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:23.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:23.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:23.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:23.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:23.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:23.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:23.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:23.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4499999999999997e-07 2023-11-25T04:23:23.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:23.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034087+0000", 2023-11-25T04:23:23.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9770000000000002e-06 2023-11-25T04:23:23.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.777 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:23.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034094+0000", 2023-11-25T04:23:23.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8809999999999998e-06 2023-11-25T04:23:23.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.777 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:23.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034100+0000", 2023-11-25T04:23:23.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8440000000000003e-06 2023-11-25T04:23:23.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:23.778 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034993+0000", 2023-11-25T04:23:23.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00089336799999999996 2023-11-25T04:23:23.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:23.778 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035099+0000", 2023-11-25T04:23:23.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106037 2023-11-25T04:23:23.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:23.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035276+0000", 2023-11-25T04:23:23.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017631799999999999 2023-11-25T04:23:23.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:23.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037815+0000", 2023-11-25T04:23:23.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025395959999999999 2023-11-25T04:23:23.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:23.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044420+0000", 2023-11-25T04:23:23.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066045749999999997 2023-11-25T04:23:23.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:23.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044463+0000", 2023-11-25T04:23:23.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3205999999999999e-05 2023-11-25T04:23:23.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:23.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044497+0000", 2023-11-25T04:23:23.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3476000000000003e-05 2023-11-25T04:23:23.784 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:23.784 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:23.784 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:23.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.784 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:17 3.1d 3:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:23.784 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:23.784 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.70264031, 2023-11-25T04:23:23.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01127356, 2023-11-25T04:23:23.784 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:23.785 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:23.785 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:23.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.785 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:23.785 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:23.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:23.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.785 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:23.785 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:23.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:23.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:23.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:23.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1100000000000001e-07 2023-11-25T04:23:23.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:23.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034112+0000", 2023-11-25T04:23:23.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8600000000000003e-06 2023-11-25T04:23:23.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:23.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034120+0000", 2023-11-25T04:23:23.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7700000000000001e-06 2023-11-25T04:23:23.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:23.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034127+0000", 2023-11-25T04:23:23.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0999999999999998e-06 2023-11-25T04:23:23.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:23.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035601+0000", 2023-11-25T04:23:23.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014739410000000001 2023-11-25T04:23:23.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:23.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035685+0000", 2023-11-25T04:23:23.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3677999999999994e-05 2023-11-25T04:23:23.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:23.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035827+0000", 2023-11-25T04:23:23.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014232599999999999 2023-11-25T04:23:23.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:23.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038725+0000", 2023-11-25T04:23:23.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028982779999999998 2023-11-25T04:23:23.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:23.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045323+0000", 2023-11-25T04:23:23.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0065980780000000003 2023-11-25T04:23:23.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.790 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:23.790 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045354+0000", 2023-11-25T04:23:23.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0463e-05 2023-11-25T04:23:23.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.790 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:23.790 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045381+0000", 2023-11-25T04:23:23.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7654999999999999e-05 2023-11-25T04:23:23.790 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:23.790 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:23.790 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:23.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.791 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:22 3.e 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:23.791 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:23.791 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.702613896999999, 2023-11-25T04:23:23.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010133422, 2023-11-25T04:23:23.791 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:23.791 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:23.791 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:23.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:23.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:23.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:23.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:23.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:23.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:23.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:23.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034135+0000", 2023-11-25T04:23:23.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-07 2023-11-25T04:23:23.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:23.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034138+0000", 2023-11-25T04:23:23.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6009999999999999e-06 2023-11-25T04:23:23.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.794 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:23.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034145+0000", 2023-11-25T04:23:23.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1169999999999998e-06 2023-11-25T04:23:23.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.794 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:23.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034150+0000", 2023-11-25T04:23:23.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9080000000000003e-06 2023-11-25T04:23:23.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:23.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034209+0000", 2023-11-25T04:23:23.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8149000000000002e-05 2023-11-25T04:23:23.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:23.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034415+0000", 2023-11-25T04:23:23.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020637699999999999 2023-11-25T04:23:23.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:23.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034584+0000", 2023-11-25T04:23:23.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016870399999999999 2023-11-25T04:23:23.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:23.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.036979+0000", 2023-11-25T04:23:23.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023957509999999998 2023-11-25T04:23:23.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:23.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044161+0000", 2023-11-25T04:23:23.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0071811590000000003 2023-11-25T04:23:23.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:23.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044224+0000", 2023-11-25T04:23:23.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3454000000000006e-05 2023-11-25T04:23:23.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:23.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044268+0000", 2023-11-25T04:23:23.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3761999999999999e-05 2023-11-25T04:23:23.797 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:23.797 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:23.797 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:23.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.798 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:23 3.e 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:23.798 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:23.798 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.702582487999997, 2023-11-25T04:23:23.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010281676, 2023-11-25T04:23:23.798 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:23.798 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:23.798 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:23.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.798 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:23.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:23.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:23.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:23.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:23.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:23.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:23.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:23.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5799999999999999e-07 2023-11-25T04:23:23.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:23.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034169+0000", 2023-11-25T04:23:23.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1879999999999998e-06 2023-11-25T04:23:23.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:23.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034177+0000", 2023-11-25T04:23:23.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7020000000000001e-06 2023-11-25T04:23:23.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:23.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034184+0000", 2023-11-25T04:23:23.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3910000000000003e-06 2023-11-25T04:23:23.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:23.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034924+0000", 2023-11-25T04:23:23.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00074032999999999998 2023-11-25T04:23:23.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:23.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034991+0000", 2023-11-25T04:23:23.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7347000000000004e-05 2023-11-25T04:23:23.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:23.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035096+0000", 2023-11-25T04:23:23.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010488 2023-11-25T04:23:23.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:23.803 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037588+0000", 2023-11-25T04:23:23.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024918309999999999 2023-11-25T04:23:23.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:23.803 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044367+0000", 2023-11-25T04:23:23.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067787589999999997 2023-11-25T04:23:23.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:23.803 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044415+0000", 2023-11-25T04:23:23.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7803000000000003e-05 2023-11-25T04:23:23.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.804 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:23.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044447+0000", 2023-11-25T04:23:23.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2886999999999999e-05 2023-11-25T04:23:23.804 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:23.804 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:23.804 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:23.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.805 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:30 3.15 3:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:23.805 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:23.805 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.702524048000001, 2023-11-25T04:23:23.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010365070000000001, 2023-11-25T04:23:23.805 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:23.805 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:23.805 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:23.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:23.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:23.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:23.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:23.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:23.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:23.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:23.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034225+0000", 2023-11-25T04:23:23.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7400000000000002e-07 2023-11-25T04:23:23.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:23.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034229+0000", 2023-11-25T04:23:23.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3100000000000002e-06 2023-11-25T04:23:23.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:23.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034239+0000", 2023-11-25T04:23:23.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9140000000000003e-06 2023-11-25T04:23:23.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:23.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034246+0000", 2023-11-25T04:23:23.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0149999999999998e-06 2023-11-25T04:23:23.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:23.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035572+0000", 2023-11-25T04:23:23.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001325655 2023-11-25T04:23:23.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:23.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035730+0000", 2023-11-25T04:23:23.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015827899999999999 2023-11-25T04:23:23.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:23.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035857+0000", 2023-11-25T04:23:23.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012644699999999999 2023-11-25T04:23:23.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:23.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038737+0000", 2023-11-25T04:23:23.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002880077 2023-11-25T04:23:23.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:23.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044540+0000", 2023-11-25T04:23:23.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058034239999999997 2023-11-25T04:23:23.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:23.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044569+0000", 2023-11-25T04:23:23.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8388999999999999e-05 2023-11-25T04:23:23.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:23.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044589+0000", 2023-11-25T04:23:23.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0686000000000002e-05 2023-11-25T04:23:23.811 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:23.811 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:23.811 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:23.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.811 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:33 3.1b 3:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:23.811 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:23.811 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.702478550000002, 2023-11-25T04:23:23.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099754049999999997, 2023-11-25T04:23:23.811 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:23.811 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:23:23.811 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:23.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.812 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:23.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:23.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:23.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.812 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:23.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:23.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:23.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:23.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:23.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4799999999999998e-07 2023-11-25T04:23:23.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:23.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034273+0000", 2023-11-25T04:23:23.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7819999999999998e-06 2023-11-25T04:23:23.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:23.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034285+0000", 2023-11-25T04:23:23.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2058e-05 2023-11-25T04:23:23.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:23.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034291+0000", 2023-11-25T04:23:23.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0680000000000002e-06 2023-11-25T04:23:23.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:23.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034370+0000", 2023-11-25T04:23:23.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8452e-05 2023-11-25T04:23:23.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:23.816 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034477+0000", 2023-11-25T04:23:23.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107471 2023-11-25T04:23:23.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:23.816 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044245+0000", 2023-11-25T04:23:23.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0097679259999999997 2023-11-25T04:23:23.816 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:23.816 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:23.816 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:23.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.817 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:36 3.1a 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:23.817 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:23.817 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.702431877000002, 2023-11-25T04:23:23.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058179099999999999, 2023-11-25T04:23:23.817 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:23.817 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:23:23.817 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:23.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:23.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:23.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:23.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:23.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:23.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:23.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:23.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034317+0000", 2023-11-25T04:23:23.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1999999999999999e-07 2023-11-25T04:23:23.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:23.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034321+0000", 2023-11-25T04:23:23.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.233e-06 2023-11-25T04:23:23.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:23.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034337+0000", 2023-11-25T04:23:23.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5308999999999999e-05 2023-11-25T04:23:23.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:23.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034354+0000", 2023-11-25T04:23:23.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7411999999999999e-05 2023-11-25T04:23:23.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:23.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034465+0000", 2023-11-25T04:23:23.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011154699999999999 2023-11-25T04:23:23.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:23.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034560+0000", 2023-11-25T04:23:23.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4186999999999998e-05 2023-11-25T04:23:23.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:23.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.040134+0000", 2023-11-25T04:23:23.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055746019999999997 2023-11-25T04:23:23.821 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:23.821 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:23.821 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:23.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.821 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:9 3.10 e18/16 3:09401a81:::601.00000000:head v 18'4, mlcod=0'0)", 2023-11-25T04:23:23.821 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:23.821 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.693537569, 2023-11-25T04:23:23.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061641020000000003, 2023-11-25T04:23:23.821 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:23.821 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:23.822 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:23.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:23.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:23.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:23.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:23.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:23.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:23.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.823 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:23.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:23.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5300000000000001e-07 2023-11-25T04:23:23.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:23.824 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043222+0000", 2023-11-25T04:23:23.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.082e-05 2023-11-25T04:23:23.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:23.824 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043228+0000", 2023-11-25T04:23:23.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6699999999999997e-06 2023-11-25T04:23:23.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:23.824 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043236+0000", 2023-11-25T04:23:23.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8189999999999992e-06 2023-11-25T04:23:23.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:23.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043507+0000", 2023-11-25T04:23:23.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000271012 2023-11-25T04:23:23.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:23.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043530+0000", 2023-11-25T04:23:23.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2659000000000001e-05 2023-11-25T04:23:23.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:23.826 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049321+0000", 2023-11-25T04:23:23.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057915129999999999 2023-11-25T04:23:23.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:23.826 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049375+0000", 2023-11-25T04:23:23.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3455999999999998e-05 2023-11-25T04:23:23.826 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:23.826 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:23.827 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:23.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.827 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:1 3.14 e18/16 3:292cf221:::200.00000000:head v 18'3, mlcod=0'0)", 2023-11-25T04:23:23.827 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:23.827 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.693500856, 2023-11-25T04:23:23.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0060983599999999997, 2023-11-25T04:23:23.827 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:23.827 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:23.827 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:23.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:23.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:23.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:23.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:23.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:23.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:23.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:23.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043248+0000", 2023-11-25T04:23:23.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9399999999999995e-07 2023-11-25T04:23:23.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:23.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043252+0000", 2023-11-25T04:23:23.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1799999999999998e-06 2023-11-25T04:23:23.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:23.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043259+0000", 2023-11-25T04:23:23.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0049999999999998e-06 2023-11-25T04:23:23.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:23.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043265+0000", 2023-11-25T04:23:23.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0859999999999998e-06 2023-11-25T04:23:23.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:23.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043311+0000", 2023-11-25T04:23:23.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6082999999999999e-05 2023-11-25T04:23:23.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:23.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043330+0000", 2023-11-25T04:23:23.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8556e-05 2023-11-25T04:23:23.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:23.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049303+0000", 2023-11-25T04:23:23.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059733470000000004 2023-11-25T04:23:23.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:23.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049346+0000", 2023-11-25T04:23:23.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2608999999999998e-05 2023-11-25T04:23:23.832 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:23.832 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:23.832 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:23.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.832 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:3 3.14 e18/16 3:287034c4:::1.00000000.inode:head v 18'4, mlcod=0'0)", 2023-11-25T04:23:23.832 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:23.832 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.692984350000003, 2023-11-25T04:23:23.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057040090000000003, 2023-11-25T04:23:23.832 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:23.832 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:23.832 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:23.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:23.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:23.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:23.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:23.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:23.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:23.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:23.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:23.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9799999999999999e-07 2023-11-25T04:23:23.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.835 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:23.835 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043785+0000", 2023-11-25T04:23:23.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0721999999999999e-05 2023-11-25T04:23:23.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.835 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:23.835 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043793+0000", 2023-11-25T04:23:23.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5839999999999995e-06 2023-11-25T04:23:23.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.835 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:23.835 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043803+0000", 2023-11-25T04:23:23.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0729999999999994e-06 2023-11-25T04:23:23.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:23.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043843+0000", 2023-11-25T04:23:23.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0747999999999997e-05 2023-11-25T04:23:23.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:23.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043856+0000", 2023-11-25T04:23:23.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2697000000000001e-05 2023-11-25T04:23:23.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.837 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:23.837 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049426+0000", 2023-11-25T04:23:23.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055704409999999998 2023-11-25T04:23:23.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.837 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:23.837 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049468+0000", 2023-11-25T04:23:23.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1446000000000003e-05 2023-11-25T04:23:23.837 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:23.837 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:23.837 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:23.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.838 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "pg_update_log_missing(3.14 epoch 18/16 rep_tid 171 entries 18'5 (0'0) error 3:2e2fa760:::200.00000001:head by mds.0.3:32 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:23:23.838 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:23.838 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.692823386999997, 2023-11-25T04:23:23.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005556938, 2023-11-25T04:23:23.838 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:23.838 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2023-11-25T04:23:23.838 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:23.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:23.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:23.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:23.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:23.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:23.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:23.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:23.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:23.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.03e-07 2023-11-25T04:23:23.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:23.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043931+0000", 2023-11-25T04:23:23.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1079999999999998e-06 2023-11-25T04:23:23.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:23.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043944+0000", 2023-11-25T04:23:23.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2852e-05 2023-11-25T04:23:23.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:23.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043955+0000", 2023-11-25T04:23:23.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1037e-05 2023-11-25T04:23:23.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:23.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044088+0000", 2023-11-25T04:23:23.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013269599999999999 2023-11-25T04:23:23.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:23.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049482+0000", 2023-11-25T04:23:23.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053940419999999999 2023-11-25T04:23:23.842 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:23.842 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:23.842 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:23.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.842 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:10 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'5, mlcod=0'0)", 2023-11-25T04:23:23.842 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:23.842 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.692733359000002, 2023-11-25T04:23:23.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054493099999999997, 2023-11-25T04:23:23.842 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:23.843 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:23.843 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:23.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:23.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:23.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:23.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.844 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:23.844 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:23.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:23.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.844 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:23.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:23.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6100000000000002e-07 2023-11-25T04:23:23.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.845 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:23.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044019+0000", 2023-11-25T04:23:23.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.839e-06 2023-11-25T04:23:23.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.845 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:23.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044026+0000", 2023-11-25T04:23:23.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5869999999999998e-06 2023-11-25T04:23:23.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:23.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044031+0000", 2023-11-25T04:23:23.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4909999999999996e-06 2023-11-25T04:23:23.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:23.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044070+0000", 2023-11-25T04:23:23.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8829999999999999e-05 2023-11-25T04:23:23.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:23.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044081+0000", 2023-11-25T04:23:23.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1447999999999999e-05 2023-11-25T04:23:23.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:23.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049406+0000", 2023-11-25T04:23:23.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053246480000000004 2023-11-25T04:23:23.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:23.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049464+0000", 2023-11-25T04:23:23.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8205999999999998e-05 2023-11-25T04:23:23.847 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:23.848 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:23.848 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:23.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.848 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:11 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'6, mlcod=0'0)", 2023-11-25T04:23:23.848 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:23.848 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 36.692532094999997, 2023-11-25T04:23:23.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052673499999999996, 2023-11-25T04:23:23.848 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:23.848 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:23.849 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:23.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:23.849 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:23.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:23.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:23.849 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:23.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:23.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.850 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:23.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:23.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0199999999999998e-07 2023-11-25T04:23:23.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.850 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:23.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044220+0000", 2023-11-25T04:23:23.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7720000000000001e-06 2023-11-25T04:23:23.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.850 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:23.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044226+0000", 2023-11-25T04:23:23.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8579999999999996e-06 2023-11-25T04:23:23.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.851 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:23.851 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044232+0000", 2023-11-25T04:23:23.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1700000000000002e-06 2023-11-25T04:23:23.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.851 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:23.851 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044274+0000", 2023-11-25T04:23:23.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1254999999999997e-05 2023-11-25T04:23:23.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:23.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:23:23.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6880999999999999e-05 2023-11-25T04:23:23.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:23.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049466+0000", 2023-11-25T04:23:23.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051659219999999999 2023-11-25T04:23:23.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:23.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:23.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:23.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049483+0000", 2023-11-25T04:23:23.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.719e-05 2023-11-25T04:23:23.853 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:23.853 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:23.853 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:23.853 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:23.853 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:23.853 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:23.854 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:23.853+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:23.855 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:23:23.955 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:23.954+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:23.965 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:23.965 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:23:23.965 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2023-11-25T04:23:23.965 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:23.974 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:23:24.059 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:24.057+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:24.119 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:24.119 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:23:24.119 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2023-11-25T04:23:24.119 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2023-11-25T04:23:24.119 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:24.130 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:23:24.171 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:24.169+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:24.258 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:24.256+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:24.278 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:24.278 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2023-11-25T04:23:24.278 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2023-11-25T04:23:24.278 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2023-11-25T04:23:24.278 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.278 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:24.278 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:24.279 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.244068783000003, 2023-11-25T04:23:24.279 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01072943, 2023-11-25T04:23:24.279 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:24.279 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:24.279 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:24.279 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.279 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:24.279 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:24.279 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:24.279 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.279 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.279 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:24.279 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:24.280 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:24.280 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.280 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.280 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:24.280 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031829+0000", 2023-11-25T04:23:24.280 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.666e-06 2023-11-25T04:23:24.280 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.280 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.280 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:24.280 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031839+0000", 2023-11-25T04:23:24.280 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9490000000000008e-06 2023-11-25T04:23:24.280 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.280 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.281 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:24.281 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031857+0000", 2023-11-25T04:23:24.281 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8093000000000001e-05 2023-11-25T04:23:24.281 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.281 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.281 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:24.281 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031882+0000", 2023-11-25T04:23:24.281 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4294000000000002e-05 2023-11-25T04:23:24.281 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.281 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.281 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:24.281 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033130+0000", 2023-11-25T04:23:24.282 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001248264 2023-11-25T04:23:24.282 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.282 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.282 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:24.282 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033426+0000", 2023-11-25T04:23:24.282 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029620999999999998 2023-11-25T04:23:24.282 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.282 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.282 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:24.282 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033814+0000", 2023-11-25T04:23:24.282 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000387742 2023-11-25T04:23:24.282 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.282 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.283 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:24.283 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036793+0000", 2023-11-25T04:23:24.283 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002979598 2023-11-25T04:23:24.283 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.283 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.283 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:24.283 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042455+0000", 2023-11-25T04:23:24.283 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0056619260000000003 2023-11-25T04:23:24.283 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.283 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.283 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:24.283 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042502+0000", 2023-11-25T04:23:24.283 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7139999999999999e-05 2023-11-25T04:23:24.284 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.284 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.284 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:24.284 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042557+0000", 2023-11-25T04:23:24.284 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4548e-05 2023-11-25T04:23:24.284 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:24.284 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:24.284 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:24.284 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.284 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.284 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:24.284 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:24.284 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.242870248000003, 2023-11-25T04:23:24.285 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011505202000000001, 2023-11-25T04:23:24.285 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:24.285 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:24.285 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:24.285 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.285 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:24.285 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:24.285 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:24.285 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.285 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.285 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:24.285 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:24.285 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:24.286 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.286 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.286 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:24.286 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033027+0000", 2023-11-25T04:23:24.286 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0699999999999999e-06 2023-11-25T04:23:24.286 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.286 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.286 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:24.286 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033035+0000", 2023-11-25T04:23:24.286 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0830000000000004e-06 2023-11-25T04:23:24.286 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.286 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.287 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:24.287 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033059+0000", 2023-11-25T04:23:24.287 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3975e-05 2023-11-25T04:23:24.287 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.287 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.287 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:24.287 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033482+0000", 2023-11-25T04:23:24.287 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042253700000000001 2023-11-25T04:23:24.287 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.287 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.288 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:24.288 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034520+0000", 2023-11-25T04:23:24.288 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001037991 2023-11-25T04:23:24.288 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.288 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.288 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:24.288 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034611+0000", 2023-11-25T04:23:24.288 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1113999999999995e-05 2023-11-25T04:23:24.288 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.288 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.288 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:24.289 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034769+0000", 2023-11-25T04:23:24.289 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000157641 2023-11-25T04:23:24.289 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.289 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.289 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:24.289 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038292+0000", 2023-11-25T04:23:24.289 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035231160000000002 2023-11-25T04:23:24.289 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.289 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.289 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:24.289 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044440+0000", 2023-11-25T04:23:24.290 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0061486980000000002 2023-11-25T04:23:24.290 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.290 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.290 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:24.290 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044495+0000", 2023-11-25T04:23:24.290 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4339000000000003e-05 2023-11-25T04:23:24.290 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.290 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.290 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:24.290 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044531+0000", 2023-11-25T04:23:24.290 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6637999999999997e-05 2023-11-25T04:23:24.290 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:24.291 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:24.291 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:24.291 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.291 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.291 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:24.291 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:24.291 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.242395893999998, 2023-11-25T04:23:24.291 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0057634039999999997, 2023-11-25T04:23:24.291 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:24.291 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:24.291 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:24.291 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.292 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:24.292 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:24.292 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:24.292 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.292 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.292 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:24.292 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:24.292 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:24.292 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.292 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.292 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:24.292 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:24.292 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7900000000000007e-07 2023-11-25T04:23:24.293 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.293 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.293 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:24.293 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033507+0000", 2023-11-25T04:23:24.293 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8900000000000004e-06 2023-11-25T04:23:24.293 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.293 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.293 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:24.293 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033519+0000", 2023-11-25T04:23:24.293 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1756000000000001e-05 2023-11-25T04:23:24.293 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.293 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.293 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:24.294 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033527+0000", 2023-11-25T04:23:24.294 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3610000000000001e-06 2023-11-25T04:23:24.294 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.294 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.294 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:24.294 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034451+0000", 2023-11-25T04:23:24.294 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00092372400000000001 2023-11-25T04:23:24.294 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.294 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.294 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:24.294 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034809+0000", 2023-11-25T04:23:24.294 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00035757699999999998 2023-11-25T04:23:24.295 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.295 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.295 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:23:24.295 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034965+0000", 2023-11-25T04:23:24.295 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015638699999999999 2023-11-25T04:23:24.295 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.295 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.295 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:24.295 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038299+0000", 2023-11-25T04:23:24.295 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0033336559999999999 2023-11-25T04:23:24.295 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.295 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.296 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:24.296 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039169+0000", 2023-11-25T04:23:24.296 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00087066400000000005 2023-11-25T04:23:24.296 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.296 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.296 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:24.296 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039226+0000", 2023-11-25T04:23:24.296 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6511999999999997e-05 2023-11-25T04:23:24.296 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.296 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.296 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:24.296 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039264+0000", 2023-11-25T04:23:24.296 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7898000000000002e-05 2023-11-25T04:23:24.297 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:24.297 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:24.297 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:24.297 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.297 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.297 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:25 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:24.297 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:24.297 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.242317309999997, 2023-11-25T04:23:24.297 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096656169999999996, 2023-11-25T04:23:24.297 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:24.297 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:24.297 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:24.297 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.297 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:24.298 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:24.298 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:24.298 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.298 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.298 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:24.298 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:24.298 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:24.298 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.298 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.298 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:24.298 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033580+0000", 2023-11-25T04:23:24.299 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8199999999999999e-07 2023-11-25T04:23:24.299 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.299 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.299 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:24.299 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033585+0000", 2023-11-25T04:23:24.299 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3680000000000001e-06 2023-11-25T04:23:24.299 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.299 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.299 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:24.299 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033600+0000", 2023-11-25T04:23:24.299 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4331000000000001e-05 2023-11-25T04:23:24.299 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.299 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.300 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:24.300 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033606+0000", 2023-11-25T04:23:24.300 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5780000000000004e-06 2023-11-25T04:23:24.300 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.300 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.300 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:24.300 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035747+0000", 2023-11-25T04:23:24.300 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0021406530000000002 2023-11-25T04:23:24.300 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.300 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.300 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:24.300 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035901+0000", 2023-11-25T04:23:24.300 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000153931 2023-11-25T04:23:24.301 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.301 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.301 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:23:24.301 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036193+0000", 2023-11-25T04:23:24.301 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029211299999999999 2023-11-25T04:23:24.301 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.301 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.301 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:24.301 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038655+0000", 2023-11-25T04:23:24.301 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0024621640000000002 2023-11-25T04:23:24.301 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.301 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.302 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:24.302 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043130+0000", 2023-11-25T04:23:24.302 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.004474698 2023-11-25T04:23:24.302 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.302 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.302 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:24.302 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043204+0000", 2023-11-25T04:23:24.302 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4091999999999995e-05 2023-11-25T04:23:24.302 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.303 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.303 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:24.303 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043245+0000", 2023-11-25T04:23:24.304 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0907e-05 2023-11-25T04:23:24.304 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:24.304 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:24.304 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:24.304 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.304 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.304 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:26 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:24.304 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:24.304 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.242281486000003, 2023-11-25T04:23:24.304 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011086945000000001, 2023-11-25T04:23:24.304 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:24.304 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:24.305 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:24.305 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.305 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:24.305 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:24.305 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:24.305 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.305 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.305 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:24.305 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:24.305 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:24.305 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.305 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.305 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:24.305 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:24.306 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1699999999999998e-07 2023-11-25T04:23:24.306 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.306 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.306 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:24.306 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033627+0000", 2023-11-25T04:23:24.306 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.163e-05 2023-11-25T04:23:24.306 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.306 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.306 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:24.306 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033636+0000", 2023-11-25T04:23:24.306 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6859999999999995e-06 2023-11-25T04:23:24.306 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.306 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.307 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:24.307 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033642+0000", 2023-11-25T04:23:24.307 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3149999999999997e-06 2023-11-25T04:23:24.307 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.307 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.307 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:24.307 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035414+0000", 2023-11-25T04:23:24.307 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0017716380000000001 2023-11-25T04:23:24.307 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.307 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.307 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:24.307 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035539+0000", 2023-11-25T04:23:24.307 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000125477 2023-11-25T04:23:24.308 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.308 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.308 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:24.308 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035672+0000", 2023-11-25T04:23:24.308 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000133316 2023-11-25T04:23:24.308 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.308 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.308 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:24.308 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038649+0000", 2023-11-25T04:23:24.308 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029764679999999999 2023-11-25T04:23:24.308 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.308 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.308 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:24.308 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044578+0000", 2023-11-25T04:23:24.309 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0059293510000000002 2023-11-25T04:23:24.309 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.309 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.309 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:24.309 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044651+0000", 2023-11-25T04:23:24.309 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2513000000000002e-05 2023-11-25T04:23:24.309 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.309 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.309 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:24.309 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044702+0000", 2023-11-25T04:23:24.309 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1034e-05 2023-11-25T04:23:24.309 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:24.310 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:24.310 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:24.310 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.310 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.310 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:29 3.b 3:d0630e4c:::mds0_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:24.310 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:24.310 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.242243565999999, 2023-11-25T04:23:24.310 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010980227, 2023-11-25T04:23:24.310 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:24.310 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:24.310 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:24.310 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.310 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:24.311 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:24.311 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:24.311 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.311 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.311 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:24.311 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:24.311 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:24.311 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.311 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.311 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:24.311 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:24.311 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3499999999999996e-07 2023-11-25T04:23:24.311 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.311 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.312 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:24.312 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033658+0000", 2023-11-25T04:23:24.312 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1280000000000001e-06 2023-11-25T04:23:24.312 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.312 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.312 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:24.312 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033667+0000", 2023-11-25T04:23:24.312 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1430000000000004e-06 2023-11-25T04:23:24.312 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.312 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.312 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:24.312 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033678+0000", 2023-11-25T04:23:24.312 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0906e-05 2023-11-25T04:23:24.312 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.313 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.313 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:24.313 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034456+0000", 2023-11-25T04:23:24.313 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00077833899999999996 2023-11-25T04:23:24.313 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.313 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.313 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:24.313 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034575+0000", 2023-11-25T04:23:24.313 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011926000000000001 2023-11-25T04:23:24.313 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.313 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.313 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:24.313 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035001+0000", 2023-11-25T04:23:24.314 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042550999999999998 2023-11-25T04:23:24.314 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.314 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.314 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:24.314 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.037923+0000", 2023-11-25T04:23:24.314 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029219670000000001 2023-11-25T04:23:24.314 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.314 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.314 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:24.314 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044538+0000", 2023-11-25T04:23:24.314 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0066152349999999997 2023-11-25T04:23:24.314 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.315 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.315 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:24.315 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044594+0000", 2023-11-25T04:23:24.315 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5593000000000001e-05 2023-11-25T04:23:24.315 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.315 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.315 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:24.315 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044633+0000", 2023-11-25T04:23:24.315 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9511000000000002e-05 2023-11-25T04:23:24.315 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:24.315 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:24.315 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:24.315 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.316 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.316 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:34 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:24.316 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:24.316 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.241993452999999, 2023-11-25T04:23:24.316 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010920445000000001, 2023-11-25T04:23:24.316 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:24.316 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:24.316 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:24.316 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.316 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:24.316 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:24.316 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:24.316 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.316 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.317 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:24.317 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:24.317 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:24.317 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.317 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.317 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:24.317 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033904+0000", 2023-11-25T04:23:24.317 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6599999999999994e-07 2023-11-25T04:23:24.317 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.317 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.317 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:24.317 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033909+0000", 2023-11-25T04:23:24.317 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5639999999999996e-06 2023-11-25T04:23:24.317 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.318 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.318 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:24.318 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033923+0000", 2023-11-25T04:23:24.318 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3028e-05 2023-11-25T04:23:24.318 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.318 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.318 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:24.318 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034043+0000", 2023-11-25T04:23:24.318 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000120328 2023-11-25T04:23:24.318 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.318 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.318 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:24.318 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035576+0000", 2023-11-25T04:23:24.319 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015333720000000001 2023-11-25T04:23:24.319 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.319 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.319 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:24.319 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035699+0000", 2023-11-25T04:23:24.319 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000123001 2023-11-25T04:23:24.319 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.319 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.319 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:24.319 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044823+0000", 2023-11-25T04:23:24.319 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0091241859999999994 2023-11-25T04:23:24.319 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:24.319 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:24.320 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:24.320 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.320 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.320 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:35 3.1f 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:24.320 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:24.320 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.240689248000002, 2023-11-25T04:23:24.320 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096164779999999995, 2023-11-25T04:23:24.320 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:24.320 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:24.320 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:24.320 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.320 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:24.320 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:24.320 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:24.321 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.321 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.321 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:24.321 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:24.321 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:24.321 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.321 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.321 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:24.321 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035208+0000", 2023-11-25T04:23:24.321 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2699999999999998e-07 2023-11-25T04:23:24.321 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.321 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.321 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:24.322 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035213+0000", 2023-11-25T04:23:24.322 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.6070000000000001e-06 2023-11-25T04:23:24.322 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.322 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.322 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:24.322 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035350+0000", 2023-11-25T04:23:24.322 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013693599999999999 2023-11-25T04:23:24.322 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.322 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.322 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:24.330 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035358+0000", 2023-11-25T04:23:24.331 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0099999999999995e-06 2023-11-25T04:23:24.331 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.331 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.331 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:24.331 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036260+0000", 2023-11-25T04:23:24.331 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00090240100000000005 2023-11-25T04:23:24.331 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.331 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.331 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:24.331 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036517+0000", 2023-11-25T04:23:24.331 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00025736299999999998 2023-11-25T04:23:24.331 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.331 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.332 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:24.332 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044824+0000", 2023-11-25T04:23:24.332 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0083063340000000003 2023-11-25T04:23:24.332 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:24.332 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:24.332 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:24.332 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.332 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.332 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:37 3.1e 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:24.332 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:24.332 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.240522562000002, 2023-11-25T04:23:24.332 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0078322640000000002, 2023-11-25T04:23:24.332 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:24.332 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:24.333 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:24.333 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.333 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:24.333 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:24.333 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:24.333 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.333 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.333 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:24.333 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:24.333 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:24.333 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.334 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.334 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:24.334 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035375+0000", 2023-11-25T04:23:24.334 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9100000000000003e-07 2023-11-25T04:23:24.334 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.334 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.334 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:24.334 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035379+0000", 2023-11-25T04:23:24.334 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9430000000000002e-06 2023-11-25T04:23:24.334 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.334 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.335 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:24.335 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035388+0000", 2023-11-25T04:23:24.335 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2760000000000006e-06 2023-11-25T04:23:24.335 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.335 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.335 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:24.335 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035396+0000", 2023-11-25T04:23:24.335 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8480000000000001e-06 2023-11-25T04:23:24.335 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.335 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.335 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:24.335 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035452+0000", 2023-11-25T04:23:24.335 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5831000000000001e-05 2023-11-25T04:23:24.335 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.336 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.336 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:24.336 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035934+0000", 2023-11-25T04:23:24.336 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00048265599999999998 2023-11-25T04:23:24.336 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.336 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.336 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:24.336 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043206+0000", 2023-11-25T04:23:24.336 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.007271919 2023-11-25T04:23:24.336 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:24.336 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:24.336 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:24.336 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.337 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.337 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing(3.1a epoch 18/16 rep_tid 3 entries 18'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:36 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:23:24.337 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:24.337 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.239246975999997, 2023-11-25T04:23:24.337 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002073414, 2023-11-25T04:23:24.337 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:24.337 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:24.337 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:24.337 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.337 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:24.337 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:24.337 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:24.337 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.337 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.338 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:24.338 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:24.338 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:24.338 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.338 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.338 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:24.338 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036650+0000", 2023-11-25T04:23:24.338 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7300000000000002e-07 2023-11-25T04:23:24.338 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.338 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.338 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:24.338 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036659+0000", 2023-11-25T04:23:24.338 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.7499999999999992e-06 2023-11-25T04:23:24.339 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.339 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.339 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:24.339 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036673+0000", 2023-11-25T04:23:24.339 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4917e-05 2023-11-25T04:23:24.339 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.339 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.339 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:24.339 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036687+0000", 2023-11-25T04:23:24.339 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3883000000000001e-05 2023-11-25T04:23:24.339 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.339 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.339 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:24.339 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036781+0000", 2023-11-25T04:23:24.340 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3219000000000003e-05 2023-11-25T04:23:24.340 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.340 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.340 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:24.340 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038723+0000", 2023-11-25T04:23:24.340 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019422720000000001 2023-11-25T04:23:24.340 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:24.340 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:24.340 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:24.340 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.340 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.340 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:28 3.13 e18/16 3:cd5a64a3:::100.00000000:head v 18'1, mlcod=0'0)", 2023-11-25T04:23:24.340 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:24.341 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.235585817999997, 2023-11-25T04:23:24.341 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001850546, 2023-11-25T04:23:24.341 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:24.341 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:24.341 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:24.341 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.341 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:24.341 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:24.341 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:24.341 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.341 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.341 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:24.341 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:24.341 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:24.341 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.342 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.342 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:24.342 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:24.342 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7800000000000002e-07 2023-11-25T04:23:24.342 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.342 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.342 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:24.342 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040331+0000", 2023-11-25T04:23:24.342 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0449999999999999e-05 2023-11-25T04:23:24.343 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.343 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.343 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:24.343 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040339+0000", 2023-11-25T04:23:24.343 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0709999999999995e-06 2023-11-25T04:23:24.343 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.343 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.343 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:24.343 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040347+0000", 2023-11-25T04:23:24.343 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7079999999999997e-06 2023-11-25T04:23:24.343 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.343 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.344 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:24.344 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040410+0000", 2023-11-25T04:23:24.344 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2453000000000007e-05 2023-11-25T04:23:24.344 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.344 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.344 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:24.344 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040422+0000", 2023-11-25T04:23:24.345 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2429000000000001e-05 2023-11-25T04:23:24.345 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.345 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.345 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:24.345 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042129+0000", 2023-11-25T04:23:24.345 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001707266 2023-11-25T04:23:24.345 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.345 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.345 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:24.345 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042161+0000", 2023-11-25T04:23:24.345 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1791000000000001e-05 2023-11-25T04:23:24.346 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:24.346 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:24.346 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:24.346 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.346 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.346 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:27 3.13 e18/16 3:c91d4a1d:::mds0_inotable:head v 18'2, mlcod=0'0)", 2023-11-25T04:23:24.346 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:24.346 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.235382295000001, 2023-11-25T04:23:24.346 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0016886430000000001, 2023-11-25T04:23:24.346 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:24.346 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:24.346 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:24.346 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.347 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:24.347 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:24.347 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:24.347 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.347 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.347 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:24.347 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:24.347 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:24.347 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.347 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.348 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:24.348 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:24.348 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3999999999999997e-07 2023-11-25T04:23:24.348 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.348 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.348 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:24.348 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040520+0000", 2023-11-25T04:23:24.348 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2989999999999996e-06 2023-11-25T04:23:24.348 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.348 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.348 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:24.348 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040527+0000", 2023-11-25T04:23:24.349 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5730000000000002e-06 2023-11-25T04:23:24.349 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.349 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.349 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:24.349 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040535+0000", 2023-11-25T04:23:24.349 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8630000000000008e-06 2023-11-25T04:23:24.349 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.349 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.349 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:24.349 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040683+0000", 2023-11-25T04:23:24.349 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000148002 2023-11-25T04:23:24.349 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.349 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.349 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:24.350 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040700+0000", 2023-11-25T04:23:24.350 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7065e-05 2023-11-25T04:23:24.350 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.350 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.350 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:24.350 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042163+0000", 2023-11-25T04:23:24.350 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001462767 2023-11-25T04:23:24.350 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.350 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.350 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:24.350 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042203+0000", 2023-11-25T04:23:24.350 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9734000000000003e-05 2023-11-25T04:23:24.350 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:24.351 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:24.351 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:24.351 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.351 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.351 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:20 3.1c e18/16 3:3ed09add:::607.00000000:head v 18'1, mlcod=0'0)", 2023-11-25T04:23:24.351 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:24.351 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.235347253999997, 2023-11-25T04:23:24.351 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015271, 2023-11-25T04:23:24.351 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:24.351 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:24.351 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:24.351 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.352 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:24.352 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:24.352 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:24.352 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.352 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.352 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:24.352 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:24.352 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:24.352 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.352 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.352 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:24.352 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:24.352 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4799999999999999e-07 2023-11-25T04:23:24.353 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.353 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.353 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:24.353 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040561+0000", 2023-11-25T04:23:24.353 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1800000000000001e-05 2023-11-25T04:23:24.353 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.353 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.353 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:24.353 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040568+0000", 2023-11-25T04:23:24.353 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7730000000000002e-06 2023-11-25T04:23:24.353 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.353 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.353 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:24.354 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040587+0000", 2023-11-25T04:23:24.354 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8691000000000001e-05 2023-11-25T04:23:24.354 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.354 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.354 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:24.354 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040638+0000", 2023-11-25T04:23:24.354 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1068e-05 2023-11-25T04:23:24.354 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.354 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.354 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:24.354 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040670+0000", 2023-11-25T04:23:24.355 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2125999999999997e-05 2023-11-25T04:23:24.355 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.355 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.355 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:24.355 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042032+0000", 2023-11-25T04:23:24.355 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001361821 2023-11-25T04:23:24.355 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.355 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.355 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:24.355 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042076+0000", 2023-11-25T04:23:24.355 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4472999999999999e-05 2023-11-25T04:23:24.355 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:24.355 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:24.355 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:24.356 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.356 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.356 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:21 3.1c e18/16 3:3ed09add:::607.00000000:head v 18'2, mlcod=0'0)", 2023-11-25T04:23:24.356 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:24.356 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.235112528999998, 2023-11-25T04:23:24.356 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0013228160000000001, 2023-11-25T04:23:24.356 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:24.356 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:24.356 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:24.356 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.356 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:24.356 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:24.356 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:24.357 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.357 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.357 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:24.357 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:24.357 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:24.357 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.357 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.357 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:24.357 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:24.357 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9299999999999999e-07 2023-11-25T04:23:24.357 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.357 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.357 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:24.357 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040790+0000", 2023-11-25T04:23:24.358 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5770000000000001e-06 2023-11-25T04:23:24.358 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.358 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.358 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:24.358 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040799+0000", 2023-11-25T04:23:24.358 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6439999999999995e-06 2023-11-25T04:23:24.358 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.358 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.358 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:24.358 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040806+0000", 2023-11-25T04:23:24.358 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0180000000000004e-06 2023-11-25T04:23:24.358 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.358 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.359 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:24.359 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040907+0000", 2023-11-25T04:23:24.359 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000100096 2023-11-25T04:23:24.359 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.359 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.359 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:24.359 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040925+0000", 2023-11-25T04:23:24.359 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8536999999999999e-05 2023-11-25T04:23:24.359 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.359 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.359 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:24.359 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042079+0000", 2023-11-25T04:23:24.360 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00115358 2023-11-25T04:23:24.360 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.360 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.360 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:24.360 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042107+0000", 2023-11-25T04:23:24.360 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7971e-05 2023-11-25T04:23:24.360 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:24.360 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:24.360 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:24.360 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.360 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.360 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop_reply(mds.0.3:4 3.1f e18/16 ondisk, result = 0)", 2023-11-25T04:23:24.360 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:24.361 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.233773139, 2023-11-25T04:23:24.361 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00044092900000000001, 2023-11-25T04:23:24.361 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:24.361 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:24.361 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:24.361 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.361 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:24.361 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:24.361 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:24.361 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.361 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.361 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:24.361 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:24.362 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:24.362 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.362 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.362 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:24.362 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042124+0000", 2023-11-25T04:23:24.362 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1800000000000005e-07 2023-11-25T04:23:24.362 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.362 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.362 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:24.362 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042128+0000", 2023-11-25T04:23:24.362 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2289999999999997e-06 2023-11-25T04:23:24.362 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.363 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.363 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:24.363 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042138+0000", 2023-11-25T04:23:24.363 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9729999999999992e-06 2023-11-25T04:23:24.363 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.363 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.363 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:24.363 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042145+0000", 2023-11-25T04:23:24.363 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.336e-06 2023-11-25T04:23:24.364 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.364 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.364 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:24.364 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042441+0000", 2023-11-25T04:23:24.364 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029650500000000002 2023-11-25T04:23:24.364 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.364 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.364 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:24.364 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042448+0000", 2023-11-25T04:23:24.364 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4100000000000002e-06 2023-11-25T04:23:24.364 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.364 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.365 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:24.365 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042564+0000", 2023-11-25T04:23:24.365 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000115758 2023-11-25T04:23:24.365 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:24.365 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:24.365 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:24.365 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.365 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.365 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing_reply(3.1f epoch 18/16 rep_tid 8 lcod 18'4)", 2023-11-25T04:23:24.365 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:24.365 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.231447727000003, 2023-11-25T04:23:24.365 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00038154699999999998, 2023-11-25T04:23:24.365 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:24.365 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:24.366 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:24.366 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.366 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:24.366 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:24.366 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:24.366 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.366 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.366 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:24.366 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:24.366 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:24.366 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.366 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.367 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:24.367 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:24.367 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2700000000000001e-07 2023-11-25T04:23:24.367 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.367 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.367 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:24.367 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044451+0000", 2023-11-25T04:23:24.367 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7460000000000001e-06 2023-11-25T04:23:24.367 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.367 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.367 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:24.367 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044455+0000", 2023-11-25T04:23:24.368 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5809999999999998e-06 2023-11-25T04:23:24.368 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.368 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.368 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:24.368 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044458+0000", 2023-11-25T04:23:24.368 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0170000000000001e-06 2023-11-25T04:23:24.368 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.368 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.368 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:24.368 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044734+0000", 2023-11-25T04:23:24.368 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00027582799999999999 2023-11-25T04:23:24.368 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.368 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.369 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:24.369 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044830+0000", 2023-11-25T04:23:24.369 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6248000000000004e-05 2023-11-25T04:23:24.369 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:24.369 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:24.369 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:24.369 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.369 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.369 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing_reply(3.b epoch 18/16 rep_tid 5 lcod 18'2)", 2023-11-25T04:23:24.369 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:24.369 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.231411066, 2023-11-25T04:23:24.369 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000344488, 2023-11-25T04:23:24.369 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:24.370 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:24.370 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:24.370 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.370 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:24.370 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:24.370 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:24.370 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.370 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.371 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:24.371 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:24.371 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:24.371 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.371 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.371 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:24.371 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:24.371 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.55e-07 2023-11-25T04:23:24.371 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.371 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.371 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:24.371 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044487+0000", 2023-11-25T04:23:24.372 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2470000000000001e-06 2023-11-25T04:23:24.372 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.372 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.372 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:24.372 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044491+0000", 2023-11-25T04:23:24.372 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9659999999999998e-06 2023-11-25T04:23:24.372 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.372 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.372 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:24.372 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044494+0000", 2023-11-25T04:23:24.372 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2310000000000001e-06 2023-11-25T04:23:24.372 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.372 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.372 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:24.373 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044756+0000", 2023-11-25T04:23:24.373 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00026172800000000003 2023-11-25T04:23:24.373 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.373 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.373 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:24.373 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044830+0000", 2023-11-25T04:23:24.373 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4160999999999997e-05 2023-11-25T04:23:24.373 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:24.373 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:24.373 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:24.373 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.373 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.373 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:39 3.6 3:654134d2:::mds0_openfiles.0:head [omap-clear,omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2023-11-25T04:23:24.374 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:24.374 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 32.171585024000002, 2023-11-25T04:23:24.374 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001788658, 2023-11-25T04:23:24.374 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:24.374 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:24.374 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:24.374 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.374 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:24.374 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:24.374 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:24.374 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.374 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.374 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:24.375 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:24.375 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:24.375 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.375 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.375 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:24.375 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104314+0000", 2023-11-25T04:23:24.375 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3099999999999999e-06 2023-11-25T04:23:24.375 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.375 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.375 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:24.375 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104325+0000", 2023-11-25T04:23:24.375 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1511999999999999e-05 2023-11-25T04:23:24.375 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.375 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.376 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:24.376 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104362+0000", 2023-11-25T04:23:24.376 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6903000000000002e-05 2023-11-25T04:23:24.376 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.376 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.376 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:24.376 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104387+0000", 2023-11-25T04:23:24.376 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4766e-05 2023-11-25T04:23:24.376 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.376 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.376 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:24.376 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104465+0000", 2023-11-25T04:23:24.376 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8119000000000002e-05 2023-11-25T04:23:24.377 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.377 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.377 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:24.377 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104557+0000", 2023-11-25T04:23:24.377 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2496000000000003e-05 2023-11-25T04:23:24.377 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.377 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.377 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:24.377 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104646+0000", 2023-11-25T04:23:24.377 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8818000000000002e-05 2023-11-25T04:23:24.378 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.378 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.378 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:24.378 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105201+0000", 2023-11-25T04:23:24.378 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00055506000000000002 2023-11-25T04:23:24.378 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.378 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.378 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:24.378 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105994+0000", 2023-11-25T04:23:24.378 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00079294700000000001 2023-11-25T04:23:24.378 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.378 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.379 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:24.379 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.106055+0000", 2023-11-25T04:23:24.379 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0566000000000003e-05 2023-11-25T04:23:24.379 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.379 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.379 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:24.379 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.106100+0000", 2023-11-25T04:23:24.379 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5160999999999997e-05 2023-11-25T04:23:24.379 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:24.379 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:24.379 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:24.379 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.379 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.380 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:43 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2023-11-25T04:23:24.380 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:24.380 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 17.171360027999999, 2023-11-25T04:23:24.380 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001751166, 2023-11-25T04:23:24.380 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:24.380 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:24.380 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:24.380 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.380 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:24.380 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:24.380 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:24.380 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.381 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.381 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:24.381 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:24.381 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:24.381 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.381 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.381 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:24.381 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104539+0000", 2023-11-25T04:23:24.381 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9529999999999999e-06 2023-11-25T04:23:24.381 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.381 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.382 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:24.382 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104557+0000", 2023-11-25T04:23:24.382 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7464000000000001e-05 2023-11-25T04:23:24.382 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.382 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.382 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:24.382 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104590+0000", 2023-11-25T04:23:24.382 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2966000000000003e-05 2023-11-25T04:23:24.382 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.382 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.382 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:24.383 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104608+0000", 2023-11-25T04:23:24.383 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8193e-05 2023-11-25T04:23:24.383 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.383 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.383 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:24.383 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104767+0000", 2023-11-25T04:23:24.383 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015882 2023-11-25T04:23:24.383 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.383 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.383 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:24.383 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104909+0000", 2023-11-25T04:23:24.383 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014244999999999999 2023-11-25T04:23:24.384 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.384 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.384 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:24.384 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.105059+0000", 2023-11-25T04:23:24.384 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014964399999999999 2023-11-25T04:23:24.384 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.384 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.384 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:24.384 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.105701+0000", 2023-11-25T04:23:24.384 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00064201099999999995 2023-11-25T04:23:24.384 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.384 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.384 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:24.385 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106233+0000", 2023-11-25T04:23:24.385 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00053178500000000001 2023-11-25T04:23:24.385 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.385 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.385 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:24.385 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106265+0000", 2023-11-25T04:23:24.385 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1986000000000002e-05 2023-11-25T04:23:24.385 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.385 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.385 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:24.385 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106288+0000", 2023-11-25T04:23:24.385 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2894000000000001e-05 2023-11-25T04:23:24.386 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:24.386 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:24.386 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:24.386 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.386 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.386 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:45 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e23)", 2023-11-25T04:23:24.386 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:23:24.386 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 12.171161988, 2023-11-25T04:23:24.386 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00098423299999999994, 2023-11-25T04:23:24.386 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:24.386 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:24.386 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:24.386 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.387 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:24.387 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:23:24.387 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:24.387 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.387 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.387 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:24.387 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:23:24.387 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:24.387 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.387 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.387 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:24.388 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104737+0000", 2023-11-25T04:23:24.388 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3309999999999998e-06 2023-11-25T04:23:24.388 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.388 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.388 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:24.388 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104748+0000", 2023-11-25T04:23:24.388 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1080999999999999e-05 2023-11-25T04:23:24.388 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.388 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.389 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:24.389 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104771+0000", 2023-11-25T04:23:24.389 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2707000000000001e-05 2023-11-25T04:23:24.389 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.389 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.389 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:24.389 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104800+0000", 2023-11-25T04:23:24.389 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9595e-05 2023-11-25T04:23:24.389 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.389 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.389 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:24.389 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104890+0000", 2023-11-25T04:23:24.390 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.9808999999999993e-05 2023-11-25T04:23:24.390 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.390 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.390 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:24.390 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105004+0000", 2023-11-25T04:23:24.390 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000114508 2023-11-25T04:23:24.390 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.390 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.390 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:24.390 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105634+0000", 2023-11-25T04:23:24.390 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00062985799999999996 2023-11-25T04:23:24.390 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.391 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.391 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:24.391 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105671+0000", 2023-11-25T04:23:24.391 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6640000000000002e-05 2023-11-25T04:23:24.391 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:24.391 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:24.391 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:24.391 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105719+0000", 2023-11-25T04:23:24.391 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7704000000000003e-05 2023-11-25T04:23:24.391 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:24.391 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:24.391 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:24.393 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:24.393 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:24.393 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:24.394 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:24.393+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:24.395 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:23:24.495 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:24.493+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:24.507 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:24.507 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:23:24.507 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2023-11-25T04:23:24.507 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:24.517 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:23:24.601 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:24.599+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:24.662 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:24.662 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:23:24.662 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2023-11-25T04:23:24.662 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2023-11-25T04:23:24.662 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:24.671 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:23:24.712 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:24.711+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:24.801 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:24.800+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:24.817 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:24.817 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2023-11-25T04:23:24.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2023-11-25T04:23:24.818 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2023-11-25T04:23:24.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.818 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:155 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'109, mlcod=11'105)", 2023-11-25T04:23:24.818 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:24.818 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.030471622, 2023-11-25T04:23:24.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061719019999999999, 2023-11-25T04:23:24.818 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:24.818 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:24.818 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:24.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:24.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:24.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:24.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:24.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:24.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:24.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:24.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784884+0000", 2023-11-25T04:23:24.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3099999999999999e-07 2023-11-25T04:23:24.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:24.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784942+0000", 2023-11-25T04:23:24.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8570000000000003e-05 2023-11-25T04:23:24.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:24.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784949+0000", 2023-11-25T04:23:24.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0310000000000001e-06 2023-11-25T04:23:24.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:24.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784953+0000", 2023-11-25T04:23:24.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6320000000000001e-06 2023-11-25T04:23:24.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:24.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784993+0000", 2023-11-25T04:23:24.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9737000000000003e-05 2023-11-25T04:23:24.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:24.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785011+0000", 2023-11-25T04:23:24.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8349999999999999e-05 2023-11-25T04:23:24.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:24.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791024+0000", 2023-11-25T04:23:24.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006012997 2023-11-25T04:23:24.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:24.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791055+0000", 2023-11-25T04:23:24.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1254000000000003e-05 2023-11-25T04:23:24.822 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:24.823 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:24.823 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:24.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.823 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:156 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'110, mlcod=11'105)", 2023-11-25T04:23:24.823 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:24.823 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.029885573000001, 2023-11-25T04:23:24.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005616116, 2023-11-25T04:23:24.823 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:24.823 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:24.823 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:24.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.823 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:24.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:24.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:24.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:24.824 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:24.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:24.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:24.824 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785470+0000", 2023-11-25T04:23:24.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0699999999999998e-07 2023-11-25T04:23:24.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:24.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785474+0000", 2023-11-25T04:23:24.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.048e-06 2023-11-25T04:23:24.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:24.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785480+0000", 2023-11-25T04:23:24.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7729999999999996e-06 2023-11-25T04:23:24.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:24.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785483+0000", 2023-11-25T04:23:24.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1970000000000001e-06 2023-11-25T04:23:24.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:24.826 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787936+0000", 2023-11-25T04:23:24.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024529040000000001 2023-11-25T04:23:24.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:24.826 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787947+0000", 2023-11-25T04:23:24.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1669e-05 2023-11-25T04:23:24.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:24.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791057+0000", 2023-11-25T04:23:24.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031095200000000002 2023-11-25T04:23:24.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:24.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791086+0000", 2023-11-25T04:23:24.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8697999999999999e-05 2023-11-25T04:23:24.827 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:24.827 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:24.827 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:24.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.828 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:157 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'111, mlcod=11'106)", 2023-11-25T04:23:24.828 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:24.828 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.029293015, 2023-11-25T04:23:24.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050590280000000001, 2023-11-25T04:23:24.828 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:24.828 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:24.828 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:24.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:24.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:24.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:24.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:24.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:24.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:24.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:24.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786063+0000", 2023-11-25T04:23:24.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2600000000000002e-07 2023-11-25T04:23:24.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:24.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786069+0000", 2023-11-25T04:23:24.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8839999999999999e-06 2023-11-25T04:23:24.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:24.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786079+0000", 2023-11-25T04:23:24.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0124e-05 2023-11-25T04:23:24.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:24.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786085+0000", 2023-11-25T04:23:24.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6660000000000002e-06 2023-11-25T04:23:24.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:24.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788160+0000", 2023-11-25T04:23:24.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020747669999999999 2023-11-25T04:23:24.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:24.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788180+0000", 2023-11-25T04:23:24.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0208999999999999e-05 2023-11-25T04:23:24.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:24.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791087+0000", 2023-11-25T04:23:24.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029064630000000002 2023-11-25T04:23:24.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.832 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:24.832 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791121+0000", 2023-11-25T04:23:24.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4289000000000003e-05 2023-11-25T04:23:24.832 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:24.832 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:24.832 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:24.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.832 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:158 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'112, mlcod=11'106)", 2023-11-25T04:23:24.832 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:24.832 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.028876621999999, 2023-11-25T04:23:24.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046663030000000001, 2023-11-25T04:23:24.833 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:24.833 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:24.833 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:24.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:24.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:24.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:24.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:24.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:24.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:24.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:24.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786479+0000", 2023-11-25T04:23:24.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2799999999999996e-07 2023-11-25T04:23:24.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:24.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786498+0000", 2023-11-25T04:23:24.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9069e-05 2023-11-25T04:23:24.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:24.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786508+0000", 2023-11-25T04:23:24.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5810000000000003e-06 2023-11-25T04:23:24.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:24.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786512+0000", 2023-11-25T04:23:24.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4889999999999997e-06 2023-11-25T04:23:24.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:24.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788438+0000", 2023-11-25T04:23:24.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019257059999999999 2023-11-25T04:23:24.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:24.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788449+0000", 2023-11-25T04:23:24.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1419e-05 2023-11-25T04:23:24.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.837 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:24.837 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791122+0000", 2023-11-25T04:23:24.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002672822 2023-11-25T04:23:24.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.837 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:24.837 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791145+0000", 2023-11-25T04:23:24.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2589e-05 2023-11-25T04:23:24.837 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:24.837 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:24.837 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:24.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.838 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:206 1.0 e12/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 12'152, mlcod=12'148)", 2023-11-25T04:23:24.838 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:24.838 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 46.517108151999999, 2023-11-25T04:23:24.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046728189999999999, 2023-11-25T04:23:24.838 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:24.838 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:24.838 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:24.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:24.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:24.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:24.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:24.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:24.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:24.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:24.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298248+0000", 2023-11-25T04:23:24.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3600000000000003e-07 2023-11-25T04:23:24.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:24.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298278+0000", 2023-11-25T04:23:24.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0474e-05 2023-11-25T04:23:24.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:24.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298288+0000", 2023-11-25T04:23:24.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0156000000000001e-05 2023-11-25T04:23:24.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:24.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298293+0000", 2023-11-25T04:23:24.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7400000000000004e-06 2023-11-25T04:23:24.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:24.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298333+0000", 2023-11-25T04:23:24.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9666000000000003e-05 2023-11-25T04:23:24.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:24.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298351+0000", 2023-11-25T04:23:24.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8544999999999999e-05 2023-11-25T04:23:24.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:24.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302891+0000", 2023-11-25T04:23:24.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045398519999999996 2023-11-25T04:23:24.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:24.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302920+0000", 2023-11-25T04:23:24.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8750000000000001e-05 2023-11-25T04:23:24.842 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:24.842 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:24.842 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:24.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.842 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:14 3.15 3:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:24.842 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:24.842 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.781973794999999, 2023-11-25T04:23:24.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010990437, 2023-11-25T04:23:24.842 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:24.842 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:24.843 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:24.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:24.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:24.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:24.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:24.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:24.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:24.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:24.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033383+0000", 2023-11-25T04:23:24.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9870000000000002e-06 2023-11-25T04:23:24.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.844 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:24.844 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033398+0000", 2023-11-25T04:23:24.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4569999999999999e-05 2023-11-25T04:23:24.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.844 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:24.844 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033411+0000", 2023-11-25T04:23:24.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3543999999999999e-05 2023-11-25T04:23:24.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.845 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:24.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033423+0000", 2023-11-25T04:23:24.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1905999999999999e-05 2023-11-25T04:23:24.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.845 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:24.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034433+0000", 2023-11-25T04:23:24.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00101022 2023-11-25T04:23:24.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.845 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:24.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034539+0000", 2023-11-25T04:23:24.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010597199999999999 2023-11-25T04:23:24.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:24.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034667+0000", 2023-11-25T04:23:24.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012765500000000001 2023-11-25T04:23:24.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:24.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037803+0000", 2023-11-25T04:23:24.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031362220000000001 2023-11-25T04:23:24.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:24.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:23:24.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0064964669999999997 2023-11-25T04:23:24.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:24.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044331+0000", 2023-11-25T04:23:24.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1544999999999997e-05 2023-11-25T04:23:24.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:24.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044372+0000", 2023-11-25T04:23:24.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0349000000000003e-05 2023-11-25T04:23:24.847 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:24.847 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:24.848 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:24.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.848 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:15 3.15 3:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:24.848 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:24.848 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.781314145000003, 2023-11-25T04:23:24.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010464362, 2023-11-25T04:23:24.848 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:24.848 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:24.848 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:24.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.848 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:24.849 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:24.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:24.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:24.849 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:24.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:24.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:24.849 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034042+0000", 2023-11-25T04:23:24.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0999999999999997e-07 2023-11-25T04:23:24.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.850 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:24.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034048+0000", 2023-11-25T04:23:24.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0390000000000001e-06 2023-11-25T04:23:24.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.850 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:24.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034059+0000", 2023-11-25T04:23:24.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1182999999999999e-05 2023-11-25T04:23:24.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.850 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:24.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034067+0000", 2023-11-25T04:23:24.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1559999999999995e-06 2023-11-25T04:23:24.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.851 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:24.851 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035005+0000", 2023-11-25T04:23:24.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00093747399999999999 2023-11-25T04:23:24.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.851 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:24.851 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035077+0000", 2023-11-25T04:23:24.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2747e-05 2023-11-25T04:23:24.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:24.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035219+0000", 2023-11-25T04:23:24.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000141877 2023-11-25T04:23:24.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:24.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037810+0000", 2023-11-25T04:23:24.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002590752 2023-11-25T04:23:24.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:24.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044437+0000", 2023-11-25T04:23:24.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066270110000000004 2023-11-25T04:23:24.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:24.853 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044481+0000", 2023-11-25T04:23:24.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4106000000000001e-05 2023-11-25T04:23:24.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:24.853 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044505+0000", 2023-11-25T04:23:24.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4207e-05 2023-11-25T04:23:24.853 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:24.853 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:24.854 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:24.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.854 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:16 3.1d 3:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:24.854 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:24.854 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.781272297000001, 2023-11-25T04:23:24.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010413923, 2023-11-25T04:23:24.854 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:24.854 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:24.854 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:24.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.854 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:24.854 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:24.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:24.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.855 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:24.855 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:24.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:24.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.855 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:24.855 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:24.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4499999999999997e-07 2023-11-25T04:23:24.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.855 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:24.856 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034087+0000", 2023-11-25T04:23:24.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9770000000000002e-06 2023-11-25T04:23:24.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.856 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.856 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:24.856 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034094+0000", 2023-11-25T04:23:24.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8809999999999998e-06 2023-11-25T04:23:24.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.856 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.856 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:24.856 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034100+0000", 2023-11-25T04:23:24.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8440000000000003e-06 2023-11-25T04:23:24.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.856 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.857 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:24.857 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034993+0000", 2023-11-25T04:23:24.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00089336799999999996 2023-11-25T04:23:24.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.857 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:24.857 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035099+0000", 2023-11-25T04:23:24.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106037 2023-11-25T04:23:24.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.857 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:24.857 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035276+0000", 2023-11-25T04:23:24.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017631799999999999 2023-11-25T04:23:24.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.858 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.858 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:24.858 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037815+0000", 2023-11-25T04:23:24.858 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025395959999999999 2023-11-25T04:23:24.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.858 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.858 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:24.858 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044420+0000", 2023-11-25T04:23:24.858 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066045749999999997 2023-11-25T04:23:24.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.858 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.859 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:24.859 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044463+0000", 2023-11-25T04:23:24.859 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3205999999999999e-05 2023-11-25T04:23:24.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.859 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.859 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:24.859 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044497+0000", 2023-11-25T04:23:24.859 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3476000000000003e-05 2023-11-25T04:23:24.859 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:24.859 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:24.859 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:24.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.860 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.860 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:17 3.1d 3:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:24.860 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:24.860 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.781247088000001, 2023-11-25T04:23:24.860 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01127356, 2023-11-25T04:23:24.860 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:24.860 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:24.860 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:24.860 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.860 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:24.860 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:24.860 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:24.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.860 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.861 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:24.861 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:24.861 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:24.861 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.861 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.861 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:24.861 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:24.861 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1100000000000001e-07 2023-11-25T04:23:24.861 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.861 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.861 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:24.861 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034112+0000", 2023-11-25T04:23:24.861 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8600000000000003e-06 2023-11-25T04:23:24.861 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.862 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.862 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:24.862 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034120+0000", 2023-11-25T04:23:24.862 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7700000000000001e-06 2023-11-25T04:23:24.862 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.862 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.862 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:24.862 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034127+0000", 2023-11-25T04:23:24.862 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0999999999999998e-06 2023-11-25T04:23:24.862 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.862 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.862 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:24.862 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035601+0000", 2023-11-25T04:23:24.862 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014739410000000001 2023-11-25T04:23:24.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.863 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:24.863 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035685+0000", 2023-11-25T04:23:24.863 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3677999999999994e-05 2023-11-25T04:23:24.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.863 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:24.863 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035827+0000", 2023-11-25T04:23:24.863 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014232599999999999 2023-11-25T04:23:24.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.864 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:24.864 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038725+0000", 2023-11-25T04:23:24.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028982779999999998 2023-11-25T04:23:24.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.864 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.864 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:24.864 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045323+0000", 2023-11-25T04:23:24.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0065980780000000003 2023-11-25T04:23:24.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.864 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.864 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:24.864 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045354+0000", 2023-11-25T04:23:24.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0463e-05 2023-11-25T04:23:24.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.864 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.865 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:24.865 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045381+0000", 2023-11-25T04:23:24.865 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7654999999999999e-05 2023-11-25T04:23:24.865 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:24.865 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:24.865 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:24.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.865 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.865 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:22 3.e 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:24.865 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:24.865 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.781220675, 2023-11-25T04:23:24.865 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010133422, 2023-11-25T04:23:24.865 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:24.866 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:24.866 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:24.866 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.866 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:24.866 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:24.866 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:24.866 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.866 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.866 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:24.866 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:24.866 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:24.866 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.866 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.867 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:24.867 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034135+0000", 2023-11-25T04:23:24.867 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-07 2023-11-25T04:23:24.867 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.867 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.867 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:24.867 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034138+0000", 2023-11-25T04:23:24.867 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6009999999999999e-06 2023-11-25T04:23:24.867 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.867 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.867 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:24.867 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034145+0000", 2023-11-25T04:23:24.867 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1169999999999998e-06 2023-11-25T04:23:24.868 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.868 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.868 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:24.868 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034150+0000", 2023-11-25T04:23:24.868 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9080000000000003e-06 2023-11-25T04:23:24.868 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.868 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.868 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:24.868 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034209+0000", 2023-11-25T04:23:24.868 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8149000000000002e-05 2023-11-25T04:23:24.868 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.868 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.868 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:24.868 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034415+0000", 2023-11-25T04:23:24.868 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020637699999999999 2023-11-25T04:23:24.869 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.869 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.869 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:24.869 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034584+0000", 2023-11-25T04:23:24.869 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016870399999999999 2023-11-25T04:23:24.869 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.869 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.869 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:24.869 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.036979+0000", 2023-11-25T04:23:24.869 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023957509999999998 2023-11-25T04:23:24.869 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.869 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.869 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:24.869 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044161+0000", 2023-11-25T04:23:24.870 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0071811590000000003 2023-11-25T04:23:24.870 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.870 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.870 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:24.870 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044224+0000", 2023-11-25T04:23:24.870 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3454000000000006e-05 2023-11-25T04:23:24.870 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.870 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.870 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:24.870 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044268+0000", 2023-11-25T04:23:24.870 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3761999999999999e-05 2023-11-25T04:23:24.870 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:24.870 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:24.870 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:24.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.871 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.871 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:23 3.e 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:24.871 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:24.871 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.781189265999998, 2023-11-25T04:23:24.871 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010281676, 2023-11-25T04:23:24.871 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:24.871 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:24.871 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:24.871 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.871 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:24.871 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:24.871 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:24.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.872 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.872 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:24.872 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:24.872 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:24.872 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.872 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.872 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:24.872 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:24.872 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5799999999999999e-07 2023-11-25T04:23:24.872 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.872 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.872 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:24.872 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034169+0000", 2023-11-25T04:23:24.872 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1879999999999998e-06 2023-11-25T04:23:24.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.873 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.873 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:24.873 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034177+0000", 2023-11-25T04:23:24.873 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7020000000000001e-06 2023-11-25T04:23:24.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.873 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.873 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:24.873 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034184+0000", 2023-11-25T04:23:24.873 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3910000000000003e-06 2023-11-25T04:23:24.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.873 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.873 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:24.873 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034924+0000", 2023-11-25T04:23:24.874 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00074032999999999998 2023-11-25T04:23:24.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.874 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:24.874 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034991+0000", 2023-11-25T04:23:24.874 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7347000000000004e-05 2023-11-25T04:23:24.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.874 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:24.874 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035096+0000", 2023-11-25T04:23:24.874 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010488 2023-11-25T04:23:24.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.875 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:24.875 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037588+0000", 2023-11-25T04:23:24.875 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024918309999999999 2023-11-25T04:23:24.875 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.875 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.875 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:24.875 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044367+0000", 2023-11-25T04:23:24.875 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067787589999999997 2023-11-25T04:23:24.875 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.875 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.875 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:24.875 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044415+0000", 2023-11-25T04:23:24.875 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7803000000000003e-05 2023-11-25T04:23:24.875 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.876 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.876 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:24.876 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044447+0000", 2023-11-25T04:23:24.876 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2886999999999999e-05 2023-11-25T04:23:24.876 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:24.876 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:24.876 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:24.876 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.876 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.876 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:30 3.15 3:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:24.876 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:24.876 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.781130826000002, 2023-11-25T04:23:24.876 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010365070000000001, 2023-11-25T04:23:24.876 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:24.877 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:24.877 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:24.877 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.877 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:24.877 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:24.877 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:24.877 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.877 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.877 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:24.877 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:24.877 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:24.877 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.877 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.878 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:24.878 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034225+0000", 2023-11-25T04:23:24.878 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7400000000000002e-07 2023-11-25T04:23:24.878 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.878 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.878 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:24.878 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034229+0000", 2023-11-25T04:23:24.878 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3100000000000002e-06 2023-11-25T04:23:24.878 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.878 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.878 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:24.878 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034239+0000", 2023-11-25T04:23:24.878 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9140000000000003e-06 2023-11-25T04:23:24.879 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.879 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.879 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:24.879 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034246+0000", 2023-11-25T04:23:24.879 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0149999999999998e-06 2023-11-25T04:23:24.879 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.879 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.879 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:24.879 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035572+0000", 2023-11-25T04:23:24.879 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001325655 2023-11-25T04:23:24.879 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.879 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.879 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:24.880 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035730+0000", 2023-11-25T04:23:24.880 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015827899999999999 2023-11-25T04:23:24.880 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.880 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.880 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:24.880 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035857+0000", 2023-11-25T04:23:24.880 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012644699999999999 2023-11-25T04:23:24.880 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.880 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.880 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:24.880 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038737+0000", 2023-11-25T04:23:24.880 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002880077 2023-11-25T04:23:24.880 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.880 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.881 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:24.881 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044540+0000", 2023-11-25T04:23:24.881 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058034239999999997 2023-11-25T04:23:24.881 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.881 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.881 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:24.881 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044569+0000", 2023-11-25T04:23:24.881 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8388999999999999e-05 2023-11-25T04:23:24.881 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.881 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.881 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:24.881 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044589+0000", 2023-11-25T04:23:24.881 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0686000000000002e-05 2023-11-25T04:23:24.881 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:24.882 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:24.882 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:24.882 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.882 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.882 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:33 3.1b 3:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:24.882 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:24.882 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.781085328000003, 2023-11-25T04:23:24.882 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099754049999999997, 2023-11-25T04:23:24.882 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:24.882 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:23:24.882 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:24.882 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.882 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:24.883 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:24.883 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:24.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.883 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.883 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:24.883 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:24.883 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:24.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.883 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.883 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:24.883 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:24.883 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4799999999999998e-07 2023-11-25T04:23:24.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.883 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.884 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:24.884 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034273+0000", 2023-11-25T04:23:24.884 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7819999999999998e-06 2023-11-25T04:23:24.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.884 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.884 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:24.884 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034285+0000", 2023-11-25T04:23:24.884 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2058e-05 2023-11-25T04:23:24.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.884 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.884 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:24.884 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034291+0000", 2023-11-25T04:23:24.884 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0680000000000002e-06 2023-11-25T04:23:24.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.884 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.885 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:24.885 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034370+0000", 2023-11-25T04:23:24.885 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8452e-05 2023-11-25T04:23:24.885 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.885 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.885 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:24.885 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034477+0000", 2023-11-25T04:23:24.885 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107471 2023-11-25T04:23:24.885 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.885 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.885 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:24.885 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044245+0000", 2023-11-25T04:23:24.885 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0097679259999999997 2023-11-25T04:23:24.886 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:24.886 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:24.886 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:24.886 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.886 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.886 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:36 3.1a 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:24.886 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:24.886 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.781038655000003, 2023-11-25T04:23:24.886 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058179099999999999, 2023-11-25T04:23:24.886 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:24.886 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:23:24.886 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:24.886 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.887 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:24.887 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:24.887 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:24.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.887 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.887 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:24.887 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:24.887 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:24.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.887 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.887 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:24.887 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034317+0000", 2023-11-25T04:23:24.887 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1999999999999999e-07 2023-11-25T04:23:24.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.888 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.888 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:24.888 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034321+0000", 2023-11-25T04:23:24.888 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.233e-06 2023-11-25T04:23:24.888 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.888 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.888 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:24.888 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034337+0000", 2023-11-25T04:23:24.888 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5308999999999999e-05 2023-11-25T04:23:24.888 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.888 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.888 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:24.888 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034354+0000", 2023-11-25T04:23:24.889 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7411999999999999e-05 2023-11-25T04:23:24.889 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.889 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.889 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:24.889 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034465+0000", 2023-11-25T04:23:24.889 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011154699999999999 2023-11-25T04:23:24.889 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.889 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.889 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:24.889 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034560+0000", 2023-11-25T04:23:24.889 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4186999999999998e-05 2023-11-25T04:23:24.889 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.889 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.890 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:24.890 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.040134+0000", 2023-11-25T04:23:24.890 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055746019999999997 2023-11-25T04:23:24.890 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:24.890 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:24.890 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:24.890 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.890 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.890 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:9 3.10 e18/16 3:09401a81:::601.00000000:head v 18'4, mlcod=0'0)", 2023-11-25T04:23:24.890 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:24.890 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.772144347000001, 2023-11-25T04:23:24.890 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061641020000000003, 2023-11-25T04:23:24.891 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:24.891 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:24.891 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:24.891 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.891 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:24.891 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:24.891 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:24.891 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.891 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.891 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:24.891 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:24.891 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:24.891 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.892 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.892 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:24.892 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:24.892 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5300000000000001e-07 2023-11-25T04:23:24.892 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.892 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.892 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:24.892 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043222+0000", 2023-11-25T04:23:24.892 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.082e-05 2023-11-25T04:23:24.892 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.892 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.892 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:24.893 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043228+0000", 2023-11-25T04:23:24.893 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6699999999999997e-06 2023-11-25T04:23:24.893 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.893 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.893 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:24.893 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043236+0000", 2023-11-25T04:23:24.893 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8189999999999992e-06 2023-11-25T04:23:24.893 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.893 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.893 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:24.893 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043507+0000", 2023-11-25T04:23:24.893 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000271012 2023-11-25T04:23:24.893 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.894 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.894 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:24.894 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043530+0000", 2023-11-25T04:23:24.894 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2659000000000001e-05 2023-11-25T04:23:24.894 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.894 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.894 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:24.894 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049321+0000", 2023-11-25T04:23:24.894 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057915129999999999 2023-11-25T04:23:24.894 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.894 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.894 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:24.895 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049375+0000", 2023-11-25T04:23:24.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3455999999999998e-05 2023-11-25T04:23:24.895 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:24.895 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:24.895 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:24.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.895 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:1 3.14 e18/16 3:292cf221:::200.00000000:head v 18'3, mlcod=0'0)", 2023-11-25T04:23:24.895 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:24.895 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.772107634000001, 2023-11-25T04:23:24.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0060983599999999997, 2023-11-25T04:23:24.895 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:24.895 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:24.896 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:24.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.896 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:24.896 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:24.896 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:24.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.896 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:24.896 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:24.896 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:24.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.897 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:24.897 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043248+0000", 2023-11-25T04:23:24.897 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9399999999999995e-07 2023-11-25T04:23:24.897 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.897 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.897 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:24.897 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043252+0000", 2023-11-25T04:23:24.897 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1799999999999998e-06 2023-11-25T04:23:24.897 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.897 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.897 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:24.897 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043259+0000", 2023-11-25T04:23:24.897 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0049999999999998e-06 2023-11-25T04:23:24.898 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.898 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:24.898 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043265+0000", 2023-11-25T04:23:24.898 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0859999999999998e-06 2023-11-25T04:23:24.898 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.898 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:24.898 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043311+0000", 2023-11-25T04:23:24.898 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6082999999999999e-05 2023-11-25T04:23:24.898 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.898 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:24.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043330+0000", 2023-11-25T04:23:24.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8556e-05 2023-11-25T04:23:24.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.899 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:24.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049303+0000", 2023-11-25T04:23:24.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059733470000000004 2023-11-25T04:23:24.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.899 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:24.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049346+0000", 2023-11-25T04:23:24.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2608999999999998e-05 2023-11-25T04:23:24.899 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:24.900 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:24.900 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:24.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.900 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:3 3.14 e18/16 3:287034c4:::1.00000000.inode:head v 18'4, mlcod=0'0)", 2023-11-25T04:23:24.900 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:24.900 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.771591127999997, 2023-11-25T04:23:24.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057040090000000003, 2023-11-25T04:23:24.900 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:24.900 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:24.900 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:24.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.901 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:24.901 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:24.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:24.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.901 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:24.901 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:24.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:24.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.901 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:24.901 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:24.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9799999999999999e-07 2023-11-25T04:23:24.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.902 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:24.902 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043785+0000", 2023-11-25T04:23:24.902 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0721999999999999e-05 2023-11-25T04:23:24.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.902 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:24.902 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043793+0000", 2023-11-25T04:23:24.902 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5839999999999995e-06 2023-11-25T04:23:24.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.902 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:24.902 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043803+0000", 2023-11-25T04:23:24.903 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0729999999999994e-06 2023-11-25T04:23:24.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.903 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.903 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:24.903 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043843+0000", 2023-11-25T04:23:24.903 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0747999999999997e-05 2023-11-25T04:23:24.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.903 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.903 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:24.903 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043856+0000", 2023-11-25T04:23:24.903 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2697000000000001e-05 2023-11-25T04:23:24.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.903 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.903 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:24.904 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049426+0000", 2023-11-25T04:23:24.904 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055704409999999998 2023-11-25T04:23:24.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.904 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:24.904 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049468+0000", 2023-11-25T04:23:24.904 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1446000000000003e-05 2023-11-25T04:23:24.904 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:24.904 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:24.904 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:24.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.904 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "pg_update_log_missing(3.14 epoch 18/16 rep_tid 171 entries 18'5 (0'0) error 3:2e2fa760:::200.00000001:head by mds.0.3:32 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:23:24.904 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:24.905 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.771430164999998, 2023-11-25T04:23:24.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005556938, 2023-11-25T04:23:24.905 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:24.905 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2023-11-25T04:23:24.905 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:24.905 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.905 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:24.905 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:24.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:24.905 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.905 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.905 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:24.905 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:24.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:24.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.906 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.906 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:24.906 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:24.906 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.03e-07 2023-11-25T04:23:24.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.906 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.906 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:24.906 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043931+0000", 2023-11-25T04:23:24.906 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1079999999999998e-06 2023-11-25T04:23:24.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.906 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.906 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:24.907 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043944+0000", 2023-11-25T04:23:24.907 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2852e-05 2023-11-25T04:23:24.907 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.907 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.907 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:24.907 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043955+0000", 2023-11-25T04:23:24.907 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1037e-05 2023-11-25T04:23:24.907 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.907 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.907 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:24.907 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044088+0000", 2023-11-25T04:23:24.907 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013269599999999999 2023-11-25T04:23:24.907 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.907 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.908 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:24.908 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049482+0000", 2023-11-25T04:23:24.908 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053940419999999999 2023-11-25T04:23:24.908 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:24.908 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:24.908 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:24.908 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.908 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.908 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:10 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'5, mlcod=0'0)", 2023-11-25T04:23:24.908 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:24.908 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.771340137000003, 2023-11-25T04:23:24.908 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054493099999999997, 2023-11-25T04:23:24.908 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:24.909 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:24.909 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:24.909 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.909 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:24.909 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:24.909 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:24.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.909 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.909 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:24.909 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:24.909 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:24.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.910 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.910 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:24.910 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:24.910 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6100000000000002e-07 2023-11-25T04:23:24.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.910 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.910 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:24.910 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044019+0000", 2023-11-25T04:23:24.910 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.839e-06 2023-11-25T04:23:24.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.910 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.910 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:24.910 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044026+0000", 2023-11-25T04:23:24.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5869999999999998e-06 2023-11-25T04:23:24.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.911 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:24.911 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044031+0000", 2023-11-25T04:23:24.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4909999999999996e-06 2023-11-25T04:23:24.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.911 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:24.911 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044070+0000", 2023-11-25T04:23:24.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8829999999999999e-05 2023-11-25T04:23:24.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.912 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:24.912 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044081+0000", 2023-11-25T04:23:24.912 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1447999999999999e-05 2023-11-25T04:23:24.912 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.912 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:24.912 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049406+0000", 2023-11-25T04:23:24.912 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053246480000000004 2023-11-25T04:23:24.912 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.912 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:24.912 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049464+0000", 2023-11-25T04:23:24.913 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8205999999999998e-05 2023-11-25T04:23:24.913 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:24.913 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:24.913 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:24.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.913 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.913 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:11 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'6, mlcod=0'0)", 2023-11-25T04:23:24.913 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:24.913 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 37.771138872999998, 2023-11-25T04:23:24.913 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052673499999999996, 2023-11-25T04:23:24.913 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:24.913 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:24.913 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:24.914 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.914 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:24.914 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:24.914 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:24.914 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.914 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.914 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:24.914 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:24.914 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:24.914 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.914 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.914 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:24.914 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:24.915 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0199999999999998e-07 2023-11-25T04:23:24.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.915 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:24.915 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044220+0000", 2023-11-25T04:23:24.915 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7720000000000001e-06 2023-11-25T04:23:24.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.915 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:24.915 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044226+0000", 2023-11-25T04:23:24.915 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8579999999999996e-06 2023-11-25T04:23:24.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.916 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:24.916 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044232+0000", 2023-11-25T04:23:24.916 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1700000000000002e-06 2023-11-25T04:23:24.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.916 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:24.916 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044274+0000", 2023-11-25T04:23:24.916 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1254999999999997e-05 2023-11-25T04:23:24.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.916 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:24.916 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:23:24.917 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6880999999999999e-05 2023-11-25T04:23:24.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.917 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:24.917 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049466+0000", 2023-11-25T04:23:24.917 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051659219999999999 2023-11-25T04:23:24.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:24.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:24.917 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:24.917 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049483+0000", 2023-11-25T04:23:24.917 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.719e-05 2023-11-25T04:23:24.917 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:24.917 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:24.918 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:24.918 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:24.918 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:24.918 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:24.919 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:23:24.920 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:24.918+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:25.020 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:25.018+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:25.027 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:25.027 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:23:25.027 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2023-11-25T04:23:25.028 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:25.038 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:23:25.121 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:25.120+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:25.196 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:25.196 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:23:25.196 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2023-11-25T04:23:25.196 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2023-11-25T04:23:25.196 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:25.205 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:23:25.222 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:25.221+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:25.304 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2023-11-25T04:23:25.312 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:25.312 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2023-11-25T04:23:25.312 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2023-11-25T04:23:25.312 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2023-11-25T04:23:25.312 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.312 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:25.312 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:25.313 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 38.278278563000001, 2023-11-25T04:23:25.313 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01072943, 2023-11-25T04:23:25.313 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:25.313 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:25.313 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:25.313 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.313 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:25.313 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:25.313 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:25.313 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.313 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.313 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:25.313 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:25.313 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:25.314 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.314 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.314 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:25.314 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031829+0000", 2023-11-25T04:23:25.314 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.666e-06 2023-11-25T04:23:25.314 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.314 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.314 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:25.314 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031839+0000", 2023-11-25T04:23:25.314 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9490000000000008e-06 2023-11-25T04:23:25.314 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.314 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.314 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:25.314 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031857+0000", 2023-11-25T04:23:25.315 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8093000000000001e-05 2023-11-25T04:23:25.315 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.315 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.315 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:25.315 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031882+0000", 2023-11-25T04:23:25.315 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4294000000000002e-05 2023-11-25T04:23:25.315 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.315 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.315 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:25.315 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033130+0000", 2023-11-25T04:23:25.315 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001248264 2023-11-25T04:23:25.315 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.315 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.315 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:25.316 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033426+0000", 2023-11-25T04:23:25.316 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029620999999999998 2023-11-25T04:23:25.316 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.316 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.316 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:25.316 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033814+0000", 2023-11-25T04:23:25.316 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000387742 2023-11-25T04:23:25.316 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.316 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.316 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:25.316 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036793+0000", 2023-11-25T04:23:25.316 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002979598 2023-11-25T04:23:25.317 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.317 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.317 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:25.317 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042455+0000", 2023-11-25T04:23:25.317 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0056619260000000003 2023-11-25T04:23:25.317 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.317 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.317 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:25.317 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042502+0000", 2023-11-25T04:23:25.317 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7139999999999999e-05 2023-11-25T04:23:25.317 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.317 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.317 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:25.317 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042557+0000", 2023-11-25T04:23:25.318 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4548e-05 2023-11-25T04:23:25.318 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:25.318 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:25.318 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:25.318 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.318 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.318 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:25.318 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:25.318 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 38.277080028, 2023-11-25T04:23:25.318 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011505202000000001, 2023-11-25T04:23:25.318 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:25.318 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:25.318 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:25.319 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.319 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:25.319 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:25.319 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:25.319 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.319 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.319 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:25.319 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:25.319 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:25.319 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.319 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.319 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:25.319 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033027+0000", 2023-11-25T04:23:25.319 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0699999999999999e-06 2023-11-25T04:23:25.320 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.320 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.320 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:25.320 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033035+0000", 2023-11-25T04:23:25.320 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0830000000000004e-06 2023-11-25T04:23:25.320 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.320 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.320 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:25.320 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033059+0000", 2023-11-25T04:23:25.320 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3975e-05 2023-11-25T04:23:25.320 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.320 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.320 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:25.320 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033482+0000", 2023-11-25T04:23:25.321 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042253700000000001 2023-11-25T04:23:25.321 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.321 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.321 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:25.321 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034520+0000", 2023-11-25T04:23:25.321 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001037991 2023-11-25T04:23:25.321 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.321 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.321 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:25.321 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034611+0000", 2023-11-25T04:23:25.321 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1113999999999995e-05 2023-11-25T04:23:25.321 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.321 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.321 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:25.322 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034769+0000", 2023-11-25T04:23:25.322 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000157641 2023-11-25T04:23:25.322 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.322 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.322 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:25.322 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038292+0000", 2023-11-25T04:23:25.322 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035231160000000002 2023-11-25T04:23:25.322 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.322 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.322 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:25.322 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044440+0000", 2023-11-25T04:23:25.322 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0061486980000000002 2023-11-25T04:23:25.322 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.323 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.323 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:25.323 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044495+0000", 2023-11-25T04:23:25.323 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4339000000000003e-05 2023-11-25T04:23:25.323 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.323 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.323 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:25.323 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044531+0000", 2023-11-25T04:23:25.323 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6637999999999997e-05 2023-11-25T04:23:25.323 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:25.323 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:25.323 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:25.323 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.323 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.324 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:25.324 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:25.324 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 38.276605674000002, 2023-11-25T04:23:25.324 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0057634039999999997, 2023-11-25T04:23:25.324 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:25.324 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:25.324 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:25.324 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.324 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:25.324 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:25.324 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:25.324 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.324 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.324 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:25.325 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:25.325 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:25.325 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.325 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.325 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:25.325 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:25.325 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7900000000000007e-07 2023-11-25T04:23:25.325 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.325 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.325 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:25.325 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033507+0000", 2023-11-25T04:23:25.325 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8900000000000004e-06 2023-11-25T04:23:25.325 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.325 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.326 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:25.326 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033519+0000", 2023-11-25T04:23:25.326 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1756000000000001e-05 2023-11-25T04:23:25.326 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.326 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.326 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:25.326 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033527+0000", 2023-11-25T04:23:25.326 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3610000000000001e-06 2023-11-25T04:23:25.326 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.326 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.326 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:25.326 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034451+0000", 2023-11-25T04:23:25.326 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00092372400000000001 2023-11-25T04:23:25.327 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.327 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.327 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:25.327 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034809+0000", 2023-11-25T04:23:25.327 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00035757699999999998 2023-11-25T04:23:25.327 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.327 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.327 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:23:25.327 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034965+0000", 2023-11-25T04:23:25.327 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015638699999999999 2023-11-25T04:23:25.327 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.327 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.327 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:25.327 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038299+0000", 2023-11-25T04:23:25.328 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0033336559999999999 2023-11-25T04:23:25.328 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.328 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.328 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:25.328 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039169+0000", 2023-11-25T04:23:25.328 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00087066400000000005 2023-11-25T04:23:25.328 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.328 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.328 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:25.328 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039226+0000", 2023-11-25T04:23:25.328 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6511999999999997e-05 2023-11-25T04:23:25.328 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.328 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.329 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:25.329 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039264+0000", 2023-11-25T04:23:25.329 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7898000000000002e-05 2023-11-25T04:23:25.329 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:25.329 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:25.329 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:25.329 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.329 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.329 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:25 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:25.329 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:25.329 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 38.276527090000002, 2023-11-25T04:23:25.329 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096656169999999996, 2023-11-25T04:23:25.329 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:25.329 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:25.330 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:25.330 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.330 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:25.330 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:25.330 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:25.330 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.330 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.330 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:25.330 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:25.330 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:25.330 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.330 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.330 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:25.330 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033580+0000", 2023-11-25T04:23:25.331 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8199999999999999e-07 2023-11-25T04:23:25.331 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.331 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.331 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:25.331 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033585+0000", 2023-11-25T04:23:25.331 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3680000000000001e-06 2023-11-25T04:23:25.331 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.331 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.331 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:25.331 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033600+0000", 2023-11-25T04:23:25.331 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4331000000000001e-05 2023-11-25T04:23:25.331 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.331 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.332 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:25.332 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033606+0000", 2023-11-25T04:23:25.332 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5780000000000004e-06 2023-11-25T04:23:25.332 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.332 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.332 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:25.332 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035747+0000", 2023-11-25T04:23:25.332 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0021406530000000002 2023-11-25T04:23:25.332 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.332 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.332 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:25.332 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035901+0000", 2023-11-25T04:23:25.332 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000153931 2023-11-25T04:23:25.332 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.333 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.333 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:23:25.333 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036193+0000", 2023-11-25T04:23:25.333 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029211299999999999 2023-11-25T04:23:25.333 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.333 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.333 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:25.333 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038655+0000", 2023-11-25T04:23:25.333 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0024621640000000002 2023-11-25T04:23:25.333 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.333 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.333 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:25.333 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043130+0000", 2023-11-25T04:23:25.334 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.004474698 2023-11-25T04:23:25.334 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.334 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.334 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:25.334 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043204+0000", 2023-11-25T04:23:25.334 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4091999999999995e-05 2023-11-25T04:23:25.334 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.334 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.334 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:25.334 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043245+0000", 2023-11-25T04:23:25.334 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0907e-05 2023-11-25T04:23:25.334 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:25.334 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:25.335 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:25.335 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.335 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.335 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:26 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:25.335 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:25.335 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 38.276491266000001, 2023-11-25T04:23:25.335 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011086945000000001, 2023-11-25T04:23:25.335 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:25.335 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:25.335 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:25.335 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.335 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:25.335 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:25.336 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:25.336 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.336 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.336 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:25.336 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:25.336 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:25.336 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.336 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.336 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:25.336 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:25.336 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1699999999999998e-07 2023-11-25T04:23:25.336 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.336 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.337 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:25.337 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033627+0000", 2023-11-25T04:23:25.337 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.163e-05 2023-11-25T04:23:25.337 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.337 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.337 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:25.337 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033636+0000", 2023-11-25T04:23:25.337 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6859999999999995e-06 2023-11-25T04:23:25.337 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.337 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.337 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:25.337 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033642+0000", 2023-11-25T04:23:25.337 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3149999999999997e-06 2023-11-25T04:23:25.337 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.338 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.338 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:25.338 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035414+0000", 2023-11-25T04:23:25.338 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0017716380000000001 2023-11-25T04:23:25.338 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.338 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.338 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:25.338 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035539+0000", 2023-11-25T04:23:25.338 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000125477 2023-11-25T04:23:25.338 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.338 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.338 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:25.338 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035672+0000", 2023-11-25T04:23:25.339 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000133316 2023-11-25T04:23:25.339 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.339 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.339 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:25.339 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038649+0000", 2023-11-25T04:23:25.339 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029764679999999999 2023-11-25T04:23:25.339 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.339 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.339 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:25.339 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044578+0000", 2023-11-25T04:23:25.339 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0059293510000000002 2023-11-25T04:23:25.339 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.339 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.340 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:25.340 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044651+0000", 2023-11-25T04:23:25.340 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2513000000000002e-05 2023-11-25T04:23:25.340 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.340 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.340 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:25.340 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044702+0000", 2023-11-25T04:23:25.340 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1034e-05 2023-11-25T04:23:25.340 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:25.340 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:25.340 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:25.340 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.340 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.340 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:29 3.b 3:d0630e4c:::mds0_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:25.341 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:25.341 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 38.276453345999997, 2023-11-25T04:23:25.341 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010980227, 2023-11-25T04:23:25.341 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:25.341 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:25.341 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:25.341 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.341 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:25.341 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:25.341 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:25.341 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.341 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.341 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:25.342 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:25.342 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:25.342 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.342 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.342 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:25.342 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:25.342 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3499999999999996e-07 2023-11-25T04:23:25.342 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.342 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.342 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:25.342 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033658+0000", 2023-11-25T04:23:25.342 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1280000000000001e-06 2023-11-25T04:23:25.342 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.343 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.343 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:25.343 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033667+0000", 2023-11-25T04:23:25.343 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1430000000000004e-06 2023-11-25T04:23:25.343 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.343 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.343 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:25.343 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033678+0000", 2023-11-25T04:23:25.343 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0906e-05 2023-11-25T04:23:25.343 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.343 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.343 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:25.343 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034456+0000", 2023-11-25T04:23:25.343 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00077833899999999996 2023-11-25T04:23:25.344 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.344 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.344 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:25.344 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034575+0000", 2023-11-25T04:23:25.344 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011926000000000001 2023-11-25T04:23:25.344 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.344 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.344 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:25.344 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035001+0000", 2023-11-25T04:23:25.344 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042550999999999998 2023-11-25T04:23:25.344 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.344 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.344 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:25.344 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.037923+0000", 2023-11-25T04:23:25.345 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029219670000000001 2023-11-25T04:23:25.345 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.345 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.345 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:25.345 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044538+0000", 2023-11-25T04:23:25.345 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0066152349999999997 2023-11-25T04:23:25.345 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.345 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.345 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:25.345 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044594+0000", 2023-11-25T04:23:25.345 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5593000000000001e-05 2023-11-25T04:23:25.345 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.345 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.345 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:25.346 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044633+0000", 2023-11-25T04:23:25.346 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9511000000000002e-05 2023-11-25T04:23:25.346 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:25.346 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:25.346 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:25.346 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.346 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.346 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:34 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:25.346 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:25.346 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 38.276203232999997, 2023-11-25T04:23:25.346 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010920445000000001, 2023-11-25T04:23:25.346 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:25.346 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:25.347 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:25.347 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.347 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:25.347 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:25.347 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:25.347 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.347 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.347 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:25.347 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:25.347 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:25.347 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.347 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.347 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:25.348 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033904+0000", 2023-11-25T04:23:25.348 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6599999999999994e-07 2023-11-25T04:23:25.348 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.348 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.348 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:25.348 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033909+0000", 2023-11-25T04:23:25.348 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5639999999999996e-06 2023-11-25T04:23:25.348 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.348 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.348 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:25.348 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033923+0000", 2023-11-25T04:23:25.348 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3028e-05 2023-11-25T04:23:25.348 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.349 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.349 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:25.349 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034043+0000", 2023-11-25T04:23:25.349 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000120328 2023-11-25T04:23:25.349 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.349 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.349 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:25.349 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035576+0000", 2023-11-25T04:23:25.349 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015333720000000001 2023-11-25T04:23:25.349 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.349 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.349 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:25.349 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035699+0000", 2023-11-25T04:23:25.349 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000123001 2023-11-25T04:23:25.350 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.350 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.350 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:25.350 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044823+0000", 2023-11-25T04:23:25.350 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0091241859999999994 2023-11-25T04:23:25.350 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:25.350 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:25.350 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:25.350 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.350 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.350 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:35 3.1f 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:25.351 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:25.351 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 38.274899028, 2023-11-25T04:23:25.351 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096164779999999995, 2023-11-25T04:23:25.351 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:25.351 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:25.351 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:25.351 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.351 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:25.351 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:25.351 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:25.351 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.351 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.351 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:25.352 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:25.352 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:25.352 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.352 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.352 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:25.352 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035208+0000", 2023-11-25T04:23:25.352 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2699999999999998e-07 2023-11-25T04:23:25.352 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.352 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.352 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:25.352 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035213+0000", 2023-11-25T04:23:25.352 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.6070000000000001e-06 2023-11-25T04:23:25.352 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.352 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.353 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:25.353 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035350+0000", 2023-11-25T04:23:25.353 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013693599999999999 2023-11-25T04:23:25.353 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.353 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.353 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:25.354 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035358+0000", 2023-11-25T04:23:25.354 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0099999999999995e-06 2023-11-25T04:23:25.354 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.354 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.354 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:25.354 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036260+0000", 2023-11-25T04:23:25.354 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00090240100000000005 2023-11-25T04:23:25.354 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.354 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.354 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:25.355 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036517+0000", 2023-11-25T04:23:25.355 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00025736299999999998 2023-11-25T04:23:25.355 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.355 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.355 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:25.355 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044824+0000", 2023-11-25T04:23:25.355 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0083063340000000003 2023-11-25T04:23:25.356 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:25.356 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:25.356 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:25.356 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.356 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.356 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:37 3.1e 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:25.356 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:25.356 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 38.274732342, 2023-11-25T04:23:25.356 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0078322640000000002, 2023-11-25T04:23:25.356 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:25.356 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:25.356 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:25.357 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.357 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:25.357 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:25.357 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:25.357 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.357 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.357 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:25.357 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:25.357 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:25.357 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.357 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.357 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:25.358 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035375+0000", 2023-11-25T04:23:25.358 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9100000000000003e-07 2023-11-25T04:23:25.358 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.358 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.358 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:25.358 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035379+0000", 2023-11-25T04:23:25.358 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9430000000000002e-06 2023-11-25T04:23:25.358 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.358 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.358 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:25.358 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035388+0000", 2023-11-25T04:23:25.358 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2760000000000006e-06 2023-11-25T04:23:25.359 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.359 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.359 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:25.359 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035396+0000", 2023-11-25T04:23:25.359 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8480000000000001e-06 2023-11-25T04:23:25.359 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.359 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.359 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:25.359 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035452+0000", 2023-11-25T04:23:25.359 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5831000000000001e-05 2023-11-25T04:23:25.359 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.359 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.359 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:25.360 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035934+0000", 2023-11-25T04:23:25.360 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00048265599999999998 2023-11-25T04:23:25.360 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.360 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.360 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:25.360 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043206+0000", 2023-11-25T04:23:25.360 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.007271919 2023-11-25T04:23:25.360 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:25.360 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:25.360 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:25.360 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.360 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.361 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing(3.1a epoch 18/16 rep_tid 3 entries 18'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:36 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:23:25.361 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:25.361 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 38.273456756000002, 2023-11-25T04:23:25.361 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002073414, 2023-11-25T04:23:25.361 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:25.361 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:25.361 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:25.361 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.361 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:25.361 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:25.361 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:25.361 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.362 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.362 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:25.362 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:25.362 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:25.362 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.362 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.362 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:25.362 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036650+0000", 2023-11-25T04:23:25.362 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7300000000000002e-07 2023-11-25T04:23:25.362 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.362 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.362 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:25.363 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036659+0000", 2023-11-25T04:23:25.363 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.7499999999999992e-06 2023-11-25T04:23:25.363 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.363 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.363 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:25.363 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036673+0000", 2023-11-25T04:23:25.363 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4917e-05 2023-11-25T04:23:25.363 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.363 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.363 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:25.363 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036687+0000", 2023-11-25T04:23:25.363 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3883000000000001e-05 2023-11-25T04:23:25.363 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.363 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.364 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:25.364 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036781+0000", 2023-11-25T04:23:25.364 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3219000000000003e-05 2023-11-25T04:23:25.364 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.364 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.364 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:25.364 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038723+0000", 2023-11-25T04:23:25.364 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019422720000000001 2023-11-25T04:23:25.364 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:25.364 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:25.364 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:25.364 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.364 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.365 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:28 3.13 e18/16 3:cd5a64a3:::100.00000000:head v 18'1, mlcod=0'0)", 2023-11-25T04:23:25.365 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:25.365 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 38.269795598000002, 2023-11-25T04:23:25.365 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001850546, 2023-11-25T04:23:25.365 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:25.365 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:25.365 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:25.365 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.365 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:25.365 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:25.365 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:25.365 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.366 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.366 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:25.366 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:25.366 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:25.366 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.366 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.366 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:25.366 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:25.366 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7800000000000002e-07 2023-11-25T04:23:25.366 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.366 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.366 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:25.367 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040331+0000", 2023-11-25T04:23:25.367 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0449999999999999e-05 2023-11-25T04:23:25.367 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.367 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.367 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:25.367 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040339+0000", 2023-11-25T04:23:25.367 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0709999999999995e-06 2023-11-25T04:23:25.367 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.367 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.367 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:25.367 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040347+0000", 2023-11-25T04:23:25.367 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7079999999999997e-06 2023-11-25T04:23:25.367 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.368 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.368 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:25.368 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040410+0000", 2023-11-25T04:23:25.368 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2453000000000007e-05 2023-11-25T04:23:25.368 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.368 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.368 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:25.368 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040422+0000", 2023-11-25T04:23:25.368 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2429000000000001e-05 2023-11-25T04:23:25.368 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.368 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.368 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:25.368 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042129+0000", 2023-11-25T04:23:25.369 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001707266 2023-11-25T04:23:25.369 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.369 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.369 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:25.369 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042161+0000", 2023-11-25T04:23:25.369 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1791000000000001e-05 2023-11-25T04:23:25.369 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:25.369 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:25.369 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:25.369 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.369 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.369 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:27 3.13 e18/16 3:c91d4a1d:::mds0_inotable:head v 18'2, mlcod=0'0)", 2023-11-25T04:23:25.369 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:25.370 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 38.269592074999998, 2023-11-25T04:23:25.370 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0016886430000000001, 2023-11-25T04:23:25.370 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:25.370 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:25.370 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:25.370 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.370 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:25.370 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:25.370 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:25.370 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.370 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.370 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:25.371 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:25.371 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:25.371 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.371 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.371 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:25.371 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:25.371 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3999999999999997e-07 2023-11-25T04:23:25.371 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.371 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.371 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:25.371 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040520+0000", 2023-11-25T04:23:25.371 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2989999999999996e-06 2023-11-25T04:23:25.372 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.372 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.372 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:25.372 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040527+0000", 2023-11-25T04:23:25.372 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5730000000000002e-06 2023-11-25T04:23:25.372 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.372 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.372 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:25.372 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040535+0000", 2023-11-25T04:23:25.372 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8630000000000008e-06 2023-11-25T04:23:25.372 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.372 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.372 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:25.373 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040683+0000", 2023-11-25T04:23:25.373 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000148002 2023-11-25T04:23:25.373 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.373 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.373 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:25.373 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040700+0000", 2023-11-25T04:23:25.373 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7065e-05 2023-11-25T04:23:25.373 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.373 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.373 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:25.373 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042163+0000", 2023-11-25T04:23:25.373 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001462767 2023-11-25T04:23:25.373 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.374 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.374 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:25.374 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042203+0000", 2023-11-25T04:23:25.374 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9734000000000003e-05 2023-11-25T04:23:25.374 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:25.374 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:25.374 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:25.374 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.374 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.374 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:20 3.1c e18/16 3:3ed09add:::607.00000000:head v 18'1, mlcod=0'0)", 2023-11-25T04:23:25.374 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:25.374 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 38.269557034000002, 2023-11-25T04:23:25.375 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015271, 2023-11-25T04:23:25.375 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:25.375 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:25.375 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:25.375 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.375 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:25.375 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:25.375 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:25.375 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.375 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.375 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:25.375 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:25.375 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:25.376 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.376 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.376 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:25.376 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:25.376 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4799999999999999e-07 2023-11-25T04:23:25.376 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.376 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.376 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:25.376 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040561+0000", 2023-11-25T04:23:25.376 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1800000000000001e-05 2023-11-25T04:23:25.376 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.376 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.376 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:25.377 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040568+0000", 2023-11-25T04:23:25.377 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7730000000000002e-06 2023-11-25T04:23:25.377 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.377 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.377 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:25.377 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040587+0000", 2023-11-25T04:23:25.377 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8691000000000001e-05 2023-11-25T04:23:25.377 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.377 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.377 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:25.377 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040638+0000", 2023-11-25T04:23:25.377 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1068e-05 2023-11-25T04:23:25.377 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.378 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.378 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:25.378 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040670+0000", 2023-11-25T04:23:25.378 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2125999999999997e-05 2023-11-25T04:23:25.378 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.378 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.378 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:25.378 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042032+0000", 2023-11-25T04:23:25.378 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001361821 2023-11-25T04:23:25.378 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.378 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.378 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:25.378 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042076+0000", 2023-11-25T04:23:25.379 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4472999999999999e-05 2023-11-25T04:23:25.379 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:25.379 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:25.379 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:25.379 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.379 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.379 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:21 3.1c e18/16 3:3ed09add:::607.00000000:head v 18'2, mlcod=0'0)", 2023-11-25T04:23:25.379 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:25.379 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 38.269322309000003, 2023-11-25T04:23:25.379 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0013228160000000001, 2023-11-25T04:23:25.379 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:25.379 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:25.379 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:25.380 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.380 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:25.380 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:25.380 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:25.380 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.380 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.380 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:25.380 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:25.380 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:25.380 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.380 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.380 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:25.380 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:25.381 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9299999999999999e-07 2023-11-25T04:23:25.381 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.381 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.381 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:25.381 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040790+0000", 2023-11-25T04:23:25.381 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5770000000000001e-06 2023-11-25T04:23:25.381 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.381 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.381 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:25.381 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040799+0000", 2023-11-25T04:23:25.381 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6439999999999995e-06 2023-11-25T04:23:25.381 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.381 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.382 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:25.382 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040806+0000", 2023-11-25T04:23:25.382 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0180000000000004e-06 2023-11-25T04:23:25.382 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.382 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.382 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:25.382 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040907+0000", 2023-11-25T04:23:25.382 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000100096 2023-11-25T04:23:25.382 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.382 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.382 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:25.382 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040925+0000", 2023-11-25T04:23:25.383 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8536999999999999e-05 2023-11-25T04:23:25.383 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.383 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.383 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:25.383 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042079+0000", 2023-11-25T04:23:25.383 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00115358 2023-11-25T04:23:25.383 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.383 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.383 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:25.383 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042107+0000", 2023-11-25T04:23:25.383 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7971e-05 2023-11-25T04:23:25.383 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:25.384 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:25.384 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:25.384 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.384 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.384 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop_reply(mds.0.3:4 3.1f e18/16 ondisk, result = 0)", 2023-11-25T04:23:25.384 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:25.384 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 38.267982918999998, 2023-11-25T04:23:25.384 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00044092900000000001, 2023-11-25T04:23:25.384 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:25.384 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:25.384 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:25.384 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.384 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:25.385 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:25.385 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:25.385 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.385 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.385 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:25.385 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:25.385 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:25.385 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.385 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.385 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:25.385 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042124+0000", 2023-11-25T04:23:25.385 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1800000000000005e-07 2023-11-25T04:23:25.385 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.386 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.386 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:25.386 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042128+0000", 2023-11-25T04:23:25.386 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2289999999999997e-06 2023-11-25T04:23:25.386 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.386 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.386 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:25.386 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042138+0000", 2023-11-25T04:23:25.386 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9729999999999992e-06 2023-11-25T04:23:25.386 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.386 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.386 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:25.386 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042145+0000", 2023-11-25T04:23:25.387 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.336e-06 2023-11-25T04:23:25.387 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.387 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.387 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:25.387 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042441+0000", 2023-11-25T04:23:25.388 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029650500000000002 2023-11-25T04:23:25.388 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.388 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.388 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:25.388 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042448+0000", 2023-11-25T04:23:25.388 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4100000000000002e-06 2023-11-25T04:23:25.388 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.389 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.389 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:25.389 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042564+0000", 2023-11-25T04:23:25.389 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000115758 2023-11-25T04:23:25.389 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:25.389 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:25.389 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:25.389 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.389 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.389 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing_reply(3.1f epoch 18/16 rep_tid 8 lcod 18'4)", 2023-11-25T04:23:25.389 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:25.389 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 38.265657507, 2023-11-25T04:23:25.389 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00038154699999999998, 2023-11-25T04:23:25.390 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:25.390 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:25.390 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:25.390 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.390 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:25.390 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:25.390 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:25.390 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.390 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.390 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:25.390 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:25.390 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:25.390 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.391 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.391 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:25.391 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:25.391 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2700000000000001e-07 2023-11-25T04:23:25.391 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.391 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.391 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:25.391 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044451+0000", 2023-11-25T04:23:25.391 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7460000000000001e-06 2023-11-25T04:23:25.391 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.391 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.391 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:25.391 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044455+0000", 2023-11-25T04:23:25.392 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5809999999999998e-06 2023-11-25T04:23:25.392 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.392 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.392 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:25.392 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044458+0000", 2023-11-25T04:23:25.392 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0170000000000001e-06 2023-11-25T04:23:25.392 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.392 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.392 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:25.392 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044734+0000", 2023-11-25T04:23:25.392 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00027582799999999999 2023-11-25T04:23:25.392 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.393 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.393 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:25.393 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044830+0000", 2023-11-25T04:23:25.393 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6248000000000004e-05 2023-11-25T04:23:25.393 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:25.393 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:25.393 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:25.393 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.393 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.393 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing_reply(3.b epoch 18/16 rep_tid 5 lcod 18'2)", 2023-11-25T04:23:25.393 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:25.393 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 38.265620845999997, 2023-11-25T04:23:25.393 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000344488, 2023-11-25T04:23:25.394 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:25.394 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:25.394 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:25.394 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.394 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:25.394 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:25.394 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:25.394 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.394 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.394 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:25.394 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:25.394 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:25.394 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.395 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.395 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:25.395 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:25.395 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.55e-07 2023-11-25T04:23:25.395 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.395 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.395 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:25.395 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044487+0000", 2023-11-25T04:23:25.395 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2470000000000001e-06 2023-11-25T04:23:25.395 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.395 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.395 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:25.395 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044491+0000", 2023-11-25T04:23:25.396 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9659999999999998e-06 2023-11-25T04:23:25.396 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.396 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.396 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:25.396 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044494+0000", 2023-11-25T04:23:25.396 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2310000000000001e-06 2023-11-25T04:23:25.396 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.396 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.396 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:25.396 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044756+0000", 2023-11-25T04:23:25.396 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00026172800000000003 2023-11-25T04:23:25.396 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.396 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.397 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:25.397 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044830+0000", 2023-11-25T04:23:25.397 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4160999999999997e-05 2023-11-25T04:23:25.397 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:25.397 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:25.397 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:25.397 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.397 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.397 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:39 3.6 3:654134d2:::mds0_openfiles.0:head [omap-clear,omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2023-11-25T04:23:25.397 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:25.397 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 33.205794804, 2023-11-25T04:23:25.397 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001788658, 2023-11-25T04:23:25.397 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:25.398 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:25.398 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:25.398 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.398 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:25.398 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:25.398 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:25.398 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.398 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.398 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:25.398 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:25.398 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:25.398 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.398 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.399 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:25.399 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104314+0000", 2023-11-25T04:23:25.399 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3099999999999999e-06 2023-11-25T04:23:25.399 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.399 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.399 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:25.399 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104325+0000", 2023-11-25T04:23:25.399 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1511999999999999e-05 2023-11-25T04:23:25.399 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.399 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.399 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:25.399 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104362+0000", 2023-11-25T04:23:25.399 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6903000000000002e-05 2023-11-25T04:23:25.400 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.400 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.400 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:25.400 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104387+0000", 2023-11-25T04:23:25.400 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4766e-05 2023-11-25T04:23:25.400 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.400 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.400 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:25.400 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104465+0000", 2023-11-25T04:23:25.400 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8119000000000002e-05 2023-11-25T04:23:25.400 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.400 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.400 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:25.401 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104557+0000", 2023-11-25T04:23:25.401 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2496000000000003e-05 2023-11-25T04:23:25.401 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.401 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.401 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:25.401 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104646+0000", 2023-11-25T04:23:25.401 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8818000000000002e-05 2023-11-25T04:23:25.401 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.401 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.401 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:25.401 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105201+0000", 2023-11-25T04:23:25.401 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00055506000000000002 2023-11-25T04:23:25.401 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.402 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.402 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:25.402 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105994+0000", 2023-11-25T04:23:25.402 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00079294700000000001 2023-11-25T04:23:25.402 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.402 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.402 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:25.402 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.106055+0000", 2023-11-25T04:23:25.402 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0566000000000003e-05 2023-11-25T04:23:25.402 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.402 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.403 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:25.403 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.106100+0000", 2023-11-25T04:23:25.403 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5160999999999997e-05 2023-11-25T04:23:25.403 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:25.403 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:25.403 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:25.403 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.403 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.403 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:43 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2023-11-25T04:23:25.403 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:25.403 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 18.205569808, 2023-11-25T04:23:25.403 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001751166, 2023-11-25T04:23:25.404 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:25.404 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:25.404 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:25.404 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.404 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:25.404 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:25.404 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:25.404 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.404 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.404 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:25.404 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:25.404 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:25.404 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.405 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.405 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:25.405 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104539+0000", 2023-11-25T04:23:25.405 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9529999999999999e-06 2023-11-25T04:23:25.405 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.405 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.405 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:25.405 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104557+0000", 2023-11-25T04:23:25.405 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7464000000000001e-05 2023-11-25T04:23:25.405 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.405 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.405 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:25.405 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104590+0000", 2023-11-25T04:23:25.406 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2966000000000003e-05 2023-11-25T04:23:25.406 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.406 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.406 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:25.406 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104608+0000", 2023-11-25T04:23:25.406 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8193e-05 2023-11-25T04:23:25.406 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.406 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.406 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:25.406 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104767+0000", 2023-11-25T04:23:25.406 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015882 2023-11-25T04:23:25.406 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.407 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.407 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:25.407 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104909+0000", 2023-11-25T04:23:25.407 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014244999999999999 2023-11-25T04:23:25.407 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.407 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.407 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:25.407 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.105059+0000", 2023-11-25T04:23:25.407 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014964399999999999 2023-11-25T04:23:25.407 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.407 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.407 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:25.407 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.105701+0000", 2023-11-25T04:23:25.408 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00064201099999999995 2023-11-25T04:23:25.408 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.408 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.408 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:25.408 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106233+0000", 2023-11-25T04:23:25.408 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00053178500000000001 2023-11-25T04:23:25.408 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.408 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.408 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:25.408 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106265+0000", 2023-11-25T04:23:25.408 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1986000000000002e-05 2023-11-25T04:23:25.408 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.408 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.409 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:25.409 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106288+0000", 2023-11-25T04:23:25.409 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2894000000000001e-05 2023-11-25T04:23:25.409 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:25.409 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:25.409 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:25.409 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.409 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.409 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:45 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e23)", 2023-11-25T04:23:25.409 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:23:25.409 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 13.205371767999999, 2023-11-25T04:23:25.409 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00098423299999999994, 2023-11-25T04:23:25.409 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:25.409 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:25.410 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:25.410 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.410 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:25.410 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:23:25.410 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:25.410 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.410 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.410 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:25.410 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:23:25.410 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:25.410 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.410 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.411 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:25.411 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104737+0000", 2023-11-25T04:23:25.411 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3309999999999998e-06 2023-11-25T04:23:25.411 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.411 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.411 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:25.411 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104748+0000", 2023-11-25T04:23:25.411 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1080999999999999e-05 2023-11-25T04:23:25.411 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.411 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.411 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:25.411 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104771+0000", 2023-11-25T04:23:25.411 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2707000000000001e-05 2023-11-25T04:23:25.411 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.412 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.412 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:25.412 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104800+0000", 2023-11-25T04:23:25.412 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9595e-05 2023-11-25T04:23:25.412 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.412 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.412 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:25.412 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104890+0000", 2023-11-25T04:23:25.412 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.9808999999999993e-05 2023-11-25T04:23:25.412 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.412 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.412 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:25.412 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105004+0000", 2023-11-25T04:23:25.413 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000114508 2023-11-25T04:23:25.413 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.413 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.413 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:25.413 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105634+0000", 2023-11-25T04:23:25.413 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00062985799999999996 2023-11-25T04:23:25.413 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.413 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.413 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:25.413 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105671+0000", 2023-11-25T04:23:25.413 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6640000000000002e-05 2023-11-25T04:23:25.413 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:25.413 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:25.413 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:25.414 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105719+0000", 2023-11-25T04:23:25.414 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7704000000000003e-05 2023-11-25T04:23:25.414 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:25.414 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:25.414 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:25.414 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:25.414 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:25.414 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:25.417 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:23:25.417 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:25.414+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:25.517 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:25.516+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:25.524 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:25.524 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:23:25.524 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2023-11-25T04:23:25.524 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:25.533 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:23:25.618 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:25.617+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:25.633 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-11-25T04:23:25.636 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:25.637 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:23:25.637 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2023-11-25T04:23:25.637 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2023-11-25T04:23:25.637 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:25.646 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:23:25.719 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:25.717+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:25.748 INFO:teuthology.orchestra.run.smithi067.stderr:Error ENXIO: problem getting command descriptions from osd.0 2023-11-25T04:23:25.752 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:25.752 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2023-11-25T04:23:25.753 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2023-11-25T04:23:25.753 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2023-11-25T04:23:25.753 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.753 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:155 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'109, mlcod=11'105)", 2023-11-25T04:23:25.753 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:25.753 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.965570235000001, 2023-11-25T04:23:25.753 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061719019999999999, 2023-11-25T04:23:25.753 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:25.753 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:25.753 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:25.753 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.754 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:25.754 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:25.754 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:25.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.754 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:25.754 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:25.754 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:25.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.754 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:25.754 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784884+0000", 2023-11-25T04:23:25.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3099999999999999e-07 2023-11-25T04:23:25.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:25.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784942+0000", 2023-11-25T04:23:25.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8570000000000003e-05 2023-11-25T04:23:25.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:25.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784949+0000", 2023-11-25T04:23:25.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0310000000000001e-06 2023-11-25T04:23:25.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.756 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.756 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:25.756 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784953+0000", 2023-11-25T04:23:25.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6320000000000001e-06 2023-11-25T04:23:25.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.756 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.756 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:25.756 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784993+0000", 2023-11-25T04:23:25.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9737000000000003e-05 2023-11-25T04:23:25.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.756 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.757 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:25.757 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785011+0000", 2023-11-25T04:23:25.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8349999999999999e-05 2023-11-25T04:23:25.757 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.757 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.757 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:25.757 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791024+0000", 2023-11-25T04:23:25.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006012997 2023-11-25T04:23:25.757 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.757 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.757 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:25.757 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791055+0000", 2023-11-25T04:23:25.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1254000000000003e-05 2023-11-25T04:23:25.758 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:25.758 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:25.758 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:25.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.758 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:156 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'110, mlcod=11'105)", 2023-11-25T04:23:25.758 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:25.758 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.964984186000002, 2023-11-25T04:23:25.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005616116, 2023-11-25T04:23:25.758 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:25.758 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:25.759 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:25.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:25.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:25.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:25.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:25.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:25.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:25.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:25.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785470+0000", 2023-11-25T04:23:25.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0699999999999998e-07 2023-11-25T04:23:25.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:25.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785474+0000", 2023-11-25T04:23:25.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.048e-06 2023-11-25T04:23:25.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:25.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785480+0000", 2023-11-25T04:23:25.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7729999999999996e-06 2023-11-25T04:23:25.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:25.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785483+0000", 2023-11-25T04:23:25.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1970000000000001e-06 2023-11-25T04:23:25.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:25.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787936+0000", 2023-11-25T04:23:25.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024529040000000001 2023-11-25T04:23:25.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.762 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:25.762 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787947+0000", 2023-11-25T04:23:25.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1669e-05 2023-11-25T04:23:25.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.762 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:25.762 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791057+0000", 2023-11-25T04:23:25.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031095200000000002 2023-11-25T04:23:25.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.763 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:25.763 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791086+0000", 2023-11-25T04:23:25.763 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8697999999999999e-05 2023-11-25T04:23:25.763 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:25.763 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:25.763 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:25.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.763 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:157 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'111, mlcod=11'106)", 2023-11-25T04:23:25.763 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:25.764 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.964391628000001, 2023-11-25T04:23:25.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050590280000000001, 2023-11-25T04:23:25.764 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:25.764 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:25.764 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:25.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:25.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:25.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:25.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:25.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:25.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:25.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:25.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786063+0000", 2023-11-25T04:23:25.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2600000000000002e-07 2023-11-25T04:23:25.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:25.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786069+0000", 2023-11-25T04:23:25.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8839999999999999e-06 2023-11-25T04:23:25.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.766 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:25.766 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786079+0000", 2023-11-25T04:23:25.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0124e-05 2023-11-25T04:23:25.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.766 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:25.766 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786085+0000", 2023-11-25T04:23:25.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6660000000000002e-06 2023-11-25T04:23:25.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.767 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:25.767 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788160+0000", 2023-11-25T04:23:25.767 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020747669999999999 2023-11-25T04:23:25.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.767 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:25.767 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788180+0000", 2023-11-25T04:23:25.767 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0208999999999999e-05 2023-11-25T04:23:25.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.767 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:25.768 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791087+0000", 2023-11-25T04:23:25.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029064630000000002 2023-11-25T04:23:25.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.768 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:25.768 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791121+0000", 2023-11-25T04:23:25.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4289000000000003e-05 2023-11-25T04:23:25.768 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:25.768 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:25.768 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:25.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.769 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:158 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'112, mlcod=11'106)", 2023-11-25T04:23:25.769 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:25.769 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.963975234999999, 2023-11-25T04:23:25.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046663030000000001, 2023-11-25T04:23:25.769 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:25.769 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:25.769 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:25.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:25.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:25.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:25.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:25.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:25.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:25.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:25.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786479+0000", 2023-11-25T04:23:25.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2799999999999996e-07 2023-11-25T04:23:25.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.771 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:25.771 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786498+0000", 2023-11-25T04:23:25.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9069e-05 2023-11-25T04:23:25.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.771 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:25.771 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786508+0000", 2023-11-25T04:23:25.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5810000000000003e-06 2023-11-25T04:23:25.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.771 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:25.772 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786512+0000", 2023-11-25T04:23:25.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4889999999999997e-06 2023-11-25T04:23:25.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.772 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.772 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:25.772 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788438+0000", 2023-11-25T04:23:25.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019257059999999999 2023-11-25T04:23:25.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.772 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.772 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:25.772 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788449+0000", 2023-11-25T04:23:25.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1419e-05 2023-11-25T04:23:25.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.773 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:25.773 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791122+0000", 2023-11-25T04:23:25.773 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002672822 2023-11-25T04:23:25.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.773 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:25.773 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791145+0000", 2023-11-25T04:23:25.773 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2589e-05 2023-11-25T04:23:25.773 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:25.773 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:25.774 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:25.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.774 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:206 1.0 e12/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 12'152, mlcod=12'148)", 2023-11-25T04:23:25.774 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:25.774 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.452206765, 2023-11-25T04:23:25.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046728189999999999, 2023-11-25T04:23:25.774 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:25.774 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:25.775 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:25.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:25.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:25.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:25.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:25.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:25.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:25.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:25.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298248+0000", 2023-11-25T04:23:25.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3600000000000003e-07 2023-11-25T04:23:25.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:25.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298278+0000", 2023-11-25T04:23:25.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0474e-05 2023-11-25T04:23:25.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.777 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:25.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298288+0000", 2023-11-25T04:23:25.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0156000000000001e-05 2023-11-25T04:23:25.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.777 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:25.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298293+0000", 2023-11-25T04:23:25.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7400000000000004e-06 2023-11-25T04:23:25.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:25.778 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298333+0000", 2023-11-25T04:23:25.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9666000000000003e-05 2023-11-25T04:23:25.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:25.778 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298351+0000", 2023-11-25T04:23:25.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8544999999999999e-05 2023-11-25T04:23:25.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:25.779 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302891+0000", 2023-11-25T04:23:25.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045398519999999996 2023-11-25T04:23:25.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:25.779 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302920+0000", 2023-11-25T04:23:25.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8750000000000001e-05 2023-11-25T04:23:25.779 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:25.779 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:25.779 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:25.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.780 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:14 3.15 3:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:25.780 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:25.780 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 38.717072408, 2023-11-25T04:23:25.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010990437, 2023-11-25T04:23:25.780 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:25.780 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:25.780 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:25.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.780 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:25.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:25.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:25.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:25.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:25.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:25.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:25.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033383+0000", 2023-11-25T04:23:25.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9870000000000002e-06 2023-11-25T04:23:25.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:25.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033398+0000", 2023-11-25T04:23:25.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4569999999999999e-05 2023-11-25T04:23:25.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:25.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033411+0000", 2023-11-25T04:23:25.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3543999999999999e-05 2023-11-25T04:23:25.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:25.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033423+0000", 2023-11-25T04:23:25.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1905999999999999e-05 2023-11-25T04:23:25.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:25.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034433+0000", 2023-11-25T04:23:25.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00101022 2023-11-25T04:23:25.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:25.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034539+0000", 2023-11-25T04:23:25.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010597199999999999 2023-11-25T04:23:25.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:25.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034667+0000", 2023-11-25T04:23:25.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012765500000000001 2023-11-25T04:23:25.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:25.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037803+0000", 2023-11-25T04:23:25.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031362220000000001 2023-11-25T04:23:25.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.785 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:25.785 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:23:25.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0064964669999999997 2023-11-25T04:23:25.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.785 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:25.785 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044331+0000", 2023-11-25T04:23:25.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1544999999999997e-05 2023-11-25T04:23:25.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:25.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044372+0000", 2023-11-25T04:23:25.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0349000000000003e-05 2023-11-25T04:23:25.786 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:25.786 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:25.786 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:25.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.786 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:15 3.15 3:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:25.786 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:25.786 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 38.716412757999997, 2023-11-25T04:23:25.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010464362, 2023-11-25T04:23:25.787 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:25.787 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:25.787 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:25.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:25.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:25.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:25.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:25.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:25.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:25.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:25.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034042+0000", 2023-11-25T04:23:25.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0999999999999997e-07 2023-11-25T04:23:25.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:25.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034048+0000", 2023-11-25T04:23:25.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0390000000000001e-06 2023-11-25T04:23:25.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:25.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034059+0000", 2023-11-25T04:23:25.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1182999999999999e-05 2023-11-25T04:23:25.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:25.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034067+0000", 2023-11-25T04:23:25.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1559999999999995e-06 2023-11-25T04:23:25.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:25.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035005+0000", 2023-11-25T04:23:25.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00093747399999999999 2023-11-25T04:23:25.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.790 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:25.790 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035077+0000", 2023-11-25T04:23:25.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2747e-05 2023-11-25T04:23:25.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.790 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:25.790 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035219+0000", 2023-11-25T04:23:25.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000141877 2023-11-25T04:23:25.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:25.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037810+0000", 2023-11-25T04:23:25.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002590752 2023-11-25T04:23:25.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:25.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044437+0000", 2023-11-25T04:23:25.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066270110000000004 2023-11-25T04:23:25.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:25.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044481+0000", 2023-11-25T04:23:25.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4106000000000001e-05 2023-11-25T04:23:25.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:25.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044505+0000", 2023-11-25T04:23:25.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4207e-05 2023-11-25T04:23:25.792 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:25.792 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:25.792 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:25.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.792 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:16 3.1d 3:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:25.793 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:25.793 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 38.716370910000002, 2023-11-25T04:23:25.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010413923, 2023-11-25T04:23:25.793 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:25.793 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:25.793 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:25.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:25.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:25.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:25.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:25.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:25.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:25.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.794 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:25.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:25.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4499999999999997e-07 2023-11-25T04:23:25.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.794 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:25.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034087+0000", 2023-11-25T04:23:25.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9770000000000002e-06 2023-11-25T04:23:25.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:25.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034094+0000", 2023-11-25T04:23:25.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8809999999999998e-06 2023-11-25T04:23:25.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:25.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034100+0000", 2023-11-25T04:23:25.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8440000000000003e-06 2023-11-25T04:23:25.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:25.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034993+0000", 2023-11-25T04:23:25.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00089336799999999996 2023-11-25T04:23:25.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:25.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035099+0000", 2023-11-25T04:23:25.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106037 2023-11-25T04:23:25.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:25.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035276+0000", 2023-11-25T04:23:25.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017631799999999999 2023-11-25T04:23:25.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:25.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037815+0000", 2023-11-25T04:23:25.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025395959999999999 2023-11-25T04:23:25.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:25.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044420+0000", 2023-11-25T04:23:25.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066045749999999997 2023-11-25T04:23:25.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:25.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044463+0000", 2023-11-25T04:23:25.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3205999999999999e-05 2023-11-25T04:23:25.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.798 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:25.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044497+0000", 2023-11-25T04:23:25.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3476000000000003e-05 2023-11-25T04:23:25.798 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:25.798 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:25.798 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:25.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.798 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:17 3.1d 3:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:25.798 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:25.799 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 38.716345701000002, 2023-11-25T04:23:25.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01127356, 2023-11-25T04:23:25.799 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:25.799 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:25.799 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:25.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:25.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:25.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:25.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:25.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:25.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:25.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:25.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:25.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1100000000000001e-07 2023-11-25T04:23:25.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:25.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034112+0000", 2023-11-25T04:23:25.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8600000000000003e-06 2023-11-25T04:23:25.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:25.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034120+0000", 2023-11-25T04:23:25.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7700000000000001e-06 2023-11-25T04:23:25.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:25.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034127+0000", 2023-11-25T04:23:25.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0999999999999998e-06 2023-11-25T04:23:25.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:25.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035601+0000", 2023-11-25T04:23:25.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014739410000000001 2023-11-25T04:23:25.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:25.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035685+0000", 2023-11-25T04:23:25.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3677999999999994e-05 2023-11-25T04:23:25.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:25.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035827+0000", 2023-11-25T04:23:25.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014232599999999999 2023-11-25T04:23:25.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:25.803 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038725+0000", 2023-11-25T04:23:25.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028982779999999998 2023-11-25T04:23:25.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:25.803 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045323+0000", 2023-11-25T04:23:25.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0065980780000000003 2023-11-25T04:23:25.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:25.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045354+0000", 2023-11-25T04:23:25.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0463e-05 2023-11-25T04:23:25.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.804 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:25.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045381+0000", 2023-11-25T04:23:25.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7654999999999999e-05 2023-11-25T04:23:25.804 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:25.804 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:25.804 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:25.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.805 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:22 3.e 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:25.805 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:25.805 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 38.716319288000001, 2023-11-25T04:23:25.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010133422, 2023-11-25T04:23:25.805 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:25.805 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:25.805 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:25.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:25.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:25.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:25.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:25.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:25.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:25.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:25.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034135+0000", 2023-11-25T04:23:25.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-07 2023-11-25T04:23:25.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:25.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034138+0000", 2023-11-25T04:23:25.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6009999999999999e-06 2023-11-25T04:23:25.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:25.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034145+0000", 2023-11-25T04:23:25.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1169999999999998e-06 2023-11-25T04:23:25.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:25.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034150+0000", 2023-11-25T04:23:25.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9080000000000003e-06 2023-11-25T04:23:25.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:25.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034209+0000", 2023-11-25T04:23:25.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8149000000000002e-05 2023-11-25T04:23:25.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:25.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034415+0000", 2023-11-25T04:23:25.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020637699999999999 2023-11-25T04:23:25.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:25.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034584+0000", 2023-11-25T04:23:25.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016870399999999999 2023-11-25T04:23:25.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:25.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.036979+0000", 2023-11-25T04:23:25.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023957509999999998 2023-11-25T04:23:25.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:25.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044161+0000", 2023-11-25T04:23:25.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0071811590000000003 2023-11-25T04:23:25.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:25.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044224+0000", 2023-11-25T04:23:25.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3454000000000006e-05 2023-11-25T04:23:25.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:25.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044268+0000", 2023-11-25T04:23:25.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3761999999999999e-05 2023-11-25T04:23:25.810 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:25.810 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:25.810 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:25.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.811 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:23 3.e 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:25.811 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:25.811 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 38.716287878999999, 2023-11-25T04:23:25.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010281676, 2023-11-25T04:23:25.811 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:25.811 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:25.811 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:25.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.811 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:25.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:25.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:25.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.812 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:25.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:25.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:25.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.812 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:25.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:25.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5799999999999999e-07 2023-11-25T04:23:25.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:25.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034169+0000", 2023-11-25T04:23:25.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1879999999999998e-06 2023-11-25T04:23:25.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:25.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034177+0000", 2023-11-25T04:23:25.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7020000000000001e-06 2023-11-25T04:23:25.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:25.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034184+0000", 2023-11-25T04:23:25.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3910000000000003e-06 2023-11-25T04:23:25.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:25.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034924+0000", 2023-11-25T04:23:25.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00074032999999999998 2023-11-25T04:23:25.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:25.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034991+0000", 2023-11-25T04:23:25.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7347000000000004e-05 2023-11-25T04:23:25.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:25.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035096+0000", 2023-11-25T04:23:25.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010488 2023-11-25T04:23:25.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:25.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037588+0000", 2023-11-25T04:23:25.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024918309999999999 2023-11-25T04:23:25.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:25.816 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044367+0000", 2023-11-25T04:23:25.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067787589999999997 2023-11-25T04:23:25.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:25.816 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044415+0000", 2023-11-25T04:23:25.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7803000000000003e-05 2023-11-25T04:23:25.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:25.816 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044447+0000", 2023-11-25T04:23:25.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2886999999999999e-05 2023-11-25T04:23:25.817 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:25.817 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:25.817 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:25.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.817 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:30 3.15 3:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:25.817 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:25.817 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 38.716229439000003, 2023-11-25T04:23:25.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010365070000000001, 2023-11-25T04:23:25.817 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:25.817 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:25.818 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:25.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:25.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:25.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:25.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:25.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:25.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:25.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:25.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034225+0000", 2023-11-25T04:23:25.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7400000000000002e-07 2023-11-25T04:23:25.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:25.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034229+0000", 2023-11-25T04:23:25.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3100000000000002e-06 2023-11-25T04:23:25.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:25.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034239+0000", 2023-11-25T04:23:25.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9140000000000003e-06 2023-11-25T04:23:25.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:25.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034246+0000", 2023-11-25T04:23:25.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0149999999999998e-06 2023-11-25T04:23:25.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:25.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035572+0000", 2023-11-25T04:23:25.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001325655 2023-11-25T04:23:25.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:25.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035730+0000", 2023-11-25T04:23:25.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015827899999999999 2023-11-25T04:23:25.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:25.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035857+0000", 2023-11-25T04:23:25.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012644699999999999 2023-11-25T04:23:25.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:25.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038737+0000", 2023-11-25T04:23:25.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002880077 2023-11-25T04:23:25.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:25.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044540+0000", 2023-11-25T04:23:25.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058034239999999997 2023-11-25T04:23:25.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:25.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044569+0000", 2023-11-25T04:23:25.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8388999999999999e-05 2023-11-25T04:23:25.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:25.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044589+0000", 2023-11-25T04:23:25.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0686000000000002e-05 2023-11-25T04:23:25.823 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:25.823 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:25.823 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:25.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.823 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:33 3.1b 3:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:25.823 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:25.823 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 38.716183940999997, 2023-11-25T04:23:25.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099754049999999997, 2023-11-25T04:23:25.823 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:25.823 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:23:25.824 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:25.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:25.824 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:25.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:25.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:25.824 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:25.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:25.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:25.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:25.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4799999999999998e-07 2023-11-25T04:23:25.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:25.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034273+0000", 2023-11-25T04:23:25.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7819999999999998e-06 2023-11-25T04:23:25.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:25.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034285+0000", 2023-11-25T04:23:25.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2058e-05 2023-11-25T04:23:25.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:25.826 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034291+0000", 2023-11-25T04:23:25.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0680000000000002e-06 2023-11-25T04:23:25.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:25.826 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034370+0000", 2023-11-25T04:23:25.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8452e-05 2023-11-25T04:23:25.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:25.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034477+0000", 2023-11-25T04:23:25.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107471 2023-11-25T04:23:25.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:25.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044245+0000", 2023-11-25T04:23:25.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0097679259999999997 2023-11-25T04:23:25.827 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:25.827 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:25.827 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:25.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.828 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:36 3.1a 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:25.828 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:25.828 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 38.716137267999997, 2023-11-25T04:23:25.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058179099999999999, 2023-11-25T04:23:25.828 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:25.828 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:23:25.828 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:25.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:25.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:25.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:25.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:25.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:25.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:25.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:25.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034317+0000", 2023-11-25T04:23:25.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1999999999999999e-07 2023-11-25T04:23:25.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:25.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034321+0000", 2023-11-25T04:23:25.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.233e-06 2023-11-25T04:23:25.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:25.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034337+0000", 2023-11-25T04:23:25.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5308999999999999e-05 2023-11-25T04:23:25.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:25.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034354+0000", 2023-11-25T04:23:25.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7411999999999999e-05 2023-11-25T04:23:25.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:25.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034465+0000", 2023-11-25T04:23:25.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011154699999999999 2023-11-25T04:23:25.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:25.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034560+0000", 2023-11-25T04:23:25.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4186999999999998e-05 2023-11-25T04:23:25.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.832 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:25.832 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.040134+0000", 2023-11-25T04:23:25.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055746019999999997 2023-11-25T04:23:25.832 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:25.832 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:25.832 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:25.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.832 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:9 3.10 e18/16 3:09401a81:::601.00000000:head v 18'4, mlcod=0'0)", 2023-11-25T04:23:25.832 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:25.832 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 38.707242960000002, 2023-11-25T04:23:25.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061641020000000003, 2023-11-25T04:23:25.833 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:25.833 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:25.833 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:25.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:25.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:25.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:25.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:25.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:25.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:25.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:25.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:25.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5300000000000001e-07 2023-11-25T04:23:25.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:25.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043222+0000", 2023-11-25T04:23:25.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.082e-05 2023-11-25T04:23:25.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.835 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:25.835 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043228+0000", 2023-11-25T04:23:25.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6699999999999997e-06 2023-11-25T04:23:25.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.835 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:25.835 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043236+0000", 2023-11-25T04:23:25.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8189999999999992e-06 2023-11-25T04:23:25.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.835 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:25.835 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043507+0000", 2023-11-25T04:23:25.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000271012 2023-11-25T04:23:25.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:25.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043530+0000", 2023-11-25T04:23:25.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2659000000000001e-05 2023-11-25T04:23:25.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:25.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049321+0000", 2023-11-25T04:23:25.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057915129999999999 2023-11-25T04:23:25.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:25.837 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049375+0000", 2023-11-25T04:23:25.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3455999999999998e-05 2023-11-25T04:23:25.837 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:25.837 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:25.837 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:25.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.837 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:1 3.14 e18/16 3:292cf221:::200.00000000:head v 18'3, mlcod=0'0)", 2023-11-25T04:23:25.837 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:25.837 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 38.707206247000002, 2023-11-25T04:23:25.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0060983599999999997, 2023-11-25T04:23:25.837 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:25.837 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:25.838 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:25.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:25.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:25.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:25.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:25.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:25.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:25.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:25.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043248+0000", 2023-11-25T04:23:25.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9399999999999995e-07 2023-11-25T04:23:25.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:25.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043252+0000", 2023-11-25T04:23:25.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1799999999999998e-06 2023-11-25T04:23:25.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:25.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043259+0000", 2023-11-25T04:23:25.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0049999999999998e-06 2023-11-25T04:23:25.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:25.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043265+0000", 2023-11-25T04:23:25.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0859999999999998e-06 2023-11-25T04:23:25.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:25.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043311+0000", 2023-11-25T04:23:25.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6082999999999999e-05 2023-11-25T04:23:25.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:25.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043330+0000", 2023-11-25T04:23:25.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8556e-05 2023-11-25T04:23:25.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:25.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049303+0000", 2023-11-25T04:23:25.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059733470000000004 2023-11-25T04:23:25.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:25.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049346+0000", 2023-11-25T04:23:25.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2608999999999998e-05 2023-11-25T04:23:25.841 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:25.842 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:25.842 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:25.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.842 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:3 3.14 e18/16 3:287034c4:::1.00000000.inode:head v 18'4, mlcod=0'0)", 2023-11-25T04:23:25.842 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:25.842 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 38.706689740999998, 2023-11-25T04:23:25.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057040090000000003, 2023-11-25T04:23:25.842 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:25.842 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:25.842 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:25.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:25.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:25.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:25.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:25.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:25.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:25.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:25.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:25.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9799999999999999e-07 2023-11-25T04:23:25.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.844 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:25.844 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043785+0000", 2023-11-25T04:23:25.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0721999999999999e-05 2023-11-25T04:23:25.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.844 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:25.844 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043793+0000", 2023-11-25T04:23:25.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5839999999999995e-06 2023-11-25T04:23:25.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.844 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:25.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043803+0000", 2023-11-25T04:23:25.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0729999999999994e-06 2023-11-25T04:23:25.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.845 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:25.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043843+0000", 2023-11-25T04:23:25.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0747999999999997e-05 2023-11-25T04:23:25.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.845 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:25.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043856+0000", 2023-11-25T04:23:25.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2697000000000001e-05 2023-11-25T04:23:25.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:25.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049426+0000", 2023-11-25T04:23:25.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055704409999999998 2023-11-25T04:23:25.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:25.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049468+0000", 2023-11-25T04:23:25.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1446000000000003e-05 2023-11-25T04:23:25.846 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:25.846 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:25.846 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:25.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.847 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "pg_update_log_missing(3.14 epoch 18/16 rep_tid 171 entries 18'5 (0'0) error 3:2e2fa760:::200.00000001:head by mds.0.3:32 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:23:25.847 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:25.847 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 38.706528777999999, 2023-11-25T04:23:25.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005556938, 2023-11-25T04:23:25.847 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:25.847 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2023-11-25T04:23:25.847 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:25.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:25.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:25.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:25.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.848 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:25.848 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:25.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:25.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.848 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:25.848 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:25.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.03e-07 2023-11-25T04:23:25.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.848 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:25.848 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043931+0000", 2023-11-25T04:23:25.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1079999999999998e-06 2023-11-25T04:23:25.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:25.849 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043944+0000", 2023-11-25T04:23:25.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2852e-05 2023-11-25T04:23:25.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:25.849 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043955+0000", 2023-11-25T04:23:25.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1037e-05 2023-11-25T04:23:25.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.850 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:25.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044088+0000", 2023-11-25T04:23:25.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013269599999999999 2023-11-25T04:23:25.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.850 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:25.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049482+0000", 2023-11-25T04:23:25.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053940419999999999 2023-11-25T04:23:25.850 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:25.850 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:25.850 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:25.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.851 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:10 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'5, mlcod=0'0)", 2023-11-25T04:23:25.851 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:25.851 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 38.706438749999997, 2023-11-25T04:23:25.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054493099999999997, 2023-11-25T04:23:25.851 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:25.851 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:25.851 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:25.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.851 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:25.851 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:25.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:25.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:25.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:25.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:25.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:25.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:25.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6100000000000002e-07 2023-11-25T04:23:25.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:25.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044019+0000", 2023-11-25T04:23:25.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.839e-06 2023-11-25T04:23:25.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:25.853 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044026+0000", 2023-11-25T04:23:25.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5869999999999998e-06 2023-11-25T04:23:25.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:25.853 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044031+0000", 2023-11-25T04:23:25.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4909999999999996e-06 2023-11-25T04:23:25.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.854 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:25.854 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044070+0000", 2023-11-25T04:23:25.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8829999999999999e-05 2023-11-25T04:23:25.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.854 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:25.854 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044081+0000", 2023-11-25T04:23:25.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1447999999999999e-05 2023-11-25T04:23:25.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.854 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:25.854 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049406+0000", 2023-11-25T04:23:25.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053246480000000004 2023-11-25T04:23:25.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.855 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:25.855 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049464+0000", 2023-11-25T04:23:25.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8205999999999998e-05 2023-11-25T04:23:25.855 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:25.855 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:25.855 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:25.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.855 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:11 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'6, mlcod=0'0)", 2023-11-25T04:23:25.855 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:25.856 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 38.706237485999999, 2023-11-25T04:23:25.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052673499999999996, 2023-11-25T04:23:25.856 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:25.856 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:25.856 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:25.856 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.856 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:25.856 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:25.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:25.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.856 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.856 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:25.857 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:25.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:25.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.857 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:25.857 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:25.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0199999999999998e-07 2023-11-25T04:23:25.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.857 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:25.857 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044220+0000", 2023-11-25T04:23:25.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7720000000000001e-06 2023-11-25T04:23:25.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.858 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.858 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:25.858 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044226+0000", 2023-11-25T04:23:25.858 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8579999999999996e-06 2023-11-25T04:23:25.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.858 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.858 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:25.858 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044232+0000", 2023-11-25T04:23:25.858 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1700000000000002e-06 2023-11-25T04:23:25.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.858 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.858 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:25.859 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044274+0000", 2023-11-25T04:23:25.859 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1254999999999997e-05 2023-11-25T04:23:25.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.859 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.859 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:25.859 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:23:25.859 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6880999999999999e-05 2023-11-25T04:23:25.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.859 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.859 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:25.859 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049466+0000", 2023-11-25T04:23:25.859 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051659219999999999 2023-11-25T04:23:25.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:25.860 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:25.860 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:25.860 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049483+0000", 2023-11-25T04:23:25.860 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.719e-05 2023-11-25T04:23:25.860 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:25.860 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:25.860 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:25.860 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:25.860 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:25.860 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:25.861 INFO:teuthology.orchestra.run.smithi067.stdout:osd.1: {} 2023-11-25T04:23:25.862 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:25.861+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:25.862 INFO:teuthology.orchestra.run.smithi067.stderr:osd.1: osd_enable_op_tracker = '' 2023-11-25T04:23:25.863 INFO:teuthology.orchestra.run.smithi067.stdout:osd.2: {} 2023-11-25T04:23:25.863 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:23:25.864 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-11-25T04:23:25.864 INFO:teuthology.orchestra.run.smithi067.stderr:osd.2: osd_enable_op_tracker = '' 2023-11-25T04:23:25.864 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-11-25T04:23:25.896 INFO:teuthology.orchestra.run.smithi067.stderr:nodeep-scrub is unset 2023-11-25T04:23:25.962 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:25.961+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:25.972 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:25.972 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:25.980 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:23:26.063 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:26.062+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:26.126 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:26.126 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:26.135 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:23:26.178 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:26.177+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:26.241 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:26.241 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:26.250 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:23:26.295 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:26.293+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:26.360 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:26.360 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:26.365 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:26.363+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:26.369 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:23:26.467 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:26.465+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:26.515 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:26.515 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:26.523 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:23:26.567 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~18s 2023-11-25T04:23:26.567 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:26.565+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:26.628 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:26.628 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:26.637 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:23:26.680 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:26.679+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:26.745 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:26.745 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:26.754 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:23:26.769 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:26.767+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:26.871 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:26.869+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:26.899 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:26.899 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:26.908 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:23:26.971 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:26.969+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:27.013 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:27.014 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:27.023 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:23:27.071 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:27.069+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:27.131 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:27.131 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:27.140 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:23:27.181 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:27.179+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:27.199 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2] out_osds: [] dead_osds: [0] live_osds: [1, 2] 2023-11-25T04:23:27.199 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 1.00 2023-11-25T04:23:27.199 INFO:tasks.thrashosds.thrasher:Reviving osd 0 2023-11-25T04:23:27.200 INFO:tasks.ceph.ceph_manager.ceph:kill_osd on osd.0 doing powercycle of ubuntu@smithi076.front.sepia.ceph.com 2023-11-25T04:23:27.200 INFO:teuthology.orchestra.console.smithi076:Power on 2023-11-25T04:23:27.200 DEBUG:teuthology.orchestra.console.smithi076:pexpect command: ipmitool -H smithi076.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-11-25T04:23:27.227 DEBUG:teuthology.orchestra.console.smithi076:power on output: Chassis Power Control: Up/On 2023-11-25T04:23:27.227 DEBUG:teuthology.orchestra.console.smithi076:pexpect command: ipmitool -H smithi076.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-25T04:23:27.253 DEBUG:teuthology.orchestra.console.smithi076:check power output: Chassis Power is off 2023-11-25T04:23:27.274 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:27.272+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:27.286 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:27.286 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:27.296 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:23:27.374 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:27.372+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:27.401 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:27.401 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:27.411 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:23:27.475 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:27.473+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:27.518 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:27.518 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:27.527 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:23:27.611 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:27.610+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:27.672 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:27.673 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:27.677 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:27.675+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:27.681 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:23:27.778 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:27.776+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:27.826 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:27.826 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:27.835 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:23:27.865 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-11-25T04:23:27.878 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:27.876+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:27.913 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2023-11-25T04:23:27.941 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:27.941 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:27.949 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:23:27.979 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:27.978+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:27.996 INFO:teuthology.orchestra.run.smithi067.stderr:Error ENXIO: problem getting command descriptions from osd.0 2023-11-25T04:23:28.030 INFO:teuthology.orchestra.run.smithi067.stdout:osd.1: {} 2023-11-25T04:23:28.030 INFO:teuthology.orchestra.run.smithi067.stderr:osd.1: osd_enable_op_tracker = '' 2023-11-25T04:23:28.064 INFO:teuthology.orchestra.run.smithi067.stdout:osd.2: {} 2023-11-25T04:23:28.064 INFO:teuthology.orchestra.run.smithi067.stderr:osd.2: osd_enable_op_tracker = '' 2023-11-25T04:23:28.068 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-11-25T04:23:28.069 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-11-25T04:23:28.080 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:28.078+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:28.095 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:28.095 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:23:28.095 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2023-11-25T04:23:28.095 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2023-11-25T04:23:28.095 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:28.107 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:23:28.181 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:28.178+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:28.214 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:28.214 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2023-11-25T04:23:28.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2023-11-25T04:23:28.214 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2023-11-25T04:23:28.214 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.214 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:155 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'109, mlcod=11'105)", 2023-11-25T04:23:28.214 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:28.214 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.426571209000002, 2023-11-25T04:23:28.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061719019999999999, 2023-11-25T04:23:28.215 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:28.215 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:28.215 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:28.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:28.215 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:28.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:28.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:28.215 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:28.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:28.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:28.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784884+0000", 2023-11-25T04:23:28.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3099999999999999e-07 2023-11-25T04:23:28.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:28.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784942+0000", 2023-11-25T04:23:28.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8570000000000003e-05 2023-11-25T04:23:28.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.217 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:28.217 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784949+0000", 2023-11-25T04:23:28.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0310000000000001e-06 2023-11-25T04:23:28.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.217 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:28.217 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784953+0000", 2023-11-25T04:23:28.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6320000000000001e-06 2023-11-25T04:23:28.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.217 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:28.218 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784993+0000", 2023-11-25T04:23:28.218 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9737000000000003e-05 2023-11-25T04:23:28.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.218 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:28.218 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785011+0000", 2023-11-25T04:23:28.218 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8349999999999999e-05 2023-11-25T04:23:28.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.219 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:28.219 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791024+0000", 2023-11-25T04:23:28.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006012997 2023-11-25T04:23:28.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.219 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:28.219 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791055+0000", 2023-11-25T04:23:28.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1254000000000003e-05 2023-11-25T04:23:28.219 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:28.219 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:28.219 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:28.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.220 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:156 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'110, mlcod=11'105)", 2023-11-25T04:23:28.220 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:28.220 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.425985160000003, 2023-11-25T04:23:28.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005616116, 2023-11-25T04:23:28.220 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:28.220 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:28.220 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:28.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:28.220 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:28.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:28.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.221 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.221 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:28.221 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:28.221 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:28.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.221 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.221 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:28.221 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785470+0000", 2023-11-25T04:23:28.221 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0699999999999998e-07 2023-11-25T04:23:28.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.221 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.221 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:28.222 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785474+0000", 2023-11-25T04:23:28.222 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.048e-06 2023-11-25T04:23:28.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.222 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:28.222 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785480+0000", 2023-11-25T04:23:28.222 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7729999999999996e-06 2023-11-25T04:23:28.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.222 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:28.222 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785483+0000", 2023-11-25T04:23:28.222 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1970000000000001e-06 2023-11-25T04:23:28.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.223 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.223 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:28.223 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787936+0000", 2023-11-25T04:23:28.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024529040000000001 2023-11-25T04:23:28.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.223 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.223 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:28.223 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787947+0000", 2023-11-25T04:23:28.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1669e-05 2023-11-25T04:23:28.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.223 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.223 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:28.224 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791057+0000", 2023-11-25T04:23:28.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031095200000000002 2023-11-25T04:23:28.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:28.224 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791086+0000", 2023-11-25T04:23:28.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8697999999999999e-05 2023-11-25T04:23:28.224 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:28.224 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:28.224 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:28.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.224 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:157 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'111, mlcod=11'106)", 2023-11-25T04:23:28.225 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:28.225 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.425392602000002, 2023-11-25T04:23:28.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050590280000000001, 2023-11-25T04:23:28.225 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:28.225 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:28.225 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:28.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.225 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:28.225 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:28.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:28.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.226 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:28.226 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:28.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:28.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.226 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:28.226 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786063+0000", 2023-11-25T04:23:28.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2600000000000002e-07 2023-11-25T04:23:28.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:28.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786069+0000", 2023-11-25T04:23:28.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8839999999999999e-06 2023-11-25T04:23:28.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:28.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786079+0000", 2023-11-25T04:23:28.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0124e-05 2023-11-25T04:23:28.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:28.228 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786085+0000", 2023-11-25T04:23:28.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6660000000000002e-06 2023-11-25T04:23:28.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.228 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:28.228 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788160+0000", 2023-11-25T04:23:28.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020747669999999999 2023-11-25T04:23:28.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.228 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:28.229 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788180+0000", 2023-11-25T04:23:28.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0208999999999999e-05 2023-11-25T04:23:28.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.229 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:28.229 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791087+0000", 2023-11-25T04:23:28.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029064630000000002 2023-11-25T04:23:28.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.229 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:28.229 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791121+0000", 2023-11-25T04:23:28.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4289000000000003e-05 2023-11-25T04:23:28.230 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:28.230 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:28.230 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:28.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.230 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:158 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'112, mlcod=11'106)", 2023-11-25T04:23:28.230 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:28.230 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.424976209, 2023-11-25T04:23:28.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046663030000000001, 2023-11-25T04:23:28.230 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:28.230 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:28.230 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:28.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:28.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:28.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:28.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:28.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:28.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:28.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:28.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786479+0000", 2023-11-25T04:23:28.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2799999999999996e-07 2023-11-25T04:23:28.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:28.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786498+0000", 2023-11-25T04:23:28.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9069e-05 2023-11-25T04:23:28.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:28.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786508+0000", 2023-11-25T04:23:28.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5810000000000003e-06 2023-11-25T04:23:28.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.233 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:28.233 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786512+0000", 2023-11-25T04:23:28.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4889999999999997e-06 2023-11-25T04:23:28.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.233 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:28.234 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788438+0000", 2023-11-25T04:23:28.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019257059999999999 2023-11-25T04:23:28.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:28.234 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788449+0000", 2023-11-25T04:23:28.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1419e-05 2023-11-25T04:23:28.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:28.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791122+0000", 2023-11-25T04:23:28.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002672822 2023-11-25T04:23:28.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.235 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:28.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791145+0000", 2023-11-25T04:23:28.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2589e-05 2023-11-25T04:23:28.235 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:28.235 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:28.235 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:28.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.236 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:206 1.0 e12/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 12'152, mlcod=12'148)", 2023-11-25T04:23:28.236 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:28.236 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 49.913207739000001, 2023-11-25T04:23:28.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046728189999999999, 2023-11-25T04:23:28.236 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:28.236 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:28.236 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:28.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:28.236 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:28.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:28.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:28.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:28.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:28.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:28.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298248+0000", 2023-11-25T04:23:28.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3600000000000003e-07 2023-11-25T04:23:28.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:28.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298278+0000", 2023-11-25T04:23:28.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0474e-05 2023-11-25T04:23:28.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:28.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298288+0000", 2023-11-25T04:23:28.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0156000000000001e-05 2023-11-25T04:23:28.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:28.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298293+0000", 2023-11-25T04:23:28.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7400000000000004e-06 2023-11-25T04:23:28.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.239 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:28.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298333+0000", 2023-11-25T04:23:28.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9666000000000003e-05 2023-11-25T04:23:28.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.239 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:28.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298351+0000", 2023-11-25T04:23:28.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8544999999999999e-05 2023-11-25T04:23:28.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:28.240 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302891+0000", 2023-11-25T04:23:28.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045398519999999996 2023-11-25T04:23:28.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:28.240 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302920+0000", 2023-11-25T04:23:28.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8750000000000001e-05 2023-11-25T04:23:28.240 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:28.240 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:28.240 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:28.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.241 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:14 3.15 3:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:28.241 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:28.241 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 41.178073382000001, 2023-11-25T04:23:28.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010990437, 2023-11-25T04:23:28.241 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:28.241 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:28.241 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:28.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:28.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:28.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:28.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:28.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:28.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:28.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:28.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033383+0000", 2023-11-25T04:23:28.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9870000000000002e-06 2023-11-25T04:23:28.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:28.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033398+0000", 2023-11-25T04:23:28.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4569999999999999e-05 2023-11-25T04:23:28.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:28.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033411+0000", 2023-11-25T04:23:28.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3543999999999999e-05 2023-11-25T04:23:28.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.244 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:28.244 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033423+0000", 2023-11-25T04:23:28.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1905999999999999e-05 2023-11-25T04:23:28.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.244 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:28.244 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034433+0000", 2023-11-25T04:23:28.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00101022 2023-11-25T04:23:28.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.244 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:28.244 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034539+0000", 2023-11-25T04:23:28.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010597199999999999 2023-11-25T04:23:28.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.245 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:28.245 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034667+0000", 2023-11-25T04:23:28.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012765500000000001 2023-11-25T04:23:28.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.245 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:28.245 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037803+0000", 2023-11-25T04:23:28.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031362220000000001 2023-11-25T04:23:28.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:28.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:23:28.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0064964669999999997 2023-11-25T04:23:28.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:28.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044331+0000", 2023-11-25T04:23:28.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1544999999999997e-05 2023-11-25T04:23:28.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:28.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044372+0000", 2023-11-25T04:23:28.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0349000000000003e-05 2023-11-25T04:23:28.247 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:28.247 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:28.247 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:28.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.247 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:15 3.15 3:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:28.247 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:28.247 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 41.177413731999998, 2023-11-25T04:23:28.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010464362, 2023-11-25T04:23:28.247 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:28.247 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:28.248 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:28.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:28.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:28.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:28.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:28.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:28.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:28.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:28.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034042+0000", 2023-11-25T04:23:28.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0999999999999997e-07 2023-11-25T04:23:28.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:28.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034048+0000", 2023-11-25T04:23:28.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0390000000000001e-06 2023-11-25T04:23:28.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:28.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034059+0000", 2023-11-25T04:23:28.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1182999999999999e-05 2023-11-25T04:23:28.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.250 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:28.250 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034067+0000", 2023-11-25T04:23:28.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1559999999999995e-06 2023-11-25T04:23:28.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.250 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:28.250 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035005+0000", 2023-11-25T04:23:28.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00093747399999999999 2023-11-25T04:23:28.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:28.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035077+0000", 2023-11-25T04:23:28.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2747e-05 2023-11-25T04:23:28.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:28.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035219+0000", 2023-11-25T04:23:28.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000141877 2023-11-25T04:23:28.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:28.252 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037810+0000", 2023-11-25T04:23:28.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002590752 2023-11-25T04:23:28.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:28.252 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044437+0000", 2023-11-25T04:23:28.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066270110000000004 2023-11-25T04:23:28.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:28.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044481+0000", 2023-11-25T04:23:28.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4106000000000001e-05 2023-11-25T04:23:28.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:28.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044505+0000", 2023-11-25T04:23:28.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4207e-05 2023-11-25T04:23:28.253 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:28.253 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:28.253 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:28.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.254 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:16 3.1d 3:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:28.254 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:28.254 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 41.177371884000003, 2023-11-25T04:23:28.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010413923, 2023-11-25T04:23:28.254 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:28.254 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:28.254 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:28.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:28.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:28.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:28.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:28.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:28.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:28.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:28.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:28.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4499999999999997e-07 2023-11-25T04:23:28.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:28.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034087+0000", 2023-11-25T04:23:28.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9770000000000002e-06 2023-11-25T04:23:28.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:28.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034094+0000", 2023-11-25T04:23:28.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8809999999999998e-06 2023-11-25T04:23:28.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:28.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034100+0000", 2023-11-25T04:23:28.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8440000000000003e-06 2023-11-25T04:23:28.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:28.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034993+0000", 2023-11-25T04:23:28.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00089336799999999996 2023-11-25T04:23:28.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:28.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035099+0000", 2023-11-25T04:23:28.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106037 2023-11-25T04:23:28.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:28.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035276+0000", 2023-11-25T04:23:28.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017631799999999999 2023-11-25T04:23:28.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:28.258 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037815+0000", 2023-11-25T04:23:28.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025395959999999999 2023-11-25T04:23:28.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:28.258 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044420+0000", 2023-11-25T04:23:28.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066045749999999997 2023-11-25T04:23:28.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:28.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044463+0000", 2023-11-25T04:23:28.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3205999999999999e-05 2023-11-25T04:23:28.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:28.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044497+0000", 2023-11-25T04:23:28.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3476000000000003e-05 2023-11-25T04:23:28.259 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:28.259 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:28.259 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:28.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.260 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:17 3.1d 3:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:28.260 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:28.260 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 41.177346675000003, 2023-11-25T04:23:28.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01127356, 2023-11-25T04:23:28.260 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:28.260 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:28.261 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:28.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:28.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:28.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:28.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:28.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:28.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:28.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:28.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:28.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1100000000000001e-07 2023-11-25T04:23:28.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:28.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034112+0000", 2023-11-25T04:23:28.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8600000000000003e-06 2023-11-25T04:23:28.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:28.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034120+0000", 2023-11-25T04:23:28.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7700000000000001e-06 2023-11-25T04:23:28.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:28.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034127+0000", 2023-11-25T04:23:28.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0999999999999998e-06 2023-11-25T04:23:28.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:28.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035601+0000", 2023-11-25T04:23:28.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014739410000000001 2023-11-25T04:23:28.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:28.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035685+0000", 2023-11-25T04:23:28.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3677999999999994e-05 2023-11-25T04:23:28.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:28.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035827+0000", 2023-11-25T04:23:28.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014232599999999999 2023-11-25T04:23:28.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:28.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038725+0000", 2023-11-25T04:23:28.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028982779999999998 2023-11-25T04:23:28.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:28.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045323+0000", 2023-11-25T04:23:28.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0065980780000000003 2023-11-25T04:23:28.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:28.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045354+0000", 2023-11-25T04:23:28.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0463e-05 2023-11-25T04:23:28.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:28.267 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045381+0000", 2023-11-25T04:23:28.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7654999999999999e-05 2023-11-25T04:23:28.267 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:28.267 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:28.267 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:28.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.267 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:22 3.e 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:28.267 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:28.267 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 41.177320262000002, 2023-11-25T04:23:28.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010133422, 2023-11-25T04:23:28.268 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:28.268 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:28.268 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:28.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:28.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:28.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:28.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:28.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:28.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:28.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:28.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034135+0000", 2023-11-25T04:23:28.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-07 2023-11-25T04:23:28.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:28.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034138+0000", 2023-11-25T04:23:28.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6009999999999999e-06 2023-11-25T04:23:28.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:28.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034145+0000", 2023-11-25T04:23:28.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1169999999999998e-06 2023-11-25T04:23:28.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:28.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034150+0000", 2023-11-25T04:23:28.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9080000000000003e-06 2023-11-25T04:23:28.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:28.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034209+0000", 2023-11-25T04:23:28.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8149000000000002e-05 2023-11-25T04:23:28.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:28.272 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034415+0000", 2023-11-25T04:23:28.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020637699999999999 2023-11-25T04:23:28.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:28.272 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034584+0000", 2023-11-25T04:23:28.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016870399999999999 2023-11-25T04:23:28.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:28.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.036979+0000", 2023-11-25T04:23:28.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023957509999999998 2023-11-25T04:23:28.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:28.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044161+0000", 2023-11-25T04:23:28.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0071811590000000003 2023-11-25T04:23:28.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:28.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044224+0000", 2023-11-25T04:23:28.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3454000000000006e-05 2023-11-25T04:23:28.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:28.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044268+0000", 2023-11-25T04:23:28.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3761999999999999e-05 2023-11-25T04:23:28.274 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:28.274 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:28.274 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:28.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.274 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:23 3.e 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:28.274 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:28.274 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 41.177288853, 2023-11-25T04:23:28.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010281676, 2023-11-25T04:23:28.275 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:28.275 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:28.275 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:28.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:28.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:28.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:28.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:28.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:28.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:28.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:28.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:28.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5799999999999999e-07 2023-11-25T04:23:28.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:28.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034169+0000", 2023-11-25T04:23:28.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1879999999999998e-06 2023-11-25T04:23:28.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:28.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034177+0000", 2023-11-25T04:23:28.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7020000000000001e-06 2023-11-25T04:23:28.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:28.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034184+0000", 2023-11-25T04:23:28.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3910000000000003e-06 2023-11-25T04:23:28.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:28.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034924+0000", 2023-11-25T04:23:28.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00074032999999999998 2023-11-25T04:23:28.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:28.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034991+0000", 2023-11-25T04:23:28.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7347000000000004e-05 2023-11-25T04:23:28.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:28.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035096+0000", 2023-11-25T04:23:28.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010488 2023-11-25T04:23:28.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:28.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037588+0000", 2023-11-25T04:23:28.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024918309999999999 2023-11-25T04:23:28.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:28.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044367+0000", 2023-11-25T04:23:28.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067787589999999997 2023-11-25T04:23:28.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:28.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044415+0000", 2023-11-25T04:23:28.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7803000000000003e-05 2023-11-25T04:23:28.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:28.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044447+0000", 2023-11-25T04:23:28.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2886999999999999e-05 2023-11-25T04:23:28.281 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:28.281 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:28.281 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:28.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.281 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:30 3.15 3:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:28.281 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:28.281 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 41.177230412999997, 2023-11-25T04:23:28.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010365070000000001, 2023-11-25T04:23:28.281 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:28.281 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:28.281 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:28.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:28.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:28.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:28.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:28.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:28.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:28.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:28.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034225+0000", 2023-11-25T04:23:28.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7400000000000002e-07 2023-11-25T04:23:28.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:28.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034229+0000", 2023-11-25T04:23:28.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3100000000000002e-06 2023-11-25T04:23:28.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:28.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034239+0000", 2023-11-25T04:23:28.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9140000000000003e-06 2023-11-25T04:23:28.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:28.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034246+0000", 2023-11-25T04:23:28.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0149999999999998e-06 2023-11-25T04:23:28.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:28.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035572+0000", 2023-11-25T04:23:28.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001325655 2023-11-25T04:23:28.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:28.286 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035730+0000", 2023-11-25T04:23:28.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015827899999999999 2023-11-25T04:23:28.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:28.286 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035857+0000", 2023-11-25T04:23:28.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012644699999999999 2023-11-25T04:23:28.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:28.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038737+0000", 2023-11-25T04:23:28.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002880077 2023-11-25T04:23:28.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:28.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044540+0000", 2023-11-25T04:23:28.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058034239999999997 2023-11-25T04:23:28.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:28.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044569+0000", 2023-11-25T04:23:28.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8388999999999999e-05 2023-11-25T04:23:28.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:28.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044589+0000", 2023-11-25T04:23:28.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0686000000000002e-05 2023-11-25T04:23:28.288 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:28.288 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:28.288 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:28.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.288 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:33 3.1b 3:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:28.289 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:28.289 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 41.177184914999998, 2023-11-25T04:23:28.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099754049999999997, 2023-11-25T04:23:28.289 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:28.289 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:23:28.289 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:28.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:28.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:28.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:28.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.290 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:28.290 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:28.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:28.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.290 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:28.290 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:28.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4799999999999998e-07 2023-11-25T04:23:28.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.291 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:28.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034273+0000", 2023-11-25T04:23:28.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7819999999999998e-06 2023-11-25T04:23:28.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.291 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:28.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034285+0000", 2023-11-25T04:23:28.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2058e-05 2023-11-25T04:23:28.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.291 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:28.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034291+0000", 2023-11-25T04:23:28.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0680000000000002e-06 2023-11-25T04:23:28.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:28.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034370+0000", 2023-11-25T04:23:28.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8452e-05 2023-11-25T04:23:28.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:28.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034477+0000", 2023-11-25T04:23:28.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107471 2023-11-25T04:23:28.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:28.293 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044245+0000", 2023-11-25T04:23:28.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0097679259999999997 2023-11-25T04:23:28.293 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:28.293 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:28.293 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:28.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.293 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:36 3.1a 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:28.293 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:28.293 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 41.177138241999998, 2023-11-25T04:23:28.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058179099999999999, 2023-11-25T04:23:28.293 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:28.294 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:23:28.294 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:28.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:28.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:28.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:28.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:28.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:28.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:28.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:28.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034317+0000", 2023-11-25T04:23:28.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1999999999999999e-07 2023-11-25T04:23:28.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:28.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034321+0000", 2023-11-25T04:23:28.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.233e-06 2023-11-25T04:23:28.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.296 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:28.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034337+0000", 2023-11-25T04:23:28.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5308999999999999e-05 2023-11-25T04:23:28.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.296 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:28.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034354+0000", 2023-11-25T04:23:28.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7411999999999999e-05 2023-11-25T04:23:28.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:28.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034465+0000", 2023-11-25T04:23:28.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011154699999999999 2023-11-25T04:23:28.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:28.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034560+0000", 2023-11-25T04:23:28.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4186999999999998e-05 2023-11-25T04:23:28.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:28.298 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.040134+0000", 2023-11-25T04:23:28.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055746019999999997 2023-11-25T04:23:28.298 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:28.298 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:28.298 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:28.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.298 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:9 3.10 e18/16 3:09401a81:::601.00000000:head v 18'4, mlcod=0'0)", 2023-11-25T04:23:28.298 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:28.299 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 41.168243934000003, 2023-11-25T04:23:28.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061641020000000003, 2023-11-25T04:23:28.299 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:28.299 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:28.299 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:28.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:28.299 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:28.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:28.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.300 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:28.300 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:28.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:28.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.300 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:28.300 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:28.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5300000000000001e-07 2023-11-25T04:23:28.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.301 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:28.301 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043222+0000", 2023-11-25T04:23:28.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.082e-05 2023-11-25T04:23:28.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.301 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:28.301 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043228+0000", 2023-11-25T04:23:28.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6699999999999997e-06 2023-11-25T04:23:28.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.301 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:28.301 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043236+0000", 2023-11-25T04:23:28.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8189999999999992e-06 2023-11-25T04:23:28.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:28.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043507+0000", 2023-11-25T04:23:28.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000271012 2023-11-25T04:23:28.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:28.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043530+0000", 2023-11-25T04:23:28.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2659000000000001e-05 2023-11-25T04:23:28.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.303 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:28.303 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049321+0000", 2023-11-25T04:23:28.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057915129999999999 2023-11-25T04:23:28.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.303 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:28.303 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049375+0000", 2023-11-25T04:23:28.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3455999999999998e-05 2023-11-25T04:23:28.303 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:28.304 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:28.304 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:28.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.304 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:1 3.14 e18/16 3:292cf221:::200.00000000:head v 18'3, mlcod=0'0)", 2023-11-25T04:23:28.304 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:28.304 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 41.168207221000003, 2023-11-25T04:23:28.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0060983599999999997, 2023-11-25T04:23:28.304 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:28.304 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:28.304 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:28.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:28.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:28.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:28.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:28.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:28.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:28.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:28.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043248+0000", 2023-11-25T04:23:28.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9399999999999995e-07 2023-11-25T04:23:28.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:28.306 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043252+0000", 2023-11-25T04:23:28.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1799999999999998e-06 2023-11-25T04:23:28.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:28.306 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043259+0000", 2023-11-25T04:23:28.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0049999999999998e-06 2023-11-25T04:23:28.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:28.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043265+0000", 2023-11-25T04:23:28.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0859999999999998e-06 2023-11-25T04:23:28.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:28.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043311+0000", 2023-11-25T04:23:28.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6082999999999999e-05 2023-11-25T04:23:28.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:28.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043330+0000", 2023-11-25T04:23:28.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8556e-05 2023-11-25T04:23:28.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.308 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:28.308 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049303+0000", 2023-11-25T04:23:28.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059733470000000004 2023-11-25T04:23:28.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.308 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:28.308 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049346+0000", 2023-11-25T04:23:28.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2608999999999998e-05 2023-11-25T04:23:28.308 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:28.309 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:28.309 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:28.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.309 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:3 3.14 e18/16 3:287034c4:::1.00000000.inode:head v 18'4, mlcod=0'0)", 2023-11-25T04:23:28.309 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:28.309 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 41.167690714999999, 2023-11-25T04:23:28.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057040090000000003, 2023-11-25T04:23:28.309 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:28.309 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:28.309 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:28.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:28.310 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:28.310 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:28.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:28.310 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:28.310 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:28.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:28.310 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:28.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9799999999999999e-07 2023-11-25T04:23:28.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:28.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043785+0000", 2023-11-25T04:23:28.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0721999999999999e-05 2023-11-25T04:23:28.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:28.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043793+0000", 2023-11-25T04:23:28.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5839999999999995e-06 2023-11-25T04:23:28.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.312 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:28.312 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043803+0000", 2023-11-25T04:23:28.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0729999999999994e-06 2023-11-25T04:23:28.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.312 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:28.312 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043843+0000", 2023-11-25T04:23:28.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0747999999999997e-05 2023-11-25T04:23:28.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.312 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:28.312 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043856+0000", 2023-11-25T04:23:28.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2697000000000001e-05 2023-11-25T04:23:28.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.313 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:28.313 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049426+0000", 2023-11-25T04:23:28.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055704409999999998 2023-11-25T04:23:28.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.313 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:28.314 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049468+0000", 2023-11-25T04:23:28.314 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1446000000000003e-05 2023-11-25T04:23:28.314 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:28.314 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:28.314 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:28.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.314 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "pg_update_log_missing(3.14 epoch 18/16 rep_tid 171 entries 18'5 (0'0) error 3:2e2fa760:::200.00000001:head by mds.0.3:32 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:23:28.314 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:28.314 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 41.167529752, 2023-11-25T04:23:28.314 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005556938, 2023-11-25T04:23:28.314 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:28.315 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2023-11-25T04:23:28.315 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:28.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.315 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:28.315 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:28.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:28.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.315 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:28.315 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:28.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:28.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.316 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:28.316 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:28.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.03e-07 2023-11-25T04:23:28.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.316 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:28.316 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043931+0000", 2023-11-25T04:23:28.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1079999999999998e-06 2023-11-25T04:23:28.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.317 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:28.317 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043944+0000", 2023-11-25T04:23:28.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2852e-05 2023-11-25T04:23:28.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.317 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:28.317 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043955+0000", 2023-11-25T04:23:28.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1037e-05 2023-11-25T04:23:28.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.318 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:28.318 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044088+0000", 2023-11-25T04:23:28.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013269599999999999 2023-11-25T04:23:28.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.318 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:28.318 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049482+0000", 2023-11-25T04:23:28.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053940419999999999 2023-11-25T04:23:28.318 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:28.318 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:28.318 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:28.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.319 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:10 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'5, mlcod=0'0)", 2023-11-25T04:23:28.319 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:28.319 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 41.167439723999998, 2023-11-25T04:23:28.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054493099999999997, 2023-11-25T04:23:28.319 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:28.319 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:28.319 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:28.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.319 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:28.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:28.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:28.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:28.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:28.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:28.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:28.321 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:28.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6100000000000002e-07 2023-11-25T04:23:28.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:28.321 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044019+0000", 2023-11-25T04:23:28.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.839e-06 2023-11-25T04:23:28.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:28.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044026+0000", 2023-11-25T04:23:28.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5869999999999998e-06 2023-11-25T04:23:28.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:28.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044031+0000", 2023-11-25T04:23:28.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4909999999999996e-06 2023-11-25T04:23:28.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:28.323 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044070+0000", 2023-11-25T04:23:28.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8829999999999999e-05 2023-11-25T04:23:28.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.323 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:28.323 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044081+0000", 2023-11-25T04:23:28.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1447999999999999e-05 2023-11-25T04:23:28.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.323 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:28.323 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049406+0000", 2023-11-25T04:23:28.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053246480000000004 2023-11-25T04:23:28.324 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.324 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.324 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:28.324 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049464+0000", 2023-11-25T04:23:28.324 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8205999999999998e-05 2023-11-25T04:23:28.324 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:28.324 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:28.324 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:28.324 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.324 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.324 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:11 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'6, mlcod=0'0)", 2023-11-25T04:23:28.324 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:28.325 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 41.16723846, 2023-11-25T04:23:28.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052673499999999996, 2023-11-25T04:23:28.325 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:28.325 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:28.325 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:28.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.325 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:28.325 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:28.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:28.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.325 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:28.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:28.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:28.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:28.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:28.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0199999999999998e-07 2023-11-25T04:23:28.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.327 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:28.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044220+0000", 2023-11-25T04:23:28.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7720000000000001e-06 2023-11-25T04:23:28.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.327 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:28.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044226+0000", 2023-11-25T04:23:28.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8579999999999996e-06 2023-11-25T04:23:28.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.327 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:28.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044232+0000", 2023-11-25T04:23:28.328 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1700000000000002e-06 2023-11-25T04:23:28.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.328 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.328 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:28.328 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044274+0000", 2023-11-25T04:23:28.328 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1254999999999997e-05 2023-11-25T04:23:28.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.328 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.328 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:28.328 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:23:28.328 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6880999999999999e-05 2023-11-25T04:23:28.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.329 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.329 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:28.329 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049466+0000", 2023-11-25T04:23:28.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051659219999999999 2023-11-25T04:23:28.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:28.329 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:28.329 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:28.329 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049483+0000", 2023-11-25T04:23:28.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.719e-05 2023-11-25T04:23:28.329 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:28.329 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:28.329 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:28.330 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:28.330 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:28.330 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:28.331 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:23:28.332 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:28.329+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:28.432 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:28.430+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:28.440 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:28.440 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:23:28.440 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2023-11-25T04:23:28.440 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:28.449 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:23:28.532 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:28.531+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:28.596 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:28.596 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:23:28.596 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2023-11-25T04:23:28.596 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2023-11-25T04:23:28.596 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:28.605 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:23:28.637 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:28.632+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:28.712 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:28.712 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2023-11-25T04:23:28.712 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2023-11-25T04:23:28.712 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2023-11-25T04:23:28.712 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.712 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:28.712 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:28.712 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 41.677925385000002, 2023-11-25T04:23:28.713 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01072943, 2023-11-25T04:23:28.713 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:28.713 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:28.713 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:28.713 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.713 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:28.713 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:28.713 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:28.713 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.713 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.713 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:28.713 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:28.713 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:28.714 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.714 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.714 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:28.714 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031829+0000", 2023-11-25T04:23:28.714 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.666e-06 2023-11-25T04:23:28.714 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.714 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.714 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:28.714 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031839+0000", 2023-11-25T04:23:28.714 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9490000000000008e-06 2023-11-25T04:23:28.715 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.715 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.715 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:28.715 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031857+0000", 2023-11-25T04:23:28.715 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8093000000000001e-05 2023-11-25T04:23:28.715 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.715 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.715 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:28.716 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031882+0000", 2023-11-25T04:23:28.716 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4294000000000002e-05 2023-11-25T04:23:28.716 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.716 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.716 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:28.716 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033130+0000", 2023-11-25T04:23:28.716 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001248264 2023-11-25T04:23:28.716 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.716 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.716 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:28.716 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033426+0000", 2023-11-25T04:23:28.716 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029620999999999998 2023-11-25T04:23:28.717 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.717 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.717 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:28.717 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033814+0000", 2023-11-25T04:23:28.717 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000387742 2023-11-25T04:23:28.717 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.717 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.717 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:28.717 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036793+0000", 2023-11-25T04:23:28.717 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002979598 2023-11-25T04:23:28.717 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.717 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.717 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:28.718 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042455+0000", 2023-11-25T04:23:28.718 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0056619260000000003 2023-11-25T04:23:28.718 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.718 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.718 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:28.718 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042502+0000", 2023-11-25T04:23:28.719 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7139999999999999e-05 2023-11-25T04:23:28.719 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.719 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.719 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:28.719 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042557+0000", 2023-11-25T04:23:28.719 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4548e-05 2023-11-25T04:23:28.719 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:28.719 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:28.719 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:28.719 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.719 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.719 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:28.720 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:28.720 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 41.676726850000001, 2023-11-25T04:23:28.720 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011505202000000001, 2023-11-25T04:23:28.720 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:28.720 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:28.720 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:28.720 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.720 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:28.720 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:28.720 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:28.720 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.720 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.721 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:28.721 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:28.721 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:28.721 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.721 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.721 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:28.721 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033027+0000", 2023-11-25T04:23:28.721 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0699999999999999e-06 2023-11-25T04:23:28.721 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.721 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.721 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:28.722 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033035+0000", 2023-11-25T04:23:28.722 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0830000000000004e-06 2023-11-25T04:23:28.722 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.722 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.722 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:28.722 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033059+0000", 2023-11-25T04:23:28.722 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3975e-05 2023-11-25T04:23:28.722 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.722 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.722 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:28.722 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033482+0000", 2023-11-25T04:23:28.722 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042253700000000001 2023-11-25T04:23:28.723 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.723 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.723 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:28.723 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034520+0000", 2023-11-25T04:23:28.723 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001037991 2023-11-25T04:23:28.723 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.723 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.723 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:28.723 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034611+0000", 2023-11-25T04:23:28.723 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1113999999999995e-05 2023-11-25T04:23:28.723 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.723 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.723 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:28.724 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034769+0000", 2023-11-25T04:23:28.724 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000157641 2023-11-25T04:23:28.724 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.724 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.724 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:28.724 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038292+0000", 2023-11-25T04:23:28.724 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035231160000000002 2023-11-25T04:23:28.724 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.724 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.724 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:28.724 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044440+0000", 2023-11-25T04:23:28.724 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0061486980000000002 2023-11-25T04:23:28.725 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.725 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.725 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:28.725 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044495+0000", 2023-11-25T04:23:28.725 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4339000000000003e-05 2023-11-25T04:23:28.725 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.725 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.725 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:28.725 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044531+0000", 2023-11-25T04:23:28.725 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6637999999999997e-05 2023-11-25T04:23:28.725 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:28.725 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:28.726 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:28.726 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.726 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.726 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:28.726 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:28.726 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 41.676252496000004, 2023-11-25T04:23:28.726 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0057634039999999997, 2023-11-25T04:23:28.726 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:28.726 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:28.726 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:28.726 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.726 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:28.727 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:28.727 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:28.727 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.727 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.727 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:28.727 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:28.727 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:28.727 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.727 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.727 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:28.727 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:28.727 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7900000000000007e-07 2023-11-25T04:23:28.727 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.728 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.728 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:28.728 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033507+0000", 2023-11-25T04:23:28.728 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8900000000000004e-06 2023-11-25T04:23:28.728 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.728 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.728 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:28.728 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033519+0000", 2023-11-25T04:23:28.728 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1756000000000001e-05 2023-11-25T04:23:28.728 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.728 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.728 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:28.729 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033527+0000", 2023-11-25T04:23:28.729 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3610000000000001e-06 2023-11-25T04:23:28.729 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.729 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.729 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:28.729 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034451+0000", 2023-11-25T04:23:28.729 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00092372400000000001 2023-11-25T04:23:28.729 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.729 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.729 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:28.729 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034809+0000", 2023-11-25T04:23:28.729 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00035757699999999998 2023-11-25T04:23:28.730 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.730 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.730 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:23:28.730 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034965+0000", 2023-11-25T04:23:28.730 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015638699999999999 2023-11-25T04:23:28.730 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.730 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.730 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:28.730 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038299+0000", 2023-11-25T04:23:28.730 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0033336559999999999 2023-11-25T04:23:28.730 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.730 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.731 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:28.731 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039169+0000", 2023-11-25T04:23:28.731 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00087066400000000005 2023-11-25T04:23:28.731 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.731 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.731 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:28.731 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039226+0000", 2023-11-25T04:23:28.731 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6511999999999997e-05 2023-11-25T04:23:28.731 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.731 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.731 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:28.731 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039264+0000", 2023-11-25T04:23:28.732 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7898000000000002e-05 2023-11-25T04:23:28.732 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:28.732 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:28.732 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:28.732 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.732 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.732 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:25 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:28.732 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:28.732 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 41.676173912000003, 2023-11-25T04:23:28.732 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096656169999999996, 2023-11-25T04:23:28.732 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:28.732 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:28.733 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:28.733 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.733 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:28.733 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:28.733 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:28.733 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.733 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.733 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:28.733 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:28.733 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:28.733 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.733 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.733 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:28.734 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033580+0000", 2023-11-25T04:23:28.734 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8199999999999999e-07 2023-11-25T04:23:28.734 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.734 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.734 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:28.734 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033585+0000", 2023-11-25T04:23:28.734 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3680000000000001e-06 2023-11-25T04:23:28.734 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.734 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.734 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:28.734 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033600+0000", 2023-11-25T04:23:28.734 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4331000000000001e-05 2023-11-25T04:23:28.734 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.735 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.735 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:28.735 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033606+0000", 2023-11-25T04:23:28.735 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5780000000000004e-06 2023-11-25T04:23:28.735 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.735 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.735 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:28.735 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035747+0000", 2023-11-25T04:23:28.736 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0021406530000000002 2023-11-25T04:23:28.736 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.736 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.736 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:28.736 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035901+0000", 2023-11-25T04:23:28.736 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000153931 2023-11-25T04:23:28.736 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.736 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.736 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:23:28.736 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036193+0000", 2023-11-25T04:23:28.736 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029211299999999999 2023-11-25T04:23:28.736 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.737 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.737 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:28.737 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038655+0000", 2023-11-25T04:23:28.737 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0024621640000000002 2023-11-25T04:23:28.737 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.737 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.737 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:28.737 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043130+0000", 2023-11-25T04:23:28.737 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.004474698 2023-11-25T04:23:28.737 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.737 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.737 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:28.737 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043204+0000", 2023-11-25T04:23:28.738 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4091999999999995e-05 2023-11-25T04:23:28.738 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.738 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.738 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:28.738 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043245+0000", 2023-11-25T04:23:28.738 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0907e-05 2023-11-25T04:23:28.738 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:28.738 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:28.738 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:28.738 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.738 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.739 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:26 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:28.739 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:28.739 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 41.676138088000002, 2023-11-25T04:23:28.739 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011086945000000001, 2023-11-25T04:23:28.739 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:28.739 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:28.739 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:28.739 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.739 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:28.739 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:28.740 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:28.740 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.740 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.740 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:28.740 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:28.741 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:28.741 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.741 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.741 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:28.741 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:28.741 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1699999999999998e-07 2023-11-25T04:23:28.741 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.741 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.741 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:28.741 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033627+0000", 2023-11-25T04:23:28.741 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.163e-05 2023-11-25T04:23:28.741 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.742 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.742 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:28.742 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033636+0000", 2023-11-25T04:23:28.742 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6859999999999995e-06 2023-11-25T04:23:28.742 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.742 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.742 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:28.742 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033642+0000", 2023-11-25T04:23:28.742 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3149999999999997e-06 2023-11-25T04:23:28.742 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.742 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.742 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:28.743 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035414+0000", 2023-11-25T04:23:28.743 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0017716380000000001 2023-11-25T04:23:28.743 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.743 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.743 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:28.743 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035539+0000", 2023-11-25T04:23:28.743 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000125477 2023-11-25T04:23:28.743 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.743 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.743 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:28.743 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035672+0000", 2023-11-25T04:23:28.743 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000133316 2023-11-25T04:23:28.743 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.744 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.744 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:28.744 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038649+0000", 2023-11-25T04:23:28.744 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029764679999999999 2023-11-25T04:23:28.744 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.744 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.744 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:28.744 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044578+0000", 2023-11-25T04:23:28.744 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0059293510000000002 2023-11-25T04:23:28.744 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.744 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.744 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:28.744 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044651+0000", 2023-11-25T04:23:28.745 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2513000000000002e-05 2023-11-25T04:23:28.745 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.745 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.745 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:28.745 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044702+0000", 2023-11-25T04:23:28.745 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1034e-05 2023-11-25T04:23:28.745 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:28.745 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:28.745 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:28.745 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.745 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.745 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:29 3.b 3:d0630e4c:::mds0_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:28.746 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:28.746 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 41.676100167999998, 2023-11-25T04:23:28.746 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010980227, 2023-11-25T04:23:28.746 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:28.746 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:28.746 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:28.746 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.746 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:28.746 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:28.746 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:28.746 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.746 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.747 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:28.747 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:28.747 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:28.747 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.747 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.747 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:28.747 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:28.747 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3499999999999996e-07 2023-11-25T04:23:28.747 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.747 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.747 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:28.747 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033658+0000", 2023-11-25T04:23:28.748 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1280000000000001e-06 2023-11-25T04:23:28.748 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.748 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.748 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:28.748 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033667+0000", 2023-11-25T04:23:28.748 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1430000000000004e-06 2023-11-25T04:23:28.748 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.748 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.748 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:28.748 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033678+0000", 2023-11-25T04:23:28.748 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0906e-05 2023-11-25T04:23:28.748 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.749 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.749 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:28.749 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034456+0000", 2023-11-25T04:23:28.749 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00077833899999999996 2023-11-25T04:23:28.749 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.749 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.749 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:28.749 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034575+0000", 2023-11-25T04:23:28.749 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011926000000000001 2023-11-25T04:23:28.749 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.749 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.749 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:28.749 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035001+0000", 2023-11-25T04:23:28.750 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042550999999999998 2023-11-25T04:23:28.750 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.750 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.750 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:28.750 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.037923+0000", 2023-11-25T04:23:28.750 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029219670000000001 2023-11-25T04:23:28.750 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.750 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.750 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:28.750 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044538+0000", 2023-11-25T04:23:28.750 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0066152349999999997 2023-11-25T04:23:28.750 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.751 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.751 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:28.751 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044594+0000", 2023-11-25T04:23:28.751 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5593000000000001e-05 2023-11-25T04:23:28.751 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.751 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.751 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:28.751 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044633+0000", 2023-11-25T04:23:28.751 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9511000000000002e-05 2023-11-25T04:23:28.751 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:28.751 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:28.751 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:28.751 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.752 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.752 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:34 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:28.752 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:28.752 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 41.675850054999998, 2023-11-25T04:23:28.752 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010920445000000001, 2023-11-25T04:23:28.752 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:28.752 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:28.752 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:28.752 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.752 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:28.752 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:28.752 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:28.753 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.753 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.753 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:28.753 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:28.753 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:28.753 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.753 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.753 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:28.753 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033904+0000", 2023-11-25T04:23:28.753 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6599999999999994e-07 2023-11-25T04:23:28.753 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.753 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.753 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:28.754 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033909+0000", 2023-11-25T04:23:28.754 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5639999999999996e-06 2023-11-25T04:23:28.754 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.754 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.754 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:28.754 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033923+0000", 2023-11-25T04:23:28.754 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3028e-05 2023-11-25T04:23:28.754 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.754 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.754 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:28.754 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034043+0000", 2023-11-25T04:23:28.754 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000120328 2023-11-25T04:23:28.755 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.755 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.755 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:28.755 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035576+0000", 2023-11-25T04:23:28.755 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015333720000000001 2023-11-25T04:23:28.755 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.755 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.755 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:28.755 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035699+0000", 2023-11-25T04:23:28.755 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000123001 2023-11-25T04:23:28.755 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.755 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.755 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:28.756 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044823+0000", 2023-11-25T04:23:28.756 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0091241859999999994 2023-11-25T04:23:28.756 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:28.756 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:28.756 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:28.756 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.756 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.756 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:35 3.1f 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:28.756 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:28.756 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 41.674545850000001, 2023-11-25T04:23:28.756 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096164779999999995, 2023-11-25T04:23:28.756 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:28.757 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:28.757 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:28.757 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.757 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:28.757 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:28.757 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:28.757 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.757 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.757 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:28.757 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:28.757 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:28.757 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.757 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.758 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:28.758 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035208+0000", 2023-11-25T04:23:28.758 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2699999999999998e-07 2023-11-25T04:23:28.758 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.758 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.758 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:28.758 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035213+0000", 2023-11-25T04:23:28.758 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.6070000000000001e-06 2023-11-25T04:23:28.758 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.758 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.758 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:28.758 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035350+0000", 2023-11-25T04:23:28.759 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013693599999999999 2023-11-25T04:23:28.759 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.759 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.759 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:28.759 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035358+0000", 2023-11-25T04:23:28.759 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0099999999999995e-06 2023-11-25T04:23:28.759 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.759 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.759 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:28.759 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036260+0000", 2023-11-25T04:23:28.759 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00090240100000000005 2023-11-25T04:23:28.759 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.759 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.760 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:28.760 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036517+0000", 2023-11-25T04:23:28.760 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00025736299999999998 2023-11-25T04:23:28.760 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.760 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.760 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:28.760 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044824+0000", 2023-11-25T04:23:28.760 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0083063340000000003 2023-11-25T04:23:28.760 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:28.760 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:28.760 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:28.760 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.760 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.761 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:37 3.1e 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:28.761 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:28.761 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 41.674379164000001, 2023-11-25T04:23:28.761 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0078322640000000002, 2023-11-25T04:23:28.761 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:28.761 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:28.761 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:28.761 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.761 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:28.761 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:28.761 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:28.761 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.762 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.762 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:28.762 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:28.762 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:28.762 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.762 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.762 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:28.762 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035375+0000", 2023-11-25T04:23:28.762 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9100000000000003e-07 2023-11-25T04:23:28.762 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.762 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.762 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:28.763 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035379+0000", 2023-11-25T04:23:28.763 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9430000000000002e-06 2023-11-25T04:23:28.763 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.763 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.763 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:28.763 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035388+0000", 2023-11-25T04:23:28.763 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2760000000000006e-06 2023-11-25T04:23:28.763 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.763 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.763 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:28.763 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035396+0000", 2023-11-25T04:23:28.763 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8480000000000001e-06 2023-11-25T04:23:28.764 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.764 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.764 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:28.764 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035452+0000", 2023-11-25T04:23:28.764 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5831000000000001e-05 2023-11-25T04:23:28.764 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.764 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.764 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:28.764 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035934+0000", 2023-11-25T04:23:28.764 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00048265599999999998 2023-11-25T04:23:28.764 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.764 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.764 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:28.765 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043206+0000", 2023-11-25T04:23:28.765 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.007271919 2023-11-25T04:23:28.765 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:28.765 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:28.765 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:28.765 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.765 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.765 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing(3.1a epoch 18/16 rep_tid 3 entries 18'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:36 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:23:28.765 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:28.765 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 41.673103578000003, 2023-11-25T04:23:28.765 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002073414, 2023-11-25T04:23:28.765 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:28.765 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:28.766 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:28.766 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.766 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:28.766 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:28.766 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:28.766 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.766 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.766 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:28.766 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:28.766 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:28.766 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.766 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.767 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:28.767 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036650+0000", 2023-11-25T04:23:28.767 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7300000000000002e-07 2023-11-25T04:23:28.767 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.767 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.767 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:28.767 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036659+0000", 2023-11-25T04:23:28.767 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.7499999999999992e-06 2023-11-25T04:23:28.767 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.767 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.767 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:28.767 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036673+0000", 2023-11-25T04:23:28.768 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4917e-05 2023-11-25T04:23:28.768 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.768 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.768 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:28.768 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036687+0000", 2023-11-25T04:23:28.768 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3883000000000001e-05 2023-11-25T04:23:28.768 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.768 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.768 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:28.768 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036781+0000", 2023-11-25T04:23:28.768 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3219000000000003e-05 2023-11-25T04:23:28.768 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.769 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.769 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:28.769 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038723+0000", 2023-11-25T04:23:28.769 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019422720000000001 2023-11-25T04:23:28.769 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:28.769 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:28.769 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:28.769 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.769 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.769 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:28 3.13 e18/16 3:cd5a64a3:::100.00000000:head v 18'1, mlcod=0'0)", 2023-11-25T04:23:28.769 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:28.769 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 41.669442420000003, 2023-11-25T04:23:28.770 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001850546, 2023-11-25T04:23:28.770 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:28.770 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:28.770 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:28.770 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.770 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:28.770 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:28.770 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:28.770 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.770 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.770 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:28.770 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:28.770 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:28.771 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.771 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.771 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:28.771 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:28.771 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7800000000000002e-07 2023-11-25T04:23:28.771 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.771 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.771 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:28.771 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040331+0000", 2023-11-25T04:23:28.771 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0449999999999999e-05 2023-11-25T04:23:28.771 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.771 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.772 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:28.772 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040339+0000", 2023-11-25T04:23:28.772 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0709999999999995e-06 2023-11-25T04:23:28.772 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.772 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.772 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:28.772 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040347+0000", 2023-11-25T04:23:28.772 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7079999999999997e-06 2023-11-25T04:23:28.772 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.772 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.772 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:28.772 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040410+0000", 2023-11-25T04:23:28.773 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2453000000000007e-05 2023-11-25T04:23:28.773 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.773 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.773 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:28.773 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040422+0000", 2023-11-25T04:23:28.773 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2429000000000001e-05 2023-11-25T04:23:28.773 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.773 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.773 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:28.773 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042129+0000", 2023-11-25T04:23:28.773 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001707266 2023-11-25T04:23:28.773 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.773 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.774 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:28.774 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042161+0000", 2023-11-25T04:23:28.774 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1791000000000001e-05 2023-11-25T04:23:28.774 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:28.774 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:28.774 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:28.774 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.774 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.774 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:27 3.13 e18/16 3:c91d4a1d:::mds0_inotable:head v 18'2, mlcod=0'0)", 2023-11-25T04:23:28.774 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:28.774 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 41.669238897, 2023-11-25T04:23:28.774 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0016886430000000001, 2023-11-25T04:23:28.775 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:28.775 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:28.775 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:28.775 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.775 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:28.775 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:28.775 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:28.775 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.775 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.775 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:28.775 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:28.775 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:28.775 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.776 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.776 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:28.776 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:28.776 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3999999999999997e-07 2023-11-25T04:23:28.776 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.776 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.776 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:28.776 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040520+0000", 2023-11-25T04:23:28.776 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2989999999999996e-06 2023-11-25T04:23:28.776 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.776 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.776 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:28.777 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040527+0000", 2023-11-25T04:23:28.777 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5730000000000002e-06 2023-11-25T04:23:28.777 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.777 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.777 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:28.777 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040535+0000", 2023-11-25T04:23:28.777 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8630000000000008e-06 2023-11-25T04:23:28.777 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.777 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.777 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:28.777 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040683+0000", 2023-11-25T04:23:28.777 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000148002 2023-11-25T04:23:28.778 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.778 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.778 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:28.778 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040700+0000", 2023-11-25T04:23:28.778 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7065e-05 2023-11-25T04:23:28.778 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.778 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.778 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:28.778 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042163+0000", 2023-11-25T04:23:28.778 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001462767 2023-11-25T04:23:28.778 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.778 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.779 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:28.779 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042203+0000", 2023-11-25T04:23:28.779 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9734000000000003e-05 2023-11-25T04:23:28.779 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:28.779 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:28.779 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:28.779 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.779 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.779 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:20 3.1c e18/16 3:3ed09add:::607.00000000:head v 18'1, mlcod=0'0)", 2023-11-25T04:23:28.779 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:28.779 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 41.669203856000003, 2023-11-25T04:23:28.779 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015271, 2023-11-25T04:23:28.780 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:28.780 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:28.780 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:28.780 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.780 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:28.780 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:28.780 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:28.780 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.780 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.780 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:28.780 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:28.780 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:28.780 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.781 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.781 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:28.781 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:28.781 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4799999999999999e-07 2023-11-25T04:23:28.781 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.781 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.781 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:28.781 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040561+0000", 2023-11-25T04:23:28.781 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1800000000000001e-05 2023-11-25T04:23:28.781 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.781 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.781 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:28.782 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040568+0000", 2023-11-25T04:23:28.782 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7730000000000002e-06 2023-11-25T04:23:28.782 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.782 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.782 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:28.782 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040587+0000", 2023-11-25T04:23:28.782 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8691000000000001e-05 2023-11-25T04:23:28.782 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.782 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.782 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:28.782 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040638+0000", 2023-11-25T04:23:28.782 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1068e-05 2023-11-25T04:23:28.783 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.783 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.783 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:28.783 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040670+0000", 2023-11-25T04:23:28.783 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2125999999999997e-05 2023-11-25T04:23:28.783 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.783 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.783 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:28.783 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042032+0000", 2023-11-25T04:23:28.783 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001361821 2023-11-25T04:23:28.783 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.783 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.783 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:28.784 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042076+0000", 2023-11-25T04:23:28.784 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4472999999999999e-05 2023-11-25T04:23:28.784 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:28.784 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:28.784 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:28.784 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.784 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.784 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:21 3.1c e18/16 3:3ed09add:::607.00000000:head v 18'2, mlcod=0'0)", 2023-11-25T04:23:28.784 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:28.784 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 41.668969130999997, 2023-11-25T04:23:28.784 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0013228160000000001, 2023-11-25T04:23:28.784 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:28.784 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:28.785 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:28.785 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.785 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:28.785 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:28.785 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:28.785 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.785 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.785 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:28.785 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:28.785 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:28.785 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.785 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.786 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:28.786 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:28.786 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9299999999999999e-07 2023-11-25T04:23:28.786 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.786 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.786 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:28.786 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040790+0000", 2023-11-25T04:23:28.786 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5770000000000001e-06 2023-11-25T04:23:28.786 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.786 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.786 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:28.786 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040799+0000", 2023-11-25T04:23:28.787 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6439999999999995e-06 2023-11-25T04:23:28.787 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.787 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.787 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:28.787 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040806+0000", 2023-11-25T04:23:28.787 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0180000000000004e-06 2023-11-25T04:23:28.787 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.787 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.787 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:28.787 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040907+0000", 2023-11-25T04:23:28.787 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000100096 2023-11-25T04:23:28.787 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.788 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.788 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:28.788 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040925+0000", 2023-11-25T04:23:28.788 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8536999999999999e-05 2023-11-25T04:23:28.788 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.788 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.788 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:28.788 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042079+0000", 2023-11-25T04:23:28.788 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00115358 2023-11-25T04:23:28.788 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.788 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.788 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:28.788 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042107+0000", 2023-11-25T04:23:28.789 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7971e-05 2023-11-25T04:23:28.789 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:28.789 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:28.789 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:28.789 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.789 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.789 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop_reply(mds.0.3:4 3.1f e18/16 ondisk, result = 0)", 2023-11-25T04:23:28.789 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:28.789 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 41.667629740999999, 2023-11-25T04:23:28.789 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00044092900000000001, 2023-11-25T04:23:28.789 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:28.789 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:28.790 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:28.790 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.790 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:28.790 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:28.790 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:28.790 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.790 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.790 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:28.790 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:28.790 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:28.790 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.790 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.791 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:28.791 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042124+0000", 2023-11-25T04:23:28.791 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1800000000000005e-07 2023-11-25T04:23:28.791 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.791 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.791 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:28.791 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042128+0000", 2023-11-25T04:23:28.791 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2289999999999997e-06 2023-11-25T04:23:28.791 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.791 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.791 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:28.791 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042138+0000", 2023-11-25T04:23:28.791 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9729999999999992e-06 2023-11-25T04:23:28.792 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.792 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.792 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:28.792 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042145+0000", 2023-11-25T04:23:28.792 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.336e-06 2023-11-25T04:23:28.792 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.792 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.792 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:28.792 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042441+0000", 2023-11-25T04:23:28.792 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029650500000000002 2023-11-25T04:23:28.792 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.792 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.793 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:28.793 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042448+0000", 2023-11-25T04:23:28.793 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4100000000000002e-06 2023-11-25T04:23:28.793 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.793 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.793 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:28.793 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042564+0000", 2023-11-25T04:23:28.793 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000115758 2023-11-25T04:23:28.793 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:28.793 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:28.793 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:28.793 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.794 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.794 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing_reply(3.1f epoch 18/16 rep_tid 8 lcod 18'4)", 2023-11-25T04:23:28.794 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:28.794 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 41.665304329000001, 2023-11-25T04:23:28.794 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00038154699999999998, 2023-11-25T04:23:28.794 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:28.794 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:28.794 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:28.794 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.794 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:28.794 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:28.794 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:28.795 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.795 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.795 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:28.795 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:28.795 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:28.795 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.795 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.795 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:28.795 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:28.795 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2700000000000001e-07 2023-11-25T04:23:28.795 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.795 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.796 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:28.796 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044451+0000", 2023-11-25T04:23:28.796 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7460000000000001e-06 2023-11-25T04:23:28.796 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.796 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.796 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:28.796 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044455+0000", 2023-11-25T04:23:28.796 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5809999999999998e-06 2023-11-25T04:23:28.796 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.796 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.796 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:28.796 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044458+0000", 2023-11-25T04:23:28.797 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0170000000000001e-06 2023-11-25T04:23:28.797 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.797 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.797 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:28.797 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044734+0000", 2023-11-25T04:23:28.797 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00027582799999999999 2023-11-25T04:23:28.797 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.797 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.797 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:28.797 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044830+0000", 2023-11-25T04:23:28.797 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6248000000000004e-05 2023-11-25T04:23:28.797 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:28.797 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:28.798 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:28.798 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.798 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.798 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing_reply(3.b epoch 18/16 rep_tid 5 lcod 18'2)", 2023-11-25T04:23:28.798 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:28.798 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 41.665267667999998, 2023-11-25T04:23:28.798 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000344488, 2023-11-25T04:23:28.798 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:28.798 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:28.798 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:28.798 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.798 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:28.799 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:28.799 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:28.799 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.799 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.799 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:28.799 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:28.799 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:28.799 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.799 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.799 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:28.799 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:28.799 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.55e-07 2023-11-25T04:23:28.800 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.800 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.800 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:28.800 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044487+0000", 2023-11-25T04:23:28.800 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2470000000000001e-06 2023-11-25T04:23:28.800 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.800 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.800 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:28.800 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044491+0000", 2023-11-25T04:23:28.800 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9659999999999998e-06 2023-11-25T04:23:28.800 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.800 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.800 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:28.801 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044494+0000", 2023-11-25T04:23:28.801 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2310000000000001e-06 2023-11-25T04:23:28.801 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.801 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.801 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:28.801 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044756+0000", 2023-11-25T04:23:28.801 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00026172800000000003 2023-11-25T04:23:28.801 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.801 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.801 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:28.801 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044830+0000", 2023-11-25T04:23:28.801 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4160999999999997e-05 2023-11-25T04:23:28.802 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:28.802 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:28.802 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:28.802 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.802 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.802 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:39 3.6 3:654134d2:::mds0_openfiles.0:head [omap-clear,omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2023-11-25T04:23:28.802 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:28.802 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 36.605441626000001, 2023-11-25T04:23:28.802 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001788658, 2023-11-25T04:23:28.802 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:28.802 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:28.802 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:28.803 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.803 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:28.803 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:28.803 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:28.803 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.803 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.803 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:28.803 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:28.803 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:28.803 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.803 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.803 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:28.804 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104314+0000", 2023-11-25T04:23:28.804 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3099999999999999e-06 2023-11-25T04:23:28.804 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.804 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.804 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:28.804 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104325+0000", 2023-11-25T04:23:28.804 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1511999999999999e-05 2023-11-25T04:23:28.804 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.804 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.804 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:28.804 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104362+0000", 2023-11-25T04:23:28.804 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6903000000000002e-05 2023-11-25T04:23:28.804 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.805 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.805 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:28.805 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104387+0000", 2023-11-25T04:23:28.805 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4766e-05 2023-11-25T04:23:28.805 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.805 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.805 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:28.805 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104465+0000", 2023-11-25T04:23:28.805 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8119000000000002e-05 2023-11-25T04:23:28.805 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.805 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.805 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:28.805 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104557+0000", 2023-11-25T04:23:28.806 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2496000000000003e-05 2023-11-25T04:23:28.806 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.806 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.806 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:28.806 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104646+0000", 2023-11-25T04:23:28.806 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8818000000000002e-05 2023-11-25T04:23:28.806 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.806 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.806 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:28.806 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105201+0000", 2023-11-25T04:23:28.806 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00055506000000000002 2023-11-25T04:23:28.806 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.807 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.807 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:28.807 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105994+0000", 2023-11-25T04:23:28.807 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00079294700000000001 2023-11-25T04:23:28.807 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.807 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.807 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:28.807 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.106055+0000", 2023-11-25T04:23:28.807 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0566000000000003e-05 2023-11-25T04:23:28.807 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.807 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.807 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:28.807 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.106100+0000", 2023-11-25T04:23:28.808 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5160999999999997e-05 2023-11-25T04:23:28.808 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:28.808 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:28.808 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:28.808 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.808 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.808 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:43 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2023-11-25T04:23:28.808 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:28.808 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 21.605216630000001, 2023-11-25T04:23:28.808 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001751166, 2023-11-25T04:23:28.808 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:28.808 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:28.809 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:28.809 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.809 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:28.809 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:28.809 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:28.809 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.809 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.809 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:28.809 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:28.809 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:28.809 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.809 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.809 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:28.810 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104539+0000", 2023-11-25T04:23:28.810 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9529999999999999e-06 2023-11-25T04:23:28.810 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.810 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.810 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:28.810 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104557+0000", 2023-11-25T04:23:28.810 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7464000000000001e-05 2023-11-25T04:23:28.810 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.810 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.810 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:28.810 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104590+0000", 2023-11-25T04:23:28.810 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2966000000000003e-05 2023-11-25T04:23:28.811 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.811 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.811 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:28.811 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104608+0000", 2023-11-25T04:23:28.811 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8193e-05 2023-11-25T04:23:28.811 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.811 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.811 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:28.811 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104767+0000", 2023-11-25T04:23:28.811 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015882 2023-11-25T04:23:28.811 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.811 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.811 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:28.812 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104909+0000", 2023-11-25T04:23:28.812 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014244999999999999 2023-11-25T04:23:28.812 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.812 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.812 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:28.812 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.105059+0000", 2023-11-25T04:23:28.812 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014964399999999999 2023-11-25T04:23:28.812 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.812 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.812 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:28.812 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.105701+0000", 2023-11-25T04:23:28.812 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00064201099999999995 2023-11-25T04:23:28.812 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.813 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.813 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:28.813 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106233+0000", 2023-11-25T04:23:28.813 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00053178500000000001 2023-11-25T04:23:28.813 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.813 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.813 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:28.813 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106265+0000", 2023-11-25T04:23:28.813 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1986000000000002e-05 2023-11-25T04:23:28.813 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.813 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.813 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:28.813 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106288+0000", 2023-11-25T04:23:28.814 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2894000000000001e-05 2023-11-25T04:23:28.814 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:28.814 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:28.814 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:28.814 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.814 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.814 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:45 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e23)", 2023-11-25T04:23:28.814 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:23:28.814 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 16.60501859, 2023-11-25T04:23:28.815 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00098423299999999994, 2023-11-25T04:23:28.815 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:28.815 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:28.815 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:28.815 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.815 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:28.815 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:23:28.815 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:28.815 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.815 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.815 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:28.815 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:23:28.816 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:28.816 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.816 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.816 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:28.816 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104737+0000", 2023-11-25T04:23:28.816 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3309999999999998e-06 2023-11-25T04:23:28.816 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.816 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.816 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:28.816 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104748+0000", 2023-11-25T04:23:28.816 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1080999999999999e-05 2023-11-25T04:23:28.816 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.817 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.817 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:28.817 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104771+0000", 2023-11-25T04:23:28.817 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2707000000000001e-05 2023-11-25T04:23:28.817 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.817 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.817 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:28.817 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104800+0000", 2023-11-25T04:23:28.817 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9595e-05 2023-11-25T04:23:28.817 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.817 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.817 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:28.818 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104890+0000", 2023-11-25T04:23:28.818 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.9808999999999993e-05 2023-11-25T04:23:28.818 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.818 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.818 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:28.818 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105004+0000", 2023-11-25T04:23:28.818 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000114508 2023-11-25T04:23:28.818 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.818 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.818 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:28.818 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105634+0000", 2023-11-25T04:23:28.818 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00062985799999999996 2023-11-25T04:23:28.819 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.819 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.819 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:28.819 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105671+0000", 2023-11-25T04:23:28.819 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6640000000000002e-05 2023-11-25T04:23:28.819 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:28.819 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:28.819 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:28.819 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105719+0000", 2023-11-25T04:23:28.819 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7704000000000003e-05 2023-11-25T04:23:28.819 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:28.819 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:28.819 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:28.820 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:28.820 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:28.820 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:28.821 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:23:28.822 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:28.819+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:28.905 INFO:teuthology.orchestra.run.smithi067.stderr:noscrub is set 2023-11-25T04:23:28.922 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:28.921+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:28.932 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:28.932 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:23:28.932 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2023-11-25T04:23:28.932 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:28.941 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:23:29.022 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:29.021+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:29.046 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:29.046 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:23:29.046 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2023-11-25T04:23:29.046 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2023-11-25T04:23:29.046 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:29.055 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:23:29.126 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:29.124+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:29.161 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:29.162 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2023-11-25T04:23:29.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2023-11-25T04:23:29.162 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2023-11-25T04:23:29.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.163 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:155 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'109, mlcod=11'105)", 2023-11-25T04:23:29.163 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:29.163 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.374055108, 2023-11-25T04:23:29.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061719019999999999, 2023-11-25T04:23:29.163 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:29.163 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:29.163 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:29.164 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.164 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:29.164 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:29.164 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:29.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.164 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.165 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:29.165 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:29.165 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:29.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.165 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.165 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:29.165 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784884+0000", 2023-11-25T04:23:29.166 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3099999999999999e-07 2023-11-25T04:23:29.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.166 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:29.166 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784942+0000", 2023-11-25T04:23:29.166 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8570000000000003e-05 2023-11-25T04:23:29.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.167 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:29.167 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784949+0000", 2023-11-25T04:23:29.167 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0310000000000001e-06 2023-11-25T04:23:29.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.168 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.168 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:29.168 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784953+0000", 2023-11-25T04:23:29.168 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6320000000000001e-06 2023-11-25T04:23:29.168 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.168 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.169 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:29.169 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784993+0000", 2023-11-25T04:23:29.169 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9737000000000003e-05 2023-11-25T04:23:29.169 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.169 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.169 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:29.170 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785011+0000", 2023-11-25T04:23:29.170 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8349999999999999e-05 2023-11-25T04:23:29.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.170 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:29.170 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791024+0000", 2023-11-25T04:23:29.171 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006012997 2023-11-25T04:23:29.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.171 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:29.171 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791055+0000", 2023-11-25T04:23:29.171 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1254000000000003e-05 2023-11-25T04:23:29.171 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:29.172 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:29.172 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:29.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.172 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:156 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'110, mlcod=11'105)", 2023-11-25T04:23:29.173 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:29.173 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.373469059000001, 2023-11-25T04:23:29.173 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005616116, 2023-11-25T04:23:29.173 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:29.173 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:29.173 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:29.174 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.174 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:29.174 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:29.174 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:29.174 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.174 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.175 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:29.175 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:29.175 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:29.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.175 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.175 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:29.176 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785470+0000", 2023-11-25T04:23:29.176 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0699999999999998e-07 2023-11-25T04:23:29.176 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.176 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.176 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:29.176 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785474+0000", 2023-11-25T04:23:29.176 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.048e-06 2023-11-25T04:23:29.177 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.177 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.177 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:29.177 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785480+0000", 2023-11-25T04:23:29.177 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7729999999999996e-06 2023-11-25T04:23:29.178 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.178 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.178 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:29.178 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785483+0000", 2023-11-25T04:23:29.178 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1970000000000001e-06 2023-11-25T04:23:29.178 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.179 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.179 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:29.179 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787936+0000", 2023-11-25T04:23:29.179 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024529040000000001 2023-11-25T04:23:29.179 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.179 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.179 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:29.180 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787947+0000", 2023-11-25T04:23:29.180 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1669e-05 2023-11-25T04:23:29.180 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.180 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.180 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:29.180 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791057+0000", 2023-11-25T04:23:29.181 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031095200000000002 2023-11-25T04:23:29.181 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.181 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.181 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:29.181 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791086+0000", 2023-11-25T04:23:29.181 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8697999999999999e-05 2023-11-25T04:23:29.182 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:29.182 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:29.182 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:29.182 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.182 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.182 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:157 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'111, mlcod=11'106)", 2023-11-25T04:23:29.183 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:29.183 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.372876501, 2023-11-25T04:23:29.183 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050590280000000001, 2023-11-25T04:23:29.183 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:29.183 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:29.183 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:29.183 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.184 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:29.184 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:29.184 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:29.185 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.185 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.185 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:29.185 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:29.185 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:29.185 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.186 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.186 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:29.186 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786063+0000", 2023-11-25T04:23:29.186 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2600000000000002e-07 2023-11-25T04:23:29.186 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.186 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.187 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:29.187 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786069+0000", 2023-11-25T04:23:29.187 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8839999999999999e-06 2023-11-25T04:23:29.187 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.187 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.187 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:29.188 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786079+0000", 2023-11-25T04:23:29.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0124e-05 2023-11-25T04:23:29.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.188 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:29.188 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786085+0000", 2023-11-25T04:23:29.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6660000000000002e-06 2023-11-25T04:23:29.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:29.189 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788160+0000", 2023-11-25T04:23:29.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020747669999999999 2023-11-25T04:23:29.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.190 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:29.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788180+0000", 2023-11-25T04:23:29.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0208999999999999e-05 2023-11-25T04:23:29.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.191 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:29.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791087+0000", 2023-11-25T04:23:29.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029064630000000002 2023-11-25T04:23:29.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.191 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:29.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791121+0000", 2023-11-25T04:23:29.192 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4289000000000003e-05 2023-11-25T04:23:29.192 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:29.192 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:29.192 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:29.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.192 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:158 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'112, mlcod=11'106)", 2023-11-25T04:23:29.192 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:29.192 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.372460107999999, 2023-11-25T04:23:29.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046663030000000001, 2023-11-25T04:23:29.193 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:29.193 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:29.193 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:29.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.193 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:29.193 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:29.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:29.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:29.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:29.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:29.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:29.195 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786479+0000", 2023-11-25T04:23:29.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2799999999999996e-07 2023-11-25T04:23:29.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.195 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:29.195 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786498+0000", 2023-11-25T04:23:29.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9069e-05 2023-11-25T04:23:29.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.196 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:29.196 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786508+0000", 2023-11-25T04:23:29.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5810000000000003e-06 2023-11-25T04:23:29.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:29.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786512+0000", 2023-11-25T04:23:29.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4889999999999997e-06 2023-11-25T04:23:29.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:29.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788438+0000", 2023-11-25T04:23:29.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019257059999999999 2023-11-25T04:23:29.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:29.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788449+0000", 2023-11-25T04:23:29.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1419e-05 2023-11-25T04:23:29.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:29.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791122+0000", 2023-11-25T04:23:29.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002672822 2023-11-25T04:23:29.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.200 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:29.200 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791145+0000", 2023-11-25T04:23:29.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2589e-05 2023-11-25T04:23:29.200 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:29.200 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:29.201 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:29.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.201 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:206 1.0 e12/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 12'152, mlcod=12'148)", 2023-11-25T04:23:29.201 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:29.201 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.860691637999999, 2023-11-25T04:23:29.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046728189999999999, 2023-11-25T04:23:29.202 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:29.202 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:29.202 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:29.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.202 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:29.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:29.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:29.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:29.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:29.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:29.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.204 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:29.204 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298248+0000", 2023-11-25T04:23:29.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3600000000000003e-07 2023-11-25T04:23:29.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.205 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:29.205 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298278+0000", 2023-11-25T04:23:29.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0474e-05 2023-11-25T04:23:29.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:29.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298288+0000", 2023-11-25T04:23:29.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0156000000000001e-05 2023-11-25T04:23:29.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:29.207 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298293+0000", 2023-11-25T04:23:29.207 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7400000000000004e-06 2023-11-25T04:23:29.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.207 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:29.207 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298333+0000", 2023-11-25T04:23:29.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9666000000000003e-05 2023-11-25T04:23:29.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.208 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:29.208 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298351+0000", 2023-11-25T04:23:29.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8544999999999999e-05 2023-11-25T04:23:29.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.209 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.209 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:29.209 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302891+0000", 2023-11-25T04:23:29.209 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045398519999999996 2023-11-25T04:23:29.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.209 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.210 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:29.210 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302920+0000", 2023-11-25T04:23:29.210 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8750000000000001e-05 2023-11-25T04:23:29.210 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:29.210 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:29.210 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:29.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.211 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:14 3.15 3:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:29.211 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:29.211 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.125557280999999, 2023-11-25T04:23:29.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010990437, 2023-11-25T04:23:29.211 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:29.211 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:29.212 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:29.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.212 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:29.212 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:29.212 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:29.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.213 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:29.213 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:29.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:29.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.213 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:29.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033383+0000", 2023-11-25T04:23:29.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9870000000000002e-06 2023-11-25T04:23:29.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:29.215 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033398+0000", 2023-11-25T04:23:29.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4569999999999999e-05 2023-11-25T04:23:29.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:29.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033411+0000", 2023-11-25T04:23:29.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3543999999999999e-05 2023-11-25T04:23:29.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:29.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033423+0000", 2023-11-25T04:23:29.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1905999999999999e-05 2023-11-25T04:23:29.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:29.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034433+0000", 2023-11-25T04:23:29.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00101022 2023-11-25T04:23:29.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.217 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:29.217 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034539+0000", 2023-11-25T04:23:29.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010597199999999999 2023-11-25T04:23:29.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.217 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:29.217 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034667+0000", 2023-11-25T04:23:29.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012765500000000001 2023-11-25T04:23:29.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.218 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:29.218 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037803+0000", 2023-11-25T04:23:29.218 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031362220000000001 2023-11-25T04:23:29.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.218 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:29.218 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:23:29.218 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0064964669999999997 2023-11-25T04:23:29.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.219 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:29.219 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044331+0000", 2023-11-25T04:23:29.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1544999999999997e-05 2023-11-25T04:23:29.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.219 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:29.219 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044372+0000", 2023-11-25T04:23:29.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0349000000000003e-05 2023-11-25T04:23:29.219 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:29.219 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:29.219 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:29.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.220 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:15 3.15 3:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:29.220 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:29.220 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.124897631000003, 2023-11-25T04:23:29.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010464362, 2023-11-25T04:23:29.220 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:29.220 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:29.220 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:29.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:29.220 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:29.221 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:29.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.221 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.221 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:29.221 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:29.221 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:29.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.221 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.221 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:29.221 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034042+0000", 2023-11-25T04:23:29.221 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0999999999999997e-07 2023-11-25T04:23:29.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.222 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:29.222 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034048+0000", 2023-11-25T04:23:29.222 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0390000000000001e-06 2023-11-25T04:23:29.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.222 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:29.222 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034059+0000", 2023-11-25T04:23:29.222 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1182999999999999e-05 2023-11-25T04:23:29.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.223 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.223 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:29.223 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034067+0000", 2023-11-25T04:23:29.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1559999999999995e-06 2023-11-25T04:23:29.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.223 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.223 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:29.223 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035005+0000", 2023-11-25T04:23:29.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00093747399999999999 2023-11-25T04:23:29.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.223 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.223 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:29.223 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035077+0000", 2023-11-25T04:23:29.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2747e-05 2023-11-25T04:23:29.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:29.224 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035219+0000", 2023-11-25T04:23:29.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000141877 2023-11-25T04:23:29.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:29.224 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037810+0000", 2023-11-25T04:23:29.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002590752 2023-11-25T04:23:29.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.225 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:29.225 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044437+0000", 2023-11-25T04:23:29.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066270110000000004 2023-11-25T04:23:29.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.225 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:29.225 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044481+0000", 2023-11-25T04:23:29.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4106000000000001e-05 2023-11-25T04:23:29.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.226 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:29.226 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044505+0000", 2023-11-25T04:23:29.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4207e-05 2023-11-25T04:23:29.226 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:29.226 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:29.226 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:29.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.226 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:16 3.1d 3:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:29.226 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:29.226 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.124855783000001, 2023-11-25T04:23:29.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010413923, 2023-11-25T04:23:29.227 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:29.227 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:29.227 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:29.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:29.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:29.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:29.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:29.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:29.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:29.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.228 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:29.228 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:29.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4499999999999997e-07 2023-11-25T04:23:29.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.228 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:29.228 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034087+0000", 2023-11-25T04:23:29.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9770000000000002e-06 2023-11-25T04:23:29.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.228 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:29.229 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034094+0000", 2023-11-25T04:23:29.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8809999999999998e-06 2023-11-25T04:23:29.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.229 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:29.229 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034100+0000", 2023-11-25T04:23:29.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8440000000000003e-06 2023-11-25T04:23:29.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.229 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:29.229 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034993+0000", 2023-11-25T04:23:29.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00089336799999999996 2023-11-25T04:23:29.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:29.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035099+0000", 2023-11-25T04:23:29.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106037 2023-11-25T04:23:29.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:29.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035276+0000", 2023-11-25T04:23:29.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017631799999999999 2023-11-25T04:23:29.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:29.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037815+0000", 2023-11-25T04:23:29.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025395959999999999 2023-11-25T04:23:29.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:29.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044420+0000", 2023-11-25T04:23:29.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066045749999999997 2023-11-25T04:23:29.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:29.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044463+0000", 2023-11-25T04:23:29.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3205999999999999e-05 2023-11-25T04:23:29.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:29.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044497+0000", 2023-11-25T04:23:29.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3476000000000003e-05 2023-11-25T04:23:29.232 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:29.232 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:29.232 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:29.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.232 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:17 3.1d 3:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:29.232 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:29.232 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.124830574000001, 2023-11-25T04:23:29.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01127356, 2023-11-25T04:23:29.233 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:29.233 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:29.233 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:29.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.233 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:29.233 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:29.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:29.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.233 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:29.233 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:29.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:29.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:29.234 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:29.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1100000000000001e-07 2023-11-25T04:23:29.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:29.234 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034112+0000", 2023-11-25T04:23:29.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8600000000000003e-06 2023-11-25T04:23:29.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:29.234 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034120+0000", 2023-11-25T04:23:29.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7700000000000001e-06 2023-11-25T04:23:29.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.235 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:29.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034127+0000", 2023-11-25T04:23:29.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0999999999999998e-06 2023-11-25T04:23:29.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.235 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:29.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035601+0000", 2023-11-25T04:23:29.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014739410000000001 2023-11-25T04:23:29.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:29.236 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035685+0000", 2023-11-25T04:23:29.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3677999999999994e-05 2023-11-25T04:23:29.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:29.236 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035827+0000", 2023-11-25T04:23:29.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014232599999999999 2023-11-25T04:23:29.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:29.236 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038725+0000", 2023-11-25T04:23:29.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028982779999999998 2023-11-25T04:23:29.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:29.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045323+0000", 2023-11-25T04:23:29.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0065980780000000003 2023-11-25T04:23:29.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:29.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045354+0000", 2023-11-25T04:23:29.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0463e-05 2023-11-25T04:23:29.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:29.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045381+0000", 2023-11-25T04:23:29.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7654999999999999e-05 2023-11-25T04:23:29.238 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:29.238 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:29.238 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:29.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.238 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:22 3.e 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:29.238 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:29.238 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.124804161, 2023-11-25T04:23:29.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010133422, 2023-11-25T04:23:29.239 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:29.239 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:29.239 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:29.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.239 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:29.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:29.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:29.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.239 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:29.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:29.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:29.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:29.240 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034135+0000", 2023-11-25T04:23:29.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-07 2023-11-25T04:23:29.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:29.240 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034138+0000", 2023-11-25T04:23:29.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6009999999999999e-06 2023-11-25T04:23:29.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:29.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034145+0000", 2023-11-25T04:23:29.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1169999999999998e-06 2023-11-25T04:23:29.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:29.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034150+0000", 2023-11-25T04:23:29.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9080000000000003e-06 2023-11-25T04:23:29.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:29.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034209+0000", 2023-11-25T04:23:29.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8149000000000002e-05 2023-11-25T04:23:29.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:29.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034415+0000", 2023-11-25T04:23:29.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020637699999999999 2023-11-25T04:23:29.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:29.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034584+0000", 2023-11-25T04:23:29.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016870399999999999 2023-11-25T04:23:29.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:29.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.036979+0000", 2023-11-25T04:23:29.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023957509999999998 2023-11-25T04:23:29.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:29.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044161+0000", 2023-11-25T04:23:29.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0071811590000000003 2023-11-25T04:23:29.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:29.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044224+0000", 2023-11-25T04:23:29.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3454000000000006e-05 2023-11-25T04:23:29.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.244 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:29.244 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044268+0000", 2023-11-25T04:23:29.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3761999999999999e-05 2023-11-25T04:23:29.244 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:29.244 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:29.244 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:29.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.244 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:23 3.e 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:29.244 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:29.244 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.124772751999998, 2023-11-25T04:23:29.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010281676, 2023-11-25T04:23:29.245 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:29.245 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:29.245 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:29.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.245 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:29.245 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:29.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:29.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.245 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:29.245 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:29.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:29.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:29.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:29.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5799999999999999e-07 2023-11-25T04:23:29.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:29.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034169+0000", 2023-11-25T04:23:29.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1879999999999998e-06 2023-11-25T04:23:29.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:29.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034177+0000", 2023-11-25T04:23:29.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7020000000000001e-06 2023-11-25T04:23:29.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:29.247 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034184+0000", 2023-11-25T04:23:29.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3910000000000003e-06 2023-11-25T04:23:29.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:29.247 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034924+0000", 2023-11-25T04:23:29.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00074032999999999998 2023-11-25T04:23:29.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:29.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034991+0000", 2023-11-25T04:23:29.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7347000000000004e-05 2023-11-25T04:23:29.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:29.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035096+0000", 2023-11-25T04:23:29.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010488 2023-11-25T04:23:29.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:29.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037588+0000", 2023-11-25T04:23:29.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024918309999999999 2023-11-25T04:23:29.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:29.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044367+0000", 2023-11-25T04:23:29.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067787589999999997 2023-11-25T04:23:29.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:29.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044415+0000", 2023-11-25T04:23:29.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7803000000000003e-05 2023-11-25T04:23:29.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.250 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:29.250 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044447+0000", 2023-11-25T04:23:29.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2886999999999999e-05 2023-11-25T04:23:29.250 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:29.250 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:29.250 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:29.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.250 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:30 3.15 3:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:29.250 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:29.251 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.124714312000002, 2023-11-25T04:23:29.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010365070000000001, 2023-11-25T04:23:29.251 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:29.251 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:29.251 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:29.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:29.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:29.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:29.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:29.252 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:29.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:29.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:29.252 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034225+0000", 2023-11-25T04:23:29.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7400000000000002e-07 2023-11-25T04:23:29.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:29.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034229+0000", 2023-11-25T04:23:29.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3100000000000002e-06 2023-11-25T04:23:29.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:29.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034239+0000", 2023-11-25T04:23:29.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9140000000000003e-06 2023-11-25T04:23:29.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:29.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034246+0000", 2023-11-25T04:23:29.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0149999999999998e-06 2023-11-25T04:23:29.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:29.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035572+0000", 2023-11-25T04:23:29.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001325655 2023-11-25T04:23:29.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:29.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035730+0000", 2023-11-25T04:23:29.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015827899999999999 2023-11-25T04:23:29.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:29.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035857+0000", 2023-11-25T04:23:29.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012644699999999999 2023-11-25T04:23:29.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:29.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038737+0000", 2023-11-25T04:23:29.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002880077 2023-11-25T04:23:29.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:29.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044540+0000", 2023-11-25T04:23:29.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058034239999999997 2023-11-25T04:23:29.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:29.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044569+0000", 2023-11-25T04:23:29.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8388999999999999e-05 2023-11-25T04:23:29.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:29.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044589+0000", 2023-11-25T04:23:29.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0686000000000002e-05 2023-11-25T04:23:29.257 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:29.257 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:29.257 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:29.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.257 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:33 3.1b 3:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:29.257 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:29.257 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.124668814000003, 2023-11-25T04:23:29.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099754049999999997, 2023-11-25T04:23:29.257 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:29.257 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:23:29.257 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:29.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:29.258 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:29.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:29.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:29.258 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:29.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:29.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:29.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:29.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4799999999999998e-07 2023-11-25T04:23:29.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:29.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034273+0000", 2023-11-25T04:23:29.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7819999999999998e-06 2023-11-25T04:23:29.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:29.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034285+0000", 2023-11-25T04:23:29.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2058e-05 2023-11-25T04:23:29.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:29.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034291+0000", 2023-11-25T04:23:29.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0680000000000002e-06 2023-11-25T04:23:29.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:29.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034370+0000", 2023-11-25T04:23:29.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8452e-05 2023-11-25T04:23:29.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:29.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034477+0000", 2023-11-25T04:23:29.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107471 2023-11-25T04:23:29.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:29.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044245+0000", 2023-11-25T04:23:29.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0097679259999999997 2023-11-25T04:23:29.261 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:29.261 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:29.261 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:29.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.262 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:36 3.1a 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:29.262 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:29.262 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.124622141000003, 2023-11-25T04:23:29.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058179099999999999, 2023-11-25T04:23:29.262 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:29.262 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:23:29.262 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:29.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:29.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:29.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:29.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:29.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:29.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:29.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:29.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034317+0000", 2023-11-25T04:23:29.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1999999999999999e-07 2023-11-25T04:23:29.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:29.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034321+0000", 2023-11-25T04:23:29.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.233e-06 2023-11-25T04:23:29.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:29.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034337+0000", 2023-11-25T04:23:29.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5308999999999999e-05 2023-11-25T04:23:29.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:29.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034354+0000", 2023-11-25T04:23:29.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7411999999999999e-05 2023-11-25T04:23:29.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:29.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034465+0000", 2023-11-25T04:23:29.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011154699999999999 2023-11-25T04:23:29.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:29.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034560+0000", 2023-11-25T04:23:29.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4186999999999998e-05 2023-11-25T04:23:29.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:29.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.040134+0000", 2023-11-25T04:23:29.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055746019999999997 2023-11-25T04:23:29.266 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:29.266 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:29.266 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:29.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.266 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:9 3.10 e18/16 3:09401a81:::601.00000000:head v 18'4, mlcod=0'0)", 2023-11-25T04:23:29.267 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:29.267 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.115727833000001, 2023-11-25T04:23:29.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061641020000000003, 2023-11-25T04:23:29.267 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:29.267 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:29.267 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:29.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.267 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:29.267 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:29.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:29.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:29.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:29.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:29.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:29.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:29.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5300000000000001e-07 2023-11-25T04:23:29.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:29.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043222+0000", 2023-11-25T04:23:29.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.082e-05 2023-11-25T04:23:29.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:29.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043228+0000", 2023-11-25T04:23:29.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6699999999999997e-06 2023-11-25T04:23:29.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:29.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043236+0000", 2023-11-25T04:23:29.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8189999999999992e-06 2023-11-25T04:23:29.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:29.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043507+0000", 2023-11-25T04:23:29.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000271012 2023-11-25T04:23:29.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:29.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043530+0000", 2023-11-25T04:23:29.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2659000000000001e-05 2023-11-25T04:23:29.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:29.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049321+0000", 2023-11-25T04:23:29.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057915129999999999 2023-11-25T04:23:29.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:29.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049375+0000", 2023-11-25T04:23:29.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3455999999999998e-05 2023-11-25T04:23:29.271 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:29.271 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:29.271 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:29.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.272 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:1 3.14 e18/16 3:292cf221:::200.00000000:head v 18'3, mlcod=0'0)", 2023-11-25T04:23:29.272 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:29.272 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.115691120000001, 2023-11-25T04:23:29.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0060983599999999997, 2023-11-25T04:23:29.272 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:29.272 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:29.272 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:29.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:29.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:29.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:29.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:29.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:29.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:29.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:29.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043248+0000", 2023-11-25T04:23:29.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9399999999999995e-07 2023-11-25T04:23:29.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:29.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043252+0000", 2023-11-25T04:23:29.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1799999999999998e-06 2023-11-25T04:23:29.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:29.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043259+0000", 2023-11-25T04:23:29.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0049999999999998e-06 2023-11-25T04:23:29.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:29.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043265+0000", 2023-11-25T04:23:29.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0859999999999998e-06 2023-11-25T04:23:29.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:29.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043311+0000", 2023-11-25T04:23:29.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6082999999999999e-05 2023-11-25T04:23:29.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:29.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043330+0000", 2023-11-25T04:23:29.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8556e-05 2023-11-25T04:23:29.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:29.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049303+0000", 2023-11-25T04:23:29.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059733470000000004 2023-11-25T04:23:29.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:29.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049346+0000", 2023-11-25T04:23:29.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2608999999999998e-05 2023-11-25T04:23:29.276 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:29.277 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:29.277 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:29.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.277 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:3 3.14 e18/16 3:287034c4:::1.00000000.inode:head v 18'4, mlcod=0'0)", 2023-11-25T04:23:29.277 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:29.277 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.115174613999997, 2023-11-25T04:23:29.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057040090000000003, 2023-11-25T04:23:29.277 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:29.277 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:29.277 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:29.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:29.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:29.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:29.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:29.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:29.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:29.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:29.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:29.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9799999999999999e-07 2023-11-25T04:23:29.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:29.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043785+0000", 2023-11-25T04:23:29.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0721999999999999e-05 2023-11-25T04:23:29.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:29.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043793+0000", 2023-11-25T04:23:29.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5839999999999995e-06 2023-11-25T04:23:29.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:29.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043803+0000", 2023-11-25T04:23:29.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0729999999999994e-06 2023-11-25T04:23:29.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:29.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043843+0000", 2023-11-25T04:23:29.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0747999999999997e-05 2023-11-25T04:23:29.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:29.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043856+0000", 2023-11-25T04:23:29.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2697000000000001e-05 2023-11-25T04:23:29.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:29.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049426+0000", 2023-11-25T04:23:29.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055704409999999998 2023-11-25T04:23:29.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:29.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049468+0000", 2023-11-25T04:23:29.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1446000000000003e-05 2023-11-25T04:23:29.282 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:29.282 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:29.282 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:29.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.282 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "pg_update_log_missing(3.14 epoch 18/16 rep_tid 171 entries 18'5 (0'0) error 3:2e2fa760:::200.00000001:head by mds.0.3:32 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:23:29.282 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:29.282 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.115013650999998, 2023-11-25T04:23:29.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005556938, 2023-11-25T04:23:29.282 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:29.282 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2023-11-25T04:23:29.282 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:29.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:29.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:29.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:29.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:29.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:29.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:29.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:29.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:29.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.03e-07 2023-11-25T04:23:29.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:29.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043931+0000", 2023-11-25T04:23:29.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1079999999999998e-06 2023-11-25T04:23:29.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:29.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043944+0000", 2023-11-25T04:23:29.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2852e-05 2023-11-25T04:23:29.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:29.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043955+0000", 2023-11-25T04:23:29.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1037e-05 2023-11-25T04:23:29.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:29.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044088+0000", 2023-11-25T04:23:29.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013269599999999999 2023-11-25T04:23:29.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:29.286 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049482+0000", 2023-11-25T04:23:29.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053940419999999999 2023-11-25T04:23:29.286 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:29.286 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:29.286 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:29.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.286 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:10 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'5, mlcod=0'0)", 2023-11-25T04:23:29.286 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:29.287 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.114923623000003, 2023-11-25T04:23:29.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054493099999999997, 2023-11-25T04:23:29.287 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:29.287 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:29.287 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:29.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:29.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:29.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:29.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:29.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:29.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:29.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:29.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:29.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6100000000000002e-07 2023-11-25T04:23:29.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:29.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044019+0000", 2023-11-25T04:23:29.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.839e-06 2023-11-25T04:23:29.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:29.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044026+0000", 2023-11-25T04:23:29.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5869999999999998e-06 2023-11-25T04:23:29.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:29.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044031+0000", 2023-11-25T04:23:29.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4909999999999996e-06 2023-11-25T04:23:29.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.290 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:29.290 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044070+0000", 2023-11-25T04:23:29.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8829999999999999e-05 2023-11-25T04:23:29.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.290 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:29.290 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044081+0000", 2023-11-25T04:23:29.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1447999999999999e-05 2023-11-25T04:23:29.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.291 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:29.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049406+0000", 2023-11-25T04:23:29.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053246480000000004 2023-11-25T04:23:29.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.291 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:29.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049464+0000", 2023-11-25T04:23:29.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8205999999999998e-05 2023-11-25T04:23:29.291 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:29.291 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:29.291 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:29.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.292 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:11 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'6, mlcod=0'0)", 2023-11-25T04:23:29.292 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:29.292 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 42.114722358999998, 2023-11-25T04:23:29.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052673499999999996, 2023-11-25T04:23:29.292 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:29.292 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:29.292 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:29.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:29.293 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:29.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:29.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.293 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:29.293 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:29.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:29.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.293 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:29.293 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:29.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0199999999999998e-07 2023-11-25T04:23:29.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:29.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044220+0000", 2023-11-25T04:23:29.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7720000000000001e-06 2023-11-25T04:23:29.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:29.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044226+0000", 2023-11-25T04:23:29.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8579999999999996e-06 2023-11-25T04:23:29.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:29.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044232+0000", 2023-11-25T04:23:29.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1700000000000002e-06 2023-11-25T04:23:29.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:29.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044274+0000", 2023-11-25T04:23:29.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1254999999999997e-05 2023-11-25T04:23:29.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:29.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:23:29.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6880999999999999e-05 2023-11-25T04:23:29.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.296 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:29.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049466+0000", 2023-11-25T04:23:29.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051659219999999999 2023-11-25T04:23:29.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:29.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:29.296 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:29.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049483+0000", 2023-11-25T04:23:29.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.719e-05 2023-11-25T04:23:29.296 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:29.296 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:29.297 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:29.298 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:29.298 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:29.298 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:29.299 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:23:29.300 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:29.297+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:29.400 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:29.398+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:29.408 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:29.408 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:23:29.408 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2023-11-25T04:23:29.408 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:29.417 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:23:29.501 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:29.499+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:29.523 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:29.523 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:23:29.523 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2023-11-25T04:23:29.523 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2023-11-25T04:23:29.523 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:29.532 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:23:29.602 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:29.600+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:29.639 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:29.639 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2023-11-25T04:23:29.639 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2023-11-25T04:23:29.639 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2023-11-25T04:23:29.639 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.640 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:29.640 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:29.640 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 42.605075333000002, 2023-11-25T04:23:29.640 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01072943, 2023-11-25T04:23:29.640 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:29.640 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:29.640 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:29.640 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.640 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:29.640 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:29.640 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:29.641 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.641 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.641 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:29.641 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:29.641 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:29.641 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.641 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.641 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:29.641 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031829+0000", 2023-11-25T04:23:29.641 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.666e-06 2023-11-25T04:23:29.641 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.642 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.642 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:29.642 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031839+0000", 2023-11-25T04:23:29.642 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9490000000000008e-06 2023-11-25T04:23:29.642 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.642 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.642 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:29.642 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031857+0000", 2023-11-25T04:23:29.642 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8093000000000001e-05 2023-11-25T04:23:29.642 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.642 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.642 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:29.642 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031882+0000", 2023-11-25T04:23:29.643 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4294000000000002e-05 2023-11-25T04:23:29.643 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.643 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.643 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:29.643 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033130+0000", 2023-11-25T04:23:29.643 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001248264 2023-11-25T04:23:29.643 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.644 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.644 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:29.644 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033426+0000", 2023-11-25T04:23:29.644 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029620999999999998 2023-11-25T04:23:29.644 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.644 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.644 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:29.644 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033814+0000", 2023-11-25T04:23:29.644 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000387742 2023-11-25T04:23:29.644 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.644 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.644 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:29.644 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036793+0000", 2023-11-25T04:23:29.645 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002979598 2023-11-25T04:23:29.645 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.645 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.645 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:29.645 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042455+0000", 2023-11-25T04:23:29.645 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0056619260000000003 2023-11-25T04:23:29.645 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.645 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.645 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:29.645 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042502+0000", 2023-11-25T04:23:29.645 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7139999999999999e-05 2023-11-25T04:23:29.645 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.645 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.646 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:29.646 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042557+0000", 2023-11-25T04:23:29.646 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4548e-05 2023-11-25T04:23:29.646 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:29.646 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:29.646 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:29.646 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.646 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.646 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:29.646 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:29.646 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 42.603876798000002, 2023-11-25T04:23:29.646 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011505202000000001, 2023-11-25T04:23:29.646 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:29.647 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:29.647 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:29.647 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.647 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:29.647 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:29.647 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:29.647 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.647 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.647 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:29.647 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:29.647 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:29.647 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.647 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.647 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:29.648 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033027+0000", 2023-11-25T04:23:29.648 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0699999999999999e-06 2023-11-25T04:23:29.648 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.648 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.648 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:29.648 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033035+0000", 2023-11-25T04:23:29.648 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0830000000000004e-06 2023-11-25T04:23:29.648 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.648 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.648 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:29.648 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033059+0000", 2023-11-25T04:23:29.648 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3975e-05 2023-11-25T04:23:29.648 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.649 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.649 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:29.649 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033482+0000", 2023-11-25T04:23:29.649 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042253700000000001 2023-11-25T04:23:29.649 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.649 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.649 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:29.649 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034520+0000", 2023-11-25T04:23:29.649 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001037991 2023-11-25T04:23:29.649 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.649 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.649 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:29.649 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034611+0000", 2023-11-25T04:23:29.649 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1113999999999995e-05 2023-11-25T04:23:29.650 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.650 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.650 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:29.650 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034769+0000", 2023-11-25T04:23:29.650 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000157641 2023-11-25T04:23:29.650 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.650 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.650 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:29.650 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038292+0000", 2023-11-25T04:23:29.650 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035231160000000002 2023-11-25T04:23:29.650 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.650 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.650 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:29.650 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044440+0000", 2023-11-25T04:23:29.651 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0061486980000000002 2023-11-25T04:23:29.651 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.651 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.651 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:29.651 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044495+0000", 2023-11-25T04:23:29.651 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4339000000000003e-05 2023-11-25T04:23:29.651 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.651 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.651 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:29.652 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044531+0000", 2023-11-25T04:23:29.652 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6637999999999997e-05 2023-11-25T04:23:29.652 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:29.652 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:29.652 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:29.652 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.652 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.652 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:29.652 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:29.652 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 42.603402443999997, 2023-11-25T04:23:29.652 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0057634039999999997, 2023-11-25T04:23:29.652 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:29.653 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:29.653 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:29.653 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.653 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:29.653 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:29.653 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:29.653 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.653 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.653 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:29.653 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:29.653 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:29.653 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.654 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.654 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:29.654 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:29.654 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7900000000000007e-07 2023-11-25T04:23:29.654 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.654 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.654 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:29.654 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033507+0000", 2023-11-25T04:23:29.654 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8900000000000004e-06 2023-11-25T04:23:29.654 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.654 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.655 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:29.655 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033519+0000", 2023-11-25T04:23:29.655 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1756000000000001e-05 2023-11-25T04:23:29.655 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.655 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.655 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:29.655 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033527+0000", 2023-11-25T04:23:29.655 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3610000000000001e-06 2023-11-25T04:23:29.655 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.655 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.655 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:29.655 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034451+0000", 2023-11-25T04:23:29.655 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00092372400000000001 2023-11-25T04:23:29.655 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.656 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.656 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:29.656 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034809+0000", 2023-11-25T04:23:29.656 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00035757699999999998 2023-11-25T04:23:29.656 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.656 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.656 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:23:29.656 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034965+0000", 2023-11-25T04:23:29.656 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015638699999999999 2023-11-25T04:23:29.656 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.656 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.656 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:29.656 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038299+0000", 2023-11-25T04:23:29.657 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0033336559999999999 2023-11-25T04:23:29.657 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.657 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.657 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:29.657 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039169+0000", 2023-11-25T04:23:29.657 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00087066400000000005 2023-11-25T04:23:29.657 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.657 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.657 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:29.657 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039226+0000", 2023-11-25T04:23:29.657 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6511999999999997e-05 2023-11-25T04:23:29.657 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.657 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.657 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:29.658 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039264+0000", 2023-11-25T04:23:29.658 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7898000000000002e-05 2023-11-25T04:23:29.658 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:29.658 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:29.658 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:29.658 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.658 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.658 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:25 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:29.658 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:29.658 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 42.603323860000003, 2023-11-25T04:23:29.658 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096656169999999996, 2023-11-25T04:23:29.658 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:29.658 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:29.659 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:29.659 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.659 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:29.659 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:29.659 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:29.659 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.659 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.659 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:29.659 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:29.659 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:29.659 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.659 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.660 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:29.660 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033580+0000", 2023-11-25T04:23:29.660 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8199999999999999e-07 2023-11-25T04:23:29.660 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.660 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.660 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:29.660 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033585+0000", 2023-11-25T04:23:29.660 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3680000000000001e-06 2023-11-25T04:23:29.660 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.660 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.660 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:29.660 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033600+0000", 2023-11-25T04:23:29.661 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4331000000000001e-05 2023-11-25T04:23:29.661 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.661 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.661 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:29.661 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033606+0000", 2023-11-25T04:23:29.661 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5780000000000004e-06 2023-11-25T04:23:29.661 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.661 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.661 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:29.661 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035747+0000", 2023-11-25T04:23:29.661 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0021406530000000002 2023-11-25T04:23:29.661 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.662 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.662 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:29.662 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035901+0000", 2023-11-25T04:23:29.662 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000153931 2023-11-25T04:23:29.662 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.662 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.662 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:23:29.662 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036193+0000", 2023-11-25T04:23:29.662 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029211299999999999 2023-11-25T04:23:29.662 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.662 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.662 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:29.663 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038655+0000", 2023-11-25T04:23:29.663 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0024621640000000002 2023-11-25T04:23:29.663 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.663 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.663 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:29.663 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043130+0000", 2023-11-25T04:23:29.663 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.004474698 2023-11-25T04:23:29.663 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.663 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.663 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:29.663 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043204+0000", 2023-11-25T04:23:29.663 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4091999999999995e-05 2023-11-25T04:23:29.663 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.664 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.664 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:29.664 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043245+0000", 2023-11-25T04:23:29.664 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0907e-05 2023-11-25T04:23:29.664 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:29.664 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:29.664 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:29.664 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.664 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.664 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:26 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:29.664 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:29.664 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 42.603288036000002, 2023-11-25T04:23:29.664 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011086945000000001, 2023-11-25T04:23:29.664 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:29.665 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:29.665 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:29.665 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.665 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:29.665 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:29.665 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:29.665 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.665 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.665 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:29.665 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:29.665 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:29.665 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.665 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.665 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:29.666 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:29.666 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1699999999999998e-07 2023-11-25T04:23:29.666 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.666 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.666 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:29.666 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033627+0000", 2023-11-25T04:23:29.666 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.163e-05 2023-11-25T04:23:29.666 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.666 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.666 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:29.666 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033636+0000", 2023-11-25T04:23:29.666 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6859999999999995e-06 2023-11-25T04:23:29.666 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.666 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.667 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:29.667 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033642+0000", 2023-11-25T04:23:29.667 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3149999999999997e-06 2023-11-25T04:23:29.667 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.667 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.667 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:29.667 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035414+0000", 2023-11-25T04:23:29.667 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0017716380000000001 2023-11-25T04:23:29.667 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.667 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.667 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:29.667 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035539+0000", 2023-11-25T04:23:29.667 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000125477 2023-11-25T04:23:29.667 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.668 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.668 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:29.668 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035672+0000", 2023-11-25T04:23:29.668 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000133316 2023-11-25T04:23:29.668 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.668 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.668 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:29.668 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038649+0000", 2023-11-25T04:23:29.668 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029764679999999999 2023-11-25T04:23:29.668 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.668 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.668 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:29.668 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044578+0000", 2023-11-25T04:23:29.668 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0059293510000000002 2023-11-25T04:23:29.669 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.669 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.669 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:29.669 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044651+0000", 2023-11-25T04:23:29.669 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2513000000000002e-05 2023-11-25T04:23:29.669 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.669 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.669 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:29.669 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044702+0000", 2023-11-25T04:23:29.670 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1034e-05 2023-11-25T04:23:29.670 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:29.670 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:29.670 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:29.670 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.670 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.670 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:29 3.b 3:d0630e4c:::mds0_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:29.670 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:29.670 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 42.603250115999998, 2023-11-25T04:23:29.670 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010980227, 2023-11-25T04:23:29.670 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:29.670 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:29.671 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:29.671 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.671 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:29.671 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:29.671 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:29.671 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.671 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.671 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:29.671 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:29.671 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:29.671 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.671 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.671 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:29.671 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:29.672 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3499999999999996e-07 2023-11-25T04:23:29.672 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.672 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.672 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:29.672 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033658+0000", 2023-11-25T04:23:29.672 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1280000000000001e-06 2023-11-25T04:23:29.672 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.672 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.672 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:29.672 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033667+0000", 2023-11-25T04:23:29.672 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1430000000000004e-06 2023-11-25T04:23:29.672 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.672 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.673 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:29.673 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033678+0000", 2023-11-25T04:23:29.673 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0906e-05 2023-11-25T04:23:29.673 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.673 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.673 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:29.673 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034456+0000", 2023-11-25T04:23:29.673 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00077833899999999996 2023-11-25T04:23:29.673 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.673 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.673 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:29.674 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034575+0000", 2023-11-25T04:23:29.674 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011926000000000001 2023-11-25T04:23:29.674 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.674 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.674 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:29.674 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035001+0000", 2023-11-25T04:23:29.674 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042550999999999998 2023-11-25T04:23:29.674 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.674 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.674 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:29.674 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.037923+0000", 2023-11-25T04:23:29.674 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029219670000000001 2023-11-25T04:23:29.674 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.675 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.675 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:29.675 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044538+0000", 2023-11-25T04:23:29.675 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0066152349999999997 2023-11-25T04:23:29.675 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.675 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.675 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:29.675 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044594+0000", 2023-11-25T04:23:29.675 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5593000000000001e-05 2023-11-25T04:23:29.675 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.675 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.675 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:29.675 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044633+0000", 2023-11-25T04:23:29.676 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9511000000000002e-05 2023-11-25T04:23:29.676 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:29.676 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:29.676 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:29.676 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.676 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.676 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:34 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:29.676 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:29.676 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 42.603000002999998, 2023-11-25T04:23:29.676 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010920445000000001, 2023-11-25T04:23:29.676 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:29.676 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:29.676 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:29.677 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.677 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:29.677 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:29.677 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:29.677 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.677 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.677 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:29.677 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:29.677 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:29.677 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.677 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.677 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:29.677 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033904+0000", 2023-11-25T04:23:29.677 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6599999999999994e-07 2023-11-25T04:23:29.678 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.678 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.678 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:29.678 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033909+0000", 2023-11-25T04:23:29.678 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5639999999999996e-06 2023-11-25T04:23:29.678 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.678 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.678 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:29.678 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033923+0000", 2023-11-25T04:23:29.678 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3028e-05 2023-11-25T04:23:29.678 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.679 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.679 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:29.679 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034043+0000", 2023-11-25T04:23:29.679 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000120328 2023-11-25T04:23:29.679 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.679 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.679 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:29.679 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035576+0000", 2023-11-25T04:23:29.679 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015333720000000001 2023-11-25T04:23:29.679 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.679 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.679 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:29.679 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035699+0000", 2023-11-25T04:23:29.680 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000123001 2023-11-25T04:23:29.680 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.680 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.680 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:29.680 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044823+0000", 2023-11-25T04:23:29.680 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0091241859999999994 2023-11-25T04:23:29.680 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:29.681 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:29.681 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:29.681 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.681 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.681 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:35 3.1f 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:29.681 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:29.681 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 42.601695798000002, 2023-11-25T04:23:29.681 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096164779999999995, 2023-11-25T04:23:29.681 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:29.681 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:29.681 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:29.681 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.681 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:29.681 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:29.682 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:29.682 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.682 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.682 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:29.682 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:29.682 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:29.682 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.682 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.682 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:29.682 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035208+0000", 2023-11-25T04:23:29.683 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2699999999999998e-07 2023-11-25T04:23:29.683 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.683 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.683 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:29.683 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035213+0000", 2023-11-25T04:23:29.683 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.6070000000000001e-06 2023-11-25T04:23:29.683 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.683 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.683 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:29.683 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035350+0000", 2023-11-25T04:23:29.683 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013693599999999999 2023-11-25T04:23:29.683 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.683 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.683 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:29.684 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035358+0000", 2023-11-25T04:23:29.684 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0099999999999995e-06 2023-11-25T04:23:29.684 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.684 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.684 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:29.684 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036260+0000", 2023-11-25T04:23:29.684 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00090240100000000005 2023-11-25T04:23:29.684 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.684 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.684 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:29.684 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036517+0000", 2023-11-25T04:23:29.684 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00025736299999999998 2023-11-25T04:23:29.684 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.684 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.685 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:29.685 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044824+0000", 2023-11-25T04:23:29.685 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0083063340000000003 2023-11-25T04:23:29.685 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:29.685 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:29.685 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:29.685 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.685 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.685 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:37 3.1e 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:29.685 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:29.685 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 42.601529112000001, 2023-11-25T04:23:29.685 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0078322640000000002, 2023-11-25T04:23:29.685 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:29.686 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:29.686 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:29.686 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.686 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:29.686 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:29.686 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:29.686 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.686 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.686 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:29.686 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:29.686 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:29.686 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.687 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.687 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:29.687 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035375+0000", 2023-11-25T04:23:29.687 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9100000000000003e-07 2023-11-25T04:23:29.687 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.687 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.687 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:29.687 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035379+0000", 2023-11-25T04:23:29.687 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9430000000000002e-06 2023-11-25T04:23:29.687 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.687 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.687 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:29.687 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035388+0000", 2023-11-25T04:23:29.687 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2760000000000006e-06 2023-11-25T04:23:29.688 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.688 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.688 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:29.688 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035396+0000", 2023-11-25T04:23:29.688 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8480000000000001e-06 2023-11-25T04:23:29.688 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.688 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.688 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:29.688 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035452+0000", 2023-11-25T04:23:29.688 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5831000000000001e-05 2023-11-25T04:23:29.688 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.688 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.688 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:29.689 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035934+0000", 2023-11-25T04:23:29.689 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00048265599999999998 2023-11-25T04:23:29.689 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.689 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.689 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:29.689 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043206+0000", 2023-11-25T04:23:29.689 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.007271919 2023-11-25T04:23:29.689 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:29.689 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:29.689 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:29.689 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.689 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.689 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing(3.1a epoch 18/16 rep_tid 3 entries 18'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:36 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:23:29.690 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:29.690 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 42.600253526000003, 2023-11-25T04:23:29.690 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002073414, 2023-11-25T04:23:29.690 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:29.690 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:29.691 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:29.691 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.691 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:29.691 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:29.691 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:29.691 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.691 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.691 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:29.691 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:29.691 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:29.691 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.691 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.691 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:29.691 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036650+0000", 2023-11-25T04:23:29.692 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7300000000000002e-07 2023-11-25T04:23:29.692 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.692 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.692 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:29.692 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036659+0000", 2023-11-25T04:23:29.692 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.7499999999999992e-06 2023-11-25T04:23:29.692 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.692 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.692 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:29.692 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036673+0000", 2023-11-25T04:23:29.692 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4917e-05 2023-11-25T04:23:29.692 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.692 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.693 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:29.693 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036687+0000", 2023-11-25T04:23:29.693 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3883000000000001e-05 2023-11-25T04:23:29.693 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.693 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.693 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:29.693 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036781+0000", 2023-11-25T04:23:29.693 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3219000000000003e-05 2023-11-25T04:23:29.693 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.693 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.693 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:29.693 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038723+0000", 2023-11-25T04:23:29.693 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019422720000000001 2023-11-25T04:23:29.693 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:29.694 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:29.694 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:29.694 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.694 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.694 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:28 3.13 e18/16 3:cd5a64a3:::100.00000000:head v 18'1, mlcod=0'0)", 2023-11-25T04:23:29.694 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:29.694 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 42.596592368000003, 2023-11-25T04:23:29.694 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001850546, 2023-11-25T04:23:29.694 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:29.694 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:29.694 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:29.694 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.695 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:29.695 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:29.695 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:29.695 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.695 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.695 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:29.695 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:29.695 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:29.695 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.695 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.695 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:29.695 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:29.695 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7800000000000002e-07 2023-11-25T04:23:29.695 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.696 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.696 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:29.696 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040331+0000", 2023-11-25T04:23:29.696 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0449999999999999e-05 2023-11-25T04:23:29.696 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.696 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.696 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:29.696 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040339+0000", 2023-11-25T04:23:29.696 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0709999999999995e-06 2023-11-25T04:23:29.696 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.696 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.696 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:29.696 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040347+0000", 2023-11-25T04:23:29.696 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7079999999999997e-06 2023-11-25T04:23:29.697 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.697 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.697 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:29.697 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040410+0000", 2023-11-25T04:23:29.697 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2453000000000007e-05 2023-11-25T04:23:29.697 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.697 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.697 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:29.697 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040422+0000", 2023-11-25T04:23:29.697 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2429000000000001e-05 2023-11-25T04:23:29.698 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.698 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.698 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:29.698 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042129+0000", 2023-11-25T04:23:29.698 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001707266 2023-11-25T04:23:29.698 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.698 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.698 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:29.698 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042161+0000", 2023-11-25T04:23:29.699 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1791000000000001e-05 2023-11-25T04:23:29.699 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:29.699 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:29.699 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:29.699 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.699 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.699 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:27 3.13 e18/16 3:c91d4a1d:::mds0_inotable:head v 18'2, mlcod=0'0)", 2023-11-25T04:23:29.699 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:29.699 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 42.596388845, 2023-11-25T04:23:29.699 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0016886430000000001, 2023-11-25T04:23:29.699 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:29.699 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:29.699 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:29.699 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.700 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:29.700 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:29.700 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:29.700 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.700 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.700 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:29.700 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:29.700 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:29.700 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.700 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.700 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:29.700 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:29.700 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3999999999999997e-07 2023-11-25T04:23:29.700 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.701 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.701 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:29.701 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040520+0000", 2023-11-25T04:23:29.701 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2989999999999996e-06 2023-11-25T04:23:29.701 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.701 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.701 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:29.701 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040527+0000", 2023-11-25T04:23:29.701 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5730000000000002e-06 2023-11-25T04:23:29.701 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.701 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.701 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:29.701 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040535+0000", 2023-11-25T04:23:29.701 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8630000000000008e-06 2023-11-25T04:23:29.702 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.702 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.702 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:29.702 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040683+0000", 2023-11-25T04:23:29.702 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000148002 2023-11-25T04:23:29.702 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.702 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.702 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:29.702 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040700+0000", 2023-11-25T04:23:29.702 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7065e-05 2023-11-25T04:23:29.702 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.702 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.703 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:29.703 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042163+0000", 2023-11-25T04:23:29.703 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001462767 2023-11-25T04:23:29.703 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.703 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.703 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:29.703 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042203+0000", 2023-11-25T04:23:29.703 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9734000000000003e-05 2023-11-25T04:23:29.703 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:29.704 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:29.704 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:29.704 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.704 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.704 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:20 3.1c e18/16 3:3ed09add:::607.00000000:head v 18'1, mlcod=0'0)", 2023-11-25T04:23:29.704 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:29.704 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 42.596353804000003, 2023-11-25T04:23:29.704 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015271, 2023-11-25T04:23:29.704 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:29.704 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:29.704 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:29.704 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.704 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:29.705 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:29.705 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:29.705 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.705 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.705 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:29.705 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:29.705 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:29.705 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.705 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.705 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:29.705 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:29.705 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4799999999999999e-07 2023-11-25T04:23:29.705 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.706 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.706 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:29.706 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040561+0000", 2023-11-25T04:23:29.706 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1800000000000001e-05 2023-11-25T04:23:29.706 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.706 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.706 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:29.706 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040568+0000", 2023-11-25T04:23:29.706 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7730000000000002e-06 2023-11-25T04:23:29.707 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.707 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.707 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:29.707 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040587+0000", 2023-11-25T04:23:29.707 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8691000000000001e-05 2023-11-25T04:23:29.707 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.707 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.707 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:29.707 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040638+0000", 2023-11-25T04:23:29.707 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1068e-05 2023-11-25T04:23:29.707 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.707 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.707 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:29.708 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040670+0000", 2023-11-25T04:23:29.708 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2125999999999997e-05 2023-11-25T04:23:29.708 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.708 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.708 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:29.708 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042032+0000", 2023-11-25T04:23:29.708 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001361821 2023-11-25T04:23:29.708 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.708 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.708 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:29.708 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042076+0000", 2023-11-25T04:23:29.708 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4472999999999999e-05 2023-11-25T04:23:29.708 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:29.709 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:29.709 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:29.709 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.709 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.709 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:21 3.1c e18/16 3:3ed09add:::607.00000000:head v 18'2, mlcod=0'0)", 2023-11-25T04:23:29.709 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:29.709 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 42.596119078999997, 2023-11-25T04:23:29.709 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0013228160000000001, 2023-11-25T04:23:29.709 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:29.709 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:29.709 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:29.709 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.709 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:29.710 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:29.710 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:29.710 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.710 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.710 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:29.710 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:29.710 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:29.710 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.710 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.710 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:29.710 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:29.710 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9299999999999999e-07 2023-11-25T04:23:29.710 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.711 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.711 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:29.711 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040790+0000", 2023-11-25T04:23:29.711 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5770000000000001e-06 2023-11-25T04:23:29.711 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.711 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.711 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:29.711 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040799+0000", 2023-11-25T04:23:29.711 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6439999999999995e-06 2023-11-25T04:23:29.711 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.711 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.712 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:29.712 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040806+0000", 2023-11-25T04:23:29.712 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0180000000000004e-06 2023-11-25T04:23:29.712 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.712 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.712 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:29.712 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040907+0000", 2023-11-25T04:23:29.712 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000100096 2023-11-25T04:23:29.712 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.712 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.712 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:29.712 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040925+0000", 2023-11-25T04:23:29.713 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8536999999999999e-05 2023-11-25T04:23:29.713 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.713 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.713 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:29.713 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042079+0000", 2023-11-25T04:23:29.713 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00115358 2023-11-25T04:23:29.713 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.713 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.713 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:29.714 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042107+0000", 2023-11-25T04:23:29.714 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7971e-05 2023-11-25T04:23:29.714 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:29.714 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:29.714 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:29.714 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.714 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.714 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop_reply(mds.0.3:4 3.1f e18/16 ondisk, result = 0)", 2023-11-25T04:23:29.714 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:29.714 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 42.594779688999999, 2023-11-25T04:23:29.714 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00044092900000000001, 2023-11-25T04:23:29.714 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:29.714 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:29.715 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:29.715 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.715 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:29.715 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:29.715 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:29.715 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.715 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.715 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:29.715 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:29.715 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:29.715 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.715 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.715 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:29.716 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042124+0000", 2023-11-25T04:23:29.716 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1800000000000005e-07 2023-11-25T04:23:29.716 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.716 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.716 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:29.716 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042128+0000", 2023-11-25T04:23:29.716 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2289999999999997e-06 2023-11-25T04:23:29.716 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.716 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.716 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:29.716 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042138+0000", 2023-11-25T04:23:29.716 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9729999999999992e-06 2023-11-25T04:23:29.716 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.717 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.717 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:29.717 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042145+0000", 2023-11-25T04:23:29.717 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.336e-06 2023-11-25T04:23:29.717 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.717 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.717 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:29.717 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042441+0000", 2023-11-25T04:23:29.717 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029650500000000002 2023-11-25T04:23:29.717 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.718 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.718 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:29.718 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042448+0000", 2023-11-25T04:23:29.718 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4100000000000002e-06 2023-11-25T04:23:29.718 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.718 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.718 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:29.718 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042564+0000", 2023-11-25T04:23:29.718 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000115758 2023-11-25T04:23:29.718 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:29.718 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:29.718 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:29.719 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.719 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.719 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing_reply(3.1f epoch 18/16 rep_tid 8 lcod 18'4)", 2023-11-25T04:23:29.719 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:29.719 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 42.592454277000002, 2023-11-25T04:23:29.719 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00038154699999999998, 2023-11-25T04:23:29.719 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:29.719 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:29.719 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:29.719 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.719 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:29.719 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:29.719 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:29.720 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.720 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.720 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:29.720 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:29.720 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:29.720 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.720 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.720 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:29.720 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:29.721 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2700000000000001e-07 2023-11-25T04:23:29.721 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.721 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.721 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:29.721 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044451+0000", 2023-11-25T04:23:29.721 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7460000000000001e-06 2023-11-25T04:23:29.721 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.721 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.722 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:29.722 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044455+0000", 2023-11-25T04:23:29.722 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5809999999999998e-06 2023-11-25T04:23:29.722 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.722 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.722 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:29.722 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044458+0000", 2023-11-25T04:23:29.722 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0170000000000001e-06 2023-11-25T04:23:29.722 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.722 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.722 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:29.722 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044734+0000", 2023-11-25T04:23:29.723 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00027582799999999999 2023-11-25T04:23:29.723 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.723 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.723 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:29.723 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044830+0000", 2023-11-25T04:23:29.723 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6248000000000004e-05 2023-11-25T04:23:29.723 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:29.723 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:29.723 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:29.723 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.723 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.723 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing_reply(3.b epoch 18/16 rep_tid 5 lcod 18'2)", 2023-11-25T04:23:29.723 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:29.724 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 42.592417615999999, 2023-11-25T04:23:29.724 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000344488, 2023-11-25T04:23:29.724 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:29.724 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:29.724 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:29.724 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.724 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:29.724 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:29.724 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:29.724 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.724 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.724 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:29.725 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:29.725 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:29.725 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.725 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.725 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:29.725 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:29.725 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.55e-07 2023-11-25T04:23:29.725 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.725 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.725 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:29.725 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044487+0000", 2023-11-25T04:23:29.725 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2470000000000001e-06 2023-11-25T04:23:29.725 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.726 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.726 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:29.726 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044491+0000", 2023-11-25T04:23:29.726 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9659999999999998e-06 2023-11-25T04:23:29.726 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.726 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.726 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:29.726 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044494+0000", 2023-11-25T04:23:29.726 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2310000000000001e-06 2023-11-25T04:23:29.726 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.726 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.726 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:29.727 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044756+0000", 2023-11-25T04:23:29.727 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00026172800000000003 2023-11-25T04:23:29.727 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.727 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.727 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:29.727 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044830+0000", 2023-11-25T04:23:29.727 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4160999999999997e-05 2023-11-25T04:23:29.727 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:29.727 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:29.727 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:29.727 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.727 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.727 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:39 3.6 3:654134d2:::mds0_openfiles.0:head [omap-clear,omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2023-11-25T04:23:29.728 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:29.728 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.532591574000001, 2023-11-25T04:23:29.728 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001788658, 2023-11-25T04:23:29.728 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:29.728 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:29.728 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:29.728 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.728 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:29.728 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:29.728 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:29.728 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.728 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.728 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:29.729 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:29.729 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:29.729 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.729 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.729 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:29.729 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104314+0000", 2023-11-25T04:23:29.729 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3099999999999999e-06 2023-11-25T04:23:29.729 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.729 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.729 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:29.729 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104325+0000", 2023-11-25T04:23:29.729 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1511999999999999e-05 2023-11-25T04:23:29.730 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.730 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.730 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:29.730 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104362+0000", 2023-11-25T04:23:29.730 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6903000000000002e-05 2023-11-25T04:23:29.730 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.730 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.730 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:29.730 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104387+0000", 2023-11-25T04:23:29.731 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4766e-05 2023-11-25T04:23:29.731 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.731 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.731 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:29.731 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104465+0000", 2023-11-25T04:23:29.731 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8119000000000002e-05 2023-11-25T04:23:29.731 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.731 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.731 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:29.731 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104557+0000", 2023-11-25T04:23:29.731 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2496000000000003e-05 2023-11-25T04:23:29.731 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.732 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.732 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:29.732 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104646+0000", 2023-11-25T04:23:29.732 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8818000000000002e-05 2023-11-25T04:23:29.732 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.732 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.732 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:29.732 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105201+0000", 2023-11-25T04:23:29.732 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00055506000000000002 2023-11-25T04:23:29.732 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.732 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.732 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:29.732 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105994+0000", 2023-11-25T04:23:29.733 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00079294700000000001 2023-11-25T04:23:29.733 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.733 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.733 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:29.733 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.106055+0000", 2023-11-25T04:23:29.733 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0566000000000003e-05 2023-11-25T04:23:29.733 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.733 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.733 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:29.733 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.106100+0000", 2023-11-25T04:23:29.733 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5160999999999997e-05 2023-11-25T04:23:29.733 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:29.733 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:29.734 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:29.734 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.734 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.734 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:43 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2023-11-25T04:23:29.734 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:29.734 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 22.532366578000001, 2023-11-25T04:23:29.734 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001751166, 2023-11-25T04:23:29.734 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:29.734 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:29.734 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:29.734 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.734 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:29.735 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:29.735 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:29.735 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.735 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.735 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:29.735 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:29.735 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:29.735 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.735 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.735 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:29.736 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104539+0000", 2023-11-25T04:23:29.736 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9529999999999999e-06 2023-11-25T04:23:29.736 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.736 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.736 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:29.736 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104557+0000", 2023-11-25T04:23:29.736 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7464000000000001e-05 2023-11-25T04:23:29.736 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.736 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.736 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:29.736 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104590+0000", 2023-11-25T04:23:29.736 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2966000000000003e-05 2023-11-25T04:23:29.736 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.737 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.737 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:29.737 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104608+0000", 2023-11-25T04:23:29.737 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8193e-05 2023-11-25T04:23:29.737 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.737 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.737 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:29.737 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104767+0000", 2023-11-25T04:23:29.737 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015882 2023-11-25T04:23:29.737 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.737 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.737 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:29.737 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104909+0000", 2023-11-25T04:23:29.738 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014244999999999999 2023-11-25T04:23:29.738 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.738 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.738 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:29.738 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.105059+0000", 2023-11-25T04:23:29.738 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014964399999999999 2023-11-25T04:23:29.738 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.738 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.738 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:29.738 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.105701+0000", 2023-11-25T04:23:29.739 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00064201099999999995 2023-11-25T04:23:29.739 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.739 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.739 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:29.739 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106233+0000", 2023-11-25T04:23:29.739 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00053178500000000001 2023-11-25T04:23:29.739 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.739 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.739 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:29.739 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106265+0000", 2023-11-25T04:23:29.739 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1986000000000002e-05 2023-11-25T04:23:29.739 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.739 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.740 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:29.740 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106288+0000", 2023-11-25T04:23:29.740 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2894000000000001e-05 2023-11-25T04:23:29.740 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:29.740 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:29.740 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:29.740 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.740 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.740 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:45 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e23)", 2023-11-25T04:23:29.740 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:23:29.740 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 17.532168538000001, 2023-11-25T04:23:29.740 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00098423299999999994, 2023-11-25T04:23:29.741 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:29.741 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:29.741 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:29.741 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.741 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:29.741 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:23:29.741 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:29.741 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.741 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.741 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:29.741 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:23:29.742 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:29.742 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.742 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.742 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:29.742 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104737+0000", 2023-11-25T04:23:29.742 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3309999999999998e-06 2023-11-25T04:23:29.742 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.742 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.742 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:29.742 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104748+0000", 2023-11-25T04:23:29.742 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1080999999999999e-05 2023-11-25T04:23:29.742 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.743 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.743 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:29.743 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104771+0000", 2023-11-25T04:23:29.743 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2707000000000001e-05 2023-11-25T04:23:29.743 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.743 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.743 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:29.743 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104800+0000", 2023-11-25T04:23:29.744 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9595e-05 2023-11-25T04:23:29.744 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.744 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.744 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:29.744 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104890+0000", 2023-11-25T04:23:29.744 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.9808999999999993e-05 2023-11-25T04:23:29.745 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.745 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.745 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:29.745 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105004+0000", 2023-11-25T04:23:29.745 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000114508 2023-11-25T04:23:29.745 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.745 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.745 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:29.745 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105634+0000", 2023-11-25T04:23:29.745 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00062985799999999996 2023-11-25T04:23:29.745 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.745 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.746 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:29.746 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105671+0000", 2023-11-25T04:23:29.746 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6640000000000002e-05 2023-11-25T04:23:29.746 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:29.746 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:29.746 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:29.746 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105719+0000", 2023-11-25T04:23:29.746 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7704000000000003e-05 2023-11-25T04:23:29.746 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:29.746 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:29.746 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:29.746 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:29.747 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:29.747 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:29.748 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:23:29.749 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:29.746+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:29.849 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:29.848+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:29.862 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:29.862 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:23:29.862 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2023-11-25T04:23:29.862 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:29.871 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:23:29.950 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:29.949+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:29.974 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:29.975 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:23:29.975 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2023-11-25T04:23:29.975 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2023-11-25T04:23:29.975 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:29.983 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:23:30.051 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:30.050+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:30.070 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-11-25T04:23:30.090 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:30.090 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2023-11-25T04:23:30.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2023-11-25T04:23:30.090 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2023-11-25T04:23:30.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.091 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:155 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'109, mlcod=11'105)", 2023-11-25T04:23:30.091 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:30.091 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 52.303092212000003, 2023-11-25T04:23:30.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061719019999999999, 2023-11-25T04:23:30.091 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:30.091 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:30.091 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:30.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:30.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:30.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:30.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.092 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:30.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:30.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:30.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.092 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:30.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784884+0000", 2023-11-25T04:23:30.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3099999999999999e-07 2023-11-25T04:23:30.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.092 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:30.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784942+0000", 2023-11-25T04:23:30.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8570000000000003e-05 2023-11-25T04:23:30.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:30.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784949+0000", 2023-11-25T04:23:30.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0310000000000001e-06 2023-11-25T04:23:30.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:30.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784953+0000", 2023-11-25T04:23:30.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6320000000000001e-06 2023-11-25T04:23:30.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:30.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784993+0000", 2023-11-25T04:23:30.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9737000000000003e-05 2023-11-25T04:23:30.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:30.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785011+0000", 2023-11-25T04:23:30.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8349999999999999e-05 2023-11-25T04:23:30.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:30.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791024+0000", 2023-11-25T04:23:30.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006012997 2023-11-25T04:23:30.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:30.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791055+0000", 2023-11-25T04:23:30.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1254000000000003e-05 2023-11-25T04:23:30.095 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:30.095 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:30.095 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:30.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.095 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:156 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'110, mlcod=11'105)", 2023-11-25T04:23:30.096 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:30.096 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 52.302506162999997, 2023-11-25T04:23:30.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005616116, 2023-11-25T04:23:30.096 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:30.096 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:30.096 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:30.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:30.096 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:30.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:30.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:30.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:30.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:30.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:30.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785470+0000", 2023-11-25T04:23:30.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0699999999999998e-07 2023-11-25T04:23:30.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:30.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785474+0000", 2023-11-25T04:23:30.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.048e-06 2023-11-25T04:23:30.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.098 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:30.098 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785480+0000", 2023-11-25T04:23:30.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7729999999999996e-06 2023-11-25T04:23:30.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.098 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:30.098 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785483+0000", 2023-11-25T04:23:30.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1970000000000001e-06 2023-11-25T04:23:30.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.099 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:30.099 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787936+0000", 2023-11-25T04:23:30.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024529040000000001 2023-11-25T04:23:30.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.099 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:30.099 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787947+0000", 2023-11-25T04:23:30.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1669e-05 2023-11-25T04:23:30.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:30.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791057+0000", 2023-11-25T04:23:30.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031095200000000002 2023-11-25T04:23:30.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:30.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791086+0000", 2023-11-25T04:23:30.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8697999999999999e-05 2023-11-25T04:23:30.100 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:30.100 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:30.100 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:30.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.101 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:157 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'111, mlcod=11'106)", 2023-11-25T04:23:30.101 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:30.101 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 52.301913605000003, 2023-11-25T04:23:30.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050590280000000001, 2023-11-25T04:23:30.101 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:30.101 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:30.101 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:30.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:30.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:30.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:30.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.102 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:30.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:30.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:30.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.102 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:30.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786063+0000", 2023-11-25T04:23:30.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2600000000000002e-07 2023-11-25T04:23:30.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.103 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:30.103 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786069+0000", 2023-11-25T04:23:30.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8839999999999999e-06 2023-11-25T04:23:30.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.103 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:30.103 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786079+0000", 2023-11-25T04:23:30.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0124e-05 2023-11-25T04:23:30.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.103 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:30.103 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786085+0000", 2023-11-25T04:23:30.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6660000000000002e-06 2023-11-25T04:23:30.104 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.104 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:30.104 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788160+0000", 2023-11-25T04:23:30.104 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020747669999999999 2023-11-25T04:23:30.104 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.104 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:30.104 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788180+0000", 2023-11-25T04:23:30.104 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0208999999999999e-05 2023-11-25T04:23:30.104 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.105 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:30.105 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791087+0000", 2023-11-25T04:23:30.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029064630000000002 2023-11-25T04:23:30.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.105 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:30.105 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791121+0000", 2023-11-25T04:23:30.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4289000000000003e-05 2023-11-25T04:23:30.105 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:30.105 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:30.105 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:30.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.106 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:158 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'112, mlcod=11'106)", 2023-11-25T04:23:30.106 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:30.106 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 52.301497212000001, 2023-11-25T04:23:30.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046663030000000001, 2023-11-25T04:23:30.106 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:30.106 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:30.106 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:30.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.106 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:30.106 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:30.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:30.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.107 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:30.107 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:30.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:30.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.107 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:30.107 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786479+0000", 2023-11-25T04:23:30.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2799999999999996e-07 2023-11-25T04:23:30.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.107 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:30.107 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786498+0000", 2023-11-25T04:23:30.108 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9069e-05 2023-11-25T04:23:30.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.108 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:30.108 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786508+0000", 2023-11-25T04:23:30.108 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5810000000000003e-06 2023-11-25T04:23:30.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.108 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:30.108 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786512+0000", 2023-11-25T04:23:30.108 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4889999999999997e-06 2023-11-25T04:23:30.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.109 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:30.109 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788438+0000", 2023-11-25T04:23:30.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019257059999999999 2023-11-25T04:23:30.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.109 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:30.109 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788449+0000", 2023-11-25T04:23:30.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1419e-05 2023-11-25T04:23:30.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.109 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:30.110 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791122+0000", 2023-11-25T04:23:30.110 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002672822 2023-11-25T04:23:30.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.110 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.110 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:30.110 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791145+0000", 2023-11-25T04:23:30.110 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2589e-05 2023-11-25T04:23:30.110 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:30.110 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:30.110 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:30.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.110 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.111 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:206 1.0 e12/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 12'152, mlcod=12'148)", 2023-11-25T04:23:30.111 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:30.111 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.789728742000001, 2023-11-25T04:23:30.111 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046728189999999999, 2023-11-25T04:23:30.111 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:30.111 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:30.111 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:30.111 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.113 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:30.113 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:30.113 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:30.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.113 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:30.113 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:30.113 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:30.114 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.114 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.114 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:30.114 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298248+0000", 2023-11-25T04:23:30.114 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3600000000000003e-07 2023-11-25T04:23:30.114 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.114 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.114 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:30.114 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298278+0000", 2023-11-25T04:23:30.114 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0474e-05 2023-11-25T04:23:30.114 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.115 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:30.115 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298288+0000", 2023-11-25T04:23:30.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0156000000000001e-05 2023-11-25T04:23:30.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.115 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:30.115 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298293+0000", 2023-11-25T04:23:30.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7400000000000004e-06 2023-11-25T04:23:30.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.115 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:30.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298333+0000", 2023-11-25T04:23:30.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9666000000000003e-05 2023-11-25T04:23:30.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.116 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:30.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298351+0000", 2023-11-25T04:23:30.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8544999999999999e-05 2023-11-25T04:23:30.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.116 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:30.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302891+0000", 2023-11-25T04:23:30.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045398519999999996 2023-11-25T04:23:30.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.117 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:30.117 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302920+0000", 2023-11-25T04:23:30.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8750000000000001e-05 2023-11-25T04:23:30.117 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:30.117 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:30.117 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:30.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.117 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:14 3.15 3:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:30.117 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:30.118 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 43.054594385000001, 2023-11-25T04:23:30.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010990437, 2023-11-25T04:23:30.118 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:30.118 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:30.118 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:30.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.118 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:30.118 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:30.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:30.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:30.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:30.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:30.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.120 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:30.120 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033383+0000", 2023-11-25T04:23:30.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9870000000000002e-06 2023-11-25T04:23:30.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.120 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:30.120 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033398+0000", 2023-11-25T04:23:30.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4569999999999999e-05 2023-11-25T04:23:30.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.121 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:30.121 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033411+0000", 2023-11-25T04:23:30.121 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3543999999999999e-05 2023-11-25T04:23:30.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.121 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:30.121 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033423+0000", 2023-11-25T04:23:30.121 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1905999999999999e-05 2023-11-25T04:23:30.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.121 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:30.121 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034433+0000", 2023-11-25T04:23:30.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00101022 2023-11-25T04:23:30.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.122 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:30.123 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034539+0000", 2023-11-25T04:23:30.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010597199999999999 2023-11-25T04:23:30.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.123 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:30.123 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034667+0000", 2023-11-25T04:23:30.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012765500000000001 2023-11-25T04:23:30.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.124 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:30.124 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037803+0000", 2023-11-25T04:23:30.124 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031362220000000001 2023-11-25T04:23:30.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.124 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:30.124 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:23:30.124 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0064964669999999997 2023-11-25T04:23:30.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.124 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:30.124 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044331+0000", 2023-11-25T04:23:30.125 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1544999999999997e-05 2023-11-25T04:23:30.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.125 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:30.125 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044372+0000", 2023-11-25T04:23:30.125 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0349000000000003e-05 2023-11-25T04:23:30.125 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:30.125 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:30.125 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:30.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.126 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:15 3.15 3:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:30.126 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:30.126 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 43.053934734999999, 2023-11-25T04:23:30.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010464362, 2023-11-25T04:23:30.126 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:30.126 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:30.126 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:30.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.126 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:30.126 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:30.127 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:30.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.127 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:30.127 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:30.127 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:30.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.127 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:30.127 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034042+0000", 2023-11-25T04:23:30.127 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0999999999999997e-07 2023-11-25T04:23:30.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.128 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.128 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:30.128 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034048+0000", 2023-11-25T04:23:30.128 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0390000000000001e-06 2023-11-25T04:23:30.128 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.128 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.128 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:30.128 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034059+0000", 2023-11-25T04:23:30.128 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1182999999999999e-05 2023-11-25T04:23:30.129 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.129 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.129 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:30.129 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034067+0000", 2023-11-25T04:23:30.129 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1559999999999995e-06 2023-11-25T04:23:30.129 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.129 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.129 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:30.129 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035005+0000", 2023-11-25T04:23:30.129 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00093747399999999999 2023-11-25T04:23:30.129 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.129 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.130 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:30.130 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035077+0000", 2023-11-25T04:23:30.130 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2747e-05 2023-11-25T04:23:30.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.130 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.130 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:30.130 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035219+0000", 2023-11-25T04:23:30.130 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000141877 2023-11-25T04:23:30.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.130 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.130 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:30.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037810+0000", 2023-11-25T04:23:30.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002590752 2023-11-25T04:23:30.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:30.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044437+0000", 2023-11-25T04:23:30.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066270110000000004 2023-11-25T04:23:30.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:30.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044481+0000", 2023-11-25T04:23:30.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4106000000000001e-05 2023-11-25T04:23:30.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.132 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:30.132 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044505+0000", 2023-11-25T04:23:30.132 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4207e-05 2023-11-25T04:23:30.132 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:30.132 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:30.132 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:30.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.132 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:16 3.1d 3:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:30.132 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:30.133 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 43.053892887000003, 2023-11-25T04:23:30.133 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010413923, 2023-11-25T04:23:30.133 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:30.133 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:30.134 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:30.134 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.134 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:30.134 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:30.134 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:30.134 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.134 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.134 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:30.134 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:30.134 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:30.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.135 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:30.135 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:30.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4499999999999997e-07 2023-11-25T04:23:30.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.135 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:30.135 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034087+0000", 2023-11-25T04:23:30.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9770000000000002e-06 2023-11-25T04:23:30.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:30.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034094+0000", 2023-11-25T04:23:30.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8809999999999998e-06 2023-11-25T04:23:30.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:30.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034100+0000", 2023-11-25T04:23:30.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8440000000000003e-06 2023-11-25T04:23:30.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:30.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034993+0000", 2023-11-25T04:23:30.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00089336799999999996 2023-11-25T04:23:30.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:30.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035099+0000", 2023-11-25T04:23:30.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106037 2023-11-25T04:23:30.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:30.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035276+0000", 2023-11-25T04:23:30.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017631799999999999 2023-11-25T04:23:30.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.138 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:30.138 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037815+0000", 2023-11-25T04:23:30.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025395959999999999 2023-11-25T04:23:30.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.138 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:30.138 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044420+0000", 2023-11-25T04:23:30.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066045749999999997 2023-11-25T04:23:30.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.138 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:30.138 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044463+0000", 2023-11-25T04:23:30.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3205999999999999e-05 2023-11-25T04:23:30.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.139 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:30.139 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044497+0000", 2023-11-25T04:23:30.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3476000000000003e-05 2023-11-25T04:23:30.139 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:30.139 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:30.139 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:30.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.139 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:17 3.1d 3:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:30.140 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:30.140 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 43.053867678000003, 2023-11-25T04:23:30.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01127356, 2023-11-25T04:23:30.140 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:30.140 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:30.140 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:30.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.140 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:30.140 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:30.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:30.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.140 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:30.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:30.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:30.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.141 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.141 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:30.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:30.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1100000000000001e-07 2023-11-25T04:23:30.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.141 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.141 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:30.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034112+0000", 2023-11-25T04:23:30.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8600000000000003e-06 2023-11-25T04:23:30.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.142 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:30.142 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034120+0000", 2023-11-25T04:23:30.142 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7700000000000001e-06 2023-11-25T04:23:30.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.142 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:30.142 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034127+0000", 2023-11-25T04:23:30.142 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0999999999999998e-06 2023-11-25T04:23:30.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:30.143 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035601+0000", 2023-11-25T04:23:30.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014739410000000001 2023-11-25T04:23:30.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:30.143 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035685+0000", 2023-11-25T04:23:30.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3677999999999994e-05 2023-11-25T04:23:30.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:30.143 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035827+0000", 2023-11-25T04:23:30.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014232599999999999 2023-11-25T04:23:30.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:30.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038725+0000", 2023-11-25T04:23:30.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028982779999999998 2023-11-25T04:23:30.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:30.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045323+0000", 2023-11-25T04:23:30.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0065980780000000003 2023-11-25T04:23:30.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:30.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045354+0000", 2023-11-25T04:23:30.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0463e-05 2023-11-25T04:23:30.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:30.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045381+0000", 2023-11-25T04:23:30.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7654999999999999e-05 2023-11-25T04:23:30.145 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:30.145 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:30.145 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:30.146 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.146 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:22 3.e 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:30.146 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:30.146 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 43.053841265000003, 2023-11-25T04:23:30.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010133422, 2023-11-25T04:23:30.146 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:30.146 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:30.146 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:30.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.146 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:30.146 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:30.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:30.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.147 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:30.147 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:30.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:30.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.147 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:30.147 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034135+0000", 2023-11-25T04:23:30.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-07 2023-11-25T04:23:30.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.148 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:30.148 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034138+0000", 2023-11-25T04:23:30.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6009999999999999e-06 2023-11-25T04:23:30.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.148 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:30.148 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034145+0000", 2023-11-25T04:23:30.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1169999999999998e-06 2023-11-25T04:23:30.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.148 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:30.148 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034150+0000", 2023-11-25T04:23:30.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9080000000000003e-06 2023-11-25T04:23:30.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:30.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034209+0000", 2023-11-25T04:23:30.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8149000000000002e-05 2023-11-25T04:23:30.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:30.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034415+0000", 2023-11-25T04:23:30.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020637699999999999 2023-11-25T04:23:30.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:30.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034584+0000", 2023-11-25T04:23:30.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016870399999999999 2023-11-25T04:23:30.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:30.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.036979+0000", 2023-11-25T04:23:30.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023957509999999998 2023-11-25T04:23:30.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:30.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044161+0000", 2023-11-25T04:23:30.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0071811590000000003 2023-11-25T04:23:30.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.151 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:30.151 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044224+0000", 2023-11-25T04:23:30.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3454000000000006e-05 2023-11-25T04:23:30.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.151 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:30.151 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044268+0000", 2023-11-25T04:23:30.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3761999999999999e-05 2023-11-25T04:23:30.151 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:30.151 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:30.152 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:30.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.152 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:23 3.e 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:30.152 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:30.152 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 43.053809856000001, 2023-11-25T04:23:30.152 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010281676, 2023-11-25T04:23:30.152 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:30.152 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:30.152 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:30.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.152 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:30.152 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:30.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:30.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:30.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:30.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:30.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:30.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:30.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5799999999999999e-07 2023-11-25T04:23:30.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:30.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034169+0000", 2023-11-25T04:23:30.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1879999999999998e-06 2023-11-25T04:23:30.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:30.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034177+0000", 2023-11-25T04:23:30.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7020000000000001e-06 2023-11-25T04:23:30.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:30.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034184+0000", 2023-11-25T04:23:30.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3910000000000003e-06 2023-11-25T04:23:30.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.155 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:30.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034924+0000", 2023-11-25T04:23:30.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00074032999999999998 2023-11-25T04:23:30.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.155 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:30.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034991+0000", 2023-11-25T04:23:30.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7347000000000004e-05 2023-11-25T04:23:30.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.156 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:30.156 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035096+0000", 2023-11-25T04:23:30.156 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010488 2023-11-25T04:23:30.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.156 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:30.156 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037588+0000", 2023-11-25T04:23:30.156 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024918309999999999 2023-11-25T04:23:30.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.156 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:30.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044367+0000", 2023-11-25T04:23:30.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067787589999999997 2023-11-25T04:23:30.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.157 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:30.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044415+0000", 2023-11-25T04:23:30.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7803000000000003e-05 2023-11-25T04:23:30.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.157 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:30.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044447+0000", 2023-11-25T04:23:30.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2886999999999999e-05 2023-11-25T04:23:30.157 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:30.158 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:30.158 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:30.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.158 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:30 3.15 3:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:30.158 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:30.158 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 43.053751415999997, 2023-11-25T04:23:30.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010365070000000001, 2023-11-25T04:23:30.158 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:30.158 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:30.158 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:30.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:30.159 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:30.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:30.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.159 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:30.159 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:30.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:30.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.159 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:30.159 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034225+0000", 2023-11-25T04:23:30.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7400000000000002e-07 2023-11-25T04:23:30.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.160 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.160 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:30.160 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034229+0000", 2023-11-25T04:23:30.160 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3100000000000002e-06 2023-11-25T04:23:30.160 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.160 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.160 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:30.160 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034239+0000", 2023-11-25T04:23:30.160 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9140000000000003e-06 2023-11-25T04:23:30.160 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.160 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.160 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:30.160 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034246+0000", 2023-11-25T04:23:30.161 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0149999999999998e-06 2023-11-25T04:23:30.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.161 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.161 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:30.161 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035572+0000", 2023-11-25T04:23:30.161 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001325655 2023-11-25T04:23:30.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.161 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.161 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:30.161 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035730+0000", 2023-11-25T04:23:30.161 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015827899999999999 2023-11-25T04:23:30.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.162 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:30.162 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035857+0000", 2023-11-25T04:23:30.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012644699999999999 2023-11-25T04:23:30.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.162 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:30.162 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038737+0000", 2023-11-25T04:23:30.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002880077 2023-11-25T04:23:30.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.162 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:30.163 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044540+0000", 2023-11-25T04:23:30.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058034239999999997 2023-11-25T04:23:30.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.163 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.163 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:30.163 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044569+0000", 2023-11-25T04:23:30.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8388999999999999e-05 2023-11-25T04:23:30.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.163 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.163 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:30.163 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044589+0000", 2023-11-25T04:23:30.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0686000000000002e-05 2023-11-25T04:23:30.164 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:30.164 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:30.164 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:30.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.164 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.164 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:33 3.1b 3:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:30.164 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:30.164 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 43.053705917999999, 2023-11-25T04:23:30.164 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099754049999999997, 2023-11-25T04:23:30.164 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:30.164 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:23:30.164 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:30.165 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.165 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:30.165 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:30.165 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:30.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.165 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.165 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:30.165 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:30.165 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:30.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.165 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.165 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:30.165 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:30.166 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4799999999999998e-07 2023-11-25T04:23:30.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.166 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:30.166 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034273+0000", 2023-11-25T04:23:30.166 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7819999999999998e-06 2023-11-25T04:23:30.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.166 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:30.166 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034285+0000", 2023-11-25T04:23:30.166 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2058e-05 2023-11-25T04:23:30.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.167 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:30.167 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034291+0000", 2023-11-25T04:23:30.167 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0680000000000002e-06 2023-11-25T04:23:30.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.167 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:30.167 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034370+0000", 2023-11-25T04:23:30.167 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8452e-05 2023-11-25T04:23:30.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.167 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:30.167 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034477+0000", 2023-11-25T04:23:30.168 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107471 2023-11-25T04:23:30.168 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.168 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.168 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:30.168 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044245+0000", 2023-11-25T04:23:30.168 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0097679259999999997 2023-11-25T04:23:30.168 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:30.168 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:30.168 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:30.168 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.168 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.168 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:36 3.1a 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:30.168 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:30.169 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 43.053659244999999, 2023-11-25T04:23:30.169 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058179099999999999, 2023-11-25T04:23:30.169 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:30.169 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:23:30.169 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:30.169 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.169 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:30.169 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:30.169 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:30.169 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.169 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.169 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:30.169 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:30.170 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:30.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.170 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:30.170 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034317+0000", 2023-11-25T04:23:30.170 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1999999999999999e-07 2023-11-25T04:23:30.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.170 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:30.170 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034321+0000", 2023-11-25T04:23:30.170 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.233e-06 2023-11-25T04:23:30.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.171 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:30.171 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034337+0000", 2023-11-25T04:23:30.171 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5308999999999999e-05 2023-11-25T04:23:30.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.171 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:30.171 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034354+0000", 2023-11-25T04:23:30.171 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7411999999999999e-05 2023-11-25T04:23:30.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.171 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:30.172 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034465+0000", 2023-11-25T04:23:30.172 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011154699999999999 2023-11-25T04:23:30.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.172 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:30.172 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034560+0000", 2023-11-25T04:23:30.172 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4186999999999998e-05 2023-11-25T04:23:30.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.172 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:30.172 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.040134+0000", 2023-11-25T04:23:30.172 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055746019999999997 2023-11-25T04:23:30.172 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:30.173 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:30.173 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:30.173 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.173 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.173 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:9 3.10 e18/16 3:09401a81:::601.00000000:head v 18'4, mlcod=0'0)", 2023-11-25T04:23:30.173 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:30.173 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 43.044764936999997, 2023-11-25T04:23:30.173 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061641020000000003, 2023-11-25T04:23:30.173 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:30.173 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:30.173 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:30.173 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.174 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:30.174 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:30.174 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:30.174 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.174 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.174 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:30.174 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:30.174 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:30.174 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.174 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.174 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:30.174 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:30.174 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5300000000000001e-07 2023-11-25T04:23:30.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.175 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.175 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:30.175 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043222+0000", 2023-11-25T04:23:30.175 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.082e-05 2023-11-25T04:23:30.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.175 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.175 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:30.175 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043228+0000", 2023-11-25T04:23:30.175 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6699999999999997e-06 2023-11-25T04:23:30.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.175 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.176 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:30.176 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043236+0000", 2023-11-25T04:23:30.176 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8189999999999992e-06 2023-11-25T04:23:30.176 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.176 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.176 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:30.176 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043507+0000", 2023-11-25T04:23:30.176 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000271012 2023-11-25T04:23:30.176 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.176 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.176 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:30.176 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043530+0000", 2023-11-25T04:23:30.176 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2659000000000001e-05 2023-11-25T04:23:30.177 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.177 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.177 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:30.177 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049321+0000", 2023-11-25T04:23:30.177 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057915129999999999 2023-11-25T04:23:30.177 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.177 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.177 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:30.177 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049375+0000", 2023-11-25T04:23:30.177 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3455999999999998e-05 2023-11-25T04:23:30.177 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:30.177 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:30.177 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:30.178 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.178 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.178 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:1 3.14 e18/16 3:292cf221:::200.00000000:head v 18'3, mlcod=0'0)", 2023-11-25T04:23:30.178 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:30.178 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 43.044728224000004, 2023-11-25T04:23:30.178 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0060983599999999997, 2023-11-25T04:23:30.178 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:30.178 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:30.178 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:30.178 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.178 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:30.178 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:30.179 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:30.179 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.179 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.179 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:30.179 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:30.179 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:30.179 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.179 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.179 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:30.179 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043248+0000", 2023-11-25T04:23:30.179 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9399999999999995e-07 2023-11-25T04:23:30.180 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.180 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.180 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:30.180 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043252+0000", 2023-11-25T04:23:30.180 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1799999999999998e-06 2023-11-25T04:23:30.180 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.180 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.180 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:30.180 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043259+0000", 2023-11-25T04:23:30.180 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0049999999999998e-06 2023-11-25T04:23:30.180 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.180 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.180 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:30.181 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043265+0000", 2023-11-25T04:23:30.181 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0859999999999998e-06 2023-11-25T04:23:30.181 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.181 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.181 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:30.181 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043311+0000", 2023-11-25T04:23:30.181 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6082999999999999e-05 2023-11-25T04:23:30.181 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.181 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.181 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:30.181 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043330+0000", 2023-11-25T04:23:30.181 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8556e-05 2023-11-25T04:23:30.182 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.182 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.182 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:30.182 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049303+0000", 2023-11-25T04:23:30.182 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059733470000000004 2023-11-25T04:23:30.182 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.182 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.182 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:30.182 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049346+0000", 2023-11-25T04:23:30.182 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2608999999999998e-05 2023-11-25T04:23:30.182 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:30.182 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:30.182 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:30.183 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.183 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.183 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:3 3.14 e18/16 3:287034c4:::1.00000000.inode:head v 18'4, mlcod=0'0)", 2023-11-25T04:23:30.183 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:30.183 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 43.044211718, 2023-11-25T04:23:30.183 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057040090000000003, 2023-11-25T04:23:30.183 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:30.183 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:30.183 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:30.183 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.183 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:30.183 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:30.184 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:30.184 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.184 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.184 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:30.184 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:30.184 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:30.184 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.184 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.184 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:30.184 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:30.184 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9799999999999999e-07 2023-11-25T04:23:30.184 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.184 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.185 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:30.185 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043785+0000", 2023-11-25T04:23:30.185 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0721999999999999e-05 2023-11-25T04:23:30.185 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.185 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.185 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:30.185 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043793+0000", 2023-11-25T04:23:30.185 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5839999999999995e-06 2023-11-25T04:23:30.185 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.185 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.185 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:30.185 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043803+0000", 2023-11-25T04:23:30.186 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0729999999999994e-06 2023-11-25T04:23:30.186 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.186 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.186 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:30.186 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043843+0000", 2023-11-25T04:23:30.186 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0747999999999997e-05 2023-11-25T04:23:30.186 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.186 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.186 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:30.186 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043856+0000", 2023-11-25T04:23:30.186 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2697000000000001e-05 2023-11-25T04:23:30.186 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.186 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.187 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:30.187 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049426+0000", 2023-11-25T04:23:30.187 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055704409999999998 2023-11-25T04:23:30.187 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.187 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.187 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:30.187 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049468+0000", 2023-11-25T04:23:30.187 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1446000000000003e-05 2023-11-25T04:23:30.187 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:30.187 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:30.187 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:30.187 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.187 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.188 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "pg_update_log_missing(3.14 epoch 18/16 rep_tid 171 entries 18'5 (0'0) error 3:2e2fa760:::200.00000001:head by mds.0.3:32 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:23:30.188 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:30.188 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 43.044050755000001, 2023-11-25T04:23:30.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005556938, 2023-11-25T04:23:30.188 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:30.188 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2023-11-25T04:23:30.188 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:30.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.188 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:30.188 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:30.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:30.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:30.189 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:30.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:30.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:30.189 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:30.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.03e-07 2023-11-25T04:23:30.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:30.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043931+0000", 2023-11-25T04:23:30.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1079999999999998e-06 2023-11-25T04:23:30.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.190 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:30.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043944+0000", 2023-11-25T04:23:30.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2852e-05 2023-11-25T04:23:30.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.190 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:30.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043955+0000", 2023-11-25T04:23:30.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1037e-05 2023-11-25T04:23:30.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.191 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:30.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044088+0000", 2023-11-25T04:23:30.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013269599999999999 2023-11-25T04:23:30.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.191 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:30.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049482+0000", 2023-11-25T04:23:30.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053940419999999999 2023-11-25T04:23:30.191 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:30.191 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:30.191 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:30.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.192 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:10 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'5, mlcod=0'0)", 2023-11-25T04:23:30.192 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:30.192 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 43.043960726999998, 2023-11-25T04:23:30.192 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054493099999999997, 2023-11-25T04:23:30.192 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:30.192 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:30.192 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:30.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.192 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:30.192 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:30.192 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:30.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.193 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:30.193 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:30.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:30.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.193 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:30.193 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:30.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6100000000000002e-07 2023-11-25T04:23:30.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.193 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:30.193 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044019+0000", 2023-11-25T04:23:30.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.839e-06 2023-11-25T04:23:30.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:30.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044026+0000", 2023-11-25T04:23:30.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5869999999999998e-06 2023-11-25T04:23:30.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:30.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044031+0000", 2023-11-25T04:23:30.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4909999999999996e-06 2023-11-25T04:23:30.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.195 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:30.195 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044070+0000", 2023-11-25T04:23:30.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8829999999999999e-05 2023-11-25T04:23:30.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.195 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:30.195 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044081+0000", 2023-11-25T04:23:30.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1447999999999999e-05 2023-11-25T04:23:30.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.195 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:30.195 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049406+0000", 2023-11-25T04:23:30.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053246480000000004 2023-11-25T04:23:30.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.196 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:30.196 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049464+0000", 2023-11-25T04:23:30.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8205999999999998e-05 2023-11-25T04:23:30.196 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:30.196 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:30.196 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:30.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.196 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:11 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'6, mlcod=0'0)", 2023-11-25T04:23:30.196 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:30.197 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 43.043759463000001, 2023-11-25T04:23:30.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052673499999999996, 2023-11-25T04:23:30.197 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:30.197 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:30.197 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:30.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:30.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:30.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:30.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:30.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:30.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:30.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:30.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:30.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0199999999999998e-07 2023-11-25T04:23:30.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:30.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044220+0000", 2023-11-25T04:23:30.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7720000000000001e-06 2023-11-25T04:23:30.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:30.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044226+0000", 2023-11-25T04:23:30.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8579999999999996e-06 2023-11-25T04:23:30.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:30.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044232+0000", 2023-11-25T04:23:30.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1700000000000002e-06 2023-11-25T04:23:30.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:30.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044274+0000", 2023-11-25T04:23:30.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1254999999999997e-05 2023-11-25T04:23:30.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.200 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:30.200 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:23:30.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6880999999999999e-05 2023-11-25T04:23:30.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.200 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:30.200 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049466+0000", 2023-11-25T04:23:30.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051659219999999999 2023-11-25T04:23:30.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:30.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:30.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:30.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049483+0000", 2023-11-25T04:23:30.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.719e-05 2023-11-25T04:23:30.201 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:30.201 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:30.201 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:30.201 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:30.201 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:30.201 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:30.203 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:23:30.203 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:30.201+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:30.245 INFO:teuthology.orchestra.run.smithi067.stderr:Error ENXIO: problem getting command descriptions from osd.0 2023-11-25T04:23:30.279 INFO:teuthology.orchestra.run.smithi067.stdout:osd.1: {} 2023-11-25T04:23:30.280 INFO:teuthology.orchestra.run.smithi067.stderr:osd.1: osd_enable_op_tracker = '' 2023-11-25T04:23:30.303 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:30.301+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:30.311 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:30.311 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:23:30.312 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2023-11-25T04:23:30.312 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:30.313 INFO:teuthology.orchestra.run.smithi067.stdout:osd.2: {} 2023-11-25T04:23:30.314 INFO:teuthology.orchestra.run.smithi067.stderr:osd.2: osd_enable_op_tracker = '' 2023-11-25T04:23:30.318 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-11-25T04:23:30.318 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-11-25T04:23:30.320 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:23:30.404 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:30.402+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:30.425 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:30.425 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:30.434 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:23:30.504 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:30.503+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:30.579 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:30.579 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:30.588 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:23:30.605 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:30.603+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:30.702 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:30.702 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:30.705 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:30.703+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:30.711 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:23:30.807 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:30.804+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:30.858 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:30.858 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:30.867 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:23:30.908 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:30.907+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:30.924 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2023-11-25T04:23:31.009 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:31.008+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:31.011 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:31.012 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:31.020 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:23:31.110 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:31.108+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:31.129 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:31.129 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:31.138 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:23:31.222 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:31.220+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:31.254 DEBUG:teuthology.orchestra.console.smithi076:pexpect command: ipmitool -H smithi076.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-25T04:23:31.284 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:31.284 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:31.293 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:23:31.313 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:31.310+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:31.366 DEBUG:teuthology.orchestra.console.smithi076:check power output: Chassis Power is off 2023-11-25T04:23:31.413 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:31.411+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:31.438 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:31.438 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:31.447 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:23:31.514 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:31.512+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:31.556 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:31.556 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:31.565 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:23:31.615 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:31.614+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:31.710 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:31.710 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:31.716 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:31.715+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:31.718 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:23:31.817 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:31.815+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:31.824 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:31.824 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:31.833 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:23:31.908 INFO:teuthology.orchestra.run.smithi067.stderr:nodeep-scrub is set 2023-11-25T04:23:31.918 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:31.916+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:31.942 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:31.942 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:31.951 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:23:32.019 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:32.018+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:32.056 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:32.056 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:32.065 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:23:32.121 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:32.119+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:32.171 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:32.172 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:32.182 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:23:32.223 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:32.221+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:32.286 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:32.287 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:32.295 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:23:32.319 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-11-25T04:23:32.340 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:32.338+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:32.372 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~23s 2023-11-25T04:23:32.401 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:32.402 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:32.410 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:23:32.451 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:32.449+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:32.485 INFO:teuthology.orchestra.run.smithi067.stderr:Error ENXIO: problem getting command descriptions from osd.0 2023-11-25T04:23:32.503 INFO:teuthology.orchestra.run.smithi067.stdout:osd.1: {} 2023-11-25T04:23:32.503 INFO:teuthology.orchestra.run.smithi067.stderr:osd.1: osd_enable_op_tracker = '' 2023-11-25T04:23:32.521 INFO:teuthology.orchestra.run.smithi067.stdout:osd.2: {} 2023-11-25T04:23:32.521 INFO:teuthology.orchestra.run.smithi067.stderr:osd.2: osd_enable_op_tracker = '' 2023-11-25T04:23:32.523 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:32.521+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:32.524 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-11-25T04:23:32.524 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-11-25T04:23:32.559 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:32.559 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2023-11-25T04:23:32.559 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2023-11-25T04:23:32.559 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2023-11-25T04:23:32.559 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.559 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:155 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'109, mlcod=11'105)", 2023-11-25T04:23:32.559 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:32.560 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 54.771765858999998, 2023-11-25T04:23:32.560 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061719019999999999, 2023-11-25T04:23:32.560 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:32.560 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:32.560 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:32.560 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.560 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:32.560 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:32.560 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:32.560 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.561 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:32.561 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:32.561 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:32.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.561 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:32.561 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784884+0000", 2023-11-25T04:23:32.561 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3099999999999999e-07 2023-11-25T04:23:32.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.562 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:32.562 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784942+0000", 2023-11-25T04:23:32.562 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8570000000000003e-05 2023-11-25T04:23:32.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.562 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.562 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:32.562 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784949+0000", 2023-11-25T04:23:32.562 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0310000000000001e-06 2023-11-25T04:23:32.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.562 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.562 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:32.562 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784953+0000", 2023-11-25T04:23:32.562 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6320000000000001e-06 2023-11-25T04:23:32.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.563 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:32.563 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784993+0000", 2023-11-25T04:23:32.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9737000000000003e-05 2023-11-25T04:23:32.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.563 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:32.563 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785011+0000", 2023-11-25T04:23:32.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8349999999999999e-05 2023-11-25T04:23:32.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:32.564 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791024+0000", 2023-11-25T04:23:32.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006012997 2023-11-25T04:23:32.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:32.564 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791055+0000", 2023-11-25T04:23:32.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1254000000000003e-05 2023-11-25T04:23:32.564 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:32.564 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:32.564 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:32.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.565 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:156 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'110, mlcod=11'105)", 2023-11-25T04:23:32.565 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:32.565 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 54.77117981, 2023-11-25T04:23:32.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005616116, 2023-11-25T04:23:32.565 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:32.565 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:32.565 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:32.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.565 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:32.565 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:32.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:32.566 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.566 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.566 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:32.566 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:32.566 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:32.566 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.566 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.566 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:32.566 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785470+0000", 2023-11-25T04:23:32.566 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0699999999999998e-07 2023-11-25T04:23:32.566 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:32.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785474+0000", 2023-11-25T04:23:32.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.048e-06 2023-11-25T04:23:32.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:32.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785480+0000", 2023-11-25T04:23:32.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7729999999999996e-06 2023-11-25T04:23:32.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:32.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785483+0000", 2023-11-25T04:23:32.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1970000000000001e-06 2023-11-25T04:23:32.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:32.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787936+0000", 2023-11-25T04:23:32.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024529040000000001 2023-11-25T04:23:32.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:32.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787947+0000", 2023-11-25T04:23:32.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1669e-05 2023-11-25T04:23:32.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:32.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791057+0000", 2023-11-25T04:23:32.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031095200000000002 2023-11-25T04:23:32.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:32.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791086+0000", 2023-11-25T04:23:32.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8697999999999999e-05 2023-11-25T04:23:32.569 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:32.569 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:32.569 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:32.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.570 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:157 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'111, mlcod=11'106)", 2023-11-25T04:23:32.570 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:32.570 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 54.770587251999999, 2023-11-25T04:23:32.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050590280000000001, 2023-11-25T04:23:32.570 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:32.570 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:32.570 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:32.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.570 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:32.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:32.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:32.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:32.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:32.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:32.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:32.572 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786063+0000", 2023-11-25T04:23:32.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2600000000000002e-07 2023-11-25T04:23:32.572 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.572 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.572 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:32.572 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786069+0000", 2023-11-25T04:23:32.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8839999999999999e-06 2023-11-25T04:23:32.572 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.572 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.572 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:32.572 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786079+0000", 2023-11-25T04:23:32.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0124e-05 2023-11-25T04:23:32.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.573 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:32.573 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786085+0000", 2023-11-25T04:23:32.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6660000000000002e-06 2023-11-25T04:23:32.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:32.574 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788160+0000", 2023-11-25T04:23:32.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020747669999999999 2023-11-25T04:23:32.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:32.574 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788180+0000", 2023-11-25T04:23:32.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0208999999999999e-05 2023-11-25T04:23:32.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:32.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791087+0000", 2023-11-25T04:23:32.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029064630000000002 2023-11-25T04:23:32.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:32.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791121+0000", 2023-11-25T04:23:32.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4289000000000003e-05 2023-11-25T04:23:32.575 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:32.575 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:32.576 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:32.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.576 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:158 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'112, mlcod=11'106)", 2023-11-25T04:23:32.576 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:32.576 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 54.770170858999997, 2023-11-25T04:23:32.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046663030000000001, 2023-11-25T04:23:32.576 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:32.576 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:32.577 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:32.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.577 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:32.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:32.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:32.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.577 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:32.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:32.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:32.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.578 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:32.578 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786479+0000", 2023-11-25T04:23:32.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2799999999999996e-07 2023-11-25T04:23:32.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.578 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:32.578 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786498+0000", 2023-11-25T04:23:32.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9069e-05 2023-11-25T04:23:32.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:32.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786508+0000", 2023-11-25T04:23:32.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5810000000000003e-06 2023-11-25T04:23:32.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:32.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786512+0000", 2023-11-25T04:23:32.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4889999999999997e-06 2023-11-25T04:23:32.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:32.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788438+0000", 2023-11-25T04:23:32.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019257059999999999 2023-11-25T04:23:32.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:32.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788449+0000", 2023-11-25T04:23:32.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1419e-05 2023-11-25T04:23:32.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:32.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791122+0000", 2023-11-25T04:23:32.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002672822 2023-11-25T04:23:32.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:32.581 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791145+0000", 2023-11-25T04:23:32.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2589e-05 2023-11-25T04:23:32.581 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:32.581 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:32.581 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:32.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.581 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:206 1.0 e12/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 12'152, mlcod=12'148)", 2023-11-25T04:23:32.581 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:32.581 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 54.258402388999997, 2023-11-25T04:23:32.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046728189999999999, 2023-11-25T04:23:32.581 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:32.581 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:32.582 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:32.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:32.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:32.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:32.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:32.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:32.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:32.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.583 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:32.583 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298248+0000", 2023-11-25T04:23:32.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3600000000000003e-07 2023-11-25T04:23:32.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.583 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:32.583 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298278+0000", 2023-11-25T04:23:32.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0474e-05 2023-11-25T04:23:32.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.583 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:32.583 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298288+0000", 2023-11-25T04:23:32.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0156000000000001e-05 2023-11-25T04:23:32.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.584 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:32.584 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298293+0000", 2023-11-25T04:23:32.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7400000000000004e-06 2023-11-25T04:23:32.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.584 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:32.584 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298333+0000", 2023-11-25T04:23:32.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9666000000000003e-05 2023-11-25T04:23:32.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:32.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298351+0000", 2023-11-25T04:23:32.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8544999999999999e-05 2023-11-25T04:23:32.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:32.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302891+0000", 2023-11-25T04:23:32.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045398519999999996 2023-11-25T04:23:32.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.587 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:32.587 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302920+0000", 2023-11-25T04:23:32.587 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8750000000000001e-05 2023-11-25T04:23:32.587 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:32.587 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:32.587 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:32.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.587 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:14 3.15 3:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:32.587 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:32.587 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 45.523268031999997, 2023-11-25T04:23:32.587 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010990437, 2023-11-25T04:23:32.587 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:32.588 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:32.588 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:32.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.588 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:32.588 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:32.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:32.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:32.589 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:32.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:32.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:32.589 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033383+0000", 2023-11-25T04:23:32.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9870000000000002e-06 2023-11-25T04:23:32.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:32.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033398+0000", 2023-11-25T04:23:32.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4569999999999999e-05 2023-11-25T04:23:32.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:32.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033411+0000", 2023-11-25T04:23:32.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3543999999999999e-05 2023-11-25T04:23:32.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:32.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033423+0000", 2023-11-25T04:23:32.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1905999999999999e-05 2023-11-25T04:23:32.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:32.593 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034433+0000", 2023-11-25T04:23:32.593 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00101022 2023-11-25T04:23:32.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:32.593 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034539+0000", 2023-11-25T04:23:32.593 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010597199999999999 2023-11-25T04:23:32.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:32.594 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034667+0000", 2023-11-25T04:23:32.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012765500000000001 2023-11-25T04:23:32.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.594 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:32.594 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037803+0000", 2023-11-25T04:23:32.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031362220000000001 2023-11-25T04:23:32.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.594 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:32.594 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:23:32.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0064964669999999997 2023-11-25T04:23:32.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:32.596 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044331+0000", 2023-11-25T04:23:32.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1544999999999997e-05 2023-11-25T04:23:32.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.596 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.596 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:32.596 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044372+0000", 2023-11-25T04:23:32.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0349000000000003e-05 2023-11-25T04:23:32.596 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:32.596 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:32.596 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:32.597 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.597 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.597 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:15 3.15 3:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:32.597 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:32.597 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 45.522608382000001, 2023-11-25T04:23:32.597 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010464362, 2023-11-25T04:23:32.597 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:32.597 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:32.597 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:32.597 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.597 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:32.597 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:32.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:32.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:32.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:32.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:32.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:32.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034042+0000", 2023-11-25T04:23:32.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0999999999999997e-07 2023-11-25T04:23:32.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:32.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034048+0000", 2023-11-25T04:23:32.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0390000000000001e-06 2023-11-25T04:23:32.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.600 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:32.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034059+0000", 2023-11-25T04:23:32.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1182999999999999e-05 2023-11-25T04:23:32.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.600 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:32.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034067+0000", 2023-11-25T04:23:32.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1559999999999995e-06 2023-11-25T04:23:32.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.600 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:32.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035005+0000", 2023-11-25T04:23:32.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00093747399999999999 2023-11-25T04:23:32.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:32.601 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035077+0000", 2023-11-25T04:23:32.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2747e-05 2023-11-25T04:23:32.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:32.601 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035219+0000", 2023-11-25T04:23:32.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000141877 2023-11-25T04:23:32.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:32.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037810+0000", 2023-11-25T04:23:32.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002590752 2023-11-25T04:23:32.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:32.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044437+0000", 2023-11-25T04:23:32.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066270110000000004 2023-11-25T04:23:32.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:32.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044481+0000", 2023-11-25T04:23:32.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4106000000000001e-05 2023-11-25T04:23:32.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:32.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044505+0000", 2023-11-25T04:23:32.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4207e-05 2023-11-25T04:23:32.604 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:32.604 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:32.604 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:32.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.604 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:16 3.1d 3:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:32.604 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:32.604 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 45.522566533999999, 2023-11-25T04:23:32.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010413923, 2023-11-25T04:23:32.604 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:32.604 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:32.605 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:32.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.605 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:32.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:32.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:32.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.605 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:32.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:32.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:32.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:32.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:32.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4499999999999997e-07 2023-11-25T04:23:32.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:32.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034087+0000", 2023-11-25T04:23:32.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9770000000000002e-06 2023-11-25T04:23:32.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:32.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034094+0000", 2023-11-25T04:23:32.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8809999999999998e-06 2023-11-25T04:23:32.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:32.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034100+0000", 2023-11-25T04:23:32.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8440000000000003e-06 2023-11-25T04:23:32.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:32.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034993+0000", 2023-11-25T04:23:32.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00089336799999999996 2023-11-25T04:23:32.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:32.608 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035099+0000", 2023-11-25T04:23:32.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106037 2023-11-25T04:23:32.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:32.608 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035276+0000", 2023-11-25T04:23:32.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017631799999999999 2023-11-25T04:23:32.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:32.608 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037815+0000", 2023-11-25T04:23:32.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025395959999999999 2023-11-25T04:23:32.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:32.609 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044420+0000", 2023-11-25T04:23:32.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066045749999999997 2023-11-25T04:23:32.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:32.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044463+0000", 2023-11-25T04:23:32.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3205999999999999e-05 2023-11-25T04:23:32.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.610 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:32.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044497+0000", 2023-11-25T04:23:32.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3476000000000003e-05 2023-11-25T04:23:32.610 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:32.610 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:32.610 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:32.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.610 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:17 3.1d 3:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:32.611 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:32.611 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 45.522541324999999, 2023-11-25T04:23:32.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01127356, 2023-11-25T04:23:32.611 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:32.611 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:32.611 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:32.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:32.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:32.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:32.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:32.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:32.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:32.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:32.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:32.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1100000000000001e-07 2023-11-25T04:23:32.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:32.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034112+0000", 2023-11-25T04:23:32.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8600000000000003e-06 2023-11-25T04:23:32.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.613 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:32.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034120+0000", 2023-11-25T04:23:32.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7700000000000001e-06 2023-11-25T04:23:32.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.613 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:32.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034127+0000", 2023-11-25T04:23:32.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0999999999999998e-06 2023-11-25T04:23:32.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.613 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:32.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035601+0000", 2023-11-25T04:23:32.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014739410000000001 2023-11-25T04:23:32.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:32.614 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035685+0000", 2023-11-25T04:23:32.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3677999999999994e-05 2023-11-25T04:23:32.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:32.614 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035827+0000", 2023-11-25T04:23:32.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014232599999999999 2023-11-25T04:23:32.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:32.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038725+0000", 2023-11-25T04:23:32.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028982779999999998 2023-11-25T04:23:32.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:32.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045323+0000", 2023-11-25T04:23:32.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0065980780000000003 2023-11-25T04:23:32.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:32.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045354+0000", 2023-11-25T04:23:32.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0463e-05 2023-11-25T04:23:32.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:32.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045381+0000", 2023-11-25T04:23:32.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7654999999999999e-05 2023-11-25T04:23:32.616 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:32.617 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:32.617 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:32.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.617 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:22 3.e 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:32.617 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:32.617 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 45.522514911999998, 2023-11-25T04:23:32.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010133422, 2023-11-25T04:23:32.617 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:32.617 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:32.617 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:32.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.617 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:32.618 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:32.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:32.618 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.618 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:32.618 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:32.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:32.618 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.618 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:32.618 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034135+0000", 2023-11-25T04:23:32.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-07 2023-11-25T04:23:32.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:32.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034138+0000", 2023-11-25T04:23:32.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6009999999999999e-06 2023-11-25T04:23:32.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:32.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034145+0000", 2023-11-25T04:23:32.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1169999999999998e-06 2023-11-25T04:23:32.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:32.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034150+0000", 2023-11-25T04:23:32.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9080000000000003e-06 2023-11-25T04:23:32.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:32.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034209+0000", 2023-11-25T04:23:32.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8149000000000002e-05 2023-11-25T04:23:32.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:32.621 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034415+0000", 2023-11-25T04:23:32.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020637699999999999 2023-11-25T04:23:32.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:32.621 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034584+0000", 2023-11-25T04:23:32.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016870399999999999 2023-11-25T04:23:32.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:32.621 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.036979+0000", 2023-11-25T04:23:32.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023957509999999998 2023-11-25T04:23:32.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.622 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:32.622 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044161+0000", 2023-11-25T04:23:32.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0071811590000000003 2023-11-25T04:23:32.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.622 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:32.622 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044224+0000", 2023-11-25T04:23:32.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3454000000000006e-05 2023-11-25T04:23:32.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.623 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:32.623 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044268+0000", 2023-11-25T04:23:32.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3761999999999999e-05 2023-11-25T04:23:32.623 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:32.623 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:32.623 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:32.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.623 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:23 3.e 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:32.623 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:32.623 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 45.522483502999997, 2023-11-25T04:23:32.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010281676, 2023-11-25T04:23:32.623 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:32.624 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:32.624 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:32.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:32.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:32.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:32.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:32.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:32.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:32.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:32.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:32.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5799999999999999e-07 2023-11-25T04:23:32.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:32.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034169+0000", 2023-11-25T04:23:32.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1879999999999998e-06 2023-11-25T04:23:32.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:32.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034177+0000", 2023-11-25T04:23:32.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7020000000000001e-06 2023-11-25T04:23:32.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:32.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034184+0000", 2023-11-25T04:23:32.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3910000000000003e-06 2023-11-25T04:23:32.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:32.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034924+0000", 2023-11-25T04:23:32.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00074032999999999998 2023-11-25T04:23:32.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:32.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034991+0000", 2023-11-25T04:23:32.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7347000000000004e-05 2023-11-25T04:23:32.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.627 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:32.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035096+0000", 2023-11-25T04:23:32.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010488 2023-11-25T04:23:32.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.627 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:32.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037588+0000", 2023-11-25T04:23:32.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024918309999999999 2023-11-25T04:23:32.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:32.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044367+0000", 2023-11-25T04:23:32.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067787589999999997 2023-11-25T04:23:32.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:32.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044415+0000", 2023-11-25T04:23:32.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7803000000000003e-05 2023-11-25T04:23:32.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:32.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044447+0000", 2023-11-25T04:23:32.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2886999999999999e-05 2023-11-25T04:23:32.629 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:32.629 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:32.629 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:32.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.629 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:30 3.15 3:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:32.629 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:32.629 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 45.522425063, 2023-11-25T04:23:32.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010365070000000001, 2023-11-25T04:23:32.629 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:32.630 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:32.630 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:32.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:32.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:32.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:32.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.631 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:32.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:32.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:32.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.631 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:32.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034225+0000", 2023-11-25T04:23:32.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7400000000000002e-07 2023-11-25T04:23:32.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.631 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:32.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034229+0000", 2023-11-25T04:23:32.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3100000000000002e-06 2023-11-25T04:23:32.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:32.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034239+0000", 2023-11-25T04:23:32.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9140000000000003e-06 2023-11-25T04:23:32.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:32.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034246+0000", 2023-11-25T04:23:32.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0149999999999998e-06 2023-11-25T04:23:32.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:32.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035572+0000", 2023-11-25T04:23:32.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001325655 2023-11-25T04:23:32.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:32.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035730+0000", 2023-11-25T04:23:32.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015827899999999999 2023-11-25T04:23:32.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:32.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035857+0000", 2023-11-25T04:23:32.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012644699999999999 2023-11-25T04:23:32.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:32.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038737+0000", 2023-11-25T04:23:32.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002880077 2023-11-25T04:23:32.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:32.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044540+0000", 2023-11-25T04:23:32.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058034239999999997 2023-11-25T04:23:32.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:32.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044569+0000", 2023-11-25T04:23:32.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8388999999999999e-05 2023-11-25T04:23:32.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:32.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044589+0000", 2023-11-25T04:23:32.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0686000000000002e-05 2023-11-25T04:23:32.635 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:32.635 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:32.635 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:32.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.636 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:33 3.1b 3:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:32.636 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:32.636 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 45.522379565000001, 2023-11-25T04:23:32.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099754049999999997, 2023-11-25T04:23:32.636 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:32.636 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:23:32.636 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:32.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:32.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:32.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:32.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:32.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:32.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:32.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:32.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:32.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4799999999999998e-07 2023-11-25T04:23:32.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:32.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034273+0000", 2023-11-25T04:23:32.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7819999999999998e-06 2023-11-25T04:23:32.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:32.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034285+0000", 2023-11-25T04:23:32.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2058e-05 2023-11-25T04:23:32.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:32.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034291+0000", 2023-11-25T04:23:32.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0680000000000002e-06 2023-11-25T04:23:32.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:32.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034370+0000", 2023-11-25T04:23:32.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8452e-05 2023-11-25T04:23:32.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:32.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034477+0000", 2023-11-25T04:23:32.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107471 2023-11-25T04:23:32.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:32.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044245+0000", 2023-11-25T04:23:32.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0097679259999999997 2023-11-25T04:23:32.640 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:32.640 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:32.640 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:32.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.640 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:36 3.1a 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:32.640 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:32.640 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 45.522332892000001, 2023-11-25T04:23:32.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058179099999999999, 2023-11-25T04:23:32.641 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:32.641 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:23:32.641 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:32.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:32.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:32.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:32.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:32.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:32.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:32.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:32.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034317+0000", 2023-11-25T04:23:32.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1999999999999999e-07 2023-11-25T04:23:32.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:32.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034321+0000", 2023-11-25T04:23:32.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.233e-06 2023-11-25T04:23:32.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:32.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034337+0000", 2023-11-25T04:23:32.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5308999999999999e-05 2023-11-25T04:23:32.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:32.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034354+0000", 2023-11-25T04:23:32.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7411999999999999e-05 2023-11-25T04:23:32.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:32.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034465+0000", 2023-11-25T04:23:32.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011154699999999999 2023-11-25T04:23:32.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.644 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:32.644 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034560+0000", 2023-11-25T04:23:32.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4186999999999998e-05 2023-11-25T04:23:32.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.644 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:32.644 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.040134+0000", 2023-11-25T04:23:32.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055746019999999997 2023-11-25T04:23:32.644 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:32.644 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:32.645 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:32.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.645 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:9 3.10 e18/16 3:09401a81:::601.00000000:head v 18'4, mlcod=0'0)", 2023-11-25T04:23:32.645 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:32.645 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 45.513438583999999, 2023-11-25T04:23:32.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061641020000000003, 2023-11-25T04:23:32.645 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:32.645 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:32.645 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:32.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:32.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:32.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:32.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:32.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:32.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:32.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:32.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:32.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5300000000000001e-07 2023-11-25T04:23:32.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.647 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:32.647 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043222+0000", 2023-11-25T04:23:32.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.082e-05 2023-11-25T04:23:32.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.647 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:32.647 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043228+0000", 2023-11-25T04:23:32.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6699999999999997e-06 2023-11-25T04:23:32.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.648 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:32.648 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043236+0000", 2023-11-25T04:23:32.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8189999999999992e-06 2023-11-25T04:23:32.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.648 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:32.648 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043507+0000", 2023-11-25T04:23:32.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000271012 2023-11-25T04:23:32.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.648 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:32.648 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043530+0000", 2023-11-25T04:23:32.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2659000000000001e-05 2023-11-25T04:23:32.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.649 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:32.649 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049321+0000", 2023-11-25T04:23:32.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057915129999999999 2023-11-25T04:23:32.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.649 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:32.649 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049375+0000", 2023-11-25T04:23:32.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3455999999999998e-05 2023-11-25T04:23:32.649 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:32.649 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:32.650 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:32.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.650 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:1 3.14 e18/16 3:292cf221:::200.00000000:head v 18'3, mlcod=0'0)", 2023-11-25T04:23:32.650 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:32.650 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 45.513401870999999, 2023-11-25T04:23:32.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0060983599999999997, 2023-11-25T04:23:32.650 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:32.650 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:32.650 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:32.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:32.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:32.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:32.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:32.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:32.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:32.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:32.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043248+0000", 2023-11-25T04:23:32.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9399999999999995e-07 2023-11-25T04:23:32.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.652 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.652 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:32.652 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043252+0000", 2023-11-25T04:23:32.652 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1799999999999998e-06 2023-11-25T04:23:32.652 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.652 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.652 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:32.652 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043259+0000", 2023-11-25T04:23:32.652 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0049999999999998e-06 2023-11-25T04:23:32.652 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.652 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.652 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:32.652 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043265+0000", 2023-11-25T04:23:32.653 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0859999999999998e-06 2023-11-25T04:23:32.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.653 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:32.653 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043311+0000", 2023-11-25T04:23:32.653 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6082999999999999e-05 2023-11-25T04:23:32.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.653 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:32.653 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043330+0000", 2023-11-25T04:23:32.653 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8556e-05 2023-11-25T04:23:32.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:32.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049303+0000", 2023-11-25T04:23:32.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059733470000000004 2023-11-25T04:23:32.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:32.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049346+0000", 2023-11-25T04:23:32.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2608999999999998e-05 2023-11-25T04:23:32.654 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:32.654 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:32.654 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:32.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.655 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:3 3.14 e18/16 3:287034c4:::1.00000000.inode:head v 18'4, mlcod=0'0)", 2023-11-25T04:23:32.655 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:32.655 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 45.512885365000002, 2023-11-25T04:23:32.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057040090000000003, 2023-11-25T04:23:32.655 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:32.655 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:32.655 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:32.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:32.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:32.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:32.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.656 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:32.656 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:32.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:32.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.656 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:32.656 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:32.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9799999999999999e-07 2023-11-25T04:23:32.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.656 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:32.656 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043785+0000", 2023-11-25T04:23:32.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0721999999999999e-05 2023-11-25T04:23:32.657 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.657 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.657 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:32.657 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043793+0000", 2023-11-25T04:23:32.657 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5839999999999995e-06 2023-11-25T04:23:32.657 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.657 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.657 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:32.657 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043803+0000", 2023-11-25T04:23:32.657 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0729999999999994e-06 2023-11-25T04:23:32.657 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.657 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.657 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:32.658 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043843+0000", 2023-11-25T04:23:32.658 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0747999999999997e-05 2023-11-25T04:23:32.658 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.658 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.658 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:32.658 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043856+0000", 2023-11-25T04:23:32.658 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2697000000000001e-05 2023-11-25T04:23:32.658 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.658 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.658 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:32.658 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049426+0000", 2023-11-25T04:23:32.658 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055704409999999998 2023-11-25T04:23:32.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.659 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.659 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:32.659 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049468+0000", 2023-11-25T04:23:32.659 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1446000000000003e-05 2023-11-25T04:23:32.659 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:32.659 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:32.659 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:32.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.659 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.659 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "pg_update_log_missing(3.14 epoch 18/16 rep_tid 171 entries 18'5 (0'0) error 3:2e2fa760:::200.00000001:head by mds.0.3:32 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:23:32.659 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:32.659 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 45.512724402000003, 2023-11-25T04:23:32.660 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005556938, 2023-11-25T04:23:32.660 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:32.660 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2023-11-25T04:23:32.660 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:32.660 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.660 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:32.660 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:32.660 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:32.660 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.660 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.660 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:32.660 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:32.661 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:32.661 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.661 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.661 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:32.661 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:32.661 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.03e-07 2023-11-25T04:23:32.661 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.661 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.661 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:32.661 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043931+0000", 2023-11-25T04:23:32.661 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1079999999999998e-06 2023-11-25T04:23:32.661 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.661 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.662 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:32.662 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043944+0000", 2023-11-25T04:23:32.662 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2852e-05 2023-11-25T04:23:32.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.662 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.662 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:32.662 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043955+0000", 2023-11-25T04:23:32.662 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1037e-05 2023-11-25T04:23:32.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.662 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.662 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:32.662 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044088+0000", 2023-11-25T04:23:32.662 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013269599999999999 2023-11-25T04:23:32.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:32.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049482+0000", 2023-11-25T04:23:32.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053940419999999999 2023-11-25T04:23:32.663 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:32.663 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:32.663 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:32.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.663 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:10 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'5, mlcod=0'0)", 2023-11-25T04:23:32.663 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:32.663 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 45.512634374000001, 2023-11-25T04:23:32.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054493099999999997, 2023-11-25T04:23:32.664 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:32.664 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:32.664 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:32.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:32.664 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:32.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:32.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:32.664 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:32.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:32.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.665 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.665 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:32.665 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:32.665 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6100000000000002e-07 2023-11-25T04:23:32.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.665 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.665 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:32.665 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044019+0000", 2023-11-25T04:23:32.665 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.839e-06 2023-11-25T04:23:32.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.665 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.665 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:32.666 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044026+0000", 2023-11-25T04:23:32.666 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5869999999999998e-06 2023-11-25T04:23:32.666 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.666 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.666 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:32.666 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044031+0000", 2023-11-25T04:23:32.666 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4909999999999996e-06 2023-11-25T04:23:32.666 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.666 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.666 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:32.666 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044070+0000", 2023-11-25T04:23:32.666 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8829999999999999e-05 2023-11-25T04:23:32.666 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.667 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:32.667 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044081+0000", 2023-11-25T04:23:32.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1447999999999999e-05 2023-11-25T04:23:32.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.667 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:32.667 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049406+0000", 2023-11-25T04:23:32.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053246480000000004 2023-11-25T04:23:32.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.667 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:32.667 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049464+0000", 2023-11-25T04:23:32.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8205999999999998e-05 2023-11-25T04:23:32.668 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:32.668 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:32.668 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:32.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.668 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:11 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'6, mlcod=0'0)", 2023-11-25T04:23:32.668 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:32.668 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 45.512433110000003, 2023-11-25T04:23:32.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052673499999999996, 2023-11-25T04:23:32.668 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:32.668 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:32.668 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:32.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.669 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:32.669 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:32.669 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:32.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.669 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:32.669 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:32.669 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:32.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.669 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:32.669 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:32.669 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0199999999999998e-07 2023-11-25T04:23:32.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.670 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:32.670 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044220+0000", 2023-11-25T04:23:32.670 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7720000000000001e-06 2023-11-25T04:23:32.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.670 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:32.670 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044226+0000", 2023-11-25T04:23:32.670 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8579999999999996e-06 2023-11-25T04:23:32.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.670 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:32.670 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044232+0000", 2023-11-25T04:23:32.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1700000000000002e-06 2023-11-25T04:23:32.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:32.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044274+0000", 2023-11-25T04:23:32.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1254999999999997e-05 2023-11-25T04:23:32.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:32.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:23:32.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6880999999999999e-05 2023-11-25T04:23:32.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.672 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:32.672 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049466+0000", 2023-11-25T04:23:32.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051659219999999999 2023-11-25T04:23:32.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:32.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:32.672 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:32.672 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049483+0000", 2023-11-25T04:23:32.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.719e-05 2023-11-25T04:23:32.672 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:32.672 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:32.673 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:32.673 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:32.673 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:32.673 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:32.674 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:23:32.675 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:32.672+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:32.775 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:32.773+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:32.785 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:32.785 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:23:32.785 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2023-11-25T04:23:32.785 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:32.794 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:23:32.879 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:32.878+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:32.939 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:32.939 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:23:32.939 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2023-11-25T04:23:32.939 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2023-11-25T04:23:32.939 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:32.949 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:23:32.990 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:32.988+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:33.078 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:33.077+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:33.095 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:33.095 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2023-11-25T04:23:33.095 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2023-11-25T04:23:33.095 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2023-11-25T04:23:33.096 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.096 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:33.096 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:33.096 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 46.061208016999998, 2023-11-25T04:23:33.096 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01072943, 2023-11-25T04:23:33.096 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:33.096 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:33.096 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:33.096 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.096 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:33.096 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:33.096 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:33.097 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.097 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.097 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:33.097 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:33.097 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:33.097 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.097 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.097 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:33.097 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031829+0000", 2023-11-25T04:23:33.097 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.666e-06 2023-11-25T04:23:33.097 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.097 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.098 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:33.098 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031839+0000", 2023-11-25T04:23:33.098 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9490000000000008e-06 2023-11-25T04:23:33.098 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.098 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.098 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:33.098 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031857+0000", 2023-11-25T04:23:33.098 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8093000000000001e-05 2023-11-25T04:23:33.098 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.098 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.098 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:33.098 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031882+0000", 2023-11-25T04:23:33.098 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4294000000000002e-05 2023-11-25T04:23:33.099 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.099 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.099 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:33.099 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033130+0000", 2023-11-25T04:23:33.099 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001248264 2023-11-25T04:23:33.099 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.099 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.099 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:33.099 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033426+0000", 2023-11-25T04:23:33.099 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029620999999999998 2023-11-25T04:23:33.099 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.099 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.100 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:33.100 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033814+0000", 2023-11-25T04:23:33.100 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000387742 2023-11-25T04:23:33.100 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.100 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.100 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:33.100 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036793+0000", 2023-11-25T04:23:33.100 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002979598 2023-11-25T04:23:33.100 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.100 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.100 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:33.100 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042455+0000", 2023-11-25T04:23:33.100 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0056619260000000003 2023-11-25T04:23:33.101 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.101 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.101 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:33.101 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042502+0000", 2023-11-25T04:23:33.101 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7139999999999999e-05 2023-11-25T04:23:33.101 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.101 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.101 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:33.101 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042557+0000", 2023-11-25T04:23:33.101 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4548e-05 2023-11-25T04:23:33.101 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:33.101 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:33.102 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:33.102 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.102 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.102 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:33.102 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:33.102 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 46.060009481999998, 2023-11-25T04:23:33.102 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011505202000000001, 2023-11-25T04:23:33.102 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:33.102 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:33.102 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:33.102 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.102 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:33.102 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:33.103 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:33.103 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.103 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.103 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:33.103 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:33.103 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:33.103 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.103 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.103 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:33.103 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033027+0000", 2023-11-25T04:23:33.103 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0699999999999999e-06 2023-11-25T04:23:33.103 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.104 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.104 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:33.104 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033035+0000", 2023-11-25T04:23:33.104 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0830000000000004e-06 2023-11-25T04:23:33.104 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.104 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.104 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:33.104 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033059+0000", 2023-11-25T04:23:33.104 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3975e-05 2023-11-25T04:23:33.104 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.104 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.104 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:33.105 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033482+0000", 2023-11-25T04:23:33.105 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042253700000000001 2023-11-25T04:23:33.105 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.105 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.105 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:33.105 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034520+0000", 2023-11-25T04:23:33.105 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001037991 2023-11-25T04:23:33.105 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.105 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.105 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:33.105 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034611+0000", 2023-11-25T04:23:33.105 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1113999999999995e-05 2023-11-25T04:23:33.105 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.106 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.106 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:33.106 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034769+0000", 2023-11-25T04:23:33.106 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000157641 2023-11-25T04:23:33.106 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.106 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.106 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:33.106 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038292+0000", 2023-11-25T04:23:33.106 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035231160000000002 2023-11-25T04:23:33.107 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.107 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.107 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:33.107 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044440+0000", 2023-11-25T04:23:33.107 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0061486980000000002 2023-11-25T04:23:33.107 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.107 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.107 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:33.107 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044495+0000", 2023-11-25T04:23:33.107 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4339000000000003e-05 2023-11-25T04:23:33.107 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.107 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.107 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:33.108 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044531+0000", 2023-11-25T04:23:33.108 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6637999999999997e-05 2023-11-25T04:23:33.108 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:33.108 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:33.108 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:33.108 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.108 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.108 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:33.108 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:33.108 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 46.059535128, 2023-11-25T04:23:33.108 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0057634039999999997, 2023-11-25T04:23:33.108 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:33.109 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:33.109 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:33.109 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.109 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:33.109 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:33.109 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:33.109 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.109 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.109 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:33.109 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:33.109 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:33.109 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.109 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.110 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:33.110 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:33.110 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7900000000000007e-07 2023-11-25T04:23:33.110 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.110 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.110 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:33.110 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033507+0000", 2023-11-25T04:23:33.110 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8900000000000004e-06 2023-11-25T04:23:33.110 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.110 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.110 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:33.110 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033519+0000", 2023-11-25T04:23:33.111 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1756000000000001e-05 2023-11-25T04:23:33.111 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.111 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.111 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:33.111 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033527+0000", 2023-11-25T04:23:33.111 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3610000000000001e-06 2023-11-25T04:23:33.111 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.111 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.111 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:33.111 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034451+0000", 2023-11-25T04:23:33.111 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00092372400000000001 2023-11-25T04:23:33.111 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.112 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.112 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:33.112 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034809+0000", 2023-11-25T04:23:33.112 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00035757699999999998 2023-11-25T04:23:33.112 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.112 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.112 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:23:33.112 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034965+0000", 2023-11-25T04:23:33.112 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015638699999999999 2023-11-25T04:23:33.112 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.112 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.112 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:33.113 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038299+0000", 2023-11-25T04:23:33.113 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0033336559999999999 2023-11-25T04:23:33.113 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.113 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.113 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:33.113 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039169+0000", 2023-11-25T04:23:33.113 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00087066400000000005 2023-11-25T04:23:33.113 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.113 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.113 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:33.113 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039226+0000", 2023-11-25T04:23:33.113 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6511999999999997e-05 2023-11-25T04:23:33.113 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.114 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.114 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:33.114 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039264+0000", 2023-11-25T04:23:33.114 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7898000000000002e-05 2023-11-25T04:23:33.114 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:33.114 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:33.114 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:33.114 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.114 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.114 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:25 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:33.114 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:33.114 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 46.059456544, 2023-11-25T04:23:33.114 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096656169999999996, 2023-11-25T04:23:33.115 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:33.115 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:33.115 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:33.115 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.115 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:33.115 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:33.115 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:33.115 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.115 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.115 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:33.115 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:33.115 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:33.115 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.116 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.116 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:33.116 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033580+0000", 2023-11-25T04:23:33.116 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8199999999999999e-07 2023-11-25T04:23:33.116 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.116 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.116 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:33.116 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033585+0000", 2023-11-25T04:23:33.116 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3680000000000001e-06 2023-11-25T04:23:33.116 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.116 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.116 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:33.116 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033600+0000", 2023-11-25T04:23:33.117 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4331000000000001e-05 2023-11-25T04:23:33.117 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.117 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.117 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:33.117 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033606+0000", 2023-11-25T04:23:33.117 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5780000000000004e-06 2023-11-25T04:23:33.117 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.117 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.117 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:33.117 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035747+0000", 2023-11-25T04:23:33.117 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0021406530000000002 2023-11-25T04:23:33.117 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.117 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.118 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:33.118 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035901+0000", 2023-11-25T04:23:33.118 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000153931 2023-11-25T04:23:33.118 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.118 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.118 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:23:33.118 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036193+0000", 2023-11-25T04:23:33.118 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029211299999999999 2023-11-25T04:23:33.118 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.118 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.118 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:33.118 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038655+0000", 2023-11-25T04:23:33.119 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0024621640000000002 2023-11-25T04:23:33.119 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.119 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.119 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:33.119 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043130+0000", 2023-11-25T04:23:33.119 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.004474698 2023-11-25T04:23:33.119 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.119 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.119 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:33.119 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043204+0000", 2023-11-25T04:23:33.119 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4091999999999995e-05 2023-11-25T04:23:33.119 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.119 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.120 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:33.120 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043245+0000", 2023-11-25T04:23:33.120 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0907e-05 2023-11-25T04:23:33.120 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:33.120 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:33.120 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:33.120 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.120 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.120 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:26 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:33.120 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:33.120 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 46.059420719999999, 2023-11-25T04:23:33.120 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011086945000000001, 2023-11-25T04:23:33.120 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:33.121 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:33.121 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:33.121 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.121 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:33.121 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:33.121 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:33.121 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.121 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.122 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:33.122 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:33.122 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:33.122 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.122 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.122 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:33.122 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:33.122 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1699999999999998e-07 2023-11-25T04:23:33.122 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.122 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.122 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:33.122 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033627+0000", 2023-11-25T04:23:33.123 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.163e-05 2023-11-25T04:23:33.123 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.123 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.123 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:33.123 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033636+0000", 2023-11-25T04:23:33.123 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6859999999999995e-06 2023-11-25T04:23:33.123 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.123 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.123 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:33.123 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033642+0000", 2023-11-25T04:23:33.123 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3149999999999997e-06 2023-11-25T04:23:33.123 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.123 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.124 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:33.124 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035414+0000", 2023-11-25T04:23:33.124 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0017716380000000001 2023-11-25T04:23:33.124 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.124 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.124 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:33.124 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035539+0000", 2023-11-25T04:23:33.124 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000125477 2023-11-25T04:23:33.124 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.124 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.124 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:33.124 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035672+0000", 2023-11-25T04:23:33.125 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000133316 2023-11-25T04:23:33.125 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.125 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.125 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:33.125 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038649+0000", 2023-11-25T04:23:33.125 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029764679999999999 2023-11-25T04:23:33.125 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.125 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.125 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:33.125 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044578+0000", 2023-11-25T04:23:33.125 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0059293510000000002 2023-11-25T04:23:33.125 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.126 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.126 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:33.126 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044651+0000", 2023-11-25T04:23:33.126 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2513000000000002e-05 2023-11-25T04:23:33.126 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.126 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.126 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:33.126 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044702+0000", 2023-11-25T04:23:33.126 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1034e-05 2023-11-25T04:23:33.126 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:33.126 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:33.126 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:33.127 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.127 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.127 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:29 3.b 3:d0630e4c:::mds0_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:33.127 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:33.127 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 46.059382800000002, 2023-11-25T04:23:33.127 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010980227, 2023-11-25T04:23:33.127 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:33.127 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:33.127 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:33.127 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.127 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:33.127 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:33.128 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:33.128 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.128 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.128 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:33.128 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:33.128 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:33.128 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.128 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.128 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:33.128 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:33.128 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3499999999999996e-07 2023-11-25T04:23:33.128 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.128 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.129 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:33.129 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033658+0000", 2023-11-25T04:23:33.129 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1280000000000001e-06 2023-11-25T04:23:33.129 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.129 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.129 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:33.129 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033667+0000", 2023-11-25T04:23:33.129 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1430000000000004e-06 2023-11-25T04:23:33.129 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.129 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.129 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:33.129 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033678+0000", 2023-11-25T04:23:33.130 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0906e-05 2023-11-25T04:23:33.130 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.130 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.130 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:33.130 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034456+0000", 2023-11-25T04:23:33.130 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00077833899999999996 2023-11-25T04:23:33.130 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.130 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.130 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:33.130 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034575+0000", 2023-11-25T04:23:33.130 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011926000000000001 2023-11-25T04:23:33.130 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.131 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.131 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:33.131 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035001+0000", 2023-11-25T04:23:33.131 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042550999999999998 2023-11-25T04:23:33.131 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.131 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.131 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:33.131 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.037923+0000", 2023-11-25T04:23:33.131 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029219670000000001 2023-11-25T04:23:33.131 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.131 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.131 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:33.131 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044538+0000", 2023-11-25T04:23:33.132 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0066152349999999997 2023-11-25T04:23:33.132 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.132 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.132 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:33.132 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044594+0000", 2023-11-25T04:23:33.132 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5593000000000001e-05 2023-11-25T04:23:33.132 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.132 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.132 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:33.132 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044633+0000", 2023-11-25T04:23:33.132 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9511000000000002e-05 2023-11-25T04:23:33.132 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:33.132 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:33.133 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:33.133 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.133 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.133 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:34 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:33.133 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:33.133 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 46.059132687000002, 2023-11-25T04:23:33.133 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010920445000000001, 2023-11-25T04:23:33.133 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:33.133 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:33.133 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:33.133 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.133 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:33.134 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:33.134 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:33.134 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.134 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.134 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:33.134 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:33.134 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:33.134 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.134 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.134 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:33.134 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033904+0000", 2023-11-25T04:23:33.134 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6599999999999994e-07 2023-11-25T04:23:33.134 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.135 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.135 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:33.135 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033909+0000", 2023-11-25T04:23:33.135 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5639999999999996e-06 2023-11-25T04:23:33.135 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.135 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.135 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:33.135 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033923+0000", 2023-11-25T04:23:33.135 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3028e-05 2023-11-25T04:23:33.135 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.135 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.135 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:33.136 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034043+0000", 2023-11-25T04:23:33.136 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000120328 2023-11-25T04:23:33.136 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.136 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.136 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:33.136 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035576+0000", 2023-11-25T04:23:33.136 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015333720000000001 2023-11-25T04:23:33.136 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.136 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.136 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:33.136 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035699+0000", 2023-11-25T04:23:33.136 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000123001 2023-11-25T04:23:33.137 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.137 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.137 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:33.137 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044823+0000", 2023-11-25T04:23:33.137 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0091241859999999994 2023-11-25T04:23:33.137 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:33.137 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:33.137 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:33.137 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.137 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.137 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:35 3.1f 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:33.137 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:33.138 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 46.057828481999998, 2023-11-25T04:23:33.138 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096164779999999995, 2023-11-25T04:23:33.138 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:33.138 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:33.138 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:33.138 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.138 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:33.138 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:33.138 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:33.138 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.138 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.138 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:33.138 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:33.139 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:33.139 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.139 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.139 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:33.139 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035208+0000", 2023-11-25T04:23:33.139 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2699999999999998e-07 2023-11-25T04:23:33.139 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.139 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.139 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:33.139 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035213+0000", 2023-11-25T04:23:33.139 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.6070000000000001e-06 2023-11-25T04:23:33.139 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.140 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.140 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:33.140 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035350+0000", 2023-11-25T04:23:33.140 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013693599999999999 2023-11-25T04:23:33.140 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.140 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.140 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:33.140 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035358+0000", 2023-11-25T04:23:33.140 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0099999999999995e-06 2023-11-25T04:23:33.140 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.140 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.140 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:33.140 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036260+0000", 2023-11-25T04:23:33.141 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00090240100000000005 2023-11-25T04:23:33.141 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.141 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.141 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:33.141 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036517+0000", 2023-11-25T04:23:33.141 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00025736299999999998 2023-11-25T04:23:33.141 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.141 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.141 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:33.141 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044824+0000", 2023-11-25T04:23:33.141 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0083063340000000003 2023-11-25T04:23:33.141 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:33.142 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:33.142 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:33.142 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.142 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.142 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:37 3.1e 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:33.142 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:33.142 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 46.057661795999998, 2023-11-25T04:23:33.142 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0078322640000000002, 2023-11-25T04:23:33.142 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:33.143 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:33.143 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:33.143 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.143 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:33.143 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:33.143 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:33.143 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.143 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.143 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:33.143 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:33.143 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:33.144 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.144 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.144 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:33.144 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035375+0000", 2023-11-25T04:23:33.144 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9100000000000003e-07 2023-11-25T04:23:33.144 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.144 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.144 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:33.144 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035379+0000", 2023-11-25T04:23:33.144 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9430000000000002e-06 2023-11-25T04:23:33.144 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.144 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.145 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:33.145 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035388+0000", 2023-11-25T04:23:33.145 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2760000000000006e-06 2023-11-25T04:23:33.145 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.145 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.145 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:33.145 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035396+0000", 2023-11-25T04:23:33.145 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8480000000000001e-06 2023-11-25T04:23:33.145 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.145 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.145 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:33.145 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035452+0000", 2023-11-25T04:23:33.146 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5831000000000001e-05 2023-11-25T04:23:33.146 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.146 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.146 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:33.146 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035934+0000", 2023-11-25T04:23:33.146 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00048265599999999998 2023-11-25T04:23:33.146 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.146 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.146 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:33.146 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043206+0000", 2023-11-25T04:23:33.146 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.007271919 2023-11-25T04:23:33.146 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:33.147 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:33.147 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:33.147 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.147 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.147 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing(3.1a epoch 18/16 rep_tid 3 entries 18'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:36 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:23:33.147 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:33.147 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 46.056386209999999, 2023-11-25T04:23:33.147 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002073414, 2023-11-25T04:23:33.147 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:33.147 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:33.147 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:33.148 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.148 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:33.148 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:33.148 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:33.148 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.148 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.148 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:33.148 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:33.148 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:33.148 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.148 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.148 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:33.149 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036650+0000", 2023-11-25T04:23:33.149 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7300000000000002e-07 2023-11-25T04:23:33.149 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.149 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.149 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:33.149 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036659+0000", 2023-11-25T04:23:33.149 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.7499999999999992e-06 2023-11-25T04:23:33.149 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.149 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.149 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:33.149 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036673+0000", 2023-11-25T04:23:33.149 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4917e-05 2023-11-25T04:23:33.150 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.150 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.150 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:33.150 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036687+0000", 2023-11-25T04:23:33.150 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3883000000000001e-05 2023-11-25T04:23:33.150 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.150 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.150 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:33.150 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036781+0000", 2023-11-25T04:23:33.150 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3219000000000003e-05 2023-11-25T04:23:33.150 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.151 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.151 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:33.151 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038723+0000", 2023-11-25T04:23:33.151 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019422720000000001 2023-11-25T04:23:33.151 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:33.151 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:33.151 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:33.151 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.151 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.151 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:28 3.13 e18/16 3:cd5a64a3:::100.00000000:head v 18'1, mlcod=0'0)", 2023-11-25T04:23:33.151 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:33.151 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 46.052725052, 2023-11-25T04:23:33.152 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001850546, 2023-11-25T04:23:33.152 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:33.152 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:33.152 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:33.152 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.152 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:33.152 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:33.152 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:33.152 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.152 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.152 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:33.152 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:33.152 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:33.153 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.153 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.153 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:33.153 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:33.153 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7800000000000002e-07 2023-11-25T04:23:33.153 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.153 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.153 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:33.153 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040331+0000", 2023-11-25T04:23:33.153 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0449999999999999e-05 2023-11-25T04:23:33.153 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.153 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.154 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:33.154 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040339+0000", 2023-11-25T04:23:33.154 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0709999999999995e-06 2023-11-25T04:23:33.154 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.154 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.154 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:33.154 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040347+0000", 2023-11-25T04:23:33.154 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7079999999999997e-06 2023-11-25T04:23:33.154 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.154 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.154 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:33.154 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040410+0000", 2023-11-25T04:23:33.155 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2453000000000007e-05 2023-11-25T04:23:33.155 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.155 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.155 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:33.155 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040422+0000", 2023-11-25T04:23:33.155 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2429000000000001e-05 2023-11-25T04:23:33.155 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.155 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.155 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:33.155 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042129+0000", 2023-11-25T04:23:33.155 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001707266 2023-11-25T04:23:33.155 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.156 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.156 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:33.156 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042161+0000", 2023-11-25T04:23:33.156 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1791000000000001e-05 2023-11-25T04:23:33.156 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:33.156 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:33.156 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:33.156 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.156 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.156 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:27 3.13 e18/16 3:c91d4a1d:::mds0_inotable:head v 18'2, mlcod=0'0)", 2023-11-25T04:23:33.156 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:33.156 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 46.052521529000003, 2023-11-25T04:23:33.157 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0016886430000000001, 2023-11-25T04:23:33.157 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:33.157 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:33.157 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:33.157 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.157 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:33.157 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:33.157 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:33.157 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.157 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.157 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:33.157 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:33.158 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:33.158 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.158 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.158 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:33.158 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:33.158 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3999999999999997e-07 2023-11-25T04:23:33.158 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.158 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.158 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:33.158 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040520+0000", 2023-11-25T04:23:33.158 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2989999999999996e-06 2023-11-25T04:23:33.158 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.159 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.159 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:33.159 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040527+0000", 2023-11-25T04:23:33.159 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5730000000000002e-06 2023-11-25T04:23:33.159 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.159 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.159 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:33.159 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040535+0000", 2023-11-25T04:23:33.159 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8630000000000008e-06 2023-11-25T04:23:33.159 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.159 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.159 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:33.160 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040683+0000", 2023-11-25T04:23:33.160 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000148002 2023-11-25T04:23:33.160 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.160 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.160 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:33.160 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040700+0000", 2023-11-25T04:23:33.160 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7065e-05 2023-11-25T04:23:33.160 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.160 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.160 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:33.160 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042163+0000", 2023-11-25T04:23:33.160 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001462767 2023-11-25T04:23:33.161 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.161 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.161 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:33.161 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042203+0000", 2023-11-25T04:23:33.161 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9734000000000003e-05 2023-11-25T04:23:33.161 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:33.161 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:33.161 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:33.161 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.161 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.161 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:20 3.1c e18/16 3:3ed09add:::607.00000000:head v 18'1, mlcod=0'0)", 2023-11-25T04:23:33.161 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:33.162 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 46.052486488, 2023-11-25T04:23:33.162 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015271, 2023-11-25T04:23:33.162 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:33.162 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:33.162 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:33.162 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.162 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:33.162 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:33.162 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:33.162 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.162 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.162 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:33.163 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:33.163 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:33.163 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.163 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.163 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:33.163 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:33.163 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4799999999999999e-07 2023-11-25T04:23:33.163 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.163 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.163 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:33.163 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040561+0000", 2023-11-25T04:23:33.163 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1800000000000001e-05 2023-11-25T04:23:33.164 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.164 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.164 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:33.164 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040568+0000", 2023-11-25T04:23:33.164 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7730000000000002e-06 2023-11-25T04:23:33.164 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.164 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.164 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:33.164 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040587+0000", 2023-11-25T04:23:33.164 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8691000000000001e-05 2023-11-25T04:23:33.164 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.164 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.165 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:33.165 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040638+0000", 2023-11-25T04:23:33.165 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1068e-05 2023-11-25T04:23:33.165 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.165 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.165 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:33.165 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040670+0000", 2023-11-25T04:23:33.165 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2125999999999997e-05 2023-11-25T04:23:33.165 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.165 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.165 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:33.165 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042032+0000", 2023-11-25T04:23:33.165 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001361821 2023-11-25T04:23:33.166 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.166 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.166 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:33.166 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042076+0000", 2023-11-25T04:23:33.166 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4472999999999999e-05 2023-11-25T04:23:33.166 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:33.166 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:33.166 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:33.166 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.166 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.166 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:21 3.1c e18/16 3:3ed09add:::607.00000000:head v 18'2, mlcod=0'0)", 2023-11-25T04:23:33.166 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:33.167 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 46.052251763000001, 2023-11-25T04:23:33.167 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0013228160000000001, 2023-11-25T04:23:33.167 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:33.167 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:33.167 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:33.167 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.167 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:33.167 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:33.167 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:33.167 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.167 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.167 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:33.168 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:33.168 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:33.168 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.168 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.168 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:33.168 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:33.168 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9299999999999999e-07 2023-11-25T04:23:33.168 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.168 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.168 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:33.168 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040790+0000", 2023-11-25T04:23:33.168 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5770000000000001e-06 2023-11-25T04:23:33.169 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.169 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.169 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:33.169 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040799+0000", 2023-11-25T04:23:33.169 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6439999999999995e-06 2023-11-25T04:23:33.169 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.169 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.169 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:33.169 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040806+0000", 2023-11-25T04:23:33.169 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0180000000000004e-06 2023-11-25T04:23:33.169 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.169 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.170 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:33.170 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040907+0000", 2023-11-25T04:23:33.170 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000100096 2023-11-25T04:23:33.170 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.170 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.170 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:33.170 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040925+0000", 2023-11-25T04:23:33.170 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8536999999999999e-05 2023-11-25T04:23:33.170 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.170 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.170 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:33.170 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042079+0000", 2023-11-25T04:23:33.171 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00115358 2023-11-25T04:23:33.171 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.171 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.171 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:33.171 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042107+0000", 2023-11-25T04:23:33.171 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7971e-05 2023-11-25T04:23:33.171 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:33.171 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:33.171 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:33.171 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.171 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.171 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop_reply(mds.0.3:4 3.1f e18/16 ondisk, result = 0)", 2023-11-25T04:23:33.172 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:33.172 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 46.050912373000003, 2023-11-25T04:23:33.172 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00044092900000000001, 2023-11-25T04:23:33.172 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:33.172 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:33.172 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:33.172 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.172 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:33.172 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:33.172 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:33.172 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.172 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.173 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:33.173 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:33.173 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:33.173 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.173 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.173 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:33.173 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042124+0000", 2023-11-25T04:23:33.173 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1800000000000005e-07 2023-11-25T04:23:33.173 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.173 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.173 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:33.173 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042128+0000", 2023-11-25T04:23:33.173 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2289999999999997e-06 2023-11-25T04:23:33.174 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.174 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.174 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:33.174 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042138+0000", 2023-11-25T04:23:33.174 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9729999999999992e-06 2023-11-25T04:23:33.174 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.174 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.174 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:33.174 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042145+0000", 2023-11-25T04:23:33.174 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.336e-06 2023-11-25T04:23:33.174 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.174 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.175 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:33.175 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042441+0000", 2023-11-25T04:23:33.175 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029650500000000002 2023-11-25T04:23:33.175 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.175 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.175 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:33.175 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042448+0000", 2023-11-25T04:23:33.175 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4100000000000002e-06 2023-11-25T04:23:33.175 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.175 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.175 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:33.175 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042564+0000", 2023-11-25T04:23:33.176 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000115758 2023-11-25T04:23:33.176 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:33.176 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:33.176 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:33.176 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.176 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.176 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing_reply(3.1f epoch 18/16 rep_tid 8 lcod 18'4)", 2023-11-25T04:23:33.176 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:33.177 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 46.048586960999998, 2023-11-25T04:23:33.177 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00038154699999999998, 2023-11-25T04:23:33.177 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:33.177 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:33.177 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:33.177 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.177 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:33.177 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:33.177 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:33.177 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.177 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.177 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:33.178 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:33.178 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:33.178 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.178 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.178 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:33.178 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:33.178 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2700000000000001e-07 2023-11-25T04:23:33.178 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.178 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.178 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:33.178 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044451+0000", 2023-11-25T04:23:33.178 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7460000000000001e-06 2023-11-25T04:23:33.179 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.179 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.179 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:33.179 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044455+0000", 2023-11-25T04:23:33.179 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5809999999999998e-06 2023-11-25T04:23:33.179 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.179 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.179 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:33.179 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044458+0000", 2023-11-25T04:23:33.179 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0170000000000001e-06 2023-11-25T04:23:33.179 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.179 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.180 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:33.180 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044734+0000", 2023-11-25T04:23:33.180 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00027582799999999999 2023-11-25T04:23:33.180 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.180 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.180 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:33.180 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044830+0000", 2023-11-25T04:23:33.180 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6248000000000004e-05 2023-11-25T04:23:33.180 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:33.180 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:33.180 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:33.180 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.180 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.181 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing_reply(3.b epoch 18/16 rep_tid 5 lcod 18'2)", 2023-11-25T04:23:33.181 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:33.181 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 46.048550300000002, 2023-11-25T04:23:33.181 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000344488, 2023-11-25T04:23:33.181 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:33.181 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:33.181 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:33.181 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.181 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:33.181 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:33.181 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:33.181 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.181 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.182 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:33.182 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:33.182 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:33.182 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.182 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.182 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:33.182 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:33.182 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.55e-07 2023-11-25T04:23:33.182 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.182 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.182 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:33.182 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044487+0000", 2023-11-25T04:23:33.182 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2470000000000001e-06 2023-11-25T04:23:33.183 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.183 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.183 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:33.183 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044491+0000", 2023-11-25T04:23:33.183 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9659999999999998e-06 2023-11-25T04:23:33.183 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.183 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.183 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:33.183 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044494+0000", 2023-11-25T04:23:33.183 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2310000000000001e-06 2023-11-25T04:23:33.183 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.184 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.184 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:33.184 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044756+0000", 2023-11-25T04:23:33.184 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00026172800000000003 2023-11-25T04:23:33.184 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.184 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.184 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:33.184 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044830+0000", 2023-11-25T04:23:33.184 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4160999999999997e-05 2023-11-25T04:23:33.184 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:33.184 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:33.184 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:33.185 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.185 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.185 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:39 3.6 3:654134d2:::mds0_openfiles.0:head [omap-clear,omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2023-11-25T04:23:33.185 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:33.185 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 40.988724257999998, 2023-11-25T04:23:33.185 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001788658, 2023-11-25T04:23:33.185 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:33.185 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:33.185 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:33.185 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.185 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:33.185 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:33.186 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:33.186 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.186 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.186 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:33.186 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:33.186 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:33.186 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.186 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.186 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:33.186 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104314+0000", 2023-11-25T04:23:33.186 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3099999999999999e-06 2023-11-25T04:23:33.186 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.187 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.187 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:33.187 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104325+0000", 2023-11-25T04:23:33.187 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1511999999999999e-05 2023-11-25T04:23:33.187 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.187 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.187 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:33.187 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104362+0000", 2023-11-25T04:23:33.187 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6903000000000002e-05 2023-11-25T04:23:33.187 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.187 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.187 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:33.187 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104387+0000", 2023-11-25T04:23:33.188 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4766e-05 2023-11-25T04:23:33.188 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.188 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.188 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:33.188 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104465+0000", 2023-11-25T04:23:33.188 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8119000000000002e-05 2023-11-25T04:23:33.188 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.188 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.188 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:33.188 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104557+0000", 2023-11-25T04:23:33.188 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2496000000000003e-05 2023-11-25T04:23:33.188 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.188 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.189 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:33.189 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104646+0000", 2023-11-25T04:23:33.189 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8818000000000002e-05 2023-11-25T04:23:33.189 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.189 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.189 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:33.189 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105201+0000", 2023-11-25T04:23:33.189 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00055506000000000002 2023-11-25T04:23:33.189 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.189 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.189 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:33.189 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105994+0000", 2023-11-25T04:23:33.189 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00079294700000000001 2023-11-25T04:23:33.190 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.190 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.190 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:33.190 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.106055+0000", 2023-11-25T04:23:33.190 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0566000000000003e-05 2023-11-25T04:23:33.190 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.190 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.190 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:33.190 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.106100+0000", 2023-11-25T04:23:33.190 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5160999999999997e-05 2023-11-25T04:23:33.190 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:33.190 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:33.190 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:33.191 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.191 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.191 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:43 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2023-11-25T04:23:33.191 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:33.191 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 25.988499262000001, 2023-11-25T04:23:33.191 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001751166, 2023-11-25T04:23:33.191 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:33.191 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:33.191 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:33.191 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.191 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:33.191 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:33.191 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:33.192 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.192 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.192 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:33.192 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:33.192 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:33.192 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.192 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.192 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:33.192 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104539+0000", 2023-11-25T04:23:33.192 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9529999999999999e-06 2023-11-25T04:23:33.192 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.192 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.193 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:33.193 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104557+0000", 2023-11-25T04:23:33.193 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7464000000000001e-05 2023-11-25T04:23:33.193 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.193 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.193 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:33.193 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104590+0000", 2023-11-25T04:23:33.193 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2966000000000003e-05 2023-11-25T04:23:33.193 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.193 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.193 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:33.193 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104608+0000", 2023-11-25T04:23:33.193 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8193e-05 2023-11-25T04:23:33.193 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.194 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.194 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:33.194 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104767+0000", 2023-11-25T04:23:33.194 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015882 2023-11-25T04:23:33.194 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.194 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.194 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:33.194 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104909+0000", 2023-11-25T04:23:33.194 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014244999999999999 2023-11-25T04:23:33.194 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.194 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.194 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:33.194 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.105059+0000", 2023-11-25T04:23:33.195 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014964399999999999 2023-11-25T04:23:33.195 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.195 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.195 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:33.195 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.105701+0000", 2023-11-25T04:23:33.195 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00064201099999999995 2023-11-25T04:23:33.195 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.195 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.195 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:33.195 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106233+0000", 2023-11-25T04:23:33.195 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00053178500000000001 2023-11-25T04:23:33.195 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.195 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.196 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:33.196 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106265+0000", 2023-11-25T04:23:33.196 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1986000000000002e-05 2023-11-25T04:23:33.196 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.196 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.196 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:33.196 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106288+0000", 2023-11-25T04:23:33.196 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2894000000000001e-05 2023-11-25T04:23:33.196 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:33.196 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:33.196 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:33.196 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.196 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.197 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:45 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e23)", 2023-11-25T04:23:33.197 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:23:33.197 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 20.988301222, 2023-11-25T04:23:33.197 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00098423299999999994, 2023-11-25T04:23:33.197 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:33.197 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:33.197 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:33.197 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.197 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:33.197 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:23:33.197 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:33.197 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.197 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.197 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:33.198 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:23:33.198 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:33.198 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.198 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.198 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:33.198 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104737+0000", 2023-11-25T04:23:33.198 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3309999999999998e-06 2023-11-25T04:23:33.198 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.198 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.198 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:33.198 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104748+0000", 2023-11-25T04:23:33.198 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1080999999999999e-05 2023-11-25T04:23:33.198 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.199 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.199 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:33.199 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104771+0000", 2023-11-25T04:23:33.199 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2707000000000001e-05 2023-11-25T04:23:33.199 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.199 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.199 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:33.199 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104800+0000", 2023-11-25T04:23:33.199 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9595e-05 2023-11-25T04:23:33.199 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.199 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.199 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:33.199 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104890+0000", 2023-11-25T04:23:33.200 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.9808999999999993e-05 2023-11-25T04:23:33.200 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.200 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.200 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:33.200 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105004+0000", 2023-11-25T04:23:33.200 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000114508 2023-11-25T04:23:33.200 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.200 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.200 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:33.200 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105634+0000", 2023-11-25T04:23:33.200 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00062985799999999996 2023-11-25T04:23:33.200 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.200 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.201 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:33.201 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105671+0000", 2023-11-25T04:23:33.201 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6640000000000002e-05 2023-11-25T04:23:33.201 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:33.201 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:33.201 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:33.201 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105719+0000", 2023-11-25T04:23:33.201 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7704000000000003e-05 2023-11-25T04:23:33.201 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:33.201 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:33.201 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:33.201 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:33.201 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:33.202 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:33.203 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:23:33.203 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:33.201+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:33.304 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:33.302+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:33.310 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:33.310 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:23:33.311 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2023-11-25T04:23:33.311 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:33.319 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:23:33.405 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:33.403+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:33.464 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:33.464 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:23:33.465 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2023-11-25T04:23:33.465 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2023-11-25T04:23:33.465 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:33.473 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:23:33.514 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:33.512+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:33.606 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:33.605+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:33.619 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:33.619 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2023-11-25T04:23:33.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2023-11-25T04:23:33.620 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2023-11-25T04:23:33.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.620 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:155 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'109, mlcod=11'105)", 2023-11-25T04:23:33.620 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:33.620 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.831681056999997, 2023-11-25T04:23:33.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061719019999999999, 2023-11-25T04:23:33.621 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:33.621 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:33.621 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:33.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:33.621 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:33.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:33.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.622 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:33.622 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:33.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:33.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.623 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:33.623 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784884+0000", 2023-11-25T04:23:33.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3099999999999999e-07 2023-11-25T04:23:33.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:33.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784942+0000", 2023-11-25T04:23:33.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8570000000000003e-05 2023-11-25T04:23:33.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:33.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784949+0000", 2023-11-25T04:23:33.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0310000000000001e-06 2023-11-25T04:23:33.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:33.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784953+0000", 2023-11-25T04:23:33.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6320000000000001e-06 2023-11-25T04:23:33.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.627 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:33.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784993+0000", 2023-11-25T04:23:33.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9737000000000003e-05 2023-11-25T04:23:33.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:33.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785011+0000", 2023-11-25T04:23:33.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8349999999999999e-05 2023-11-25T04:23:33.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:33.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791024+0000", 2023-11-25T04:23:33.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006012997 2023-11-25T04:23:33.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:33.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791055+0000", 2023-11-25T04:23:33.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1254000000000003e-05 2023-11-25T04:23:33.629 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:33.629 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:33.629 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:33.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.629 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:156 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'110, mlcod=11'105)", 2023-11-25T04:23:33.629 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:33.629 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.831095007999998, 2023-11-25T04:23:33.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005616116, 2023-11-25T04:23:33.629 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:33.629 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:33.629 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:33.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:33.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:33.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:33.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:33.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:33.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:33.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:33.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785470+0000", 2023-11-25T04:23:33.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0699999999999998e-07 2023-11-25T04:23:33.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.631 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:33.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785474+0000", 2023-11-25T04:23:33.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.048e-06 2023-11-25T04:23:33.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.631 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:33.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785480+0000", 2023-11-25T04:23:33.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7729999999999996e-06 2023-11-25T04:23:33.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:33.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785483+0000", 2023-11-25T04:23:33.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1970000000000001e-06 2023-11-25T04:23:33.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:33.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787936+0000", 2023-11-25T04:23:33.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024529040000000001 2023-11-25T04:23:33.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:33.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787947+0000", 2023-11-25T04:23:33.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1669e-05 2023-11-25T04:23:33.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:33.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791057+0000", 2023-11-25T04:23:33.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031095200000000002 2023-11-25T04:23:33.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:33.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791086+0000", 2023-11-25T04:23:33.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8697999999999999e-05 2023-11-25T04:23:33.633 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:33.634 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:33.634 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:33.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.634 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:157 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'111, mlcod=11'106)", 2023-11-25T04:23:33.634 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:33.634 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.830502449999997, 2023-11-25T04:23:33.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050590280000000001, 2023-11-25T04:23:33.634 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:33.634 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:33.634 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:33.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:33.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:33.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:33.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:33.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:33.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:33.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:33.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786063+0000", 2023-11-25T04:23:33.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2600000000000002e-07 2023-11-25T04:23:33.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:33.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786069+0000", 2023-11-25T04:23:33.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8839999999999999e-06 2023-11-25T04:23:33.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:33.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786079+0000", 2023-11-25T04:23:33.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0124e-05 2023-11-25T04:23:33.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:33.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786085+0000", 2023-11-25T04:23:33.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6660000000000002e-06 2023-11-25T04:23:33.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:33.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788160+0000", 2023-11-25T04:23:33.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020747669999999999 2023-11-25T04:23:33.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:33.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788180+0000", 2023-11-25T04:23:33.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0208999999999999e-05 2023-11-25T04:23:33.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:33.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791087+0000", 2023-11-25T04:23:33.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029064630000000002 2023-11-25T04:23:33.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:33.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791121+0000", 2023-11-25T04:23:33.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4289000000000003e-05 2023-11-25T04:23:33.639 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:33.639 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:33.639 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:33.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.639 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:158 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'112, mlcod=11'106)", 2023-11-25T04:23:33.639 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:33.639 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.830086057000003, 2023-11-25T04:23:33.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046663030000000001, 2023-11-25T04:23:33.639 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:33.639 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:33.640 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:33.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:33.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:33.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:33.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:33.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:33.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:33.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:33.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786479+0000", 2023-11-25T04:23:33.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2799999999999996e-07 2023-11-25T04:23:33.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:33.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786498+0000", 2023-11-25T04:23:33.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9069e-05 2023-11-25T04:23:33.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:33.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786508+0000", 2023-11-25T04:23:33.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5810000000000003e-06 2023-11-25T04:23:33.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:33.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786512+0000", 2023-11-25T04:23:33.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4889999999999997e-06 2023-11-25T04:23:33.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:33.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788438+0000", 2023-11-25T04:23:33.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019257059999999999 2023-11-25T04:23:33.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:33.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788449+0000", 2023-11-25T04:23:33.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1419e-05 2023-11-25T04:23:33.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:33.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791122+0000", 2023-11-25T04:23:33.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002672822 2023-11-25T04:23:33.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.644 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:33.644 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791145+0000", 2023-11-25T04:23:33.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2589e-05 2023-11-25T04:23:33.644 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:33.644 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:33.644 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:33.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.644 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:206 1.0 e12/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 12'152, mlcod=12'148)", 2023-11-25T04:23:33.644 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:33.644 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.318317587000003, 2023-11-25T04:23:33.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046728189999999999, 2023-11-25T04:23:33.644 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:33.645 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:33.645 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:33.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:33.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:33.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:33.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:33.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:33.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:33.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:33.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298248+0000", 2023-11-25T04:23:33.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3600000000000003e-07 2023-11-25T04:23:33.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:33.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298278+0000", 2023-11-25T04:23:33.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0474e-05 2023-11-25T04:23:33.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.647 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:33.647 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298288+0000", 2023-11-25T04:23:33.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0156000000000001e-05 2023-11-25T04:23:33.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.647 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:33.647 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298293+0000", 2023-11-25T04:23:33.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7400000000000004e-06 2023-11-25T04:23:33.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.647 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:33.647 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298333+0000", 2023-11-25T04:23:33.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9666000000000003e-05 2023-11-25T04:23:33.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.648 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:33.648 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298351+0000", 2023-11-25T04:23:33.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8544999999999999e-05 2023-11-25T04:23:33.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.648 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:33.648 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302891+0000", 2023-11-25T04:23:33.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045398519999999996 2023-11-25T04:23:33.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.648 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:33.649 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302920+0000", 2023-11-25T04:23:33.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8750000000000001e-05 2023-11-25T04:23:33.649 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:33.649 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:33.649 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:33.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.649 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:14 3.15 3:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:33.649 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:33.649 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 46.583183230000003, 2023-11-25T04:23:33.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010990437, 2023-11-25T04:23:33.649 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:33.649 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:33.650 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:33.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:33.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:33.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:33.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:33.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:33.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:33.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:33.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033383+0000", 2023-11-25T04:23:33.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9870000000000002e-06 2023-11-25T04:23:33.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:33.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033398+0000", 2023-11-25T04:23:33.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4569999999999999e-05 2023-11-25T04:23:33.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:33.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033411+0000", 2023-11-25T04:23:33.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3543999999999999e-05 2023-11-25T04:23:33.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:33.652 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033423+0000", 2023-11-25T04:23:33.652 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1905999999999999e-05 2023-11-25T04:23:33.652 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.652 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.652 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:33.652 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034433+0000", 2023-11-25T04:23:33.652 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00101022 2023-11-25T04:23:33.652 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.652 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.652 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:33.652 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034539+0000", 2023-11-25T04:23:33.652 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010597199999999999 2023-11-25T04:23:33.652 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.653 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:33.653 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034667+0000", 2023-11-25T04:23:33.653 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012765500000000001 2023-11-25T04:23:33.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.653 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:33.653 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037803+0000", 2023-11-25T04:23:33.653 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031362220000000001 2023-11-25T04:23:33.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.653 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:33.653 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:23:33.653 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0064964669999999997 2023-11-25T04:23:33.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:33.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044331+0000", 2023-11-25T04:23:33.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1544999999999997e-05 2023-11-25T04:23:33.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:33.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044372+0000", 2023-11-25T04:23:33.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0349000000000003e-05 2023-11-25T04:23:33.654 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:33.654 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:33.654 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:33.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.655 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:15 3.15 3:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:33.655 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:33.655 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 46.58252358, 2023-11-25T04:23:33.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010464362, 2023-11-25T04:23:33.655 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:33.655 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:33.655 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:33.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:33.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:33.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:33.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.656 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:33.656 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:33.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:33.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.656 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:33.656 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034042+0000", 2023-11-25T04:23:33.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0999999999999997e-07 2023-11-25T04:23:33.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.657 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:33.657 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034048+0000", 2023-11-25T04:23:33.657 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0390000000000001e-06 2023-11-25T04:23:33.657 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.657 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.657 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:33.657 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034059+0000", 2023-11-25T04:23:33.657 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1182999999999999e-05 2023-11-25T04:23:33.657 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.657 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.657 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:33.657 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034067+0000", 2023-11-25T04:23:33.657 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1559999999999995e-06 2023-11-25T04:23:33.658 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.658 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.658 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:33.658 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035005+0000", 2023-11-25T04:23:33.658 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00093747399999999999 2023-11-25T04:23:33.658 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.658 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.658 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:33.658 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035077+0000", 2023-11-25T04:23:33.658 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2747e-05 2023-11-25T04:23:33.658 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.658 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.658 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:33.659 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035219+0000", 2023-11-25T04:23:33.659 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000141877 2023-11-25T04:23:33.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.659 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.659 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:33.659 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037810+0000", 2023-11-25T04:23:33.659 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002590752 2023-11-25T04:23:33.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.659 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.659 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:33.659 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044437+0000", 2023-11-25T04:23:33.659 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066270110000000004 2023-11-25T04:23:33.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.660 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.660 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:33.660 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044481+0000", 2023-11-25T04:23:33.660 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4106000000000001e-05 2023-11-25T04:23:33.660 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.660 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.660 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:33.660 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044505+0000", 2023-11-25T04:23:33.660 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4207e-05 2023-11-25T04:23:33.660 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:33.660 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:33.660 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:33.660 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.660 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.661 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:16 3.1d 3:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:33.661 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:33.661 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 46.582481731999998, 2023-11-25T04:23:33.661 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010413923, 2023-11-25T04:23:33.661 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:33.661 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:33.661 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:33.661 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.661 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:33.661 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:33.661 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:33.661 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.661 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.661 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:33.662 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:33.662 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:33.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.662 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.662 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:33.662 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:33.662 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4499999999999997e-07 2023-11-25T04:23:33.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.662 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.662 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:33.662 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034087+0000", 2023-11-25T04:23:33.662 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9770000000000002e-06 2023-11-25T04:23:33.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:33.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034094+0000", 2023-11-25T04:23:33.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8809999999999998e-06 2023-11-25T04:23:33.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:33.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034100+0000", 2023-11-25T04:23:33.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8440000000000003e-06 2023-11-25T04:23:33.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:33.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034993+0000", 2023-11-25T04:23:33.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00089336799999999996 2023-11-25T04:23:33.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:33.664 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035099+0000", 2023-11-25T04:23:33.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106037 2023-11-25T04:23:33.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:33.664 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035276+0000", 2023-11-25T04:23:33.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017631799999999999 2023-11-25T04:23:33.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:33.665 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037815+0000", 2023-11-25T04:23:33.665 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025395959999999999 2023-11-25T04:23:33.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.665 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.665 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:33.665 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044420+0000", 2023-11-25T04:23:33.665 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066045749999999997 2023-11-25T04:23:33.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.665 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.665 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:33.665 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044463+0000", 2023-11-25T04:23:33.665 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3205999999999999e-05 2023-11-25T04:23:33.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.666 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.666 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:33.666 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044497+0000", 2023-11-25T04:23:33.666 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3476000000000003e-05 2023-11-25T04:23:33.666 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:33.666 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:33.666 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:33.666 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.666 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.666 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:17 3.1d 3:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:33.666 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:33.666 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 46.582456522999998, 2023-11-25T04:23:33.666 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01127356, 2023-11-25T04:23:33.667 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:33.667 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:33.667 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:33.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.667 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:33.667 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:33.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:33.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.667 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:33.667 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:33.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:33.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.668 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:33.668 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:33.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1100000000000001e-07 2023-11-25T04:23:33.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.668 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:33.668 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034112+0000", 2023-11-25T04:23:33.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8600000000000003e-06 2023-11-25T04:23:33.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.668 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:33.668 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034120+0000", 2023-11-25T04:23:33.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7700000000000001e-06 2023-11-25T04:23:33.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.669 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:33.669 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034127+0000", 2023-11-25T04:23:33.669 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0999999999999998e-06 2023-11-25T04:23:33.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.669 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:33.669 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035601+0000", 2023-11-25T04:23:33.669 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014739410000000001 2023-11-25T04:23:33.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.669 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:33.669 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035685+0000", 2023-11-25T04:23:33.669 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3677999999999994e-05 2023-11-25T04:23:33.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.670 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:33.670 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035827+0000", 2023-11-25T04:23:33.670 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014232599999999999 2023-11-25T04:23:33.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.670 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:33.670 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038725+0000", 2023-11-25T04:23:33.670 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028982779999999998 2023-11-25T04:23:33.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.670 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:33.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045323+0000", 2023-11-25T04:23:33.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0065980780000000003 2023-11-25T04:23:33.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:33.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045354+0000", 2023-11-25T04:23:33.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0463e-05 2023-11-25T04:23:33.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:33.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045381+0000", 2023-11-25T04:23:33.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7654999999999999e-05 2023-11-25T04:23:33.671 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:33.672 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:33.672 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:33.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.672 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:22 3.e 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:33.672 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:33.672 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 46.582430109999997, 2023-11-25T04:23:33.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010133422, 2023-11-25T04:23:33.672 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:33.672 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:33.672 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:33.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.699 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:33.699 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:33.700 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:33.700 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.700 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.700 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:33.700 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:33.700 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:33.700 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.700 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.700 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:33.700 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034135+0000", 2023-11-25T04:23:33.701 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-07 2023-11-25T04:23:33.701 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.701 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.702 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:33.702 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034138+0000", 2023-11-25T04:23:33.702 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6009999999999999e-06 2023-11-25T04:23:33.702 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.702 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:33.702 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034145+0000", 2023-11-25T04:23:33.702 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1169999999999998e-06 2023-11-25T04:23:33.702 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.702 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:33.702 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034150+0000", 2023-11-25T04:23:33.702 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9080000000000003e-06 2023-11-25T04:23:33.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.703 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:33.703 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034209+0000", 2023-11-25T04:23:33.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8149000000000002e-05 2023-11-25T04:23:33.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.703 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:33.703 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034415+0000", 2023-11-25T04:23:33.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020637699999999999 2023-11-25T04:23:33.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.703 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:33.704 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034584+0000", 2023-11-25T04:23:33.704 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016870399999999999 2023-11-25T04:23:33.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.704 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.704 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:33.704 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.036979+0000", 2023-11-25T04:23:33.704 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023957509999999998 2023-11-25T04:23:33.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.704 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.704 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:33.704 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044161+0000", 2023-11-25T04:23:33.704 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0071811590000000003 2023-11-25T04:23:33.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.704 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.705 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:33.705 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044224+0000", 2023-11-25T04:23:33.705 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3454000000000006e-05 2023-11-25T04:23:33.705 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.705 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.705 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:33.705 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044268+0000", 2023-11-25T04:23:33.705 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3761999999999999e-05 2023-11-25T04:23:33.705 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:33.705 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:33.705 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:33.705 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.705 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.706 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:23 3.e 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:33.706 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:33.706 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 46.582398701000002, 2023-11-25T04:23:33.706 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010281676, 2023-11-25T04:23:33.706 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:33.706 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:33.706 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:33.706 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.706 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:33.706 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:33.706 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:33.706 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.706 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.707 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:33.707 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:33.707 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:33.707 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.707 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.707 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:33.707 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:33.707 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5799999999999999e-07 2023-11-25T04:23:33.707 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.707 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.707 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:33.707 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034169+0000", 2023-11-25T04:23:33.707 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1879999999999998e-06 2023-11-25T04:23:33.708 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.708 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.708 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:33.708 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034177+0000", 2023-11-25T04:23:33.708 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7020000000000001e-06 2023-11-25T04:23:33.708 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.708 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.708 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:33.708 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034184+0000", 2023-11-25T04:23:33.708 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3910000000000003e-06 2023-11-25T04:23:33.708 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.708 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.708 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:33.709 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034924+0000", 2023-11-25T04:23:33.709 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00074032999999999998 2023-11-25T04:23:33.709 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.709 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.709 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:33.709 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034991+0000", 2023-11-25T04:23:33.709 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7347000000000004e-05 2023-11-25T04:23:33.709 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.709 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.709 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:33.709 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035096+0000", 2023-11-25T04:23:33.709 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010488 2023-11-25T04:23:33.709 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.710 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.710 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:33.710 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037588+0000", 2023-11-25T04:23:33.710 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024918309999999999 2023-11-25T04:23:33.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.710 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.710 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:33.710 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044367+0000", 2023-11-25T04:23:33.710 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067787589999999997 2023-11-25T04:23:33.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.710 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.710 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:33.711 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044415+0000", 2023-11-25T04:23:33.711 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7803000000000003e-05 2023-11-25T04:23:33.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.711 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:33.711 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044447+0000", 2023-11-25T04:23:33.711 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2886999999999999e-05 2023-11-25T04:23:33.711 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:33.711 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:33.711 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:33.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.712 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:30 3.15 3:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:33.712 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:33.712 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 46.582340260999999, 2023-11-25T04:23:33.712 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010365070000000001, 2023-11-25T04:23:33.712 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:33.712 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:33.712 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:33.712 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.712 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:33.712 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:33.712 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:33.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.713 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:33.713 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:33.713 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:33.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.713 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:33.713 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034225+0000", 2023-11-25T04:23:33.713 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7400000000000002e-07 2023-11-25T04:23:33.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.713 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:33.714 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034229+0000", 2023-11-25T04:23:33.714 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3100000000000002e-06 2023-11-25T04:23:33.714 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.714 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.714 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:33.714 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034239+0000", 2023-11-25T04:23:33.714 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9140000000000003e-06 2023-11-25T04:23:33.714 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.714 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.714 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:33.714 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034246+0000", 2023-11-25T04:23:33.714 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0149999999999998e-06 2023-11-25T04:23:33.715 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.715 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.715 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:33.715 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035572+0000", 2023-11-25T04:23:33.715 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001325655 2023-11-25T04:23:33.715 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.715 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.715 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:33.715 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035730+0000", 2023-11-25T04:23:33.715 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015827899999999999 2023-11-25T04:23:33.715 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.715 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.715 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:33.716 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035857+0000", 2023-11-25T04:23:33.716 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012644699999999999 2023-11-25T04:23:33.716 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.716 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.716 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:33.716 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038737+0000", 2023-11-25T04:23:33.716 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002880077 2023-11-25T04:23:33.716 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.716 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.716 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:33.716 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044540+0000", 2023-11-25T04:23:33.716 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058034239999999997 2023-11-25T04:23:33.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.717 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:33.717 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044569+0000", 2023-11-25T04:23:33.717 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8388999999999999e-05 2023-11-25T04:23:33.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.717 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:33.717 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044589+0000", 2023-11-25T04:23:33.717 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0686000000000002e-05 2023-11-25T04:23:33.717 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:33.717 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:33.718 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:33.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.718 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:33 3.1b 3:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:33.718 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:33.718 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 46.582294763, 2023-11-25T04:23:33.718 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099754049999999997, 2023-11-25T04:23:33.718 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:33.718 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:23:33.718 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:33.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.718 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:33.719 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:33.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:33.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.719 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.719 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:33.719 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:33.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:33.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.719 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.719 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:33.719 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:33.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4799999999999998e-07 2023-11-25T04:23:33.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.720 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.720 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:33.720 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034273+0000", 2023-11-25T04:23:33.720 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7819999999999998e-06 2023-11-25T04:23:33.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.720 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.720 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:33.720 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034285+0000", 2023-11-25T04:23:33.720 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2058e-05 2023-11-25T04:23:33.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.720 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.720 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:33.721 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034291+0000", 2023-11-25T04:23:33.721 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0680000000000002e-06 2023-11-25T04:23:33.721 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.721 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.721 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:33.721 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034370+0000", 2023-11-25T04:23:33.721 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8452e-05 2023-11-25T04:23:33.721 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.721 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.721 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:33.721 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034477+0000", 2023-11-25T04:23:33.721 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107471 2023-11-25T04:23:33.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.722 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.722 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:33.722 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044245+0000", 2023-11-25T04:23:33.722 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0097679259999999997 2023-11-25T04:23:33.722 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:33.722 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:33.722 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:33.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.722 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.722 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:36 3.1a 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:33.722 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:33.723 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 46.58224809, 2023-11-25T04:23:33.723 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058179099999999999, 2023-11-25T04:23:33.723 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:33.723 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:23:33.723 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:33.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.723 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:33.723 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:33.723 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:33.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.723 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:33.724 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:33.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:33.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.724 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.724 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:33.724 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034317+0000", 2023-11-25T04:23:33.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1999999999999999e-07 2023-11-25T04:23:33.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.724 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.724 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:33.724 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034321+0000", 2023-11-25T04:23:33.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.233e-06 2023-11-25T04:23:33.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:33.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034337+0000", 2023-11-25T04:23:33.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5308999999999999e-05 2023-11-25T04:23:33.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:33.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034354+0000", 2023-11-25T04:23:33.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7411999999999999e-05 2023-11-25T04:23:33.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:33.726 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034465+0000", 2023-11-25T04:23:33.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011154699999999999 2023-11-25T04:23:33.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.726 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:33.726 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034560+0000", 2023-11-25T04:23:33.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4186999999999998e-05 2023-11-25T04:23:33.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.726 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:33.726 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.040134+0000", 2023-11-25T04:23:33.727 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055746019999999997 2023-11-25T04:23:33.727 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:33.727 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:33.727 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:33.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.727 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.727 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:9 3.10 e18/16 3:09401a81:::601.00000000:head v 18'4, mlcod=0'0)", 2023-11-25T04:23:33.727 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:33.727 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 46.573353781999998, 2023-11-25T04:23:33.727 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061641020000000003, 2023-11-25T04:23:33.727 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:33.727 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:33.728 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:33.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.728 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:33.728 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:33.728 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:33.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.728 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:33.728 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:33.728 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:33.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.729 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:33.729 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:33.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5300000000000001e-07 2023-11-25T04:23:33.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.729 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:33.729 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043222+0000", 2023-11-25T04:23:33.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.082e-05 2023-11-25T04:23:33.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.729 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:33.729 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043228+0000", 2023-11-25T04:23:33.730 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6699999999999997e-06 2023-11-25T04:23:33.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.730 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:33.730 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043236+0000", 2023-11-25T04:23:33.730 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8189999999999992e-06 2023-11-25T04:23:33.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.730 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:33.730 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043507+0000", 2023-11-25T04:23:33.730 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000271012 2023-11-25T04:23:33.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.731 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:33.731 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043530+0000", 2023-11-25T04:23:33.731 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2659000000000001e-05 2023-11-25T04:23:33.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.731 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:33.731 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049321+0000", 2023-11-25T04:23:33.731 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057915129999999999 2023-11-25T04:23:33.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:33.732 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049375+0000", 2023-11-25T04:23:33.732 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3455999999999998e-05 2023-11-25T04:23:33.732 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:33.732 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:33.732 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:33.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.732 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:1 3.14 e18/16 3:292cf221:::200.00000000:head v 18'3, mlcod=0'0)", 2023-11-25T04:23:33.732 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:33.732 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 46.573317068999998, 2023-11-25T04:23:33.732 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0060983599999999997, 2023-11-25T04:23:33.733 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:33.733 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:33.733 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:33.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.733 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:33.733 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:33.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:33.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.733 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:33.733 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:33.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:33.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.734 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:33.734 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043248+0000", 2023-11-25T04:23:33.734 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9399999999999995e-07 2023-11-25T04:23:33.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.734 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:33.734 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043252+0000", 2023-11-25T04:23:33.734 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1799999999999998e-06 2023-11-25T04:23:33.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.735 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:33.735 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043259+0000", 2023-11-25T04:23:33.735 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0049999999999998e-06 2023-11-25T04:23:33.735 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.735 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.735 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:33.735 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043265+0000", 2023-11-25T04:23:33.735 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0859999999999998e-06 2023-11-25T04:23:33.735 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.735 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.735 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:33.735 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043311+0000", 2023-11-25T04:23:33.736 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6082999999999999e-05 2023-11-25T04:23:33.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.736 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.736 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:33.736 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043330+0000", 2023-11-25T04:23:33.736 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8556e-05 2023-11-25T04:23:33.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.736 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.736 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:33.736 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049303+0000", 2023-11-25T04:23:33.736 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059733470000000004 2023-11-25T04:23:33.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:33.737 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049346+0000", 2023-11-25T04:23:33.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2608999999999998e-05 2023-11-25T04:23:33.737 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:33.737 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:33.737 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:33.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.737 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:3 3.14 e18/16 3:287034c4:::1.00000000.inode:head v 18'4, mlcod=0'0)", 2023-11-25T04:23:33.737 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:33.738 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 46.572800563000001, 2023-11-25T04:23:33.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057040090000000003, 2023-11-25T04:23:33.738 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:33.738 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:33.738 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:33.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:33.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:33.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:33.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:33.739 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:33.739 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:33.739 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.739 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.739 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:33.739 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:33.739 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9799999999999999e-07 2023-11-25T04:23:33.739 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.739 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.739 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:33.739 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043785+0000", 2023-11-25T04:23:33.739 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0721999999999999e-05 2023-11-25T04:23:33.740 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.740 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.740 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:33.740 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043793+0000", 2023-11-25T04:23:33.740 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5839999999999995e-06 2023-11-25T04:23:33.740 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.740 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.740 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:33.740 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043803+0000", 2023-11-25T04:23:33.740 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0729999999999994e-06 2023-11-25T04:23:33.740 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.740 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:33.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043843+0000", 2023-11-25T04:23:33.741 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0747999999999997e-05 2023-11-25T04:23:33.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:33.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043856+0000", 2023-11-25T04:23:33.741 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2697000000000001e-05 2023-11-25T04:23:33.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:33.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049426+0000", 2023-11-25T04:23:33.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055704409999999998 2023-11-25T04:23:33.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.742 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:33.742 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049468+0000", 2023-11-25T04:23:33.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1446000000000003e-05 2023-11-25T04:23:33.742 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:33.742 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:33.742 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:33.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.742 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "pg_update_log_missing(3.14 epoch 18/16 rep_tid 171 entries 18'5 (0'0) error 3:2e2fa760:::200.00000001:head by mds.0.3:32 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:23:33.743 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:33.743 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 46.572639600000002, 2023-11-25T04:23:33.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005556938, 2023-11-25T04:23:33.743 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:33.743 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2023-11-25T04:23:33.743 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:33.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.743 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:33.743 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:33.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:33.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.744 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:33.744 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:33.744 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:33.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.744 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:33.744 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:33.744 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.03e-07 2023-11-25T04:23:33.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.744 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:33.744 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043931+0000", 2023-11-25T04:23:33.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1079999999999998e-06 2023-11-25T04:23:33.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:33.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043944+0000", 2023-11-25T04:23:33.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2852e-05 2023-11-25T04:23:33.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:33.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043955+0000", 2023-11-25T04:23:33.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1037e-05 2023-11-25T04:23:33.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:33.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044088+0000", 2023-11-25T04:23:33.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013269599999999999 2023-11-25T04:23:33.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:33.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049482+0000", 2023-11-25T04:23:33.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053940419999999999 2023-11-25T04:23:33.747 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:33.747 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:33.748 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:33.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.749 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.749 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:10 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'5, mlcod=0'0)", 2023-11-25T04:23:33.749 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:33.749 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 46.572549572, 2023-11-25T04:23:33.749 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054493099999999997, 2023-11-25T04:23:33.749 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:33.749 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:33.749 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:33.749 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.749 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:33.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:33.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:33.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:33.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:33.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:33.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:33.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:33.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6100000000000002e-07 2023-11-25T04:23:33.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:33.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044019+0000", 2023-11-25T04:23:33.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.839e-06 2023-11-25T04:23:33.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:33.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044026+0000", 2023-11-25T04:23:33.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5869999999999998e-06 2023-11-25T04:23:33.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.752 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:33.752 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044031+0000", 2023-11-25T04:23:33.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4909999999999996e-06 2023-11-25T04:23:33.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.752 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:33.752 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044070+0000", 2023-11-25T04:23:33.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8829999999999999e-05 2023-11-25T04:23:33.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.752 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:33.753 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044081+0000", 2023-11-25T04:23:33.753 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1447999999999999e-05 2023-11-25T04:23:33.753 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.753 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.753 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:33.753 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049406+0000", 2023-11-25T04:23:33.753 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053246480000000004 2023-11-25T04:23:33.753 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.753 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.753 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:33.753 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049464+0000", 2023-11-25T04:23:33.753 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8205999999999998e-05 2023-11-25T04:23:33.754 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:33.754 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:33.754 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:33.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.754 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:11 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'6, mlcod=0'0)", 2023-11-25T04:23:33.754 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:33.754 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 46.572348308000002, 2023-11-25T04:23:33.754 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052673499999999996, 2023-11-25T04:23:33.754 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:33.754 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:33.754 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:33.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:33.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:33.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:33.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:33.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:33.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:33.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:33.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:33.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0199999999999998e-07 2023-11-25T04:23:33.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.756 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.756 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:33.756 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044220+0000", 2023-11-25T04:23:33.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7720000000000001e-06 2023-11-25T04:23:33.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.756 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.756 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:33.756 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044226+0000", 2023-11-25T04:23:33.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8579999999999996e-06 2023-11-25T04:23:33.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.757 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.757 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:33.757 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044232+0000", 2023-11-25T04:23:33.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1700000000000002e-06 2023-11-25T04:23:33.757 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.757 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.757 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:33.757 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044274+0000", 2023-11-25T04:23:33.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1254999999999997e-05 2023-11-25T04:23:33.757 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.757 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.757 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:33.757 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:23:33.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6880999999999999e-05 2023-11-25T04:23:33.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.758 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:33.758 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049466+0000", 2023-11-25T04:23:33.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051659219999999999 2023-11-25T04:23:33.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:33.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:33.758 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:33.758 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049483+0000", 2023-11-25T04:23:33.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.719e-05 2023-11-25T04:23:33.758 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:33.759 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:33.759 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:33.759 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:33.759 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:33.759 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:33.760 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:23:33.761 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:33.759+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:33.861 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:33.859+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:33.868 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:33.868 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:23:33.868 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2023-11-25T04:23:33.868 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:33.877 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:23:33.927 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2023-11-25T04:23:33.962 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:33.960+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:34.022 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:34.022 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:23:34.022 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2023-11-25T04:23:34.022 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2023-11-25T04:23:34.023 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:34.034 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:23:34.077 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:34.076+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:34.142 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:34.142 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2023-11-25T04:23:34.142 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2023-11-25T04:23:34.142 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2023-11-25T04:23:34.142 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.143 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:34.143 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:34.143 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.108129327999997, 2023-11-25T04:23:34.143 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01072943, 2023-11-25T04:23:34.143 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:34.143 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:34.143 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:34.143 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.143 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:34.143 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:34.143 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:34.143 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.144 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.144 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:34.144 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:34.144 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:34.144 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.144 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.144 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:34.144 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031829+0000", 2023-11-25T04:23:34.144 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.666e-06 2023-11-25T04:23:34.144 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.144 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.145 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:34.145 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031839+0000", 2023-11-25T04:23:34.145 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9490000000000008e-06 2023-11-25T04:23:34.145 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.145 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.145 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:34.145 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031857+0000", 2023-11-25T04:23:34.145 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8093000000000001e-05 2023-11-25T04:23:34.145 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.145 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.145 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:34.145 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031882+0000", 2023-11-25T04:23:34.146 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4294000000000002e-05 2023-11-25T04:23:34.146 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.146 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.146 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:34.146 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033130+0000", 2023-11-25T04:23:34.146 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001248264 2023-11-25T04:23:34.146 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.146 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.146 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:34.146 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033426+0000", 2023-11-25T04:23:34.147 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029620999999999998 2023-11-25T04:23:34.147 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.147 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.147 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:34.147 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033814+0000", 2023-11-25T04:23:34.147 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000387742 2023-11-25T04:23:34.147 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.147 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.147 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:34.147 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036793+0000", 2023-11-25T04:23:34.147 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002979598 2023-11-25T04:23:34.147 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.148 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.148 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:34.148 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042455+0000", 2023-11-25T04:23:34.148 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0056619260000000003 2023-11-25T04:23:34.148 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.148 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.148 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:34.148 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042502+0000", 2023-11-25T04:23:34.148 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7139999999999999e-05 2023-11-25T04:23:34.148 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.148 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.148 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:34.148 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042557+0000", 2023-11-25T04:23:34.149 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4548e-05 2023-11-25T04:23:34.149 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:34.149 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:34.149 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:34.149 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.149 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.149 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:34.149 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:34.149 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.106930792999997, 2023-11-25T04:23:34.149 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011505202000000001, 2023-11-25T04:23:34.149 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:34.149 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:34.150 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:34.150 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.150 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:34.150 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:34.150 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:34.150 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.150 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.150 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:34.150 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:34.150 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:34.150 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.150 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.151 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:34.151 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033027+0000", 2023-11-25T04:23:34.151 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0699999999999999e-06 2023-11-25T04:23:34.151 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.151 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.151 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:34.151 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033035+0000", 2023-11-25T04:23:34.151 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0830000000000004e-06 2023-11-25T04:23:34.151 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.151 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.151 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:34.151 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033059+0000", 2023-11-25T04:23:34.152 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3975e-05 2023-11-25T04:23:34.152 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.152 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.152 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:34.152 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033482+0000", 2023-11-25T04:23:34.152 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042253700000000001 2023-11-25T04:23:34.152 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.152 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.152 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:34.152 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034520+0000", 2023-11-25T04:23:34.152 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001037991 2023-11-25T04:23:34.152 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.153 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.153 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:34.153 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034611+0000", 2023-11-25T04:23:34.154 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1113999999999995e-05 2023-11-25T04:23:34.154 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.154 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.154 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:34.154 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034769+0000", 2023-11-25T04:23:34.155 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000157641 2023-11-25T04:23:34.155 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.155 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.155 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:34.155 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038292+0000", 2023-11-25T04:23:34.155 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035231160000000002 2023-11-25T04:23:34.155 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.155 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.155 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:34.155 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044440+0000", 2023-11-25T04:23:34.155 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0061486980000000002 2023-11-25T04:23:34.155 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.155 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.156 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:34.156 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044495+0000", 2023-11-25T04:23:34.156 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4339000000000003e-05 2023-11-25T04:23:34.156 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.156 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.156 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:34.156 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044531+0000", 2023-11-25T04:23:34.156 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6637999999999997e-05 2023-11-25T04:23:34.156 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:34.156 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:34.156 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:34.156 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.156 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.157 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:34.157 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:34.157 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.106456438999999, 2023-11-25T04:23:34.157 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0057634039999999997, 2023-11-25T04:23:34.157 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:34.157 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:34.157 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:34.157 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.157 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:34.157 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:34.157 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:34.157 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.157 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.158 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:34.158 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:34.158 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:34.158 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.158 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.158 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:34.158 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:34.158 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7900000000000007e-07 2023-11-25T04:23:34.158 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.158 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.158 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:34.158 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033507+0000", 2023-11-25T04:23:34.159 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8900000000000004e-06 2023-11-25T04:23:34.159 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.159 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.159 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:34.159 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033519+0000", 2023-11-25T04:23:34.159 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1756000000000001e-05 2023-11-25T04:23:34.159 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.159 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.159 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:34.159 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033527+0000", 2023-11-25T04:23:34.159 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3610000000000001e-06 2023-11-25T04:23:34.159 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.159 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.160 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:34.160 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034451+0000", 2023-11-25T04:23:34.160 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00092372400000000001 2023-11-25T04:23:34.160 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.160 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.160 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:34.160 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034809+0000", 2023-11-25T04:23:34.160 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00035757699999999998 2023-11-25T04:23:34.160 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.160 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.160 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:23:34.160 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034965+0000", 2023-11-25T04:23:34.160 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015638699999999999 2023-11-25T04:23:34.161 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.161 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.161 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:34.161 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038299+0000", 2023-11-25T04:23:34.161 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0033336559999999999 2023-11-25T04:23:34.161 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.161 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.161 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:34.161 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039169+0000", 2023-11-25T04:23:34.161 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00087066400000000005 2023-11-25T04:23:34.161 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.161 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.161 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:34.162 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039226+0000", 2023-11-25T04:23:34.162 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6511999999999997e-05 2023-11-25T04:23:34.162 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.162 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.162 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:34.162 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039264+0000", 2023-11-25T04:23:34.162 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7898000000000002e-05 2023-11-25T04:23:34.162 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:34.162 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:34.162 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:34.162 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.162 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.162 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:25 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:34.163 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:34.163 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.106377854999998, 2023-11-25T04:23:34.163 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096656169999999996, 2023-11-25T04:23:34.163 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:34.163 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:34.163 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:34.163 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.163 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:34.163 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:34.163 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:34.163 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.163 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.163 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:34.164 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:34.164 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:34.164 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.164 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.164 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:34.164 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033580+0000", 2023-11-25T04:23:34.164 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8199999999999999e-07 2023-11-25T04:23:34.164 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.164 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.164 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:34.164 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033585+0000", 2023-11-25T04:23:34.164 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3680000000000001e-06 2023-11-25T04:23:34.164 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.165 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.165 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:34.165 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033600+0000", 2023-11-25T04:23:34.165 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4331000000000001e-05 2023-11-25T04:23:34.165 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.165 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.165 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:34.165 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033606+0000", 2023-11-25T04:23:34.165 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5780000000000004e-06 2023-11-25T04:23:34.165 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.165 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.165 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:34.165 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035747+0000", 2023-11-25T04:23:34.166 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0021406530000000002 2023-11-25T04:23:34.166 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.166 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.166 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:34.166 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035901+0000", 2023-11-25T04:23:34.166 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000153931 2023-11-25T04:23:34.166 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.166 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.166 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:23:34.166 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036193+0000", 2023-11-25T04:23:34.166 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029211299999999999 2023-11-25T04:23:34.166 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.167 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.167 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:34.167 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038655+0000", 2023-11-25T04:23:34.167 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0024621640000000002 2023-11-25T04:23:34.167 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.167 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.167 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:34.167 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043130+0000", 2023-11-25T04:23:34.167 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.004474698 2023-11-25T04:23:34.167 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.167 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.167 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:34.167 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043204+0000", 2023-11-25T04:23:34.167 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4091999999999995e-05 2023-11-25T04:23:34.168 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.168 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.168 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:34.168 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043245+0000", 2023-11-25T04:23:34.168 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0907e-05 2023-11-25T04:23:34.168 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:34.168 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:34.168 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:34.168 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.168 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.168 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:26 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:34.168 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:34.168 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.106342030999997, 2023-11-25T04:23:34.168 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011086945000000001, 2023-11-25T04:23:34.169 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:34.169 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:34.169 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:34.169 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.169 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:34.169 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:34.169 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:34.169 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.169 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.169 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:34.169 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:34.169 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:34.169 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.170 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.170 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:34.170 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:34.170 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1699999999999998e-07 2023-11-25T04:23:34.170 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.170 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.170 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:34.170 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033627+0000", 2023-11-25T04:23:34.170 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.163e-05 2023-11-25T04:23:34.170 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.170 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.170 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:34.171 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033636+0000", 2023-11-25T04:23:34.171 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6859999999999995e-06 2023-11-25T04:23:34.171 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.171 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.171 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:34.171 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033642+0000", 2023-11-25T04:23:34.171 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3149999999999997e-06 2023-11-25T04:23:34.171 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.171 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.171 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:34.172 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035414+0000", 2023-11-25T04:23:34.172 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0017716380000000001 2023-11-25T04:23:34.172 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.172 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.172 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:34.172 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035539+0000", 2023-11-25T04:23:34.172 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000125477 2023-11-25T04:23:34.172 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.172 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.172 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:34.172 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035672+0000", 2023-11-25T04:23:34.172 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000133316 2023-11-25T04:23:34.172 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.173 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.173 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:34.173 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038649+0000", 2023-11-25T04:23:34.173 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029764679999999999 2023-11-25T04:23:34.173 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.173 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.173 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:34.173 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044578+0000", 2023-11-25T04:23:34.173 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0059293510000000002 2023-11-25T04:23:34.173 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.173 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.173 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:34.173 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044651+0000", 2023-11-25T04:23:34.174 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2513000000000002e-05 2023-11-25T04:23:34.174 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.174 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.174 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:34.174 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044702+0000", 2023-11-25T04:23:34.174 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1034e-05 2023-11-25T04:23:34.174 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:34.174 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:34.174 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:34.174 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.174 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.174 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:29 3.b 3:d0630e4c:::mds0_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:34.175 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:34.175 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.106304111, 2023-11-25T04:23:34.175 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010980227, 2023-11-25T04:23:34.175 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:34.175 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:34.175 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:34.175 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.175 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:34.175 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:34.175 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:34.175 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.175 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.175 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:34.176 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:34.176 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:34.176 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.176 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.176 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:34.176 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:34.176 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3499999999999996e-07 2023-11-25T04:23:34.176 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.176 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.176 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:34.176 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033658+0000", 2023-11-25T04:23:34.176 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1280000000000001e-06 2023-11-25T04:23:34.176 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.177 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.177 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:34.177 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033667+0000", 2023-11-25T04:23:34.177 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1430000000000004e-06 2023-11-25T04:23:34.177 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.177 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.177 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:34.177 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033678+0000", 2023-11-25T04:23:34.177 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0906e-05 2023-11-25T04:23:34.177 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.177 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.177 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:34.177 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034456+0000", 2023-11-25T04:23:34.178 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00077833899999999996 2023-11-25T04:23:34.178 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.178 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.178 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:34.178 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034575+0000", 2023-11-25T04:23:34.178 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011926000000000001 2023-11-25T04:23:34.178 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.178 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.178 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:34.178 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035001+0000", 2023-11-25T04:23:34.178 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042550999999999998 2023-11-25T04:23:34.178 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.179 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.179 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:34.179 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.037923+0000", 2023-11-25T04:23:34.179 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029219670000000001 2023-11-25T04:23:34.179 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.179 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.179 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:34.179 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044538+0000", 2023-11-25T04:23:34.179 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0066152349999999997 2023-11-25T04:23:34.179 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.179 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.179 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:34.179 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044594+0000", 2023-11-25T04:23:34.179 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5593000000000001e-05 2023-11-25T04:23:34.180 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.180 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.180 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:34.180 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044633+0000", 2023-11-25T04:23:34.180 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9511000000000002e-05 2023-11-25T04:23:34.180 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:34.180 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:34.180 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:34.180 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.180 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.180 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:34 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:34.180 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:34.181 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.106053998, 2023-11-25T04:23:34.181 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010920445000000001, 2023-11-25T04:23:34.181 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:34.181 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:34.181 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:34.181 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.181 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:34.181 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:34.181 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:34.181 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.181 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.181 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:34.181 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:34.182 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:34.182 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.182 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.182 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:34.182 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033904+0000", 2023-11-25T04:23:34.182 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6599999999999994e-07 2023-11-25T04:23:34.182 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.182 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.182 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:34.182 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033909+0000", 2023-11-25T04:23:34.182 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5639999999999996e-06 2023-11-25T04:23:34.182 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.182 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.182 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:34.183 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033923+0000", 2023-11-25T04:23:34.183 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3028e-05 2023-11-25T04:23:34.183 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.183 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.183 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:34.183 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034043+0000", 2023-11-25T04:23:34.183 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000120328 2023-11-25T04:23:34.183 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.183 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.183 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:34.183 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035576+0000", 2023-11-25T04:23:34.183 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015333720000000001 2023-11-25T04:23:34.183 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.184 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.184 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:34.184 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035699+0000", 2023-11-25T04:23:34.184 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000123001 2023-11-25T04:23:34.184 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.184 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.184 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:34.184 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044823+0000", 2023-11-25T04:23:34.184 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0091241859999999994 2023-11-25T04:23:34.184 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:34.184 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:34.184 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:34.184 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.184 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.185 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:35 3.1f 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:34.185 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:34.185 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.104749793000003, 2023-11-25T04:23:34.185 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096164779999999995, 2023-11-25T04:23:34.185 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:34.185 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:34.185 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:34.185 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.185 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:34.185 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:34.185 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:34.186 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.186 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.186 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:34.186 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:34.186 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:34.186 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.186 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.186 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:34.186 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035208+0000", 2023-11-25T04:23:34.186 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2699999999999998e-07 2023-11-25T04:23:34.186 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.186 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.186 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:34.187 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035213+0000", 2023-11-25T04:23:34.187 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.6070000000000001e-06 2023-11-25T04:23:34.187 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.187 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.187 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:34.187 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035350+0000", 2023-11-25T04:23:34.187 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013693599999999999 2023-11-25T04:23:34.187 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.187 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.187 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:34.187 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035358+0000", 2023-11-25T04:23:34.187 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0099999999999995e-06 2023-11-25T04:23:34.187 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.187 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.188 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:34.188 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036260+0000", 2023-11-25T04:23:34.188 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00090240100000000005 2023-11-25T04:23:34.188 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.188 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.188 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:34.188 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036517+0000", 2023-11-25T04:23:34.188 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00025736299999999998 2023-11-25T04:23:34.188 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.188 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.188 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:34.188 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044824+0000", 2023-11-25T04:23:34.188 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0083063340000000003 2023-11-25T04:23:34.188 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:34.189 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:34.189 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:34.189 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.189 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.189 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:37 3.1e 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:34.189 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:34.189 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.104583107000003, 2023-11-25T04:23:34.189 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0078322640000000002, 2023-11-25T04:23:34.189 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:34.189 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:34.189 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:34.189 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.189 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:34.190 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:34.190 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:34.190 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.190 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.190 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:34.190 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:34.190 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:34.190 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.190 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.190 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:34.190 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035375+0000", 2023-11-25T04:23:34.190 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9100000000000003e-07 2023-11-25T04:23:34.190 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.191 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.191 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:34.191 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035379+0000", 2023-11-25T04:23:34.191 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9430000000000002e-06 2023-11-25T04:23:34.191 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.191 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.191 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:34.191 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035388+0000", 2023-11-25T04:23:34.191 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2760000000000006e-06 2023-11-25T04:23:34.191 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.191 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.191 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:34.191 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035396+0000", 2023-11-25T04:23:34.191 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8480000000000001e-06 2023-11-25T04:23:34.192 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.192 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.192 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:34.192 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035452+0000", 2023-11-25T04:23:34.192 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5831000000000001e-05 2023-11-25T04:23:34.192 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.192 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.192 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:34.192 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035934+0000", 2023-11-25T04:23:34.192 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00048265599999999998 2023-11-25T04:23:34.192 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.192 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.192 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:34.192 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043206+0000", 2023-11-25T04:23:34.193 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.007271919 2023-11-25T04:23:34.193 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:34.193 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:34.193 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:34.193 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.193 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.193 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing(3.1a epoch 18/16 rep_tid 3 entries 18'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:36 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:23:34.193 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:34.193 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.103307520999998, 2023-11-25T04:23:34.193 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002073414, 2023-11-25T04:23:34.193 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:34.193 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:34.193 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:34.193 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.194 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:34.194 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:34.194 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:34.194 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.194 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.194 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:34.194 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:34.194 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:34.194 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.194 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.194 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:34.194 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036650+0000", 2023-11-25T04:23:34.194 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7300000000000002e-07 2023-11-25T04:23:34.194 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.195 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.195 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:34.195 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036659+0000", 2023-11-25T04:23:34.195 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.7499999999999992e-06 2023-11-25T04:23:34.195 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.195 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.195 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:34.195 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036673+0000", 2023-11-25T04:23:34.195 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4917e-05 2023-11-25T04:23:34.195 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.195 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.195 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:34.195 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036687+0000", 2023-11-25T04:23:34.196 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3883000000000001e-05 2023-11-25T04:23:34.196 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.196 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.196 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:34.196 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036781+0000", 2023-11-25T04:23:34.196 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3219000000000003e-05 2023-11-25T04:23:34.196 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.196 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.196 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:34.196 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038723+0000", 2023-11-25T04:23:34.196 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019422720000000001 2023-11-25T04:23:34.196 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:34.196 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:34.196 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:34.197 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.197 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.197 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:28 3.13 e18/16 3:cd5a64a3:::100.00000000:head v 18'1, mlcod=0'0)", 2023-11-25T04:23:34.197 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:34.197 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.099646362999998, 2023-11-25T04:23:34.197 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001850546, 2023-11-25T04:23:34.197 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:34.197 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:34.197 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:34.197 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.197 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:34.197 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:34.197 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:34.198 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.198 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.198 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:34.198 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:34.198 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:34.198 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.198 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.198 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:34.198 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:34.198 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7800000000000002e-07 2023-11-25T04:23:34.198 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.198 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.198 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:34.198 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040331+0000", 2023-11-25T04:23:34.199 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0449999999999999e-05 2023-11-25T04:23:34.199 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.199 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.199 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:34.199 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040339+0000", 2023-11-25T04:23:34.199 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0709999999999995e-06 2023-11-25T04:23:34.199 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.199 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.199 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:34.199 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040347+0000", 2023-11-25T04:23:34.199 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7079999999999997e-06 2023-11-25T04:23:34.199 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.199 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.199 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:34.200 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040410+0000", 2023-11-25T04:23:34.200 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2453000000000007e-05 2023-11-25T04:23:34.200 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.200 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.200 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:34.200 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040422+0000", 2023-11-25T04:23:34.200 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2429000000000001e-05 2023-11-25T04:23:34.200 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.200 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.200 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:34.200 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042129+0000", 2023-11-25T04:23:34.200 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001707266 2023-11-25T04:23:34.200 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.201 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.201 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:34.201 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042161+0000", 2023-11-25T04:23:34.201 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1791000000000001e-05 2023-11-25T04:23:34.201 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:34.201 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:34.201 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:34.201 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.201 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.201 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:27 3.13 e18/16 3:c91d4a1d:::mds0_inotable:head v 18'2, mlcod=0'0)", 2023-11-25T04:23:34.201 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:34.201 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.099442840000002, 2023-11-25T04:23:34.201 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0016886430000000001, 2023-11-25T04:23:34.201 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:34.202 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:34.202 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:34.202 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.202 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:34.202 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:34.202 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:34.202 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.202 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.202 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:34.202 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:34.202 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:34.202 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.202 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.203 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:34.203 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:34.203 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3999999999999997e-07 2023-11-25T04:23:34.203 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.203 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.203 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:34.203 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040520+0000", 2023-11-25T04:23:34.203 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2989999999999996e-06 2023-11-25T04:23:34.203 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.203 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.203 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:34.203 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040527+0000", 2023-11-25T04:23:34.203 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5730000000000002e-06 2023-11-25T04:23:34.204 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.204 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.204 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:34.204 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040535+0000", 2023-11-25T04:23:34.204 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8630000000000008e-06 2023-11-25T04:23:34.204 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.204 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.204 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:34.204 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040683+0000", 2023-11-25T04:23:34.204 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000148002 2023-11-25T04:23:34.204 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.204 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.204 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:34.204 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040700+0000", 2023-11-25T04:23:34.205 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7065e-05 2023-11-25T04:23:34.205 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.205 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.205 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:34.205 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042163+0000", 2023-11-25T04:23:34.205 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001462767 2023-11-25T04:23:34.205 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.205 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.205 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:34.205 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042203+0000", 2023-11-25T04:23:34.205 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9734000000000003e-05 2023-11-25T04:23:34.205 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:34.205 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:34.206 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:34.206 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.206 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.206 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:20 3.1c e18/16 3:3ed09add:::607.00000000:head v 18'1, mlcod=0'0)", 2023-11-25T04:23:34.206 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:34.206 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.099407798999998, 2023-11-25T04:23:34.206 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015271, 2023-11-25T04:23:34.206 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:34.206 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:34.206 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:34.206 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.206 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:34.206 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:34.206 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:34.207 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.207 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.207 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:34.207 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:34.207 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:34.207 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.207 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.207 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:34.207 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:34.207 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4799999999999999e-07 2023-11-25T04:23:34.207 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.207 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.207 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:34.208 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040561+0000", 2023-11-25T04:23:34.208 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1800000000000001e-05 2023-11-25T04:23:34.208 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.208 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.208 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:34.208 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040568+0000", 2023-11-25T04:23:34.208 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7730000000000002e-06 2023-11-25T04:23:34.208 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.208 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.208 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:34.208 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040587+0000", 2023-11-25T04:23:34.208 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8691000000000001e-05 2023-11-25T04:23:34.208 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.208 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.209 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:34.209 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040638+0000", 2023-11-25T04:23:34.209 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1068e-05 2023-11-25T04:23:34.209 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.209 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.209 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:34.209 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040670+0000", 2023-11-25T04:23:34.209 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2125999999999997e-05 2023-11-25T04:23:34.209 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.209 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.209 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:34.209 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042032+0000", 2023-11-25T04:23:34.209 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001361821 2023-11-25T04:23:34.210 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.210 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.210 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:34.210 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042076+0000", 2023-11-25T04:23:34.210 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4472999999999999e-05 2023-11-25T04:23:34.210 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:34.210 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:34.210 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:34.210 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.210 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.210 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:21 3.1c e18/16 3:3ed09add:::607.00000000:head v 18'2, mlcod=0'0)", 2023-11-25T04:23:34.210 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:34.210 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.099173073999999, 2023-11-25T04:23:34.210 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0013228160000000001, 2023-11-25T04:23:34.211 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:34.211 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:34.211 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:34.211 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.211 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:34.211 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:34.211 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:34.211 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.211 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.211 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:34.211 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:34.211 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:34.211 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.212 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.212 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:34.212 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:34.212 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9299999999999999e-07 2023-11-25T04:23:34.212 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.212 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.212 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:34.212 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040790+0000", 2023-11-25T04:23:34.212 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5770000000000001e-06 2023-11-25T04:23:34.212 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.212 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.212 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:34.212 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040799+0000", 2023-11-25T04:23:34.212 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6439999999999995e-06 2023-11-25T04:23:34.212 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.213 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.213 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:34.213 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040806+0000", 2023-11-25T04:23:34.213 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0180000000000004e-06 2023-11-25T04:23:34.213 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.213 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.213 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:34.213 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040907+0000", 2023-11-25T04:23:34.213 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000100096 2023-11-25T04:23:34.213 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.213 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.213 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:34.213 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040925+0000", 2023-11-25T04:23:34.214 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8536999999999999e-05 2023-11-25T04:23:34.214 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.214 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.214 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:34.214 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042079+0000", 2023-11-25T04:23:34.214 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00115358 2023-11-25T04:23:34.214 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.214 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.214 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:34.214 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042107+0000", 2023-11-25T04:23:34.214 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7971e-05 2023-11-25T04:23:34.214 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:34.214 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:34.214 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:34.215 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.215 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.215 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop_reply(mds.0.3:4 3.1f e18/16 ondisk, result = 0)", 2023-11-25T04:23:34.215 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:34.215 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.097833684000001, 2023-11-25T04:23:34.215 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00044092900000000001, 2023-11-25T04:23:34.215 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:34.215 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:34.215 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:34.215 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.215 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:34.215 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:34.215 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:34.215 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.216 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.216 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:34.216 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:34.216 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:34.216 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.216 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.216 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:34.216 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042124+0000", 2023-11-25T04:23:34.216 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1800000000000005e-07 2023-11-25T04:23:34.216 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.216 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.216 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:34.216 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042128+0000", 2023-11-25T04:23:34.217 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2289999999999997e-06 2023-11-25T04:23:34.217 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.217 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.217 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:34.217 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042138+0000", 2023-11-25T04:23:34.217 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9729999999999992e-06 2023-11-25T04:23:34.217 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.217 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.217 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:34.217 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042145+0000", 2023-11-25T04:23:34.217 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.336e-06 2023-11-25T04:23:34.217 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.218 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.218 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:34.218 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042441+0000", 2023-11-25T04:23:34.218 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029650500000000002 2023-11-25T04:23:34.218 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.218 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.218 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:34.218 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042448+0000", 2023-11-25T04:23:34.218 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4100000000000002e-06 2023-11-25T04:23:34.218 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.218 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.218 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:34.218 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042564+0000", 2023-11-25T04:23:34.219 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000115758 2023-11-25T04:23:34.219 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:34.219 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:34.219 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:34.219 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.219 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.219 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing_reply(3.1f epoch 18/16 rep_tid 8 lcod 18'4)", 2023-11-25T04:23:34.219 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:34.219 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.095508272000004, 2023-11-25T04:23:34.219 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00038154699999999998, 2023-11-25T04:23:34.219 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:34.219 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:34.219 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:34.220 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.220 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:34.220 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:34.220 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:34.220 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.220 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.220 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:34.220 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:34.220 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:34.220 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.220 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.220 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:34.220 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:34.220 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2700000000000001e-07 2023-11-25T04:23:34.221 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.221 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.221 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:34.221 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044451+0000", 2023-11-25T04:23:34.221 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7460000000000001e-06 2023-11-25T04:23:34.221 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.221 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.221 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:34.221 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044455+0000", 2023-11-25T04:23:34.221 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5809999999999998e-06 2023-11-25T04:23:34.221 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.221 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.221 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:34.221 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044458+0000", 2023-11-25T04:23:34.222 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0170000000000001e-06 2023-11-25T04:23:34.222 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.222 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.222 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:34.222 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044734+0000", 2023-11-25T04:23:34.222 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00027582799999999999 2023-11-25T04:23:34.222 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.222 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.222 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:34.222 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044830+0000", 2023-11-25T04:23:34.222 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6248000000000004e-05 2023-11-25T04:23:34.222 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:34.222 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:34.223 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:34.223 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.223 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.223 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing_reply(3.b epoch 18/16 rep_tid 5 lcod 18'2)", 2023-11-25T04:23:34.223 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:34.223 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.095471611000001, 2023-11-25T04:23:34.223 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000344488, 2023-11-25T04:23:34.223 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:34.223 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:34.223 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:34.223 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.223 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:34.223 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:34.223 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:34.224 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.224 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.224 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:34.224 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:34.224 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:34.224 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.224 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.224 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:34.224 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:34.224 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.55e-07 2023-11-25T04:23:34.224 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.224 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.224 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:34.225 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044487+0000", 2023-11-25T04:23:34.225 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2470000000000001e-06 2023-11-25T04:23:34.225 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.225 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.225 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:34.225 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044491+0000", 2023-11-25T04:23:34.225 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9659999999999998e-06 2023-11-25T04:23:34.225 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.225 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.225 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:34.225 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044494+0000", 2023-11-25T04:23:34.225 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2310000000000001e-06 2023-11-25T04:23:34.225 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.225 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.226 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:34.226 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044756+0000", 2023-11-25T04:23:34.226 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00026172800000000003 2023-11-25T04:23:34.226 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.226 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.226 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:34.226 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044830+0000", 2023-11-25T04:23:34.226 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4160999999999997e-05 2023-11-25T04:23:34.226 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:34.226 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:34.226 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:34.226 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.226 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.226 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:39 3.6 3:654134d2:::mds0_openfiles.0:head [omap-clear,omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2023-11-25T04:23:34.227 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:34.227 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 42.035645569000003, 2023-11-25T04:23:34.227 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001788658, 2023-11-25T04:23:34.227 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:34.227 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:34.227 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:34.227 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.227 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:34.227 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:34.227 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:34.227 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.227 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.227 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:34.228 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:34.228 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:34.228 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.228 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.228 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:34.228 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104314+0000", 2023-11-25T04:23:34.228 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3099999999999999e-06 2023-11-25T04:23:34.228 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.228 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.228 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:34.228 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104325+0000", 2023-11-25T04:23:34.228 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1511999999999999e-05 2023-11-25T04:23:34.228 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.228 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.229 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:34.229 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104362+0000", 2023-11-25T04:23:34.229 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6903000000000002e-05 2023-11-25T04:23:34.229 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.229 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.229 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:34.229 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104387+0000", 2023-11-25T04:23:34.229 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4766e-05 2023-11-25T04:23:34.229 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.229 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.229 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:34.229 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104465+0000", 2023-11-25T04:23:34.229 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8119000000000002e-05 2023-11-25T04:23:34.230 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.230 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.230 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:34.230 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104557+0000", 2023-11-25T04:23:34.230 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2496000000000003e-05 2023-11-25T04:23:34.230 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.230 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.230 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:34.230 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104646+0000", 2023-11-25T04:23:34.230 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8818000000000002e-05 2023-11-25T04:23:34.230 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.230 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.230 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:34.230 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105201+0000", 2023-11-25T04:23:34.231 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00055506000000000002 2023-11-25T04:23:34.231 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.231 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.231 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:34.231 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105994+0000", 2023-11-25T04:23:34.231 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00079294700000000001 2023-11-25T04:23:34.231 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.231 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.231 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:34.231 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.106055+0000", 2023-11-25T04:23:34.231 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0566000000000003e-05 2023-11-25T04:23:34.231 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.231 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.231 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:34.232 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.106100+0000", 2023-11-25T04:23:34.232 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5160999999999997e-05 2023-11-25T04:23:34.232 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:34.232 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:34.232 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:34.232 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.232 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.232 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:43 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2023-11-25T04:23:34.232 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:34.232 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 27.035420573, 2023-11-25T04:23:34.232 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001751166, 2023-11-25T04:23:34.232 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:34.232 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:34.232 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:34.233 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.233 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:34.233 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:34.233 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:34.233 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.233 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.233 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:34.233 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:34.233 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:34.233 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.233 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.233 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:34.233 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104539+0000", 2023-11-25T04:23:34.233 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9529999999999999e-06 2023-11-25T04:23:34.234 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.234 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.234 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:34.234 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104557+0000", 2023-11-25T04:23:34.234 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7464000000000001e-05 2023-11-25T04:23:34.234 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.234 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.234 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:34.234 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104590+0000", 2023-11-25T04:23:34.234 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2966000000000003e-05 2023-11-25T04:23:34.234 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.234 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.234 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:34.234 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104608+0000", 2023-11-25T04:23:34.235 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8193e-05 2023-11-25T04:23:34.235 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.237 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.237 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:34.237 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104767+0000", 2023-11-25T04:23:34.237 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015882 2023-11-25T04:23:34.237 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.237 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.237 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:34.237 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104909+0000", 2023-11-25T04:23:34.237 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014244999999999999 2023-11-25T04:23:34.237 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.238 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.238 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:34.238 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.105059+0000", 2023-11-25T04:23:34.238 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014964399999999999 2023-11-25T04:23:34.238 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.238 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.238 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:34.238 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.105701+0000", 2023-11-25T04:23:34.238 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00064201099999999995 2023-11-25T04:23:34.238 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.238 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.238 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:34.238 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106233+0000", 2023-11-25T04:23:34.239 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00053178500000000001 2023-11-25T04:23:34.239 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.239 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.239 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:34.239 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106265+0000", 2023-11-25T04:23:34.239 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1986000000000002e-05 2023-11-25T04:23:34.239 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.239 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.239 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:34.239 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106288+0000", 2023-11-25T04:23:34.239 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2894000000000001e-05 2023-11-25T04:23:34.239 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:34.239 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:34.239 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:34.240 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.240 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.240 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:45 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e23)", 2023-11-25T04:23:34.240 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:23:34.240 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 22.035222532999999, 2023-11-25T04:23:34.240 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00098423299999999994, 2023-11-25T04:23:34.240 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:34.240 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:34.240 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:34.240 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.240 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:34.240 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:23:34.240 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:34.240 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.241 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.241 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:34.241 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:23:34.241 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:34.241 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.241 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.241 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:34.241 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104737+0000", 2023-11-25T04:23:34.241 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3309999999999998e-06 2023-11-25T04:23:34.241 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.241 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.241 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:34.241 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104748+0000", 2023-11-25T04:23:34.241 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1080999999999999e-05 2023-11-25T04:23:34.242 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.242 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.242 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:34.242 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104771+0000", 2023-11-25T04:23:34.242 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2707000000000001e-05 2023-11-25T04:23:34.242 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.242 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.242 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:34.242 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104800+0000", 2023-11-25T04:23:34.242 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9595e-05 2023-11-25T04:23:34.242 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.242 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.242 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:34.243 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104890+0000", 2023-11-25T04:23:34.243 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.9808999999999993e-05 2023-11-25T04:23:34.243 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.243 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.243 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:34.243 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105004+0000", 2023-11-25T04:23:34.243 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000114508 2023-11-25T04:23:34.243 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.243 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.243 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:34.243 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105634+0000", 2023-11-25T04:23:34.243 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00062985799999999996 2023-11-25T04:23:34.243 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.243 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.244 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:34.244 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105671+0000", 2023-11-25T04:23:34.244 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6640000000000002e-05 2023-11-25T04:23:34.244 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:34.244 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:34.244 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:34.244 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105719+0000", 2023-11-25T04:23:34.244 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7704000000000003e-05 2023-11-25T04:23:34.244 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:34.244 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:34.244 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:34.244 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:34.244 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:34.244 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:34.246 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:34.244+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:34.246 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:23:34.347 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:34.345+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:34.355 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:34.355 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:23:34.355 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2023-11-25T04:23:34.355 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:34.364 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:23:34.448 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:34.446+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:34.468 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:34.469 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:23:34.469 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2023-11-25T04:23:34.469 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2023-11-25T04:23:34.469 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:34.477 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:23:34.525 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-11-25T04:23:34.561 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:34.559+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:34.623 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:34.623 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2023-11-25T04:23:34.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2023-11-25T04:23:34.623 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2023-11-25T04:23:34.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.624 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:155 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'109, mlcod=11'105)", 2023-11-25T04:23:34.624 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:34.624 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 56.835704630999999, 2023-11-25T04:23:34.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061719019999999999, 2023-11-25T04:23:34.624 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:34.624 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:34.624 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:34.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:34.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:34.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:34.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:34.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:34.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:34.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:34.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784884+0000", 2023-11-25T04:23:34.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3099999999999999e-07 2023-11-25T04:23:34.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:34.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784942+0000", 2023-11-25T04:23:34.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8570000000000003e-05 2023-11-25T04:23:34.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.627 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:34.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784949+0000", 2023-11-25T04:23:34.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0310000000000001e-06 2023-11-25T04:23:34.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.627 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:34.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784953+0000", 2023-11-25T04:23:34.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6320000000000001e-06 2023-11-25T04:23:34.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:34.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784993+0000", 2023-11-25T04:23:34.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9737000000000003e-05 2023-11-25T04:23:34.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:34.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785011+0000", 2023-11-25T04:23:34.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8349999999999999e-05 2023-11-25T04:23:34.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:34.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791024+0000", 2023-11-25T04:23:34.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006012997 2023-11-25T04:23:34.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:34.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791055+0000", 2023-11-25T04:23:34.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1254000000000003e-05 2023-11-25T04:23:34.630 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:34.630 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:34.630 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:34.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.631 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:156 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'110, mlcod=11'105)", 2023-11-25T04:23:34.631 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:34.631 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 56.835118582, 2023-11-25T04:23:34.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005616116, 2023-11-25T04:23:34.631 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:34.631 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:34.631 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:34.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:34.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:34.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:34.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:34.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:34.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:34.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:34.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785470+0000", 2023-11-25T04:23:34.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0699999999999998e-07 2023-11-25T04:23:34.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:34.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785474+0000", 2023-11-25T04:23:34.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.048e-06 2023-11-25T04:23:34.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:34.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785480+0000", 2023-11-25T04:23:34.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7729999999999996e-06 2023-11-25T04:23:34.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:34.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785483+0000", 2023-11-25T04:23:34.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1970000000000001e-06 2023-11-25T04:23:34.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:34.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787936+0000", 2023-11-25T04:23:34.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024529040000000001 2023-11-25T04:23:34.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:34.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787947+0000", 2023-11-25T04:23:34.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1669e-05 2023-11-25T04:23:34.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:34.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791057+0000", 2023-11-25T04:23:34.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031095200000000002 2023-11-25T04:23:34.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:34.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791086+0000", 2023-11-25T04:23:34.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8697999999999999e-05 2023-11-25T04:23:34.636 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:34.637 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:34.637 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:34.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.637 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:157 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'111, mlcod=11'106)", 2023-11-25T04:23:34.637 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:34.637 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 56.834526023999999, 2023-11-25T04:23:34.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050590280000000001, 2023-11-25T04:23:34.637 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:34.638 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:34.638 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:34.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:34.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:34.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:34.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:34.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:34.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:34.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:34.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786063+0000", 2023-11-25T04:23:34.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2600000000000002e-07 2023-11-25T04:23:34.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:34.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786069+0000", 2023-11-25T04:23:34.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8839999999999999e-06 2023-11-25T04:23:34.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:34.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786079+0000", 2023-11-25T04:23:34.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0124e-05 2023-11-25T04:23:34.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:34.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786085+0000", 2023-11-25T04:23:34.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6660000000000002e-06 2023-11-25T04:23:34.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:34.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788160+0000", 2023-11-25T04:23:34.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020747669999999999 2023-11-25T04:23:34.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:34.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788180+0000", 2023-11-25T04:23:34.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0208999999999999e-05 2023-11-25T04:23:34.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:34.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791087+0000", 2023-11-25T04:23:34.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029064630000000002 2023-11-25T04:23:34.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:34.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791121+0000", 2023-11-25T04:23:34.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4289000000000003e-05 2023-11-25T04:23:34.643 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:34.643 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:34.643 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:34.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.644 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:158 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'112, mlcod=11'106)", 2023-11-25T04:23:34.644 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:34.644 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 56.834109630999997, 2023-11-25T04:23:34.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046663030000000001, 2023-11-25T04:23:34.644 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:34.644 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:34.644 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:34.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:34.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:34.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:34.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:34.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:34.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:34.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:34.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786479+0000", 2023-11-25T04:23:34.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2799999999999996e-07 2023-11-25T04:23:34.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:34.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786498+0000", 2023-11-25T04:23:34.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9069e-05 2023-11-25T04:23:34.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.647 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:34.647 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786508+0000", 2023-11-25T04:23:34.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5810000000000003e-06 2023-11-25T04:23:34.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.647 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:34.647 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786512+0000", 2023-11-25T04:23:34.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4889999999999997e-06 2023-11-25T04:23:34.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.648 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:34.649 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788438+0000", 2023-11-25T04:23:34.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019257059999999999 2023-11-25T04:23:34.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:34.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788449+0000", 2023-11-25T04:23:34.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1419e-05 2023-11-25T04:23:34.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:34.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791122+0000", 2023-11-25T04:23:34.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002672822 2023-11-25T04:23:34.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:34.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791145+0000", 2023-11-25T04:23:34.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2589e-05 2023-11-25T04:23:34.651 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:34.651 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:34.651 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:34.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.652 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:206 1.0 e12/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 12'152, mlcod=12'148)", 2023-11-25T04:23:34.652 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:34.652 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 56.322341160999997, 2023-11-25T04:23:34.652 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046728189999999999, 2023-11-25T04:23:34.652 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:34.652 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:34.652 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:34.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.653 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:34.653 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:34.653 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:34.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.653 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:34.653 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:34.653 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:34.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:34.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298248+0000", 2023-11-25T04:23:34.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3600000000000003e-07 2023-11-25T04:23:34.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:34.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298278+0000", 2023-11-25T04:23:34.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0474e-05 2023-11-25T04:23:34.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:34.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298288+0000", 2023-11-25T04:23:34.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0156000000000001e-05 2023-11-25T04:23:34.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:34.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298293+0000", 2023-11-25T04:23:34.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7400000000000004e-06 2023-11-25T04:23:34.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.656 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:34.656 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298333+0000", 2023-11-25T04:23:34.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9666000000000003e-05 2023-11-25T04:23:34.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.656 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:34.656 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298351+0000", 2023-11-25T04:23:34.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8544999999999999e-05 2023-11-25T04:23:34.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.657 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:34.657 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302891+0000", 2023-11-25T04:23:34.657 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045398519999999996 2023-11-25T04:23:34.657 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.657 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.657 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:34.657 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302920+0000", 2023-11-25T04:23:34.657 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8750000000000001e-05 2023-11-25T04:23:34.657 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:34.658 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:34.658 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:34.658 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.658 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.658 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:14 3.15 3:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:34.658 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:34.658 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.587206803999997, 2023-11-25T04:23:34.658 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010990437, 2023-11-25T04:23:34.658 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:34.658 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:34.658 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:34.659 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.659 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:34.659 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:34.659 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:34.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.659 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.659 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:34.659 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:34.659 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:34.660 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.660 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.660 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:34.660 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033383+0000", 2023-11-25T04:23:34.660 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9870000000000002e-06 2023-11-25T04:23:34.660 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.660 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.660 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:34.660 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033398+0000", 2023-11-25T04:23:34.661 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4569999999999999e-05 2023-11-25T04:23:34.661 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.661 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.661 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:34.661 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033411+0000", 2023-11-25T04:23:34.661 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3543999999999999e-05 2023-11-25T04:23:34.661 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.661 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.661 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:34.661 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033423+0000", 2023-11-25T04:23:34.662 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1905999999999999e-05 2023-11-25T04:23:34.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.662 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.662 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:34.662 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034433+0000", 2023-11-25T04:23:34.662 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00101022 2023-11-25T04:23:34.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.662 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.662 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:34.662 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034539+0000", 2023-11-25T04:23:34.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010597199999999999 2023-11-25T04:23:34.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:34.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034667+0000", 2023-11-25T04:23:34.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012765500000000001 2023-11-25T04:23:34.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:34.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037803+0000", 2023-11-25T04:23:34.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031362220000000001 2023-11-25T04:23:34.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:34.664 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:23:34.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0064964669999999997 2023-11-25T04:23:34.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:34.664 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044331+0000", 2023-11-25T04:23:34.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1544999999999997e-05 2023-11-25T04:23:34.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.665 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.665 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:34.665 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044372+0000", 2023-11-25T04:23:34.665 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0349000000000003e-05 2023-11-25T04:23:34.665 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:34.665 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:34.665 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:34.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.665 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.666 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:15 3.15 3:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:34.666 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:34.666 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.586547154000002, 2023-11-25T04:23:34.666 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010464362, 2023-11-25T04:23:34.666 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:34.666 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:34.666 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:34.666 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.666 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:34.666 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:34.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:34.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.667 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:34.667 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:34.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:34.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.667 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:34.668 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034042+0000", 2023-11-25T04:23:34.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0999999999999997e-07 2023-11-25T04:23:34.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.668 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:34.668 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034048+0000", 2023-11-25T04:23:34.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0390000000000001e-06 2023-11-25T04:23:34.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.669 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:34.669 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034059+0000", 2023-11-25T04:23:34.669 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1182999999999999e-05 2023-11-25T04:23:34.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.669 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:34.669 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034067+0000", 2023-11-25T04:23:34.669 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1559999999999995e-06 2023-11-25T04:23:34.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.670 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:34.670 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035005+0000", 2023-11-25T04:23:34.670 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00093747399999999999 2023-11-25T04:23:34.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.670 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:34.670 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035077+0000", 2023-11-25T04:23:34.670 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2747e-05 2023-11-25T04:23:34.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:34.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035219+0000", 2023-11-25T04:23:34.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000141877 2023-11-25T04:23:34.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:34.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037810+0000", 2023-11-25T04:23:34.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002590752 2023-11-25T04:23:34.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.672 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:34.672 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044437+0000", 2023-11-25T04:23:34.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066270110000000004 2023-11-25T04:23:34.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.673 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:34.673 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044481+0000", 2023-11-25T04:23:34.673 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4106000000000001e-05 2023-11-25T04:23:34.673 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.673 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.673 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:34.673 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044505+0000", 2023-11-25T04:23:34.673 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4207e-05 2023-11-25T04:23:34.673 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:34.674 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:34.674 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:34.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.674 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.674 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:16 3.1d 3:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:34.674 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:34.674 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.586505305999999, 2023-11-25T04:23:34.674 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010413923, 2023-11-25T04:23:34.674 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:34.675 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:34.675 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:34.675 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.675 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:34.675 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:34.675 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:34.675 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.675 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.675 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:34.675 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:34.676 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:34.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.676 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:34.676 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:34.676 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4499999999999997e-07 2023-11-25T04:23:34.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.676 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:34.677 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034087+0000", 2023-11-25T04:23:34.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9770000000000002e-06 2023-11-25T04:23:34.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.677 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:34.677 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034094+0000", 2023-11-25T04:23:34.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8809999999999998e-06 2023-11-25T04:23:34.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.677 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:34.678 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034100+0000", 2023-11-25T04:23:34.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8440000000000003e-06 2023-11-25T04:23:34.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.678 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.678 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:34.678 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034993+0000", 2023-11-25T04:23:34.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00089336799999999996 2023-11-25T04:23:34.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.679 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.679 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:34.679 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035099+0000", 2023-11-25T04:23:34.679 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106037 2023-11-25T04:23:34.679 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.679 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.679 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:34.679 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035276+0000", 2023-11-25T04:23:34.679 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017631799999999999 2023-11-25T04:23:34.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.680 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.680 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:34.680 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037815+0000", 2023-11-25T04:23:34.680 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025395959999999999 2023-11-25T04:23:34.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.680 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.680 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:34.680 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044420+0000", 2023-11-25T04:23:34.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066045749999999997 2023-11-25T04:23:34.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.681 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:34.681 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044463+0000", 2023-11-25T04:23:34.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3205999999999999e-05 2023-11-25T04:23:34.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.681 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:34.681 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044497+0000", 2023-11-25T04:23:34.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3476000000000003e-05 2023-11-25T04:23:34.682 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:34.682 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:34.682 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:34.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.682 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:17 3.1d 3:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:34.682 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:34.682 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.586480096999999, 2023-11-25T04:23:34.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01127356, 2023-11-25T04:23:34.683 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:34.683 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:34.683 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:34.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.683 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:34.683 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:34.683 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:34.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.684 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:34.684 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:34.684 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:34.684 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.684 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.684 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:34.684 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:34.684 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1100000000000001e-07 2023-11-25T04:23:34.684 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.684 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.685 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:34.685 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034112+0000", 2023-11-25T04:23:34.685 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8600000000000003e-06 2023-11-25T04:23:34.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.685 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.685 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:34.685 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034120+0000", 2023-11-25T04:23:34.685 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7700000000000001e-06 2023-11-25T04:23:34.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:34.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034127+0000", 2023-11-25T04:23:34.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0999999999999998e-06 2023-11-25T04:23:34.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:34.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035601+0000", 2023-11-25T04:23:34.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014739410000000001 2023-11-25T04:23:34.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.687 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:34.687 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035685+0000", 2023-11-25T04:23:34.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3677999999999994e-05 2023-11-25T04:23:34.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.687 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:34.687 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035827+0000", 2023-11-25T04:23:34.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014232599999999999 2023-11-25T04:23:34.688 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.688 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.688 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:34.688 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038725+0000", 2023-11-25T04:23:34.688 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028982779999999998 2023-11-25T04:23:34.688 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.688 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.688 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:34.688 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045323+0000", 2023-11-25T04:23:34.688 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0065980780000000003 2023-11-25T04:23:34.689 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.689 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:34.689 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045354+0000", 2023-11-25T04:23:34.689 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0463e-05 2023-11-25T04:23:34.689 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.689 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:34.689 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045381+0000", 2023-11-25T04:23:34.689 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7654999999999999e-05 2023-11-25T04:23:34.690 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:34.690 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:34.690 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:34.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.690 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.690 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:22 3.e 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:34.690 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:34.690 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.586453683999999, 2023-11-25T04:23:34.690 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010133422, 2023-11-25T04:23:34.690 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:34.691 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:34.691 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:34.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.691 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:34.691 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:34.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:34.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.691 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:34.692 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:34.692 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:34.692 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.692 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.692 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:34.692 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034135+0000", 2023-11-25T04:23:34.692 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-07 2023-11-25T04:23:34.692 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.692 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.692 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:34.693 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034138+0000", 2023-11-25T04:23:34.693 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6009999999999999e-06 2023-11-25T04:23:34.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.693 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:34.693 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034145+0000", 2023-11-25T04:23:34.693 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1169999999999998e-06 2023-11-25T04:23:34.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.693 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:34.694 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034150+0000", 2023-11-25T04:23:34.694 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9080000000000003e-06 2023-11-25T04:23:34.694 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.694 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.694 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:34.694 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034209+0000", 2023-11-25T04:23:34.694 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8149000000000002e-05 2023-11-25T04:23:34.694 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.694 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.695 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:34.695 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034415+0000", 2023-11-25T04:23:34.695 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020637699999999999 2023-11-25T04:23:34.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.695 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.695 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:34.695 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034584+0000", 2023-11-25T04:23:34.695 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016870399999999999 2023-11-25T04:23:34.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.696 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.696 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:34.696 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.036979+0000", 2023-11-25T04:23:34.696 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023957509999999998 2023-11-25T04:23:34.696 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.696 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.696 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:34.696 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044161+0000", 2023-11-25T04:23:34.696 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0071811590000000003 2023-11-25T04:23:34.696 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.697 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:34.697 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044224+0000", 2023-11-25T04:23:34.697 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3454000000000006e-05 2023-11-25T04:23:34.697 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.697 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:34.697 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044268+0000", 2023-11-25T04:23:34.697 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3761999999999999e-05 2023-11-25T04:23:34.697 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:34.698 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:34.698 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:34.698 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.698 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.698 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:23 3.e 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:34.698 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:34.698 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.586422274999997, 2023-11-25T04:23:34.698 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010281676, 2023-11-25T04:23:34.699 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:34.699 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:34.699 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:34.699 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.699 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:34.699 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:34.699 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:34.699 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.699 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.699 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:34.700 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:34.700 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:34.700 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.700 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.700 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:34.700 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:34.700 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5799999999999999e-07 2023-11-25T04:23:34.700 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.700 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.701 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:34.701 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034169+0000", 2023-11-25T04:23:34.701 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1879999999999998e-06 2023-11-25T04:23:34.701 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.701 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.701 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:34.701 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034177+0000", 2023-11-25T04:23:34.701 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7020000000000001e-06 2023-11-25T04:23:34.701 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.702 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:34.702 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034184+0000", 2023-11-25T04:23:34.702 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3910000000000003e-06 2023-11-25T04:23:34.702 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.702 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:34.702 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034924+0000", 2023-11-25T04:23:34.702 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00074032999999999998 2023-11-25T04:23:34.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.703 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:34.703 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034991+0000", 2023-11-25T04:23:34.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7347000000000004e-05 2023-11-25T04:23:34.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.703 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:34.703 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035096+0000", 2023-11-25T04:23:34.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010488 2023-11-25T04:23:34.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.704 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.704 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:34.704 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037588+0000", 2023-11-25T04:23:34.704 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024918309999999999 2023-11-25T04:23:34.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.704 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.704 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:34.704 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044367+0000", 2023-11-25T04:23:34.704 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067787589999999997 2023-11-25T04:23:34.705 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.705 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.705 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:34.705 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044415+0000", 2023-11-25T04:23:34.705 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7803000000000003e-05 2023-11-25T04:23:34.705 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.705 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.705 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:34.705 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044447+0000", 2023-11-25T04:23:34.706 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2886999999999999e-05 2023-11-25T04:23:34.706 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:34.706 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:34.706 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:34.706 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.706 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.706 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:30 3.15 3:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:34.706 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:34.707 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.586363835, 2023-11-25T04:23:34.707 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010365070000000001, 2023-11-25T04:23:34.707 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:34.707 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:34.707 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:34.707 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.707 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:34.707 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:34.707 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:34.707 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.708 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.708 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:34.708 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:34.708 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:34.708 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.708 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.708 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:34.708 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034225+0000", 2023-11-25T04:23:34.708 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7400000000000002e-07 2023-11-25T04:23:34.709 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.709 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.709 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:34.709 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034229+0000", 2023-11-25T04:23:34.709 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3100000000000002e-06 2023-11-25T04:23:34.709 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.710 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.710 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:34.710 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034239+0000", 2023-11-25T04:23:34.710 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9140000000000003e-06 2023-11-25T04:23:34.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.710 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.710 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:34.710 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034246+0000", 2023-11-25T04:23:34.710 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0149999999999998e-06 2023-11-25T04:23:34.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.711 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:34.711 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035572+0000", 2023-11-25T04:23:34.711 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001325655 2023-11-25T04:23:34.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.711 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:34.711 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035730+0000", 2023-11-25T04:23:34.712 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015827899999999999 2023-11-25T04:23:34.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.712 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.712 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:34.712 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035857+0000", 2023-11-25T04:23:34.712 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012644699999999999 2023-11-25T04:23:34.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.712 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.712 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:34.712 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038737+0000", 2023-11-25T04:23:34.713 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002880077 2023-11-25T04:23:34.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.713 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:34.713 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044540+0000", 2023-11-25T04:23:34.713 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058034239999999997 2023-11-25T04:23:34.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.713 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:34.714 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044569+0000", 2023-11-25T04:23:34.714 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8388999999999999e-05 2023-11-25T04:23:34.714 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.714 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.714 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:34.714 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044589+0000", 2023-11-25T04:23:34.714 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0686000000000002e-05 2023-11-25T04:23:34.714 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:34.714 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:34.715 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:34.715 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.715 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.715 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:33 3.1b 3:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:34.715 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:34.715 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.586318337000002, 2023-11-25T04:23:34.715 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099754049999999997, 2023-11-25T04:23:34.715 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:34.715 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:23:34.716 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:34.716 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.716 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:34.716 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:34.716 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:34.716 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.716 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.716 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:34.716 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:34.717 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:34.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.717 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:34.717 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:34.717 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4799999999999998e-07 2023-11-25T04:23:34.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.717 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:34.717 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034273+0000", 2023-11-25T04:23:34.718 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7819999999999998e-06 2023-11-25T04:23:34.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.718 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:34.718 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034285+0000", 2023-11-25T04:23:34.718 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2058e-05 2023-11-25T04:23:34.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.718 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:34.718 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034291+0000", 2023-11-25T04:23:34.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0680000000000002e-06 2023-11-25T04:23:34.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.719 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.719 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:34.719 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034370+0000", 2023-11-25T04:23:34.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8452e-05 2023-11-25T04:23:34.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.719 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.719 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:34.720 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034477+0000", 2023-11-25T04:23:34.720 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107471 2023-11-25T04:23:34.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.720 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.720 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:34.720 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044245+0000", 2023-11-25T04:23:34.720 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0097679259999999997 2023-11-25T04:23:34.720 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:34.720 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:34.721 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:34.721 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.721 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.721 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:36 3.1a 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:34.721 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:34.721 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.586271664000002, 2023-11-25T04:23:34.721 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058179099999999999, 2023-11-25T04:23:34.721 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:34.721 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:23:34.721 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:34.722 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.722 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:34.722 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:34.722 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:34.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.722 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.722 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:34.722 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:34.722 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:34.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.723 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:34.723 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034317+0000", 2023-11-25T04:23:34.723 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1999999999999999e-07 2023-11-25T04:23:34.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.723 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:34.723 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034321+0000", 2023-11-25T04:23:34.723 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.233e-06 2023-11-25T04:23:34.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.724 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.724 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:34.724 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034337+0000", 2023-11-25T04:23:34.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5308999999999999e-05 2023-11-25T04:23:34.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.724 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.724 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:34.724 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034354+0000", 2023-11-25T04:23:34.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7411999999999999e-05 2023-11-25T04:23:34.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:34.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034465+0000", 2023-11-25T04:23:34.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011154699999999999 2023-11-25T04:23:34.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:34.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034560+0000", 2023-11-25T04:23:34.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4186999999999998e-05 2023-11-25T04:23:34.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.726 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:34.726 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.040134+0000", 2023-11-25T04:23:34.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055746019999999997 2023-11-25T04:23:34.726 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:34.726 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:34.726 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:34.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.727 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:9 3.10 e18/16 3:09401a81:::601.00000000:head v 18'4, mlcod=0'0)", 2023-11-25T04:23:34.727 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:34.727 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.577377356, 2023-11-25T04:23:34.727 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061641020000000003, 2023-11-25T04:23:34.727 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:34.727 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:34.727 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:34.727 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.727 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:34.728 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:34.728 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:34.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.728 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:34.728 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:34.728 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:34.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.729 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:34.729 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:34.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5300000000000001e-07 2023-11-25T04:23:34.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.729 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:34.729 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043222+0000", 2023-11-25T04:23:34.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.082e-05 2023-11-25T04:23:34.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.730 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:34.730 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043228+0000", 2023-11-25T04:23:34.730 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6699999999999997e-06 2023-11-25T04:23:34.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.730 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:34.730 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043236+0000", 2023-11-25T04:23:34.730 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8189999999999992e-06 2023-11-25T04:23:34.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.731 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:34.731 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043507+0000", 2023-11-25T04:23:34.731 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000271012 2023-11-25T04:23:34.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.731 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:34.731 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043530+0000", 2023-11-25T04:23:34.731 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2659000000000001e-05 2023-11-25T04:23:34.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:34.732 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049321+0000", 2023-11-25T04:23:34.732 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057915129999999999 2023-11-25T04:23:34.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:34.732 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049375+0000", 2023-11-25T04:23:34.732 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3455999999999998e-05 2023-11-25T04:23:34.733 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:34.733 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:34.733 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:34.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.733 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:1 3.14 e18/16 3:292cf221:::200.00000000:head v 18'3, mlcod=0'0)", 2023-11-25T04:23:34.733 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:34.733 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.577340642999999, 2023-11-25T04:23:34.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0060983599999999997, 2023-11-25T04:23:34.733 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:34.734 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:34.734 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:34.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.734 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:34.734 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:34.734 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:34.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.734 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:34.735 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:34.735 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:34.735 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.735 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.735 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:34.735 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043248+0000", 2023-11-25T04:23:34.735 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9399999999999995e-07 2023-11-25T04:23:34.735 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.735 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.735 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:34.736 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043252+0000", 2023-11-25T04:23:34.736 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1799999999999998e-06 2023-11-25T04:23:34.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.736 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.736 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:34.736 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043259+0000", 2023-11-25T04:23:34.736 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0049999999999998e-06 2023-11-25T04:23:34.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:34.737 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043265+0000", 2023-11-25T04:23:34.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0859999999999998e-06 2023-11-25T04:23:34.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:34.737 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043311+0000", 2023-11-25T04:23:34.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6082999999999999e-05 2023-11-25T04:23:34.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:34.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043330+0000", 2023-11-25T04:23:34.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8556e-05 2023-11-25T04:23:34.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:34.739 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049303+0000", 2023-11-25T04:23:34.739 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059733470000000004 2023-11-25T04:23:34.739 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.739 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.739 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:34.739 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049346+0000", 2023-11-25T04:23:34.739 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2608999999999998e-05 2023-11-25T04:23:34.739 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:34.739 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:34.740 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:34.740 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.740 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.740 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:3 3.14 e18/16 3:287034c4:::1.00000000.inode:head v 18'4, mlcod=0'0)", 2023-11-25T04:23:34.740 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:34.740 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.576824137000003, 2023-11-25T04:23:34.740 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057040090000000003, 2023-11-25T04:23:34.740 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:34.741 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:34.741 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:34.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:34.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:34.741 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:34.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:34.742 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:34.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:34.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.742 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:34.742 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:34.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9799999999999999e-07 2023-11-25T04:23:34.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.743 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:34.743 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043785+0000", 2023-11-25T04:23:34.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0721999999999999e-05 2023-11-25T04:23:34.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.743 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:34.743 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043793+0000", 2023-11-25T04:23:34.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5839999999999995e-06 2023-11-25T04:23:34.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.744 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:34.744 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043803+0000", 2023-11-25T04:23:34.744 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0729999999999994e-06 2023-11-25T04:23:34.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.744 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:34.744 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043843+0000", 2023-11-25T04:23:34.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0747999999999997e-05 2023-11-25T04:23:34.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:34.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043856+0000", 2023-11-25T04:23:34.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2697000000000001e-05 2023-11-25T04:23:34.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:34.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049426+0000", 2023-11-25T04:23:34.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055704409999999998 2023-11-25T04:23:34.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:34.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049468+0000", 2023-11-25T04:23:34.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1446000000000003e-05 2023-11-25T04:23:34.747 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:34.747 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:34.747 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:34.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.747 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.747 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "pg_update_log_missing(3.14 epoch 18/16 rep_tid 171 entries 18'5 (0'0) error 3:2e2fa760:::200.00000001:head by mds.0.3:32 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:23:34.747 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:34.747 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.576663173999997, 2023-11-25T04:23:34.748 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005556938, 2023-11-25T04:23:34.748 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:34.748 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2023-11-25T04:23:34.748 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:34.748 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.748 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:34.748 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:34.748 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:34.748 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.748 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.749 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:34.749 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:34.749 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:34.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.749 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.749 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:34.749 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:34.749 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.03e-07 2023-11-25T04:23:34.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:34.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043931+0000", 2023-11-25T04:23:34.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1079999999999998e-06 2023-11-25T04:23:34.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:34.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043944+0000", 2023-11-25T04:23:34.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2852e-05 2023-11-25T04:23:34.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:34.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043955+0000", 2023-11-25T04:23:34.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1037e-05 2023-11-25T04:23:34.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:34.752 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044088+0000", 2023-11-25T04:23:34.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013269599999999999 2023-11-25T04:23:34.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.752 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:34.752 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049482+0000", 2023-11-25T04:23:34.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053940419999999999 2023-11-25T04:23:34.752 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:34.752 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:34.752 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:34.753 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.753 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.753 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:10 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'5, mlcod=0'0)", 2023-11-25T04:23:34.753 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:34.753 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.576573146000001, 2023-11-25T04:23:34.753 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054493099999999997, 2023-11-25T04:23:34.753 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:34.753 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:34.753 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:34.753 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.754 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:34.754 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:34.754 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:34.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.754 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:34.754 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:34.754 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:34.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:34.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:34.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6100000000000002e-07 2023-11-25T04:23:34.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:34.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044019+0000", 2023-11-25T04:23:34.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.839e-06 2023-11-25T04:23:34.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.756 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.756 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:34.756 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044026+0000", 2023-11-25T04:23:34.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5869999999999998e-06 2023-11-25T04:23:34.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.756 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.756 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:34.756 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044031+0000", 2023-11-25T04:23:34.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4909999999999996e-06 2023-11-25T04:23:34.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.757 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.757 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:34.757 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044070+0000", 2023-11-25T04:23:34.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8829999999999999e-05 2023-11-25T04:23:34.757 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.757 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.757 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:34.757 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044081+0000", 2023-11-25T04:23:34.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1447999999999999e-05 2023-11-25T04:23:34.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.758 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:34.758 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049406+0000", 2023-11-25T04:23:34.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053246480000000004 2023-11-25T04:23:34.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.758 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:34.758 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049464+0000", 2023-11-25T04:23:34.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8205999999999998e-05 2023-11-25T04:23:34.759 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:34.759 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:34.759 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:34.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.759 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:11 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'6, mlcod=0'0)", 2023-11-25T04:23:34.759 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:34.759 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 47.576371881999997, 2023-11-25T04:23:34.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052673499999999996, 2023-11-25T04:23:34.759 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:34.760 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:34.760 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:34.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:34.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:34.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:34.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:34.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:34.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:34.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:34.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:34.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0199999999999998e-07 2023-11-25T04:23:34.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:34.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044220+0000", 2023-11-25T04:23:34.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7720000000000001e-06 2023-11-25T04:23:34.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.762 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:34.762 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044226+0000", 2023-11-25T04:23:34.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8579999999999996e-06 2023-11-25T04:23:34.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.762 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:34.762 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044232+0000", 2023-11-25T04:23:34.763 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1700000000000002e-06 2023-11-25T04:23:34.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.763 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:34.763 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044274+0000", 2023-11-25T04:23:34.763 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1254999999999997e-05 2023-11-25T04:23:34.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.763 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:34.763 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:23:34.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6880999999999999e-05 2023-11-25T04:23:34.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:34.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049466+0000", 2023-11-25T04:23:34.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051659219999999999 2023-11-25T04:23:34.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:34.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:34.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:34.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049483+0000", 2023-11-25T04:23:34.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.719e-05 2023-11-25T04:23:34.765 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:34.765 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:34.765 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:34.765 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:34.765 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:34.765 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:34.766 INFO:teuthology.orchestra.run.smithi067.stderr:Error ENXIO: problem getting command descriptions from osd.0 2023-11-25T04:23:34.767 INFO:teuthology.orchestra.run.smithi067.stdout:osd.1: {} 2023-11-25T04:23:34.768 INFO:teuthology.orchestra.run.smithi067.stderr:osd.1: osd_enable_op_tracker = '' 2023-11-25T04:23:34.768 INFO:teuthology.orchestra.run.smithi067.stdout:osd.2: {} 2023-11-25T04:23:34.768 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:23:34.769 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:34.765+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:34.769 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-11-25T04:23:34.770 INFO:teuthology.orchestra.run.smithi067.stderr:osd.2: osd_enable_op_tracker = '' 2023-11-25T04:23:34.770 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-11-25T04:23:34.867 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:34.866+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:34.883 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:34.883 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:34.892 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:23:34.918 INFO:teuthology.orchestra.run.smithi067.stderr:noscrub is unset 2023-11-25T04:23:34.976 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:34.975+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:35.039 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:35.039 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:35.048 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:23:35.069 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:35.067+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:35.170 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:35.169+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:35.194 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:35.194 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:35.203 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:23:35.271 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:35.270+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:35.309 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:35.309 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:35.318 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:23:35.367 DEBUG:teuthology.orchestra.console.smithi076:pexpect command: ipmitool -H smithi076.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-25T04:23:35.376 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:35.375+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:35.423 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:35.423 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:35.432 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:23:35.475 DEBUG:teuthology.orchestra.console.smithi076:check power output: Chassis Power is on 2023-11-25T04:23:35.476 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:35.474+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:35.537 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:35.537 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:35.546 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:23:35.577 INFO:teuthology.orchestra.console.smithi076:Power on completed 2023-11-25T04:23:35.586 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:35.585+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:35.653 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:35.653 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:35.662 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:23:35.678 DEBUG:teuthology.orchestra.console.smithi076:Waiting for login prompt 2023-11-25T04:23:35.678 DEBUG:teuthology.orchestra.console.smithi076:pexpect command: console -M conserver.front.sepia.ceph.com -p 3109 -f smithi076 2023-11-25T04:23:35.686 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:35.677+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:35.767 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:35.767 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:35.776 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:23:35.819 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:35.818+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:35.880 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:35.879+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:35.881 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:35.881 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:35.891 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:23:35.981 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:35.980+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:36.001 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:36.002 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:36.010 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:23:36.082 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:36.081+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:36.115 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:36.115 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:36.123 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:23:36.183 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:36.182+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:36.269 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:36.269 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:36.278 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:23:36.284 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:36.283+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:36.385 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:36.383+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:36.386 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:36.386 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:36.395 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:23:36.485 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:36.484+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:36.540 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:36.540 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:36.549 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:23:36.592 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:36.591+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:36.655 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:36.655 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:36.664 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:23:36.705 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:36.703+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:36.770 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:36.770 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:36.771 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-11-25T04:23:36.779 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:23:36.824 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:36.822+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:36.883 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:36.884 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:36.886 INFO:teuthology.orchestra.run.smithi067.stderr:Error ENXIO: problem getting command descriptions from osd.0 2023-11-25T04:23:36.890 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:36.889+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:36.892 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:23:36.921 INFO:teuthology.orchestra.run.smithi067.stdout:osd.1: {} 2023-11-25T04:23:36.921 INFO:teuthology.orchestra.run.smithi067.stderr:osd.1: osd_enable_op_tracker = '' 2023-11-25T04:23:36.935 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2023-11-25T04:23:36.956 INFO:teuthology.orchestra.run.smithi067.stdout:osd.2: {} 2023-11-25T04:23:36.956 INFO:teuthology.orchestra.run.smithi067.stderr:osd.2: osd_enable_op_tracker = '' 2023-11-25T04:23:36.961 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-11-25T04:23:36.961 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-11-25T04:23:36.991 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:36.989+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:37.040 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:37.040 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2023-11-25T04:23:37.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2023-11-25T04:23:37.040 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2023-11-25T04:23:37.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.041 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:155 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'109, mlcod=11'105)", 2023-11-25T04:23:37.041 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:37.041 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 59.252921067000003, 2023-11-25T04:23:37.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061719019999999999, 2023-11-25T04:23:37.041 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:37.041 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:37.041 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:37.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:37.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:37.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:37.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:37.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:37.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:37.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:37.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784884+0000", 2023-11-25T04:23:37.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3099999999999999e-07 2023-11-25T04:23:37.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:37.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784942+0000", 2023-11-25T04:23:37.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8570000000000003e-05 2023-11-25T04:23:37.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:37.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784949+0000", 2023-11-25T04:23:37.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0310000000000001e-06 2023-11-25T04:23:37.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:37.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784953+0000", 2023-11-25T04:23:37.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6320000000000001e-06 2023-11-25T04:23:37.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:37.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784993+0000", 2023-11-25T04:23:37.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9737000000000003e-05 2023-11-25T04:23:37.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:37.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785011+0000", 2023-11-25T04:23:37.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8349999999999999e-05 2023-11-25T04:23:37.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:37.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791024+0000", 2023-11-25T04:23:37.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006012997 2023-11-25T04:23:37.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:37.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791055+0000", 2023-11-25T04:23:37.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1254000000000003e-05 2023-11-25T04:23:37.045 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:37.045 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:37.045 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:37.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.046 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:156 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'110, mlcod=11'105)", 2023-11-25T04:23:37.046 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:37.046 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 59.252335017999997, 2023-11-25T04:23:37.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005616116, 2023-11-25T04:23:37.046 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:37.046 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:37.046 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:37.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:37.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:37.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:37.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:37.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:37.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:37.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:37.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785470+0000", 2023-11-25T04:23:37.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0699999999999998e-07 2023-11-25T04:23:37.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:37.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785474+0000", 2023-11-25T04:23:37.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.048e-06 2023-11-25T04:23:37.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:37.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785480+0000", 2023-11-25T04:23:37.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7729999999999996e-06 2023-11-25T04:23:37.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:37.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785483+0000", 2023-11-25T04:23:37.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1970000000000001e-06 2023-11-25T04:23:37.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:37.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787936+0000", 2023-11-25T04:23:37.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024529040000000001 2023-11-25T04:23:37.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:37.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787947+0000", 2023-11-25T04:23:37.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1669e-05 2023-11-25T04:23:37.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:37.050 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791057+0000", 2023-11-25T04:23:37.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031095200000000002 2023-11-25T04:23:37.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:37.050 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791086+0000", 2023-11-25T04:23:37.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8697999999999999e-05 2023-11-25T04:23:37.050 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:37.050 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:37.050 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:37.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.051 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:157 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'111, mlcod=11'106)", 2023-11-25T04:23:37.051 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:37.051 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 59.251742460000003, 2023-11-25T04:23:37.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050590280000000001, 2023-11-25T04:23:37.051 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:37.051 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:37.051 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:37.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:37.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:37.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:37.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:37.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:37.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:37.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:37.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786063+0000", 2023-11-25T04:23:37.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2600000000000002e-07 2023-11-25T04:23:37.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:37.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786069+0000", 2023-11-25T04:23:37.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8839999999999999e-06 2023-11-25T04:23:37.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.054 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:37.054 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786079+0000", 2023-11-25T04:23:37.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0124e-05 2023-11-25T04:23:37.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.054 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:37.054 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786085+0000", 2023-11-25T04:23:37.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6660000000000002e-06 2023-11-25T04:23:37.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.054 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:37.054 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788160+0000", 2023-11-25T04:23:37.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020747669999999999 2023-11-25T04:23:37.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:37.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788180+0000", 2023-11-25T04:23:37.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0208999999999999e-05 2023-11-25T04:23:37.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:37.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791087+0000", 2023-11-25T04:23:37.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029064630000000002 2023-11-25T04:23:37.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:37.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791121+0000", 2023-11-25T04:23:37.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4289000000000003e-05 2023-11-25T04:23:37.056 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:37.056 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:37.056 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:37.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.056 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:158 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'112, mlcod=11'106)", 2023-11-25T04:23:37.056 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:37.056 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 59.251326067000001, 2023-11-25T04:23:37.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046663030000000001, 2023-11-25T04:23:37.056 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:37.057 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:37.057 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:37.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:37.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:37.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:37.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:37.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:37.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:37.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:37.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786479+0000", 2023-11-25T04:23:37.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2799999999999996e-07 2023-11-25T04:23:37.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:37.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786498+0000", 2023-11-25T04:23:37.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9069e-05 2023-11-25T04:23:37.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:37.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786508+0000", 2023-11-25T04:23:37.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5810000000000003e-06 2023-11-25T04:23:37.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:37.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786512+0000", 2023-11-25T04:23:37.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4889999999999997e-06 2023-11-25T04:23:37.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.060 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:37.060 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788438+0000", 2023-11-25T04:23:37.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019257059999999999 2023-11-25T04:23:37.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.060 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:37.060 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788449+0000", 2023-11-25T04:23:37.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1419e-05 2023-11-25T04:23:37.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:37.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791122+0000", 2023-11-25T04:23:37.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002672822 2023-11-25T04:23:37.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:37.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791145+0000", 2023-11-25T04:23:37.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2589e-05 2023-11-25T04:23:37.061 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:37.061 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:37.061 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:37.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.062 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:206 1.0 e12/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 12'152, mlcod=12'148)", 2023-11-25T04:23:37.062 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:37.062 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 58.739557597000001, 2023-11-25T04:23:37.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046728189999999999, 2023-11-25T04:23:37.062 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:37.062 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:37.062 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:37.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:37.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:37.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:37.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:37.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:37.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:37.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:37.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298248+0000", 2023-11-25T04:23:37.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3600000000000003e-07 2023-11-25T04:23:37.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:37.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298278+0000", 2023-11-25T04:23:37.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0474e-05 2023-11-25T04:23:37.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:37.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298288+0000", 2023-11-25T04:23:37.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0156000000000001e-05 2023-11-25T04:23:37.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:37.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298293+0000", 2023-11-25T04:23:37.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7400000000000004e-06 2023-11-25T04:23:37.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:37.065 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298333+0000", 2023-11-25T04:23:37.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9666000000000003e-05 2023-11-25T04:23:37.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:37.065 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298351+0000", 2023-11-25T04:23:37.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8544999999999999e-05 2023-11-25T04:23:37.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:37.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302891+0000", 2023-11-25T04:23:37.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045398519999999996 2023-11-25T04:23:37.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:37.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302920+0000", 2023-11-25T04:23:37.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8750000000000001e-05 2023-11-25T04:23:37.066 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:37.066 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:37.067 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:37.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.067 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:14 3.15 3:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:37.067 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:37.067 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.004423240000001, 2023-11-25T04:23:37.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010990437, 2023-11-25T04:23:37.067 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:37.067 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:37.067 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:37.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:37.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:37.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:37.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:37.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:37.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:37.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:37.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033383+0000", 2023-11-25T04:23:37.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9870000000000002e-06 2023-11-25T04:23:37.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.069 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:37.069 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033398+0000", 2023-11-25T04:23:37.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4569999999999999e-05 2023-11-25T04:23:37.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.069 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:37.069 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033411+0000", 2023-11-25T04:23:37.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3543999999999999e-05 2023-11-25T04:23:37.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:37.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033423+0000", 2023-11-25T04:23:37.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1905999999999999e-05 2023-11-25T04:23:37.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:37.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034433+0000", 2023-11-25T04:23:37.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00101022 2023-11-25T04:23:37.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:37.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034539+0000", 2023-11-25T04:23:37.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010597199999999999 2023-11-25T04:23:37.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:37.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034667+0000", 2023-11-25T04:23:37.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012765500000000001 2023-11-25T04:23:37.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:37.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037803+0000", 2023-11-25T04:23:37.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031362220000000001 2023-11-25T04:23:37.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:37.072 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:23:37.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0064964669999999997 2023-11-25T04:23:37.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:37.072 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044331+0000", 2023-11-25T04:23:37.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1544999999999997e-05 2023-11-25T04:23:37.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:37.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044372+0000", 2023-11-25T04:23:37.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0349000000000003e-05 2023-11-25T04:23:37.073 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:37.073 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:37.073 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:37.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.073 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:15 3.15 3:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:37.073 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:37.073 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.003763589999998, 2023-11-25T04:23:37.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010464362, 2023-11-25T04:23:37.073 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:37.074 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:37.074 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:37.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:37.074 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:37.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:37.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:37.074 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:37.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:37.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:37.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034042+0000", 2023-11-25T04:23:37.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0999999999999997e-07 2023-11-25T04:23:37.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:37.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034048+0000", 2023-11-25T04:23:37.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0390000000000001e-06 2023-11-25T04:23:37.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:37.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034059+0000", 2023-11-25T04:23:37.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1182999999999999e-05 2023-11-25T04:23:37.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:37.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034067+0000", 2023-11-25T04:23:37.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1559999999999995e-06 2023-11-25T04:23:37.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:37.077 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035005+0000", 2023-11-25T04:23:37.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00093747399999999999 2023-11-25T04:23:37.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.077 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:37.077 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035077+0000", 2023-11-25T04:23:37.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2747e-05 2023-11-25T04:23:37.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.077 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:37.077 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035219+0000", 2023-11-25T04:23:37.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000141877 2023-11-25T04:23:37.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.078 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:37.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037810+0000", 2023-11-25T04:23:37.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002590752 2023-11-25T04:23:37.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.078 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:37.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044437+0000", 2023-11-25T04:23:37.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066270110000000004 2023-11-25T04:23:37.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:37.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044481+0000", 2023-11-25T04:23:37.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4106000000000001e-05 2023-11-25T04:23:37.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:37.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044505+0000", 2023-11-25T04:23:37.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4207e-05 2023-11-25T04:23:37.079 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:37.079 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:37.079 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:37.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.080 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:16 3.1d 3:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:37.080 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:37.080 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.003721742000003, 2023-11-25T04:23:37.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010413923, 2023-11-25T04:23:37.080 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:37.080 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:37.080 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:37.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.080 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:37.080 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:37.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:37.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:37.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:37.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:37.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:37.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:37.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4499999999999997e-07 2023-11-25T04:23:37.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:37.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034087+0000", 2023-11-25T04:23:37.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9770000000000002e-06 2023-11-25T04:23:37.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:37.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034094+0000", 2023-11-25T04:23:37.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8809999999999998e-06 2023-11-25T04:23:37.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.083 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:37.083 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034100+0000", 2023-11-25T04:23:37.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8440000000000003e-06 2023-11-25T04:23:37.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.083 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:37.083 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034993+0000", 2023-11-25T04:23:37.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00089336799999999996 2023-11-25T04:23:37.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:37.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035099+0000", 2023-11-25T04:23:37.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106037 2023-11-25T04:23:37.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:37.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035276+0000", 2023-11-25T04:23:37.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017631799999999999 2023-11-25T04:23:37.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:37.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037815+0000", 2023-11-25T04:23:37.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025395959999999999 2023-11-25T04:23:37.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:37.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044420+0000", 2023-11-25T04:23:37.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066045749999999997 2023-11-25T04:23:37.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:37.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044463+0000", 2023-11-25T04:23:37.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3205999999999999e-05 2023-11-25T04:23:37.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.086 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:37.086 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044497+0000", 2023-11-25T04:23:37.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3476000000000003e-05 2023-11-25T04:23:37.086 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:37.086 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:37.086 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:37.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.086 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:17 3.1d 3:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:37.086 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:37.086 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.003696533000003, 2023-11-25T04:23:37.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01127356, 2023-11-25T04:23:37.087 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:37.087 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:37.087 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:37.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.087 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:37.087 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:37.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:37.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.087 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:37.087 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:37.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:37.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:37.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:37.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1100000000000001e-07 2023-11-25T04:23:37.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:37.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034112+0000", 2023-11-25T04:23:37.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8600000000000003e-06 2023-11-25T04:23:37.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:37.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034120+0000", 2023-11-25T04:23:37.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7700000000000001e-06 2023-11-25T04:23:37.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:37.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034127+0000", 2023-11-25T04:23:37.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0999999999999998e-06 2023-11-25T04:23:37.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:37.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035601+0000", 2023-11-25T04:23:37.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014739410000000001 2023-11-25T04:23:37.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:37.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035685+0000", 2023-11-25T04:23:37.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3677999999999994e-05 2023-11-25T04:23:37.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:37.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035827+0000", 2023-11-25T04:23:37.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014232599999999999 2023-11-25T04:23:37.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:37.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038725+0000", 2023-11-25T04:23:37.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028982779999999998 2023-11-25T04:23:37.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.092 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:37.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045323+0000", 2023-11-25T04:23:37.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0065980780000000003 2023-11-25T04:23:37.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.092 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:37.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045354+0000", 2023-11-25T04:23:37.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0463e-05 2023-11-25T04:23:37.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:37.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045381+0000", 2023-11-25T04:23:37.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7654999999999999e-05 2023-11-25T04:23:37.093 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:37.093 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:37.093 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:37.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.093 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:22 3.e 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:37.093 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:37.093 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.003670120000002, 2023-11-25T04:23:37.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010133422, 2023-11-25T04:23:37.094 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:37.094 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:37.094 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:37.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:37.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:37.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:37.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:37.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:37.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:37.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:37.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034135+0000", 2023-11-25T04:23:37.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-07 2023-11-25T04:23:37.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:37.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034138+0000", 2023-11-25T04:23:37.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6009999999999999e-06 2023-11-25T04:23:37.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:37.096 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034145+0000", 2023-11-25T04:23:37.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1169999999999998e-06 2023-11-25T04:23:37.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:37.096 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034150+0000", 2023-11-25T04:23:37.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9080000000000003e-06 2023-11-25T04:23:37.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:37.096 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034209+0000", 2023-11-25T04:23:37.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8149000000000002e-05 2023-11-25T04:23:37.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:37.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034415+0000", 2023-11-25T04:23:37.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020637699999999999 2023-11-25T04:23:37.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:37.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034584+0000", 2023-11-25T04:23:37.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016870399999999999 2023-11-25T04:23:37.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:37.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.036979+0000", 2023-11-25T04:23:37.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023957509999999998 2023-11-25T04:23:37.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.098 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:37.098 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044161+0000", 2023-11-25T04:23:37.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0071811590000000003 2023-11-25T04:23:37.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.098 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:37.098 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044224+0000", 2023-11-25T04:23:37.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3454000000000006e-05 2023-11-25T04:23:37.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.099 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:37.099 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044268+0000", 2023-11-25T04:23:37.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3761999999999999e-05 2023-11-25T04:23:37.099 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:37.099 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:37.099 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:37.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.099 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:23 3.e 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:37.099 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:37.099 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.003638711000001, 2023-11-25T04:23:37.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010281676, 2023-11-25T04:23:37.100 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:37.100 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:37.100 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:37.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:37.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:37.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:37.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:37.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:37.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:37.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:37.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:37.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5799999999999999e-07 2023-11-25T04:23:37.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:37.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034169+0000", 2023-11-25T04:23:37.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1879999999999998e-06 2023-11-25T04:23:37.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:37.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034177+0000", 2023-11-25T04:23:37.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7020000000000001e-06 2023-11-25T04:23:37.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.102 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:37.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034184+0000", 2023-11-25T04:23:37.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3910000000000003e-06 2023-11-25T04:23:37.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.102 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:37.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034924+0000", 2023-11-25T04:23:37.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00074032999999999998 2023-11-25T04:23:37.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.103 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:37.103 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034991+0000", 2023-11-25T04:23:37.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7347000000000004e-05 2023-11-25T04:23:37.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.103 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:37.103 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035096+0000", 2023-11-25T04:23:37.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010488 2023-11-25T04:23:37.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.103 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:37.103 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037588+0000", 2023-11-25T04:23:37.104 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024918309999999999 2023-11-25T04:23:37.104 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.104 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:37.104 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044367+0000", 2023-11-25T04:23:37.104 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067787589999999997 2023-11-25T04:23:37.104 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.104 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:37.104 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044415+0000", 2023-11-25T04:23:37.104 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7803000000000003e-05 2023-11-25T04:23:37.104 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.104 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:37.105 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044447+0000", 2023-11-25T04:23:37.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2886999999999999e-05 2023-11-25T04:23:37.105 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:37.105 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:37.105 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:37.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.105 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:30 3.15 3:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:37.105 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:37.105 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.003580270999997, 2023-11-25T04:23:37.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010365070000000001, 2023-11-25T04:23:37.105 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:37.106 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:37.106 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:37.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.106 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:37.106 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:37.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:37.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.106 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:37.106 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:37.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:37.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.107 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:37.107 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034225+0000", 2023-11-25T04:23:37.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7400000000000002e-07 2023-11-25T04:23:37.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.107 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:37.107 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034229+0000", 2023-11-25T04:23:37.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3100000000000002e-06 2023-11-25T04:23:37.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.107 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:37.107 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034239+0000", 2023-11-25T04:23:37.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9140000000000003e-06 2023-11-25T04:23:37.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.108 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:37.109 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034246+0000", 2023-11-25T04:23:37.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0149999999999998e-06 2023-11-25T04:23:37.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.110 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.110 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:37.110 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035572+0000", 2023-11-25T04:23:37.110 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001325655 2023-11-25T04:23:37.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.110 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.110 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:37.110 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035730+0000", 2023-11-25T04:23:37.110 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015827899999999999 2023-11-25T04:23:37.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.110 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.111 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:37.111 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035857+0000", 2023-11-25T04:23:37.111 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012644699999999999 2023-11-25T04:23:37.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.111 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.111 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:37.111 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038737+0000", 2023-11-25T04:23:37.111 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002880077 2023-11-25T04:23:37.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.111 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.111 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:37.111 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044540+0000", 2023-11-25T04:23:37.111 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058034239999999997 2023-11-25T04:23:37.112 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.112 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.112 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:37.112 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044569+0000", 2023-11-25T04:23:37.112 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8388999999999999e-05 2023-11-25T04:23:37.112 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.112 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.112 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:37.112 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044589+0000", 2023-11-25T04:23:37.112 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0686000000000002e-05 2023-11-25T04:23:37.112 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:37.112 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:37.112 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:37.112 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.113 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:33 3.1b 3:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:37.113 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:37.113 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.003534772999998, 2023-11-25T04:23:37.113 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099754049999999997, 2023-11-25T04:23:37.113 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:37.113 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:23:37.113 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:37.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.113 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:37.113 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:37.113 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:37.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.114 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.114 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:37.114 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:37.114 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:37.114 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.114 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.114 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:37.114 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:37.114 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4799999999999998e-07 2023-11-25T04:23:37.114 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.114 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.114 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:37.115 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034273+0000", 2023-11-25T04:23:37.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7819999999999998e-06 2023-11-25T04:23:37.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.115 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:37.115 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034285+0000", 2023-11-25T04:23:37.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2058e-05 2023-11-25T04:23:37.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.115 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:37.115 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034291+0000", 2023-11-25T04:23:37.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0680000000000002e-06 2023-11-25T04:23:37.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.116 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:37.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034370+0000", 2023-11-25T04:23:37.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8452e-05 2023-11-25T04:23:37.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.116 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:37.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034477+0000", 2023-11-25T04:23:37.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107471 2023-11-25T04:23:37.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.116 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:37.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044245+0000", 2023-11-25T04:23:37.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0097679259999999997 2023-11-25T04:23:37.117 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:37.117 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:37.117 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:37.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.117 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:36 3.1a 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:37.117 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:37.117 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.003488099999998, 2023-11-25T04:23:37.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058179099999999999, 2023-11-25T04:23:37.117 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:37.117 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:23:37.117 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:37.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.118 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:37.118 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:37.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:37.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.118 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:37.118 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:37.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:37.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.118 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:37.118 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034317+0000", 2023-11-25T04:23:37.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1999999999999999e-07 2023-11-25T04:23:37.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:37.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034321+0000", 2023-11-25T04:23:37.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.233e-06 2023-11-25T04:23:37.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:37.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034337+0000", 2023-11-25T04:23:37.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5308999999999999e-05 2023-11-25T04:23:37.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.120 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:37.120 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034354+0000", 2023-11-25T04:23:37.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7411999999999999e-05 2023-11-25T04:23:37.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.120 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:37.120 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034465+0000", 2023-11-25T04:23:37.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011154699999999999 2023-11-25T04:23:37.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.120 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:37.120 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034560+0000", 2023-11-25T04:23:37.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4186999999999998e-05 2023-11-25T04:23:37.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.121 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:37.121 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.040134+0000", 2023-11-25T04:23:37.121 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055746019999999997 2023-11-25T04:23:37.121 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:37.121 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:37.121 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:37.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.121 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:9 3.10 e18/16 3:09401a81:::601.00000000:head v 18'4, mlcod=0'0)", 2023-11-25T04:23:37.121 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:37.121 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 49.994593792000003, 2023-11-25T04:23:37.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061641020000000003, 2023-11-25T04:23:37.122 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:37.122 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:37.122 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:37.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.122 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:37.122 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:37.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:37.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.122 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:37.122 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:37.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:37.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.123 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:37.123 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:37.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5300000000000001e-07 2023-11-25T04:23:37.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.123 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:37.123 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043222+0000", 2023-11-25T04:23:37.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.082e-05 2023-11-25T04:23:37.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.123 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:37.124 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043228+0000", 2023-11-25T04:23:37.124 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6699999999999997e-06 2023-11-25T04:23:37.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.124 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:37.124 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043236+0000", 2023-11-25T04:23:37.124 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8189999999999992e-06 2023-11-25T04:23:37.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.124 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:37.124 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043507+0000", 2023-11-25T04:23:37.124 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000271012 2023-11-25T04:23:37.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.125 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:37.125 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043530+0000", 2023-11-25T04:23:37.125 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2659000000000001e-05 2023-11-25T04:23:37.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.125 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:37.125 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049321+0000", 2023-11-25T04:23:37.125 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057915129999999999 2023-11-25T04:23:37.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.125 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:37.126 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049375+0000", 2023-11-25T04:23:37.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3455999999999998e-05 2023-11-25T04:23:37.126 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:37.126 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:37.126 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:37.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.126 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:1 3.14 e18/16 3:292cf221:::200.00000000:head v 18'3, mlcod=0'0)", 2023-11-25T04:23:37.126 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:37.126 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 49.994557079000003, 2023-11-25T04:23:37.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0060983599999999997, 2023-11-25T04:23:37.126 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:37.126 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:37.127 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:37.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.127 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:37.127 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:37.127 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:37.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.127 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:37.127 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:37.127 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:37.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.127 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:37.128 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043248+0000", 2023-11-25T04:23:37.128 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9399999999999995e-07 2023-11-25T04:23:37.128 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.128 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.128 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:37.128 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043252+0000", 2023-11-25T04:23:37.128 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1799999999999998e-06 2023-11-25T04:23:37.128 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.128 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.128 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:37.128 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043259+0000", 2023-11-25T04:23:37.128 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0049999999999998e-06 2023-11-25T04:23:37.128 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.129 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.129 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:37.129 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043265+0000", 2023-11-25T04:23:37.129 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0859999999999998e-06 2023-11-25T04:23:37.129 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.129 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.129 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:37.129 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043311+0000", 2023-11-25T04:23:37.129 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6082999999999999e-05 2023-11-25T04:23:37.129 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.129 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.129 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:37.129 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043330+0000", 2023-11-25T04:23:37.129 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8556e-05 2023-11-25T04:23:37.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.130 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.130 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:37.130 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049303+0000", 2023-11-25T04:23:37.130 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059733470000000004 2023-11-25T04:23:37.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.130 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.130 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:37.130 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049346+0000", 2023-11-25T04:23:37.130 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2608999999999998e-05 2023-11-25T04:23:37.130 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:37.130 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:37.130 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:37.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.131 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:3 3.14 e18/16 3:287034c4:::1.00000000.inode:head v 18'4, mlcod=0'0)", 2023-11-25T04:23:37.131 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:37.131 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 49.994040572999999, 2023-11-25T04:23:37.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057040090000000003, 2023-11-25T04:23:37.131 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:37.131 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:37.131 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:37.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:37.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:37.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:37.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.132 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:37.132 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:37.132 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:37.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.132 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:37.132 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:37.132 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9799999999999999e-07 2023-11-25T04:23:37.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.132 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:37.133 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043785+0000", 2023-11-25T04:23:37.133 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0721999999999999e-05 2023-11-25T04:23:37.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.133 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.133 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:37.133 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043793+0000", 2023-11-25T04:23:37.133 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5839999999999995e-06 2023-11-25T04:23:37.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.133 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.133 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:37.133 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043803+0000", 2023-11-25T04:23:37.133 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0729999999999994e-06 2023-11-25T04:23:37.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.134 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.134 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:37.134 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043843+0000", 2023-11-25T04:23:37.134 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0747999999999997e-05 2023-11-25T04:23:37.134 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.134 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.134 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:37.134 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043856+0000", 2023-11-25T04:23:37.134 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2697000000000001e-05 2023-11-25T04:23:37.134 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.134 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.134 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:37.134 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049426+0000", 2023-11-25T04:23:37.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055704409999999998 2023-11-25T04:23:37.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.135 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:37.135 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049468+0000", 2023-11-25T04:23:37.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1446000000000003e-05 2023-11-25T04:23:37.135 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:37.135 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:37.135 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:37.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.135 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "pg_update_log_missing(3.14 epoch 18/16 rep_tid 171 entries 18'5 (0'0) error 3:2e2fa760:::200.00000001:head by mds.0.3:32 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:23:37.136 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:37.136 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 49.99387961, 2023-11-25T04:23:37.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005556938, 2023-11-25T04:23:37.136 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:37.136 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2023-11-25T04:23:37.136 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:37.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:37.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:37.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:37.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:37.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:37.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:37.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:37.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:37.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.03e-07 2023-11-25T04:23:37.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:37.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043931+0000", 2023-11-25T04:23:37.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1079999999999998e-06 2023-11-25T04:23:37.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.138 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:37.138 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043944+0000", 2023-11-25T04:23:37.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2852e-05 2023-11-25T04:23:37.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.138 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:37.138 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043955+0000", 2023-11-25T04:23:37.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1037e-05 2023-11-25T04:23:37.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.139 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:37.139 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044088+0000", 2023-11-25T04:23:37.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013269599999999999 2023-11-25T04:23:37.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.139 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:37.139 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049482+0000", 2023-11-25T04:23:37.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053940419999999999 2023-11-25T04:23:37.139 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:37.139 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:37.139 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:37.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.139 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:10 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'5, mlcod=0'0)", 2023-11-25T04:23:37.140 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:37.140 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 49.993789581999998, 2023-11-25T04:23:37.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054493099999999997, 2023-11-25T04:23:37.140 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:37.140 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:37.140 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:37.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.140 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:37.140 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:37.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:37.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.141 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.141 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:37.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:37.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:37.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.141 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.141 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:37.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:37.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6100000000000002e-07 2023-11-25T04:23:37.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.142 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:37.142 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044019+0000", 2023-11-25T04:23:37.142 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.839e-06 2023-11-25T04:23:37.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.142 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:37.142 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044026+0000", 2023-11-25T04:23:37.142 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5869999999999998e-06 2023-11-25T04:23:37.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:37.143 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044031+0000", 2023-11-25T04:23:37.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4909999999999996e-06 2023-11-25T04:23:37.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:37.143 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044070+0000", 2023-11-25T04:23:37.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8829999999999999e-05 2023-11-25T04:23:37.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:37.143 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044081+0000", 2023-11-25T04:23:37.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1447999999999999e-05 2023-11-25T04:23:37.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:37.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049406+0000", 2023-11-25T04:23:37.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053246480000000004 2023-11-25T04:23:37.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:37.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049464+0000", 2023-11-25T04:23:37.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8205999999999998e-05 2023-11-25T04:23:37.144 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:37.144 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:37.144 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:37.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.145 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:11 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'6, mlcod=0'0)", 2023-11-25T04:23:37.145 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:37.145 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 49.993588318, 2023-11-25T04:23:37.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052673499999999996, 2023-11-25T04:23:37.145 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:37.145 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:37.145 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:37.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:37.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:37.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:37.146 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.146 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:37.146 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:37.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:37.146 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.146 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:37.146 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:37.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0199999999999998e-07 2023-11-25T04:23:37.146 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.147 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:37.147 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044220+0000", 2023-11-25T04:23:37.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7720000000000001e-06 2023-11-25T04:23:37.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.147 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:37.147 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044226+0000", 2023-11-25T04:23:37.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8579999999999996e-06 2023-11-25T04:23:37.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.147 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:37.147 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044232+0000", 2023-11-25T04:23:37.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1700000000000002e-06 2023-11-25T04:23:37.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.148 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:37.148 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044274+0000", 2023-11-25T04:23:37.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1254999999999997e-05 2023-11-25T04:23:37.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.148 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:37.148 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:23:37.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6880999999999999e-05 2023-11-25T04:23:37.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:37.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049466+0000", 2023-11-25T04:23:37.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051659219999999999 2023-11-25T04:23:37.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:37.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:37.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:37.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049483+0000", 2023-11-25T04:23:37.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.719e-05 2023-11-25T04:23:37.149 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:37.149 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:37.149 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:37.149 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:37.150 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:37.150 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:37.151 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:23:37.152 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:37.150+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:37.252 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:37.250+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:37.259 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:37.260 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:23:37.260 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2023-11-25T04:23:37.260 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:37.268 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:23:37.353 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:37.351+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:37.413 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:37.414 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:23:37.414 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2023-11-25T04:23:37.414 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2023-11-25T04:23:37.414 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:37.422 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:23:37.454 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:37.452+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:37.529 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:37.529 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2023-11-25T04:23:37.529 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2023-11-25T04:23:37.529 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2023-11-25T04:23:37.530 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.530 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:37.530 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:37.530 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 50.495262498000002, 2023-11-25T04:23:37.530 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01072943, 2023-11-25T04:23:37.530 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:37.530 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:37.530 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:37.530 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.530 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:37.530 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:37.530 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:37.531 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.531 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.531 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:37.531 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:37.531 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:37.531 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.531 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.531 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:37.531 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031829+0000", 2023-11-25T04:23:37.531 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.666e-06 2023-11-25T04:23:37.531 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.531 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.531 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:37.531 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031839+0000", 2023-11-25T04:23:37.532 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9490000000000008e-06 2023-11-25T04:23:37.532 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.532 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.532 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:37.532 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031857+0000", 2023-11-25T04:23:37.532 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8093000000000001e-05 2023-11-25T04:23:37.532 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.532 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.532 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:37.532 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031882+0000", 2023-11-25T04:23:37.532 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4294000000000002e-05 2023-11-25T04:23:37.532 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.532 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.533 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:37.533 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033130+0000", 2023-11-25T04:23:37.533 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001248264 2023-11-25T04:23:37.533 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.533 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.533 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:37.533 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033426+0000", 2023-11-25T04:23:37.533 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029620999999999998 2023-11-25T04:23:37.533 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.533 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.533 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:37.533 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033814+0000", 2023-11-25T04:23:37.533 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000387742 2023-11-25T04:23:37.534 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.534 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.534 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:37.534 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036793+0000", 2023-11-25T04:23:37.534 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002979598 2023-11-25T04:23:37.534 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.535 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.535 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:37.535 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042455+0000", 2023-11-25T04:23:37.535 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0056619260000000003 2023-11-25T04:23:37.535 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.535 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.535 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:37.535 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042502+0000", 2023-11-25T04:23:37.535 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7139999999999999e-05 2023-11-25T04:23:37.535 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.535 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.536 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:37.536 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042557+0000", 2023-11-25T04:23:37.536 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4548e-05 2023-11-25T04:23:37.536 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:37.536 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:37.536 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:37.536 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.536 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.552 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:37.552 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:37.552 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 50.494063963000002, 2023-11-25T04:23:37.552 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011505202000000001, 2023-11-25T04:23:37.552 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:37.552 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:37.552 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:37.552 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.552 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:37.552 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:37.552 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:37.552 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.553 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.553 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:37.553 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:37.553 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:37.553 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.553 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.553 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:37.553 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033027+0000", 2023-11-25T04:23:37.553 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0699999999999999e-06 2023-11-25T04:23:37.553 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.553 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.553 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:37.553 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033035+0000", 2023-11-25T04:23:37.553 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0830000000000004e-06 2023-11-25T04:23:37.554 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.554 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.554 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:37.554 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033059+0000", 2023-11-25T04:23:37.554 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3975e-05 2023-11-25T04:23:37.554 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.554 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.554 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:37.554 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033482+0000", 2023-11-25T04:23:37.554 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042253700000000001 2023-11-25T04:23:37.554 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.554 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.554 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:37.555 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034520+0000", 2023-11-25T04:23:37.555 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001037991 2023-11-25T04:23:37.555 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.555 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.555 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:37.555 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034611+0000", 2023-11-25T04:23:37.555 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1113999999999995e-05 2023-11-25T04:23:37.555 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.555 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.555 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:37.555 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034769+0000", 2023-11-25T04:23:37.555 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000157641 2023-11-25T04:23:37.556 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.556 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.556 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:37.556 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038292+0000", 2023-11-25T04:23:37.556 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035231160000000002 2023-11-25T04:23:37.556 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.556 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.556 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:37.556 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044440+0000", 2023-11-25T04:23:37.556 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0061486980000000002 2023-11-25T04:23:37.557 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.557 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.557 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:37.557 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044495+0000", 2023-11-25T04:23:37.557 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4339000000000003e-05 2023-11-25T04:23:37.557 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.557 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.557 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:37.557 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044531+0000", 2023-11-25T04:23:37.557 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6637999999999997e-05 2023-11-25T04:23:37.557 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:37.558 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:37.558 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:37.558 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.558 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.558 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:37.558 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:37.558 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 50.493589608999997, 2023-11-25T04:23:37.558 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0057634039999999997, 2023-11-25T04:23:37.558 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:37.558 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:37.558 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:37.559 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.559 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:37.559 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:37.559 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:37.559 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.559 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.559 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:37.559 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:37.559 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:37.559 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.559 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.559 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:37.559 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:37.560 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7900000000000007e-07 2023-11-25T04:23:37.560 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.560 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.560 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:37.560 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033507+0000", 2023-11-25T04:23:37.560 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8900000000000004e-06 2023-11-25T04:23:37.560 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.560 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.560 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:37.560 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033519+0000", 2023-11-25T04:23:37.560 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1756000000000001e-05 2023-11-25T04:23:37.560 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.560 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.560 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:37.561 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033527+0000", 2023-11-25T04:23:37.561 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3610000000000001e-06 2023-11-25T04:23:37.561 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.561 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.561 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:37.561 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034451+0000", 2023-11-25T04:23:37.561 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00092372400000000001 2023-11-25T04:23:37.561 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.561 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.561 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:37.561 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034809+0000", 2023-11-25T04:23:37.561 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00035757699999999998 2023-11-25T04:23:37.561 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.561 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.566 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:23:37.566 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034965+0000", 2023-11-25T04:23:37.566 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015638699999999999 2023-11-25T04:23:37.566 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.566 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.567 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:37.567 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038299+0000", 2023-11-25T04:23:37.567 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0033336559999999999 2023-11-25T04:23:37.567 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.567 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.567 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:37.567 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039169+0000", 2023-11-25T04:23:37.567 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00087066400000000005 2023-11-25T04:23:37.567 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.567 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.567 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:37.567 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039226+0000", 2023-11-25T04:23:37.568 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6511999999999997e-05 2023-11-25T04:23:37.568 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.568 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.568 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:37.568 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039264+0000", 2023-11-25T04:23:37.568 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7898000000000002e-05 2023-11-25T04:23:37.568 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:37.568 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:37.568 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:37.568 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.568 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.569 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:25 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:37.569 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:37.569 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 50.493511024999997, 2023-11-25T04:23:37.569 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096656169999999996, 2023-11-25T04:23:37.569 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:37.569 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:37.569 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:37.569 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.569 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:37.569 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:37.569 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:37.570 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.570 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.570 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:37.570 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:37.570 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:37.570 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.570 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.570 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:37.570 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033580+0000", 2023-11-25T04:23:37.570 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8199999999999999e-07 2023-11-25T04:23:37.570 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.570 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.572 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:37.572 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033585+0000", 2023-11-25T04:23:37.572 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3680000000000001e-06 2023-11-25T04:23:37.572 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.572 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.573 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:37.573 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033600+0000", 2023-11-25T04:23:37.573 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4331000000000001e-05 2023-11-25T04:23:37.573 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.573 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.573 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:37.573 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033606+0000", 2023-11-25T04:23:37.573 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5780000000000004e-06 2023-11-25T04:23:37.573 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.573 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.573 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:37.573 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035747+0000", 2023-11-25T04:23:37.573 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0021406530000000002 2023-11-25T04:23:37.574 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.574 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.574 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:37.574 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035901+0000", 2023-11-25T04:23:37.574 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000153931 2023-11-25T04:23:37.574 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.574 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.574 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:23:37.574 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036193+0000", 2023-11-25T04:23:37.574 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029211299999999999 2023-11-25T04:23:37.574 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.574 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.575 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:37.575 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038655+0000", 2023-11-25T04:23:37.575 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0024621640000000002 2023-11-25T04:23:37.575 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.575 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.575 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:37.575 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043130+0000", 2023-11-25T04:23:37.575 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.004474698 2023-11-25T04:23:37.575 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.575 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.575 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:37.575 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043204+0000", 2023-11-25T04:23:37.575 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4091999999999995e-05 2023-11-25T04:23:37.575 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.576 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.576 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:37.576 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043245+0000", 2023-11-25T04:23:37.576 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0907e-05 2023-11-25T04:23:37.576 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:37.576 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:37.576 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:37.576 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.576 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.576 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:26 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:37.576 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:37.576 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 50.493475201000003, 2023-11-25T04:23:37.576 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011086945000000001, 2023-11-25T04:23:37.576 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:37.577 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:37.577 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:37.577 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.577 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:37.577 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:37.577 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:37.577 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.577 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.577 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:37.577 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:37.577 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:37.577 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.577 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.577 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:37.578 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:37.578 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1699999999999998e-07 2023-11-25T04:23:37.578 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.578 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.578 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:37.578 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033627+0000", 2023-11-25T04:23:37.578 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.163e-05 2023-11-25T04:23:37.578 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.578 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.578 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:37.578 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033636+0000", 2023-11-25T04:23:37.578 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6859999999999995e-06 2023-11-25T04:23:37.578 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.578 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.579 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:37.579 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033642+0000", 2023-11-25T04:23:37.579 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3149999999999997e-06 2023-11-25T04:23:37.579 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.579 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.579 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:37.579 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035414+0000", 2023-11-25T04:23:37.579 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0017716380000000001 2023-11-25T04:23:37.579 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.579 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.579 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:37.579 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035539+0000", 2023-11-25T04:23:37.579 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000125477 2023-11-25T04:23:37.579 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.580 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.580 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:37.580 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035672+0000", 2023-11-25T04:23:37.580 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000133316 2023-11-25T04:23:37.580 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.580 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.580 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:37.580 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038649+0000", 2023-11-25T04:23:37.580 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029764679999999999 2023-11-25T04:23:37.580 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.580 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.580 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:37.580 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044578+0000", 2023-11-25T04:23:37.580 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0059293510000000002 2023-11-25T04:23:37.581 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.581 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.581 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:37.581 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044651+0000", 2023-11-25T04:23:37.581 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2513000000000002e-05 2023-11-25T04:23:37.581 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.581 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.581 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:37.581 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044702+0000", 2023-11-25T04:23:37.581 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1034e-05 2023-11-25T04:23:37.581 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:37.581 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:37.581 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:37.582 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.582 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.582 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:29 3.b 3:d0630e4c:::mds0_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:37.582 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:37.582 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 50.493437280999999, 2023-11-25T04:23:37.582 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010980227, 2023-11-25T04:23:37.583 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:37.583 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:37.583 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:37.583 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.584 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:37.584 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:37.584 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:37.584 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.584 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.584 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:37.584 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:37.584 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:37.584 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.584 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.584 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:37.584 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:37.584 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3499999999999996e-07 2023-11-25T04:23:37.585 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.585 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.585 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:37.585 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033658+0000", 2023-11-25T04:23:37.585 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1280000000000001e-06 2023-11-25T04:23:37.585 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.586 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.586 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:37.586 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033667+0000", 2023-11-25T04:23:37.586 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1430000000000004e-06 2023-11-25T04:23:37.586 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.586 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.586 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:37.587 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033678+0000", 2023-11-25T04:23:37.587 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0906e-05 2023-11-25T04:23:37.587 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.587 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.587 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:37.587 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034456+0000", 2023-11-25T04:23:37.587 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00077833899999999996 2023-11-25T04:23:37.587 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.587 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.587 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:37.587 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034575+0000", 2023-11-25T04:23:37.587 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011926000000000001 2023-11-25T04:23:37.587 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.588 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.588 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:37.588 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035001+0000", 2023-11-25T04:23:37.588 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042550999999999998 2023-11-25T04:23:37.588 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.588 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.588 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:37.588 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.037923+0000", 2023-11-25T04:23:37.588 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029219670000000001 2023-11-25T04:23:37.589 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.589 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.589 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:37.589 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044538+0000", 2023-11-25T04:23:37.589 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0066152349999999997 2023-11-25T04:23:37.589 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.589 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.589 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:37.589 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044594+0000", 2023-11-25T04:23:37.589 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5593000000000001e-05 2023-11-25T04:23:37.589 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.589 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.590 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:37.590 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044633+0000", 2023-11-25T04:23:37.590 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9511000000000002e-05 2023-11-25T04:23:37.590 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:37.590 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:37.590 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:37.590 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.590 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.590 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:34 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:37.590 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:37.590 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 50.493187167999999, 2023-11-25T04:23:37.591 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010920445000000001, 2023-11-25T04:23:37.591 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:37.591 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:37.591 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:37.591 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.591 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:37.591 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:37.591 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:37.591 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.591 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.591 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:37.591 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:37.591 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:37.591 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.592 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.592 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:37.592 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033904+0000", 2023-11-25T04:23:37.592 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6599999999999994e-07 2023-11-25T04:23:37.592 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.592 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.592 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:37.592 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033909+0000", 2023-11-25T04:23:37.592 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5639999999999996e-06 2023-11-25T04:23:37.592 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.592 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.592 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:37.592 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033923+0000", 2023-11-25T04:23:37.593 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3028e-05 2023-11-25T04:23:37.593 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.593 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.593 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:37.593 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034043+0000", 2023-11-25T04:23:37.593 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000120328 2023-11-25T04:23:37.593 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.593 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.593 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:37.593 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035576+0000", 2023-11-25T04:23:37.593 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015333720000000001 2023-11-25T04:23:37.593 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.593 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.593 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:37.594 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035699+0000", 2023-11-25T04:23:37.594 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000123001 2023-11-25T04:23:37.594 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.595 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.595 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:37.595 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044823+0000", 2023-11-25T04:23:37.595 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0091241859999999994 2023-11-25T04:23:37.595 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:37.595 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:37.595 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:37.595 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.595 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.595 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:35 3.1f 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:37.595 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:37.595 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 50.491882963000002, 2023-11-25T04:23:37.595 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096164779999999995, 2023-11-25T04:23:37.596 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:37.596 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:37.596 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:37.596 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.596 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:37.596 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:37.596 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:37.596 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.596 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.596 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:37.596 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:37.596 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:37.596 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.596 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.597 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:37.597 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035208+0000", 2023-11-25T04:23:37.597 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2699999999999998e-07 2023-11-25T04:23:37.597 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.597 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.597 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:37.597 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035213+0000", 2023-11-25T04:23:37.597 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.6070000000000001e-06 2023-11-25T04:23:37.597 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.597 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.597 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:37.598 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035350+0000", 2023-11-25T04:23:37.598 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013693599999999999 2023-11-25T04:23:37.598 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.598 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.598 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:37.598 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035358+0000", 2023-11-25T04:23:37.598 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0099999999999995e-06 2023-11-25T04:23:37.598 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.598 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.598 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:37.598 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036260+0000", 2023-11-25T04:23:37.599 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00090240100000000005 2023-11-25T04:23:37.599 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.599 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.599 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:37.599 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036517+0000", 2023-11-25T04:23:37.599 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00025736299999999998 2023-11-25T04:23:37.599 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.599 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.599 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:37.599 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044824+0000", 2023-11-25T04:23:37.599 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0083063340000000003 2023-11-25T04:23:37.599 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:37.599 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:37.599 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:37.600 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.600 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.600 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:37 3.1e 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:37.600 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:37.600 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 50.491716277000002, 2023-11-25T04:23:37.600 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0078322640000000002, 2023-11-25T04:23:37.600 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:37.600 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:37.600 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:37.600 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.600 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:37.600 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:37.600 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:37.601 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.601 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.601 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:37.601 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:37.601 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:37.601 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.601 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.601 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:37.601 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035375+0000", 2023-11-25T04:23:37.601 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9100000000000003e-07 2023-11-25T04:23:37.601 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.601 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.601 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:37.602 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035379+0000", 2023-11-25T04:23:37.602 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9430000000000002e-06 2023-11-25T04:23:37.602 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.602 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.602 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:37.602 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035388+0000", 2023-11-25T04:23:37.602 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2760000000000006e-06 2023-11-25T04:23:37.602 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.602 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.602 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:37.602 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035396+0000", 2023-11-25T04:23:37.602 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8480000000000001e-06 2023-11-25T04:23:37.602 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.603 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.603 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:37.603 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035452+0000", 2023-11-25T04:23:37.603 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5831000000000001e-05 2023-11-25T04:23:37.603 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.603 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.603 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:37.603 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035934+0000", 2023-11-25T04:23:37.603 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00048265599999999998 2023-11-25T04:23:37.603 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.603 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.603 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:37.603 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043206+0000", 2023-11-25T04:23:37.603 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.007271919 2023-11-25T04:23:37.604 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:37.604 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:37.604 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:37.604 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.604 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.604 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing(3.1a epoch 18/16 rep_tid 3 entries 18'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:36 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:23:37.604 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:37.604 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 50.490440691000003, 2023-11-25T04:23:37.604 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002073414, 2023-11-25T04:23:37.604 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:37.604 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:37.604 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:37.605 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.605 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:37.605 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:37.605 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:37.605 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.605 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.605 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:37.605 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:37.605 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:37.605 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.605 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.605 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:37.605 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036650+0000", 2023-11-25T04:23:37.605 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7300000000000002e-07 2023-11-25T04:23:37.606 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.606 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.606 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:37.606 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036659+0000", 2023-11-25T04:23:37.606 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.7499999999999992e-06 2023-11-25T04:23:37.606 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.606 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.606 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:37.606 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036673+0000", 2023-11-25T04:23:37.606 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4917e-05 2023-11-25T04:23:37.606 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.606 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.606 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:37.607 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036687+0000", 2023-11-25T04:23:37.607 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3883000000000001e-05 2023-11-25T04:23:37.607 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.607 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.607 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:37.607 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036781+0000", 2023-11-25T04:23:37.607 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3219000000000003e-05 2023-11-25T04:23:37.607 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.607 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.607 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:37.607 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038723+0000", 2023-11-25T04:23:37.607 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019422720000000001 2023-11-25T04:23:37.607 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:37.607 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:37.608 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:37.608 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.608 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.608 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:28 3.13 e18/16 3:cd5a64a3:::100.00000000:head v 18'1, mlcod=0'0)", 2023-11-25T04:23:37.608 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:37.608 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 50.486779533000004, 2023-11-25T04:23:37.608 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001850546, 2023-11-25T04:23:37.608 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:37.608 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:37.608 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:37.608 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.608 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:37.608 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:37.608 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:37.609 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.609 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.609 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:37.609 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:37.609 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:37.609 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.609 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.609 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:37.609 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:37.609 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7800000000000002e-07 2023-11-25T04:23:37.609 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.609 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.609 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:37.609 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040331+0000", 2023-11-25T04:23:37.610 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0449999999999999e-05 2023-11-25T04:23:37.610 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.610 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.610 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:37.610 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040339+0000", 2023-11-25T04:23:37.610 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0709999999999995e-06 2023-11-25T04:23:37.610 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.610 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.610 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:37.610 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040347+0000", 2023-11-25T04:23:37.610 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7079999999999997e-06 2023-11-25T04:23:37.610 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.610 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.611 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:37.611 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040410+0000", 2023-11-25T04:23:37.611 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2453000000000007e-05 2023-11-25T04:23:37.611 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.611 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.611 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:37.611 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040422+0000", 2023-11-25T04:23:37.611 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2429000000000001e-05 2023-11-25T04:23:37.611 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.611 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.611 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:37.611 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042129+0000", 2023-11-25T04:23:37.611 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001707266 2023-11-25T04:23:37.611 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.612 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.612 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:37.612 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042161+0000", 2023-11-25T04:23:37.612 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1791000000000001e-05 2023-11-25T04:23:37.612 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:37.612 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:37.612 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:37.612 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.612 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.612 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:27 3.13 e18/16 3:c91d4a1d:::mds0_inotable:head v 18'2, mlcod=0'0)", 2023-11-25T04:23:37.612 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:37.612 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 50.48657601, 2023-11-25T04:23:37.612 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0016886430000000001, 2023-11-25T04:23:37.612 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:37.613 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:37.613 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:37.613 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.613 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:37.613 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:37.613 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:37.613 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.613 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.613 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:37.613 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:37.613 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:37.613 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.614 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.614 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:37.614 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:37.614 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3999999999999997e-07 2023-11-25T04:23:37.614 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.614 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.614 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:37.614 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040520+0000", 2023-11-25T04:23:37.614 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2989999999999996e-06 2023-11-25T04:23:37.614 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.614 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.614 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:37.614 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040527+0000", 2023-11-25T04:23:37.615 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5730000000000002e-06 2023-11-25T04:23:37.615 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.615 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.615 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:37.615 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040535+0000", 2023-11-25T04:23:37.615 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8630000000000008e-06 2023-11-25T04:23:37.615 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.615 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.615 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:37.615 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040683+0000", 2023-11-25T04:23:37.615 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000148002 2023-11-25T04:23:37.615 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.615 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.615 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:37.616 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040700+0000", 2023-11-25T04:23:37.616 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7065e-05 2023-11-25T04:23:37.616 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.616 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.616 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:37.616 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042163+0000", 2023-11-25T04:23:37.616 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001462767 2023-11-25T04:23:37.616 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.616 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.616 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:37.616 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042203+0000", 2023-11-25T04:23:37.616 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9734000000000003e-05 2023-11-25T04:23:37.616 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:37.616 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:37.617 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:37.617 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.617 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.617 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:20 3.1c e18/16 3:3ed09add:::607.00000000:head v 18'1, mlcod=0'0)", 2023-11-25T04:23:37.617 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:37.617 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 50.486540969000004, 2023-11-25T04:23:37.617 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015271, 2023-11-25T04:23:37.617 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:37.617 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:37.617 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:37.617 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.617 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:37.617 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:37.617 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:37.618 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.618 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.618 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:37.618 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:37.618 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:37.618 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.618 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.618 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:37.618 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:37.618 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4799999999999999e-07 2023-11-25T04:23:37.618 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.618 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.618 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:37.618 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040561+0000", 2023-11-25T04:23:37.619 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1800000000000001e-05 2023-11-25T04:23:37.619 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.619 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.619 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:37.619 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040568+0000", 2023-11-25T04:23:37.619 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7730000000000002e-06 2023-11-25T04:23:37.619 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.619 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.619 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:37.619 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040587+0000", 2023-11-25T04:23:37.619 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8691000000000001e-05 2023-11-25T04:23:37.619 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.619 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.620 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:37.620 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040638+0000", 2023-11-25T04:23:37.620 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1068e-05 2023-11-25T04:23:37.620 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.620 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.620 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:37.620 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040670+0000", 2023-11-25T04:23:37.620 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2125999999999997e-05 2023-11-25T04:23:37.620 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.620 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.621 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:37.621 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042032+0000", 2023-11-25T04:23:37.621 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001361821 2023-11-25T04:23:37.621 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.621 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.621 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:37.621 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042076+0000", 2023-11-25T04:23:37.621 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4472999999999999e-05 2023-11-25T04:23:37.621 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:37.621 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:37.621 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:37.621 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.622 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.622 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:21 3.1c e18/16 3:3ed09add:::607.00000000:head v 18'2, mlcod=0'0)", 2023-11-25T04:23:37.622 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:37.622 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 50.486306243999998, 2023-11-25T04:23:37.622 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0013228160000000001, 2023-11-25T04:23:37.622 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:37.622 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:37.622 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:37.622 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.622 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:37.622 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:37.622 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:37.622 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.622 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.623 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:37.623 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:37.623 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:37.623 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.623 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.623 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:37.623 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:37.623 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9299999999999999e-07 2023-11-25T04:23:37.623 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.623 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.623 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:37.623 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040790+0000", 2023-11-25T04:23:37.623 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5770000000000001e-06 2023-11-25T04:23:37.623 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.624 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.624 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:37.624 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040799+0000", 2023-11-25T04:23:37.624 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6439999999999995e-06 2023-11-25T04:23:37.624 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.624 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.624 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:37.624 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040806+0000", 2023-11-25T04:23:37.624 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0180000000000004e-06 2023-11-25T04:23:37.624 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.624 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.624 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:37.624 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040907+0000", 2023-11-25T04:23:37.624 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000100096 2023-11-25T04:23:37.625 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.625 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.625 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:37.625 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040925+0000", 2023-11-25T04:23:37.625 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8536999999999999e-05 2023-11-25T04:23:37.625 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.625 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.625 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:37.625 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042079+0000", 2023-11-25T04:23:37.625 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00115358 2023-11-25T04:23:37.625 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.625 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.625 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:37.625 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042107+0000", 2023-11-25T04:23:37.626 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7971e-05 2023-11-25T04:23:37.626 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:37.626 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:37.626 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:37.626 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.626 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.626 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop_reply(mds.0.3:4 3.1f e18/16 ondisk, result = 0)", 2023-11-25T04:23:37.626 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:37.626 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 50.484966854, 2023-11-25T04:23:37.626 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00044092900000000001, 2023-11-25T04:23:37.626 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:37.626 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:37.626 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:37.627 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.627 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:37.627 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:37.627 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:37.627 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.627 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.627 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:37.627 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:37.627 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:37.627 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.627 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.627 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:37.627 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042124+0000", 2023-11-25T04:23:37.627 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1800000000000005e-07 2023-11-25T04:23:37.628 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.628 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.628 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:37.628 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042128+0000", 2023-11-25T04:23:37.628 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2289999999999997e-06 2023-11-25T04:23:37.628 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.628 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.628 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:37.628 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042138+0000", 2023-11-25T04:23:37.628 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9729999999999992e-06 2023-11-25T04:23:37.628 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.628 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.628 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:37.628 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042145+0000", 2023-11-25T04:23:37.629 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.336e-06 2023-11-25T04:23:37.629 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.629 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.629 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:37.629 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042441+0000", 2023-11-25T04:23:37.629 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029650500000000002 2023-11-25T04:23:37.629 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.629 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.629 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:37.629 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042448+0000", 2023-11-25T04:23:37.629 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4100000000000002e-06 2023-11-25T04:23:37.629 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.629 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.630 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:37.630 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042564+0000", 2023-11-25T04:23:37.630 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000115758 2023-11-25T04:23:37.630 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:37.630 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:37.630 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:37.630 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.630 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.630 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing_reply(3.1f epoch 18/16 rep_tid 8 lcod 18'4)", 2023-11-25T04:23:37.630 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:37.630 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 50.482641442000002, 2023-11-25T04:23:37.630 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00038154699999999998, 2023-11-25T04:23:37.630 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:37.631 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:37.631 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:37.631 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.631 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:37.631 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:37.631 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:37.631 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.631 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.631 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:37.631 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:37.631 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:37.631 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.631 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.632 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:37.632 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:37.632 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2700000000000001e-07 2023-11-25T04:23:37.632 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.632 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.632 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:37.632 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044451+0000", 2023-11-25T04:23:37.632 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7460000000000001e-06 2023-11-25T04:23:37.632 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.632 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.632 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:37.632 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044455+0000", 2023-11-25T04:23:37.632 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5809999999999998e-06 2023-11-25T04:23:37.633 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.633 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.633 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:37.633 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044458+0000", 2023-11-25T04:23:37.633 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0170000000000001e-06 2023-11-25T04:23:37.633 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.633 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.633 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:37.633 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044734+0000", 2023-11-25T04:23:37.633 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00027582799999999999 2023-11-25T04:23:37.633 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.633 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.633 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:37.633 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044830+0000", 2023-11-25T04:23:37.634 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6248000000000004e-05 2023-11-25T04:23:37.634 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:37.634 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:37.634 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:37.634 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.634 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.634 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing_reply(3.b epoch 18/16 rep_tid 5 lcod 18'2)", 2023-11-25T04:23:37.634 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:37.634 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 50.482604780999999, 2023-11-25T04:23:37.634 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000344488, 2023-11-25T04:23:37.634 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:37.634 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:37.634 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:37.635 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.635 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:37.635 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:37.635 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:37.635 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.635 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.635 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:37.635 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:37.635 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:37.635 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.637 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.637 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:37.637 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:37.637 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.55e-07 2023-11-25T04:23:37.637 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.637 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.637 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:37.637 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044487+0000", 2023-11-25T04:23:37.637 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2470000000000001e-06 2023-11-25T04:23:37.637 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.637 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.637 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:37.637 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044491+0000", 2023-11-25T04:23:37.638 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9659999999999998e-06 2023-11-25T04:23:37.638 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.638 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.638 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:37.638 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044494+0000", 2023-11-25T04:23:37.638 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2310000000000001e-06 2023-11-25T04:23:37.638 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.638 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.638 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:37.638 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044756+0000", 2023-11-25T04:23:37.638 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00026172800000000003 2023-11-25T04:23:37.638 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.638 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.639 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:37.639 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044830+0000", 2023-11-25T04:23:37.639 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4160999999999997e-05 2023-11-25T04:23:37.639 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:37.639 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:37.639 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:37.639 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.639 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.639 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:39 3.6 3:654134d2:::mds0_openfiles.0:head [omap-clear,omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2023-11-25T04:23:37.639 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:37.639 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 45.422778739000002, 2023-11-25T04:23:37.639 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001788658, 2023-11-25T04:23:37.639 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:37.640 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:37.640 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:37.640 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.640 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:37.640 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:37.640 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:37.640 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.640 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.640 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:37.640 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:37.640 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:37.640 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.640 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.641 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:37.641 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104314+0000", 2023-11-25T04:23:37.641 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3099999999999999e-06 2023-11-25T04:23:37.641 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.641 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.641 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:37.641 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104325+0000", 2023-11-25T04:23:37.641 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1511999999999999e-05 2023-11-25T04:23:37.641 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.641 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.641 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:37.641 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104362+0000", 2023-11-25T04:23:37.641 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6903000000000002e-05 2023-11-25T04:23:37.641 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.642 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.642 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:37.642 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104387+0000", 2023-11-25T04:23:37.642 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4766e-05 2023-11-25T04:23:37.642 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.642 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.642 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:37.642 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104465+0000", 2023-11-25T04:23:37.642 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8119000000000002e-05 2023-11-25T04:23:37.642 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.642 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.642 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:37.643 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104557+0000", 2023-11-25T04:23:37.643 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2496000000000003e-05 2023-11-25T04:23:37.643 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.643 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.643 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:37.643 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104646+0000", 2023-11-25T04:23:37.643 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8818000000000002e-05 2023-11-25T04:23:37.643 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.643 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.643 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:37.643 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105201+0000", 2023-11-25T04:23:37.643 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00055506000000000002 2023-11-25T04:23:37.643 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.644 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.644 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:37.644 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105994+0000", 2023-11-25T04:23:37.644 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00079294700000000001 2023-11-25T04:23:37.644 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.644 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.644 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:37.644 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.106055+0000", 2023-11-25T04:23:37.644 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0566000000000003e-05 2023-11-25T04:23:37.644 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.644 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.644 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:37.644 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.106100+0000", 2023-11-25T04:23:37.645 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5160999999999997e-05 2023-11-25T04:23:37.645 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:37.645 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:37.645 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:37.645 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.645 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.645 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:43 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2023-11-25T04:23:37.645 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:37.645 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 30.422553743000002, 2023-11-25T04:23:37.645 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001751166, 2023-11-25T04:23:37.645 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:37.646 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:37.646 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:37.646 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.646 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:37.646 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:37.646 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:37.646 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.646 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.646 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:37.646 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:37.646 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:37.646 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.646 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.647 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:37.647 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104539+0000", 2023-11-25T04:23:37.647 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9529999999999999e-06 2023-11-25T04:23:37.647 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.647 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.647 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:37.647 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104557+0000", 2023-11-25T04:23:37.647 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7464000000000001e-05 2023-11-25T04:23:37.647 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.647 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.647 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:37.647 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104590+0000", 2023-11-25T04:23:37.647 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2966000000000003e-05 2023-11-25T04:23:37.647 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.648 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.648 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:37.648 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104608+0000", 2023-11-25T04:23:37.648 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8193e-05 2023-11-25T04:23:37.648 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.648 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.648 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:37.648 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104767+0000", 2023-11-25T04:23:37.648 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015882 2023-11-25T04:23:37.648 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.648 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.648 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:37.648 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104909+0000", 2023-11-25T04:23:37.649 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014244999999999999 2023-11-25T04:23:37.649 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.649 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.649 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:37.649 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.105059+0000", 2023-11-25T04:23:37.649 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014964399999999999 2023-11-25T04:23:37.649 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.649 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.649 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:37.649 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.105701+0000", 2023-11-25T04:23:37.649 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00064201099999999995 2023-11-25T04:23:37.649 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.649 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.649 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:37.650 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106233+0000", 2023-11-25T04:23:37.650 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00053178500000000001 2023-11-25T04:23:37.650 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.650 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.650 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:37.650 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106265+0000", 2023-11-25T04:23:37.650 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1986000000000002e-05 2023-11-25T04:23:37.650 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.650 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.650 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:37.650 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106288+0000", 2023-11-25T04:23:37.650 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2894000000000001e-05 2023-11-25T04:23:37.650 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:37.651 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:37.651 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:37.651 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.651 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.651 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:45 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e23)", 2023-11-25T04:23:37.651 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:23:37.651 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 25.422355703000001, 2023-11-25T04:23:37.651 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00098423299999999994, 2023-11-25T04:23:37.651 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:37.651 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:37.651 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:37.651 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.651 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:37.651 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:23:37.652 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:37.652 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.652 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.652 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:37.652 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:23:37.652 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:37.652 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.652 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.652 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:37.652 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104737+0000", 2023-11-25T04:23:37.652 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3309999999999998e-06 2023-11-25T04:23:37.652 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.652 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.652 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:37.653 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104748+0000", 2023-11-25T04:23:37.653 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1080999999999999e-05 2023-11-25T04:23:37.653 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.653 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.653 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:37.653 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104771+0000", 2023-11-25T04:23:37.653 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2707000000000001e-05 2023-11-25T04:23:37.653 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.653 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.653 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:37.653 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104800+0000", 2023-11-25T04:23:37.653 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9595e-05 2023-11-25T04:23:37.653 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.654 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.654 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:37.654 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104890+0000", 2023-11-25T04:23:37.654 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.9808999999999993e-05 2023-11-25T04:23:37.654 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.654 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.654 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:37.654 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105004+0000", 2023-11-25T04:23:37.654 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000114508 2023-11-25T04:23:37.654 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.654 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.654 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:37.654 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105634+0000", 2023-11-25T04:23:37.655 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00062985799999999996 2023-11-25T04:23:37.655 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.655 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.655 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:37.655 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105671+0000", 2023-11-25T04:23:37.655 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6640000000000002e-05 2023-11-25T04:23:37.655 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:37.655 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:37.655 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:37.655 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105719+0000", 2023-11-25T04:23:37.655 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7704000000000003e-05 2023-11-25T04:23:37.655 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:37.656 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:37.656 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:37.656 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:37.656 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:37.656 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:37.657 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:23:37.658 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:37.656+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:37.758 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:37.757+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:37.767 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:37.768 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:23:37.768 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2023-11-25T04:23:37.768 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:37.777 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:23:37.859 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:37.858+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:37.922 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:37.922 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:23:37.922 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2023-11-25T04:23:37.922 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2023-11-25T04:23:37.922 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:37.925 INFO:teuthology.orchestra.run.smithi067.stderr:nodeep-scrub is unset 2023-11-25T04:23:37.932 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:23:37.960 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:37.959+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:38.039 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:38.040 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2023-11-25T04:23:38.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2023-11-25T04:23:38.040 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2023-11-25T04:23:38.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.040 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:155 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'109, mlcod=11'105)", 2023-11-25T04:23:38.040 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:38.040 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 60.252572929999999, 2023-11-25T04:23:38.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061719019999999999, 2023-11-25T04:23:38.040 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:38.041 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:38.041 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:38.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:38.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:38.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:38.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:38.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:38.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:38.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:38.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784884+0000", 2023-11-25T04:23:38.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3099999999999999e-07 2023-11-25T04:23:38.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:38.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784942+0000", 2023-11-25T04:23:38.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8570000000000003e-05 2023-11-25T04:23:38.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:38.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784949+0000", 2023-11-25T04:23:38.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0310000000000001e-06 2023-11-25T04:23:38.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:38.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784953+0000", 2023-11-25T04:23:38.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6320000000000001e-06 2023-11-25T04:23:38.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:38.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784993+0000", 2023-11-25T04:23:38.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9737000000000003e-05 2023-11-25T04:23:38.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:38.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785011+0000", 2023-11-25T04:23:38.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8349999999999999e-05 2023-11-25T04:23:38.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:38.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791024+0000", 2023-11-25T04:23:38.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006012997 2023-11-25T04:23:38.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:38.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791055+0000", 2023-11-25T04:23:38.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1254000000000003e-05 2023-11-25T04:23:38.045 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:38.045 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:38.045 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:38.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.045 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:156 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'110, mlcod=11'105)", 2023-11-25T04:23:38.045 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:38.045 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 60.251986881000001, 2023-11-25T04:23:38.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005616116, 2023-11-25T04:23:38.046 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:38.046 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:38.046 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:38.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:38.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:38.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:38.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:38.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:38.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:38.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:38.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785470+0000", 2023-11-25T04:23:38.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0699999999999998e-07 2023-11-25T04:23:38.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:38.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785474+0000", 2023-11-25T04:23:38.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.048e-06 2023-11-25T04:23:38.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:38.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785480+0000", 2023-11-25T04:23:38.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7729999999999996e-06 2023-11-25T04:23:38.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:38.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785483+0000", 2023-11-25T04:23:38.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1970000000000001e-06 2023-11-25T04:23:38.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:38.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787936+0000", 2023-11-25T04:23:38.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024529040000000001 2023-11-25T04:23:38.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:38.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787947+0000", 2023-11-25T04:23:38.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1669e-05 2023-11-25T04:23:38.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:38.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791057+0000", 2023-11-25T04:23:38.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031095200000000002 2023-11-25T04:23:38.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:38.050 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791086+0000", 2023-11-25T04:23:38.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8697999999999999e-05 2023-11-25T04:23:38.050 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:38.050 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:38.050 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:38.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.050 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:157 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'111, mlcod=11'106)", 2023-11-25T04:23:38.050 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:38.051 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 60.251394323, 2023-11-25T04:23:38.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050590280000000001, 2023-11-25T04:23:38.051 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:38.051 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:38.051 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:38.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:38.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:38.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:38.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:38.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:38.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:38.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:38.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786063+0000", 2023-11-25T04:23:38.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2600000000000002e-07 2023-11-25T04:23:38.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:38.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786069+0000", 2023-11-25T04:23:38.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8839999999999999e-06 2023-11-25T04:23:38.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:38.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786079+0000", 2023-11-25T04:23:38.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0124e-05 2023-11-25T04:23:38.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:38.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786085+0000", 2023-11-25T04:23:38.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6660000000000002e-06 2023-11-25T04:23:38.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.054 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:38.054 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788160+0000", 2023-11-25T04:23:38.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020747669999999999 2023-11-25T04:23:38.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.054 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:38.054 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788180+0000", 2023-11-25T04:23:38.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0208999999999999e-05 2023-11-25T04:23:38.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.054 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:38.054 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791087+0000", 2023-11-25T04:23:38.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029064630000000002 2023-11-25T04:23:38.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:38.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791121+0000", 2023-11-25T04:23:38.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4289000000000003e-05 2023-11-25T04:23:38.055 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:38.055 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:38.055 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:38.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.055 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:158 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'112, mlcod=11'106)", 2023-11-25T04:23:38.056 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:38.056 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 60.250977929999998, 2023-11-25T04:23:38.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046663030000000001, 2023-11-25T04:23:38.056 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:38.056 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:38.056 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:38.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:38.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:38.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:38.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:38.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:38.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:38.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:38.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786479+0000", 2023-11-25T04:23:38.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2799999999999996e-07 2023-11-25T04:23:38.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:38.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786498+0000", 2023-11-25T04:23:38.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9069e-05 2023-11-25T04:23:38.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:38.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786508+0000", 2023-11-25T04:23:38.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5810000000000003e-06 2023-11-25T04:23:38.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:38.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786512+0000", 2023-11-25T04:23:38.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4889999999999997e-06 2023-11-25T04:23:38.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:38.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788438+0000", 2023-11-25T04:23:38.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019257059999999999 2023-11-25T04:23:38.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:38.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788449+0000", 2023-11-25T04:23:38.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1419e-05 2023-11-25T04:23:38.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:38.060 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791122+0000", 2023-11-25T04:23:38.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002672822 2023-11-25T04:23:38.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.060 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:38.060 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791145+0000", 2023-11-25T04:23:38.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2589e-05 2023-11-25T04:23:38.060 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:38.060 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:38.060 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:38.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.061 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:206 1.0 e12/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 12'152, mlcod=12'148)", 2023-11-25T04:23:38.061 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:38.061 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 59.739209459999998, 2023-11-25T04:23:38.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046728189999999999, 2023-11-25T04:23:38.061 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:38.061 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:38.061 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:38.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:38.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:38.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:38.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:38.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:38.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:38.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:38.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298248+0000", 2023-11-25T04:23:38.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3600000000000003e-07 2023-11-25T04:23:38.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:38.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298278+0000", 2023-11-25T04:23:38.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0474e-05 2023-11-25T04:23:38.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:38.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298288+0000", 2023-11-25T04:23:38.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0156000000000001e-05 2023-11-25T04:23:38.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:38.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298293+0000", 2023-11-25T04:23:38.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7400000000000004e-06 2023-11-25T04:23:38.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:38.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298333+0000", 2023-11-25T04:23:38.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9666000000000003e-05 2023-11-25T04:23:38.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:38.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298351+0000", 2023-11-25T04:23:38.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8544999999999999e-05 2023-11-25T04:23:38.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:38.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302891+0000", 2023-11-25T04:23:38.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045398519999999996 2023-11-25T04:23:38.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:38.065 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302920+0000", 2023-11-25T04:23:38.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8750000000000001e-05 2023-11-25T04:23:38.065 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:38.065 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:38.065 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:38.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.065 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:14 3.15 3:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:38.066 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:38.066 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.004075102999998, 2023-11-25T04:23:38.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010990437, 2023-11-25T04:23:38.066 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:38.066 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:38.066 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:38.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:38.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:38.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:38.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:38.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:38.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:38.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:38.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033383+0000", 2023-11-25T04:23:38.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9870000000000002e-06 2023-11-25T04:23:38.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:38.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033398+0000", 2023-11-25T04:23:38.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4569999999999999e-05 2023-11-25T04:23:38.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:38.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033411+0000", 2023-11-25T04:23:38.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3543999999999999e-05 2023-11-25T04:23:38.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:38.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033423+0000", 2023-11-25T04:23:38.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1905999999999999e-05 2023-11-25T04:23:38.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.069 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:38.069 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034433+0000", 2023-11-25T04:23:38.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00101022 2023-11-25T04:23:38.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.069 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:38.069 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034539+0000", 2023-11-25T04:23:38.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010597199999999999 2023-11-25T04:23:38.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.069 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:38.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034667+0000", 2023-11-25T04:23:38.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012765500000000001 2023-11-25T04:23:38.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:38.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037803+0000", 2023-11-25T04:23:38.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031362220000000001 2023-11-25T04:23:38.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:38.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:23:38.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0064964669999999997 2023-11-25T04:23:38.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:38.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044331+0000", 2023-11-25T04:23:38.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1544999999999997e-05 2023-11-25T04:23:38.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:38.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044372+0000", 2023-11-25T04:23:38.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0349000000000003e-05 2023-11-25T04:23:38.071 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:38.071 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:38.071 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:38.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.072 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:15 3.15 3:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:38.072 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:38.072 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.003415453000002, 2023-11-25T04:23:38.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010464362, 2023-11-25T04:23:38.072 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:38.072 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:38.072 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:38.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:38.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:38.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:38.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.073 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:38.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:38.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:38.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.073 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:38.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034042+0000", 2023-11-25T04:23:38.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0999999999999997e-07 2023-11-25T04:23:38.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:38.074 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034048+0000", 2023-11-25T04:23:38.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0390000000000001e-06 2023-11-25T04:23:38.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:38.074 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034059+0000", 2023-11-25T04:23:38.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1182999999999999e-05 2023-11-25T04:23:38.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:38.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034067+0000", 2023-11-25T04:23:38.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1559999999999995e-06 2023-11-25T04:23:38.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:38.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035005+0000", 2023-11-25T04:23:38.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00093747399999999999 2023-11-25T04:23:38.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:38.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035077+0000", 2023-11-25T04:23:38.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2747e-05 2023-11-25T04:23:38.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:38.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035219+0000", 2023-11-25T04:23:38.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000141877 2023-11-25T04:23:38.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:38.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037810+0000", 2023-11-25T04:23:38.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002590752 2023-11-25T04:23:38.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.077 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:38.077 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044437+0000", 2023-11-25T04:23:38.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066270110000000004 2023-11-25T04:23:38.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.077 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:38.077 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044481+0000", 2023-11-25T04:23:38.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4106000000000001e-05 2023-11-25T04:23:38.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.077 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:38.077 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044505+0000", 2023-11-25T04:23:38.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4207e-05 2023-11-25T04:23:38.078 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:38.078 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:38.078 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:38.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.078 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:16 3.1d 3:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:38.078 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:38.078 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.003373605, 2023-11-25T04:23:38.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010413923, 2023-11-25T04:23:38.078 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:38.078 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:38.079 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:38.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:38.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:38.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:38.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:38.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:38.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:38.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:38.080 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:38.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4499999999999997e-07 2023-11-25T04:23:38.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.080 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:38.080 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034087+0000", 2023-11-25T04:23:38.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9770000000000002e-06 2023-11-25T04:23:38.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.080 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:38.080 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034094+0000", 2023-11-25T04:23:38.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8809999999999998e-06 2023-11-25T04:23:38.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:38.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034100+0000", 2023-11-25T04:23:38.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8440000000000003e-06 2023-11-25T04:23:38.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:38.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034993+0000", 2023-11-25T04:23:38.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00089336799999999996 2023-11-25T04:23:38.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:38.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035099+0000", 2023-11-25T04:23:38.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106037 2023-11-25T04:23:38.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:38.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035276+0000", 2023-11-25T04:23:38.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017631799999999999 2023-11-25T04:23:38.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:38.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037815+0000", 2023-11-25T04:23:38.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025395959999999999 2023-11-25T04:23:38.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.083 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:38.083 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044420+0000", 2023-11-25T04:23:38.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066045749999999997 2023-11-25T04:23:38.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.083 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:38.083 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044463+0000", 2023-11-25T04:23:38.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3205999999999999e-05 2023-11-25T04:23:38.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:38.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044497+0000", 2023-11-25T04:23:38.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3476000000000003e-05 2023-11-25T04:23:38.084 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:38.084 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:38.084 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:38.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.084 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:17 3.1d 3:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:38.084 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:38.084 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.003348396, 2023-11-25T04:23:38.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01127356, 2023-11-25T04:23:38.085 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:38.085 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:38.085 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:38.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:38.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:38.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:38.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:38.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:38.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:38.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.086 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:38.086 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:38.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1100000000000001e-07 2023-11-25T04:23:38.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.086 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:38.086 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034112+0000", 2023-11-25T04:23:38.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8600000000000003e-06 2023-11-25T04:23:38.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.086 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:38.087 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034120+0000", 2023-11-25T04:23:38.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7700000000000001e-06 2023-11-25T04:23:38.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.087 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:38.087 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034127+0000", 2023-11-25T04:23:38.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0999999999999998e-06 2023-11-25T04:23:38.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.087 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:38.087 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035601+0000", 2023-11-25T04:23:38.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014739410000000001 2023-11-25T04:23:38.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:38.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035685+0000", 2023-11-25T04:23:38.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3677999999999994e-05 2023-11-25T04:23:38.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:38.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035827+0000", 2023-11-25T04:23:38.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014232599999999999 2023-11-25T04:23:38.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:38.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038725+0000", 2023-11-25T04:23:38.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028982779999999998 2023-11-25T04:23:38.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:38.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045323+0000", 2023-11-25T04:23:38.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0065980780000000003 2023-11-25T04:23:38.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:38.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045354+0000", 2023-11-25T04:23:38.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0463e-05 2023-11-25T04:23:38.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:38.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045381+0000", 2023-11-25T04:23:38.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7654999999999999e-05 2023-11-25T04:23:38.090 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:38.090 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:38.090 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:38.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.090 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:22 3.e 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:38.090 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:38.090 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.003321982999999, 2023-11-25T04:23:38.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010133422, 2023-11-25T04:23:38.091 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:38.091 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:38.091 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:38.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:38.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:38.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:38.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:38.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:38.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:38.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.092 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:38.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034135+0000", 2023-11-25T04:23:38.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-07 2023-11-25T04:23:38.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.092 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:38.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034138+0000", 2023-11-25T04:23:38.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6009999999999999e-06 2023-11-25T04:23:38.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.092 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:38.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034145+0000", 2023-11-25T04:23:38.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1169999999999998e-06 2023-11-25T04:23:38.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:38.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034150+0000", 2023-11-25T04:23:38.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9080000000000003e-06 2023-11-25T04:23:38.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:38.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034209+0000", 2023-11-25T04:23:38.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8149000000000002e-05 2023-11-25T04:23:38.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:38.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034415+0000", 2023-11-25T04:23:38.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020637699999999999 2023-11-25T04:23:38.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:38.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034584+0000", 2023-11-25T04:23:38.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016870399999999999 2023-11-25T04:23:38.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:38.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.036979+0000", 2023-11-25T04:23:38.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023957509999999998 2023-11-25T04:23:38.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:38.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044161+0000", 2023-11-25T04:23:38.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0071811590000000003 2023-11-25T04:23:38.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:38.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044224+0000", 2023-11-25T04:23:38.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3454000000000006e-05 2023-11-25T04:23:38.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:38.096 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044268+0000", 2023-11-25T04:23:38.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3761999999999999e-05 2023-11-25T04:23:38.096 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:38.096 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:38.096 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:38.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.096 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:23 3.e 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:38.096 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:38.097 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.003290573999998, 2023-11-25T04:23:38.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010281676, 2023-11-25T04:23:38.097 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:38.097 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:38.097 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:38.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:38.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:38.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:38.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:38.098 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:38.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:38.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.098 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:38.098 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:38.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5799999999999999e-07 2023-11-25T04:23:38.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.098 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:38.098 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034169+0000", 2023-11-25T04:23:38.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1879999999999998e-06 2023-11-25T04:23:38.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.099 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:38.099 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034177+0000", 2023-11-25T04:23:38.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7020000000000001e-06 2023-11-25T04:23:38.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.099 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:38.099 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034184+0000", 2023-11-25T04:23:38.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3910000000000003e-06 2023-11-25T04:23:38.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.099 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:38.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034924+0000", 2023-11-25T04:23:38.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00074032999999999998 2023-11-25T04:23:38.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:38.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034991+0000", 2023-11-25T04:23:38.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7347000000000004e-05 2023-11-25T04:23:38.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:38.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035096+0000", 2023-11-25T04:23:38.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010488 2023-11-25T04:23:38.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:38.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037588+0000", 2023-11-25T04:23:38.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024918309999999999 2023-11-25T04:23:38.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:38.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044367+0000", 2023-11-25T04:23:38.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067787589999999997 2023-11-25T04:23:38.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:38.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044415+0000", 2023-11-25T04:23:38.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7803000000000003e-05 2023-11-25T04:23:38.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.102 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:38.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044447+0000", 2023-11-25T04:23:38.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2886999999999999e-05 2023-11-25T04:23:38.102 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:38.102 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:38.102 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:38.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.103 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:30 3.15 3:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:38.103 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:38.103 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.003232134000001, 2023-11-25T04:23:38.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010365070000000001, 2023-11-25T04:23:38.103 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:38.103 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:38.103 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:38.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.103 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:38.103 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:38.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:38.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.104 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:38.104 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:38.104 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:38.104 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.104 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:38.104 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034225+0000", 2023-11-25T04:23:38.104 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7400000000000002e-07 2023-11-25T04:23:38.104 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.104 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:38.104 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034229+0000", 2023-11-25T04:23:38.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3100000000000002e-06 2023-11-25T04:23:38.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.105 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:38.105 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034239+0000", 2023-11-25T04:23:38.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9140000000000003e-06 2023-11-25T04:23:38.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.105 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:38.105 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034246+0000", 2023-11-25T04:23:38.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0149999999999998e-06 2023-11-25T04:23:38.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.106 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:38.106 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035572+0000", 2023-11-25T04:23:38.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001325655 2023-11-25T04:23:38.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.106 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:38.106 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035730+0000", 2023-11-25T04:23:38.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015827899999999999 2023-11-25T04:23:38.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.106 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:38.106 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035857+0000", 2023-11-25T04:23:38.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012644699999999999 2023-11-25T04:23:38.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.107 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:38.107 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038737+0000", 2023-11-25T04:23:38.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002880077 2023-11-25T04:23:38.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.107 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:38.107 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044540+0000", 2023-11-25T04:23:38.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058034239999999997 2023-11-25T04:23:38.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.108 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:38.108 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044569+0000", 2023-11-25T04:23:38.108 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8388999999999999e-05 2023-11-25T04:23:38.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.108 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:38.108 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044589+0000", 2023-11-25T04:23:38.108 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0686000000000002e-05 2023-11-25T04:23:38.108 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:38.108 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:38.108 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:38.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.109 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:33 3.1b 3:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:38.109 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:38.109 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.003186636000002, 2023-11-25T04:23:38.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099754049999999997, 2023-11-25T04:23:38.109 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:38.109 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:23:38.109 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:38.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.109 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:38.109 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:38.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:38.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.110 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.110 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:38.110 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:38.110 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:38.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.110 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.110 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:38.110 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:38.110 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4799999999999998e-07 2023-11-25T04:23:38.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.110 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.110 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:38.110 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034273+0000", 2023-11-25T04:23:38.111 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7819999999999998e-06 2023-11-25T04:23:38.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.111 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.111 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:38.111 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034285+0000", 2023-11-25T04:23:38.111 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2058e-05 2023-11-25T04:23:38.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.111 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.112 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:38.112 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034291+0000", 2023-11-25T04:23:38.112 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0680000000000002e-06 2023-11-25T04:23:38.112 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.112 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.112 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:38.112 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034370+0000", 2023-11-25T04:23:38.112 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8452e-05 2023-11-25T04:23:38.112 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.112 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.113 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:38.113 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034477+0000", 2023-11-25T04:23:38.113 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107471 2023-11-25T04:23:38.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.113 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:38.113 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044245+0000", 2023-11-25T04:23:38.113 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0097679259999999997 2023-11-25T04:23:38.113 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:38.113 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:38.113 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:38.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.114 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.114 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:36 3.1a 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:38.114 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:38.114 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.003139963000002, 2023-11-25T04:23:38.114 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058179099999999999, 2023-11-25T04:23:38.114 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:38.114 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:23:38.114 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:38.114 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.114 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:38.114 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:38.114 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:38.114 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.115 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:38.115 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:38.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:38.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.115 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:38.115 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034317+0000", 2023-11-25T04:23:38.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1999999999999999e-07 2023-11-25T04:23:38.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.115 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:38.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034321+0000", 2023-11-25T04:23:38.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.233e-06 2023-11-25T04:23:38.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.116 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:38.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034337+0000", 2023-11-25T04:23:38.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5308999999999999e-05 2023-11-25T04:23:38.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.116 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:38.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034354+0000", 2023-11-25T04:23:38.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7411999999999999e-05 2023-11-25T04:23:38.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.117 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:38.117 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034465+0000", 2023-11-25T04:23:38.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011154699999999999 2023-11-25T04:23:38.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.117 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:38.117 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034560+0000", 2023-11-25T04:23:38.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4186999999999998e-05 2023-11-25T04:23:38.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.118 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:38.118 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.040134+0000", 2023-11-25T04:23:38.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055746019999999997 2023-11-25T04:23:38.118 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:38.118 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:38.118 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:38.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.118 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:9 3.10 e18/16 3:09401a81:::601.00000000:head v 18'4, mlcod=0'0)", 2023-11-25T04:23:38.118 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:38.118 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.994245655, 2023-11-25T04:23:38.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061641020000000003, 2023-11-25T04:23:38.118 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:38.119 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:38.119 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:38.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:38.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:38.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:38.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:38.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:38.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:38.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.120 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:38.120 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:38.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5300000000000001e-07 2023-11-25T04:23:38.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.120 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:38.120 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043222+0000", 2023-11-25T04:23:38.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.082e-05 2023-11-25T04:23:38.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.120 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:38.120 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043228+0000", 2023-11-25T04:23:38.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6699999999999997e-06 2023-11-25T04:23:38.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.121 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:38.121 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043236+0000", 2023-11-25T04:23:38.121 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8189999999999992e-06 2023-11-25T04:23:38.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.121 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:38.121 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043507+0000", 2023-11-25T04:23:38.121 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000271012 2023-11-25T04:23:38.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.121 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:38.121 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043530+0000", 2023-11-25T04:23:38.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2659000000000001e-05 2023-11-25T04:23:38.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.122 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:38.122 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049321+0000", 2023-11-25T04:23:38.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057915129999999999 2023-11-25T04:23:38.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.122 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:38.122 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049375+0000", 2023-11-25T04:23:38.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3455999999999998e-05 2023-11-25T04:23:38.122 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:38.122 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:38.123 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:38.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.123 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:1 3.14 e18/16 3:292cf221:::200.00000000:head v 18'3, mlcod=0'0)", 2023-11-25T04:23:38.123 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:38.123 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.994208942, 2023-11-25T04:23:38.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0060983599999999997, 2023-11-25T04:23:38.123 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:38.123 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:38.123 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:38.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.123 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:38.123 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:38.124 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:38.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.124 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:38.124 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:38.124 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:38.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.124 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:38.124 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043248+0000", 2023-11-25T04:23:38.124 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9399999999999995e-07 2023-11-25T04:23:38.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.124 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:38.125 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043252+0000", 2023-11-25T04:23:38.125 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1799999999999998e-06 2023-11-25T04:23:38.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.125 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:38.125 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043259+0000", 2023-11-25T04:23:38.125 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0049999999999998e-06 2023-11-25T04:23:38.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.125 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:38.125 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043265+0000", 2023-11-25T04:23:38.125 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0859999999999998e-06 2023-11-25T04:23:38.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.126 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:38.126 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043311+0000", 2023-11-25T04:23:38.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6082999999999999e-05 2023-11-25T04:23:38.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.126 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:38.126 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043330+0000", 2023-11-25T04:23:38.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8556e-05 2023-11-25T04:23:38.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.126 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:38.126 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049303+0000", 2023-11-25T04:23:38.127 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059733470000000004 2023-11-25T04:23:38.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.127 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:38.127 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049346+0000", 2023-11-25T04:23:38.127 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2608999999999998e-05 2023-11-25T04:23:38.127 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:38.127 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:38.127 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:38.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.129 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.129 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:3 3.14 e18/16 3:287034c4:::1.00000000.inode:head v 18'4, mlcod=0'0)", 2023-11-25T04:23:38.129 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:38.129 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.993692436000003, 2023-11-25T04:23:38.129 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057040090000000003, 2023-11-25T04:23:38.129 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:38.129 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:38.129 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:38.129 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.129 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:38.129 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:38.130 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:38.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.130 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.130 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:38.130 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:38.130 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:38.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.130 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.130 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:38.130 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:38.130 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9799999999999999e-07 2023-11-25T04:23:38.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:38.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043785+0000", 2023-11-25T04:23:38.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0721999999999999e-05 2023-11-25T04:23:38.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:38.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043793+0000", 2023-11-25T04:23:38.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5839999999999995e-06 2023-11-25T04:23:38.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:38.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043803+0000", 2023-11-25T04:23:38.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0729999999999994e-06 2023-11-25T04:23:38.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.132 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:38.132 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043843+0000", 2023-11-25T04:23:38.132 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0747999999999997e-05 2023-11-25T04:23:38.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.132 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:38.132 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043856+0000", 2023-11-25T04:23:38.132 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2697000000000001e-05 2023-11-25T04:23:38.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.132 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:38.133 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049426+0000", 2023-11-25T04:23:38.133 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055704409999999998 2023-11-25T04:23:38.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.133 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.133 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:38.133 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049468+0000", 2023-11-25T04:23:38.133 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1446000000000003e-05 2023-11-25T04:23:38.133 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:38.133 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:38.133 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:38.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.133 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.133 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "pg_update_log_missing(3.14 epoch 18/16 rep_tid 171 entries 18'5 (0'0) error 3:2e2fa760:::200.00000001:head by mds.0.3:32 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:23:38.134 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:38.134 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.993531472999997, 2023-11-25T04:23:38.134 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005556938, 2023-11-25T04:23:38.134 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:38.134 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2023-11-25T04:23:38.134 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:38.134 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.134 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:38.134 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:38.134 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:38.134 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.134 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.134 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:38.135 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:38.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:38.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.135 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:38.135 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:38.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.03e-07 2023-11-25T04:23:38.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.135 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:38.135 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043931+0000", 2023-11-25T04:23:38.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1079999999999998e-06 2023-11-25T04:23:38.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:38.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043944+0000", 2023-11-25T04:23:38.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2852e-05 2023-11-25T04:23:38.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:38.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043955+0000", 2023-11-25T04:23:38.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1037e-05 2023-11-25T04:23:38.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:38.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044088+0000", 2023-11-25T04:23:38.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013269599999999999 2023-11-25T04:23:38.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:38.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049482+0000", 2023-11-25T04:23:38.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053940419999999999 2023-11-25T04:23:38.137 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:38.137 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:38.137 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:38.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.138 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:10 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'5, mlcod=0'0)", 2023-11-25T04:23:38.138 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:38.138 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.993441445000002, 2023-11-25T04:23:38.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054493099999999997, 2023-11-25T04:23:38.138 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:38.138 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:38.138 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:38.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.138 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:38.138 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:38.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:38.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.139 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:38.139 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:38.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:38.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.139 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:38.139 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:38.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6100000000000002e-07 2023-11-25T04:23:38.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.139 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:38.139 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044019+0000", 2023-11-25T04:23:38.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.839e-06 2023-11-25T04:23:38.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.140 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:38.140 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044026+0000", 2023-11-25T04:23:38.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5869999999999998e-06 2023-11-25T04:23:38.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.140 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:38.140 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044031+0000", 2023-11-25T04:23:38.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4909999999999996e-06 2023-11-25T04:23:38.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.140 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:38.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044070+0000", 2023-11-25T04:23:38.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8829999999999999e-05 2023-11-25T04:23:38.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.141 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.141 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:38.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044081+0000", 2023-11-25T04:23:38.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1447999999999999e-05 2023-11-25T04:23:38.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.141 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.141 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:38.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049406+0000", 2023-11-25T04:23:38.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053246480000000004 2023-11-25T04:23:38.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.142 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:38.142 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049464+0000", 2023-11-25T04:23:38.142 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8205999999999998e-05 2023-11-25T04:23:38.142 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:38.142 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:38.142 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:38.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.142 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:11 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'6, mlcod=0'0)", 2023-11-25T04:23:38.142 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:38.142 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 50.993240180999997, 2023-11-25T04:23:38.142 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052673499999999996, 2023-11-25T04:23:38.143 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:38.143 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:38.143 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:38.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:38.143 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:38.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:38.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:38.143 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:38.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:38.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:38.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:38.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0199999999999998e-07 2023-11-25T04:23:38.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:38.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044220+0000", 2023-11-25T04:23:38.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7720000000000001e-06 2023-11-25T04:23:38.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:38.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044226+0000", 2023-11-25T04:23:38.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8579999999999996e-06 2023-11-25T04:23:38.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:38.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044232+0000", 2023-11-25T04:23:38.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1700000000000002e-06 2023-11-25T04:23:38.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:38.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044274+0000", 2023-11-25T04:23:38.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1254999999999997e-05 2023-11-25T04:23:38.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:38.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:23:38.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6880999999999999e-05 2023-11-25T04:23:38.146 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.146 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:38.146 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049466+0000", 2023-11-25T04:23:38.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051659219999999999 2023-11-25T04:23:38.146 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:38.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:38.146 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:38.146 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049483+0000", 2023-11-25T04:23:38.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.719e-05 2023-11-25T04:23:38.146 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:38.146 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:38.146 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:38.147 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:38.147 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:38.147 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:38.148 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:23:38.149 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:38.147+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:38.249 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:38.246+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:38.255 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:38.256 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:23:38.256 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2023-11-25T04:23:38.256 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:38.265 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:23:38.349 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~29s 2023-11-25T04:23:38.350 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:38.347+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:38.411 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:38.411 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:23:38.411 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2023-11-25T04:23:38.411 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2023-11-25T04:23:38.411 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:38.419 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:23:38.460 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:38.458+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:38.552 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:38.551+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:38.565 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:38.565 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2023-11-25T04:23:38.566 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2023-11-25T04:23:38.566 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2023-11-25T04:23:38.566 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.566 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:38.566 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:38.566 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 51.531626144999997, 2023-11-25T04:23:38.566 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01072943, 2023-11-25T04:23:38.566 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:38.566 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:38.566 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:38.566 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.567 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:38.567 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:38.567 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:38.567 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.567 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.567 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:38.567 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:38.567 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:38.567 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.567 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.567 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:38.567 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031829+0000", 2023-11-25T04:23:38.567 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.666e-06 2023-11-25T04:23:38.567 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.568 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.568 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:38.568 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031839+0000", 2023-11-25T04:23:38.568 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9490000000000008e-06 2023-11-25T04:23:38.568 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.568 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.568 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:38.568 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031857+0000", 2023-11-25T04:23:38.568 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8093000000000001e-05 2023-11-25T04:23:38.568 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.568 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.568 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:38.569 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031882+0000", 2023-11-25T04:23:38.569 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4294000000000002e-05 2023-11-25T04:23:38.569 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.569 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.569 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:38.569 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033130+0000", 2023-11-25T04:23:38.569 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001248264 2023-11-25T04:23:38.569 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.569 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.569 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:38.569 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033426+0000", 2023-11-25T04:23:38.569 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029620999999999998 2023-11-25T04:23:38.569 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.570 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.570 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:38.570 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033814+0000", 2023-11-25T04:23:38.570 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000387742 2023-11-25T04:23:38.570 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.570 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.570 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:38.570 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036793+0000", 2023-11-25T04:23:38.570 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002979598 2023-11-25T04:23:38.570 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.570 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.570 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:38.571 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042455+0000", 2023-11-25T04:23:38.571 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0056619260000000003 2023-11-25T04:23:38.571 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.571 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.571 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:38.571 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042502+0000", 2023-11-25T04:23:38.571 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7139999999999999e-05 2023-11-25T04:23:38.571 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.571 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.571 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:38.571 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042557+0000", 2023-11-25T04:23:38.571 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4548e-05 2023-11-25T04:23:38.571 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:38.572 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:38.572 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:38.572 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.572 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.572 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:38.572 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:38.572 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 51.530427609999997, 2023-11-25T04:23:38.572 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011505202000000001, 2023-11-25T04:23:38.572 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:38.572 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:38.572 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:38.572 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.572 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:38.573 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:38.573 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:38.573 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.573 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.573 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:38.573 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:38.573 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:38.573 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.573 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.573 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:38.573 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033027+0000", 2023-11-25T04:23:38.573 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0699999999999999e-06 2023-11-25T04:23:38.574 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.574 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.574 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:38.574 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033035+0000", 2023-11-25T04:23:38.574 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0830000000000004e-06 2023-11-25T04:23:38.574 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.574 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.574 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:38.574 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033059+0000", 2023-11-25T04:23:38.574 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3975e-05 2023-11-25T04:23:38.574 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.574 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.575 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:38.575 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033482+0000", 2023-11-25T04:23:38.575 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042253700000000001 2023-11-25T04:23:38.575 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.575 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.575 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:38.575 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034520+0000", 2023-11-25T04:23:38.575 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001037991 2023-11-25T04:23:38.575 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.575 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.575 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:38.575 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034611+0000", 2023-11-25T04:23:38.575 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1113999999999995e-05 2023-11-25T04:23:38.576 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.576 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.576 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:38.576 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034769+0000", 2023-11-25T04:23:38.576 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000157641 2023-11-25T04:23:38.576 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.576 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.576 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:38.576 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038292+0000", 2023-11-25T04:23:38.576 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035231160000000002 2023-11-25T04:23:38.576 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.577 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.577 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:38.577 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044440+0000", 2023-11-25T04:23:38.577 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0061486980000000002 2023-11-25T04:23:38.577 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.577 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.577 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:38.577 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044495+0000", 2023-11-25T04:23:38.577 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4339000000000003e-05 2023-11-25T04:23:38.577 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.577 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.577 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:38.577 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044531+0000", 2023-11-25T04:23:38.578 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6637999999999997e-05 2023-11-25T04:23:38.578 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:38.578 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:38.578 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:38.578 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.578 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.578 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:38.578 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:38.578 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 51.529953255999999, 2023-11-25T04:23:38.579 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0057634039999999997, 2023-11-25T04:23:38.579 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:38.579 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:38.579 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:38.579 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.579 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:38.579 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:38.579 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:38.579 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.579 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.579 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:38.579 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:38.580 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:38.580 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.580 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.580 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:38.580 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:38.580 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7900000000000007e-07 2023-11-25T04:23:38.580 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.580 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.580 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:38.580 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033507+0000", 2023-11-25T04:23:38.580 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8900000000000004e-06 2023-11-25T04:23:38.580 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.580 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.581 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:38.581 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033519+0000", 2023-11-25T04:23:38.581 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1756000000000001e-05 2023-11-25T04:23:38.581 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.581 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.581 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:38.581 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033527+0000", 2023-11-25T04:23:38.581 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3610000000000001e-06 2023-11-25T04:23:38.581 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.581 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.581 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:38.581 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034451+0000", 2023-11-25T04:23:38.582 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00092372400000000001 2023-11-25T04:23:38.582 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.582 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.582 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:38.582 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034809+0000", 2023-11-25T04:23:38.582 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00035757699999999998 2023-11-25T04:23:38.582 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.582 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.582 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:23:38.582 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034965+0000", 2023-11-25T04:23:38.582 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015638699999999999 2023-11-25T04:23:38.582 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.583 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.583 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:38.583 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038299+0000", 2023-11-25T04:23:38.583 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0033336559999999999 2023-11-25T04:23:38.583 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.583 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.583 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:38.583 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039169+0000", 2023-11-25T04:23:38.583 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00087066400000000005 2023-11-25T04:23:38.583 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.583 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.583 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:38.584 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039226+0000", 2023-11-25T04:23:38.584 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6511999999999997e-05 2023-11-25T04:23:38.584 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.584 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.584 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:38.584 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039264+0000", 2023-11-25T04:23:38.584 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7898000000000002e-05 2023-11-25T04:23:38.584 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:38.584 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:38.584 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:38.584 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.584 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.584 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:25 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:38.585 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:38.585 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 51.529874671999998, 2023-11-25T04:23:38.585 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096656169999999996, 2023-11-25T04:23:38.585 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:38.585 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:38.585 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:38.585 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.585 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:38.585 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:38.585 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:38.585 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.585 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.586 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:38.586 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:38.586 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:38.586 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.586 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.586 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:38.586 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033580+0000", 2023-11-25T04:23:38.586 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8199999999999999e-07 2023-11-25T04:23:38.586 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.586 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.586 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:38.586 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033585+0000", 2023-11-25T04:23:38.587 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3680000000000001e-06 2023-11-25T04:23:38.587 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.587 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.587 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:38.587 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033600+0000", 2023-11-25T04:23:38.587 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4331000000000001e-05 2023-11-25T04:23:38.587 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.587 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.587 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:38.587 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033606+0000", 2023-11-25T04:23:38.587 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5780000000000004e-06 2023-11-25T04:23:38.587 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.588 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.588 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:38.588 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035747+0000", 2023-11-25T04:23:38.588 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0021406530000000002 2023-11-25T04:23:38.588 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.588 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.588 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:38.588 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035901+0000", 2023-11-25T04:23:38.588 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000153931 2023-11-25T04:23:38.588 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.588 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.588 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:23:38.588 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036193+0000", 2023-11-25T04:23:38.589 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029211299999999999 2023-11-25T04:23:38.589 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.589 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.589 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:38.589 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038655+0000", 2023-11-25T04:23:38.589 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0024621640000000002 2023-11-25T04:23:38.589 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.589 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.589 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:38.589 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043130+0000", 2023-11-25T04:23:38.589 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.004474698 2023-11-25T04:23:38.589 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.590 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.590 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:38.590 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043204+0000", 2023-11-25T04:23:38.590 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4091999999999995e-05 2023-11-25T04:23:38.590 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.590 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.590 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:38.590 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043245+0000", 2023-11-25T04:23:38.590 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0907e-05 2023-11-25T04:23:38.590 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:38.590 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:38.590 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:38.590 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.591 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.591 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:26 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:38.591 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:38.591 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 51.529838847999997, 2023-11-25T04:23:38.591 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011086945000000001, 2023-11-25T04:23:38.591 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:38.591 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:38.591 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:38.591 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.591 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:38.591 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:38.591 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:38.591 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.592 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.592 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:38.592 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:38.592 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:38.592 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.592 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.592 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:38.592 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:38.592 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1699999999999998e-07 2023-11-25T04:23:38.592 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.592 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.592 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:38.593 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033627+0000", 2023-11-25T04:23:38.593 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.163e-05 2023-11-25T04:23:38.593 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.593 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.593 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:38.593 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033636+0000", 2023-11-25T04:23:38.593 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6859999999999995e-06 2023-11-25T04:23:38.593 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.593 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.593 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:38.593 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033642+0000", 2023-11-25T04:23:38.593 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3149999999999997e-06 2023-11-25T04:23:38.593 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.594 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.594 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:38.594 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035414+0000", 2023-11-25T04:23:38.594 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0017716380000000001 2023-11-25T04:23:38.594 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.594 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.594 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:38.594 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035539+0000", 2023-11-25T04:23:38.594 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000125477 2023-11-25T04:23:38.594 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.594 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.594 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:38.594 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035672+0000", 2023-11-25T04:23:38.595 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000133316 2023-11-25T04:23:38.595 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.595 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.595 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:38.595 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038649+0000", 2023-11-25T04:23:38.595 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029764679999999999 2023-11-25T04:23:38.595 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.595 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.595 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:38.595 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044578+0000", 2023-11-25T04:23:38.595 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0059293510000000002 2023-11-25T04:23:38.595 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.595 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.596 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:38.596 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044651+0000", 2023-11-25T04:23:38.596 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2513000000000002e-05 2023-11-25T04:23:38.596 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.596 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.596 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:38.596 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044702+0000", 2023-11-25T04:23:38.596 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1034e-05 2023-11-25T04:23:38.596 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:38.596 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:38.596 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:38.596 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.597 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.597 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:29 3.b 3:d0630e4c:::mds0_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:38.597 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:38.597 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 51.529800928, 2023-11-25T04:23:38.597 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010980227, 2023-11-25T04:23:38.597 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:38.597 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:38.597 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:38.597 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.597 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:38.597 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:38.597 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:38.597 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.598 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.598 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:38.598 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:38.598 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:38.598 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.598 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.598 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:38.598 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:38.598 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3499999999999996e-07 2023-11-25T04:23:38.598 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.598 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.599 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:38.599 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033658+0000", 2023-11-25T04:23:38.599 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1280000000000001e-06 2023-11-25T04:23:38.599 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.599 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.599 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:38.599 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033667+0000", 2023-11-25T04:23:38.599 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1430000000000004e-06 2023-11-25T04:23:38.599 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.599 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.599 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:38.599 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033678+0000", 2023-11-25T04:23:38.599 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0906e-05 2023-11-25T04:23:38.600 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.600 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.600 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:38.600 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034456+0000", 2023-11-25T04:23:38.600 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00077833899999999996 2023-11-25T04:23:38.600 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.600 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.600 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:38.600 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034575+0000", 2023-11-25T04:23:38.600 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011926000000000001 2023-11-25T04:23:38.600 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.600 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.600 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:38.601 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035001+0000", 2023-11-25T04:23:38.601 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042550999999999998 2023-11-25T04:23:38.601 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.601 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.601 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:38.601 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.037923+0000", 2023-11-25T04:23:38.601 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029219670000000001 2023-11-25T04:23:38.601 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.601 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.601 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:38.601 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044538+0000", 2023-11-25T04:23:38.601 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0066152349999999997 2023-11-25T04:23:38.601 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.602 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.602 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:38.602 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044594+0000", 2023-11-25T04:23:38.602 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5593000000000001e-05 2023-11-25T04:23:38.602 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.602 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.602 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:38.602 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044633+0000", 2023-11-25T04:23:38.602 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9511000000000002e-05 2023-11-25T04:23:38.602 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:38.602 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:38.602 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:38.603 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.603 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.603 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:34 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:38.603 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:38.603 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 51.529550815, 2023-11-25T04:23:38.603 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010920445000000001, 2023-11-25T04:23:38.603 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:38.603 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:38.603 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:38.603 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.603 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:38.603 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:38.603 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:38.603 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.604 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.604 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:38.604 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:38.604 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:38.604 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.604 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.604 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:38.604 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033904+0000", 2023-11-25T04:23:38.604 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6599999999999994e-07 2023-11-25T04:23:38.604 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.604 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.604 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:38.605 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033909+0000", 2023-11-25T04:23:38.605 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5639999999999996e-06 2023-11-25T04:23:38.605 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.605 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.605 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:38.605 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033923+0000", 2023-11-25T04:23:38.605 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3028e-05 2023-11-25T04:23:38.605 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.605 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.605 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:38.605 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034043+0000", 2023-11-25T04:23:38.605 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000120328 2023-11-25T04:23:38.605 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.605 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.606 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:38.606 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035576+0000", 2023-11-25T04:23:38.606 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015333720000000001 2023-11-25T04:23:38.606 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.606 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.606 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:38.606 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035699+0000", 2023-11-25T04:23:38.606 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000123001 2023-11-25T04:23:38.606 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.606 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.606 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:38.606 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044823+0000", 2023-11-25T04:23:38.606 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0091241859999999994 2023-11-25T04:23:38.607 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:38.607 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:38.607 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:38.607 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.607 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.607 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:35 3.1f 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:38.607 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:38.607 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 51.528246609999997, 2023-11-25T04:23:38.607 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096164779999999995, 2023-11-25T04:23:38.607 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:38.607 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:38.607 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:38.607 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.607 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:38.608 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:38.608 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:38.608 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.608 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.608 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:38.608 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:38.608 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:38.608 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.608 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.608 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:38.608 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035208+0000", 2023-11-25T04:23:38.608 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2699999999999998e-07 2023-11-25T04:23:38.608 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.608 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.609 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:38.609 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035213+0000", 2023-11-25T04:23:38.609 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.6070000000000001e-06 2023-11-25T04:23:38.609 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.609 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.609 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:38.609 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035350+0000", 2023-11-25T04:23:38.609 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013693599999999999 2023-11-25T04:23:38.609 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.609 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.609 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:38.609 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035358+0000", 2023-11-25T04:23:38.609 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0099999999999995e-06 2023-11-25T04:23:38.610 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.610 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.610 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:38.610 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036260+0000", 2023-11-25T04:23:38.610 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00090240100000000005 2023-11-25T04:23:38.610 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.610 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.610 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:38.610 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036517+0000", 2023-11-25T04:23:38.610 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00025736299999999998 2023-11-25T04:23:38.610 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.610 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.610 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:38.610 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044824+0000", 2023-11-25T04:23:38.611 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0083063340000000003 2023-11-25T04:23:38.611 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:38.611 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:38.611 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:38.611 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.611 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.611 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:37 3.1e 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:38.611 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:38.611 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 51.528079923999996, 2023-11-25T04:23:38.611 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0078322640000000002, 2023-11-25T04:23:38.611 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:38.611 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:38.611 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:38.612 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.612 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:38.612 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:38.612 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:38.612 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.612 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.612 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:38.612 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:38.612 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:38.612 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.612 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.612 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:38.612 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035375+0000", 2023-11-25T04:23:38.612 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9100000000000003e-07 2023-11-25T04:23:38.612 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.613 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.613 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:38.613 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035379+0000", 2023-11-25T04:23:38.613 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9430000000000002e-06 2023-11-25T04:23:38.613 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.613 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.613 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:38.613 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035388+0000", 2023-11-25T04:23:38.613 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2760000000000006e-06 2023-11-25T04:23:38.613 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.613 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.613 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:38.613 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035396+0000", 2023-11-25T04:23:38.613 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8480000000000001e-06 2023-11-25T04:23:38.614 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.614 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.614 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:38.614 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035452+0000", 2023-11-25T04:23:38.614 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5831000000000001e-05 2023-11-25T04:23:38.614 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.614 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.614 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:38.614 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035934+0000", 2023-11-25T04:23:38.614 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00048265599999999998 2023-11-25T04:23:38.614 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.614 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.614 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:38.615 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043206+0000", 2023-11-25T04:23:38.615 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.007271919 2023-11-25T04:23:38.615 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:38.615 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:38.615 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:38.615 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.615 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.615 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing(3.1a epoch 18/16 rep_tid 3 entries 18'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:36 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:23:38.615 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:38.615 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 51.526804337999998, 2023-11-25T04:23:38.615 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002073414, 2023-11-25T04:23:38.615 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:38.615 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:38.615 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:38.616 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.616 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:38.616 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:38.616 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:38.616 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.616 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.616 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:38.616 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:38.616 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:38.616 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.616 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.616 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:38.616 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036650+0000", 2023-11-25T04:23:38.616 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7300000000000002e-07 2023-11-25T04:23:38.617 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.617 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.617 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:38.617 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036659+0000", 2023-11-25T04:23:38.617 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.7499999999999992e-06 2023-11-25T04:23:38.617 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.617 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.617 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:38.617 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036673+0000", 2023-11-25T04:23:38.617 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4917e-05 2023-11-25T04:23:38.617 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.617 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.617 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:38.618 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036687+0000", 2023-11-25T04:23:38.618 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3883000000000001e-05 2023-11-25T04:23:38.618 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.618 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.618 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:38.618 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036781+0000", 2023-11-25T04:23:38.618 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3219000000000003e-05 2023-11-25T04:23:38.618 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.618 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.618 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:38.618 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038723+0000", 2023-11-25T04:23:38.618 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019422720000000001 2023-11-25T04:23:38.618 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:38.618 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:38.619 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:38.619 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.619 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.619 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:28 3.13 e18/16 3:cd5a64a3:::100.00000000:head v 18'1, mlcod=0'0)", 2023-11-25T04:23:38.619 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:38.619 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 51.523143179999998, 2023-11-25T04:23:38.619 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001850546, 2023-11-25T04:23:38.619 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:38.619 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:38.619 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:38.619 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.619 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:38.619 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:38.620 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:38.620 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.620 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.620 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:38.620 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:38.620 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:38.620 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.620 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.620 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:38.620 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:38.620 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7800000000000002e-07 2023-11-25T04:23:38.620 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.620 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.620 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:38.621 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040331+0000", 2023-11-25T04:23:38.621 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0449999999999999e-05 2023-11-25T04:23:38.621 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.621 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.621 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:38.621 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040339+0000", 2023-11-25T04:23:38.621 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0709999999999995e-06 2023-11-25T04:23:38.621 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.621 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.621 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:38.621 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040347+0000", 2023-11-25T04:23:38.621 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7079999999999997e-06 2023-11-25T04:23:38.621 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.621 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.622 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:38.622 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040410+0000", 2023-11-25T04:23:38.622 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2453000000000007e-05 2023-11-25T04:23:38.622 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.622 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.622 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:38.622 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040422+0000", 2023-11-25T04:23:38.622 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2429000000000001e-05 2023-11-25T04:23:38.622 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.622 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.622 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:38.622 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042129+0000", 2023-11-25T04:23:38.622 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001707266 2023-11-25T04:23:38.622 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.623 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.623 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:38.623 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042161+0000", 2023-11-25T04:23:38.623 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1791000000000001e-05 2023-11-25T04:23:38.623 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:38.623 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:38.623 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:38.623 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.623 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.623 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:27 3.13 e18/16 3:c91d4a1d:::mds0_inotable:head v 18'2, mlcod=0'0)", 2023-11-25T04:23:38.623 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:38.623 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 51.522939657000002, 2023-11-25T04:23:38.623 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0016886430000000001, 2023-11-25T04:23:38.623 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:38.624 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:38.624 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:38.624 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.624 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:38.624 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:38.624 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:38.624 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.624 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.624 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:38.624 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:38.624 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:38.624 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.624 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.624 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:38.625 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:38.625 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3999999999999997e-07 2023-11-25T04:23:38.625 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.625 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.625 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:38.625 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040520+0000", 2023-11-25T04:23:38.625 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2989999999999996e-06 2023-11-25T04:23:38.625 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.625 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.625 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:38.625 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040527+0000", 2023-11-25T04:23:38.625 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5730000000000002e-06 2023-11-25T04:23:38.625 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.625 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.626 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:38.626 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040535+0000", 2023-11-25T04:23:38.626 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8630000000000008e-06 2023-11-25T04:23:38.626 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.626 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.626 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:38.626 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040683+0000", 2023-11-25T04:23:38.626 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000148002 2023-11-25T04:23:38.626 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.626 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.626 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:38.626 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040700+0000", 2023-11-25T04:23:38.626 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7065e-05 2023-11-25T04:23:38.626 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.627 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.627 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:38.627 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042163+0000", 2023-11-25T04:23:38.627 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001462767 2023-11-25T04:23:38.627 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.627 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.627 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:38.627 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042203+0000", 2023-11-25T04:23:38.627 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9734000000000003e-05 2023-11-25T04:23:38.627 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:38.627 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:38.627 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:38.627 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.627 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.628 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:20 3.1c e18/16 3:3ed09add:::607.00000000:head v 18'1, mlcod=0'0)", 2023-11-25T04:23:38.628 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:38.628 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 51.522904615999998, 2023-11-25T04:23:38.628 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015271, 2023-11-25T04:23:38.628 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:38.628 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:38.628 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:38.628 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.628 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:38.628 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:38.628 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:38.628 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.628 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.628 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:38.629 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:38.629 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:38.629 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.629 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.629 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:38.629 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:38.629 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4799999999999999e-07 2023-11-25T04:23:38.629 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.629 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.629 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:38.629 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040561+0000", 2023-11-25T04:23:38.629 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1800000000000001e-05 2023-11-25T04:23:38.629 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.629 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.630 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:38.630 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040568+0000", 2023-11-25T04:23:38.630 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7730000000000002e-06 2023-11-25T04:23:38.630 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.630 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.630 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:38.630 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040587+0000", 2023-11-25T04:23:38.630 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8691000000000001e-05 2023-11-25T04:23:38.630 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.630 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.630 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:38.630 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040638+0000", 2023-11-25T04:23:38.630 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1068e-05 2023-11-25T04:23:38.630 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.631 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.631 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:38.631 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040670+0000", 2023-11-25T04:23:38.631 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2125999999999997e-05 2023-11-25T04:23:38.631 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.631 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.631 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:38.631 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042032+0000", 2023-11-25T04:23:38.631 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001361821 2023-11-25T04:23:38.631 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.631 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.631 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:38.631 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042076+0000", 2023-11-25T04:23:38.631 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4472999999999999e-05 2023-11-25T04:23:38.632 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:38.632 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:38.632 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:38.632 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.632 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.632 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:21 3.1c e18/16 3:3ed09add:::607.00000000:head v 18'2, mlcod=0'0)", 2023-11-25T04:23:38.632 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:38.632 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 51.522669891, 2023-11-25T04:23:38.632 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0013228160000000001, 2023-11-25T04:23:38.632 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:38.632 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:38.632 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:38.632 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.632 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:38.633 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:38.633 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:38.633 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.633 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.633 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:38.633 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:38.633 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:38.633 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.633 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.633 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:38.633 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:38.633 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9299999999999999e-07 2023-11-25T04:23:38.633 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.633 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.634 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:38.634 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040790+0000", 2023-11-25T04:23:38.634 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5770000000000001e-06 2023-11-25T04:23:38.634 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.634 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.634 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:38.634 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040799+0000", 2023-11-25T04:23:38.634 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6439999999999995e-06 2023-11-25T04:23:38.634 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.634 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.634 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:38.634 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040806+0000", 2023-11-25T04:23:38.634 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0180000000000004e-06 2023-11-25T04:23:38.634 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.635 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.635 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:38.635 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040907+0000", 2023-11-25T04:23:38.635 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000100096 2023-11-25T04:23:38.635 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.635 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.635 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:38.635 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040925+0000", 2023-11-25T04:23:38.635 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8536999999999999e-05 2023-11-25T04:23:38.635 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.635 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.635 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:38.635 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042079+0000", 2023-11-25T04:23:38.636 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00115358 2023-11-25T04:23:38.636 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.636 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.636 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:38.636 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042107+0000", 2023-11-25T04:23:38.636 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7971e-05 2023-11-25T04:23:38.636 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:38.636 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:38.636 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:38.636 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.636 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.636 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop_reply(mds.0.3:4 3.1f e18/16 ondisk, result = 0)", 2023-11-25T04:23:38.636 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:38.636 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 51.521330501000001, 2023-11-25T04:23:38.637 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00044092900000000001, 2023-11-25T04:23:38.637 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:38.637 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:38.637 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:38.637 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.637 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:38.637 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:38.637 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:38.637 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.637 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.637 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:38.637 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:38.637 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:38.637 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.637 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.638 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:38.638 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042124+0000", 2023-11-25T04:23:38.638 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1800000000000005e-07 2023-11-25T04:23:38.638 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.638 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.638 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:38.638 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042128+0000", 2023-11-25T04:23:38.638 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2289999999999997e-06 2023-11-25T04:23:38.638 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.638 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.638 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:38.638 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042138+0000", 2023-11-25T04:23:38.638 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9729999999999992e-06 2023-11-25T04:23:38.639 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.639 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.639 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:38.639 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042145+0000", 2023-11-25T04:23:38.639 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.336e-06 2023-11-25T04:23:38.639 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.639 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.639 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:38.639 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042441+0000", 2023-11-25T04:23:38.639 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029650500000000002 2023-11-25T04:23:38.639 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.639 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.639 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:38.639 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042448+0000", 2023-11-25T04:23:38.640 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4100000000000002e-06 2023-11-25T04:23:38.640 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.640 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.640 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:38.640 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042564+0000", 2023-11-25T04:23:38.640 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000115758 2023-11-25T04:23:38.640 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:38.640 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:38.640 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:38.640 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.640 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.640 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing_reply(3.1f epoch 18/16 rep_tid 8 lcod 18'4)", 2023-11-25T04:23:38.640 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:38.640 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 51.519005088999997, 2023-11-25T04:23:38.641 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00038154699999999998, 2023-11-25T04:23:38.641 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:38.641 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:38.641 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:38.641 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.641 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:38.641 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:38.641 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:38.641 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.641 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.641 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:38.641 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:38.641 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:38.641 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.642 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.642 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:38.642 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:38.642 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2700000000000001e-07 2023-11-25T04:23:38.642 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.642 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.642 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:38.642 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044451+0000", 2023-11-25T04:23:38.642 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7460000000000001e-06 2023-11-25T04:23:38.642 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.642 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.642 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:38.642 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044455+0000", 2023-11-25T04:23:38.642 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5809999999999998e-06 2023-11-25T04:23:38.643 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.643 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.643 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:38.643 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044458+0000", 2023-11-25T04:23:38.643 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0170000000000001e-06 2023-11-25T04:23:38.643 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.643 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.643 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:38.643 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044734+0000", 2023-11-25T04:23:38.643 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00027582799999999999 2023-11-25T04:23:38.643 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.643 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.643 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:38.643 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044830+0000", 2023-11-25T04:23:38.644 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6248000000000004e-05 2023-11-25T04:23:38.644 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:38.644 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:38.644 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:38.644 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.644 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.644 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing_reply(3.b epoch 18/16 rep_tid 5 lcod 18'2)", 2023-11-25T04:23:38.644 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:38.644 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 51.518968428000001, 2023-11-25T04:23:38.644 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000344488, 2023-11-25T04:23:38.644 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:38.645 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:38.645 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:38.645 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.645 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:38.645 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:38.645 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:38.645 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.645 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.645 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:38.645 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:38.645 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:38.645 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.645 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.645 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:38.646 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:38.646 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.55e-07 2023-11-25T04:23:38.646 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.646 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.646 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:38.646 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044487+0000", 2023-11-25T04:23:38.646 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2470000000000001e-06 2023-11-25T04:23:38.646 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.646 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.646 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:38.646 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044491+0000", 2023-11-25T04:23:38.646 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9659999999999998e-06 2023-11-25T04:23:38.646 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.647 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.647 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:38.647 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044494+0000", 2023-11-25T04:23:38.647 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2310000000000001e-06 2023-11-25T04:23:38.647 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.647 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.647 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:38.647 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044756+0000", 2023-11-25T04:23:38.647 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00026172800000000003 2023-11-25T04:23:38.647 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.647 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.647 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:38.647 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044830+0000", 2023-11-25T04:23:38.647 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4160999999999997e-05 2023-11-25T04:23:38.647 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:38.648 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:38.648 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:38.648 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.648 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.648 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:39 3.6 3:654134d2:::mds0_openfiles.0:head [omap-clear,omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2023-11-25T04:23:38.648 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:38.648 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 46.459142386000003, 2023-11-25T04:23:38.648 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001788658, 2023-11-25T04:23:38.648 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:38.648 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:38.648 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:38.648 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.648 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:38.648 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:38.649 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:38.649 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.649 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.649 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:38.649 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:38.649 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:38.649 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.649 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.649 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:38.649 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104314+0000", 2023-11-25T04:23:38.649 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3099999999999999e-06 2023-11-25T04:23:38.649 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.649 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.649 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:38.650 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104325+0000", 2023-11-25T04:23:38.650 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1511999999999999e-05 2023-11-25T04:23:38.650 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.650 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.650 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:38.650 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104362+0000", 2023-11-25T04:23:38.650 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6903000000000002e-05 2023-11-25T04:23:38.650 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.650 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.650 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:38.650 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104387+0000", 2023-11-25T04:23:38.650 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4766e-05 2023-11-25T04:23:38.651 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.651 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.651 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:38.651 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104465+0000", 2023-11-25T04:23:38.651 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8119000000000002e-05 2023-11-25T04:23:38.651 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.651 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.651 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:38.651 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104557+0000", 2023-11-25T04:23:38.651 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2496000000000003e-05 2023-11-25T04:23:38.651 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.651 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.651 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:38.652 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104646+0000", 2023-11-25T04:23:38.652 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8818000000000002e-05 2023-11-25T04:23:38.652 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.652 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.652 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:38.652 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105201+0000", 2023-11-25T04:23:38.652 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00055506000000000002 2023-11-25T04:23:38.652 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.652 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.652 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:38.652 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105994+0000", 2023-11-25T04:23:38.652 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00079294700000000001 2023-11-25T04:23:38.652 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.652 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.653 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:38.653 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.106055+0000", 2023-11-25T04:23:38.653 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0566000000000003e-05 2023-11-25T04:23:38.653 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.653 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.653 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:38.653 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.106100+0000", 2023-11-25T04:23:38.653 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5160999999999997e-05 2023-11-25T04:23:38.653 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:38.653 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:38.653 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:38.653 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.653 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.653 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:43 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2023-11-25T04:23:38.654 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:38.654 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 31.45891739, 2023-11-25T04:23:38.654 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001751166, 2023-11-25T04:23:38.654 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:38.654 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:38.654 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:38.654 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.654 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:38.654 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:38.654 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:38.654 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.654 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.654 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:38.655 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:38.655 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:38.655 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.655 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.655 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:38.655 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104539+0000", 2023-11-25T04:23:38.655 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9529999999999999e-06 2023-11-25T04:23:38.655 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.655 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.655 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:38.655 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104557+0000", 2023-11-25T04:23:38.655 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7464000000000001e-05 2023-11-25T04:23:38.655 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.656 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.656 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:38.656 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104590+0000", 2023-11-25T04:23:38.656 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2966000000000003e-05 2023-11-25T04:23:38.656 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.656 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.656 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:38.656 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104608+0000", 2023-11-25T04:23:38.656 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8193e-05 2023-11-25T04:23:38.656 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.656 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.656 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:38.656 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104767+0000", 2023-11-25T04:23:38.656 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015882 2023-11-25T04:23:38.657 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.657 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.657 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:38.657 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104909+0000", 2023-11-25T04:23:38.657 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014244999999999999 2023-11-25T04:23:38.657 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.657 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.657 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:38.657 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.105059+0000", 2023-11-25T04:23:38.657 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014964399999999999 2023-11-25T04:23:38.657 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.657 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.657 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:38.657 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.105701+0000", 2023-11-25T04:23:38.658 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00064201099999999995 2023-11-25T04:23:38.658 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.658 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.658 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:38.658 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106233+0000", 2023-11-25T04:23:38.658 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00053178500000000001 2023-11-25T04:23:38.658 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.658 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.658 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:38.658 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106265+0000", 2023-11-25T04:23:38.658 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1986000000000002e-05 2023-11-25T04:23:38.658 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.658 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.658 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:38.659 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106288+0000", 2023-11-25T04:23:38.659 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2894000000000001e-05 2023-11-25T04:23:38.659 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:38.659 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:38.659 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:38.659 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.659 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.659 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:45 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e23)", 2023-11-25T04:23:38.659 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:23:38.659 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 26.458719349999999, 2023-11-25T04:23:38.659 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00098423299999999994, 2023-11-25T04:23:38.659 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:38.659 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:38.659 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:38.660 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.660 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:38.660 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:23:38.660 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:38.660 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.660 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.660 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:38.660 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:23:38.660 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:38.660 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.660 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.660 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:38.660 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104737+0000", 2023-11-25T04:23:38.660 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3309999999999998e-06 2023-11-25T04:23:38.661 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.661 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.661 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:38.661 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104748+0000", 2023-11-25T04:23:38.661 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1080999999999999e-05 2023-11-25T04:23:38.661 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.661 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.661 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:38.661 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104771+0000", 2023-11-25T04:23:38.661 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2707000000000001e-05 2023-11-25T04:23:38.661 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.661 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.661 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:38.661 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104800+0000", 2023-11-25T04:23:38.662 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9595e-05 2023-11-25T04:23:38.662 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.662 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.662 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:38.662 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104890+0000", 2023-11-25T04:23:38.662 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.9808999999999993e-05 2023-11-25T04:23:38.662 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.662 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.662 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:38.662 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105004+0000", 2023-11-25T04:23:38.662 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000114508 2023-11-25T04:23:38.662 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.663 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.663 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:38.663 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105634+0000", 2023-11-25T04:23:38.663 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00062985799999999996 2023-11-25T04:23:38.663 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.663 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.663 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:38.663 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105671+0000", 2023-11-25T04:23:38.663 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6640000000000002e-05 2023-11-25T04:23:38.663 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:38.663 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:38.663 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:38.663 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105719+0000", 2023-11-25T04:23:38.663 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7704000000000003e-05 2023-11-25T04:23:38.664 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:38.664 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:38.664 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:38.664 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:38.664 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:38.664 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:38.665 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:38.664+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:38.666 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:23:38.766 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:38.764+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:38.771 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:38.772 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:23:38.772 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2023-11-25T04:23:38.772 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:38.780 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:23:38.867 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:38.865+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:38.884 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:38.884 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:23:38.884 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2023-11-25T04:23:38.884 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2023-11-25T04:23:38.884 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:38.892 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:23:38.963 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-11-25T04:23:38.968 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:38.967+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:39.001 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:39.001 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2023-11-25T04:23:39.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2023-11-25T04:23:39.001 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2023-11-25T04:23:39.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.001 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:155 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'109, mlcod=11'105)", 2023-11-25T04:23:39.002 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:39.002 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 61.214027674999997, 2023-11-25T04:23:39.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061719019999999999, 2023-11-25T04:23:39.002 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:39.002 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:39.002 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:39.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:39.002 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:39.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:39.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:39.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:39.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:39.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:39.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784884+0000", 2023-11-25T04:23:39.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3099999999999999e-07 2023-11-25T04:23:39.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:39.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784942+0000", 2023-11-25T04:23:39.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8570000000000003e-05 2023-11-25T04:23:39.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:39.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784949+0000", 2023-11-25T04:23:39.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0310000000000001e-06 2023-11-25T04:23:39.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:39.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784953+0000", 2023-11-25T04:23:39.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6320000000000001e-06 2023-11-25T04:23:39.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:39.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784993+0000", 2023-11-25T04:23:39.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9737000000000003e-05 2023-11-25T04:23:39.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:39.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785011+0000", 2023-11-25T04:23:39.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8349999999999999e-05 2023-11-25T04:23:39.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:39.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791024+0000", 2023-11-25T04:23:39.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006012997 2023-11-25T04:23:39.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:39.006 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791055+0000", 2023-11-25T04:23:39.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1254000000000003e-05 2023-11-25T04:23:39.006 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:39.006 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:39.006 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:39.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.006 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:156 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'110, mlcod=11'105)", 2023-11-25T04:23:39.006 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:39.006 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 61.213441625999998, 2023-11-25T04:23:39.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005616116, 2023-11-25T04:23:39.007 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:39.007 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:39.007 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:39.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:39.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:39.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:39.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:39.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:39.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:39.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:39.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785470+0000", 2023-11-25T04:23:39.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0699999999999998e-07 2023-11-25T04:23:39.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:39.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785474+0000", 2023-11-25T04:23:39.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.048e-06 2023-11-25T04:23:39.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:39.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785480+0000", 2023-11-25T04:23:39.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7729999999999996e-06 2023-11-25T04:23:39.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:39.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785483+0000", 2023-11-25T04:23:39.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1970000000000001e-06 2023-11-25T04:23:39.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:39.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787936+0000", 2023-11-25T04:23:39.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024529040000000001 2023-11-25T04:23:39.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:39.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787947+0000", 2023-11-25T04:23:39.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1669e-05 2023-11-25T04:23:39.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:39.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791057+0000", 2023-11-25T04:23:39.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031095200000000002 2023-11-25T04:23:39.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:39.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791086+0000", 2023-11-25T04:23:39.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8697999999999999e-05 2023-11-25T04:23:39.010 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:39.011 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:39.011 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:39.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.011 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:157 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'111, mlcod=11'106)", 2023-11-25T04:23:39.011 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:39.011 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 61.212849067999997, 2023-11-25T04:23:39.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050590280000000001, 2023-11-25T04:23:39.011 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:39.011 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:39.011 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:39.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:39.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:39.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:39.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:39.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:39.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:39.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:39.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786063+0000", 2023-11-25T04:23:39.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2600000000000002e-07 2023-11-25T04:23:39.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:39.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786069+0000", 2023-11-25T04:23:39.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8839999999999999e-06 2023-11-25T04:23:39.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:39.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786079+0000", 2023-11-25T04:23:39.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0124e-05 2023-11-25T04:23:39.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:39.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786085+0000", 2023-11-25T04:23:39.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6660000000000002e-06 2023-11-25T04:23:39.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:39.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788160+0000", 2023-11-25T04:23:39.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020747669999999999 2023-11-25T04:23:39.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:39.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788180+0000", 2023-11-25T04:23:39.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0208999999999999e-05 2023-11-25T04:23:39.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:39.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791087+0000", 2023-11-25T04:23:39.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029064630000000002 2023-11-25T04:23:39.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:39.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791121+0000", 2023-11-25T04:23:39.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4289000000000003e-05 2023-11-25T04:23:39.015 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:39.015 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:39.015 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:39.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.016 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:158 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'112, mlcod=11'106)", 2023-11-25T04:23:39.016 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:39.016 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 61.212432675000002, 2023-11-25T04:23:39.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046663030000000001, 2023-11-25T04:23:39.016 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:39.016 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:39.016 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:39.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.016 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:39.016 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:39.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:39.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:39.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:39.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:39.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:39.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786479+0000", 2023-11-25T04:23:39.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2799999999999996e-07 2023-11-25T04:23:39.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:39.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786498+0000", 2023-11-25T04:23:39.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9069e-05 2023-11-25T04:23:39.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:39.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786508+0000", 2023-11-25T04:23:39.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5810000000000003e-06 2023-11-25T04:23:39.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:39.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786512+0000", 2023-11-25T04:23:39.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4889999999999997e-06 2023-11-25T04:23:39.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:39.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788438+0000", 2023-11-25T04:23:39.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019257059999999999 2023-11-25T04:23:39.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:39.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788449+0000", 2023-11-25T04:23:39.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1419e-05 2023-11-25T04:23:39.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:39.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791122+0000", 2023-11-25T04:23:39.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002672822 2023-11-25T04:23:39.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.020 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:39.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791145+0000", 2023-11-25T04:23:39.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2589e-05 2023-11-25T04:23:39.020 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:39.020 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:39.020 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:39.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.021 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:206 1.0 e12/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 12'152, mlcod=12'148)", 2023-11-25T04:23:39.021 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:39.021 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 60.700664205000002, 2023-11-25T04:23:39.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046728189999999999, 2023-11-25T04:23:39.021 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:39.021 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:39.021 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:39.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.021 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:39.021 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:39.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:39.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:39.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:39.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:39.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:39.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298248+0000", 2023-11-25T04:23:39.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3600000000000003e-07 2023-11-25T04:23:39.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:39.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298278+0000", 2023-11-25T04:23:39.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0474e-05 2023-11-25T04:23:39.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:39.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298288+0000", 2023-11-25T04:23:39.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0156000000000001e-05 2023-11-25T04:23:39.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:39.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298293+0000", 2023-11-25T04:23:39.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7400000000000004e-06 2023-11-25T04:23:39.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:39.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298333+0000", 2023-11-25T04:23:39.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9666000000000003e-05 2023-11-25T04:23:39.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:39.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298351+0000", 2023-11-25T04:23:39.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8544999999999999e-05 2023-11-25T04:23:39.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:39.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302891+0000", 2023-11-25T04:23:39.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045398519999999996 2023-11-25T04:23:39.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:39.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302920+0000", 2023-11-25T04:23:39.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8750000000000001e-05 2023-11-25T04:23:39.025 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:39.025 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:39.025 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:39.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.025 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:14 3.15 3:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:39.025 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:39.025 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.965529848000003, 2023-11-25T04:23:39.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010990437, 2023-11-25T04:23:39.026 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:39.026 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:39.026 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:39.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:39.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:39.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:39.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:39.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:39.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:39.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:39.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033383+0000", 2023-11-25T04:23:39.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9870000000000002e-06 2023-11-25T04:23:39.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:39.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033398+0000", 2023-11-25T04:23:39.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4569999999999999e-05 2023-11-25T04:23:39.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:39.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033411+0000", 2023-11-25T04:23:39.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3543999999999999e-05 2023-11-25T04:23:39.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:39.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033423+0000", 2023-11-25T04:23:39.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1905999999999999e-05 2023-11-25T04:23:39.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:39.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034433+0000", 2023-11-25T04:23:39.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00101022 2023-11-25T04:23:39.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:39.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034539+0000", 2023-11-25T04:23:39.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010597199999999999 2023-11-25T04:23:39.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:39.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034667+0000", 2023-11-25T04:23:39.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012765500000000001 2023-11-25T04:23:39.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:39.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037803+0000", 2023-11-25T04:23:39.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031362220000000001 2023-11-25T04:23:39.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:39.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:23:39.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0064964669999999997 2023-11-25T04:23:39.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:39.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044331+0000", 2023-11-25T04:23:39.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1544999999999997e-05 2023-11-25T04:23:39.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:39.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044372+0000", 2023-11-25T04:23:39.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0349000000000003e-05 2023-11-25T04:23:39.031 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:39.031 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:39.031 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:39.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.031 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:15 3.15 3:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:39.031 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:39.031 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.964870198, 2023-11-25T04:23:39.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010464362, 2023-11-25T04:23:39.032 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:39.032 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:39.032 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:39.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:39.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:39.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:39.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:39.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:39.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:39.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.033 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:39.033 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034042+0000", 2023-11-25T04:23:39.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0999999999999997e-07 2023-11-25T04:23:39.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.033 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:39.033 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034048+0000", 2023-11-25T04:23:39.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0390000000000001e-06 2023-11-25T04:23:39.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.033 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:39.033 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034059+0000", 2023-11-25T04:23:39.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1182999999999999e-05 2023-11-25T04:23:39.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:39.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034067+0000", 2023-11-25T04:23:39.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1559999999999995e-06 2023-11-25T04:23:39.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:39.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035005+0000", 2023-11-25T04:23:39.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00093747399999999999 2023-11-25T04:23:39.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:39.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035077+0000", 2023-11-25T04:23:39.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2747e-05 2023-11-25T04:23:39.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:39.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035219+0000", 2023-11-25T04:23:39.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000141877 2023-11-25T04:23:39.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:39.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037810+0000", 2023-11-25T04:23:39.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002590752 2023-11-25T04:23:39.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:39.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044437+0000", 2023-11-25T04:23:39.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066270110000000004 2023-11-25T04:23:39.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:39.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044481+0000", 2023-11-25T04:23:39.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4106000000000001e-05 2023-11-25T04:23:39.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.037 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:39.037 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044505+0000", 2023-11-25T04:23:39.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4207e-05 2023-11-25T04:23:39.037 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:39.037 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:39.037 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:39.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.037 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:16 3.1d 3:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:39.037 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:39.037 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.964828349999998, 2023-11-25T04:23:39.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010413923, 2023-11-25T04:23:39.038 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:39.038 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:39.038 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:39.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:39.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:39.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:39.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:39.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:39.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:39.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:39.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:39.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4499999999999997e-07 2023-11-25T04:23:39.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:39.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034087+0000", 2023-11-25T04:23:39.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9770000000000002e-06 2023-11-25T04:23:39.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:39.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034094+0000", 2023-11-25T04:23:39.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8809999999999998e-06 2023-11-25T04:23:39.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:39.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034100+0000", 2023-11-25T04:23:39.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8440000000000003e-06 2023-11-25T04:23:39.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:39.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034993+0000", 2023-11-25T04:23:39.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00089336799999999996 2023-11-25T04:23:39.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:39.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035099+0000", 2023-11-25T04:23:39.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106037 2023-11-25T04:23:39.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:39.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035276+0000", 2023-11-25T04:23:39.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017631799999999999 2023-11-25T04:23:39.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:39.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037815+0000", 2023-11-25T04:23:39.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025395959999999999 2023-11-25T04:23:39.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:39.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044420+0000", 2023-11-25T04:23:39.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066045749999999997 2023-11-25T04:23:39.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:39.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044463+0000", 2023-11-25T04:23:39.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3205999999999999e-05 2023-11-25T04:23:39.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:39.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044497+0000", 2023-11-25T04:23:39.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3476000000000003e-05 2023-11-25T04:23:39.043 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:39.043 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:39.043 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:39.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.043 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:17 3.1d 3:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:39.043 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:39.043 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.964803140999997, 2023-11-25T04:23:39.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01127356, 2023-11-25T04:23:39.044 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:39.044 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:39.044 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:39.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:39.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:39.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:39.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:39.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:39.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:39.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:39.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:39.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1100000000000001e-07 2023-11-25T04:23:39.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:39.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034112+0000", 2023-11-25T04:23:39.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8600000000000003e-06 2023-11-25T04:23:39.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:39.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034120+0000", 2023-11-25T04:23:39.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7700000000000001e-06 2023-11-25T04:23:39.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:39.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034127+0000", 2023-11-25T04:23:39.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0999999999999998e-06 2023-11-25T04:23:39.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:39.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035601+0000", 2023-11-25T04:23:39.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014739410000000001 2023-11-25T04:23:39.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:39.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035685+0000", 2023-11-25T04:23:39.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3677999999999994e-05 2023-11-25T04:23:39.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:39.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035827+0000", 2023-11-25T04:23:39.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014232599999999999 2023-11-25T04:23:39.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:39.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038725+0000", 2023-11-25T04:23:39.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028982779999999998 2023-11-25T04:23:39.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:39.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045323+0000", 2023-11-25T04:23:39.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0065980780000000003 2023-11-25T04:23:39.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:39.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045354+0000", 2023-11-25T04:23:39.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0463e-05 2023-11-25T04:23:39.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:39.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045381+0000", 2023-11-25T04:23:39.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7654999999999999e-05 2023-11-25T04:23:39.049 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:39.049 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:39.049 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:39.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.050 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:22 3.e 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:39.050 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:39.050 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.964776727999997, 2023-11-25T04:23:39.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010133422, 2023-11-25T04:23:39.050 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:39.050 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:39.050 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:39.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:39.050 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:39.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:39.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:39.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:39.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:39.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:39.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034135+0000", 2023-11-25T04:23:39.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-07 2023-11-25T04:23:39.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:39.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034138+0000", 2023-11-25T04:23:39.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6009999999999999e-06 2023-11-25T04:23:39.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:39.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034145+0000", 2023-11-25T04:23:39.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1169999999999998e-06 2023-11-25T04:23:39.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:39.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034150+0000", 2023-11-25T04:23:39.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9080000000000003e-06 2023-11-25T04:23:39.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:39.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034209+0000", 2023-11-25T04:23:39.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8149000000000002e-05 2023-11-25T04:23:39.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:39.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034415+0000", 2023-11-25T04:23:39.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020637699999999999 2023-11-25T04:23:39.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:39.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034584+0000", 2023-11-25T04:23:39.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016870399999999999 2023-11-25T04:23:39.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.054 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:39.054 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.036979+0000", 2023-11-25T04:23:39.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023957509999999998 2023-11-25T04:23:39.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.054 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:39.054 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044161+0000", 2023-11-25T04:23:39.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0071811590000000003 2023-11-25T04:23:39.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:39.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044224+0000", 2023-11-25T04:23:39.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3454000000000006e-05 2023-11-25T04:23:39.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:39.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044268+0000", 2023-11-25T04:23:39.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3761999999999999e-05 2023-11-25T04:23:39.055 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:39.055 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:39.055 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:39.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.056 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:23 3.e 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:39.056 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:39.056 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.964745319000002, 2023-11-25T04:23:39.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010281676, 2023-11-25T04:23:39.056 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:39.056 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:39.056 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:39.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:39.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:39.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:39.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:39.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:39.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:39.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:39.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:39.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5799999999999999e-07 2023-11-25T04:23:39.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:39.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034169+0000", 2023-11-25T04:23:39.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1879999999999998e-06 2023-11-25T04:23:39.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:39.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034177+0000", 2023-11-25T04:23:39.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7020000000000001e-06 2023-11-25T04:23:39.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:39.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034184+0000", 2023-11-25T04:23:39.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3910000000000003e-06 2023-11-25T04:23:39.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:39.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034924+0000", 2023-11-25T04:23:39.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00074032999999999998 2023-11-25T04:23:39.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:39.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034991+0000", 2023-11-25T04:23:39.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7347000000000004e-05 2023-11-25T04:23:39.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:39.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035096+0000", 2023-11-25T04:23:39.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010488 2023-11-25T04:23:39.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.060 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:39.060 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037588+0000", 2023-11-25T04:23:39.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024918309999999999 2023-11-25T04:23:39.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.060 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:39.060 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044367+0000", 2023-11-25T04:23:39.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067787589999999997 2023-11-25T04:23:39.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.060 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:39.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044415+0000", 2023-11-25T04:23:39.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7803000000000003e-05 2023-11-25T04:23:39.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:39.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044447+0000", 2023-11-25T04:23:39.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2886999999999999e-05 2023-11-25T04:23:39.061 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:39.061 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:39.061 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:39.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.061 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:30 3.15 3:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:39.062 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:39.062 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.964686878999998, 2023-11-25T04:23:39.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010365070000000001, 2023-11-25T04:23:39.062 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:39.062 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:39.062 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:39.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:39.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:39.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:39.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:39.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:39.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:39.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:39.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034225+0000", 2023-11-25T04:23:39.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7400000000000002e-07 2023-11-25T04:23:39.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:39.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034229+0000", 2023-11-25T04:23:39.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3100000000000002e-06 2023-11-25T04:23:39.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:39.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034239+0000", 2023-11-25T04:23:39.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9140000000000003e-06 2023-11-25T04:23:39.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:39.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034246+0000", 2023-11-25T04:23:39.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0149999999999998e-06 2023-11-25T04:23:39.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:39.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035572+0000", 2023-11-25T04:23:39.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001325655 2023-11-25T04:23:39.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:39.065 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035730+0000", 2023-11-25T04:23:39.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015827899999999999 2023-11-25T04:23:39.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:39.065 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035857+0000", 2023-11-25T04:23:39.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012644699999999999 2023-11-25T04:23:39.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:39.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038737+0000", 2023-11-25T04:23:39.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002880077 2023-11-25T04:23:39.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:39.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044540+0000", 2023-11-25T04:23:39.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058034239999999997 2023-11-25T04:23:39.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:39.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044569+0000", 2023-11-25T04:23:39.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8388999999999999e-05 2023-11-25T04:23:39.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:39.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044589+0000", 2023-11-25T04:23:39.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0686000000000002e-05 2023-11-25T04:23:39.067 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:39.067 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:39.067 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:39.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.068 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:33 3.1b 3:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:39.068 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:39.068 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.964641381, 2023-11-25T04:23:39.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099754049999999997, 2023-11-25T04:23:39.068 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:39.068 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:23:39.068 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:39.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:39.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:39.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:39.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:39.069 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:39.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:39.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.069 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:39.069 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:39.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4799999999999998e-07 2023-11-25T04:23:39.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.069 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:39.069 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034273+0000", 2023-11-25T04:23:39.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7819999999999998e-06 2023-11-25T04:23:39.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:39.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034285+0000", 2023-11-25T04:23:39.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2058e-05 2023-11-25T04:23:39.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:39.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034291+0000", 2023-11-25T04:23:39.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0680000000000002e-06 2023-11-25T04:23:39.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:39.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034370+0000", 2023-11-25T04:23:39.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8452e-05 2023-11-25T04:23:39.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:39.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034477+0000", 2023-11-25T04:23:39.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107471 2023-11-25T04:23:39.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:39.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044245+0000", 2023-11-25T04:23:39.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0097679259999999997 2023-11-25T04:23:39.071 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:39.072 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:39.072 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:39.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.072 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:36 3.1a 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:39.072 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:39.072 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.964594708, 2023-11-25T04:23:39.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058179099999999999, 2023-11-25T04:23:39.072 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:39.072 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:23:39.072 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:39.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:39.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:39.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:39.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.073 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:39.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:39.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:39.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.073 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:39.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034317+0000", 2023-11-25T04:23:39.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1999999999999999e-07 2023-11-25T04:23:39.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:39.074 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034321+0000", 2023-11-25T04:23:39.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.233e-06 2023-11-25T04:23:39.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:39.074 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034337+0000", 2023-11-25T04:23:39.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5308999999999999e-05 2023-11-25T04:23:39.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:39.074 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034354+0000", 2023-11-25T04:23:39.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7411999999999999e-05 2023-11-25T04:23:39.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:39.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034465+0000", 2023-11-25T04:23:39.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011154699999999999 2023-11-25T04:23:39.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:39.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034560+0000", 2023-11-25T04:23:39.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4186999999999998e-05 2023-11-25T04:23:39.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.076 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:39.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.040134+0000", 2023-11-25T04:23:39.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055746019999999997 2023-11-25T04:23:39.076 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:39.076 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:39.076 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:39.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.076 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:9 3.10 e18/16 3:09401a81:::601.00000000:head v 18'4, mlcod=0'0)", 2023-11-25T04:23:39.076 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:39.076 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.955700399999998, 2023-11-25T04:23:39.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061641020000000003, 2023-11-25T04:23:39.076 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:39.077 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:39.077 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:39.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.077 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:39.077 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:39.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:39.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.077 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:39.077 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:39.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:39.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.078 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:39.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:39.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5300000000000001e-07 2023-11-25T04:23:39.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.078 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:39.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043222+0000", 2023-11-25T04:23:39.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.082e-05 2023-11-25T04:23:39.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.078 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:39.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043228+0000", 2023-11-25T04:23:39.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6699999999999997e-06 2023-11-25T04:23:39.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:39.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043236+0000", 2023-11-25T04:23:39.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8189999999999992e-06 2023-11-25T04:23:39.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:39.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043507+0000", 2023-11-25T04:23:39.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000271012 2023-11-25T04:23:39.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:39.080 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043530+0000", 2023-11-25T04:23:39.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2659000000000001e-05 2023-11-25T04:23:39.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.080 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:39.080 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049321+0000", 2023-11-25T04:23:39.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057915129999999999 2023-11-25T04:23:39.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.080 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:39.080 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049375+0000", 2023-11-25T04:23:39.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3455999999999998e-05 2023-11-25T04:23:39.080 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:39.081 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:39.081 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:39.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.081 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:1 3.14 e18/16 3:292cf221:::200.00000000:head v 18'3, mlcod=0'0)", 2023-11-25T04:23:39.081 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:39.081 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.955663686999998, 2023-11-25T04:23:39.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0060983599999999997, 2023-11-25T04:23:39.081 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:39.081 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:39.081 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:39.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:39.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:39.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:39.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:39.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:39.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:39.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:39.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043248+0000", 2023-11-25T04:23:39.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9399999999999995e-07 2023-11-25T04:23:39.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.083 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:39.083 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043252+0000", 2023-11-25T04:23:39.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1799999999999998e-06 2023-11-25T04:23:39.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.083 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:39.083 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043259+0000", 2023-11-25T04:23:39.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0049999999999998e-06 2023-11-25T04:23:39.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.083 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:39.083 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043265+0000", 2023-11-25T04:23:39.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0859999999999998e-06 2023-11-25T04:23:39.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:39.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043311+0000", 2023-11-25T04:23:39.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6082999999999999e-05 2023-11-25T04:23:39.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:39.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043330+0000", 2023-11-25T04:23:39.084 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8556e-05 2023-11-25T04:23:39.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:39.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049303+0000", 2023-11-25T04:23:39.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059733470000000004 2023-11-25T04:23:39.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:39.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049346+0000", 2023-11-25T04:23:39.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2608999999999998e-05 2023-11-25T04:23:39.085 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:39.085 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:39.085 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:39.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.085 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:3 3.14 e18/16 3:287034c4:::1.00000000.inode:head v 18'4, mlcod=0'0)", 2023-11-25T04:23:39.086 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:39.086 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.955147181000001, 2023-11-25T04:23:39.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057040090000000003, 2023-11-25T04:23:39.086 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:39.086 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:39.086 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:39.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.086 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:39.086 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:39.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:39.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.087 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:39.087 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:39.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:39.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.087 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:39.087 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:39.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9799999999999999e-07 2023-11-25T04:23:39.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.087 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:39.087 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043785+0000", 2023-11-25T04:23:39.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0721999999999999e-05 2023-11-25T04:23:39.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:39.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043793+0000", 2023-11-25T04:23:39.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5839999999999995e-06 2023-11-25T04:23:39.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:39.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043803+0000", 2023-11-25T04:23:39.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0729999999999994e-06 2023-11-25T04:23:39.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:39.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043843+0000", 2023-11-25T04:23:39.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0747999999999997e-05 2023-11-25T04:23:39.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:39.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043856+0000", 2023-11-25T04:23:39.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2697000000000001e-05 2023-11-25T04:23:39.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:39.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049426+0000", 2023-11-25T04:23:39.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055704409999999998 2023-11-25T04:23:39.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:39.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049468+0000", 2023-11-25T04:23:39.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1446000000000003e-05 2023-11-25T04:23:39.090 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:39.090 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:39.090 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:39.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.090 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "pg_update_log_missing(3.14 epoch 18/16 rep_tid 171 entries 18'5 (0'0) error 3:2e2fa760:::200.00000001:head by mds.0.3:32 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:23:39.090 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:39.091 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.954986218000002, 2023-11-25T04:23:39.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005556938, 2023-11-25T04:23:39.091 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:39.091 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2023-11-25T04:23:39.091 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:39.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:39.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:39.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:39.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:39.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:39.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:39.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.092 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:39.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:39.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.03e-07 2023-11-25T04:23:39.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.092 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:39.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043931+0000", 2023-11-25T04:23:39.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1079999999999998e-06 2023-11-25T04:23:39.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:39.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043944+0000", 2023-11-25T04:23:39.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2852e-05 2023-11-25T04:23:39.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:39.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043955+0000", 2023-11-25T04:23:39.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1037e-05 2023-11-25T04:23:39.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:39.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044088+0000", 2023-11-25T04:23:39.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013269599999999999 2023-11-25T04:23:39.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:39.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049482+0000", 2023-11-25T04:23:39.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053940419999999999 2023-11-25T04:23:39.094 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:39.094 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:39.094 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:39.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.095 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:10 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'5, mlcod=0'0)", 2023-11-25T04:23:39.095 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:39.095 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.954896189999999, 2023-11-25T04:23:39.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054493099999999997, 2023-11-25T04:23:39.095 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:39.095 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:39.095 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:39.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:39.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:39.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:39.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:39.096 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:39.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:39.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:39.096 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:39.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6100000000000002e-07 2023-11-25T04:23:39.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:39.096 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044019+0000", 2023-11-25T04:23:39.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.839e-06 2023-11-25T04:23:39.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:39.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044026+0000", 2023-11-25T04:23:39.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5869999999999998e-06 2023-11-25T04:23:39.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:39.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044031+0000", 2023-11-25T04:23:39.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4909999999999996e-06 2023-11-25T04:23:39.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.098 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:39.098 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044070+0000", 2023-11-25T04:23:39.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8829999999999999e-05 2023-11-25T04:23:39.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.098 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:39.098 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044081+0000", 2023-11-25T04:23:39.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1447999999999999e-05 2023-11-25T04:23:39.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.098 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:39.098 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049406+0000", 2023-11-25T04:23:39.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053246480000000004 2023-11-25T04:23:39.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.099 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:39.099 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049464+0000", 2023-11-25T04:23:39.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8205999999999998e-05 2023-11-25T04:23:39.099 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:39.099 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:39.099 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:39.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.099 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:11 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'6, mlcod=0'0)", 2023-11-25T04:23:39.099 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:39.100 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 51.954694926000002, 2023-11-25T04:23:39.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052673499999999996, 2023-11-25T04:23:39.100 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:39.100 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:39.100 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:39.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:39.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:39.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:39.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:39.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:39.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:39.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:39.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:39.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0199999999999998e-07 2023-11-25T04:23:39.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:39.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044220+0000", 2023-11-25T04:23:39.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7720000000000001e-06 2023-11-25T04:23:39.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.102 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:39.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044226+0000", 2023-11-25T04:23:39.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8579999999999996e-06 2023-11-25T04:23:39.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.102 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:39.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044232+0000", 2023-11-25T04:23:39.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1700000000000002e-06 2023-11-25T04:23:39.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.102 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:39.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044274+0000", 2023-11-25T04:23:39.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1254999999999997e-05 2023-11-25T04:23:39.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.103 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:39.103 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:23:39.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6880999999999999e-05 2023-11-25T04:23:39.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.103 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:39.103 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049466+0000", 2023-11-25T04:23:39.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051659219999999999 2023-11-25T04:23:39.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:39.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:39.104 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:39.104 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049483+0000", 2023-11-25T04:23:39.104 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.719e-05 2023-11-25T04:23:39.104 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:39.104 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:39.104 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:39.104 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:39.104 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:39.104 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:39.105 INFO:teuthology.orchestra.run.smithi067.stderr:Error ENXIO: problem getting command descriptions from osd.0 2023-11-25T04:23:39.106 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:39.104+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:39.106 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:23:39.114 INFO:teuthology.orchestra.run.smithi067.stdout:osd.1: {} 2023-11-25T04:23:39.114 INFO:teuthology.orchestra.run.smithi067.stderr:osd.1: osd_enable_op_tracker = '' 2023-11-25T04:23:39.133 INFO:teuthology.orchestra.run.smithi067.stdout:osd.2: {} 2023-11-25T04:23:39.133 INFO:teuthology.orchestra.run.smithi067.stderr:osd.2: osd_enable_op_tracker = '' 2023-11-25T04:23:39.136 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-11-25T04:23:39.136 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-11-25T04:23:39.207 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:39.205+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:39.214 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:39.214 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:39.223 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:23:39.307 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:39.306+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:39.369 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:39.369 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:39.378 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:23:39.423 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:39.422+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:39.483 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:39.483 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:39.492 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:23:39.536 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:39.535+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:39.599 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:39.600 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:39.609 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:23:39.610 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:39.608+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:39.710 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:39.709+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:39.755 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:39.755 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:39.763 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:23:39.847 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:39.846+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:39.908 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:39.909 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:39.912 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:39.910+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:39.917 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:23:39.940 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2023-11-25T04:23:40.013 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:40.011+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:40.024 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:40.024 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:40.033 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:23:40.114 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:40.113+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:40.178 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:40.178 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:40.187 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:23:40.231 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:40.230+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:40.292 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:40.292 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:40.301 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:23:40.345 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:40.344+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:40.406 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:40.406 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:40.414 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:23:40.455 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:40.454+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:40.518 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:40.517+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:40.560 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:40.560 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:40.569 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:23:40.653 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:40.652+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:40.714 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:40.714 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:40.719 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:40.718+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:40.722 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:23:40.821 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:40.819+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:40.829 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:40.829 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:40.838 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:23:40.922 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:40.921+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:40.942 INFO:teuthology.orchestra.run.smithi067.stderr:noscrub is set 2023-11-25T04:23:40.984 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:40.984 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:40.994 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:23:41.035 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:41.033+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:41.123 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:41.122+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:41.137 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-11-25T04:23:41.140 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:41.141 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:41.150 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:23:41.224 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:41.222+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:41.256 INFO:teuthology.orchestra.run.smithi067.stderr:Error ENXIO: problem getting command descriptions from osd.0 2023-11-25T04:23:41.257 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:41.258 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:41.267 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:23:41.288 INFO:teuthology.orchestra.run.smithi067.stdout:osd.1: {} 2023-11-25T04:23:41.288 INFO:teuthology.orchestra.run.smithi067.stderr:osd.1: osd_enable_op_tracker = '' 2023-11-25T04:23:41.306 INFO:teuthology.orchestra.run.smithi067.stdout:osd.2: {} 2023-11-25T04:23:41.307 INFO:teuthology.orchestra.run.smithi067.stderr:osd.2: osd_enable_op_tracker = '' 2023-11-25T04:23:41.311 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-11-25T04:23:41.311 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-11-25T04:23:41.324 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:41.323+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:41.412 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:41.412 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:23:41.412 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2023-11-25T04:23:41.412 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2023-11-25T04:23:41.412 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:41.422 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:23:41.426 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:41.424+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:41.526 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:41.525+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:41.532 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:41.532 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2023-11-25T04:23:41.532 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2023-11-25T04:23:41.532 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2023-11-25T04:23:41.532 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.532 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:155 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'109, mlcod=11'105)", 2023-11-25T04:23:41.533 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:41.533 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 63.745280553000001, 2023-11-25T04:23:41.533 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061719019999999999, 2023-11-25T04:23:41.533 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:41.533 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:41.533 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:41.533 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.533 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:41.533 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:41.533 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:41.533 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.533 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.534 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:41.534 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:41.534 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:41.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.534 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.534 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:41.534 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784884+0000", 2023-11-25T04:23:41.534 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3099999999999999e-07 2023-11-25T04:23:41.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.534 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.534 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:41.534 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784942+0000", 2023-11-25T04:23:41.535 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8570000000000003e-05 2023-11-25T04:23:41.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.535 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:41.535 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784949+0000", 2023-11-25T04:23:41.535 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0310000000000001e-06 2023-11-25T04:23:41.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.535 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:41.535 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784953+0000", 2023-11-25T04:23:41.535 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6320000000000001e-06 2023-11-25T04:23:41.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.536 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:41.536 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784993+0000", 2023-11-25T04:23:41.536 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9737000000000003e-05 2023-11-25T04:23:41.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.536 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.536 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:41.536 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785011+0000", 2023-11-25T04:23:41.536 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8349999999999999e-05 2023-11-25T04:23:41.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.536 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.536 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:41.537 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791024+0000", 2023-11-25T04:23:41.537 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006012997 2023-11-25T04:23:41.537 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.537 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.537 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:41.537 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791055+0000", 2023-11-25T04:23:41.537 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1254000000000003e-05 2023-11-25T04:23:41.537 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:41.537 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:41.537 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:41.537 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.537 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.538 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:156 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'110, mlcod=11'105)", 2023-11-25T04:23:41.538 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:41.538 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 63.744694504000002, 2023-11-25T04:23:41.538 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005616116, 2023-11-25T04:23:41.538 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:41.538 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:41.538 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:41.538 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.538 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:41.538 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:41.538 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:41.538 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.539 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:41.539 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:41.539 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:41.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.539 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:41.539 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785470+0000", 2023-11-25T04:23:41.539 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0699999999999998e-07 2023-11-25T04:23:41.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.539 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:41.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785474+0000", 2023-11-25T04:23:41.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.048e-06 2023-11-25T04:23:41.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.540 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.540 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:41.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785480+0000", 2023-11-25T04:23:41.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7729999999999996e-06 2023-11-25T04:23:41.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.540 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.540 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:41.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785483+0000", 2023-11-25T04:23:41.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1970000000000001e-06 2023-11-25T04:23:41.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.541 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.541 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:41.541 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787936+0000", 2023-11-25T04:23:41.541 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024529040000000001 2023-11-25T04:23:41.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.541 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.541 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:41.541 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787947+0000", 2023-11-25T04:23:41.541 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1669e-05 2023-11-25T04:23:41.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.541 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.541 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:41.542 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791057+0000", 2023-11-25T04:23:41.542 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031095200000000002 2023-11-25T04:23:41.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.542 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.542 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:41.542 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791086+0000", 2023-11-25T04:23:41.542 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8697999999999999e-05 2023-11-25T04:23:41.542 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:41.542 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:41.542 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:41.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.542 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.543 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:157 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'111, mlcod=11'106)", 2023-11-25T04:23:41.543 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:41.543 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 63.744101946000001, 2023-11-25T04:23:41.543 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050590280000000001, 2023-11-25T04:23:41.543 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:41.543 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:41.543 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:41.543 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.543 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:41.543 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:41.543 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:41.543 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.544 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.544 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:41.544 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:41.544 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:41.544 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.544 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.544 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:41.544 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786063+0000", 2023-11-25T04:23:41.544 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2600000000000002e-07 2023-11-25T04:23:41.544 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.544 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.544 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:41.544 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786069+0000", 2023-11-25T04:23:41.545 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8839999999999999e-06 2023-11-25T04:23:41.545 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.545 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.545 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:41.545 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786079+0000", 2023-11-25T04:23:41.545 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0124e-05 2023-11-25T04:23:41.545 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.545 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.545 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:41.545 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786085+0000", 2023-11-25T04:23:41.545 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6660000000000002e-06 2023-11-25T04:23:41.545 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.545 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.546 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:41.546 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788160+0000", 2023-11-25T04:23:41.546 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020747669999999999 2023-11-25T04:23:41.546 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.546 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.546 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:41.546 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788180+0000", 2023-11-25T04:23:41.546 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0208999999999999e-05 2023-11-25T04:23:41.546 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.546 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.546 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:41.546 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791087+0000", 2023-11-25T04:23:41.547 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029064630000000002 2023-11-25T04:23:41.547 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.547 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.547 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:41.547 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791121+0000", 2023-11-25T04:23:41.547 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4289000000000003e-05 2023-11-25T04:23:41.547 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:41.547 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:41.547 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:41.547 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.547 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.547 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:158 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'112, mlcod=11'106)", 2023-11-25T04:23:41.548 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:41.548 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 63.743685552999999, 2023-11-25T04:23:41.548 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046663030000000001, 2023-11-25T04:23:41.548 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:41.548 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:41.548 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:41.548 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.548 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:41.548 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:41.548 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:41.548 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.548 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.549 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:41.549 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:41.549 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:41.549 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.549 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.549 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:41.549 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786479+0000", 2023-11-25T04:23:41.549 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2799999999999996e-07 2023-11-25T04:23:41.549 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.549 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.549 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:41.549 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786498+0000", 2023-11-25T04:23:41.549 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9069e-05 2023-11-25T04:23:41.550 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.550 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.550 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:41.550 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786508+0000", 2023-11-25T04:23:41.550 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5810000000000003e-06 2023-11-25T04:23:41.550 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.550 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.550 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:41.551 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786512+0000", 2023-11-25T04:23:41.551 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4889999999999997e-06 2023-11-25T04:23:41.551 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.551 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.551 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:41.551 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788438+0000", 2023-11-25T04:23:41.551 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019257059999999999 2023-11-25T04:23:41.551 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.551 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.552 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:41.552 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788449+0000", 2023-11-25T04:23:41.552 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1419e-05 2023-11-25T04:23:41.552 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.552 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.552 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:41.552 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791122+0000", 2023-11-25T04:23:41.552 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002672822 2023-11-25T04:23:41.552 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.552 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.552 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:41.552 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791145+0000", 2023-11-25T04:23:41.553 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2589e-05 2023-11-25T04:23:41.553 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:41.553 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:41.553 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:41.553 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.553 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.553 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:206 1.0 e12/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 12'152, mlcod=12'148)", 2023-11-25T04:23:41.553 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:41.553 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 63.231917082999999, 2023-11-25T04:23:41.553 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046728189999999999, 2023-11-25T04:23:41.553 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:41.553 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:41.553 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:41.554 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.554 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:41.554 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:41.554 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:41.554 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.554 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.554 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:41.554 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:41.554 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:41.554 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.554 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.554 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:41.555 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298248+0000", 2023-11-25T04:23:41.555 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3600000000000003e-07 2023-11-25T04:23:41.555 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.555 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.555 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:41.555 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298278+0000", 2023-11-25T04:23:41.555 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0474e-05 2023-11-25T04:23:41.555 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.555 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.555 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:41.555 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298288+0000", 2023-11-25T04:23:41.555 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0156000000000001e-05 2023-11-25T04:23:41.556 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.556 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.556 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:41.556 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298293+0000", 2023-11-25T04:23:41.556 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7400000000000004e-06 2023-11-25T04:23:41.556 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.556 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.556 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:41.556 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298333+0000", 2023-11-25T04:23:41.556 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9666000000000003e-05 2023-11-25T04:23:41.556 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.556 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.557 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:41.557 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298351+0000", 2023-11-25T04:23:41.557 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8544999999999999e-05 2023-11-25T04:23:41.557 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.557 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.557 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:41.557 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302891+0000", 2023-11-25T04:23:41.557 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045398519999999996 2023-11-25T04:23:41.557 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.557 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.557 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:41.557 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302920+0000", 2023-11-25T04:23:41.558 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8750000000000001e-05 2023-11-25T04:23:41.558 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:41.558 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:41.558 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:41.558 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.558 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.558 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:14 3.15 3:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:41.558 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:41.558 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 54.496782725999999, 2023-11-25T04:23:41.558 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010990437, 2023-11-25T04:23:41.558 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:41.558 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:41.559 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:41.559 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.559 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:41.559 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:41.559 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:41.559 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.559 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.559 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:41.559 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:41.559 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:41.559 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.559 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.560 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:41.560 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033383+0000", 2023-11-25T04:23:41.560 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9870000000000002e-06 2023-11-25T04:23:41.560 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.560 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.560 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:41.560 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033398+0000", 2023-11-25T04:23:41.560 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4569999999999999e-05 2023-11-25T04:23:41.560 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.560 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.560 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:41.560 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033411+0000", 2023-11-25T04:23:41.560 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3543999999999999e-05 2023-11-25T04:23:41.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.561 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:41.561 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033423+0000", 2023-11-25T04:23:41.561 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1905999999999999e-05 2023-11-25T04:23:41.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.561 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:41.561 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034433+0000", 2023-11-25T04:23:41.561 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00101022 2023-11-25T04:23:41.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.561 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:41.562 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034539+0000", 2023-11-25T04:23:41.562 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010597199999999999 2023-11-25T04:23:41.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.562 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.562 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:41.562 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034667+0000", 2023-11-25T04:23:41.562 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012765500000000001 2023-11-25T04:23:41.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.562 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.562 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:41.562 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037803+0000", 2023-11-25T04:23:41.562 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031362220000000001 2023-11-25T04:23:41.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.563 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:41.563 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:23:41.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0064964669999999997 2023-11-25T04:23:41.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.563 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:41.563 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044331+0000", 2023-11-25T04:23:41.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1544999999999997e-05 2023-11-25T04:23:41.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:41.564 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044372+0000", 2023-11-25T04:23:41.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0349000000000003e-05 2023-11-25T04:23:41.564 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:41.564 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:41.564 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:41.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.564 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:15 3.15 3:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:41.564 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:41.564 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 54.496123076000003, 2023-11-25T04:23:41.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010464362, 2023-11-25T04:23:41.565 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:41.565 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:41.565 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:41.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.565 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:41.565 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:41.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:41.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.565 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:41.565 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:41.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:41.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.566 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.566 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:41.566 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034042+0000", 2023-11-25T04:23:41.566 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0999999999999997e-07 2023-11-25T04:23:41.566 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.566 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.566 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:41.566 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034048+0000", 2023-11-25T04:23:41.566 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0390000000000001e-06 2023-11-25T04:23:41.566 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.566 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.566 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:41.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034059+0000", 2023-11-25T04:23:41.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1182999999999999e-05 2023-11-25T04:23:41.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:41.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034067+0000", 2023-11-25T04:23:41.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1559999999999995e-06 2023-11-25T04:23:41.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:41.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035005+0000", 2023-11-25T04:23:41.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00093747399999999999 2023-11-25T04:23:41.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:41.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035077+0000", 2023-11-25T04:23:41.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2747e-05 2023-11-25T04:23:41.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:41.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035219+0000", 2023-11-25T04:23:41.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000141877 2023-11-25T04:23:41.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:41.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037810+0000", 2023-11-25T04:23:41.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002590752 2023-11-25T04:23:41.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:41.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044437+0000", 2023-11-25T04:23:41.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066270110000000004 2023-11-25T04:23:41.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:41.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044481+0000", 2023-11-25T04:23:41.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4106000000000001e-05 2023-11-25T04:23:41.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.570 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:41.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044505+0000", 2023-11-25T04:23:41.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4207e-05 2023-11-25T04:23:41.570 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:41.570 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:41.570 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:41.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.570 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:16 3.1d 3:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:41.570 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:41.571 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 54.496081228000001, 2023-11-25T04:23:41.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010413923, 2023-11-25T04:23:41.571 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:41.571 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:41.571 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:41.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:41.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:41.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:41.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:41.572 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:41.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:41.572 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.572 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.572 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:41.572 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:41.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4499999999999997e-07 2023-11-25T04:23:41.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.573 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:41.573 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034087+0000", 2023-11-25T04:23:41.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9770000000000002e-06 2023-11-25T04:23:41.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.573 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:41.574 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034094+0000", 2023-11-25T04:23:41.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8809999999999998e-06 2023-11-25T04:23:41.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:41.574 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034100+0000", 2023-11-25T04:23:41.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8440000000000003e-06 2023-11-25T04:23:41.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:41.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034993+0000", 2023-11-25T04:23:41.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00089336799999999996 2023-11-25T04:23:41.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:41.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035099+0000", 2023-11-25T04:23:41.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106037 2023-11-25T04:23:41.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:41.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035276+0000", 2023-11-25T04:23:41.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017631799999999999 2023-11-25T04:23:41.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.576 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:41.576 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037815+0000", 2023-11-25T04:23:41.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025395959999999999 2023-11-25T04:23:41.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.576 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:41.576 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044420+0000", 2023-11-25T04:23:41.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066045749999999997 2023-11-25T04:23:41.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.576 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:41.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044463+0000", 2023-11-25T04:23:41.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3205999999999999e-05 2023-11-25T04:23:41.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.577 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:41.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044497+0000", 2023-11-25T04:23:41.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3476000000000003e-05 2023-11-25T04:23:41.577 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:41.577 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:41.577 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:41.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.577 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:17 3.1d 3:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:41.578 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:41.578 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 54.496056019000001, 2023-11-25T04:23:41.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01127356, 2023-11-25T04:23:41.578 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:41.578 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:41.578 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:41.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.578 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:41.578 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:41.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:41.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:41.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:41.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:41.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:41.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:41.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1100000000000001e-07 2023-11-25T04:23:41.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:41.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034112+0000", 2023-11-25T04:23:41.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8600000000000003e-06 2023-11-25T04:23:41.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:41.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034120+0000", 2023-11-25T04:23:41.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7700000000000001e-06 2023-11-25T04:23:41.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:41.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034127+0000", 2023-11-25T04:23:41.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0999999999999998e-06 2023-11-25T04:23:41.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.581 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:41.581 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035601+0000", 2023-11-25T04:23:41.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014739410000000001 2023-11-25T04:23:41.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.581 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:41.581 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035685+0000", 2023-11-25T04:23:41.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3677999999999994e-05 2023-11-25T04:23:41.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:41.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035827+0000", 2023-11-25T04:23:41.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014232599999999999 2023-11-25T04:23:41.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:41.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038725+0000", 2023-11-25T04:23:41.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028982779999999998 2023-11-25T04:23:41.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:41.583 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045323+0000", 2023-11-25T04:23:41.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0065980780000000003 2023-11-25T04:23:41.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.583 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:41.583 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045354+0000", 2023-11-25T04:23:41.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0463e-05 2023-11-25T04:23:41.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.583 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:41.583 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045381+0000", 2023-11-25T04:23:41.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7654999999999999e-05 2023-11-25T04:23:41.584 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:41.584 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:41.584 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:41.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.584 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:22 3.e 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:41.584 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:41.584 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 54.496029606, 2023-11-25T04:23:41.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010133422, 2023-11-25T04:23:41.584 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:41.584 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:41.584 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:41.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:41.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:41.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:41.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:41.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:41.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:41.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:41.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034135+0000", 2023-11-25T04:23:41.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-07 2023-11-25T04:23:41.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:41.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034138+0000", 2023-11-25T04:23:41.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6009999999999999e-06 2023-11-25T04:23:41.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:41.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034145+0000", 2023-11-25T04:23:41.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1169999999999998e-06 2023-11-25T04:23:41.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.587 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:41.587 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034150+0000", 2023-11-25T04:23:41.587 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9080000000000003e-06 2023-11-25T04:23:41.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.587 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:41.587 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034209+0000", 2023-11-25T04:23:41.587 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8149000000000002e-05 2023-11-25T04:23:41.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.588 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:41.588 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034415+0000", 2023-11-25T04:23:41.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020637699999999999 2023-11-25T04:23:41.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.588 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:41.588 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034584+0000", 2023-11-25T04:23:41.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016870399999999999 2023-11-25T04:23:41.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.588 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:41.588 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.036979+0000", 2023-11-25T04:23:41.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023957509999999998 2023-11-25T04:23:41.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:41.589 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044161+0000", 2023-11-25T04:23:41.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0071811590000000003 2023-11-25T04:23:41.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:41.589 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044224+0000", 2023-11-25T04:23:41.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3454000000000006e-05 2023-11-25T04:23:41.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:41.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044268+0000", 2023-11-25T04:23:41.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3761999999999999e-05 2023-11-25T04:23:41.590 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:41.590 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:41.590 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:41.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.590 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:23 3.e 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:41.590 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:41.590 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 54.495998196999999, 2023-11-25T04:23:41.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010281676, 2023-11-25T04:23:41.591 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:41.591 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:41.591 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:41.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.591 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:41.591 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:41.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:41.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.591 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:41.591 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:41.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:41.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.592 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.592 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:41.592 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:41.592 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5799999999999999e-07 2023-11-25T04:23:41.592 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.592 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.592 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:41.592 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034169+0000", 2023-11-25T04:23:41.592 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1879999999999998e-06 2023-11-25T04:23:41.592 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.592 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.592 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:41.592 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034177+0000", 2023-11-25T04:23:41.593 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7020000000000001e-06 2023-11-25T04:23:41.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:41.593 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034184+0000", 2023-11-25T04:23:41.593 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3910000000000003e-06 2023-11-25T04:23:41.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:41.593 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034924+0000", 2023-11-25T04:23:41.593 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00074032999999999998 2023-11-25T04:23:41.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.594 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:41.594 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034991+0000", 2023-11-25T04:23:41.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7347000000000004e-05 2023-11-25T04:23:41.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.594 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:41.594 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035096+0000", 2023-11-25T04:23:41.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010488 2023-11-25T04:23:41.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.594 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:41.594 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037588+0000", 2023-11-25T04:23:41.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024918309999999999 2023-11-25T04:23:41.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:41.595 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044367+0000", 2023-11-25T04:23:41.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067787589999999997 2023-11-25T04:23:41.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:41.595 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044415+0000", 2023-11-25T04:23:41.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7803000000000003e-05 2023-11-25T04:23:41.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.596 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:41.596 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044447+0000", 2023-11-25T04:23:41.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2886999999999999e-05 2023-11-25T04:23:41.596 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:41.596 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:41.596 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:41.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.596 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.596 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:30 3.15 3:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:41.596 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:41.596 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 54.495939757000002, 2023-11-25T04:23:41.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010365070000000001, 2023-11-25T04:23:41.597 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:41.597 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:41.597 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:41.597 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.597 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:41.597 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:41.597 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:41.597 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.597 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.597 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:41.597 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:41.597 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:41.597 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:41.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034225+0000", 2023-11-25T04:23:41.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7400000000000002e-07 2023-11-25T04:23:41.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:41.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034229+0000", 2023-11-25T04:23:41.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3100000000000002e-06 2023-11-25T04:23:41.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:41.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034239+0000", 2023-11-25T04:23:41.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9140000000000003e-06 2023-11-25T04:23:41.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:41.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034246+0000", 2023-11-25T04:23:41.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0149999999999998e-06 2023-11-25T04:23:41.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:41.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035572+0000", 2023-11-25T04:23:41.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001325655 2023-11-25T04:23:41.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.600 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:41.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035730+0000", 2023-11-25T04:23:41.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015827899999999999 2023-11-25T04:23:41.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.600 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:41.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035857+0000", 2023-11-25T04:23:41.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012644699999999999 2023-11-25T04:23:41.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:41.601 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038737+0000", 2023-11-25T04:23:41.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002880077 2023-11-25T04:23:41.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:41.601 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044540+0000", 2023-11-25T04:23:41.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058034239999999997 2023-11-25T04:23:41.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:41.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044569+0000", 2023-11-25T04:23:41.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8388999999999999e-05 2023-11-25T04:23:41.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:41.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044589+0000", 2023-11-25T04:23:41.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0686000000000002e-05 2023-11-25T04:23:41.602 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:41.602 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:41.602 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:41.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.602 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:33 3.1b 3:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:41.603 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:41.603 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 54.495894259000004, 2023-11-25T04:23:41.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099754049999999997, 2023-11-25T04:23:41.603 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:41.603 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:23:41.603 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:41.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:41.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:41.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:41.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.605 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:41.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:41.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:41.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.605 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:41.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:41.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4799999999999998e-07 2023-11-25T04:23:41.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:41.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034273+0000", 2023-11-25T04:23:41.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7819999999999998e-06 2023-11-25T04:23:41.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:41.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034285+0000", 2023-11-25T04:23:41.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2058e-05 2023-11-25T04:23:41.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:41.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034291+0000", 2023-11-25T04:23:41.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0680000000000002e-06 2023-11-25T04:23:41.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:41.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034370+0000", 2023-11-25T04:23:41.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8452e-05 2023-11-25T04:23:41.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:41.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034477+0000", 2023-11-25T04:23:41.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107471 2023-11-25T04:23:41.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:41.608 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044245+0000", 2023-11-25T04:23:41.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0097679259999999997 2023-11-25T04:23:41.608 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:41.608 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:41.608 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:41.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.608 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:36 3.1a 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:41.609 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:41.609 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 54.495847585999996, 2023-11-25T04:23:41.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058179099999999999, 2023-11-25T04:23:41.609 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:41.609 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:23:41.609 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:41.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:41.609 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:41.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:41.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.610 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:41.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:41.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:41.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.610 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:41.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034317+0000", 2023-11-25T04:23:41.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1999999999999999e-07 2023-11-25T04:23:41.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.610 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:41.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034321+0000", 2023-11-25T04:23:41.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.233e-06 2023-11-25T04:23:41.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:41.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034337+0000", 2023-11-25T04:23:41.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5308999999999999e-05 2023-11-25T04:23:41.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:41.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034354+0000", 2023-11-25T04:23:41.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7411999999999999e-05 2023-11-25T04:23:41.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:41.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034465+0000", 2023-11-25T04:23:41.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011154699999999999 2023-11-25T04:23:41.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:41.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034560+0000", 2023-11-25T04:23:41.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4186999999999998e-05 2023-11-25T04:23:41.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:41.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.040134+0000", 2023-11-25T04:23:41.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055746019999999997 2023-11-25T04:23:41.613 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:41.613 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:41.613 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:41.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.613 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:9 3.10 e18/16 3:09401a81:::601.00000000:head v 18'4, mlcod=0'0)", 2023-11-25T04:23:41.613 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:41.613 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 54.486953278000001, 2023-11-25T04:23:41.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061641020000000003, 2023-11-25T04:23:41.613 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:41.614 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:41.614 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:41.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:41.614 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:41.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:41.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:41.614 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:41.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:41.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:41.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:41.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5300000000000001e-07 2023-11-25T04:23:41.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:41.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043222+0000", 2023-11-25T04:23:41.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.082e-05 2023-11-25T04:23:41.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:41.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043228+0000", 2023-11-25T04:23:41.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6699999999999997e-06 2023-11-25T04:23:41.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:41.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043236+0000", 2023-11-25T04:23:41.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8189999999999992e-06 2023-11-25T04:23:41.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:41.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043507+0000", 2023-11-25T04:23:41.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000271012 2023-11-25T04:23:41.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.617 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:41.617 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043530+0000", 2023-11-25T04:23:41.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2659000000000001e-05 2023-11-25T04:23:41.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.617 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:41.617 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049321+0000", 2023-11-25T04:23:41.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057915129999999999 2023-11-25T04:23:41.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.618 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:41.618 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049375+0000", 2023-11-25T04:23:41.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3455999999999998e-05 2023-11-25T04:23:41.618 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:41.618 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:41.618 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:41.618 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.618 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:1 3.14 e18/16 3:292cf221:::200.00000000:head v 18'3, mlcod=0'0)", 2023-11-25T04:23:41.618 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:41.618 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 54.486916565000001, 2023-11-25T04:23:41.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0060983599999999997, 2023-11-25T04:23:41.618 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:41.619 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:41.619 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:41.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:41.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:41.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:41.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:41.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:41.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:41.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:41.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043248+0000", 2023-11-25T04:23:41.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9399999999999995e-07 2023-11-25T04:23:41.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:41.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043252+0000", 2023-11-25T04:23:41.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1799999999999998e-06 2023-11-25T04:23:41.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:41.621 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043259+0000", 2023-11-25T04:23:41.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0049999999999998e-06 2023-11-25T04:23:41.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:41.621 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043265+0000", 2023-11-25T04:23:41.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0859999999999998e-06 2023-11-25T04:23:41.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:41.621 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043311+0000", 2023-11-25T04:23:41.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6082999999999999e-05 2023-11-25T04:23:41.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.622 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:41.622 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043330+0000", 2023-11-25T04:23:41.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8556e-05 2023-11-25T04:23:41.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.622 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:41.622 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049303+0000", 2023-11-25T04:23:41.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059733470000000004 2023-11-25T04:23:41.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.622 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:41.623 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049346+0000", 2023-11-25T04:23:41.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2608999999999998e-05 2023-11-25T04:23:41.623 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:41.623 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:41.623 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:41.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.623 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:3 3.14 e18/16 3:287034c4:::1.00000000.inode:head v 18'4, mlcod=0'0)", 2023-11-25T04:23:41.623 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:41.623 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 54.486400058999997, 2023-11-25T04:23:41.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057040090000000003, 2023-11-25T04:23:41.623 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:41.624 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:41.624 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:41.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:41.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:41.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:41.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:41.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:41.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:41.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:41.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:41.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9799999999999999e-07 2023-11-25T04:23:41.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:41.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043785+0000", 2023-11-25T04:23:41.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0721999999999999e-05 2023-11-25T04:23:41.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:41.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043793+0000", 2023-11-25T04:23:41.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5839999999999995e-06 2023-11-25T04:23:41.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:41.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043803+0000", 2023-11-25T04:23:41.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0729999999999994e-06 2023-11-25T04:23:41.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:41.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043843+0000", 2023-11-25T04:23:41.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0747999999999997e-05 2023-11-25T04:23:41.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.627 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:41.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043856+0000", 2023-11-25T04:23:41.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2697000000000001e-05 2023-11-25T04:23:41.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.627 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:41.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049426+0000", 2023-11-25T04:23:41.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055704409999999998 2023-11-25T04:23:41.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:41.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049468+0000", 2023-11-25T04:23:41.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1446000000000003e-05 2023-11-25T04:23:41.628 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:41.628 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:41.628 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:41.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.628 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "pg_update_log_missing(3.14 epoch 18/16 rep_tid 171 entries 18'5 (0'0) error 3:2e2fa760:::200.00000001:head by mds.0.3:32 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:23:41.628 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:41.628 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 54.486239095999998, 2023-11-25T04:23:41.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005556938, 2023-11-25T04:23:41.629 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:41.629 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2023-11-25T04:23:41.629 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:41.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:41.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:41.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:41.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:41.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:41.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:41.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:41.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:41.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.03e-07 2023-11-25T04:23:41.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:41.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043931+0000", 2023-11-25T04:23:41.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1079999999999998e-06 2023-11-25T04:23:41.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.631 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:41.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043944+0000", 2023-11-25T04:23:41.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2852e-05 2023-11-25T04:23:41.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.631 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:41.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043955+0000", 2023-11-25T04:23:41.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1037e-05 2023-11-25T04:23:41.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.631 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:41.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044088+0000", 2023-11-25T04:23:41.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013269599999999999 2023-11-25T04:23:41.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:41.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049482+0000", 2023-11-25T04:23:41.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053940419999999999 2023-11-25T04:23:41.632 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:41.632 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:41.632 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:41.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.633 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:10 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'5, mlcod=0'0)", 2023-11-25T04:23:41.633 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:41.633 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 54.486149068000003, 2023-11-25T04:23:41.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054493099999999997, 2023-11-25T04:23:41.633 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:41.633 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:41.633 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:41.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:41.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:41.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:41.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:41.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:41.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:41.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:41.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:41.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6100000000000002e-07 2023-11-25T04:23:41.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:41.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044019+0000", 2023-11-25T04:23:41.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.839e-06 2023-11-25T04:23:41.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:41.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044026+0000", 2023-11-25T04:23:41.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5869999999999998e-06 2023-11-25T04:23:41.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:41.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044031+0000", 2023-11-25T04:23:41.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4909999999999996e-06 2023-11-25T04:23:41.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:41.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044070+0000", 2023-11-25T04:23:41.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8829999999999999e-05 2023-11-25T04:23:41.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:41.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044081+0000", 2023-11-25T04:23:41.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1447999999999999e-05 2023-11-25T04:23:41.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:41.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049406+0000", 2023-11-25T04:23:41.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053246480000000004 2023-11-25T04:23:41.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:41.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049464+0000", 2023-11-25T04:23:41.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8205999999999998e-05 2023-11-25T04:23:41.637 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:41.637 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:41.637 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:41.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.638 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:11 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'6, mlcod=0'0)", 2023-11-25T04:23:41.638 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:41.638 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 54.485947803999998, 2023-11-25T04:23:41.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052673499999999996, 2023-11-25T04:23:41.638 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:41.638 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:41.638 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:41.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:41.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:41.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:41.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:41.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:41.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:41.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:41.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:41.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0199999999999998e-07 2023-11-25T04:23:41.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:41.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044220+0000", 2023-11-25T04:23:41.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7720000000000001e-06 2023-11-25T04:23:41.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:41.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044226+0000", 2023-11-25T04:23:41.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8579999999999996e-06 2023-11-25T04:23:41.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:41.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044232+0000", 2023-11-25T04:23:41.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1700000000000002e-06 2023-11-25T04:23:41.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:41.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044274+0000", 2023-11-25T04:23:41.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1254999999999997e-05 2023-11-25T04:23:41.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:41.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:23:41.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6880999999999999e-05 2023-11-25T04:23:41.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:41.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049466+0000", 2023-11-25T04:23:41.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051659219999999999 2023-11-25T04:23:41.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:41.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:41.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:41.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049483+0000", 2023-11-25T04:23:41.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.719e-05 2023-11-25T04:23:41.642 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:41.642 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:41.642 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:41.642 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:41.642 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:41.642 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:41.644 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:23:41.645 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:41.642+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:41.745 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:41.743+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:41.750 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:41.751 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:23:41.751 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2023-11-25T04:23:41.751 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:41.759 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:23:41.846 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:41.845+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:41.864 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:41.864 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:23:41.864 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2023-11-25T04:23:41.864 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2023-11-25T04:23:41.864 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:41.874 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:23:41.947 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:41.945+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:41.982 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:41.982 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2023-11-25T04:23:41.982 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2023-11-25T04:23:41.982 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2023-11-25T04:23:41.982 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:41.982 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:41.982 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:41.982 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 54.948047002999999, 2023-11-25T04:23:41.983 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01072943, 2023-11-25T04:23:41.983 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:41.983 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:41.983 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:41.983 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:41.983 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:41.983 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:41.983 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:41.983 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:41.983 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:41.983 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:41.983 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:41.983 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:41.984 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:41.984 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:41.984 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:41.984 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031829+0000", 2023-11-25T04:23:41.984 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.666e-06 2023-11-25T04:23:41.984 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:41.984 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:41.984 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:41.984 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031839+0000", 2023-11-25T04:23:41.984 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9490000000000008e-06 2023-11-25T04:23:41.984 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:41.984 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:41.984 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:41.985 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031857+0000", 2023-11-25T04:23:41.985 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8093000000000001e-05 2023-11-25T04:23:41.985 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:41.985 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:41.985 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:41.985 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031882+0000", 2023-11-25T04:23:41.985 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4294000000000002e-05 2023-11-25T04:23:41.985 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:41.985 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:41.985 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:41.985 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033130+0000", 2023-11-25T04:23:41.985 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001248264 2023-11-25T04:23:41.985 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:41.985 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:41.986 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:41.986 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033426+0000", 2023-11-25T04:23:41.986 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029620999999999998 2023-11-25T04:23:41.986 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:41.986 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:41.986 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:41.986 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033814+0000", 2023-11-25T04:23:41.986 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000387742 2023-11-25T04:23:41.986 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:41.986 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:41.986 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:41.986 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036793+0000", 2023-11-25T04:23:41.987 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002979598 2023-11-25T04:23:41.987 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:41.987 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:41.987 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:41.987 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042455+0000", 2023-11-25T04:23:41.987 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0056619260000000003 2023-11-25T04:23:41.987 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:41.987 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:41.987 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:41.987 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042502+0000", 2023-11-25T04:23:41.987 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7139999999999999e-05 2023-11-25T04:23:41.987 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:41.987 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:41.987 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:41.988 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042557+0000", 2023-11-25T04:23:41.988 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4548e-05 2023-11-25T04:23:41.988 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:41.988 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:41.988 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:41.988 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:41.988 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:41.988 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:41.988 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:41.988 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 54.946848467999999, 2023-11-25T04:23:41.988 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011505202000000001, 2023-11-25T04:23:41.988 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:41.988 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:41.989 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:41.989 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:41.989 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:41.989 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:41.989 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:41.989 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:41.989 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:41.989 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:41.989 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:41.989 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:41.989 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:41.989 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:41.989 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:41.989 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033027+0000", 2023-11-25T04:23:41.990 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0699999999999999e-06 2023-11-25T04:23:41.990 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:41.990 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:41.990 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:41.990 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033035+0000", 2023-11-25T04:23:41.990 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0830000000000004e-06 2023-11-25T04:23:41.990 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:41.990 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:41.990 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:41.990 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033059+0000", 2023-11-25T04:23:41.990 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3975e-05 2023-11-25T04:23:41.990 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:41.990 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:41.991 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:41.991 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033482+0000", 2023-11-25T04:23:41.991 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042253700000000001 2023-11-25T04:23:41.991 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:41.991 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:41.991 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:41.991 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034520+0000", 2023-11-25T04:23:41.991 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001037991 2023-11-25T04:23:41.991 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:41.991 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:41.991 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:41.991 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034611+0000", 2023-11-25T04:23:41.991 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1113999999999995e-05 2023-11-25T04:23:41.991 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:41.992 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:41.992 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:41.992 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034769+0000", 2023-11-25T04:23:41.992 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000157641 2023-11-25T04:23:41.992 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:41.992 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:41.992 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:41.992 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038292+0000", 2023-11-25T04:23:41.992 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035231160000000002 2023-11-25T04:23:41.992 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:41.992 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:41.992 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:41.992 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044440+0000", 2023-11-25T04:23:41.992 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0061486980000000002 2023-11-25T04:23:41.993 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:41.993 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:41.993 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:41.993 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044495+0000", 2023-11-25T04:23:41.993 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4339000000000003e-05 2023-11-25T04:23:41.993 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:41.993 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:41.993 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:41.993 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044531+0000", 2023-11-25T04:23:41.993 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6637999999999997e-05 2023-11-25T04:23:41.993 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:41.993 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:41.993 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:41.993 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:41.994 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:41.994 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:41.994 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:41.994 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 54.946374114000001, 2023-11-25T04:23:41.994 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0057634039999999997, 2023-11-25T04:23:41.994 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:41.994 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:41.994 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:41.994 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:41.994 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:41.994 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:41.994 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:41.994 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:41.995 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:41.995 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:41.995 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:41.995 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:41.995 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:41.995 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:41.995 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:41.995 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:41.995 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7900000000000007e-07 2023-11-25T04:23:41.995 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:41.995 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:41.995 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:41.995 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033507+0000", 2023-11-25T04:23:41.996 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8900000000000004e-06 2023-11-25T04:23:41.996 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:41.996 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:41.996 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:41.996 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033519+0000", 2023-11-25T04:23:41.996 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1756000000000001e-05 2023-11-25T04:23:41.996 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:41.996 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:41.996 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:41.996 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033527+0000", 2023-11-25T04:23:41.997 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3610000000000001e-06 2023-11-25T04:23:41.997 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:41.997 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:41.997 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:41.997 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034451+0000", 2023-11-25T04:23:41.997 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00092372400000000001 2023-11-25T04:23:41.997 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:41.997 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:41.997 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:41.997 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034809+0000", 2023-11-25T04:23:41.997 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00035757699999999998 2023-11-25T04:23:41.998 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:41.998 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:41.998 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:23:41.998 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034965+0000", 2023-11-25T04:23:41.998 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015638699999999999 2023-11-25T04:23:41.998 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:41.998 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:41.998 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:41.998 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038299+0000", 2023-11-25T04:23:41.998 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0033336559999999999 2023-11-25T04:23:41.998 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:41.998 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:41.998 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:41.998 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039169+0000", 2023-11-25T04:23:41.999 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00087066400000000005 2023-11-25T04:23:41.999 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:41.999 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:41.999 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:41.999 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039226+0000", 2023-11-25T04:23:41.999 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6511999999999997e-05 2023-11-25T04:23:41.999 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:41.999 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:41.999 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:41.999 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039264+0000", 2023-11-25T04:23:41.999 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7898000000000002e-05 2023-11-25T04:23:41.999 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:41.999 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:42.000 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:42.000 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.000 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.000 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:25 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:42.000 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:42.000 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 54.94629553, 2023-11-25T04:23:42.000 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096656169999999996, 2023-11-25T04:23:42.000 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:42.000 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:42.000 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:42.000 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.000 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:42.000 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:42.000 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:42.001 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.001 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.001 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:42.001 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:42.001 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:42.001 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.001 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.001 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:42.001 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033580+0000", 2023-11-25T04:23:42.001 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8199999999999999e-07 2023-11-25T04:23:42.001 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.001 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.001 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:42.002 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033585+0000", 2023-11-25T04:23:42.002 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3680000000000001e-06 2023-11-25T04:23:42.002 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.002 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.002 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:42.002 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033600+0000", 2023-11-25T04:23:42.002 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4331000000000001e-05 2023-11-25T04:23:42.002 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.002 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.002 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:42.002 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033606+0000", 2023-11-25T04:23:42.002 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5780000000000004e-06 2023-11-25T04:23:42.002 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.003 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.003 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:42.003 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035747+0000", 2023-11-25T04:23:42.003 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0021406530000000002 2023-11-25T04:23:42.003 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.003 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.003 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:42.003 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035901+0000", 2023-11-25T04:23:42.003 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000153931 2023-11-25T04:23:42.003 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.003 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.003 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:23:42.003 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036193+0000", 2023-11-25T04:23:42.003 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029211299999999999 2023-11-25T04:23:42.004 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.004 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.004 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:42.004 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038655+0000", 2023-11-25T04:23:42.004 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0024621640000000002 2023-11-25T04:23:42.004 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.004 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.004 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:42.004 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043130+0000", 2023-11-25T04:23:42.004 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.004474698 2023-11-25T04:23:42.004 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.004 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.004 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:42.005 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043204+0000", 2023-11-25T04:23:42.005 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4091999999999995e-05 2023-11-25T04:23:42.005 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.005 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.005 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:42.005 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043245+0000", 2023-11-25T04:23:42.005 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0907e-05 2023-11-25T04:23:42.005 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:42.005 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:42.005 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:42.005 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.005 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.005 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:26 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:42.005 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:42.006 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 54.946259705999999, 2023-11-25T04:23:42.006 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011086945000000001, 2023-11-25T04:23:42.006 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:42.006 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:42.006 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:42.006 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.006 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:42.006 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:42.006 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:42.007 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.007 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.007 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:42.007 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:42.007 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:42.007 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.007 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.007 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:42.007 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:42.007 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1699999999999998e-07 2023-11-25T04:23:42.007 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.007 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.007 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:42.007 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033627+0000", 2023-11-25T04:23:42.008 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.163e-05 2023-11-25T04:23:42.008 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.008 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.008 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:42.008 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033636+0000", 2023-11-25T04:23:42.008 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6859999999999995e-06 2023-11-25T04:23:42.008 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.008 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.008 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:42.008 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033642+0000", 2023-11-25T04:23:42.008 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3149999999999997e-06 2023-11-25T04:23:42.008 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.008 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.008 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:42.009 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035414+0000", 2023-11-25T04:23:42.009 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0017716380000000001 2023-11-25T04:23:42.009 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.009 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.009 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:42.009 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035539+0000", 2023-11-25T04:23:42.009 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000125477 2023-11-25T04:23:42.009 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.009 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.009 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:42.009 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035672+0000", 2023-11-25T04:23:42.009 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000133316 2023-11-25T04:23:42.009 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.010 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.010 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:42.010 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038649+0000", 2023-11-25T04:23:42.010 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029764679999999999 2023-11-25T04:23:42.010 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.010 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.010 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:42.010 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044578+0000", 2023-11-25T04:23:42.010 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0059293510000000002 2023-11-25T04:23:42.010 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.010 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.010 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:42.010 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044651+0000", 2023-11-25T04:23:42.011 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2513000000000002e-05 2023-11-25T04:23:42.011 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.011 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.011 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:42.011 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044702+0000", 2023-11-25T04:23:42.011 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1034e-05 2023-11-25T04:23:42.011 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:42.011 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:42.011 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:42.011 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.011 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.011 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:29 3.b 3:d0630e4c:::mds0_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:42.011 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:42.012 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 54.946221786000002, 2023-11-25T04:23:42.012 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010980227, 2023-11-25T04:23:42.012 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:42.012 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:42.012 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:42.012 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.012 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:42.012 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:42.012 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:42.012 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.012 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.012 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:42.012 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:42.013 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:42.013 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.013 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.013 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:42.013 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:42.013 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3499999999999996e-07 2023-11-25T04:23:42.013 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.013 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.013 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:42.013 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033658+0000", 2023-11-25T04:23:42.013 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1280000000000001e-06 2023-11-25T04:23:42.013 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.013 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.014 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:42.014 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033667+0000", 2023-11-25T04:23:42.014 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1430000000000004e-06 2023-11-25T04:23:42.014 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.014 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.014 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:42.014 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033678+0000", 2023-11-25T04:23:42.014 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0906e-05 2023-11-25T04:23:42.014 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.014 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.014 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:42.014 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034456+0000", 2023-11-25T04:23:42.015 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00077833899999999996 2023-11-25T04:23:42.015 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.015 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.015 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:42.015 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034575+0000", 2023-11-25T04:23:42.015 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011926000000000001 2023-11-25T04:23:42.015 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.015 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.015 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:42.015 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035001+0000", 2023-11-25T04:23:42.015 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042550999999999998 2023-11-25T04:23:42.017 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.017 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.017 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:42.017 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.037923+0000", 2023-11-25T04:23:42.017 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029219670000000001 2023-11-25T04:23:42.017 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.017 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.017 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:42.017 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044538+0000", 2023-11-25T04:23:42.017 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0066152349999999997 2023-11-25T04:23:42.017 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.018 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.018 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:42.018 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044594+0000", 2023-11-25T04:23:42.018 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5593000000000001e-05 2023-11-25T04:23:42.018 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.018 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.018 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:42.018 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044633+0000", 2023-11-25T04:23:42.018 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9511000000000002e-05 2023-11-25T04:23:42.018 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:42.018 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:42.018 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:42.018 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.018 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.019 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:34 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:42.019 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:42.019 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 54.945971673000003, 2023-11-25T04:23:42.019 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010920445000000001, 2023-11-25T04:23:42.019 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:42.019 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:42.019 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:42.019 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.019 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:42.019 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:42.019 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:42.019 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.019 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.020 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:42.020 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:42.020 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:42.020 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.020 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.020 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:42.020 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033904+0000", 2023-11-25T04:23:42.020 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6599999999999994e-07 2023-11-25T04:23:42.020 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.020 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.020 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:42.020 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033909+0000", 2023-11-25T04:23:42.020 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5639999999999996e-06 2023-11-25T04:23:42.021 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.021 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.021 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:42.021 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033923+0000", 2023-11-25T04:23:42.021 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3028e-05 2023-11-25T04:23:42.021 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.021 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.021 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:42.021 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034043+0000", 2023-11-25T04:23:42.021 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000120328 2023-11-25T04:23:42.021 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.021 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.021 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:42.021 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035576+0000", 2023-11-25T04:23:42.022 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015333720000000001 2023-11-25T04:23:42.022 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.022 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.022 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:42.022 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035699+0000", 2023-11-25T04:23:42.022 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000123001 2023-11-25T04:23:42.022 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.022 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.022 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:42.022 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044823+0000", 2023-11-25T04:23:42.022 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0091241859999999994 2023-11-25T04:23:42.022 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:42.022 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:42.022 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:42.023 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.023 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.023 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:35 3.1f 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:42.023 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:42.023 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 54.944667467999999, 2023-11-25T04:23:42.023 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096164779999999995, 2023-11-25T04:23:42.023 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:42.023 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:42.023 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:42.023 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.023 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:42.023 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:42.023 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:42.023 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.024 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.024 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:42.024 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:42.024 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:42.024 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.024 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.024 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:42.024 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035208+0000", 2023-11-25T04:23:42.024 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2699999999999998e-07 2023-11-25T04:23:42.024 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.024 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.024 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:42.024 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035213+0000", 2023-11-25T04:23:42.025 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.6070000000000001e-06 2023-11-25T04:23:42.025 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.025 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.025 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:42.025 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035350+0000", 2023-11-25T04:23:42.025 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013693599999999999 2023-11-25T04:23:42.025 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.025 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.025 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:42.025 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035358+0000", 2023-11-25T04:23:42.025 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0099999999999995e-06 2023-11-25T04:23:42.025 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.025 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.025 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:42.026 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036260+0000", 2023-11-25T04:23:42.026 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00090240100000000005 2023-11-25T04:23:42.026 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.026 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.027 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:42.027 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036517+0000", 2023-11-25T04:23:42.027 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00025736299999999998 2023-11-25T04:23:42.027 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.027 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.027 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:42.027 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044824+0000", 2023-11-25T04:23:42.027 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0083063340000000003 2023-11-25T04:23:42.027 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:42.027 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:42.027 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:42.027 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.028 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.028 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:37 3.1e 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:42.028 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:42.028 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 54.944500781999999, 2023-11-25T04:23:42.028 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0078322640000000002, 2023-11-25T04:23:42.028 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:42.028 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:42.028 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:42.028 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.028 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:42.028 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:42.028 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:42.029 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.029 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.029 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:42.029 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:42.029 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:42.029 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.029 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.029 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:42.029 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035375+0000", 2023-11-25T04:23:42.029 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9100000000000003e-07 2023-11-25T04:23:42.029 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.029 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.030 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:42.030 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035379+0000", 2023-11-25T04:23:42.030 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9430000000000002e-06 2023-11-25T04:23:42.030 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.030 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.030 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:42.030 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035388+0000", 2023-11-25T04:23:42.030 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2760000000000006e-06 2023-11-25T04:23:42.030 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.030 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.030 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:42.030 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035396+0000", 2023-11-25T04:23:42.031 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8480000000000001e-06 2023-11-25T04:23:42.031 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.031 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.031 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:42.031 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035452+0000", 2023-11-25T04:23:42.031 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5831000000000001e-05 2023-11-25T04:23:42.031 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.031 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.031 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:42.031 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035934+0000", 2023-11-25T04:23:42.031 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00048265599999999998 2023-11-25T04:23:42.031 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.031 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.032 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:42.032 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043206+0000", 2023-11-25T04:23:42.032 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.007271919 2023-11-25T04:23:42.032 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:42.032 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:42.032 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:42.032 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.032 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.032 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing(3.1a epoch 18/16 rep_tid 3 entries 18'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:36 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:23:42.032 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:42.032 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 54.943225196, 2023-11-25T04:23:42.032 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002073414, 2023-11-25T04:23:42.033 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:42.033 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:42.033 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:42.033 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.033 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:42.033 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:42.033 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:42.033 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.033 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.033 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:42.033 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:42.033 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:42.034 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.034 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.034 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:42.034 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036650+0000", 2023-11-25T04:23:42.034 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7300000000000002e-07 2023-11-25T04:23:42.034 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.034 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.034 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:42.034 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036659+0000", 2023-11-25T04:23:42.034 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.7499999999999992e-06 2023-11-25T04:23:42.034 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.034 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.034 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:42.035 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036673+0000", 2023-11-25T04:23:42.035 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4917e-05 2023-11-25T04:23:42.035 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.035 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.035 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:42.035 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036687+0000", 2023-11-25T04:23:42.035 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3883000000000001e-05 2023-11-25T04:23:42.035 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.035 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.035 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:42.035 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036781+0000", 2023-11-25T04:23:42.035 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3219000000000003e-05 2023-11-25T04:23:42.036 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.036 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.036 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:42.036 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038723+0000", 2023-11-25T04:23:42.036 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019422720000000001 2023-11-25T04:23:42.036 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:42.036 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:42.036 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:42.036 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.036 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.036 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:28 3.13 e18/16 3:cd5a64a3:::100.00000000:head v 18'1, mlcod=0'0)", 2023-11-25T04:23:42.036 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:42.036 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 54.939564038, 2023-11-25T04:23:42.037 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001850546, 2023-11-25T04:23:42.037 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:42.037 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:42.037 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:42.037 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.037 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:42.037 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:42.037 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:42.037 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.037 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.037 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:42.037 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:42.037 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:42.038 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.038 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.038 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:42.038 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:42.038 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7800000000000002e-07 2023-11-25T04:23:42.038 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.038 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.038 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:42.038 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040331+0000", 2023-11-25T04:23:42.038 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0449999999999999e-05 2023-11-25T04:23:42.038 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.038 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.039 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:42.039 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040339+0000", 2023-11-25T04:23:42.039 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0709999999999995e-06 2023-11-25T04:23:42.039 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.039 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.039 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:42.039 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040347+0000", 2023-11-25T04:23:42.039 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7079999999999997e-06 2023-11-25T04:23:42.039 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.039 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.039 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:42.039 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040410+0000", 2023-11-25T04:23:42.040 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2453000000000007e-05 2023-11-25T04:23:42.040 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.040 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.040 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:42.040 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040422+0000", 2023-11-25T04:23:42.040 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2429000000000001e-05 2023-11-25T04:23:42.040 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.040 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.040 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:42.040 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042129+0000", 2023-11-25T04:23:42.040 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001707266 2023-11-25T04:23:42.040 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.040 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.041 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:42.041 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042161+0000", 2023-11-25T04:23:42.041 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1791000000000001e-05 2023-11-25T04:23:42.041 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:42.041 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:42.041 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:42.041 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.041 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.041 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:27 3.13 e18/16 3:c91d4a1d:::mds0_inotable:head v 18'2, mlcod=0'0)", 2023-11-25T04:23:42.041 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:42.041 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 54.939360514999997, 2023-11-25T04:23:42.041 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0016886430000000001, 2023-11-25T04:23:42.041 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:42.042 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:42.042 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:42.042 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.042 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:42.042 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:42.042 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:42.042 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.042 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.042 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:42.042 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:42.042 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:42.042 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.043 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.043 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:42.043 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:42.043 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3999999999999997e-07 2023-11-25T04:23:42.043 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.043 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.043 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:42.043 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040520+0000", 2023-11-25T04:23:42.043 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2989999999999996e-06 2023-11-25T04:23:42.043 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.043 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.043 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:42.043 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040527+0000", 2023-11-25T04:23:42.044 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5730000000000002e-06 2023-11-25T04:23:42.044 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.044 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.044 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:42.044 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040535+0000", 2023-11-25T04:23:42.044 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8630000000000008e-06 2023-11-25T04:23:42.044 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.044 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.044 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:42.044 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040683+0000", 2023-11-25T04:23:42.044 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000148002 2023-11-25T04:23:42.044 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.045 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.045 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:42.045 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040700+0000", 2023-11-25T04:23:42.045 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7065e-05 2023-11-25T04:23:42.045 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.045 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.045 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:42.045 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042163+0000", 2023-11-25T04:23:42.045 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001462767 2023-11-25T04:23:42.045 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.045 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.046 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:42.046 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042203+0000", 2023-11-25T04:23:42.046 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9734000000000003e-05 2023-11-25T04:23:42.046 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:42.046 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:42.046 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:42.046 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.046 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.046 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:20 3.1c e18/16 3:3ed09add:::607.00000000:head v 18'1, mlcod=0'0)", 2023-11-25T04:23:42.046 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:42.046 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 54.939325474, 2023-11-25T04:23:42.047 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015271, 2023-11-25T04:23:42.047 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:42.047 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:42.047 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:42.047 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.047 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:42.047 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:42.047 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:42.047 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.047 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.047 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:42.047 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:42.047 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:42.048 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.048 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.048 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:42.048 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:42.048 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4799999999999999e-07 2023-11-25T04:23:42.048 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.048 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.048 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:42.048 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040561+0000", 2023-11-25T04:23:42.048 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1800000000000001e-05 2023-11-25T04:23:42.048 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.048 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.048 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:42.049 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040568+0000", 2023-11-25T04:23:42.049 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7730000000000002e-06 2023-11-25T04:23:42.049 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.049 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.049 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:42.049 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040587+0000", 2023-11-25T04:23:42.049 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8691000000000001e-05 2023-11-25T04:23:42.049 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.049 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.049 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:42.049 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040638+0000", 2023-11-25T04:23:42.049 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1068e-05 2023-11-25T04:23:42.049 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.050 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.050 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:42.050 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040670+0000", 2023-11-25T04:23:42.050 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2125999999999997e-05 2023-11-25T04:23:42.050 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.050 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.050 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:42.050 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042032+0000", 2023-11-25T04:23:42.050 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001361821 2023-11-25T04:23:42.050 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.050 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.050 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:42.051 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042076+0000", 2023-11-25T04:23:42.051 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4472999999999999e-05 2023-11-25T04:23:42.051 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:42.051 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:42.051 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:42.051 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.051 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.051 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:21 3.1c e18/16 3:3ed09add:::607.00000000:head v 18'2, mlcod=0'0)", 2023-11-25T04:23:42.051 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:42.051 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 54.939090749000002, 2023-11-25T04:23:42.051 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0013228160000000001, 2023-11-25T04:23:42.051 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:42.052 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:42.052 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:42.052 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.052 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:42.052 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:42.052 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:42.052 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.052 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.052 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:42.052 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:42.052 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:42.052 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.053 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.053 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:42.053 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:42.053 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9299999999999999e-07 2023-11-25T04:23:42.053 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.053 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.053 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:42.053 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040790+0000", 2023-11-25T04:23:42.053 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5770000000000001e-06 2023-11-25T04:23:42.053 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.053 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.053 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:42.054 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040799+0000", 2023-11-25T04:23:42.054 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6439999999999995e-06 2023-11-25T04:23:42.054 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.054 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.054 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:42.054 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040806+0000", 2023-11-25T04:23:42.054 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0180000000000004e-06 2023-11-25T04:23:42.054 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.054 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.054 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:42.054 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040907+0000", 2023-11-25T04:23:42.054 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000100096 2023-11-25T04:23:42.055 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.055 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.055 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:42.055 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040925+0000", 2023-11-25T04:23:42.055 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8536999999999999e-05 2023-11-25T04:23:42.055 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.055 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.055 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:42.055 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042079+0000", 2023-11-25T04:23:42.055 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00115358 2023-11-25T04:23:42.055 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.055 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.056 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:42.056 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042107+0000", 2023-11-25T04:23:42.056 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7971e-05 2023-11-25T04:23:42.056 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:42.056 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:42.056 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:42.056 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.056 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.056 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop_reply(mds.0.3:4 3.1f e18/16 ondisk, result = 0)", 2023-11-25T04:23:42.056 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:42.056 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 54.937751359000004, 2023-11-25T04:23:42.057 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00044092900000000001, 2023-11-25T04:23:42.057 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:42.057 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:42.057 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:42.057 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.057 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:42.057 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:42.057 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:42.057 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.057 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.057 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:42.057 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:42.058 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:42.058 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.058 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.058 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:42.058 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042124+0000", 2023-11-25T04:23:42.058 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1800000000000005e-07 2023-11-25T04:23:42.058 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.058 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.058 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:42.058 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042128+0000", 2023-11-25T04:23:42.058 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2289999999999997e-06 2023-11-25T04:23:42.058 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.059 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.059 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:42.059 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042138+0000", 2023-11-25T04:23:42.059 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9729999999999992e-06 2023-11-25T04:23:42.059 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.059 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.059 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:42.059 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042145+0000", 2023-11-25T04:23:42.059 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.336e-06 2023-11-25T04:23:42.059 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.059 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.059 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:42.060 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042441+0000", 2023-11-25T04:23:42.060 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029650500000000002 2023-11-25T04:23:42.060 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.060 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.060 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:42.060 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042448+0000", 2023-11-25T04:23:42.060 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4100000000000002e-06 2023-11-25T04:23:42.060 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.060 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.060 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:42.060 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042564+0000", 2023-11-25T04:23:42.061 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000115758 2023-11-25T04:23:42.061 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:42.061 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:42.061 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:42.061 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.061 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.061 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing_reply(3.1f epoch 18/16 rep_tid 8 lcod 18'4)", 2023-11-25T04:23:42.061 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:42.061 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 54.935425946999999, 2023-11-25T04:23:42.061 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00038154699999999998, 2023-11-25T04:23:42.061 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:42.061 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:42.062 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:42.062 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.062 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:42.062 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:42.062 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:42.062 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.062 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.062 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:42.062 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:42.062 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:42.062 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.063 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.063 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:42.063 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:42.063 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2700000000000001e-07 2023-11-25T04:23:42.063 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.063 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.063 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:42.063 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044451+0000", 2023-11-25T04:23:42.063 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7460000000000001e-06 2023-11-25T04:23:42.063 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.064 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.064 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:42.064 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044455+0000", 2023-11-25T04:23:42.064 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5809999999999998e-06 2023-11-25T04:23:42.064 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.064 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.064 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:42.064 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044458+0000", 2023-11-25T04:23:42.064 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0170000000000001e-06 2023-11-25T04:23:42.064 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.064 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.064 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:42.065 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044734+0000", 2023-11-25T04:23:42.065 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00027582799999999999 2023-11-25T04:23:42.065 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.065 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.065 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:42.065 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044830+0000", 2023-11-25T04:23:42.065 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6248000000000004e-05 2023-11-25T04:23:42.065 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:42.065 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:42.065 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:42.065 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.065 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.066 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing_reply(3.b epoch 18/16 rep_tid 5 lcod 18'2)", 2023-11-25T04:23:42.066 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:42.066 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 54.935389286000003, 2023-11-25T04:23:42.066 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000344488, 2023-11-25T04:23:42.066 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:42.066 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:42.066 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:42.066 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.066 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:42.066 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:42.066 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:42.066 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.067 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.067 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:42.067 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:42.067 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:42.067 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.067 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.067 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:42.067 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:42.067 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.55e-07 2023-11-25T04:23:42.067 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.067 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.067 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:42.068 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044487+0000", 2023-11-25T04:23:42.068 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2470000000000001e-06 2023-11-25T04:23:42.068 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.068 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.068 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:42.068 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044491+0000", 2023-11-25T04:23:42.068 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9659999999999998e-06 2023-11-25T04:23:42.068 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.068 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.068 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:42.068 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044494+0000", 2023-11-25T04:23:42.068 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2310000000000001e-06 2023-11-25T04:23:42.069 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.069 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.069 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:42.069 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044756+0000", 2023-11-25T04:23:42.069 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00026172800000000003 2023-11-25T04:23:42.069 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.069 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.069 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:42.069 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044830+0000", 2023-11-25T04:23:42.069 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4160999999999997e-05 2023-11-25T04:23:42.069 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:42.070 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:42.070 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:42.070 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.070 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.070 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:39 3.6 3:654134d2:::mds0_openfiles.0:head [omap-clear,omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2023-11-25T04:23:42.070 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:42.070 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 49.875563243999999, 2023-11-25T04:23:42.070 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001788658, 2023-11-25T04:23:42.070 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:42.070 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:42.070 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:42.071 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.071 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:42.071 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:42.071 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:42.071 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.071 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.072 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:42.072 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:42.072 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:42.072 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.072 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.072 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:42.072 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104314+0000", 2023-11-25T04:23:42.072 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3099999999999999e-06 2023-11-25T04:23:42.072 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.072 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.072 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:42.072 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104325+0000", 2023-11-25T04:23:42.073 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1511999999999999e-05 2023-11-25T04:23:42.073 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.073 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.073 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:42.073 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104362+0000", 2023-11-25T04:23:42.073 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6903000000000002e-05 2023-11-25T04:23:42.073 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.073 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.073 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:42.073 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104387+0000", 2023-11-25T04:23:42.073 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4766e-05 2023-11-25T04:23:42.073 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.074 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.074 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:42.074 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104465+0000", 2023-11-25T04:23:42.074 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8119000000000002e-05 2023-11-25T04:23:42.074 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.074 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.074 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:42.074 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104557+0000", 2023-11-25T04:23:42.074 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2496000000000003e-05 2023-11-25T04:23:42.074 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.074 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.074 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:42.075 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104646+0000", 2023-11-25T04:23:42.075 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8818000000000002e-05 2023-11-25T04:23:42.075 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.075 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.075 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:42.075 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105201+0000", 2023-11-25T04:23:42.075 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00055506000000000002 2023-11-25T04:23:42.075 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.075 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.075 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:42.075 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105994+0000", 2023-11-25T04:23:42.075 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00079294700000000001 2023-11-25T04:23:42.076 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.076 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.076 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:42.076 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.106055+0000", 2023-11-25T04:23:42.076 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0566000000000003e-05 2023-11-25T04:23:42.076 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.076 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.076 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:42.076 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.106100+0000", 2023-11-25T04:23:42.076 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5160999999999997e-05 2023-11-25T04:23:42.076 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:42.077 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:42.077 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:42.077 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.077 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.077 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:43 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2023-11-25T04:23:42.077 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:42.077 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 34.875338247999998, 2023-11-25T04:23:42.077 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001751166, 2023-11-25T04:23:42.077 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:42.078 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:42.078 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:42.078 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.078 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:42.078 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:42.078 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:42.078 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.078 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.078 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:42.078 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:42.078 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:42.079 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.079 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.079 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:42.079 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104539+0000", 2023-11-25T04:23:42.079 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9529999999999999e-06 2023-11-25T04:23:42.079 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.079 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.079 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:42.079 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104557+0000", 2023-11-25T04:23:42.079 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7464000000000001e-05 2023-11-25T04:23:42.079 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.079 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.079 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:42.080 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104590+0000", 2023-11-25T04:23:42.080 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2966000000000003e-05 2023-11-25T04:23:42.080 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.080 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.080 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:42.080 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104608+0000", 2023-11-25T04:23:42.080 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8193e-05 2023-11-25T04:23:42.080 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.080 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.080 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:42.080 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104767+0000", 2023-11-25T04:23:42.081 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015882 2023-11-25T04:23:42.081 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.081 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.081 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:42.081 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104909+0000", 2023-11-25T04:23:42.081 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014244999999999999 2023-11-25T04:23:42.081 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.081 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.081 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:42.081 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.105059+0000", 2023-11-25T04:23:42.081 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014964399999999999 2023-11-25T04:23:42.081 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.081 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.082 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:42.082 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.105701+0000", 2023-11-25T04:23:42.082 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00064201099999999995 2023-11-25T04:23:42.082 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.082 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.082 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:42.082 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106233+0000", 2023-11-25T04:23:42.082 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00053178500000000001 2023-11-25T04:23:42.082 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.082 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.082 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:42.083 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106265+0000", 2023-11-25T04:23:42.083 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1986000000000002e-05 2023-11-25T04:23:42.083 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.083 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.083 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:42.083 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106288+0000", 2023-11-25T04:23:42.083 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2894000000000001e-05 2023-11-25T04:23:42.083 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:42.083 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:42.083 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:42.083 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.083 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.084 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:45 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e23)", 2023-11-25T04:23:42.084 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:23:42.084 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 29.875140208000001, 2023-11-25T04:23:42.084 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00098423299999999994, 2023-11-25T04:23:42.084 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:42.084 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:42.084 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:42.084 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.084 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:42.084 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:23:42.084 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:42.084 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.085 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.085 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:42.085 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:23:42.085 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:42.085 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.085 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.085 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:42.085 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104737+0000", 2023-11-25T04:23:42.085 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3309999999999998e-06 2023-11-25T04:23:42.085 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.085 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.085 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:42.086 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104748+0000", 2023-11-25T04:23:42.086 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1080999999999999e-05 2023-11-25T04:23:42.086 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.086 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.086 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:42.086 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104771+0000", 2023-11-25T04:23:42.086 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2707000000000001e-05 2023-11-25T04:23:42.086 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.086 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.086 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:42.086 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104800+0000", 2023-11-25T04:23:42.086 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9595e-05 2023-11-25T04:23:42.087 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.087 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.087 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:42.087 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104890+0000", 2023-11-25T04:23:42.087 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.9808999999999993e-05 2023-11-25T04:23:42.087 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.087 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.087 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:42.087 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105004+0000", 2023-11-25T04:23:42.087 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000114508 2023-11-25T04:23:42.087 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.087 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.088 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:42.088 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105634+0000", 2023-11-25T04:23:42.088 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00062985799999999996 2023-11-25T04:23:42.088 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.088 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.088 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:42.088 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105671+0000", 2023-11-25T04:23:42.088 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6640000000000002e-05 2023-11-25T04:23:42.088 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:42.088 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:42.088 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:42.088 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105719+0000", 2023-11-25T04:23:42.089 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7704000000000003e-05 2023-11-25T04:23:42.089 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:42.089 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:42.089 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:42.089 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:42.089 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:42.089 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:42.090 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:42.089+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:42.091 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:23:42.191 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:42.190+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:42.196 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:42.197 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:23:42.197 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2023-11-25T04:23:42.197 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:42.207 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:23:42.292 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:42.291+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:42.352 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:42.352 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:23:42.352 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2023-11-25T04:23:42.352 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2023-11-25T04:23:42.352 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:42.360 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:23:42.393 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:42.391+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:42.494 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:42.492+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:42.506 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:42.507 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2023-11-25T04:23:42.507 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2023-11-25T04:23:42.507 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2023-11-25T04:23:42.507 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.507 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:155 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'109, mlcod=11'105)", 2023-11-25T04:23:42.507 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:42.507 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 64.719582916999997, 2023-11-25T04:23:42.507 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061719019999999999, 2023-11-25T04:23:42.507 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:42.507 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:42.507 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:42.507 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.507 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:42.508 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:42.508 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:42.508 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.508 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.508 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:42.508 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:42.508 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:42.508 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.508 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.508 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:42.508 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784884+0000", 2023-11-25T04:23:42.508 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3099999999999999e-07 2023-11-25T04:23:42.508 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.509 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.509 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:42.509 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784942+0000", 2023-11-25T04:23:42.509 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8570000000000003e-05 2023-11-25T04:23:42.509 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.509 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.509 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:42.509 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784949+0000", 2023-11-25T04:23:42.509 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0310000000000001e-06 2023-11-25T04:23:42.509 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.509 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.509 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:42.509 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784953+0000", 2023-11-25T04:23:42.510 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6320000000000001e-06 2023-11-25T04:23:42.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.510 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.510 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:42.510 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784993+0000", 2023-11-25T04:23:42.510 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9737000000000003e-05 2023-11-25T04:23:42.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.510 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.510 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:42.510 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785011+0000", 2023-11-25T04:23:42.510 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8349999999999999e-05 2023-11-25T04:23:42.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.510 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.511 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:42.511 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791024+0000", 2023-11-25T04:23:42.511 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006012997 2023-11-25T04:23:42.511 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.511 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.511 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:42.511 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791055+0000", 2023-11-25T04:23:42.511 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1254000000000003e-05 2023-11-25T04:23:42.511 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:42.511 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:42.511 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:42.511 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.511 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.512 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:156 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'110, mlcod=11'105)", 2023-11-25T04:23:42.512 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:42.512 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 64.718996868000005, 2023-11-25T04:23:42.512 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005616116, 2023-11-25T04:23:42.512 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:42.512 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:42.512 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:42.512 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.512 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:42.512 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:42.512 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:42.512 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.512 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.513 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:42.513 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:42.513 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:42.513 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.513 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.513 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:42.513 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785470+0000", 2023-11-25T04:23:42.513 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0699999999999998e-07 2023-11-25T04:23:42.513 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.513 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.513 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:42.513 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785474+0000", 2023-11-25T04:23:42.513 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.048e-06 2023-11-25T04:23:42.513 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.514 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.514 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:42.514 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785480+0000", 2023-11-25T04:23:42.514 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7729999999999996e-06 2023-11-25T04:23:42.514 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.514 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.514 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:42.514 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785483+0000", 2023-11-25T04:23:42.514 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1970000000000001e-06 2023-11-25T04:23:42.514 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.514 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.514 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:42.514 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787936+0000", 2023-11-25T04:23:42.515 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024529040000000001 2023-11-25T04:23:42.515 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.515 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.515 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:42.515 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787947+0000", 2023-11-25T04:23:42.515 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1669e-05 2023-11-25T04:23:42.515 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.515 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.515 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:42.515 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791057+0000", 2023-11-25T04:23:42.515 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031095200000000002 2023-11-25T04:23:42.515 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.515 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.516 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:42.516 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791086+0000", 2023-11-25T04:23:42.516 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8697999999999999e-05 2023-11-25T04:23:42.516 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:42.516 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:42.516 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:42.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.516 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.516 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:157 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'111, mlcod=11'106)", 2023-11-25T04:23:42.516 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:42.516 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 64.718404309999997, 2023-11-25T04:23:42.516 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050590280000000001, 2023-11-25T04:23:42.516 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:42.517 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:42.517 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:42.517 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.517 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:42.517 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:42.517 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:42.517 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.517 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.517 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:42.517 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:42.517 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:42.517 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.517 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.518 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:42.518 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786063+0000", 2023-11-25T04:23:42.518 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2600000000000002e-07 2023-11-25T04:23:42.518 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.518 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.518 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:42.518 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786069+0000", 2023-11-25T04:23:42.518 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8839999999999999e-06 2023-11-25T04:23:42.518 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.518 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.518 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:42.518 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786079+0000", 2023-11-25T04:23:42.518 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0124e-05 2023-11-25T04:23:42.519 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.519 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.519 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:42.519 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786085+0000", 2023-11-25T04:23:42.519 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6660000000000002e-06 2023-11-25T04:23:42.519 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.519 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.519 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:42.519 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788160+0000", 2023-11-25T04:23:42.519 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020747669999999999 2023-11-25T04:23:42.519 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.519 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.519 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:42.519 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788180+0000", 2023-11-25T04:23:42.520 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0208999999999999e-05 2023-11-25T04:23:42.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.520 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.520 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:42.520 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791087+0000", 2023-11-25T04:23:42.520 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029064630000000002 2023-11-25T04:23:42.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.520 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.520 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:42.520 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791121+0000", 2023-11-25T04:23:42.520 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4289000000000003e-05 2023-11-25T04:23:42.520 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:42.520 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:42.520 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:42.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.521 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.521 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:158 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'112, mlcod=11'106)", 2023-11-25T04:23:42.521 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:42.521 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 64.717987917000002, 2023-11-25T04:23:42.521 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046663030000000001, 2023-11-25T04:23:42.521 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:42.521 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:42.521 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:42.521 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.521 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:42.521 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:42.521 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:42.522 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.522 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.522 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:42.522 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:42.522 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:42.522 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.522 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.522 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:42.522 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786479+0000", 2023-11-25T04:23:42.522 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2799999999999996e-07 2023-11-25T04:23:42.522 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.522 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.522 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:42.522 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786498+0000", 2023-11-25T04:23:42.523 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9069e-05 2023-11-25T04:23:42.523 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.523 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.523 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:42.523 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786508+0000", 2023-11-25T04:23:42.523 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5810000000000003e-06 2023-11-25T04:23:42.523 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.523 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.523 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:42.523 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786512+0000", 2023-11-25T04:23:42.523 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4889999999999997e-06 2023-11-25T04:23:42.523 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.523 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.523 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:42.524 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788438+0000", 2023-11-25T04:23:42.524 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019257059999999999 2023-11-25T04:23:42.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.524 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.524 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:42.524 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788449+0000", 2023-11-25T04:23:42.524 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1419e-05 2023-11-25T04:23:42.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.524 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.524 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:42.524 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791122+0000", 2023-11-25T04:23:42.524 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002672822 2023-11-25T04:23:42.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.524 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.525 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:42.525 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791145+0000", 2023-11-25T04:23:42.525 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2589e-05 2023-11-25T04:23:42.525 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:42.525 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:42.525 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:42.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.525 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.525 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:206 1.0 e12/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 12'152, mlcod=12'148)", 2023-11-25T04:23:42.525 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:42.525 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 64.206219446999995, 2023-11-25T04:23:42.525 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046728189999999999, 2023-11-25T04:23:42.525 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:42.525 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:42.526 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:42.526 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.526 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:42.526 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:42.526 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:42.526 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.526 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.526 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:42.526 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:42.526 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:42.526 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.526 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.526 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:42.527 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298248+0000", 2023-11-25T04:23:42.527 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3600000000000003e-07 2023-11-25T04:23:42.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.527 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.527 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:42.527 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298278+0000", 2023-11-25T04:23:42.527 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0474e-05 2023-11-25T04:23:42.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.527 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.527 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:42.527 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298288+0000", 2023-11-25T04:23:42.527 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0156000000000001e-05 2023-11-25T04:23:42.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.527 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.528 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:42.528 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298293+0000", 2023-11-25T04:23:42.528 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7400000000000004e-06 2023-11-25T04:23:42.528 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.528 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.528 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:42.528 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298333+0000", 2023-11-25T04:23:42.528 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9666000000000003e-05 2023-11-25T04:23:42.528 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.528 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.528 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:42.528 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298351+0000", 2023-11-25T04:23:42.528 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8544999999999999e-05 2023-11-25T04:23:42.528 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.529 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.529 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:42.529 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302891+0000", 2023-11-25T04:23:42.529 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045398519999999996 2023-11-25T04:23:42.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.529 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.529 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:42.529 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302920+0000", 2023-11-25T04:23:42.529 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8750000000000001e-05 2023-11-25T04:23:42.529 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:42.529 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:42.529 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:42.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.529 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.530 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:14 3.15 3:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:42.530 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:42.530 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.471085090000003, 2023-11-25T04:23:42.530 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010990437, 2023-11-25T04:23:42.530 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:42.530 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:42.530 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:42.530 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.530 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:42.530 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:42.530 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:42.530 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.530 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.531 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:42.531 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:42.531 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:42.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.531 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.531 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:42.531 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033383+0000", 2023-11-25T04:23:42.531 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9870000000000002e-06 2023-11-25T04:23:42.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.531 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.531 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:42.531 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033398+0000", 2023-11-25T04:23:42.531 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4569999999999999e-05 2023-11-25T04:23:42.532 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.532 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.532 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:42.532 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033411+0000", 2023-11-25T04:23:42.532 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3543999999999999e-05 2023-11-25T04:23:42.532 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.532 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.532 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:42.532 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033423+0000", 2023-11-25T04:23:42.532 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1905999999999999e-05 2023-11-25T04:23:42.532 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.532 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.532 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:42.532 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034433+0000", 2023-11-25T04:23:42.533 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00101022 2023-11-25T04:23:42.533 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.533 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.533 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:42.533 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034539+0000", 2023-11-25T04:23:42.533 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010597199999999999 2023-11-25T04:23:42.533 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.533 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.533 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:42.533 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034667+0000", 2023-11-25T04:23:42.533 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012765500000000001 2023-11-25T04:23:42.533 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.533 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.534 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:42.534 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037803+0000", 2023-11-25T04:23:42.534 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031362220000000001 2023-11-25T04:23:42.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.534 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.534 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:42.534 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:23:42.534 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0064964669999999997 2023-11-25T04:23:42.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.534 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.534 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:42.534 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044331+0000", 2023-11-25T04:23:42.534 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1544999999999997e-05 2023-11-25T04:23:42.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.535 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:42.535 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044372+0000", 2023-11-25T04:23:42.535 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0349000000000003e-05 2023-11-25T04:23:42.535 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:42.535 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:42.535 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:42.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.535 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:15 3.15 3:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:42.535 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:42.536 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.47042544, 2023-11-25T04:23:42.536 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010464362, 2023-11-25T04:23:42.536 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:42.536 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:42.536 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:42.536 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.536 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:42.536 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:42.536 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:42.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.536 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.536 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:42.536 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:42.537 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:42.537 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.537 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.537 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:42.537 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034042+0000", 2023-11-25T04:23:42.537 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0999999999999997e-07 2023-11-25T04:23:42.537 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.537 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.537 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:42.537 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034048+0000", 2023-11-25T04:23:42.537 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0390000000000001e-06 2023-11-25T04:23:42.537 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.537 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.538 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:42.538 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034059+0000", 2023-11-25T04:23:42.538 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1182999999999999e-05 2023-11-25T04:23:42.538 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.538 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.538 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:42.538 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034067+0000", 2023-11-25T04:23:42.538 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1559999999999995e-06 2023-11-25T04:23:42.538 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.538 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.538 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:42.538 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035005+0000", 2023-11-25T04:23:42.539 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00093747399999999999 2023-11-25T04:23:42.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.539 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:42.539 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035077+0000", 2023-11-25T04:23:42.539 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2747e-05 2023-11-25T04:23:42.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.539 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:42.539 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035219+0000", 2023-11-25T04:23:42.539 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000141877 2023-11-25T04:23:42.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.540 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:42.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037810+0000", 2023-11-25T04:23:42.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002590752 2023-11-25T04:23:42.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.540 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.540 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:42.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044437+0000", 2023-11-25T04:23:42.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066270110000000004 2023-11-25T04:23:42.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.540 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.540 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:42.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044481+0000", 2023-11-25T04:23:42.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4106000000000001e-05 2023-11-25T04:23:42.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.541 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.541 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:42.541 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044505+0000", 2023-11-25T04:23:42.541 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4207e-05 2023-11-25T04:23:42.541 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:42.541 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:42.541 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:42.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.541 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.541 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:16 3.1d 3:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:42.541 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:42.541 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.470383591999997, 2023-11-25T04:23:42.542 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010413923, 2023-11-25T04:23:42.542 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:42.542 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:42.542 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:42.542 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.542 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:42.542 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:42.542 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:42.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.542 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.542 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:42.542 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:42.542 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:42.543 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.543 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.543 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:42.543 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:42.543 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4499999999999997e-07 2023-11-25T04:23:42.543 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.543 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.543 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:42.543 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034087+0000", 2023-11-25T04:23:42.543 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9770000000000002e-06 2023-11-25T04:23:42.543 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.543 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.543 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:42.544 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034094+0000", 2023-11-25T04:23:42.544 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8809999999999998e-06 2023-11-25T04:23:42.544 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.544 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.544 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:42.544 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034100+0000", 2023-11-25T04:23:42.544 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8440000000000003e-06 2023-11-25T04:23:42.544 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.544 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.544 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:42.544 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034993+0000", 2023-11-25T04:23:42.544 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00089336799999999996 2023-11-25T04:23:42.545 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.545 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.545 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:42.545 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035099+0000", 2023-11-25T04:23:42.545 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106037 2023-11-25T04:23:42.545 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.545 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.545 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:42.545 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035276+0000", 2023-11-25T04:23:42.545 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017631799999999999 2023-11-25T04:23:42.545 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.545 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.545 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:42.546 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037815+0000", 2023-11-25T04:23:42.546 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025395959999999999 2023-11-25T04:23:42.546 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.546 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.546 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:42.546 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044420+0000", 2023-11-25T04:23:42.546 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066045749999999997 2023-11-25T04:23:42.546 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.546 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.546 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:42.546 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044463+0000", 2023-11-25T04:23:42.546 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3205999999999999e-05 2023-11-25T04:23:42.546 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.547 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.547 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:42.547 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044497+0000", 2023-11-25T04:23:42.547 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3476000000000003e-05 2023-11-25T04:23:42.547 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:42.547 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:42.547 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:42.547 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.547 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.547 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:17 3.1d 3:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:42.547 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:42.547 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.470358382999997, 2023-11-25T04:23:42.547 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01127356, 2023-11-25T04:23:42.548 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:42.548 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:42.548 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:42.548 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.548 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:42.548 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:42.548 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:42.548 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.548 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.548 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:42.548 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:42.548 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:42.548 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.549 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.549 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:42.549 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:42.549 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1100000000000001e-07 2023-11-25T04:23:42.549 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.549 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.549 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:42.549 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034112+0000", 2023-11-25T04:23:42.549 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8600000000000003e-06 2023-11-25T04:23:42.549 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.549 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.549 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:42.549 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034120+0000", 2023-11-25T04:23:42.549 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7700000000000001e-06 2023-11-25T04:23:42.549 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.550 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.550 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:42.550 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034127+0000", 2023-11-25T04:23:42.550 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0999999999999998e-06 2023-11-25T04:23:42.550 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.550 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.550 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:42.550 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035601+0000", 2023-11-25T04:23:42.550 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014739410000000001 2023-11-25T04:23:42.550 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.550 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.550 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:42.550 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035685+0000", 2023-11-25T04:23:42.551 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3677999999999994e-05 2023-11-25T04:23:42.551 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.551 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.551 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:42.551 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035827+0000", 2023-11-25T04:23:42.551 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014232599999999999 2023-11-25T04:23:42.551 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.551 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.551 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:42.551 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038725+0000", 2023-11-25T04:23:42.551 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028982779999999998 2023-11-25T04:23:42.551 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.551 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.551 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:42.552 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045323+0000", 2023-11-25T04:23:42.552 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0065980780000000003 2023-11-25T04:23:42.552 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.552 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.552 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:42.552 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045354+0000", 2023-11-25T04:23:42.552 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0463e-05 2023-11-25T04:23:42.552 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.552 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.552 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:42.552 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045381+0000", 2023-11-25T04:23:42.552 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7654999999999999e-05 2023-11-25T04:23:42.552 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:42.553 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:42.553 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:42.553 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.553 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.553 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:22 3.e 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:42.553 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:42.553 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.470331969999997, 2023-11-25T04:23:42.553 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010133422, 2023-11-25T04:23:42.553 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:42.553 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:42.553 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:42.553 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.553 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:42.553 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:42.554 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:42.554 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.554 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.554 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:42.554 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:42.554 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:42.554 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.554 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.554 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:42.554 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034135+0000", 2023-11-25T04:23:42.554 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-07 2023-11-25T04:23:42.554 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.554 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.554 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:42.555 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034138+0000", 2023-11-25T04:23:42.555 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6009999999999999e-06 2023-11-25T04:23:42.555 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.555 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.555 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:42.555 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034145+0000", 2023-11-25T04:23:42.555 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1169999999999998e-06 2023-11-25T04:23:42.555 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.555 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.555 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:42.555 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034150+0000", 2023-11-25T04:23:42.555 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9080000000000003e-06 2023-11-25T04:23:42.555 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.556 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.556 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:42.556 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034209+0000", 2023-11-25T04:23:42.556 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8149000000000002e-05 2023-11-25T04:23:42.556 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.556 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.556 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:42.556 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034415+0000", 2023-11-25T04:23:42.556 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020637699999999999 2023-11-25T04:23:42.556 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.556 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.556 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:42.556 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034584+0000", 2023-11-25T04:23:42.556 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016870399999999999 2023-11-25T04:23:42.557 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.557 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.557 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:42.557 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.036979+0000", 2023-11-25T04:23:42.557 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023957509999999998 2023-11-25T04:23:42.557 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.557 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.557 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:42.557 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044161+0000", 2023-11-25T04:23:42.557 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0071811590000000003 2023-11-25T04:23:42.557 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.557 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.557 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:42.557 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044224+0000", 2023-11-25T04:23:42.558 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3454000000000006e-05 2023-11-25T04:23:42.558 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.558 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.558 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:42.558 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044268+0000", 2023-11-25T04:23:42.558 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3761999999999999e-05 2023-11-25T04:23:42.558 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:42.558 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:42.558 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:42.558 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.558 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.558 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:23 3.e 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:42.558 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:42.559 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.470300561000002, 2023-11-25T04:23:42.559 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010281676, 2023-11-25T04:23:42.559 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:42.559 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:42.559 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:42.559 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.559 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:42.559 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:42.559 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:42.559 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.559 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.559 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:42.559 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:42.560 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:42.560 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.560 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.560 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:42.560 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:42.560 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5799999999999999e-07 2023-11-25T04:23:42.560 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.560 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.560 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:42.560 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034169+0000", 2023-11-25T04:23:42.560 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1879999999999998e-06 2023-11-25T04:23:42.560 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.560 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.561 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:42.561 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034177+0000", 2023-11-25T04:23:42.561 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7020000000000001e-06 2023-11-25T04:23:42.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.561 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:42.561 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034184+0000", 2023-11-25T04:23:42.561 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3910000000000003e-06 2023-11-25T04:23:42.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.561 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:42.561 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034924+0000", 2023-11-25T04:23:42.561 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00074032999999999998 2023-11-25T04:23:42.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.562 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.562 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:42.562 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034991+0000", 2023-11-25T04:23:42.562 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7347000000000004e-05 2023-11-25T04:23:42.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.562 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.562 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:42.562 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035096+0000", 2023-11-25T04:23:42.562 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010488 2023-11-25T04:23:42.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.562 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.562 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:42.562 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037588+0000", 2023-11-25T04:23:42.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024918309999999999 2023-11-25T04:23:42.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.563 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:42.563 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044367+0000", 2023-11-25T04:23:42.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067787589999999997 2023-11-25T04:23:42.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.563 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:42.563 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044415+0000", 2023-11-25T04:23:42.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7803000000000003e-05 2023-11-25T04:23:42.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:42.564 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044447+0000", 2023-11-25T04:23:42.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2886999999999999e-05 2023-11-25T04:23:42.564 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:42.564 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:42.564 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:42.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.564 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:30 3.15 3:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:42.564 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:42.564 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.470242120999998, 2023-11-25T04:23:42.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010365070000000001, 2023-11-25T04:23:42.564 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:42.564 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:42.565 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:42.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.565 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:42.565 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:42.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:42.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.565 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:42.565 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:42.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:42.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.565 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:42.565 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034225+0000", 2023-11-25T04:23:42.566 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7400000000000002e-07 2023-11-25T04:23:42.566 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.566 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.566 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:42.566 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034229+0000", 2023-11-25T04:23:42.566 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3100000000000002e-06 2023-11-25T04:23:42.566 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.566 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.566 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:42.566 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034239+0000", 2023-11-25T04:23:42.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9140000000000003e-06 2023-11-25T04:23:42.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:42.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034246+0000", 2023-11-25T04:23:42.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0149999999999998e-06 2023-11-25T04:23:42.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:42.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035572+0000", 2023-11-25T04:23:42.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001325655 2023-11-25T04:23:42.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:42.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035730+0000", 2023-11-25T04:23:42.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015827899999999999 2023-11-25T04:23:42.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:42.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035857+0000", 2023-11-25T04:23:42.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012644699999999999 2023-11-25T04:23:42.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:42.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038737+0000", 2023-11-25T04:23:42.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002880077 2023-11-25T04:23:42.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:42.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044540+0000", 2023-11-25T04:23:42.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058034239999999997 2023-11-25T04:23:42.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:42.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044569+0000", 2023-11-25T04:23:42.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8388999999999999e-05 2023-11-25T04:23:42.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:42.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044589+0000", 2023-11-25T04:23:42.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0686000000000002e-05 2023-11-25T04:23:42.570 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:42.570 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:42.570 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:42.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.570 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:33 3.1b 3:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:42.570 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:42.570 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.470196623, 2023-11-25T04:23:42.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099754049999999997, 2023-11-25T04:23:42.570 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:42.570 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:23:42.570 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:42.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.570 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:42.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:42.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:42.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:42.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:42.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:42.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:42.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:42.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4799999999999998e-07 2023-11-25T04:23:42.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.572 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.572 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:42.572 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034273+0000", 2023-11-25T04:23:42.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7819999999999998e-06 2023-11-25T04:23:42.572 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.572 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.572 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:42.572 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034285+0000", 2023-11-25T04:23:42.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2058e-05 2023-11-25T04:23:42.572 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.572 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.572 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:42.572 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034291+0000", 2023-11-25T04:23:42.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0680000000000002e-06 2023-11-25T04:23:42.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.573 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:42.573 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034370+0000", 2023-11-25T04:23:42.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8452e-05 2023-11-25T04:23:42.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.573 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:42.573 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034477+0000", 2023-11-25T04:23:42.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107471 2023-11-25T04:23:42.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.573 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:42.573 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044245+0000", 2023-11-25T04:23:42.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0097679259999999997 2023-11-25T04:23:42.574 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:42.574 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:42.574 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:42.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.574 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:36 3.1a 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:42.574 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:42.574 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.47014995, 2023-11-25T04:23:42.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058179099999999999, 2023-11-25T04:23:42.574 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:42.574 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:23:42.575 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:42.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:42.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:42.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:42.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:42.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:42.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:42.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.576 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:42.576 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034317+0000", 2023-11-25T04:23:42.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1999999999999999e-07 2023-11-25T04:23:42.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.576 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:42.576 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034321+0000", 2023-11-25T04:23:42.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.233e-06 2023-11-25T04:23:42.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.576 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:42.576 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034337+0000", 2023-11-25T04:23:42.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5308999999999999e-05 2023-11-25T04:23:42.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.577 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:42.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034354+0000", 2023-11-25T04:23:42.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7411999999999999e-05 2023-11-25T04:23:42.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.577 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:42.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034465+0000", 2023-11-25T04:23:42.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011154699999999999 2023-11-25T04:23:42.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.577 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:42.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034560+0000", 2023-11-25T04:23:42.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4186999999999998e-05 2023-11-25T04:23:42.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.578 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:42.578 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.040134+0000", 2023-11-25T04:23:42.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055746019999999997 2023-11-25T04:23:42.578 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:42.578 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:42.578 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:42.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.578 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:9 3.10 e18/16 3:09401a81:::601.00000000:head v 18'4, mlcod=0'0)", 2023-11-25T04:23:42.578 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:42.578 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.461255641999998, 2023-11-25T04:23:42.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061641020000000003, 2023-11-25T04:23:42.579 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:42.579 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:42.579 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:42.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:42.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:42.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:42.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:42.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:42.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:42.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:42.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:42.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5300000000000001e-07 2023-11-25T04:23:42.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:42.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043222+0000", 2023-11-25T04:23:42.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.082e-05 2023-11-25T04:23:42.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:42.581 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043228+0000", 2023-11-25T04:23:42.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6699999999999997e-06 2023-11-25T04:23:42.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.581 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:42.581 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043236+0000", 2023-11-25T04:23:42.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8189999999999992e-06 2023-11-25T04:23:42.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.581 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:42.581 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043507+0000", 2023-11-25T04:23:42.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000271012 2023-11-25T04:23:42.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:42.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043530+0000", 2023-11-25T04:23:42.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2659000000000001e-05 2023-11-25T04:23:42.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:42.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049321+0000", 2023-11-25T04:23:42.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057915129999999999 2023-11-25T04:23:42.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:42.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049375+0000", 2023-11-25T04:23:42.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3455999999999998e-05 2023-11-25T04:23:42.583 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:42.583 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:42.583 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:42.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.583 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:1 3.14 e18/16 3:292cf221:::200.00000000:head v 18'3, mlcod=0'0)", 2023-11-25T04:23:42.583 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:42.583 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.461218928999998, 2023-11-25T04:23:42.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0060983599999999997, 2023-11-25T04:23:42.583 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:42.583 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:42.583 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:42.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.583 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:42.584 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:42.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:42.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.584 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:42.584 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:42.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:42.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.584 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:42.584 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043248+0000", 2023-11-25T04:23:42.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9399999999999995e-07 2023-11-25T04:23:42.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:42.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043252+0000", 2023-11-25T04:23:42.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1799999999999998e-06 2023-11-25T04:23:42.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:42.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043259+0000", 2023-11-25T04:23:42.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0049999999999998e-06 2023-11-25T04:23:42.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:42.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043265+0000", 2023-11-25T04:23:42.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0859999999999998e-06 2023-11-25T04:23:42.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:42.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043311+0000", 2023-11-25T04:23:42.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6082999999999999e-05 2023-11-25T04:23:42.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:42.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043330+0000", 2023-11-25T04:23:42.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8556e-05 2023-11-25T04:23:42.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:42.587 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049303+0000", 2023-11-25T04:23:42.587 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059733470000000004 2023-11-25T04:23:42.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.587 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:42.587 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049346+0000", 2023-11-25T04:23:42.587 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2608999999999998e-05 2023-11-25T04:23:42.587 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:42.587 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:42.587 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:42.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.587 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:3 3.14 e18/16 3:287034c4:::1.00000000.inode:head v 18'4, mlcod=0'0)", 2023-11-25T04:23:42.587 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:42.588 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.460702423000001, 2023-11-25T04:23:42.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057040090000000003, 2023-11-25T04:23:42.588 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:42.588 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:42.588 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:42.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.588 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:42.588 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:42.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:42.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.588 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:42.588 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:42.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:42.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:42.589 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:42.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9799999999999999e-07 2023-11-25T04:23:42.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:42.589 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043785+0000", 2023-11-25T04:23:42.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0721999999999999e-05 2023-11-25T04:23:42.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:42.589 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043793+0000", 2023-11-25T04:23:42.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5839999999999995e-06 2023-11-25T04:23:42.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:42.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043803+0000", 2023-11-25T04:23:42.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0729999999999994e-06 2023-11-25T04:23:42.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:42.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043843+0000", 2023-11-25T04:23:42.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0747999999999997e-05 2023-11-25T04:23:42.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:42.591 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043856+0000", 2023-11-25T04:23:42.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2697000000000001e-05 2023-11-25T04:23:42.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.591 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:42.591 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049426+0000", 2023-11-25T04:23:42.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055704409999999998 2023-11-25T04:23:42.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.591 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:42.591 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049468+0000", 2023-11-25T04:23:42.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1446000000000003e-05 2023-11-25T04:23:42.591 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:42.591 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:42.592 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:42.592 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.592 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.592 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "pg_update_log_missing(3.14 epoch 18/16 rep_tid 171 entries 18'5 (0'0) error 3:2e2fa760:::200.00000001:head by mds.0.3:32 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:23:42.592 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:42.592 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.460541460000002, 2023-11-25T04:23:42.592 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005556938, 2023-11-25T04:23:42.592 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:42.592 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2023-11-25T04:23:42.592 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:42.592 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.592 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:42.592 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:42.592 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:42.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:42.593 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:42.593 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:42.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:42.593 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:42.593 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.03e-07 2023-11-25T04:23:42.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:42.594 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043931+0000", 2023-11-25T04:23:42.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1079999999999998e-06 2023-11-25T04:23:42.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.594 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:42.594 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043944+0000", 2023-11-25T04:23:42.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2852e-05 2023-11-25T04:23:42.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.594 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:42.594 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043955+0000", 2023-11-25T04:23:42.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1037e-05 2023-11-25T04:23:42.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:42.595 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044088+0000", 2023-11-25T04:23:42.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013269599999999999 2023-11-25T04:23:42.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:42.595 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049482+0000", 2023-11-25T04:23:42.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053940419999999999 2023-11-25T04:23:42.595 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:42.595 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:42.595 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:42.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.595 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:10 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'5, mlcod=0'0)", 2023-11-25T04:23:42.596 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:42.596 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.460451431999999, 2023-11-25T04:23:42.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054493099999999997, 2023-11-25T04:23:42.596 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:42.596 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:42.596 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:42.596 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.596 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:42.596 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:42.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:42.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.596 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.596 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:42.597 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:42.597 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:42.597 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.597 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.597 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:42.597 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:42.597 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6100000000000002e-07 2023-11-25T04:23:42.597 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.597 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.597 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:42.597 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044019+0000", 2023-11-25T04:23:42.597 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.839e-06 2023-11-25T04:23:42.597 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:42.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044026+0000", 2023-11-25T04:23:42.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5869999999999998e-06 2023-11-25T04:23:42.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:42.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044031+0000", 2023-11-25T04:23:42.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4909999999999996e-06 2023-11-25T04:23:42.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:42.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044070+0000", 2023-11-25T04:23:42.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8829999999999999e-05 2023-11-25T04:23:42.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:42.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044081+0000", 2023-11-25T04:23:42.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1447999999999999e-05 2023-11-25T04:23:42.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:42.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049406+0000", 2023-11-25T04:23:42.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053246480000000004 2023-11-25T04:23:42.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:42.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049464+0000", 2023-11-25T04:23:42.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8205999999999998e-05 2023-11-25T04:23:42.600 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:42.600 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:42.600 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:42.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.600 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:11 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'6, mlcod=0'0)", 2023-11-25T04:23:42.600 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:42.600 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 55.460250168000002, 2023-11-25T04:23:42.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052673499999999996, 2023-11-25T04:23:42.600 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:42.600 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:42.600 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:42.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:42.601 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:42.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:42.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:42.601 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:42.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:42.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:42.601 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:42.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0199999999999998e-07 2023-11-25T04:23:42.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:42.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044220+0000", 2023-11-25T04:23:42.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7720000000000001e-06 2023-11-25T04:23:42.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:42.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044226+0000", 2023-11-25T04:23:42.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8579999999999996e-06 2023-11-25T04:23:42.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:42.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044232+0000", 2023-11-25T04:23:42.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1700000000000002e-06 2023-11-25T04:23:42.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:42.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044274+0000", 2023-11-25T04:23:42.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1254999999999997e-05 2023-11-25T04:23:42.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:42.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:23:42.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6880999999999999e-05 2023-11-25T04:23:42.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:42.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049466+0000", 2023-11-25T04:23:42.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051659219999999999 2023-11-25T04:23:42.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:42.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:42.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:42.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049483+0000", 2023-11-25T04:23:42.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.719e-05 2023-11-25T04:23:42.604 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:42.604 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:42.604 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:42.604 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:42.604 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:42.604 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:42.606 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:23:42.606 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:42.604+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:42.707 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:42.705+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:42.714 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:42.714 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:23:42.715 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2023-11-25T04:23:42.715 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:42.723 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:23:42.807 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:42.806+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:42.869 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:42.869 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:23:42.869 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2023-11-25T04:23:42.869 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2023-11-25T04:23:42.869 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:42.878 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:23:42.919 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:42.917+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:42.961 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2023-11-25T04:23:43.009 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:43.008+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:43.024 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:43.024 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2023-11-25T04:23:43.025 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2023-11-25T04:23:43.025 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2023-11-25T04:23:43.025 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.025 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:43.025 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:43.025 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 55.990676321999999, 2023-11-25T04:23:43.025 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01072943, 2023-11-25T04:23:43.025 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:43.025 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:43.025 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:43.025 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.025 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:43.026 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:43.026 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:43.026 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.026 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.026 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:43.026 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:43.026 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:43.026 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.026 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.026 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:43.026 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031829+0000", 2023-11-25T04:23:43.026 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.666e-06 2023-11-25T04:23:43.027 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.027 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.027 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:43.027 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031839+0000", 2023-11-25T04:23:43.027 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9490000000000008e-06 2023-11-25T04:23:43.027 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.027 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.027 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:43.027 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031857+0000", 2023-11-25T04:23:43.027 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8093000000000001e-05 2023-11-25T04:23:43.027 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.027 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.027 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:43.028 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031882+0000", 2023-11-25T04:23:43.028 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4294000000000002e-05 2023-11-25T04:23:43.028 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.028 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.028 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:43.028 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033130+0000", 2023-11-25T04:23:43.028 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001248264 2023-11-25T04:23:43.028 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.028 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.028 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:43.028 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033426+0000", 2023-11-25T04:23:43.028 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029620999999999998 2023-11-25T04:23:43.028 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.029 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.029 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:43.029 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033814+0000", 2023-11-25T04:23:43.029 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000387742 2023-11-25T04:23:43.029 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.029 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.029 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:43.029 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036793+0000", 2023-11-25T04:23:43.029 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002979598 2023-11-25T04:23:43.029 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.029 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.029 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:43.030 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042455+0000", 2023-11-25T04:23:43.030 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0056619260000000003 2023-11-25T04:23:43.030 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.030 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.030 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:43.030 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042502+0000", 2023-11-25T04:23:43.030 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7139999999999999e-05 2023-11-25T04:23:43.030 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.030 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.030 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:43.030 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042557+0000", 2023-11-25T04:23:43.031 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4548e-05 2023-11-25T04:23:43.031 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:43.031 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:43.031 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:43.031 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.031 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.031 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:43.031 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:43.031 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 55.989477786999998, 2023-11-25T04:23:43.031 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011505202000000001, 2023-11-25T04:23:43.031 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:43.031 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:43.031 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:43.032 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.032 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:43.032 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:43.032 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:43.032 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.032 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.032 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:43.032 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:43.032 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:43.032 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.032 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.032 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:43.032 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033027+0000", 2023-11-25T04:23:43.033 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0699999999999999e-06 2023-11-25T04:23:43.033 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.033 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.033 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:43.033 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033035+0000", 2023-11-25T04:23:43.034 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0830000000000004e-06 2023-11-25T04:23:43.034 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.034 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.034 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:43.034 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033059+0000", 2023-11-25T04:23:43.034 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3975e-05 2023-11-25T04:23:43.034 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.034 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.034 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:43.034 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033482+0000", 2023-11-25T04:23:43.034 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042253700000000001 2023-11-25T04:23:43.035 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.035 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.035 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:43.035 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034520+0000", 2023-11-25T04:23:43.035 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001037991 2023-11-25T04:23:43.035 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.036 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.036 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:43.036 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034611+0000", 2023-11-25T04:23:43.036 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1113999999999995e-05 2023-11-25T04:23:43.036 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.037 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.037 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:43.037 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034769+0000", 2023-11-25T04:23:43.037 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000157641 2023-11-25T04:23:43.037 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.037 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.037 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:43.037 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038292+0000", 2023-11-25T04:23:43.037 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035231160000000002 2023-11-25T04:23:43.037 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.037 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.038 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:43.038 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044440+0000", 2023-11-25T04:23:43.038 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0061486980000000002 2023-11-25T04:23:43.038 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.038 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.038 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:43.038 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044495+0000", 2023-11-25T04:23:43.038 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4339000000000003e-05 2023-11-25T04:23:43.038 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.038 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.038 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:43.038 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044531+0000", 2023-11-25T04:23:43.038 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6637999999999997e-05 2023-11-25T04:23:43.039 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:43.039 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:43.039 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:43.039 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.039 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.039 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:43.039 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:43.039 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 55.989003433000001, 2023-11-25T04:23:43.039 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0057634039999999997, 2023-11-25T04:23:43.039 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:43.039 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:43.041 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:43.041 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.041 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:43.041 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:43.041 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:43.041 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.041 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.041 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:43.041 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:43.041 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:43.042 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.042 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.042 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:43.042 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:43.042 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7900000000000007e-07 2023-11-25T04:23:43.042 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.042 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.042 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:43.042 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033507+0000", 2023-11-25T04:23:43.042 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8900000000000004e-06 2023-11-25T04:23:43.042 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.042 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.042 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:43.043 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033519+0000", 2023-11-25T04:23:43.043 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1756000000000001e-05 2023-11-25T04:23:43.043 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.043 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.043 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:43.043 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033527+0000", 2023-11-25T04:23:43.043 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3610000000000001e-06 2023-11-25T04:23:43.043 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.043 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.043 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:43.043 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034451+0000", 2023-11-25T04:23:43.043 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00092372400000000001 2023-11-25T04:23:43.044 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.044 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.044 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:43.044 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034809+0000", 2023-11-25T04:23:43.044 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00035757699999999998 2023-11-25T04:23:43.044 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.044 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.044 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:23:43.044 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034965+0000", 2023-11-25T04:23:43.044 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015638699999999999 2023-11-25T04:23:43.044 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.044 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.044 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:43.045 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038299+0000", 2023-11-25T04:23:43.045 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0033336559999999999 2023-11-25T04:23:43.045 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.045 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.045 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:43.045 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039169+0000", 2023-11-25T04:23:43.045 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00087066400000000005 2023-11-25T04:23:43.045 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.046 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.046 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:43.046 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039226+0000", 2023-11-25T04:23:43.046 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6511999999999997e-05 2023-11-25T04:23:43.046 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.046 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.046 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:43.046 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039264+0000", 2023-11-25T04:23:43.046 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7898000000000002e-05 2023-11-25T04:23:43.046 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:43.046 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:43.046 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:43.046 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.047 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.047 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:25 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:43.047 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:43.047 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 55.988924849, 2023-11-25T04:23:43.047 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096656169999999996, 2023-11-25T04:23:43.047 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:43.047 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:43.047 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:43.047 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.047 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:43.047 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:43.047 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:43.047 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.048 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.048 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:43.048 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:43.048 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:43.048 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.048 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.048 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:43.048 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033580+0000", 2023-11-25T04:23:43.048 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8199999999999999e-07 2023-11-25T04:23:43.048 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.048 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.048 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:43.049 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033585+0000", 2023-11-25T04:23:43.049 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3680000000000001e-06 2023-11-25T04:23:43.049 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.049 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.049 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:43.049 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033600+0000", 2023-11-25T04:23:43.049 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4331000000000001e-05 2023-11-25T04:23:43.049 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.049 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.049 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:43.049 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033606+0000", 2023-11-25T04:23:43.049 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5780000000000004e-06 2023-11-25T04:23:43.049 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.050 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.050 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:43.050 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035747+0000", 2023-11-25T04:23:43.050 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0021406530000000002 2023-11-25T04:23:43.050 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.050 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.050 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:43.050 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035901+0000", 2023-11-25T04:23:43.050 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000153931 2023-11-25T04:23:43.050 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.050 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.050 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:23:43.051 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036193+0000", 2023-11-25T04:23:43.051 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029211299999999999 2023-11-25T04:23:43.051 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.051 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.051 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:43.051 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038655+0000", 2023-11-25T04:23:43.051 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0024621640000000002 2023-11-25T04:23:43.051 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.051 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.051 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:43.051 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043130+0000", 2023-11-25T04:23:43.051 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.004474698 2023-11-25T04:23:43.051 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.052 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.052 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:43.052 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043204+0000", 2023-11-25T04:23:43.052 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4091999999999995e-05 2023-11-25T04:23:43.052 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.052 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.052 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:43.052 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043245+0000", 2023-11-25T04:23:43.052 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0907e-05 2023-11-25T04:23:43.052 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:43.052 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:43.052 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:43.053 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.053 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.053 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:26 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:43.053 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:43.053 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 55.988889024999999, 2023-11-25T04:23:43.053 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011086945000000001, 2023-11-25T04:23:43.053 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:43.053 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:43.053 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:43.053 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.053 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:43.053 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:43.053 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:43.054 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.054 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.054 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:43.054 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:43.054 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:43.054 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.054 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.054 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:43.054 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:43.054 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1699999999999998e-07 2023-11-25T04:23:43.054 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.054 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.055 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:43.055 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033627+0000", 2023-11-25T04:23:43.055 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.163e-05 2023-11-25T04:23:43.055 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.055 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.055 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:43.055 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033636+0000", 2023-11-25T04:23:43.055 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6859999999999995e-06 2023-11-25T04:23:43.056 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.056 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.056 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:43.056 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033642+0000", 2023-11-25T04:23:43.056 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3149999999999997e-06 2023-11-25T04:23:43.056 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.056 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.056 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:43.056 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035414+0000", 2023-11-25T04:23:43.056 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0017716380000000001 2023-11-25T04:23:43.056 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.056 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.057 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:43.057 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035539+0000", 2023-11-25T04:23:43.057 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000125477 2023-11-25T04:23:43.057 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.057 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.057 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:43.057 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035672+0000", 2023-11-25T04:23:43.057 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000133316 2023-11-25T04:23:43.057 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.057 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.057 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:43.057 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038649+0000", 2023-11-25T04:23:43.058 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029764679999999999 2023-11-25T04:23:43.058 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.058 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.058 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:43.058 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044578+0000", 2023-11-25T04:23:43.058 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0059293510000000002 2023-11-25T04:23:43.058 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.058 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.058 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:43.058 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044651+0000", 2023-11-25T04:23:43.058 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2513000000000002e-05 2023-11-25T04:23:43.058 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.058 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.059 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:43.059 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044702+0000", 2023-11-25T04:23:43.059 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1034e-05 2023-11-25T04:23:43.059 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:43.059 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:43.059 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:43.059 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.059 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.059 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:29 3.b 3:d0630e4c:::mds0_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:43.059 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:43.059 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 55.988851105000002, 2023-11-25T04:23:43.060 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010980227, 2023-11-25T04:23:43.060 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:43.060 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:43.060 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:43.060 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.060 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:43.060 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:43.060 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:43.060 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.060 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.060 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:43.060 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:43.060 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:43.061 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.061 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.061 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:43.061 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:43.061 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3499999999999996e-07 2023-11-25T04:23:43.061 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.061 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.061 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:43.061 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033658+0000", 2023-11-25T04:23:43.061 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1280000000000001e-06 2023-11-25T04:23:43.061 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.061 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.061 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:43.062 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033667+0000", 2023-11-25T04:23:43.062 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1430000000000004e-06 2023-11-25T04:23:43.062 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.062 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.062 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:43.062 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033678+0000", 2023-11-25T04:23:43.062 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0906e-05 2023-11-25T04:23:43.062 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.062 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.062 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:43.062 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034456+0000", 2023-11-25T04:23:43.062 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00077833899999999996 2023-11-25T04:23:43.063 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.063 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.063 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:43.063 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034575+0000", 2023-11-25T04:23:43.063 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011926000000000001 2023-11-25T04:23:43.063 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.063 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.063 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:43.063 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035001+0000", 2023-11-25T04:23:43.063 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042550999999999998 2023-11-25T04:23:43.063 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.063 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.063 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:43.064 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.037923+0000", 2023-11-25T04:23:43.064 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029219670000000001 2023-11-25T04:23:43.064 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.064 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.064 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:43.064 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044538+0000", 2023-11-25T04:23:43.064 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0066152349999999997 2023-11-25T04:23:43.064 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.064 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.064 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:43.064 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044594+0000", 2023-11-25T04:23:43.065 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5593000000000001e-05 2023-11-25T04:23:43.065 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.065 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.065 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:43.065 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044633+0000", 2023-11-25T04:23:43.065 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9511000000000002e-05 2023-11-25T04:23:43.065 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:43.065 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:43.065 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:43.065 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.065 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.066 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:34 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:43.066 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:43.066 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 55.988600992000002, 2023-11-25T04:23:43.066 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010920445000000001, 2023-11-25T04:23:43.066 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:43.066 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:43.066 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:43.066 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.066 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:43.066 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:43.066 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:43.066 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.066 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.067 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:43.067 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:43.067 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:43.067 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.067 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.067 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:43.067 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033904+0000", 2023-11-25T04:23:43.067 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6599999999999994e-07 2023-11-25T04:23:43.067 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.067 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.067 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:43.067 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033909+0000", 2023-11-25T04:23:43.067 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5639999999999996e-06 2023-11-25T04:23:43.068 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.068 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.068 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:43.068 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033923+0000", 2023-11-25T04:23:43.068 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3028e-05 2023-11-25T04:23:43.068 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.068 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.068 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:43.068 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034043+0000", 2023-11-25T04:23:43.068 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000120328 2023-11-25T04:23:43.068 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.068 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.069 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:43.069 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035576+0000", 2023-11-25T04:23:43.069 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015333720000000001 2023-11-25T04:23:43.069 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.069 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.069 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:43.069 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035699+0000", 2023-11-25T04:23:43.069 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000123001 2023-11-25T04:23:43.069 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.069 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.069 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:43.069 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044823+0000", 2023-11-25T04:23:43.070 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0091241859999999994 2023-11-25T04:23:43.070 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:43.070 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:43.070 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:43.070 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.070 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.070 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:35 3.1f 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:43.070 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:43.070 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 55.987296786999998, 2023-11-25T04:23:43.070 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096164779999999995, 2023-11-25T04:23:43.070 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:43.070 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:43.071 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:43.071 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.071 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:43.071 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:43.071 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:43.071 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.071 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.071 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:43.071 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:43.071 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:43.071 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.071 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.072 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:43.072 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035208+0000", 2023-11-25T04:23:43.072 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2699999999999998e-07 2023-11-25T04:23:43.072 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.072 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.072 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:43.072 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035213+0000", 2023-11-25T04:23:43.072 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.6070000000000001e-06 2023-11-25T04:23:43.072 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.072 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.072 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:43.072 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035350+0000", 2023-11-25T04:23:43.073 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013693599999999999 2023-11-25T04:23:43.073 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.073 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.073 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:43.073 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035358+0000", 2023-11-25T04:23:43.073 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0099999999999995e-06 2023-11-25T04:23:43.073 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.073 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.073 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:43.073 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036260+0000", 2023-11-25T04:23:43.073 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00090240100000000005 2023-11-25T04:23:43.073 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.073 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.074 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:43.074 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036517+0000", 2023-11-25T04:23:43.074 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00025736299999999998 2023-11-25T04:23:43.074 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.074 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.074 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:43.074 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044824+0000", 2023-11-25T04:23:43.074 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0083063340000000003 2023-11-25T04:23:43.074 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:43.074 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:43.074 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:43.074 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.074 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.075 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:37 3.1e 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:43.075 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:43.075 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 55.987130100999998, 2023-11-25T04:23:43.075 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0078322640000000002, 2023-11-25T04:23:43.075 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:43.075 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:43.075 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:43.075 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.075 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:43.075 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:43.075 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:43.076 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.076 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.076 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:43.076 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:43.076 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:43.076 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.076 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.076 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:43.076 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035375+0000", 2023-11-25T04:23:43.076 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9100000000000003e-07 2023-11-25T04:23:43.076 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.076 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.077 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:43.077 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035379+0000", 2023-11-25T04:23:43.077 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9430000000000002e-06 2023-11-25T04:23:43.077 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.077 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.077 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:43.077 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035388+0000", 2023-11-25T04:23:43.077 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2760000000000006e-06 2023-11-25T04:23:43.077 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.077 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.077 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:43.077 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035396+0000", 2023-11-25T04:23:43.077 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8480000000000001e-06 2023-11-25T04:23:43.078 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.078 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.078 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:43.078 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035452+0000", 2023-11-25T04:23:43.078 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5831000000000001e-05 2023-11-25T04:23:43.078 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.078 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.078 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:43.078 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035934+0000", 2023-11-25T04:23:43.078 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00048265599999999998 2023-11-25T04:23:43.079 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.079 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.079 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:43.079 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043206+0000", 2023-11-25T04:23:43.079 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.007271919 2023-11-25T04:23:43.079 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:43.079 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:43.079 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:43.079 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.079 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.079 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing(3.1a epoch 18/16 rep_tid 3 entries 18'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:36 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:23:43.079 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:43.080 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 55.985854515, 2023-11-25T04:23:43.080 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002073414, 2023-11-25T04:23:43.080 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:43.080 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:43.080 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:43.080 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.080 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:43.080 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:43.080 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:43.080 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.080 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.081 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:43.081 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:43.081 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:43.081 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.081 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.081 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:43.081 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036650+0000", 2023-11-25T04:23:43.081 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7300000000000002e-07 2023-11-25T04:23:43.081 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.081 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.081 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:43.081 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036659+0000", 2023-11-25T04:23:43.081 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.7499999999999992e-06 2023-11-25T04:23:43.082 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.082 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.082 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:43.082 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036673+0000", 2023-11-25T04:23:43.082 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4917e-05 2023-11-25T04:23:43.082 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.082 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.082 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:43.083 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036687+0000", 2023-11-25T04:23:43.083 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3883000000000001e-05 2023-11-25T04:23:43.083 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.083 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.083 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:43.083 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036781+0000", 2023-11-25T04:23:43.083 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3219000000000003e-05 2023-11-25T04:23:43.083 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.083 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.083 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:43.083 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038723+0000", 2023-11-25T04:23:43.083 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019422720000000001 2023-11-25T04:23:43.083 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:43.084 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:43.084 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:43.084 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.084 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.084 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:28 3.13 e18/16 3:cd5a64a3:::100.00000000:head v 18'1, mlcod=0'0)", 2023-11-25T04:23:43.084 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:43.084 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 55.982193357, 2023-11-25T04:23:43.084 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001850546, 2023-11-25T04:23:43.084 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:43.084 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:43.084 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:43.084 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.085 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:43.085 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:43.085 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:43.085 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.085 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.085 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:43.085 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:43.085 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:43.085 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.085 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.085 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:43.085 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:43.086 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7800000000000002e-07 2023-11-25T04:23:43.086 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.086 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.086 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:43.086 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040331+0000", 2023-11-25T04:23:43.086 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0449999999999999e-05 2023-11-25T04:23:43.086 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.086 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.086 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:43.086 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040339+0000", 2023-11-25T04:23:43.086 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0709999999999995e-06 2023-11-25T04:23:43.086 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.087 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.087 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:43.087 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040347+0000", 2023-11-25T04:23:43.087 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7079999999999997e-06 2023-11-25T04:23:43.087 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.087 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.087 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:43.087 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040410+0000", 2023-11-25T04:23:43.087 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2453000000000007e-05 2023-11-25T04:23:43.087 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.087 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.087 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:43.088 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040422+0000", 2023-11-25T04:23:43.088 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2429000000000001e-05 2023-11-25T04:23:43.088 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.088 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.088 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:43.088 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042129+0000", 2023-11-25T04:23:43.088 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001707266 2023-11-25T04:23:43.088 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.088 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.088 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:43.088 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042161+0000", 2023-11-25T04:23:43.088 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1791000000000001e-05 2023-11-25T04:23:43.089 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:43.089 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:43.089 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:43.089 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.089 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.089 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:27 3.13 e18/16 3:c91d4a1d:::mds0_inotable:head v 18'2, mlcod=0'0)", 2023-11-25T04:23:43.089 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:43.089 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 55.981989833999997, 2023-11-25T04:23:43.089 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0016886430000000001, 2023-11-25T04:23:43.089 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:43.089 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:43.089 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:43.089 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.090 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:43.090 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:43.090 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:43.090 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.090 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.090 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:43.090 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:43.090 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:43.090 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.090 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.090 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:43.090 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:43.091 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3999999999999997e-07 2023-11-25T04:23:43.091 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.091 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.091 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:43.091 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040520+0000", 2023-11-25T04:23:43.091 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2989999999999996e-06 2023-11-25T04:23:43.091 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.091 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.091 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:43.091 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040527+0000", 2023-11-25T04:23:43.091 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5730000000000002e-06 2023-11-25T04:23:43.092 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.092 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.092 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:43.092 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040535+0000", 2023-11-25T04:23:43.092 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8630000000000008e-06 2023-11-25T04:23:43.092 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.092 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.092 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:43.092 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040683+0000", 2023-11-25T04:23:43.092 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000148002 2023-11-25T04:23:43.092 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.092 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.092 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:43.093 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040700+0000", 2023-11-25T04:23:43.093 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7065e-05 2023-11-25T04:23:43.093 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.093 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.093 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:43.093 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042163+0000", 2023-11-25T04:23:43.093 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001462767 2023-11-25T04:23:43.093 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.093 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.093 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:43.093 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042203+0000", 2023-11-25T04:23:43.093 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9734000000000003e-05 2023-11-25T04:23:43.094 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:43.094 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:43.094 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:43.094 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.094 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.094 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:20 3.1c e18/16 3:3ed09add:::607.00000000:head v 18'1, mlcod=0'0)", 2023-11-25T04:23:43.094 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:43.094 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 55.981954793, 2023-11-25T04:23:43.094 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015271, 2023-11-25T04:23:43.094 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:43.094 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:43.094 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:43.095 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.095 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:43.095 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:43.095 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:43.095 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.095 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.095 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:43.095 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:43.095 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:43.095 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.095 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.095 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:43.096 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:43.096 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4799999999999999e-07 2023-11-25T04:23:43.096 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.096 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.096 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:43.096 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040561+0000", 2023-11-25T04:23:43.096 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1800000000000001e-05 2023-11-25T04:23:43.096 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.096 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.096 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:43.096 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040568+0000", 2023-11-25T04:23:43.096 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7730000000000002e-06 2023-11-25T04:23:43.097 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.097 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.097 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:43.097 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040587+0000", 2023-11-25T04:23:43.097 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8691000000000001e-05 2023-11-25T04:23:43.097 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.097 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.097 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:43.097 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040638+0000", 2023-11-25T04:23:43.097 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1068e-05 2023-11-25T04:23:43.097 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.097 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.098 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:43.098 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040670+0000", 2023-11-25T04:23:43.098 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2125999999999997e-05 2023-11-25T04:23:43.098 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.098 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.098 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:43.098 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042032+0000", 2023-11-25T04:23:43.098 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001361821 2023-11-25T04:23:43.098 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.098 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.098 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:43.098 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042076+0000", 2023-11-25T04:23:43.099 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4472999999999999e-05 2023-11-25T04:23:43.099 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:43.099 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:43.099 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:43.099 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.099 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.099 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:21 3.1c e18/16 3:3ed09add:::607.00000000:head v 18'2, mlcod=0'0)", 2023-11-25T04:23:43.099 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:43.099 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 55.981720068000001, 2023-11-25T04:23:43.099 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0013228160000000001, 2023-11-25T04:23:43.099 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:43.099 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:43.100 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:43.100 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.100 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:43.100 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:43.100 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:43.100 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.100 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.100 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:43.100 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:43.100 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:43.100 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.100 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.101 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:43.101 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:43.101 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9299999999999999e-07 2023-11-25T04:23:43.101 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.101 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.101 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:43.101 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040790+0000", 2023-11-25T04:23:43.101 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5770000000000001e-06 2023-11-25T04:23:43.101 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.101 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.101 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:43.101 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040799+0000", 2023-11-25T04:23:43.101 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6439999999999995e-06 2023-11-25T04:23:43.102 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.102 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.102 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:43.102 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040806+0000", 2023-11-25T04:23:43.102 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0180000000000004e-06 2023-11-25T04:23:43.102 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.102 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.102 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:43.102 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040907+0000", 2023-11-25T04:23:43.102 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000100096 2023-11-25T04:23:43.102 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.102 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.103 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:43.103 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040925+0000", 2023-11-25T04:23:43.103 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8536999999999999e-05 2023-11-25T04:23:43.103 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.103 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.103 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:43.103 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042079+0000", 2023-11-25T04:23:43.103 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00115358 2023-11-25T04:23:43.103 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.103 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.103 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:43.103 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042107+0000", 2023-11-25T04:23:43.104 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7971e-05 2023-11-25T04:23:43.104 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:43.104 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:43.104 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:43.104 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.104 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.104 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop_reply(mds.0.3:4 3.1f e18/16 ondisk, result = 0)", 2023-11-25T04:23:43.104 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:43.104 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 55.980380678000003, 2023-11-25T04:23:43.104 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00044092900000000001, 2023-11-25T04:23:43.104 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:43.104 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:43.105 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:43.105 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.105 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:43.105 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:43.105 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:43.105 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.105 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.105 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:43.105 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:43.105 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:43.105 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.105 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.105 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:43.106 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042124+0000", 2023-11-25T04:23:43.106 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1800000000000005e-07 2023-11-25T04:23:43.106 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.106 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.106 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:43.106 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042128+0000", 2023-11-25T04:23:43.106 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2289999999999997e-06 2023-11-25T04:23:43.106 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.106 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.106 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:43.106 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042138+0000", 2023-11-25T04:23:43.107 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9729999999999992e-06 2023-11-25T04:23:43.107 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.107 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.107 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:43.107 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042145+0000", 2023-11-25T04:23:43.107 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.336e-06 2023-11-25T04:23:43.107 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.107 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.107 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:43.107 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042441+0000", 2023-11-25T04:23:43.107 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029650500000000002 2023-11-25T04:23:43.107 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.107 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.108 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:43.108 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042448+0000", 2023-11-25T04:23:43.108 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4100000000000002e-06 2023-11-25T04:23:43.108 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.108 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.108 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:43.108 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042564+0000", 2023-11-25T04:23:43.108 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000115758 2023-11-25T04:23:43.108 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:43.108 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:43.108 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:43.108 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.109 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.109 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing_reply(3.1f epoch 18/16 rep_tid 8 lcod 18'4)", 2023-11-25T04:23:43.109 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:43.109 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 55.978055265999998, 2023-11-25T04:23:43.109 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00038154699999999998, 2023-11-25T04:23:43.109 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:43.109 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:43.109 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:43.109 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.109 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:43.109 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:43.109 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:43.110 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.110 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.110 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:43.110 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:43.110 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:43.110 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.110 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.110 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:43.110 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:43.110 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2700000000000001e-07 2023-11-25T04:23:43.110 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.110 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.111 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:43.111 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044451+0000", 2023-11-25T04:23:43.111 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7460000000000001e-06 2023-11-25T04:23:43.111 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.111 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.111 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:43.111 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044455+0000", 2023-11-25T04:23:43.111 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5809999999999998e-06 2023-11-25T04:23:43.111 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.111 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.111 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:43.111 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044458+0000", 2023-11-25T04:23:43.112 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0170000000000001e-06 2023-11-25T04:23:43.112 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.112 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.112 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:43.112 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044734+0000", 2023-11-25T04:23:43.112 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00027582799999999999 2023-11-25T04:23:43.112 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.112 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.112 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:43.112 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044830+0000", 2023-11-25T04:23:43.112 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6248000000000004e-05 2023-11-25T04:23:43.112 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:43.113 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:43.113 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:43.113 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.113 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.113 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing_reply(3.b epoch 18/16 rep_tid 5 lcod 18'2)", 2023-11-25T04:23:43.113 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:43.113 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 55.978018605000003, 2023-11-25T04:23:43.113 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000344488, 2023-11-25T04:23:43.113 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:43.113 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:43.113 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:43.113 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.114 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:43.114 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:43.114 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:43.114 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.114 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.114 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:43.114 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:43.114 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:43.114 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.114 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.114 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:43.114 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:43.115 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.55e-07 2023-11-25T04:23:43.115 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.115 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.115 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:43.115 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044487+0000", 2023-11-25T04:23:43.115 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2470000000000001e-06 2023-11-25T04:23:43.115 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.115 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.115 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:43.115 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044491+0000", 2023-11-25T04:23:43.115 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9659999999999998e-06 2023-11-25T04:23:43.116 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.116 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.116 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:43.116 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044494+0000", 2023-11-25T04:23:43.116 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2310000000000001e-06 2023-11-25T04:23:43.116 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.116 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.116 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:43.116 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044756+0000", 2023-11-25T04:23:43.116 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00026172800000000003 2023-11-25T04:23:43.116 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.116 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.117 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:43.117 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044830+0000", 2023-11-25T04:23:43.117 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4160999999999997e-05 2023-11-25T04:23:43.117 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:43.117 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:43.117 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:43.117 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.117 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.117 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:39 3.6 3:654134d2:::mds0_openfiles.0:head [omap-clear,omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2023-11-25T04:23:43.117 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:43.117 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 50.918192562999998, 2023-11-25T04:23:43.117 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001788658, 2023-11-25T04:23:43.117 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:43.118 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:43.118 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:43.118 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.118 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:43.118 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:43.118 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:43.118 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.118 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.118 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:43.118 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:43.118 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:43.118 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.119 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.119 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:43.119 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104314+0000", 2023-11-25T04:23:43.119 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3099999999999999e-06 2023-11-25T04:23:43.119 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.119 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.119 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:43.119 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104325+0000", 2023-11-25T04:23:43.119 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1511999999999999e-05 2023-11-25T04:23:43.119 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.119 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.119 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:43.120 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104362+0000", 2023-11-25T04:23:43.120 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6903000000000002e-05 2023-11-25T04:23:43.120 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.120 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.120 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:43.120 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104387+0000", 2023-11-25T04:23:43.120 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4766e-05 2023-11-25T04:23:43.120 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.120 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.120 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:43.120 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104465+0000", 2023-11-25T04:23:43.120 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8119000000000002e-05 2023-11-25T04:23:43.121 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.121 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.121 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:43.121 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104557+0000", 2023-11-25T04:23:43.121 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2496000000000003e-05 2023-11-25T04:23:43.121 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.121 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.121 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:43.121 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104646+0000", 2023-11-25T04:23:43.121 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8818000000000002e-05 2023-11-25T04:23:43.121 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.122 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.122 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:43.122 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105201+0000", 2023-11-25T04:23:43.122 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00055506000000000002 2023-11-25T04:23:43.122 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.122 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.122 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:43.122 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105994+0000", 2023-11-25T04:23:43.122 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00079294700000000001 2023-11-25T04:23:43.122 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.122 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.122 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:43.123 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.106055+0000", 2023-11-25T04:23:43.123 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0566000000000003e-05 2023-11-25T04:23:43.123 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.123 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.123 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:43.123 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.106100+0000", 2023-11-25T04:23:43.123 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5160999999999997e-05 2023-11-25T04:23:43.123 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:43.123 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:43.123 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:43.123 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.123 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.124 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:43 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2023-11-25T04:23:43.124 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:43.124 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 35.917967566999998, 2023-11-25T04:23:43.124 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001751166, 2023-11-25T04:23:43.124 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:43.124 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:43.124 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:43.124 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.124 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:43.124 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:43.124 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:43.124 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.125 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.125 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:43.125 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:43.125 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:43.125 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.125 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.125 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:43.125 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104539+0000", 2023-11-25T04:23:43.125 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9529999999999999e-06 2023-11-25T04:23:43.125 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.125 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.125 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:43.126 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104557+0000", 2023-11-25T04:23:43.126 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7464000000000001e-05 2023-11-25T04:23:43.126 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.126 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.126 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:43.126 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104590+0000", 2023-11-25T04:23:43.126 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2966000000000003e-05 2023-11-25T04:23:43.126 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.126 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.126 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:43.126 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104608+0000", 2023-11-25T04:23:43.126 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8193e-05 2023-11-25T04:23:43.127 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.127 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.127 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:43.127 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104767+0000", 2023-11-25T04:23:43.127 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015882 2023-11-25T04:23:43.127 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.127 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.127 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:43.127 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104909+0000", 2023-11-25T04:23:43.127 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014244999999999999 2023-11-25T04:23:43.127 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.127 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.128 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:43.128 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.105059+0000", 2023-11-25T04:23:43.128 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014964399999999999 2023-11-25T04:23:43.128 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.128 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.128 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:43.128 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.105701+0000", 2023-11-25T04:23:43.128 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00064201099999999995 2023-11-25T04:23:43.128 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.128 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.128 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:43.128 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106233+0000", 2023-11-25T04:23:43.128 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00053178500000000001 2023-11-25T04:23:43.129 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.129 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.129 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:43.129 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106265+0000", 2023-11-25T04:23:43.129 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1986000000000002e-05 2023-11-25T04:23:43.129 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.129 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.129 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:43.129 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106288+0000", 2023-11-25T04:23:43.129 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2894000000000001e-05 2023-11-25T04:23:43.129 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:43.129 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:43.129 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:43.130 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.130 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.130 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:45 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e23)", 2023-11-25T04:23:43.130 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:23:43.130 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 30.917769527000001, 2023-11-25T04:23:43.130 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00098423299999999994, 2023-11-25T04:23:43.130 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:43.130 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:43.130 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:43.130 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.130 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:43.130 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:23:43.130 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:43.131 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.131 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.131 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:43.131 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:23:43.131 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:43.131 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.131 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.131 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:43.131 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104737+0000", 2023-11-25T04:23:43.131 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3309999999999998e-06 2023-11-25T04:23:43.131 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.131 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.131 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:43.132 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104748+0000", 2023-11-25T04:23:43.132 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1080999999999999e-05 2023-11-25T04:23:43.132 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.132 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.132 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:43.132 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104771+0000", 2023-11-25T04:23:43.132 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2707000000000001e-05 2023-11-25T04:23:43.132 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.132 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.132 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:43.132 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104800+0000", 2023-11-25T04:23:43.132 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9595e-05 2023-11-25T04:23:43.132 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.133 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.133 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:43.133 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104890+0000", 2023-11-25T04:23:43.133 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.9808999999999993e-05 2023-11-25T04:23:43.133 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.133 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.133 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:43.133 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105004+0000", 2023-11-25T04:23:43.133 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000114508 2023-11-25T04:23:43.133 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.133 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.133 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:43.133 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105634+0000", 2023-11-25T04:23:43.133 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00062985799999999996 2023-11-25T04:23:43.134 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.134 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.134 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:43.134 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105671+0000", 2023-11-25T04:23:43.134 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6640000000000002e-05 2023-11-25T04:23:43.134 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:43.134 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:43.134 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:43.134 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105719+0000", 2023-11-25T04:23:43.134 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7704000000000003e-05 2023-11-25T04:23:43.134 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:43.134 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:43.134 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:43.135 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:43.135 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:43.135 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:43.136 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:23:43.137 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:43.135+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:43.237 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:43.235+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:43.245 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:43.245 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:23:43.245 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2023-11-25T04:23:43.245 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:43.252 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:23:43.312 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-11-25T04:23:43.337 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:43.336+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:43.358 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:43.358 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:23:43.358 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2023-11-25T04:23:43.358 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2023-11-25T04:23:43.358 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:43.367 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:23:43.436 INFO:teuthology.orchestra.run.smithi067.stderr:Error ENXIO: problem getting command descriptions from osd.0 2023-11-25T04:23:43.439 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:43.437+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:43.454 INFO:teuthology.orchestra.run.smithi067.stdout:osd.1: {} 2023-11-25T04:23:43.454 INFO:teuthology.orchestra.run.smithi067.stderr:osd.1: osd_enable_op_tracker = '' 2023-11-25T04:23:43.472 INFO:teuthology.orchestra.run.smithi067.stdout:osd.2: {} 2023-11-25T04:23:43.473 INFO:teuthology.orchestra.run.smithi067.stderr:osd.2: osd_enable_op_tracker = '' 2023-11-25T04:23:43.475 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-11-25T04:23:43.476 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-11-25T04:23:43.512 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:43.512 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:43.521 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:23:43.565 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:43.564+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:43.629 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:43.629 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:43.638 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:23:43.640 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:43.639+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:43.741 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:43.740+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:43.784 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:43.784 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:43.793 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:23:43.842 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:43.840+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:43.899 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:43.899 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:43.908 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:23:43.948 INFO:teuthology.orchestra.run.smithi067.stderr:nodeep-scrub is set 2023-11-25T04:23:43.951 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:43.950+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:44.014 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:44.014 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:44.023 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:23:44.043 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:44.042+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:44.144 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:44.143+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:44.154 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~35s 2023-11-25T04:23:44.168 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:44.168 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:44.177 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:23:44.261 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:44.260+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:44.322 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:44.323 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:44.331 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:23:44.372 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:44.371+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:44.440 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:44.440 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:44.447 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:44.445+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:44.448 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:23:44.548 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:44.547+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:44.594 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:44.594 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:44.603 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:23:44.649 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:44.648+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:44.709 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:44.709 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:44.718 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:23:44.750 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:44.749+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:44.823 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:44.824 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:44.832 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:23:44.851 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:44.850+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:44.937 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:44.937 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:44.946 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:23:44.953 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:44.951+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:45.051 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:45.051 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:45.053 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:45.051+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:45.059 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:23:45.155 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:45.153+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:45.167 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:45.167 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:45.177 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:23:45.255 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:45.254+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:45.282 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:45.282 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:45.290 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:23:45.375 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:45.373+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:45.436 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:45.436 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:45.445 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:23:45.477 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-11-25T04:23:45.485 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:45.484+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:45.550 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:45.550 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:45.559 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:23:45.559 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:45.557+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:45.598 INFO:teuthology.orchestra.run.smithi067.stderr:Error ENXIO: problem getting command descriptions from osd.0 2023-11-25T04:23:45.616 INFO:teuthology.orchestra.run.smithi067.stdout:osd.1: {} 2023-11-25T04:23:45.617 INFO:teuthology.orchestra.run.smithi067.stderr:osd.1: osd_enable_op_tracker = '' 2023-11-25T04:23:45.635 INFO:teuthology.orchestra.run.smithi067.stdout:osd.2: {} 2023-11-25T04:23:45.635 INFO:teuthology.orchestra.run.smithi067.stderr:osd.2: osd_enable_op_tracker = '' 2023-11-25T04:23:45.638 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-11-25T04:23:45.638 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-11-25T04:23:45.660 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:45.658+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:45.703 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:45.703 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:23:45.704 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2023-11-25T04:23:45.704 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2023-11-25T04:23:45.704 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:45.713 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:23:45.761 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:45.760+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:45.821 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:45.821 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2023-11-25T04:23:45.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2023-11-25T04:23:45.821 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2023-11-25T04:23:45.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.822 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:155 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'109, mlcod=11'105)", 2023-11-25T04:23:45.822 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:45.822 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 68.033702020999996, 2023-11-25T04:23:45.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061719019999999999, 2023-11-25T04:23:45.822 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:45.822 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:45.822 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:45.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.823 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:45.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:45.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:45.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.823 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:45.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:45.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:45.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:45.824 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784884+0000", 2023-11-25T04:23:45.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3099999999999999e-07 2023-11-25T04:23:45.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:45.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784942+0000", 2023-11-25T04:23:45.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8570000000000003e-05 2023-11-25T04:23:45.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:45.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784949+0000", 2023-11-25T04:23:45.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0310000000000001e-06 2023-11-25T04:23:45.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:45.826 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784953+0000", 2023-11-25T04:23:45.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6320000000000001e-06 2023-11-25T04:23:45.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:45.826 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784993+0000", 2023-11-25T04:23:45.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9737000000000003e-05 2023-11-25T04:23:45.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:45.826 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785011+0000", 2023-11-25T04:23:45.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8349999999999999e-05 2023-11-25T04:23:45.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:45.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791024+0000", 2023-11-25T04:23:45.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006012997 2023-11-25T04:23:45.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:45.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791055+0000", 2023-11-25T04:23:45.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1254000000000003e-05 2023-11-25T04:23:45.827 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:45.827 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:45.827 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:45.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.828 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:156 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'110, mlcod=11'105)", 2023-11-25T04:23:45.828 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:45.828 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 68.033115972000004, 2023-11-25T04:23:45.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005616116, 2023-11-25T04:23:45.828 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:45.828 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:45.828 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:45.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:45.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:45.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:45.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:45.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:45.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:45.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:45.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785470+0000", 2023-11-25T04:23:45.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0699999999999998e-07 2023-11-25T04:23:45.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:45.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785474+0000", 2023-11-25T04:23:45.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.048e-06 2023-11-25T04:23:45.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:45.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785480+0000", 2023-11-25T04:23:45.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7729999999999996e-06 2023-11-25T04:23:45.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:45.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785483+0000", 2023-11-25T04:23:45.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1970000000000001e-06 2023-11-25T04:23:45.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:45.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787936+0000", 2023-11-25T04:23:45.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024529040000000001 2023-11-25T04:23:45.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:45.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787947+0000", 2023-11-25T04:23:45.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1669e-05 2023-11-25T04:23:45.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.832 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:45.832 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791057+0000", 2023-11-25T04:23:45.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031095200000000002 2023-11-25T04:23:45.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.832 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:45.832 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791086+0000", 2023-11-25T04:23:45.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8697999999999999e-05 2023-11-25T04:23:45.832 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:45.832 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:45.833 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:45.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.833 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:157 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'111, mlcod=11'106)", 2023-11-25T04:23:45.833 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:45.833 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 68.032523413999996, 2023-11-25T04:23:45.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050590280000000001, 2023-11-25T04:23:45.833 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:45.833 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:45.833 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:45.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:45.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:45.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:45.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:45.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:45.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:45.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:45.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786063+0000", 2023-11-25T04:23:45.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2600000000000002e-07 2023-11-25T04:23:45.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.835 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:45.835 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786069+0000", 2023-11-25T04:23:45.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8839999999999999e-06 2023-11-25T04:23:45.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.835 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:45.835 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786079+0000", 2023-11-25T04:23:45.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0124e-05 2023-11-25T04:23:45.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:45.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786085+0000", 2023-11-25T04:23:45.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6660000000000002e-06 2023-11-25T04:23:45.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:45.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788160+0000", 2023-11-25T04:23:45.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020747669999999999 2023-11-25T04:23:45.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:45.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788180+0000", 2023-11-25T04:23:45.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0208999999999999e-05 2023-11-25T04:23:45.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.837 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:45.837 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791087+0000", 2023-11-25T04:23:45.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029064630000000002 2023-11-25T04:23:45.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.837 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:45.837 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791121+0000", 2023-11-25T04:23:45.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4289000000000003e-05 2023-11-25T04:23:45.837 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:45.837 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:45.838 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:45.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.838 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:158 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'112, mlcod=11'106)", 2023-11-25T04:23:45.838 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:45.838 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 68.032107021000002, 2023-11-25T04:23:45.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046663030000000001, 2023-11-25T04:23:45.838 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:45.838 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:45.838 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:45.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:45.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:45.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:45.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:45.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:45.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:45.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:45.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786479+0000", 2023-11-25T04:23:45.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2799999999999996e-07 2023-11-25T04:23:45.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:45.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786498+0000", 2023-11-25T04:23:45.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9069e-05 2023-11-25T04:23:45.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:45.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786508+0000", 2023-11-25T04:23:45.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5810000000000003e-06 2023-11-25T04:23:45.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:45.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786512+0000", 2023-11-25T04:23:45.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4889999999999997e-06 2023-11-25T04:23:45.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:45.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788438+0000", 2023-11-25T04:23:45.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019257059999999999 2023-11-25T04:23:45.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:45.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788449+0000", 2023-11-25T04:23:45.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1419e-05 2023-11-25T04:23:45.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:45.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791122+0000", 2023-11-25T04:23:45.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002672822 2023-11-25T04:23:45.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:45.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791145+0000", 2023-11-25T04:23:45.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2589e-05 2023-11-25T04:23:45.842 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:45.843 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:45.843 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:45.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.843 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:206 1.0 e12/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 12'152, mlcod=12'148)", 2023-11-25T04:23:45.843 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:45.843 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 67.520338550999995, 2023-11-25T04:23:45.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046728189999999999, 2023-11-25T04:23:45.843 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:45.843 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:45.843 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:45.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:45.844 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:45.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:45.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.844 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:45.844 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:45.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:45.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.844 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:45.844 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298248+0000", 2023-11-25T04:23:45.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3600000000000003e-07 2023-11-25T04:23:45.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.845 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:45.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298278+0000", 2023-11-25T04:23:45.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0474e-05 2023-11-25T04:23:45.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.845 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:45.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298288+0000", 2023-11-25T04:23:45.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0156000000000001e-05 2023-11-25T04:23:45.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:45.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298293+0000", 2023-11-25T04:23:45.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7400000000000004e-06 2023-11-25T04:23:45.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:45.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298333+0000", 2023-11-25T04:23:45.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9666000000000003e-05 2023-11-25T04:23:45.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:45.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298351+0000", 2023-11-25T04:23:45.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8544999999999999e-05 2023-11-25T04:23:45.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:45.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302891+0000", 2023-11-25T04:23:45.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045398519999999996 2023-11-25T04:23:45.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:45.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302920+0000", 2023-11-25T04:23:45.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8750000000000001e-05 2023-11-25T04:23:45.848 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:45.848 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:45.848 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:45.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.848 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:14 3.15 3:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:45.848 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:45.848 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 58.785204194000002, 2023-11-25T04:23:45.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010990437, 2023-11-25T04:23:45.848 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:45.848 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:45.848 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:45.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:45.849 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:45.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:45.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:45.849 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:45.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:45.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.850 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:45.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033383+0000", 2023-11-25T04:23:45.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9870000000000002e-06 2023-11-25T04:23:45.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.850 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:45.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033398+0000", 2023-11-25T04:23:45.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4569999999999999e-05 2023-11-25T04:23:45.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.850 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:45.851 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033411+0000", 2023-11-25T04:23:45.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3543999999999999e-05 2023-11-25T04:23:45.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.851 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:45.851 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033423+0000", 2023-11-25T04:23:45.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1905999999999999e-05 2023-11-25T04:23:45.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.851 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:45.851 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034433+0000", 2023-11-25T04:23:45.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00101022 2023-11-25T04:23:45.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:45.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034539+0000", 2023-11-25T04:23:45.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010597199999999999 2023-11-25T04:23:45.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:45.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034667+0000", 2023-11-25T04:23:45.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012765500000000001 2023-11-25T04:23:45.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:45.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037803+0000", 2023-11-25T04:23:45.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031362220000000001 2023-11-25T04:23:45.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:45.853 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:23:45.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0064964669999999997 2023-11-25T04:23:45.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:45.853 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044331+0000", 2023-11-25T04:23:45.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1544999999999997e-05 2023-11-25T04:23:45.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.854 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:45.854 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044372+0000", 2023-11-25T04:23:45.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0349000000000003e-05 2023-11-25T04:23:45.854 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:45.854 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:45.854 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:45.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.854 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:15 3.15 3:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:45.854 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:45.854 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 58.784544543999999, 2023-11-25T04:23:45.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010464362, 2023-11-25T04:23:45.855 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:45.855 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:45.855 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:45.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.855 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:45.855 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:45.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:45.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.855 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:45.855 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:45.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:45.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.856 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.856 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:45.856 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034042+0000", 2023-11-25T04:23:45.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0999999999999997e-07 2023-11-25T04:23:45.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.856 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.856 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:45.856 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034048+0000", 2023-11-25T04:23:45.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0390000000000001e-06 2023-11-25T04:23:45.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.856 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.856 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:45.856 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034059+0000", 2023-11-25T04:23:45.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1182999999999999e-05 2023-11-25T04:23:45.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.857 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:45.857 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034067+0000", 2023-11-25T04:23:45.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1559999999999995e-06 2023-11-25T04:23:45.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.857 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:45.857 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035005+0000", 2023-11-25T04:23:45.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00093747399999999999 2023-11-25T04:23:45.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.858 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:45.858 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035077+0000", 2023-11-25T04:23:45.858 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2747e-05 2023-11-25T04:23:45.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.858 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.858 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:45.858 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035219+0000", 2023-11-25T04:23:45.858 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000141877 2023-11-25T04:23:45.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.858 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.858 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:45.858 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037810+0000", 2023-11-25T04:23:45.859 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002590752 2023-11-25T04:23:45.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.859 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.859 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:45.859 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044437+0000", 2023-11-25T04:23:45.859 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066270110000000004 2023-11-25T04:23:45.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.859 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.859 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:45.859 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044481+0000", 2023-11-25T04:23:45.859 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4106000000000001e-05 2023-11-25T04:23:45.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.859 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.860 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:45.860 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044505+0000", 2023-11-25T04:23:45.860 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4207e-05 2023-11-25T04:23:45.860 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:45.860 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:45.860 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:45.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.860 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.860 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:16 3.1d 3:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:45.860 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:45.860 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 58.784502695999997, 2023-11-25T04:23:45.860 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010413923, 2023-11-25T04:23:45.860 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:45.861 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:45.861 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:45.861 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.861 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:45.861 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:45.861 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:45.861 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.861 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.861 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:45.861 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:45.861 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:45.861 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.862 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.862 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:45.862 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:45.862 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4499999999999997e-07 2023-11-25T04:23:45.862 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.862 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.862 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:45.862 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034087+0000", 2023-11-25T04:23:45.862 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9770000000000002e-06 2023-11-25T04:23:45.862 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.862 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.863 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:45.863 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034094+0000", 2023-11-25T04:23:45.863 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8809999999999998e-06 2023-11-25T04:23:45.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.863 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:45.863 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034100+0000", 2023-11-25T04:23:45.863 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8440000000000003e-06 2023-11-25T04:23:45.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.863 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:45.863 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034993+0000", 2023-11-25T04:23:45.863 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00089336799999999996 2023-11-25T04:23:45.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.864 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.864 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:45.864 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035099+0000", 2023-11-25T04:23:45.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106037 2023-11-25T04:23:45.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.864 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.864 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:45.864 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035276+0000", 2023-11-25T04:23:45.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017631799999999999 2023-11-25T04:23:45.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.864 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.865 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:45.865 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037815+0000", 2023-11-25T04:23:45.865 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025395959999999999 2023-11-25T04:23:45.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.865 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.865 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:45.865 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044420+0000", 2023-11-25T04:23:45.865 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066045749999999997 2023-11-25T04:23:45.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.865 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.865 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:45.865 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044463+0000", 2023-11-25T04:23:45.866 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3205999999999999e-05 2023-11-25T04:23:45.866 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.866 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.866 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:45.866 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044497+0000", 2023-11-25T04:23:45.866 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3476000000000003e-05 2023-11-25T04:23:45.866 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:45.866 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:45.867 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:45.867 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.867 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.867 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:17 3.1d 3:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:45.867 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:45.867 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 58.784477486999997, 2023-11-25T04:23:45.867 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01127356, 2023-11-25T04:23:45.867 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:45.867 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:45.867 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:45.867 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.867 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:45.867 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:45.867 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:45.868 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.868 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.868 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:45.868 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:45.868 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:45.868 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.868 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.868 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:45.868 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:45.868 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1100000000000001e-07 2023-11-25T04:23:45.868 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.868 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.869 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:45.869 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034112+0000", 2023-11-25T04:23:45.869 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8600000000000003e-06 2023-11-25T04:23:45.869 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.869 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.869 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:45.869 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034120+0000", 2023-11-25T04:23:45.869 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7700000000000001e-06 2023-11-25T04:23:45.869 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.869 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.869 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:45.869 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034127+0000", 2023-11-25T04:23:45.869 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0999999999999998e-06 2023-11-25T04:23:45.870 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.870 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.870 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:45.870 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035601+0000", 2023-11-25T04:23:45.870 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014739410000000001 2023-11-25T04:23:45.870 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.870 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.870 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:45.870 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035685+0000", 2023-11-25T04:23:45.870 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3677999999999994e-05 2023-11-25T04:23:45.870 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.870 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.871 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:45.871 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035827+0000", 2023-11-25T04:23:45.871 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014232599999999999 2023-11-25T04:23:45.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.871 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.871 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:45.871 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038725+0000", 2023-11-25T04:23:45.871 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028982779999999998 2023-11-25T04:23:45.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.871 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.871 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:45.871 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045323+0000", 2023-11-25T04:23:45.872 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0065980780000000003 2023-11-25T04:23:45.872 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.872 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.872 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:45.872 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045354+0000", 2023-11-25T04:23:45.872 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0463e-05 2023-11-25T04:23:45.872 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.872 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.872 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:45.872 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045381+0000", 2023-11-25T04:23:45.872 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7654999999999999e-05 2023-11-25T04:23:45.872 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:45.873 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:45.873 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:45.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.873 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.873 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:22 3.e 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:45.873 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:45.873 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 58.784451074000003, 2023-11-25T04:23:45.873 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010133422, 2023-11-25T04:23:45.873 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:45.873 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:45.873 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:45.873 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.873 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:45.874 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:45.874 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:45.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.874 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:45.874 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:45.874 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:45.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.875 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:45.875 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034135+0000", 2023-11-25T04:23:45.875 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-07 2023-11-25T04:23:45.875 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.875 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.875 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:45.875 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034138+0000", 2023-11-25T04:23:45.875 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6009999999999999e-06 2023-11-25T04:23:45.875 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.875 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.875 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:45.875 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034145+0000", 2023-11-25T04:23:45.875 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1169999999999998e-06 2023-11-25T04:23:45.876 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.876 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.876 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:45.876 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034150+0000", 2023-11-25T04:23:45.876 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9080000000000003e-06 2023-11-25T04:23:45.876 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.876 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.876 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:45.876 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034209+0000", 2023-11-25T04:23:45.876 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8149000000000002e-05 2023-11-25T04:23:45.876 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.876 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.877 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:45.877 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034415+0000", 2023-11-25T04:23:45.877 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020637699999999999 2023-11-25T04:23:45.877 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.877 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.877 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:45.877 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034584+0000", 2023-11-25T04:23:45.877 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016870399999999999 2023-11-25T04:23:45.877 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.877 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.877 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:45.877 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.036979+0000", 2023-11-25T04:23:45.877 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023957509999999998 2023-11-25T04:23:45.878 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.878 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.878 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:45.878 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044161+0000", 2023-11-25T04:23:45.878 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0071811590000000003 2023-11-25T04:23:45.878 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.878 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.878 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:45.879 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044224+0000", 2023-11-25T04:23:45.879 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3454000000000006e-05 2023-11-25T04:23:45.879 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.879 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.879 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:45.879 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044268+0000", 2023-11-25T04:23:45.879 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3761999999999999e-05 2023-11-25T04:23:45.879 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:45.879 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:45.879 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:45.879 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.879 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.879 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:23 3.e 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:45.880 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:45.880 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 58.784419665000001, 2023-11-25T04:23:45.880 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010281676, 2023-11-25T04:23:45.880 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:45.880 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:45.880 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:45.880 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.880 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:45.880 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:45.880 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:45.880 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.880 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.880 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:45.881 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:45.881 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:45.881 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.881 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.881 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:45.881 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:45.881 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5799999999999999e-07 2023-11-25T04:23:45.881 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.881 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.881 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:45.881 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034169+0000", 2023-11-25T04:23:45.881 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1879999999999998e-06 2023-11-25T04:23:45.881 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.882 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.882 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:45.882 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034177+0000", 2023-11-25T04:23:45.882 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7020000000000001e-06 2023-11-25T04:23:45.882 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.882 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.882 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:45.882 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034184+0000", 2023-11-25T04:23:45.882 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3910000000000003e-06 2023-11-25T04:23:45.882 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.882 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.882 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:45.883 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034924+0000", 2023-11-25T04:23:45.883 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00074032999999999998 2023-11-25T04:23:45.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.883 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.883 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:45.883 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034991+0000", 2023-11-25T04:23:45.883 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7347000000000004e-05 2023-11-25T04:23:45.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.883 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.883 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:45.883 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035096+0000", 2023-11-25T04:23:45.883 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010488 2023-11-25T04:23:45.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.884 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.884 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:45.884 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037588+0000", 2023-11-25T04:23:45.884 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024918309999999999 2023-11-25T04:23:45.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.884 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.884 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:45.884 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044367+0000", 2023-11-25T04:23:45.884 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067787589999999997 2023-11-25T04:23:45.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.884 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.884 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:45.884 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044415+0000", 2023-11-25T04:23:45.885 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7803000000000003e-05 2023-11-25T04:23:45.885 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.885 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.885 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:45.885 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044447+0000", 2023-11-25T04:23:45.885 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2886999999999999e-05 2023-11-25T04:23:45.885 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:45.885 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:45.885 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:45.885 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.885 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.885 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:30 3.15 3:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:45.886 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:45.886 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 58.784361224999998, 2023-11-25T04:23:45.886 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010365070000000001, 2023-11-25T04:23:45.886 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:45.886 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:45.886 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:45.886 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.886 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:45.886 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:45.886 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:45.886 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.887 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.887 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:45.887 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:45.887 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:45.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.887 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.887 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:45.887 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034225+0000", 2023-11-25T04:23:45.887 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7400000000000002e-07 2023-11-25T04:23:45.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.887 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.887 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:45.887 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034229+0000", 2023-11-25T04:23:45.888 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3100000000000002e-06 2023-11-25T04:23:45.888 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.888 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.888 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:45.888 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034239+0000", 2023-11-25T04:23:45.888 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9140000000000003e-06 2023-11-25T04:23:45.888 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.888 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.888 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:45.889 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034246+0000", 2023-11-25T04:23:45.889 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0149999999999998e-06 2023-11-25T04:23:45.889 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.889 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.889 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:45.889 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035572+0000", 2023-11-25T04:23:45.889 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001325655 2023-11-25T04:23:45.889 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.889 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.889 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:45.890 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035730+0000", 2023-11-25T04:23:45.890 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015827899999999999 2023-11-25T04:23:45.890 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.890 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.890 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:45.890 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035857+0000", 2023-11-25T04:23:45.890 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012644699999999999 2023-11-25T04:23:45.890 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.890 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.890 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:45.890 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038737+0000", 2023-11-25T04:23:45.890 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002880077 2023-11-25T04:23:45.891 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.891 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.891 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:45.891 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044540+0000", 2023-11-25T04:23:45.891 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058034239999999997 2023-11-25T04:23:45.891 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.891 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.891 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:45.891 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044569+0000", 2023-11-25T04:23:45.891 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8388999999999999e-05 2023-11-25T04:23:45.891 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.891 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.891 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:45.892 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044589+0000", 2023-11-25T04:23:45.892 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0686000000000002e-05 2023-11-25T04:23:45.892 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:45.892 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:45.892 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:45.892 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.892 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.892 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:33 3.1b 3:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:45.892 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:45.892 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 58.784315726999999, 2023-11-25T04:23:45.892 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099754049999999997, 2023-11-25T04:23:45.892 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:45.892 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:23:45.893 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:45.893 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.893 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:45.893 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:45.893 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:45.893 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.893 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.893 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:45.893 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:45.893 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:45.893 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.893 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.894 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:45.894 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:45.894 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4799999999999998e-07 2023-11-25T04:23:45.894 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.894 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.894 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:45.894 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034273+0000", 2023-11-25T04:23:45.894 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7819999999999998e-06 2023-11-25T04:23:45.894 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.894 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.894 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:45.894 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034285+0000", 2023-11-25T04:23:45.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2058e-05 2023-11-25T04:23:45.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:45.895 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034291+0000", 2023-11-25T04:23:45.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0680000000000002e-06 2023-11-25T04:23:45.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:45.895 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034370+0000", 2023-11-25T04:23:45.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8452e-05 2023-11-25T04:23:45.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.896 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:45.896 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034477+0000", 2023-11-25T04:23:45.896 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107471 2023-11-25T04:23:45.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.896 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:45.896 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044245+0000", 2023-11-25T04:23:45.896 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0097679259999999997 2023-11-25T04:23:45.896 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:45.896 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:45.896 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:45.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.897 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:36 3.1a 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:45.897 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:45.897 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 58.784269053999999, 2023-11-25T04:23:45.897 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058179099999999999, 2023-11-25T04:23:45.897 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:45.897 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:23:45.897 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:45.897 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.897 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:45.897 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:45.897 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:45.897 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.897 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.898 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:45.898 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:45.898 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:45.898 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.898 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:45.898 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034317+0000", 2023-11-25T04:23:45.898 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1999999999999999e-07 2023-11-25T04:23:45.898 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.898 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:45.898 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034321+0000", 2023-11-25T04:23:45.898 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.233e-06 2023-11-25T04:23:45.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.899 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:45.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034337+0000", 2023-11-25T04:23:45.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5308999999999999e-05 2023-11-25T04:23:45.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.899 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:45.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034354+0000", 2023-11-25T04:23:45.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7411999999999999e-05 2023-11-25T04:23:45.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.899 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:45.900 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034465+0000", 2023-11-25T04:23:45.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011154699999999999 2023-11-25T04:23:45.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.900 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:45.900 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034560+0000", 2023-11-25T04:23:45.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4186999999999998e-05 2023-11-25T04:23:45.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.900 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:45.900 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.040134+0000", 2023-11-25T04:23:45.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055746019999999997 2023-11-25T04:23:45.900 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:45.901 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:45.901 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:45.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.901 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:9 3.10 e18/16 3:09401a81:::601.00000000:head v 18'4, mlcod=0'0)", 2023-11-25T04:23:45.901 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:45.901 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 58.775374745999997, 2023-11-25T04:23:45.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061641020000000003, 2023-11-25T04:23:45.901 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:45.901 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:45.901 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:45.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.902 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:45.902 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:45.902 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:45.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.902 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:45.902 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:45.902 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:45.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.902 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:45.902 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:45.903 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5300000000000001e-07 2023-11-25T04:23:45.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.903 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.903 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:45.903 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043222+0000", 2023-11-25T04:23:45.903 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.082e-05 2023-11-25T04:23:45.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.903 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.903 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:45.903 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043228+0000", 2023-11-25T04:23:45.903 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6699999999999997e-06 2023-11-25T04:23:45.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.903 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.904 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:45.904 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043236+0000", 2023-11-25T04:23:45.904 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8189999999999992e-06 2023-11-25T04:23:45.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.904 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:45.904 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043507+0000", 2023-11-25T04:23:45.904 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000271012 2023-11-25T04:23:45.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.904 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:45.904 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043530+0000", 2023-11-25T04:23:45.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2659000000000001e-05 2023-11-25T04:23:45.905 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.905 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.905 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:45.905 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049321+0000", 2023-11-25T04:23:45.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057915129999999999 2023-11-25T04:23:45.905 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.905 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.905 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:45.905 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049375+0000", 2023-11-25T04:23:45.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3455999999999998e-05 2023-11-25T04:23:45.905 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:45.905 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:45.906 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:45.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.906 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.906 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:1 3.14 e18/16 3:292cf221:::200.00000000:head v 18'3, mlcod=0'0)", 2023-11-25T04:23:45.906 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:45.906 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 58.775338032999997, 2023-11-25T04:23:45.906 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0060983599999999997, 2023-11-25T04:23:45.906 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:45.906 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:45.906 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:45.906 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.906 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:45.907 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:45.909 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:45.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.909 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.909 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:45.909 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:45.909 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:45.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.909 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.909 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:45.909 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043248+0000", 2023-11-25T04:23:45.909 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9399999999999995e-07 2023-11-25T04:23:45.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.910 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.910 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:45.910 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043252+0000", 2023-11-25T04:23:45.910 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1799999999999998e-06 2023-11-25T04:23:45.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.910 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.910 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:45.910 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043259+0000", 2023-11-25T04:23:45.910 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0049999999999998e-06 2023-11-25T04:23:45.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.910 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.910 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:45.911 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043265+0000", 2023-11-25T04:23:45.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0859999999999998e-06 2023-11-25T04:23:45.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.911 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:45.911 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043311+0000", 2023-11-25T04:23:45.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6082999999999999e-05 2023-11-25T04:23:45.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.911 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:45.911 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043330+0000", 2023-11-25T04:23:45.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8556e-05 2023-11-25T04:23:45.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.912 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:45.912 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049303+0000", 2023-11-25T04:23:45.912 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059733470000000004 2023-11-25T04:23:45.912 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.912 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:45.912 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049346+0000", 2023-11-25T04:23:45.912 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2608999999999998e-05 2023-11-25T04:23:45.912 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:45.912 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:45.912 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:45.912 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.913 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.913 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:3 3.14 e18/16 3:287034c4:::1.00000000.inode:head v 18'4, mlcod=0'0)", 2023-11-25T04:23:45.913 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:45.913 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 58.774821527, 2023-11-25T04:23:45.913 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057040090000000003, 2023-11-25T04:23:45.913 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:45.913 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:45.913 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:45.913 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.913 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:45.913 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:45.913 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:45.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.914 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.914 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:45.914 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:45.914 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:45.914 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.914 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.914 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:45.914 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:45.914 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9799999999999999e-07 2023-11-25T04:23:45.914 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.914 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.914 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:45.914 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043785+0000", 2023-11-25T04:23:45.915 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0721999999999999e-05 2023-11-25T04:23:45.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.915 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:45.915 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043793+0000", 2023-11-25T04:23:45.915 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5839999999999995e-06 2023-11-25T04:23:45.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.915 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:45.915 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043803+0000", 2023-11-25T04:23:45.915 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0729999999999994e-06 2023-11-25T04:23:45.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.916 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:45.916 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043843+0000", 2023-11-25T04:23:45.916 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0747999999999997e-05 2023-11-25T04:23:45.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.916 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:45.916 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043856+0000", 2023-11-25T04:23:45.916 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2697000000000001e-05 2023-11-25T04:23:45.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.916 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:45.916 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049426+0000", 2023-11-25T04:23:45.917 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055704409999999998 2023-11-25T04:23:45.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.917 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:45.917 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049468+0000", 2023-11-25T04:23:45.917 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1446000000000003e-05 2023-11-25T04:23:45.917 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:45.917 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:45.917 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:45.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.917 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "pg_update_log_missing(3.14 epoch 18/16 rep_tid 171 entries 18'5 (0'0) error 3:2e2fa760:::200.00000001:head by mds.0.3:32 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:23:45.917 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:45.918 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 58.774660564000001, 2023-11-25T04:23:45.918 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005556938, 2023-11-25T04:23:45.918 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:45.918 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2023-11-25T04:23:45.918 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:45.918 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.918 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:45.918 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:45.918 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:45.918 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.918 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.918 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:45.918 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:45.919 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:45.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.919 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.919 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:45.919 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:45.919 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.03e-07 2023-11-25T04:23:45.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.919 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.919 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:45.919 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043931+0000", 2023-11-25T04:23:45.919 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1079999999999998e-06 2023-11-25T04:23:45.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.920 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.920 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:45.920 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043944+0000", 2023-11-25T04:23:45.920 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2852e-05 2023-11-25T04:23:45.920 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.920 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.920 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:45.920 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043955+0000", 2023-11-25T04:23:45.920 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1037e-05 2023-11-25T04:23:45.920 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.920 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.920 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:45.920 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044088+0000", 2023-11-25T04:23:45.921 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013269599999999999 2023-11-25T04:23:45.921 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.921 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.921 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:45.921 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049482+0000", 2023-11-25T04:23:45.921 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053940419999999999 2023-11-25T04:23:45.921 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:45.921 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:45.921 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:45.921 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.921 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.921 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:10 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'5, mlcod=0'0)", 2023-11-25T04:23:45.921 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:45.922 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 58.774570535999999, 2023-11-25T04:23:45.922 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054493099999999997, 2023-11-25T04:23:45.922 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:45.922 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:45.922 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:45.922 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.922 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:45.922 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:45.922 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:45.922 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.922 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.922 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:45.922 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:45.923 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:45.923 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.923 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.923 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:45.923 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:45.923 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6100000000000002e-07 2023-11-25T04:23:45.923 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.923 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.923 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:45.923 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044019+0000", 2023-11-25T04:23:45.923 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.839e-06 2023-11-25T04:23:45.923 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.923 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.924 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:45.924 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044026+0000", 2023-11-25T04:23:45.924 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5869999999999998e-06 2023-11-25T04:23:45.924 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.924 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.924 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:45.924 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044031+0000", 2023-11-25T04:23:45.924 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4909999999999996e-06 2023-11-25T04:23:45.924 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.924 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.924 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:45.924 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044070+0000", 2023-11-25T04:23:45.924 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8829999999999999e-05 2023-11-25T04:23:45.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.925 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:45.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044081+0000", 2023-11-25T04:23:45.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1447999999999999e-05 2023-11-25T04:23:45.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.925 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:45.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049406+0000", 2023-11-25T04:23:45.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053246480000000004 2023-11-25T04:23:45.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.926 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:45.926 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049464+0000", 2023-11-25T04:23:45.926 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8205999999999998e-05 2023-11-25T04:23:45.926 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:45.926 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:45.926 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:45.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.926 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.926 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:11 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'6, mlcod=0'0)", 2023-11-25T04:23:45.926 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:45.926 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 58.774369272000001, 2023-11-25T04:23:45.926 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052673499999999996, 2023-11-25T04:23:45.926 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:45.927 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:45.927 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:45.927 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.927 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:45.927 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:45.927 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:45.927 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.927 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.927 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:45.927 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:45.927 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:45.927 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.928 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.928 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:45.928 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:45.928 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0199999999999998e-07 2023-11-25T04:23:45.928 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.928 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.928 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:45.928 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044220+0000", 2023-11-25T04:23:45.928 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7720000000000001e-06 2023-11-25T04:23:45.928 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.928 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.928 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:45.928 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044226+0000", 2023-11-25T04:23:45.929 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8579999999999996e-06 2023-11-25T04:23:45.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.929 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:45.929 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044232+0000", 2023-11-25T04:23:45.929 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1700000000000002e-06 2023-11-25T04:23:45.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.929 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:45.929 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044274+0000", 2023-11-25T04:23:45.929 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1254999999999997e-05 2023-11-25T04:23:45.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.929 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:45.930 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:23:45.930 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6880999999999999e-05 2023-11-25T04:23:45.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.930 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.930 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:45.930 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049466+0000", 2023-11-25T04:23:45.930 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051659219999999999 2023-11-25T04:23:45.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:45.930 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:45.930 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:45.930 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049483+0000", 2023-11-25T04:23:45.930 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.719e-05 2023-11-25T04:23:45.930 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:45.931 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:45.931 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:45.931 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:45.931 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:45.931 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:45.932 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:23:45.933 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:45.930+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:45.968 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2023-11-25T04:23:46.033 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:46.031+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:46.042 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:46.042 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:23:46.042 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2023-11-25T04:23:46.042 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:46.050 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:23:46.134 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:46.133+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:46.195 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:46.196 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:23:46.196 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2023-11-25T04:23:46.196 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2023-11-25T04:23:46.196 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:46.204 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:23:46.246 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:46.244+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:46.336 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:46.335+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:46.353 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:46.353 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2023-11-25T04:23:46.353 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2023-11-25T04:23:46.353 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2023-11-25T04:23:46.354 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.354 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:46.354 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:46.354 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 59.318220515, 2023-11-25T04:23:46.355 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01072943, 2023-11-25T04:23:46.355 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:46.355 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:46.355 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:46.355 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.355 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:46.355 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:46.355 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:46.356 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.356 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.356 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:46.356 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:46.356 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:46.356 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.356 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.357 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:46.357 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031829+0000", 2023-11-25T04:23:46.357 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.666e-06 2023-11-25T04:23:46.357 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.357 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.357 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:46.357 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031839+0000", 2023-11-25T04:23:46.357 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9490000000000008e-06 2023-11-25T04:23:46.357 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.357 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.357 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:46.357 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031857+0000", 2023-11-25T04:23:46.357 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8093000000000001e-05 2023-11-25T04:23:46.358 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.358 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.358 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:46.358 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031882+0000", 2023-11-25T04:23:46.358 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4294000000000002e-05 2023-11-25T04:23:46.358 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.358 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.358 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:46.358 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033130+0000", 2023-11-25T04:23:46.358 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001248264 2023-11-25T04:23:46.358 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.358 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.359 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:46.359 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033426+0000", 2023-11-25T04:23:46.359 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029620999999999998 2023-11-25T04:23:46.359 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.359 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.359 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:46.359 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033814+0000", 2023-11-25T04:23:46.359 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000387742 2023-11-25T04:23:46.359 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.359 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.359 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:46.359 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036793+0000", 2023-11-25T04:23:46.359 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002979598 2023-11-25T04:23:46.360 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.360 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.360 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:46.360 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042455+0000", 2023-11-25T04:23:46.360 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0056619260000000003 2023-11-25T04:23:46.360 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.360 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.360 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:46.360 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042502+0000", 2023-11-25T04:23:46.360 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7139999999999999e-05 2023-11-25T04:23:46.360 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.360 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.361 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:46.361 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042557+0000", 2023-11-25T04:23:46.361 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4548e-05 2023-11-25T04:23:46.361 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:46.361 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:46.361 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:46.361 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.361 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.361 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:46.361 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:46.361 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 59.31702198, 2023-11-25T04:23:46.361 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011505202000000001, 2023-11-25T04:23:46.361 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:46.361 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:46.362 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:46.362 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.362 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:46.362 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:46.362 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:46.362 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.362 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.362 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:46.362 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:46.362 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:46.362 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.362 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.363 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:46.363 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033027+0000", 2023-11-25T04:23:46.363 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0699999999999999e-06 2023-11-25T04:23:46.363 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.363 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.363 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:46.363 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033035+0000", 2023-11-25T04:23:46.363 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0830000000000004e-06 2023-11-25T04:23:46.363 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.363 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.363 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:46.363 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033059+0000", 2023-11-25T04:23:46.363 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3975e-05 2023-11-25T04:23:46.364 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.364 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.364 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:46.364 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033482+0000", 2023-11-25T04:23:46.364 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042253700000000001 2023-11-25T04:23:46.364 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.364 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.364 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:46.364 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034520+0000", 2023-11-25T04:23:46.364 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001037991 2023-11-25T04:23:46.364 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.364 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.365 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:46.365 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034611+0000", 2023-11-25T04:23:46.365 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1113999999999995e-05 2023-11-25T04:23:46.365 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.365 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.365 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:46.365 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034769+0000", 2023-11-25T04:23:46.365 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000157641 2023-11-25T04:23:46.365 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.365 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.365 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:46.365 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038292+0000", 2023-11-25T04:23:46.365 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035231160000000002 2023-11-25T04:23:46.366 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.366 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.366 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:46.366 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044440+0000", 2023-11-25T04:23:46.366 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0061486980000000002 2023-11-25T04:23:46.366 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.366 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.366 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:46.366 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044495+0000", 2023-11-25T04:23:46.366 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4339000000000003e-05 2023-11-25T04:23:46.366 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.366 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.367 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:46.367 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044531+0000", 2023-11-25T04:23:46.367 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6637999999999997e-05 2023-11-25T04:23:46.367 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:46.367 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:46.367 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:46.367 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.367 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.367 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:46.367 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:46.367 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 59.316547626000002, 2023-11-25T04:23:46.367 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0057634039999999997, 2023-11-25T04:23:46.367 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:46.368 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:46.368 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:46.368 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.368 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:46.368 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:46.368 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:46.368 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.368 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.368 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:46.368 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:46.368 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:46.368 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.369 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.369 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:46.369 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:46.369 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7900000000000007e-07 2023-11-25T04:23:46.369 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.369 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.369 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:46.369 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033507+0000", 2023-11-25T04:23:46.369 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8900000000000004e-06 2023-11-25T04:23:46.369 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.369 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.369 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:46.369 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033519+0000", 2023-11-25T04:23:46.370 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1756000000000001e-05 2023-11-25T04:23:46.370 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.370 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.370 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:46.370 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033527+0000", 2023-11-25T04:23:46.370 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3610000000000001e-06 2023-11-25T04:23:46.370 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.370 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.370 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:46.370 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034451+0000", 2023-11-25T04:23:46.370 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00092372400000000001 2023-11-25T04:23:46.370 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.371 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.371 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:46.371 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034809+0000", 2023-11-25T04:23:46.371 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00035757699999999998 2023-11-25T04:23:46.371 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.371 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.371 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:23:46.371 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034965+0000", 2023-11-25T04:23:46.371 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015638699999999999 2023-11-25T04:23:46.371 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.371 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.371 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:46.371 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038299+0000", 2023-11-25T04:23:46.372 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0033336559999999999 2023-11-25T04:23:46.372 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.372 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.372 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:46.372 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039169+0000", 2023-11-25T04:23:46.372 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00087066400000000005 2023-11-25T04:23:46.372 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.372 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.372 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:46.372 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039226+0000", 2023-11-25T04:23:46.372 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6511999999999997e-05 2023-11-25T04:23:46.372 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.373 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.373 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:46.373 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039264+0000", 2023-11-25T04:23:46.373 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7898000000000002e-05 2023-11-25T04:23:46.373 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:46.373 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:46.373 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:46.373 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.373 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.373 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:25 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:46.374 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:46.374 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 59.316469042000001, 2023-11-25T04:23:46.374 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096656169999999996, 2023-11-25T04:23:46.374 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:46.374 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:46.374 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:46.374 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.374 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:46.374 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:46.374 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:46.374 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.374 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.375 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:46.375 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:46.375 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:46.375 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.375 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.375 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:46.375 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033580+0000", 2023-11-25T04:23:46.375 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8199999999999999e-07 2023-11-25T04:23:46.375 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.375 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.375 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:46.375 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033585+0000", 2023-11-25T04:23:46.375 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3680000000000001e-06 2023-11-25T04:23:46.375 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.376 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.376 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:46.376 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033600+0000", 2023-11-25T04:23:46.376 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4331000000000001e-05 2023-11-25T04:23:46.376 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.376 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.376 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:46.376 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033606+0000", 2023-11-25T04:23:46.376 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5780000000000004e-06 2023-11-25T04:23:46.376 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.376 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.376 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:46.377 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035747+0000", 2023-11-25T04:23:46.377 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0021406530000000002 2023-11-25T04:23:46.377 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.377 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.377 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:46.377 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035901+0000", 2023-11-25T04:23:46.377 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000153931 2023-11-25T04:23:46.377 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.377 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.377 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:23:46.377 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036193+0000", 2023-11-25T04:23:46.377 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029211299999999999 2023-11-25T04:23:46.377 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.378 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.378 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:46.378 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038655+0000", 2023-11-25T04:23:46.378 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0024621640000000002 2023-11-25T04:23:46.378 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.378 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.378 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:46.378 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043130+0000", 2023-11-25T04:23:46.378 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.004474698 2023-11-25T04:23:46.378 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.378 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.378 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:46.378 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043204+0000", 2023-11-25T04:23:46.379 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4091999999999995e-05 2023-11-25T04:23:46.379 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.379 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.379 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:46.379 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043245+0000", 2023-11-25T04:23:46.379 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0907e-05 2023-11-25T04:23:46.379 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:46.379 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:46.379 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:46.379 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.379 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.379 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:26 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:46.380 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:46.380 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 59.316433218, 2023-11-25T04:23:46.380 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011086945000000001, 2023-11-25T04:23:46.380 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:46.380 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:46.380 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:46.380 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.380 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:46.380 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:46.380 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:46.380 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.380 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.380 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:46.381 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:46.381 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:46.381 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.381 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.381 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:46.381 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:46.381 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1699999999999998e-07 2023-11-25T04:23:46.381 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.381 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.381 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:46.381 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033627+0000", 2023-11-25T04:23:46.381 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.163e-05 2023-11-25T04:23:46.382 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.382 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.382 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:46.382 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033636+0000", 2023-11-25T04:23:46.382 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6859999999999995e-06 2023-11-25T04:23:46.382 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.382 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.382 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:46.382 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033642+0000", 2023-11-25T04:23:46.382 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3149999999999997e-06 2023-11-25T04:23:46.382 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.382 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.382 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:46.383 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035414+0000", 2023-11-25T04:23:46.383 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0017716380000000001 2023-11-25T04:23:46.383 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.383 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.383 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:46.383 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035539+0000", 2023-11-25T04:23:46.383 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000125477 2023-11-25T04:23:46.383 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.383 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.383 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:46.383 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035672+0000", 2023-11-25T04:23:46.383 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000133316 2023-11-25T04:23:46.383 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.384 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.384 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:46.384 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038649+0000", 2023-11-25T04:23:46.384 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029764679999999999 2023-11-25T04:23:46.384 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.384 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.384 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:46.384 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044578+0000", 2023-11-25T04:23:46.384 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0059293510000000002 2023-11-25T04:23:46.384 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.384 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.384 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:46.385 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044651+0000", 2023-11-25T04:23:46.385 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2513000000000002e-05 2023-11-25T04:23:46.385 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.385 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.385 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:46.385 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044702+0000", 2023-11-25T04:23:46.385 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1034e-05 2023-11-25T04:23:46.385 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:46.385 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:46.385 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:46.385 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.385 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.385 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:29 3.b 3:d0630e4c:::mds0_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:46.386 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:46.386 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 59.316395298000003, 2023-11-25T04:23:46.386 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010980227, 2023-11-25T04:23:46.386 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:46.386 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:46.386 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:46.386 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.386 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:46.386 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:46.386 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:46.386 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.386 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.387 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:46.387 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:46.387 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:46.387 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.387 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.387 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:46.387 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:46.387 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3499999999999996e-07 2023-11-25T04:23:46.387 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.387 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.387 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:46.387 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033658+0000", 2023-11-25T04:23:46.387 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1280000000000001e-06 2023-11-25T04:23:46.388 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.388 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.388 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:46.388 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033667+0000", 2023-11-25T04:23:46.388 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1430000000000004e-06 2023-11-25T04:23:46.388 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.388 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.388 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:46.388 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033678+0000", 2023-11-25T04:23:46.388 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0906e-05 2023-11-25T04:23:46.388 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.388 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.388 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:46.389 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034456+0000", 2023-11-25T04:23:46.389 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00077833899999999996 2023-11-25T04:23:46.389 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.389 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.389 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:46.389 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034575+0000", 2023-11-25T04:23:46.389 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011926000000000001 2023-11-25T04:23:46.389 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.389 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.389 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:46.389 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035001+0000", 2023-11-25T04:23:46.389 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042550999999999998 2023-11-25T04:23:46.389 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.390 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.390 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:46.390 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.037923+0000", 2023-11-25T04:23:46.390 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029219670000000001 2023-11-25T04:23:46.390 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.390 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.390 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:46.390 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044538+0000", 2023-11-25T04:23:46.390 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0066152349999999997 2023-11-25T04:23:46.390 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.390 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.390 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:46.391 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044594+0000", 2023-11-25T04:23:46.391 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5593000000000001e-05 2023-11-25T04:23:46.391 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.391 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.391 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:46.391 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044633+0000", 2023-11-25T04:23:46.391 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9511000000000002e-05 2023-11-25T04:23:46.391 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:46.391 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:46.391 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:46.391 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.391 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.391 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:34 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:46.392 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:46.392 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 59.316145185000003, 2023-11-25T04:23:46.392 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010920445000000001, 2023-11-25T04:23:46.392 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:46.392 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:46.392 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:46.392 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.392 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:46.392 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:46.392 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:46.393 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.393 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.393 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:46.393 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:46.393 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:46.393 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.393 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.393 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:46.393 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033904+0000", 2023-11-25T04:23:46.393 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6599999999999994e-07 2023-11-25T04:23:46.393 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.393 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.393 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:46.394 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033909+0000", 2023-11-25T04:23:46.394 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5639999999999996e-06 2023-11-25T04:23:46.394 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.394 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.394 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:46.394 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033923+0000", 2023-11-25T04:23:46.394 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3028e-05 2023-11-25T04:23:46.394 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.394 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.394 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:46.394 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034043+0000", 2023-11-25T04:23:46.394 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000120328 2023-11-25T04:23:46.394 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.395 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.395 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:46.395 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035576+0000", 2023-11-25T04:23:46.395 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015333720000000001 2023-11-25T04:23:46.395 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.395 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.395 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:46.395 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035699+0000", 2023-11-25T04:23:46.395 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000123001 2023-11-25T04:23:46.395 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.395 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.395 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:46.395 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044823+0000", 2023-11-25T04:23:46.396 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0091241859999999994 2023-11-25T04:23:46.396 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:46.396 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:46.396 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:46.396 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.396 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.396 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:35 3.1f 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:46.396 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:46.396 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 59.31484098, 2023-11-25T04:23:46.396 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096164779999999995, 2023-11-25T04:23:46.396 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:46.396 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:46.396 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:46.397 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.397 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:46.397 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:46.397 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:46.397 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.397 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.397 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:46.397 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:46.397 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:46.397 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.397 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.397 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:46.398 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035208+0000", 2023-11-25T04:23:46.398 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2699999999999998e-07 2023-11-25T04:23:46.398 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.398 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.398 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:46.398 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035213+0000", 2023-11-25T04:23:46.398 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.6070000000000001e-06 2023-11-25T04:23:46.398 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.398 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.398 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:46.398 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035350+0000", 2023-11-25T04:23:46.398 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013693599999999999 2023-11-25T04:23:46.398 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.399 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.399 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:46.399 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035358+0000", 2023-11-25T04:23:46.399 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0099999999999995e-06 2023-11-25T04:23:46.399 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.399 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.399 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:46.399 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036260+0000", 2023-11-25T04:23:46.399 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00090240100000000005 2023-11-25T04:23:46.399 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.400 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.400 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:46.400 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036517+0000", 2023-11-25T04:23:46.400 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00025736299999999998 2023-11-25T04:23:46.400 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.400 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.400 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:46.400 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044824+0000", 2023-11-25T04:23:46.400 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0083063340000000003 2023-11-25T04:23:46.400 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:46.400 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:46.400 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:46.400 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.400 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.401 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:37 3.1e 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:46.401 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:46.401 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 59.314674294, 2023-11-25T04:23:46.401 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0078322640000000002, 2023-11-25T04:23:46.401 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:46.401 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:46.401 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:46.401 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.401 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:46.401 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:46.401 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:46.401 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.401 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.402 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:46.402 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:46.402 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:46.402 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.402 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.402 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:46.402 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035375+0000", 2023-11-25T04:23:46.402 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9100000000000003e-07 2023-11-25T04:23:46.402 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.402 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.402 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:46.402 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035379+0000", 2023-11-25T04:23:46.403 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9430000000000002e-06 2023-11-25T04:23:46.403 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.403 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.403 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:46.403 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035388+0000", 2023-11-25T04:23:46.403 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2760000000000006e-06 2023-11-25T04:23:46.403 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.403 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.403 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:46.403 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035396+0000", 2023-11-25T04:23:46.403 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8480000000000001e-06 2023-11-25T04:23:46.403 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.403 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.403 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:46.404 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035452+0000", 2023-11-25T04:23:46.404 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5831000000000001e-05 2023-11-25T04:23:46.404 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.404 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.404 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:46.404 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035934+0000", 2023-11-25T04:23:46.404 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00048265599999999998 2023-11-25T04:23:46.404 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.404 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.404 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:46.404 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043206+0000", 2023-11-25T04:23:46.404 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.007271919 2023-11-25T04:23:46.404 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:46.405 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:46.405 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:46.405 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.405 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.405 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing(3.1a epoch 18/16 rep_tid 3 entries 18'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:36 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:23:46.405 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:46.405 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 59.313398708000001, 2023-11-25T04:23:46.405 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002073414, 2023-11-25T04:23:46.405 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:46.405 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:46.405 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:46.405 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.405 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:46.406 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:46.406 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:46.406 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.406 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.406 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:46.406 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:46.406 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:46.406 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.406 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.406 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:46.406 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036650+0000", 2023-11-25T04:23:46.406 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7300000000000002e-07 2023-11-25T04:23:46.406 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.407 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.407 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:46.407 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036659+0000", 2023-11-25T04:23:46.407 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.7499999999999992e-06 2023-11-25T04:23:46.407 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.407 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.407 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:46.407 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036673+0000", 2023-11-25T04:23:46.407 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4917e-05 2023-11-25T04:23:46.407 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.407 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.407 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:46.407 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036687+0000", 2023-11-25T04:23:46.408 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3883000000000001e-05 2023-11-25T04:23:46.408 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.408 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.408 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:46.408 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036781+0000", 2023-11-25T04:23:46.408 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3219000000000003e-05 2023-11-25T04:23:46.408 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.408 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.408 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:46.408 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038723+0000", 2023-11-25T04:23:46.408 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019422720000000001 2023-11-25T04:23:46.408 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:46.408 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:46.409 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:46.409 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.409 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.409 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:28 3.13 e18/16 3:cd5a64a3:::100.00000000:head v 18'1, mlcod=0'0)", 2023-11-25T04:23:46.409 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:46.409 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 59.309737550000001, 2023-11-25T04:23:46.409 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001850546, 2023-11-25T04:23:46.409 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:46.409 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:46.409 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:46.409 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.409 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:46.409 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:46.410 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:46.410 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.410 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.410 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:46.410 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:46.410 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:46.410 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.410 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.410 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:46.410 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:46.410 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7800000000000002e-07 2023-11-25T04:23:46.410 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.410 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.411 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:46.411 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040331+0000", 2023-11-25T04:23:46.411 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0449999999999999e-05 2023-11-25T04:23:46.411 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.411 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.411 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:46.411 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040339+0000", 2023-11-25T04:23:46.411 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0709999999999995e-06 2023-11-25T04:23:46.411 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.411 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.411 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:46.412 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040347+0000", 2023-11-25T04:23:46.412 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7079999999999997e-06 2023-11-25T04:23:46.412 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.412 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.412 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:46.412 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040410+0000", 2023-11-25T04:23:46.412 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2453000000000007e-05 2023-11-25T04:23:46.412 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.412 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.412 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:46.412 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040422+0000", 2023-11-25T04:23:46.412 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2429000000000001e-05 2023-11-25T04:23:46.412 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.413 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.413 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:46.413 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042129+0000", 2023-11-25T04:23:46.413 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001707266 2023-11-25T04:23:46.413 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.413 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.413 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:46.413 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042161+0000", 2023-11-25T04:23:46.413 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1791000000000001e-05 2023-11-25T04:23:46.413 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:46.413 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:46.413 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:46.413 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.414 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.414 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:27 3.13 e18/16 3:c91d4a1d:::mds0_inotable:head v 18'2, mlcod=0'0)", 2023-11-25T04:23:46.414 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:46.414 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 59.309534026999998, 2023-11-25T04:23:46.414 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0016886430000000001, 2023-11-25T04:23:46.414 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:46.414 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:46.414 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:46.414 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.414 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:46.414 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:46.414 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:46.415 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.415 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.415 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:46.415 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:46.415 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:46.415 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.415 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.415 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:46.415 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:46.415 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3999999999999997e-07 2023-11-25T04:23:46.415 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.415 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.415 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:46.416 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040520+0000", 2023-11-25T04:23:46.416 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2989999999999996e-06 2023-11-25T04:23:46.416 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.416 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.416 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:46.416 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040527+0000", 2023-11-25T04:23:46.416 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5730000000000002e-06 2023-11-25T04:23:46.416 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.416 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.416 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:46.416 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040535+0000", 2023-11-25T04:23:46.416 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8630000000000008e-06 2023-11-25T04:23:46.416 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.417 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.417 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:46.417 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040683+0000", 2023-11-25T04:23:46.417 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000148002 2023-11-25T04:23:46.417 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.417 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.417 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:46.417 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040700+0000", 2023-11-25T04:23:46.417 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7065e-05 2023-11-25T04:23:46.417 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.417 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.417 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:46.417 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042163+0000", 2023-11-25T04:23:46.418 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001462767 2023-11-25T04:23:46.418 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.418 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.418 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:46.418 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042203+0000", 2023-11-25T04:23:46.418 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9734000000000003e-05 2023-11-25T04:23:46.418 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:46.418 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:46.418 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:46.418 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.418 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.418 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:20 3.1c e18/16 3:3ed09add:::607.00000000:head v 18'1, mlcod=0'0)", 2023-11-25T04:23:46.418 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:46.419 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 59.309498986000001, 2023-11-25T04:23:46.419 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015271, 2023-11-25T04:23:46.419 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:46.419 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:46.419 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:46.419 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.419 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:46.419 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:46.419 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:46.419 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.419 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.419 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:46.419 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:46.420 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:46.420 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.420 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.420 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:46.420 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:46.420 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4799999999999999e-07 2023-11-25T04:23:46.420 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.420 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.420 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:46.420 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040561+0000", 2023-11-25T04:23:46.420 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1800000000000001e-05 2023-11-25T04:23:46.420 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.420 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.421 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:46.421 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040568+0000", 2023-11-25T04:23:46.421 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7730000000000002e-06 2023-11-25T04:23:46.421 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.421 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.421 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:46.421 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040587+0000", 2023-11-25T04:23:46.421 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8691000000000001e-05 2023-11-25T04:23:46.421 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.421 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.421 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:46.421 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040638+0000", 2023-11-25T04:23:46.421 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1068e-05 2023-11-25T04:23:46.422 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.422 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.422 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:46.422 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040670+0000", 2023-11-25T04:23:46.422 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2125999999999997e-05 2023-11-25T04:23:46.422 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.422 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.422 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:46.422 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042032+0000", 2023-11-25T04:23:46.422 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001361821 2023-11-25T04:23:46.422 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.422 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.422 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:46.422 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042076+0000", 2023-11-25T04:23:46.423 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4472999999999999e-05 2023-11-25T04:23:46.423 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:46.423 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:46.423 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:46.423 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.423 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.423 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:21 3.1c e18/16 3:3ed09add:::607.00000000:head v 18'2, mlcod=0'0)", 2023-11-25T04:23:46.423 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:46.423 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 59.309264261000003, 2023-11-25T04:23:46.423 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0013228160000000001, 2023-11-25T04:23:46.423 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:46.423 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:46.423 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:46.423 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.424 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:46.424 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:46.424 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:46.424 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.424 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.424 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:46.424 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:46.424 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:46.424 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.424 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.424 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:46.424 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:46.424 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9299999999999999e-07 2023-11-25T04:23:46.425 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.425 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.425 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:46.425 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040790+0000", 2023-11-25T04:23:46.425 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5770000000000001e-06 2023-11-25T04:23:46.425 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.425 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.425 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:46.425 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040799+0000", 2023-11-25T04:23:46.425 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6439999999999995e-06 2023-11-25T04:23:46.425 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.425 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.425 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:46.426 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040806+0000", 2023-11-25T04:23:46.426 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0180000000000004e-06 2023-11-25T04:23:46.426 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.426 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.426 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:46.426 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040907+0000", 2023-11-25T04:23:46.426 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000100096 2023-11-25T04:23:46.426 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.426 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.426 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:46.426 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040925+0000", 2023-11-25T04:23:46.426 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8536999999999999e-05 2023-11-25T04:23:46.426 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.427 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.427 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:46.427 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042079+0000", 2023-11-25T04:23:46.427 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00115358 2023-11-25T04:23:46.427 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.427 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.427 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:46.427 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042107+0000", 2023-11-25T04:23:46.427 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7971e-05 2023-11-25T04:23:46.427 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:46.427 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:46.427 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:46.428 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.428 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.428 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop_reply(mds.0.3:4 3.1f e18/16 ondisk, result = 0)", 2023-11-25T04:23:46.428 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:46.428 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 59.307924870999997, 2023-11-25T04:23:46.428 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00044092900000000001, 2023-11-25T04:23:46.428 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:46.428 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:46.428 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:46.428 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.428 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:46.428 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:46.428 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:46.429 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.429 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.429 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:46.429 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:46.429 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:46.429 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.429 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.429 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:46.429 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042124+0000", 2023-11-25T04:23:46.429 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1800000000000005e-07 2023-11-25T04:23:46.429 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.429 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.429 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:46.430 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042128+0000", 2023-11-25T04:23:46.430 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2289999999999997e-06 2023-11-25T04:23:46.430 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.430 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.430 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:46.430 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042138+0000", 2023-11-25T04:23:46.430 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9729999999999992e-06 2023-11-25T04:23:46.430 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.430 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.430 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:46.430 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042145+0000", 2023-11-25T04:23:46.430 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.336e-06 2023-11-25T04:23:46.430 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.431 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.431 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:46.431 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042441+0000", 2023-11-25T04:23:46.431 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029650500000000002 2023-11-25T04:23:46.431 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.431 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.431 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:46.431 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042448+0000", 2023-11-25T04:23:46.431 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4100000000000002e-06 2023-11-25T04:23:46.431 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.431 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.431 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:46.431 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042564+0000", 2023-11-25T04:23:46.432 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000115758 2023-11-25T04:23:46.432 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:46.432 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:46.432 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:46.432 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.432 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.432 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing_reply(3.1f epoch 18/16 rep_tid 8 lcod 18'4)", 2023-11-25T04:23:46.432 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:46.432 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 59.305599459, 2023-11-25T04:23:46.432 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00038154699999999998, 2023-11-25T04:23:46.432 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:46.432 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:46.432 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:46.433 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.433 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:46.433 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:46.433 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:46.433 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.433 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.433 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:46.433 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:46.433 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:46.433 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.433 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.433 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:46.434 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:46.434 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2700000000000001e-07 2023-11-25T04:23:46.434 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.434 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.434 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:46.434 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044451+0000", 2023-11-25T04:23:46.434 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7460000000000001e-06 2023-11-25T04:23:46.434 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.434 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.434 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:46.434 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044455+0000", 2023-11-25T04:23:46.434 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5809999999999998e-06 2023-11-25T04:23:46.434 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.435 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.435 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:46.435 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044458+0000", 2023-11-25T04:23:46.435 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0170000000000001e-06 2023-11-25T04:23:46.435 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.435 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.435 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:46.435 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044734+0000", 2023-11-25T04:23:46.435 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00027582799999999999 2023-11-25T04:23:46.435 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.435 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.435 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:46.435 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044830+0000", 2023-11-25T04:23:46.436 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6248000000000004e-05 2023-11-25T04:23:46.436 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:46.436 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:46.436 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:46.436 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.436 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.436 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing_reply(3.b epoch 18/16 rep_tid 5 lcod 18'2)", 2023-11-25T04:23:46.436 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:46.436 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 59.305562797999997, 2023-11-25T04:23:46.436 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000344488, 2023-11-25T04:23:46.436 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:46.436 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:46.437 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:46.437 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.437 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:46.437 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:46.437 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:46.437 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.437 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.437 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:46.437 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:46.437 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:46.437 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.437 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.437 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:46.438 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:46.438 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.55e-07 2023-11-25T04:23:46.438 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.438 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.438 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:46.438 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044487+0000", 2023-11-25T04:23:46.438 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2470000000000001e-06 2023-11-25T04:23:46.438 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.438 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.438 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:46.438 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044491+0000", 2023-11-25T04:23:46.438 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9659999999999998e-06 2023-11-25T04:23:46.438 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.439 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.439 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:46.439 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044494+0000", 2023-11-25T04:23:46.439 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2310000000000001e-06 2023-11-25T04:23:46.439 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.439 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.439 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:46.439 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044756+0000", 2023-11-25T04:23:46.439 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00026172800000000003 2023-11-25T04:23:46.439 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.439 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.439 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:46.439 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044830+0000", 2023-11-25T04:23:46.440 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4160999999999997e-05 2023-11-25T04:23:46.440 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:46.440 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:46.440 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:46.440 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.440 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.440 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:39 3.6 3:654134d2:::mds0_openfiles.0:head [omap-clear,omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2023-11-25T04:23:46.440 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:46.440 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 54.245736755999999, 2023-11-25T04:23:46.440 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001788658, 2023-11-25T04:23:46.440 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:46.440 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:46.440 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:46.441 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.441 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:46.441 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:46.441 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:46.441 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.441 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.441 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:46.441 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:46.441 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:46.441 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.441 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.441 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:46.442 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104314+0000", 2023-11-25T04:23:46.442 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3099999999999999e-06 2023-11-25T04:23:46.442 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.442 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.442 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:46.442 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104325+0000", 2023-11-25T04:23:46.442 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1511999999999999e-05 2023-11-25T04:23:46.442 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.442 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.442 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:46.442 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104362+0000", 2023-11-25T04:23:46.442 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6903000000000002e-05 2023-11-25T04:23:46.442 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.443 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.443 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:46.443 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104387+0000", 2023-11-25T04:23:46.443 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4766e-05 2023-11-25T04:23:46.443 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.443 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.443 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:46.443 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104465+0000", 2023-11-25T04:23:46.443 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8119000000000002e-05 2023-11-25T04:23:46.443 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.443 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.443 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:46.443 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104557+0000", 2023-11-25T04:23:46.443 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2496000000000003e-05 2023-11-25T04:23:46.444 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.444 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.444 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:46.444 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104646+0000", 2023-11-25T04:23:46.444 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8818000000000002e-05 2023-11-25T04:23:46.444 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.444 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.444 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:46.444 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105201+0000", 2023-11-25T04:23:46.444 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00055506000000000002 2023-11-25T04:23:46.444 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.444 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.444 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:46.445 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105994+0000", 2023-11-25T04:23:46.445 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00079294700000000001 2023-11-25T04:23:46.445 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.445 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.445 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:46.445 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.106055+0000", 2023-11-25T04:23:46.445 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0566000000000003e-05 2023-11-25T04:23:46.445 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.445 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.445 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:46.445 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.106100+0000", 2023-11-25T04:23:46.445 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5160999999999997e-05 2023-11-25T04:23:46.445 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:46.445 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:46.446 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:46.446 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.446 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.446 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:43 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2023-11-25T04:23:46.446 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:46.446 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 39.245511759999999, 2023-11-25T04:23:46.446 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001751166, 2023-11-25T04:23:46.446 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:46.446 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:46.446 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:46.446 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.446 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:46.447 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:46.447 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:46.447 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.447 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.447 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:46.447 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:46.447 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:46.447 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.447 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.447 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:46.447 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104539+0000", 2023-11-25T04:23:46.447 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9529999999999999e-06 2023-11-25T04:23:46.447 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.448 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.448 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:46.448 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104557+0000", 2023-11-25T04:23:46.448 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7464000000000001e-05 2023-11-25T04:23:46.448 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.448 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.448 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:46.448 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104590+0000", 2023-11-25T04:23:46.448 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2966000000000003e-05 2023-11-25T04:23:46.448 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.448 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.448 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:46.448 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104608+0000", 2023-11-25T04:23:46.448 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8193e-05 2023-11-25T04:23:46.449 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.449 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.449 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:46.449 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104767+0000", 2023-11-25T04:23:46.449 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015882 2023-11-25T04:23:46.449 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.449 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.449 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:46.449 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104909+0000", 2023-11-25T04:23:46.449 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014244999999999999 2023-11-25T04:23:46.449 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.449 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.449 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:46.449 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.105059+0000", 2023-11-25T04:23:46.450 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014964399999999999 2023-11-25T04:23:46.450 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.450 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.450 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:46.450 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.105701+0000", 2023-11-25T04:23:46.450 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00064201099999999995 2023-11-25T04:23:46.450 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.450 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.450 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:46.450 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106233+0000", 2023-11-25T04:23:46.450 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00053178500000000001 2023-11-25T04:23:46.450 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.450 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.450 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:46.451 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106265+0000", 2023-11-25T04:23:46.451 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1986000000000002e-05 2023-11-25T04:23:46.451 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.451 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.451 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:46.451 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106288+0000", 2023-11-25T04:23:46.451 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2894000000000001e-05 2023-11-25T04:23:46.451 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:46.451 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:46.451 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:46.451 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.451 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.451 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:45 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e23)", 2023-11-25T04:23:46.452 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:23:46.452 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 34.245313719999999, 2023-11-25T04:23:46.452 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00098423299999999994, 2023-11-25T04:23:46.452 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:46.452 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:46.452 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:46.452 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.452 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:46.452 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:23:46.452 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:46.452 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.452 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.452 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:46.452 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:23:46.453 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:46.453 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.453 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.453 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:46.453 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104737+0000", 2023-11-25T04:23:46.453 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3309999999999998e-06 2023-11-25T04:23:46.453 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.453 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.453 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:46.453 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104748+0000", 2023-11-25T04:23:46.453 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1080999999999999e-05 2023-11-25T04:23:46.453 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.453 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.454 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:46.454 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104771+0000", 2023-11-25T04:23:46.454 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2707000000000001e-05 2023-11-25T04:23:46.454 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.454 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.454 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:46.454 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104800+0000", 2023-11-25T04:23:46.454 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9595e-05 2023-11-25T04:23:46.454 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.454 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.454 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:46.454 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104890+0000", 2023-11-25T04:23:46.454 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.9808999999999993e-05 2023-11-25T04:23:46.455 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.455 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.455 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:46.455 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105004+0000", 2023-11-25T04:23:46.455 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000114508 2023-11-25T04:23:46.455 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.455 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.455 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:46.455 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105634+0000", 2023-11-25T04:23:46.455 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00062985799999999996 2023-11-25T04:23:46.455 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.455 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.455 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:46.455 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105671+0000", 2023-11-25T04:23:46.456 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6640000000000002e-05 2023-11-25T04:23:46.456 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:46.456 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:46.456 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:46.456 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105719+0000", 2023-11-25T04:23:46.456 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7704000000000003e-05 2023-11-25T04:23:46.456 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:46.456 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:46.456 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:46.456 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:46.456 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:46.456 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:46.458 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:23:46.458 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:46.456+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:46.558 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:46.557+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:46.566 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:46.566 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:23:46.566 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2023-11-25T04:23:46.566 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:46.576 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:23:46.659 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:46.658+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:46.681 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:46.681 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:23:46.681 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2023-11-25T04:23:46.681 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2023-11-25T04:23:46.681 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:46.690 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:23:46.775 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:46.774+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:46.838 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:46.838 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2023-11-25T04:23:46.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2023-11-25T04:23:46.838 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2023-11-25T04:23:46.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.838 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:155 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'109, mlcod=11'105)", 2023-11-25T04:23:46.838 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:46.838 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 69.050694858, 2023-11-25T04:23:46.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061719019999999999, 2023-11-25T04:23:46.839 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:46.839 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:46.839 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:46.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:46.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:46.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:46.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:46.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:46.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:46.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:46.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784884+0000", 2023-11-25T04:23:46.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3099999999999999e-07 2023-11-25T04:23:46.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:46.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784942+0000", 2023-11-25T04:23:46.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8570000000000003e-05 2023-11-25T04:23:46.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:46.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784949+0000", 2023-11-25T04:23:46.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0310000000000001e-06 2023-11-25T04:23:46.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:46.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784953+0000", 2023-11-25T04:23:46.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6320000000000001e-06 2023-11-25T04:23:46.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:46.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784993+0000", 2023-11-25T04:23:46.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9737000000000003e-05 2023-11-25T04:23:46.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:46.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785011+0000", 2023-11-25T04:23:46.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8349999999999999e-05 2023-11-25T04:23:46.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:46.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791024+0000", 2023-11-25T04:23:46.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006012997 2023-11-25T04:23:46.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:46.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791055+0000", 2023-11-25T04:23:46.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1254000000000003e-05 2023-11-25T04:23:46.843 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:46.844 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:46.844 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:46.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.844 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:156 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'110, mlcod=11'105)", 2023-11-25T04:23:46.844 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:46.844 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 69.050108808999994, 2023-11-25T04:23:46.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005616116, 2023-11-25T04:23:46.845 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:46.845 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:46.845 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:46.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.845 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:46.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:46.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:46.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.845 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:46.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:46.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:46.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:46.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785470+0000", 2023-11-25T04:23:46.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0699999999999998e-07 2023-11-25T04:23:46.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:46.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785474+0000", 2023-11-25T04:23:46.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.048e-06 2023-11-25T04:23:46.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:46.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785480+0000", 2023-11-25T04:23:46.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7729999999999996e-06 2023-11-25T04:23:46.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.848 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:46.848 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785483+0000", 2023-11-25T04:23:46.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1970000000000001e-06 2023-11-25T04:23:46.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.848 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:46.848 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787936+0000", 2023-11-25T04:23:46.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024529040000000001 2023-11-25T04:23:46.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:46.849 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787947+0000", 2023-11-25T04:23:46.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1669e-05 2023-11-25T04:23:46.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:46.849 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791057+0000", 2023-11-25T04:23:46.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031095200000000002 2023-11-25T04:23:46.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.850 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:46.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791086+0000", 2023-11-25T04:23:46.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8697999999999999e-05 2023-11-25T04:23:46.850 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:46.850 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:46.850 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:46.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.850 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:157 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'111, mlcod=11'106)", 2023-11-25T04:23:46.850 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:46.851 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 69.049516251, 2023-11-25T04:23:46.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050590280000000001, 2023-11-25T04:23:46.851 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:46.851 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:46.851 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:46.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.851 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:46.851 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:46.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:46.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.851 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:46.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:46.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:46.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:46.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786063+0000", 2023-11-25T04:23:46.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2600000000000002e-07 2023-11-25T04:23:46.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:46.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786069+0000", 2023-11-25T04:23:46.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8839999999999999e-06 2023-11-25T04:23:46.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:46.853 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786079+0000", 2023-11-25T04:23:46.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0124e-05 2023-11-25T04:23:46.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:46.853 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786085+0000", 2023-11-25T04:23:46.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6660000000000002e-06 2023-11-25T04:23:46.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.854 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:46.854 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788160+0000", 2023-11-25T04:23:46.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020747669999999999 2023-11-25T04:23:46.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.854 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:46.854 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788180+0000", 2023-11-25T04:23:46.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0208999999999999e-05 2023-11-25T04:23:46.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.854 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:46.855 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791087+0000", 2023-11-25T04:23:46.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029064630000000002 2023-11-25T04:23:46.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.855 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:46.855 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791121+0000", 2023-11-25T04:23:46.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4289000000000003e-05 2023-11-25T04:23:46.855 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:46.855 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:46.855 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:46.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.856 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:158 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'112, mlcod=11'106)", 2023-11-25T04:23:46.856 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:46.856 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 69.049099858000005, 2023-11-25T04:23:46.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046663030000000001, 2023-11-25T04:23:46.856 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:46.856 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:46.856 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:46.856 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.856 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:46.857 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:46.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:46.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.857 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:46.857 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:46.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:46.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.857 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:46.857 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786479+0000", 2023-11-25T04:23:46.858 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2799999999999996e-07 2023-11-25T04:23:46.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.858 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.858 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:46.858 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786498+0000", 2023-11-25T04:23:46.858 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9069e-05 2023-11-25T04:23:46.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.858 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.858 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:46.858 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786508+0000", 2023-11-25T04:23:46.858 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5810000000000003e-06 2023-11-25T04:23:46.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.859 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.859 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:46.859 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786512+0000", 2023-11-25T04:23:46.859 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4889999999999997e-06 2023-11-25T04:23:46.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.859 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.859 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:46.859 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788438+0000", 2023-11-25T04:23:46.860 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019257059999999999 2023-11-25T04:23:46.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.860 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.860 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:46.860 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788449+0000", 2023-11-25T04:23:46.860 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1419e-05 2023-11-25T04:23:46.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.860 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.860 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:46.860 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791122+0000", 2023-11-25T04:23:46.860 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002672822 2023-11-25T04:23:46.861 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.861 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.861 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:46.861 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791145+0000", 2023-11-25T04:23:46.861 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2589e-05 2023-11-25T04:23:46.862 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:46.862 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:46.862 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:46.862 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.862 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.862 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:206 1.0 e12/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 12'152, mlcod=12'148)", 2023-11-25T04:23:46.862 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:46.862 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 68.537331387999998, 2023-11-25T04:23:46.863 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046728189999999999, 2023-11-25T04:23:46.863 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:46.863 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:46.863 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:46.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.863 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:46.863 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:46.863 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:46.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.863 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:46.864 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:46.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:46.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.864 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.864 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:46.864 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298248+0000", 2023-11-25T04:23:46.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3600000000000003e-07 2023-11-25T04:23:46.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.864 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.864 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:46.865 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298278+0000", 2023-11-25T04:23:46.865 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0474e-05 2023-11-25T04:23:46.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.865 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.865 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:46.865 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298288+0000", 2023-11-25T04:23:46.865 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0156000000000001e-05 2023-11-25T04:23:46.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.865 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.865 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:46.866 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298293+0000", 2023-11-25T04:23:46.866 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7400000000000004e-06 2023-11-25T04:23:46.866 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.866 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.866 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:46.866 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298333+0000", 2023-11-25T04:23:46.866 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9666000000000003e-05 2023-11-25T04:23:46.866 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.867 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.867 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:46.867 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298351+0000", 2023-11-25T04:23:46.867 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8544999999999999e-05 2023-11-25T04:23:46.867 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.867 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.867 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:46.867 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302891+0000", 2023-11-25T04:23:46.867 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045398519999999996 2023-11-25T04:23:46.867 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.867 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.868 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:46.868 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302920+0000", 2023-11-25T04:23:46.868 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8750000000000001e-05 2023-11-25T04:23:46.868 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:46.868 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:46.868 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:46.868 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.868 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.868 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:14 3.15 3:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:46.868 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:46.869 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 59.802197030999999, 2023-11-25T04:23:46.869 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010990437, 2023-11-25T04:23:46.869 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:46.869 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:46.869 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:46.869 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.869 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:46.869 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:46.869 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:46.869 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.869 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.869 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:46.869 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:46.870 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:46.870 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.870 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.870 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:46.870 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033383+0000", 2023-11-25T04:23:46.870 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9870000000000002e-06 2023-11-25T04:23:46.870 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.870 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.870 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:46.870 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033398+0000", 2023-11-25T04:23:46.871 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4569999999999999e-05 2023-11-25T04:23:46.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.871 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.871 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:46.871 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033411+0000", 2023-11-25T04:23:46.871 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3543999999999999e-05 2023-11-25T04:23:46.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.871 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.871 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:46.872 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033423+0000", 2023-11-25T04:23:46.872 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1905999999999999e-05 2023-11-25T04:23:46.872 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.872 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.872 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:46.872 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034433+0000", 2023-11-25T04:23:46.872 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00101022 2023-11-25T04:23:46.872 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.872 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.873 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:46.873 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034539+0000", 2023-11-25T04:23:46.873 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010597199999999999 2023-11-25T04:23:46.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.873 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.873 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:46.873 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034667+0000", 2023-11-25T04:23:46.873 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012765500000000001 2023-11-25T04:23:46.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.873 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.873 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:46.874 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037803+0000", 2023-11-25T04:23:46.874 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031362220000000001 2023-11-25T04:23:46.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.874 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:46.874 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:23:46.874 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0064964669999999997 2023-11-25T04:23:46.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.874 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:46.874 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044331+0000", 2023-11-25T04:23:46.875 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1544999999999997e-05 2023-11-25T04:23:46.875 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.875 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.875 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:46.875 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044372+0000", 2023-11-25T04:23:46.875 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0349000000000003e-05 2023-11-25T04:23:46.875 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:46.875 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:46.875 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:46.875 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.875 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.876 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:15 3.15 3:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:46.876 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:46.876 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 59.801537381000003, 2023-11-25T04:23:46.876 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010464362, 2023-11-25T04:23:46.876 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:46.876 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:46.876 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:46.876 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.876 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:46.876 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:46.877 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:46.877 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.877 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.877 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:46.877 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:46.877 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:46.877 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.877 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.877 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:46.877 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034042+0000", 2023-11-25T04:23:46.877 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0999999999999997e-07 2023-11-25T04:23:46.877 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.878 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.878 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:46.878 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034048+0000", 2023-11-25T04:23:46.878 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0390000000000001e-06 2023-11-25T04:23:46.878 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.878 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.878 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:46.878 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034059+0000", 2023-11-25T04:23:46.878 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1182999999999999e-05 2023-11-25T04:23:46.878 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.878 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.878 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:46.879 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034067+0000", 2023-11-25T04:23:46.879 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1559999999999995e-06 2023-11-25T04:23:46.879 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.879 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.879 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:46.879 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035005+0000", 2023-11-25T04:23:46.879 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00093747399999999999 2023-11-25T04:23:46.879 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.879 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.879 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:46.879 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035077+0000", 2023-11-25T04:23:46.880 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2747e-05 2023-11-25T04:23:46.880 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.880 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.880 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:46.880 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035219+0000", 2023-11-25T04:23:46.880 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000141877 2023-11-25T04:23:46.880 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.880 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.880 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:46.880 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037810+0000", 2023-11-25T04:23:46.880 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002590752 2023-11-25T04:23:46.881 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.881 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.881 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:46.881 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044437+0000", 2023-11-25T04:23:46.881 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066270110000000004 2023-11-25T04:23:46.881 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.881 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.881 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:46.881 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044481+0000", 2023-11-25T04:23:46.881 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4106000000000001e-05 2023-11-25T04:23:46.881 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.882 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.882 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:46.882 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044505+0000", 2023-11-25T04:23:46.882 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4207e-05 2023-11-25T04:23:46.882 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:46.882 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:46.882 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:46.882 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.882 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.882 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:16 3.1d 3:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:46.882 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:46.882 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 59.801495533000001, 2023-11-25T04:23:46.883 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010413923, 2023-11-25T04:23:46.883 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:46.883 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:46.883 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:46.883 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.883 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:46.883 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:46.883 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:46.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.883 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.884 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:46.884 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:46.884 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:46.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.884 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.884 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:46.884 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:46.884 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4499999999999997e-07 2023-11-25T04:23:46.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.884 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.884 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:46.885 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034087+0000", 2023-11-25T04:23:46.885 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9770000000000002e-06 2023-11-25T04:23:46.885 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.885 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.885 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:46.885 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034094+0000", 2023-11-25T04:23:46.885 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8809999999999998e-06 2023-11-25T04:23:46.885 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.885 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.885 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:46.885 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034100+0000", 2023-11-25T04:23:46.886 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8440000000000003e-06 2023-11-25T04:23:46.886 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.886 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.886 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:46.886 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034993+0000", 2023-11-25T04:23:46.886 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00089336799999999996 2023-11-25T04:23:46.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.887 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.887 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:46.887 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035099+0000", 2023-11-25T04:23:46.887 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106037 2023-11-25T04:23:46.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.887 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.887 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:46.887 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035276+0000", 2023-11-25T04:23:46.888 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017631799999999999 2023-11-25T04:23:46.888 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.888 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.888 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:46.888 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037815+0000", 2023-11-25T04:23:46.888 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025395959999999999 2023-11-25T04:23:46.888 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.888 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.888 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:46.888 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044420+0000", 2023-11-25T04:23:46.889 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066045749999999997 2023-11-25T04:23:46.889 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.889 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.889 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:46.889 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044463+0000", 2023-11-25T04:23:46.889 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3205999999999999e-05 2023-11-25T04:23:46.889 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.889 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.889 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:46.889 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044497+0000", 2023-11-25T04:23:46.889 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3476000000000003e-05 2023-11-25T04:23:46.890 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:46.890 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:46.890 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:46.890 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.890 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.890 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:17 3.1d 3:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:46.890 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:46.890 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 59.801470324, 2023-11-25T04:23:46.890 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01127356, 2023-11-25T04:23:46.890 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:46.890 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:46.891 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:46.891 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.891 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:46.891 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:46.891 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:46.891 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.891 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.891 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:46.891 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:46.891 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:46.891 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.892 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.892 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:46.892 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:46.892 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1100000000000001e-07 2023-11-25T04:23:46.892 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.892 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.892 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:46.892 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034112+0000", 2023-11-25T04:23:46.892 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8600000000000003e-06 2023-11-25T04:23:46.893 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.893 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.893 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:46.893 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034120+0000", 2023-11-25T04:23:46.893 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7700000000000001e-06 2023-11-25T04:23:46.893 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.893 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.893 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:46.893 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034127+0000", 2023-11-25T04:23:46.893 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0999999999999998e-06 2023-11-25T04:23:46.893 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.894 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.894 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:46.894 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035601+0000", 2023-11-25T04:23:46.894 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014739410000000001 2023-11-25T04:23:46.894 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.894 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.894 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:46.894 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035685+0000", 2023-11-25T04:23:46.894 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3677999999999994e-05 2023-11-25T04:23:46.894 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.894 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:46.895 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035827+0000", 2023-11-25T04:23:46.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014232599999999999 2023-11-25T04:23:46.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:46.895 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038725+0000", 2023-11-25T04:23:46.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028982779999999998 2023-11-25T04:23:46.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:46.896 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045323+0000", 2023-11-25T04:23:46.896 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0065980780000000003 2023-11-25T04:23:46.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.896 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:46.896 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045354+0000", 2023-11-25T04:23:46.896 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0463e-05 2023-11-25T04:23:46.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.896 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:46.897 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045381+0000", 2023-11-25T04:23:46.897 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7654999999999999e-05 2023-11-25T04:23:46.897 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:46.897 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:46.897 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:46.897 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.897 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.897 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:22 3.e 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:46.898 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:46.898 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 59.801443911, 2023-11-25T04:23:46.898 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010133422, 2023-11-25T04:23:46.898 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:46.898 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:46.898 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:46.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.898 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:46.898 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:46.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:46.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.899 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:46.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:46.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:46.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.899 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:46.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034135+0000", 2023-11-25T04:23:46.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-07 2023-11-25T04:23:46.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.900 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:46.900 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034138+0000", 2023-11-25T04:23:46.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6009999999999999e-06 2023-11-25T04:23:46.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.900 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:46.900 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034145+0000", 2023-11-25T04:23:46.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1169999999999998e-06 2023-11-25T04:23:46.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.901 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:46.901 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034150+0000", 2023-11-25T04:23:46.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9080000000000003e-06 2023-11-25T04:23:46.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.901 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:46.901 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034209+0000", 2023-11-25T04:23:46.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8149000000000002e-05 2023-11-25T04:23:46.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.902 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:46.902 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034415+0000", 2023-11-25T04:23:46.902 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020637699999999999 2023-11-25T04:23:46.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.902 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:46.902 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034584+0000", 2023-11-25T04:23:46.902 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016870399999999999 2023-11-25T04:23:46.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.903 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:46.903 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.036979+0000", 2023-11-25T04:23:46.903 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023957509999999998 2023-11-25T04:23:46.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.903 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.903 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:46.903 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044161+0000", 2023-11-25T04:23:46.903 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0071811590000000003 2023-11-25T04:23:46.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.903 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.903 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:46.903 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044224+0000", 2023-11-25T04:23:46.904 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3454000000000006e-05 2023-11-25T04:23:46.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.904 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:46.904 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044268+0000", 2023-11-25T04:23:46.904 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3761999999999999e-05 2023-11-25T04:23:46.904 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:46.904 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:46.904 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:46.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.905 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:23 3.e 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:46.905 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:46.905 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 59.801412501999998, 2023-11-25T04:23:46.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010281676, 2023-11-25T04:23:46.905 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:46.905 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:46.905 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:46.905 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.905 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:46.905 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:46.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:46.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.906 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.906 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:46.906 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:46.906 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:46.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.906 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.906 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:46.906 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:46.907 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5799999999999999e-07 2023-11-25T04:23:46.907 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.907 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.907 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:46.907 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034169+0000", 2023-11-25T04:23:46.907 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1879999999999998e-06 2023-11-25T04:23:46.907 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.907 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.907 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:46.907 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034177+0000", 2023-11-25T04:23:46.908 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7020000000000001e-06 2023-11-25T04:23:46.908 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.908 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.908 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:46.908 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034184+0000", 2023-11-25T04:23:46.908 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3910000000000003e-06 2023-11-25T04:23:46.908 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.908 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.908 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:46.908 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034924+0000", 2023-11-25T04:23:46.908 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00074032999999999998 2023-11-25T04:23:46.908 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.909 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.909 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:46.909 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034991+0000", 2023-11-25T04:23:46.909 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7347000000000004e-05 2023-11-25T04:23:46.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.909 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.909 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:46.909 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035096+0000", 2023-11-25T04:23:46.909 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010488 2023-11-25T04:23:46.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.909 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.909 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:46.910 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037588+0000", 2023-11-25T04:23:46.910 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024918309999999999 2023-11-25T04:23:46.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.910 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.910 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:46.910 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044367+0000", 2023-11-25T04:23:46.910 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067787589999999997 2023-11-25T04:23:46.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.910 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.910 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:46.911 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044415+0000", 2023-11-25T04:23:46.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7803000000000003e-05 2023-11-25T04:23:46.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.911 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:46.911 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044447+0000", 2023-11-25T04:23:46.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2886999999999999e-05 2023-11-25T04:23:46.911 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:46.911 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:46.911 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:46.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.912 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:30 3.15 3:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:46.912 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:46.912 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 59.801354062000001, 2023-11-25T04:23:46.912 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010365070000000001, 2023-11-25T04:23:46.912 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:46.912 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:46.912 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:46.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.912 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:46.912 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:46.913 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:46.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.913 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.913 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:46.913 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:46.913 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:46.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.913 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.913 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:46.913 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034225+0000", 2023-11-25T04:23:46.913 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7400000000000002e-07 2023-11-25T04:23:46.914 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.914 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.914 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:46.914 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034229+0000", 2023-11-25T04:23:46.914 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3100000000000002e-06 2023-11-25T04:23:46.914 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.914 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.914 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:46.914 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034239+0000", 2023-11-25T04:23:46.914 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9140000000000003e-06 2023-11-25T04:23:46.914 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.914 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.915 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:46.915 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034246+0000", 2023-11-25T04:23:46.915 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0149999999999998e-06 2023-11-25T04:23:46.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.915 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:46.915 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035572+0000", 2023-11-25T04:23:46.915 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001325655 2023-11-25T04:23:46.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.915 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:46.915 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035730+0000", 2023-11-25T04:23:46.916 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015827899999999999 2023-11-25T04:23:46.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.916 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:46.916 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035857+0000", 2023-11-25T04:23:46.916 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012644699999999999 2023-11-25T04:23:46.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.916 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:46.916 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038737+0000", 2023-11-25T04:23:46.916 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002880077 2023-11-25T04:23:46.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.918 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:46.918 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044540+0000", 2023-11-25T04:23:46.918 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058034239999999997 2023-11-25T04:23:46.918 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.918 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.918 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:46.918 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044569+0000", 2023-11-25T04:23:46.918 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8388999999999999e-05 2023-11-25T04:23:46.918 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.918 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.918 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:46.918 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044589+0000", 2023-11-25T04:23:46.919 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0686000000000002e-05 2023-11-25T04:23:46.919 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:46.919 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:46.919 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:46.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.919 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.919 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:33 3.1b 3:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:46.919 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:46.919 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 59.801308564000003, 2023-11-25T04:23:46.919 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099754049999999997, 2023-11-25T04:23:46.919 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:46.919 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:23:46.919 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:46.920 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.920 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:46.920 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:46.920 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:46.920 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.920 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.920 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:46.920 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:46.920 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:46.920 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.920 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.921 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:46.921 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:46.921 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4799999999999998e-07 2023-11-25T04:23:46.921 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.921 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.921 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:46.921 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034273+0000", 2023-11-25T04:23:46.921 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7819999999999998e-06 2023-11-25T04:23:46.921 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.921 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.921 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:46.921 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034285+0000", 2023-11-25T04:23:46.921 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2058e-05 2023-11-25T04:23:46.922 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.922 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.922 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:46.922 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034291+0000", 2023-11-25T04:23:46.922 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0680000000000002e-06 2023-11-25T04:23:46.922 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.922 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.922 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:46.922 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034370+0000", 2023-11-25T04:23:46.922 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8452e-05 2023-11-25T04:23:46.922 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.923 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.923 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:46.923 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034477+0000", 2023-11-25T04:23:46.923 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107471 2023-11-25T04:23:46.923 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.923 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.923 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:46.923 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044245+0000", 2023-11-25T04:23:46.923 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0097679259999999997 2023-11-25T04:23:46.923 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:46.923 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:46.923 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:46.923 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.924 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.924 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:36 3.1a 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:46.924 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:46.924 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 59.801261891000003, 2023-11-25T04:23:46.924 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058179099999999999, 2023-11-25T04:23:46.924 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:46.924 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:23:46.924 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:46.924 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.924 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:46.924 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:46.924 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:46.924 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.925 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:46.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:46.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:46.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.925 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:46.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034317+0000", 2023-11-25T04:23:46.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1999999999999999e-07 2023-11-25T04:23:46.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.925 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:46.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034321+0000", 2023-11-25T04:23:46.926 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.233e-06 2023-11-25T04:23:46.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.926 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.926 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:46.926 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034337+0000", 2023-11-25T04:23:46.926 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5308999999999999e-05 2023-11-25T04:23:46.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.926 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.926 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:46.926 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034354+0000", 2023-11-25T04:23:46.926 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7411999999999999e-05 2023-11-25T04:23:46.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.926 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.927 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:46.927 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034465+0000", 2023-11-25T04:23:46.927 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011154699999999999 2023-11-25T04:23:46.927 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.927 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.927 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:46.927 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034560+0000", 2023-11-25T04:23:46.927 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4186999999999998e-05 2023-11-25T04:23:46.927 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.927 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.927 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:46.927 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.040134+0000", 2023-11-25T04:23:46.927 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055746019999999997 2023-11-25T04:23:46.927 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:46.928 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:46.928 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:46.928 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.928 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.928 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:9 3.10 e18/16 3:09401a81:::601.00000000:head v 18'4, mlcod=0'0)", 2023-11-25T04:23:46.928 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:46.928 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 59.792367583000001, 2023-11-25T04:23:46.928 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061641020000000003, 2023-11-25T04:23:46.928 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:46.928 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:46.928 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:46.928 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.929 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:46.929 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:46.929 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:46.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.929 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:46.929 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:46.929 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:46.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.929 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:46.929 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:46.929 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5300000000000001e-07 2023-11-25T04:23:46.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.930 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.930 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:46.930 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043222+0000", 2023-11-25T04:23:46.930 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.082e-05 2023-11-25T04:23:46.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.930 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.930 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:46.930 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043228+0000", 2023-11-25T04:23:46.930 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6699999999999997e-06 2023-11-25T04:23:46.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.930 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.931 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:46.931 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043236+0000", 2023-11-25T04:23:46.931 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8189999999999992e-06 2023-11-25T04:23:46.931 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.931 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.931 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:46.931 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043507+0000", 2023-11-25T04:23:46.931 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000271012 2023-11-25T04:23:46.931 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.931 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.931 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:46.931 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043530+0000", 2023-11-25T04:23:46.932 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2659000000000001e-05 2023-11-25T04:23:46.932 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.932 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.932 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:46.932 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049321+0000", 2023-11-25T04:23:46.932 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057915129999999999 2023-11-25T04:23:46.932 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.932 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.932 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:46.932 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049375+0000", 2023-11-25T04:23:46.932 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3455999999999998e-05 2023-11-25T04:23:46.932 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:46.933 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:46.933 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:46.933 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.933 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.933 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:1 3.14 e18/16 3:292cf221:::200.00000000:head v 18'3, mlcod=0'0)", 2023-11-25T04:23:46.933 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:46.933 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 59.792330870000001, 2023-11-25T04:23:46.933 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0060983599999999997, 2023-11-25T04:23:46.933 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:46.933 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:46.933 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:46.933 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.933 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:46.934 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:46.934 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:46.934 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.934 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.934 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:46.934 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:46.934 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:46.934 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.934 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.934 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:46.934 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043248+0000", 2023-11-25T04:23:46.934 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9399999999999995e-07 2023-11-25T04:23:46.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.935 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.935 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:46.935 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043252+0000", 2023-11-25T04:23:46.935 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1799999999999998e-06 2023-11-25T04:23:46.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.935 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.935 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:46.935 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043259+0000", 2023-11-25T04:23:46.935 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0049999999999998e-06 2023-11-25T04:23:46.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.935 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.936 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:46.936 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043265+0000", 2023-11-25T04:23:46.936 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0859999999999998e-06 2023-11-25T04:23:46.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.936 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:46.936 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043311+0000", 2023-11-25T04:23:46.936 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6082999999999999e-05 2023-11-25T04:23:46.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.936 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:46.936 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043330+0000", 2023-11-25T04:23:46.936 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8556e-05 2023-11-25T04:23:46.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.937 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.937 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:46.937 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049303+0000", 2023-11-25T04:23:46.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059733470000000004 2023-11-25T04:23:46.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.937 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.937 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:46.937 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049346+0000", 2023-11-25T04:23:46.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2608999999999998e-05 2023-11-25T04:23:46.937 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:46.937 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:46.937 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:46.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.938 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.938 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:3 3.14 e18/16 3:287034c4:::1.00000000.inode:head v 18'4, mlcod=0'0)", 2023-11-25T04:23:46.938 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:46.938 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 59.791814363999997, 2023-11-25T04:23:46.938 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057040090000000003, 2023-11-25T04:23:46.938 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:46.938 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:46.938 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:46.938 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.938 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:46.938 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:46.938 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:46.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.939 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:46.939 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:46.939 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:46.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.939 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:46.939 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:46.939 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9799999999999999e-07 2023-11-25T04:23:46.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.940 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:46.940 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043785+0000", 2023-11-25T04:23:46.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0721999999999999e-05 2023-11-25T04:23:46.940 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.940 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.940 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:46.940 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043793+0000", 2023-11-25T04:23:46.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5839999999999995e-06 2023-11-25T04:23:46.940 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.940 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.940 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:46.940 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043803+0000", 2023-11-25T04:23:46.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0729999999999994e-06 2023-11-25T04:23:46.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.941 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:46.941 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043843+0000", 2023-11-25T04:23:46.941 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0747999999999997e-05 2023-11-25T04:23:46.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.941 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:46.941 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043856+0000", 2023-11-25T04:23:46.941 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2697000000000001e-05 2023-11-25T04:23:46.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.941 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:46.942 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049426+0000", 2023-11-25T04:23:46.942 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055704409999999998 2023-11-25T04:23:46.942 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.942 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.942 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:46.942 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049468+0000", 2023-11-25T04:23:46.942 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1446000000000003e-05 2023-11-25T04:23:46.942 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:46.942 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:46.942 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:46.942 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.942 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.943 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "pg_update_log_missing(3.14 epoch 18/16 rep_tid 171 entries 18'5 (0'0) error 3:2e2fa760:::200.00000001:head by mds.0.3:32 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:23:46.943 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:46.943 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 59.791653400999998, 2023-11-25T04:23:46.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005556938, 2023-11-25T04:23:46.943 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:46.943 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2023-11-25T04:23:46.943 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:46.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.943 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:46.943 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:46.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:46.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:46.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:46.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:46.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:46.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:46.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.03e-07 2023-11-25T04:23:46.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:46.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043931+0000", 2023-11-25T04:23:46.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1079999999999998e-06 2023-11-25T04:23:46.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:46.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043944+0000", 2023-11-25T04:23:46.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2852e-05 2023-11-25T04:23:46.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:46.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043955+0000", 2023-11-25T04:23:46.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1037e-05 2023-11-25T04:23:46.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.946 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:46.946 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044088+0000", 2023-11-25T04:23:46.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013269599999999999 2023-11-25T04:23:46.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.946 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:46.946 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049482+0000", 2023-11-25T04:23:46.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053940419999999999 2023-11-25T04:23:46.946 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:46.946 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:46.947 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:46.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.947 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:10 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'5, mlcod=0'0)", 2023-11-25T04:23:46.947 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:46.947 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 59.791563373000002, 2023-11-25T04:23:46.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054493099999999997, 2023-11-25T04:23:46.947 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:46.947 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:46.947 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:46.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.947 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:46.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:46.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:46.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.948 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:46.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:46.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:46.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.948 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:46.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:46.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6100000000000002e-07 2023-11-25T04:23:46.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:46.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044019+0000", 2023-11-25T04:23:46.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.839e-06 2023-11-25T04:23:46.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:46.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044026+0000", 2023-11-25T04:23:46.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5869999999999998e-06 2023-11-25T04:23:46.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:46.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044031+0000", 2023-11-25T04:23:46.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4909999999999996e-06 2023-11-25T04:23:46.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:46.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044070+0000", 2023-11-25T04:23:46.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8829999999999999e-05 2023-11-25T04:23:46.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:46.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044081+0000", 2023-11-25T04:23:46.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1447999999999999e-05 2023-11-25T04:23:46.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:46.951 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049406+0000", 2023-11-25T04:23:46.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053246480000000004 2023-11-25T04:23:46.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:46.951 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049464+0000", 2023-11-25T04:23:46.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8205999999999998e-05 2023-11-25T04:23:46.951 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:46.951 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:46.951 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:46.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.952 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:11 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'6, mlcod=0'0)", 2023-11-25T04:23:46.952 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:46.952 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 59.791362108999998, 2023-11-25T04:23:46.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052673499999999996, 2023-11-25T04:23:46.952 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:46.952 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:46.952 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:46.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.952 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:46.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:46.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:46.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:46.953 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:46.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:46.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:46.953 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:46.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0199999999999998e-07 2023-11-25T04:23:46.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:46.953 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044220+0000", 2023-11-25T04:23:46.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7720000000000001e-06 2023-11-25T04:23:46.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.954 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:46.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044226+0000", 2023-11-25T04:23:46.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8579999999999996e-06 2023-11-25T04:23:46.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.954 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:46.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044232+0000", 2023-11-25T04:23:46.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1700000000000002e-06 2023-11-25T04:23:46.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:46.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044274+0000", 2023-11-25T04:23:46.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1254999999999997e-05 2023-11-25T04:23:46.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:46.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:23:46.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6880999999999999e-05 2023-11-25T04:23:46.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:46.956 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049466+0000", 2023-11-25T04:23:46.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051659219999999999 2023-11-25T04:23:46.956 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:46.956 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:46.956 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:46.956 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049483+0000", 2023-11-25T04:23:46.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.719e-05 2023-11-25T04:23:46.956 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:46.956 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:46.956 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:46.956 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:46.956 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:46.956 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:46.958 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:23:46.958 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:46.956+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:46.959 INFO:teuthology.orchestra.run.smithi067.stderr:noscrub is unset 2023-11-25T04:23:47.059 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:47.058+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:47.073 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:47.073 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:23:47.074 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2023-11-25T04:23:47.074 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:47.082 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:23:47.160 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:47.158+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:47.190 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:47.190 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:23:47.191 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2023-11-25T04:23:47.191 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2023-11-25T04:23:47.191 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:47.197 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:23:47.261 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:47.259+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:47.346 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:47.346 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2023-11-25T04:23:47.346 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2023-11-25T04:23:47.346 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2023-11-25T04:23:47.346 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.346 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:47.346 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:47.346 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.311473626999998, 2023-11-25T04:23:47.347 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01072943, 2023-11-25T04:23:47.347 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:47.347 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:47.347 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:47.347 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.347 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:47.347 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:47.347 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:47.347 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.347 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.347 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:47.347 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:47.348 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:47.348 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.348 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.348 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:47.348 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031829+0000", 2023-11-25T04:23:47.348 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.666e-06 2023-11-25T04:23:47.348 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.348 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.348 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:47.348 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031839+0000", 2023-11-25T04:23:47.349 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9490000000000008e-06 2023-11-25T04:23:47.349 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.349 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.349 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:47.349 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031857+0000", 2023-11-25T04:23:47.349 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8093000000000001e-05 2023-11-25T04:23:47.349 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.349 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.349 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:47.349 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031882+0000", 2023-11-25T04:23:47.349 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4294000000000002e-05 2023-11-25T04:23:47.350 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.350 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.350 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:47.350 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033130+0000", 2023-11-25T04:23:47.350 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001248264 2023-11-25T04:23:47.350 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.350 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.350 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:47.350 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033426+0000", 2023-11-25T04:23:47.350 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029620999999999998 2023-11-25T04:23:47.351 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.351 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.351 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:47.351 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033814+0000", 2023-11-25T04:23:47.351 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000387742 2023-11-25T04:23:47.351 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.351 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.351 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:47.351 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036793+0000", 2023-11-25T04:23:47.351 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002979598 2023-11-25T04:23:47.351 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.351 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.352 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:47.352 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042455+0000", 2023-11-25T04:23:47.352 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0056619260000000003 2023-11-25T04:23:47.352 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.352 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.352 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:47.352 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042502+0000", 2023-11-25T04:23:47.352 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7139999999999999e-05 2023-11-25T04:23:47.352 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.352 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.352 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:47.353 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042557+0000", 2023-11-25T04:23:47.353 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4548e-05 2023-11-25T04:23:47.353 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:47.353 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:47.353 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:47.353 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.353 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.353 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:47.353 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:47.353 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.310275091999998, 2023-11-25T04:23:47.353 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011505202000000001, 2023-11-25T04:23:47.353 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:47.354 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:47.354 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:47.354 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.354 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:47.354 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:47.354 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:47.354 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.354 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.354 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:47.354 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:47.354 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:47.354 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.355 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.355 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:47.355 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033027+0000", 2023-11-25T04:23:47.355 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0699999999999999e-06 2023-11-25T04:23:47.355 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.355 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.355 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:47.355 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033035+0000", 2023-11-25T04:23:47.355 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0830000000000004e-06 2023-11-25T04:23:47.355 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.355 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.355 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:47.356 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033059+0000", 2023-11-25T04:23:47.356 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3975e-05 2023-11-25T04:23:47.356 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.356 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.356 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:47.356 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033482+0000", 2023-11-25T04:23:47.356 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042253700000000001 2023-11-25T04:23:47.356 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.356 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.356 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:47.357 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034520+0000", 2023-11-25T04:23:47.357 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001037991 2023-11-25T04:23:47.357 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.357 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.357 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:47.357 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034611+0000", 2023-11-25T04:23:47.357 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1113999999999995e-05 2023-11-25T04:23:47.357 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.357 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.357 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:47.357 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034769+0000", 2023-11-25T04:23:47.358 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000157641 2023-11-25T04:23:47.358 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.358 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.358 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:47.358 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038292+0000", 2023-11-25T04:23:47.358 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035231160000000002 2023-11-25T04:23:47.358 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.358 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.358 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:47.358 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044440+0000", 2023-11-25T04:23:47.359 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0061486980000000002 2023-11-25T04:23:47.359 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.360 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.360 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:47.360 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044495+0000", 2023-11-25T04:23:47.360 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4339000000000003e-05 2023-11-25T04:23:47.360 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.360 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.360 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:47.360 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044531+0000", 2023-11-25T04:23:47.360 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6637999999999997e-05 2023-11-25T04:23:47.360 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:47.360 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:47.360 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:47.361 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.361 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.361 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:47.361 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:47.361 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.309800738, 2023-11-25T04:23:47.361 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0057634039999999997, 2023-11-25T04:23:47.361 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:47.361 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:47.361 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:47.361 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.361 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:47.362 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:47.362 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:47.362 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.362 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.362 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:47.362 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:47.362 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:47.362 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.362 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.362 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:47.362 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:47.363 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7900000000000007e-07 2023-11-25T04:23:47.363 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.363 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.363 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:47.363 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033507+0000", 2023-11-25T04:23:47.363 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8900000000000004e-06 2023-11-25T04:23:47.363 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.363 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.363 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:47.363 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033519+0000", 2023-11-25T04:23:47.363 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1756000000000001e-05 2023-11-25T04:23:47.363 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.363 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.364 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:47.364 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033527+0000", 2023-11-25T04:23:47.364 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3610000000000001e-06 2023-11-25T04:23:47.364 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.364 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.364 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:47.364 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034451+0000", 2023-11-25T04:23:47.364 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00092372400000000001 2023-11-25T04:23:47.364 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.364 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.364 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:47.365 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034809+0000", 2023-11-25T04:23:47.365 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00035757699999999998 2023-11-25T04:23:47.365 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.365 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.365 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:23:47.365 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034965+0000", 2023-11-25T04:23:47.365 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015638699999999999 2023-11-25T04:23:47.365 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.365 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.365 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:47.365 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038299+0000", 2023-11-25T04:23:47.366 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0033336559999999999 2023-11-25T04:23:47.366 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.366 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.366 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:47.366 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039169+0000", 2023-11-25T04:23:47.366 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00087066400000000005 2023-11-25T04:23:47.366 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.366 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.366 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:47.366 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039226+0000", 2023-11-25T04:23:47.367 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6511999999999997e-05 2023-11-25T04:23:47.367 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.367 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.367 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:47.367 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039264+0000", 2023-11-25T04:23:47.367 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7898000000000002e-05 2023-11-25T04:23:47.367 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:47.367 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:47.367 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:47.367 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.367 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.367 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:25 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:47.368 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:47.368 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.309722153999999, 2023-11-25T04:23:47.368 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096656169999999996, 2023-11-25T04:23:47.368 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:47.368 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:47.368 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:47.368 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.368 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:47.368 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:47.368 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:47.368 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.369 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.369 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:47.369 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:47.369 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:47.369 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.369 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.369 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:47.369 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033580+0000", 2023-11-25T04:23:47.369 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8199999999999999e-07 2023-11-25T04:23:47.369 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.369 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.369 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:47.370 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033585+0000", 2023-11-25T04:23:47.370 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3680000000000001e-06 2023-11-25T04:23:47.370 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.370 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.370 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:47.370 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033600+0000", 2023-11-25T04:23:47.370 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4331000000000001e-05 2023-11-25T04:23:47.370 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.370 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.370 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:47.370 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033606+0000", 2023-11-25T04:23:47.370 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5780000000000004e-06 2023-11-25T04:23:47.371 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.371 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.371 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:47.371 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035747+0000", 2023-11-25T04:23:47.371 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0021406530000000002 2023-11-25T04:23:47.371 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.371 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.371 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:47.371 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035901+0000", 2023-11-25T04:23:47.372 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000153931 2023-11-25T04:23:47.372 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.372 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.372 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:23:47.372 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036193+0000", 2023-11-25T04:23:47.372 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029211299999999999 2023-11-25T04:23:47.372 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.372 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.372 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:47.372 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038655+0000", 2023-11-25T04:23:47.373 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0024621640000000002 2023-11-25T04:23:47.373 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.373 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.373 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:47.373 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043130+0000", 2023-11-25T04:23:47.373 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.004474698 2023-11-25T04:23:47.373 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.373 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.373 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:47.373 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043204+0000", 2023-11-25T04:23:47.373 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4091999999999995e-05 2023-11-25T04:23:47.373 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.374 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.374 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:47.374 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043245+0000", 2023-11-25T04:23:47.374 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0907e-05 2023-11-25T04:23:47.374 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:47.374 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:47.374 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:47.374 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.374 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.374 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:26 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:47.375 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:47.375 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.309686329999998, 2023-11-25T04:23:47.375 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011086945000000001, 2023-11-25T04:23:47.375 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:47.375 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:47.375 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:47.375 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.375 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:47.375 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:47.375 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:47.375 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.376 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.376 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:47.376 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:47.376 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:47.376 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.376 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.376 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:47.376 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:47.376 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1699999999999998e-07 2023-11-25T04:23:47.376 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.376 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.376 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:47.377 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033627+0000", 2023-11-25T04:23:47.377 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.163e-05 2023-11-25T04:23:47.377 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.377 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.377 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:47.377 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033636+0000", 2023-11-25T04:23:47.377 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6859999999999995e-06 2023-11-25T04:23:47.377 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.377 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.377 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:47.377 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033642+0000", 2023-11-25T04:23:47.378 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3149999999999997e-06 2023-11-25T04:23:47.378 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.378 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.378 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:47.378 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035414+0000", 2023-11-25T04:23:47.378 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0017716380000000001 2023-11-25T04:23:47.378 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.378 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.378 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:47.378 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035539+0000", 2023-11-25T04:23:47.378 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000125477 2023-11-25T04:23:47.378 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.379 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.379 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:47.379 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035672+0000", 2023-11-25T04:23:47.379 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000133316 2023-11-25T04:23:47.379 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.379 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.379 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:47.379 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038649+0000", 2023-11-25T04:23:47.379 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029764679999999999 2023-11-25T04:23:47.379 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.379 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.380 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:47.380 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044578+0000", 2023-11-25T04:23:47.380 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0059293510000000002 2023-11-25T04:23:47.380 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.380 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.380 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:47.380 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044651+0000", 2023-11-25T04:23:47.380 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2513000000000002e-05 2023-11-25T04:23:47.380 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.380 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.380 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:47.380 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044702+0000", 2023-11-25T04:23:47.380 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1034e-05 2023-11-25T04:23:47.381 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:47.381 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:47.381 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:47.381 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.381 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.381 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:29 3.b 3:d0630e4c:::mds0_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:47.381 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:47.381 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.309648410000001, 2023-11-25T04:23:47.381 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010980227, 2023-11-25T04:23:47.381 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:47.381 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:47.381 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:47.382 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.382 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:47.382 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:47.382 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:47.382 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.382 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.382 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:47.382 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:47.382 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:47.382 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.382 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.382 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:47.382 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:47.383 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3499999999999996e-07 2023-11-25T04:23:47.383 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.383 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.383 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:47.383 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033658+0000", 2023-11-25T04:23:47.383 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1280000000000001e-06 2023-11-25T04:23:47.383 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.383 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.383 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:47.383 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033667+0000", 2023-11-25T04:23:47.383 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1430000000000004e-06 2023-11-25T04:23:47.383 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.383 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.384 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:47.384 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033678+0000", 2023-11-25T04:23:47.384 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0906e-05 2023-11-25T04:23:47.384 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.384 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.384 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:47.384 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034456+0000", 2023-11-25T04:23:47.384 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00077833899999999996 2023-11-25T04:23:47.384 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.384 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.384 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:47.384 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034575+0000", 2023-11-25T04:23:47.384 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011926000000000001 2023-11-25T04:23:47.385 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.385 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.385 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:47.385 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035001+0000", 2023-11-25T04:23:47.385 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042550999999999998 2023-11-25T04:23:47.385 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.385 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.385 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:47.385 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.037923+0000", 2023-11-25T04:23:47.385 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029219670000000001 2023-11-25T04:23:47.385 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.385 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.386 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:47.386 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044538+0000", 2023-11-25T04:23:47.386 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0066152349999999997 2023-11-25T04:23:47.386 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.386 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.386 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:47.386 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044594+0000", 2023-11-25T04:23:47.386 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5593000000000001e-05 2023-11-25T04:23:47.386 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.386 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.386 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:47.386 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044633+0000", 2023-11-25T04:23:47.386 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9511000000000002e-05 2023-11-25T04:23:47.387 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:47.387 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:47.387 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:47.387 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.387 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.387 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:34 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:47.387 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:47.387 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.309398297000001, 2023-11-25T04:23:47.387 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010920445000000001, 2023-11-25T04:23:47.387 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:47.387 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:47.387 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:47.388 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.388 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:47.388 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:47.388 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:47.388 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.388 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.388 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:47.388 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:47.388 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:47.388 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.388 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.388 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:47.389 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033904+0000", 2023-11-25T04:23:47.389 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6599999999999994e-07 2023-11-25T04:23:47.389 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.389 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.389 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:47.389 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033909+0000", 2023-11-25T04:23:47.389 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5639999999999996e-06 2023-11-25T04:23:47.389 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.389 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.389 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:47.389 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033923+0000", 2023-11-25T04:23:47.389 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3028e-05 2023-11-25T04:23:47.390 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.390 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.390 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:47.390 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034043+0000", 2023-11-25T04:23:47.390 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000120328 2023-11-25T04:23:47.390 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.390 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.390 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:47.390 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035576+0000", 2023-11-25T04:23:47.390 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015333720000000001 2023-11-25T04:23:47.390 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.391 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.391 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:47.391 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035699+0000", 2023-11-25T04:23:47.391 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000123001 2023-11-25T04:23:47.391 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.391 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.391 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:47.391 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044823+0000", 2023-11-25T04:23:47.391 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0091241859999999994 2023-11-25T04:23:47.391 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:47.391 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:47.391 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:47.392 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.392 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.392 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:35 3.1f 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:47.392 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:47.392 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.308094091999997, 2023-11-25T04:23:47.392 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096164779999999995, 2023-11-25T04:23:47.392 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:47.392 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:47.392 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:47.392 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.392 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:47.392 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:47.393 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:47.393 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.393 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.393 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:47.393 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:47.393 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:47.393 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.393 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.393 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:47.393 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035208+0000", 2023-11-25T04:23:47.393 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2699999999999998e-07 2023-11-25T04:23:47.393 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.394 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.394 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:47.394 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035213+0000", 2023-11-25T04:23:47.394 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.6070000000000001e-06 2023-11-25T04:23:47.394 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.394 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.394 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:47.394 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035350+0000", 2023-11-25T04:23:47.394 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013693599999999999 2023-11-25T04:23:47.394 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.394 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.395 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:47.395 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035358+0000", 2023-11-25T04:23:47.395 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0099999999999995e-06 2023-11-25T04:23:47.395 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.395 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.395 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:47.395 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036260+0000", 2023-11-25T04:23:47.395 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00090240100000000005 2023-11-25T04:23:47.395 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.395 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.395 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:47.395 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036517+0000", 2023-11-25T04:23:47.395 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00025736299999999998 2023-11-25T04:23:47.395 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.396 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.396 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:47.396 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044824+0000", 2023-11-25T04:23:47.396 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0083063340000000003 2023-11-25T04:23:47.396 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:47.396 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:47.396 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:47.396 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.396 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.396 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:37 3.1e 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:47.397 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:47.397 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.307927405999997, 2023-11-25T04:23:47.397 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0078322640000000002, 2023-11-25T04:23:47.397 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:47.397 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:47.397 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:47.397 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.397 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:47.397 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:47.397 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:47.397 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.398 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.398 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:47.398 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:47.398 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:47.398 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.398 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.398 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:47.398 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035375+0000", 2023-11-25T04:23:47.398 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9100000000000003e-07 2023-11-25T04:23:47.398 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.398 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.399 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:47.399 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035379+0000", 2023-11-25T04:23:47.399 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9430000000000002e-06 2023-11-25T04:23:47.399 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.399 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.399 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:47.399 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035388+0000", 2023-11-25T04:23:47.399 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2760000000000006e-06 2023-11-25T04:23:47.399 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.399 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.399 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:47.399 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035396+0000", 2023-11-25T04:23:47.400 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8480000000000001e-06 2023-11-25T04:23:47.400 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.400 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.400 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:47.400 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035452+0000", 2023-11-25T04:23:47.400 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5831000000000001e-05 2023-11-25T04:23:47.400 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.400 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.400 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:47.400 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035934+0000", 2023-11-25T04:23:47.400 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00048265599999999998 2023-11-25T04:23:47.400 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.401 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.401 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:47.401 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043206+0000", 2023-11-25T04:23:47.401 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.007271919 2023-11-25T04:23:47.401 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:47.401 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:47.401 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:47.401 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.401 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.401 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing(3.1a epoch 18/16 rep_tid 3 entries 18'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:36 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:23:47.401 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:47.402 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.306651819999999, 2023-11-25T04:23:47.402 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002073414, 2023-11-25T04:23:47.402 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:47.402 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:47.402 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:47.402 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.402 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:47.402 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:47.402 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:47.402 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.402 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.403 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:47.403 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:47.403 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:47.403 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.403 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.403 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:47.403 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036650+0000", 2023-11-25T04:23:47.403 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7300000000000002e-07 2023-11-25T04:23:47.403 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.403 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.403 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:47.403 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036659+0000", 2023-11-25T04:23:47.404 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.7499999999999992e-06 2023-11-25T04:23:47.404 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.404 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.404 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:47.404 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036673+0000", 2023-11-25T04:23:47.404 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4917e-05 2023-11-25T04:23:47.404 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.404 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.404 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:47.404 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036687+0000", 2023-11-25T04:23:47.404 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3883000000000001e-05 2023-11-25T04:23:47.405 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.405 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.405 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:47.405 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036781+0000", 2023-11-25T04:23:47.405 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3219000000000003e-05 2023-11-25T04:23:47.405 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.405 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.405 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:47.405 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038723+0000", 2023-11-25T04:23:47.405 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019422720000000001 2023-11-25T04:23:47.405 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:47.405 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:47.406 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:47.406 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.406 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.406 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:28 3.13 e18/16 3:cd5a64a3:::100.00000000:head v 18'1, mlcod=0'0)", 2023-11-25T04:23:47.406 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:47.406 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.302990661999999, 2023-11-25T04:23:47.406 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001850546, 2023-11-25T04:23:47.406 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:47.406 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:47.406 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:47.406 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.407 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:47.407 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:47.407 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:47.407 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.407 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.407 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:47.407 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:47.407 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:47.407 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.407 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.407 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:47.407 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:47.408 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7800000000000002e-07 2023-11-25T04:23:47.408 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.408 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.408 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:47.408 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040331+0000", 2023-11-25T04:23:47.408 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0449999999999999e-05 2023-11-25T04:23:47.408 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.408 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.408 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:47.408 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040339+0000", 2023-11-25T04:23:47.408 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0709999999999995e-06 2023-11-25T04:23:47.408 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.409 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.409 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:47.409 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040347+0000", 2023-11-25T04:23:47.409 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7079999999999997e-06 2023-11-25T04:23:47.409 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.409 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.409 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:47.409 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040410+0000", 2023-11-25T04:23:47.409 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2453000000000007e-05 2023-11-25T04:23:47.409 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.409 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.409 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:47.409 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040422+0000", 2023-11-25T04:23:47.410 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2429000000000001e-05 2023-11-25T04:23:47.410 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.410 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.410 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:47.410 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042129+0000", 2023-11-25T04:23:47.410 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001707266 2023-11-25T04:23:47.410 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.410 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.410 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:47.410 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042161+0000", 2023-11-25T04:23:47.410 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1791000000000001e-05 2023-11-25T04:23:47.410 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:47.411 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:47.411 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:47.411 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.411 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.411 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:27 3.13 e18/16 3:c91d4a1d:::mds0_inotable:head v 18'2, mlcod=0'0)", 2023-11-25T04:23:47.411 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:47.411 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.302787139000003, 2023-11-25T04:23:47.411 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0016886430000000001, 2023-11-25T04:23:47.411 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:47.411 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:47.411 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:47.412 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.412 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:47.412 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:47.412 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:47.412 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.412 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.412 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:47.412 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:47.412 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:47.412 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.412 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.412 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:47.413 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:47.413 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3999999999999997e-07 2023-11-25T04:23:47.413 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.413 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.413 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:47.413 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040520+0000", 2023-11-25T04:23:47.413 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2989999999999996e-06 2023-11-25T04:23:47.413 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.413 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.413 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:47.413 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040527+0000", 2023-11-25T04:23:47.413 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5730000000000002e-06 2023-11-25T04:23:47.414 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.414 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.414 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:47.414 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040535+0000", 2023-11-25T04:23:47.414 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8630000000000008e-06 2023-11-25T04:23:47.414 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.414 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.414 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:47.414 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040683+0000", 2023-11-25T04:23:47.414 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000148002 2023-11-25T04:23:47.414 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.414 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.415 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:47.415 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040700+0000", 2023-11-25T04:23:47.415 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7065e-05 2023-11-25T04:23:47.415 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.415 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.415 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:47.415 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042163+0000", 2023-11-25T04:23:47.415 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001462767 2023-11-25T04:23:47.415 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.415 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.415 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:47.415 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042203+0000", 2023-11-25T04:23:47.416 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9734000000000003e-05 2023-11-25T04:23:47.416 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:47.416 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:47.416 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:47.416 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.416 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.416 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:20 3.1c e18/16 3:3ed09add:::607.00000000:head v 18'1, mlcod=0'0)", 2023-11-25T04:23:47.416 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:47.416 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.302752097999999, 2023-11-25T04:23:47.416 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015271, 2023-11-25T04:23:47.416 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:47.416 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:47.417 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:47.417 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.417 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:47.417 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:47.417 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:47.417 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.417 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.417 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:47.417 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:47.417 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:47.417 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.417 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.417 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:47.418 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:47.418 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4799999999999999e-07 2023-11-25T04:23:47.418 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.418 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.418 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:47.418 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040561+0000", 2023-11-25T04:23:47.418 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1800000000000001e-05 2023-11-25T04:23:47.418 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.418 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.418 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:47.418 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040568+0000", 2023-11-25T04:23:47.418 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7730000000000002e-06 2023-11-25T04:23:47.419 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.419 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.419 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:47.419 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040587+0000", 2023-11-25T04:23:47.419 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8691000000000001e-05 2023-11-25T04:23:47.419 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.419 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.419 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:47.419 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040638+0000", 2023-11-25T04:23:47.419 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1068e-05 2023-11-25T04:23:47.419 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.419 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.420 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:47.420 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040670+0000", 2023-11-25T04:23:47.420 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2125999999999997e-05 2023-11-25T04:23:47.420 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.420 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.420 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:47.420 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042032+0000", 2023-11-25T04:23:47.420 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001361821 2023-11-25T04:23:47.420 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.420 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.420 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:47.421 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042076+0000", 2023-11-25T04:23:47.421 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4472999999999999e-05 2023-11-25T04:23:47.421 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:47.421 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:47.421 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:47.421 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.421 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.421 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:21 3.1c e18/16 3:3ed09add:::607.00000000:head v 18'2, mlcod=0'0)", 2023-11-25T04:23:47.421 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:47.422 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.302517373000001, 2023-11-25T04:23:47.422 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0013228160000000001, 2023-11-25T04:23:47.422 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:47.422 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:47.422 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:47.422 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.422 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:47.422 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:47.422 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:47.422 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.422 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.422 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:47.423 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:47.423 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:47.423 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.423 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.423 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:47.423 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:47.423 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9299999999999999e-07 2023-11-25T04:23:47.423 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.423 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.423 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:47.423 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040790+0000", 2023-11-25T04:23:47.423 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5770000000000001e-06 2023-11-25T04:23:47.423 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.424 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.424 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:47.424 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040799+0000", 2023-11-25T04:23:47.424 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6439999999999995e-06 2023-11-25T04:23:47.424 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.424 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.424 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:47.424 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040806+0000", 2023-11-25T04:23:47.424 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0180000000000004e-06 2023-11-25T04:23:47.424 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.424 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.424 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:47.425 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040907+0000", 2023-11-25T04:23:47.425 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000100096 2023-11-25T04:23:47.425 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.425 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.425 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:47.425 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040925+0000", 2023-11-25T04:23:47.425 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8536999999999999e-05 2023-11-25T04:23:47.425 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.425 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.426 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:47.426 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042079+0000", 2023-11-25T04:23:47.426 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00115358 2023-11-25T04:23:47.427 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.427 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.427 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:47.427 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042107+0000", 2023-11-25T04:23:47.427 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7971e-05 2023-11-25T04:23:47.427 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:47.427 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:47.428 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:47.428 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.428 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.428 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop_reply(mds.0.3:4 3.1f e18/16 ondisk, result = 0)", 2023-11-25T04:23:47.428 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:47.429 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.301177983000002, 2023-11-25T04:23:47.429 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00044092900000000001, 2023-11-25T04:23:47.429 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:47.429 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:47.429 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:47.429 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.429 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:47.429 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:47.430 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:47.430 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.430 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.430 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:47.430 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:47.430 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:47.430 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.430 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.430 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:47.431 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042124+0000", 2023-11-25T04:23:47.431 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1800000000000005e-07 2023-11-25T04:23:47.431 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.431 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.431 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:47.431 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042128+0000", 2023-11-25T04:23:47.431 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2289999999999997e-06 2023-11-25T04:23:47.431 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.431 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.431 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:47.431 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042138+0000", 2023-11-25T04:23:47.432 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9729999999999992e-06 2023-11-25T04:23:47.432 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.432 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.432 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:47.432 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042145+0000", 2023-11-25T04:23:47.432 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.336e-06 2023-11-25T04:23:47.432 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.432 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.432 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:47.432 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042441+0000", 2023-11-25T04:23:47.433 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029650500000000002 2023-11-25T04:23:47.433 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.433 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.433 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:47.433 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042448+0000", 2023-11-25T04:23:47.433 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4100000000000002e-06 2023-11-25T04:23:47.433 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.433 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.433 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:47.433 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042564+0000", 2023-11-25T04:23:47.433 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000115758 2023-11-25T04:23:47.433 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:47.433 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:47.434 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:47.434 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.434 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.434 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing_reply(3.1f epoch 18/16 rep_tid 8 lcod 18'4)", 2023-11-25T04:23:47.434 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:47.434 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.298852570999998, 2023-11-25T04:23:47.434 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00038154699999999998, 2023-11-25T04:23:47.434 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:47.435 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:47.435 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:47.435 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.435 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:47.435 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:47.435 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:47.435 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.436 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.436 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:47.436 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:47.436 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:47.436 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.436 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.436 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:47.436 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:47.436 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2700000000000001e-07 2023-11-25T04:23:47.436 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.436 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.437 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:47.437 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044451+0000", 2023-11-25T04:23:47.437 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7460000000000001e-06 2023-11-25T04:23:47.437 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.437 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.437 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:47.437 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044455+0000", 2023-11-25T04:23:47.437 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5809999999999998e-06 2023-11-25T04:23:47.437 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.437 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.437 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:47.438 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044458+0000", 2023-11-25T04:23:47.438 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0170000000000001e-06 2023-11-25T04:23:47.438 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.438 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.438 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:47.438 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044734+0000", 2023-11-25T04:23:47.438 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00027582799999999999 2023-11-25T04:23:47.438 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.438 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.439 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:47.439 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044830+0000", 2023-11-25T04:23:47.439 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6248000000000004e-05 2023-11-25T04:23:47.439 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:47.439 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:47.439 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:47.439 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.439 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.439 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing_reply(3.b epoch 18/16 rep_tid 5 lcod 18'2)", 2023-11-25T04:23:47.439 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:47.439 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.298815910000002, 2023-11-25T04:23:47.439 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000344488, 2023-11-25T04:23:47.439 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:47.439 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:47.440 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:47.440 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.440 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:47.440 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:47.440 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:47.440 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.440 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.440 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:47.440 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:47.440 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:47.440 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.440 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.440 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:47.441 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:47.441 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.55e-07 2023-11-25T04:23:47.441 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.441 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.441 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:47.441 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044487+0000", 2023-11-25T04:23:47.441 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2470000000000001e-06 2023-11-25T04:23:47.441 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.441 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.441 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:47.441 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044491+0000", 2023-11-25T04:23:47.441 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9659999999999998e-06 2023-11-25T04:23:47.441 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.442 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.442 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:47.442 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044494+0000", 2023-11-25T04:23:47.442 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2310000000000001e-06 2023-11-25T04:23:47.442 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.442 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.442 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:47.442 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044756+0000", 2023-11-25T04:23:47.442 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00026172800000000003 2023-11-25T04:23:47.442 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.442 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.443 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:47.443 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044830+0000", 2023-11-25T04:23:47.443 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4160999999999997e-05 2023-11-25T04:23:47.443 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:47.443 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:47.443 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:47.443 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.443 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.443 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:39 3.6 3:654134d2:::mds0_openfiles.0:head [omap-clear,omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2023-11-25T04:23:47.443 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:47.443 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 55.238989867999997, 2023-11-25T04:23:47.443 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001788658, 2023-11-25T04:23:47.444 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:47.444 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:47.444 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:47.444 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.444 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:47.444 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:47.444 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:47.444 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.444 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.444 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:47.444 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:47.445 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:47.445 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.445 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.445 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:47.445 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104314+0000", 2023-11-25T04:23:47.445 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3099999999999999e-06 2023-11-25T04:23:47.445 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.445 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.445 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:47.445 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104325+0000", 2023-11-25T04:23:47.445 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1511999999999999e-05 2023-11-25T04:23:47.445 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.446 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.446 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:47.446 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104362+0000", 2023-11-25T04:23:47.446 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6903000000000002e-05 2023-11-25T04:23:47.446 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.446 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.446 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:47.446 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104387+0000", 2023-11-25T04:23:47.446 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4766e-05 2023-11-25T04:23:47.446 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.446 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.446 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:47.447 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104465+0000", 2023-11-25T04:23:47.447 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8119000000000002e-05 2023-11-25T04:23:47.447 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.447 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.447 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:47.447 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104557+0000", 2023-11-25T04:23:47.447 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2496000000000003e-05 2023-11-25T04:23:47.447 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.447 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.447 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:47.447 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104646+0000", 2023-11-25T04:23:47.448 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8818000000000002e-05 2023-11-25T04:23:47.448 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.448 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.448 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:47.448 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105201+0000", 2023-11-25T04:23:47.448 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00055506000000000002 2023-11-25T04:23:47.448 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.448 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.448 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:47.448 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105994+0000", 2023-11-25T04:23:47.448 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00079294700000000001 2023-11-25T04:23:47.448 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.448 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.449 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:47.449 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.106055+0000", 2023-11-25T04:23:47.449 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0566000000000003e-05 2023-11-25T04:23:47.449 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.449 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.449 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:47.449 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.106100+0000", 2023-11-25T04:23:47.449 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5160999999999997e-05 2023-11-25T04:23:47.449 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:47.449 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:47.449 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:47.449 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.449 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.450 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:43 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2023-11-25T04:23:47.450 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:47.450 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 40.238764871999997, 2023-11-25T04:23:47.450 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001751166, 2023-11-25T04:23:47.450 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:47.450 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:47.450 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:47.450 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.450 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:47.450 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:47.450 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:47.450 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.450 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.451 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:47.451 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:47.451 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:47.451 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.451 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.451 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:47.451 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104539+0000", 2023-11-25T04:23:47.451 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9529999999999999e-06 2023-11-25T04:23:47.451 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.451 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.451 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:47.451 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104557+0000", 2023-11-25T04:23:47.452 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7464000000000001e-05 2023-11-25T04:23:47.452 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.452 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.452 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:47.452 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104590+0000", 2023-11-25T04:23:47.452 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2966000000000003e-05 2023-11-25T04:23:47.452 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.452 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.452 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:47.452 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104608+0000", 2023-11-25T04:23:47.452 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8193e-05 2023-11-25T04:23:47.452 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.452 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.453 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:47.453 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104767+0000", 2023-11-25T04:23:47.453 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015882 2023-11-25T04:23:47.453 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.453 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.453 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:47.453 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104909+0000", 2023-11-25T04:23:47.453 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014244999999999999 2023-11-25T04:23:47.453 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.453 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.453 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:47.453 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.105059+0000", 2023-11-25T04:23:47.453 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014964399999999999 2023-11-25T04:23:47.454 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.454 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.454 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:47.454 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.105701+0000", 2023-11-25T04:23:47.454 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00064201099999999995 2023-11-25T04:23:47.454 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.454 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.454 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:47.454 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106233+0000", 2023-11-25T04:23:47.454 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00053178500000000001 2023-11-25T04:23:47.454 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.454 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.454 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:47.455 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106265+0000", 2023-11-25T04:23:47.455 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1986000000000002e-05 2023-11-25T04:23:47.455 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.455 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.455 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:47.455 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106288+0000", 2023-11-25T04:23:47.455 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2894000000000001e-05 2023-11-25T04:23:47.455 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:47.455 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:47.455 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:47.455 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.455 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.455 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:45 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e23)", 2023-11-25T04:23:47.455 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:23:47.456 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 35.238566831999997, 2023-11-25T04:23:47.456 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00098423299999999994, 2023-11-25T04:23:47.456 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:47.456 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:47.456 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:47.456 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.456 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:47.456 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:23:47.456 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:47.456 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.456 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.456 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:47.456 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:23:47.456 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:47.457 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.457 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.457 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:47.457 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104737+0000", 2023-11-25T04:23:47.457 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3309999999999998e-06 2023-11-25T04:23:47.457 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.457 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.457 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:47.457 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104748+0000", 2023-11-25T04:23:47.457 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1080999999999999e-05 2023-11-25T04:23:47.457 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.457 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.457 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:47.457 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104771+0000", 2023-11-25T04:23:47.458 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2707000000000001e-05 2023-11-25T04:23:47.458 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.458 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.458 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:47.458 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104800+0000", 2023-11-25T04:23:47.458 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9595e-05 2023-11-25T04:23:47.458 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.458 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.458 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:47.458 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104890+0000", 2023-11-25T04:23:47.458 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.9808999999999993e-05 2023-11-25T04:23:47.458 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.458 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.458 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:47.459 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105004+0000", 2023-11-25T04:23:47.459 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000114508 2023-11-25T04:23:47.459 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.459 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.459 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:47.459 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105634+0000", 2023-11-25T04:23:47.459 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00062985799999999996 2023-11-25T04:23:47.459 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.459 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.459 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:47.459 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105671+0000", 2023-11-25T04:23:47.459 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6640000000000002e-05 2023-11-25T04:23:47.459 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:47.460 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:47.460 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:47.460 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105719+0000", 2023-11-25T04:23:47.460 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7704000000000003e-05 2023-11-25T04:23:47.460 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:47.460 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:47.460 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:47.460 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:47.460 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:47.460 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:47.462 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:47.460+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:47.462 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:23:47.563 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:47.562+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:47.569 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:47.569 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:23:47.569 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2023-11-25T04:23:47.569 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:47.578 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:23:47.639 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-11-25T04:23:47.663 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:47.662+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:47.726 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:47.729 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:23:47.730 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2023-11-25T04:23:47.731 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2023-11-25T04:23:47.733 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:47.737 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:23:47.764 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:47.763+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:47.810 INFO:teuthology.orchestra.run.smithi067.stderr:Error ENXIO: problem getting command descriptions from osd.0 2023-11-25T04:23:47.843 INFO:teuthology.orchestra.run.smithi067.stdout:osd.1: {} 2023-11-25T04:23:47.843 INFO:teuthology.orchestra.run.smithi067.stderr:osd.1: osd_enable_op_tracker = '' 2023-11-25T04:23:47.866 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:47.864+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:47.875 INFO:teuthology.orchestra.run.smithi067.stdout:osd.2: {} 2023-11-25T04:23:47.876 INFO:teuthology.orchestra.run.smithi067.stderr:osd.2: osd_enable_op_tracker = '' 2023-11-25T04:23:47.879 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-11-25T04:23:47.880 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-11-25T04:23:47.881 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:47.881 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:47.890 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:23:47.966 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:47.965+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:48.001 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:48.002 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:48.011 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:23:48.067 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:48.066+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:48.115 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:48.115 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:48.125 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:23:48.169 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:48.168+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:48.230 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:48.230 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:48.239 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:23:48.279 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:48.278+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:48.346 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:48.346 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:48.354 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:23:48.398 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:48.397+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:48.459 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:48.459 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:48.467 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:23:48.472 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:48.469+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:48.574 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:48.572+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:48.613 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:48.613 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:48.621 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:23:48.674 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:48.673+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:48.728 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:48.728 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:48.737 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:23:48.779 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:48.777+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:48.876 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:48.875+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:48.882 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:48.882 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:48.891 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:23:48.977 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:48.976+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:48.981 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2023-11-25T04:23:49.037 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:49.037 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:49.046 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:23:49.089 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:49.088+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:49.153 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:49.153 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:49.162 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:23:49.203 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:49.202+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:49.280 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:49.278+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:49.307 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:49.307 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:49.316 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:23:49.381 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:49.379+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:49.421 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:49.421 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:49.429 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:23:49.481 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:49.480+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:49.536 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:49.537 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:49.545 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:23:49.587 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:49.585+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:49.684 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:49.683+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:49.690 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:49.690 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:49.698 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:23:49.784 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:49.783+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:49.803 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:49.804 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:49.812 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:23:49.881 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-11-25T04:23:49.885 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:49.883+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:49.918 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:49.918 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:49.927 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:23:49.962 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~41s 2023-11-25T04:23:49.982 INFO:teuthology.orchestra.run.smithi067.stderr:nodeep-scrub is unset 2023-11-25T04:23:50.012 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:50.011+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:50.017 INFO:teuthology.orchestra.run.smithi067.stderr:Error ENXIO: problem getting command descriptions from osd.0 2023-11-25T04:23:50.035 INFO:teuthology.orchestra.run.smithi067.stdout:osd.1: {} 2023-11-25T04:23:50.035 INFO:teuthology.orchestra.run.smithi067.stderr:osd.1: osd_enable_op_tracker = '' 2023-11-25T04:23:50.053 INFO:teuthology.orchestra.run.smithi067.stdout:osd.2: {} 2023-11-25T04:23:50.054 INFO:teuthology.orchestra.run.smithi067.stderr:osd.2: osd_enable_op_tracker = '' 2023-11-25T04:23:50.056 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-11-25T04:23:50.057 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-11-25T04:23:50.073 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:50.073 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:23:50.073 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2023-11-25T04:23:50.074 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2023-11-25T04:23:50.074 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:50.082 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:23:50.123 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:50.122+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:50.187 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:50.186+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:50.229 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:50.229 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2023-11-25T04:23:50.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2023-11-25T04:23:50.230 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2023-11-25T04:23:50.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.230 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:155 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'109, mlcod=11'105)", 2023-11-25T04:23:50.230 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:50.230 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 72.442495440000002, 2023-11-25T04:23:50.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061719019999999999, 2023-11-25T04:23:50.230 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:50.230 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:50.230 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:50.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:50.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:50.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:50.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:50.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:50.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:50.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:50.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784884+0000", 2023-11-25T04:23:50.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3099999999999999e-07 2023-11-25T04:23:50.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:50.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784942+0000", 2023-11-25T04:23:50.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8570000000000003e-05 2023-11-25T04:23:50.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:50.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784949+0000", 2023-11-25T04:23:50.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0310000000000001e-06 2023-11-25T04:23:50.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:50.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784953+0000", 2023-11-25T04:23:50.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6320000000000001e-06 2023-11-25T04:23:50.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.233 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:50.233 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784993+0000", 2023-11-25T04:23:50.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9737000000000003e-05 2023-11-25T04:23:50.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.233 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:50.233 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785011+0000", 2023-11-25T04:23:50.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8349999999999999e-05 2023-11-25T04:23:50.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:50.234 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791024+0000", 2023-11-25T04:23:50.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006012997 2023-11-25T04:23:50.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:50.234 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791055+0000", 2023-11-25T04:23:50.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1254000000000003e-05 2023-11-25T04:23:50.234 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:50.234 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:50.234 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:50.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.235 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:156 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'110, mlcod=11'105)", 2023-11-25T04:23:50.235 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:50.235 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 72.441909390999996, 2023-11-25T04:23:50.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005616116, 2023-11-25T04:23:50.235 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:50.235 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:50.235 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:50.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.235 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:50.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:50.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:50.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:50.236 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:50.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:50.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:50.236 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785470+0000", 2023-11-25T04:23:50.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0699999999999998e-07 2023-11-25T04:23:50.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:50.236 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785474+0000", 2023-11-25T04:23:50.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.048e-06 2023-11-25T04:23:50.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:50.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785480+0000", 2023-11-25T04:23:50.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7729999999999996e-06 2023-11-25T04:23:50.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:50.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785483+0000", 2023-11-25T04:23:50.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1970000000000001e-06 2023-11-25T04:23:50.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:50.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787936+0000", 2023-11-25T04:23:50.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024529040000000001 2023-11-25T04:23:50.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:50.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787947+0000", 2023-11-25T04:23:50.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1669e-05 2023-11-25T04:23:50.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:50.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791057+0000", 2023-11-25T04:23:50.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031095200000000002 2023-11-25T04:23:50.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.239 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:50.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791086+0000", 2023-11-25T04:23:50.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8697999999999999e-05 2023-11-25T04:23:50.239 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:50.239 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:50.239 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:50.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.239 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:157 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'111, mlcod=11'106)", 2023-11-25T04:23:50.239 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:50.240 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 72.441316833000002, 2023-11-25T04:23:50.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050590280000000001, 2023-11-25T04:23:50.240 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:50.240 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:50.240 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:50.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:50.240 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:50.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:50.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:50.240 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:50.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:50.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:50.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786063+0000", 2023-11-25T04:23:50.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2600000000000002e-07 2023-11-25T04:23:50.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:50.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786069+0000", 2023-11-25T04:23:50.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8839999999999999e-06 2023-11-25T04:23:50.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:50.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786079+0000", 2023-11-25T04:23:50.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0124e-05 2023-11-25T04:23:50.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:50.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786085+0000", 2023-11-25T04:23:50.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6660000000000002e-06 2023-11-25T04:23:50.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:50.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788160+0000", 2023-11-25T04:23:50.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020747669999999999 2023-11-25T04:23:50.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.244 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:50.244 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788180+0000", 2023-11-25T04:23:50.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0208999999999999e-05 2023-11-25T04:23:50.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.244 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:50.244 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791087+0000", 2023-11-25T04:23:50.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029064630000000002 2023-11-25T04:23:50.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.245 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:50.245 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791121+0000", 2023-11-25T04:23:50.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4289000000000003e-05 2023-11-25T04:23:50.245 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:50.245 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:50.245 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:50.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.245 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:158 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'112, mlcod=11'106)", 2023-11-25T04:23:50.245 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:50.245 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 72.440900439999993, 2023-11-25T04:23:50.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046663030000000001, 2023-11-25T04:23:50.246 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:50.246 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:50.246 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:50.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:50.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:50.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:50.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:50.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:50.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:50.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:50.247 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786479+0000", 2023-11-25T04:23:50.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2799999999999996e-07 2023-11-25T04:23:50.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:50.247 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786498+0000", 2023-11-25T04:23:50.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9069e-05 2023-11-25T04:23:50.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:50.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786508+0000", 2023-11-25T04:23:50.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5810000000000003e-06 2023-11-25T04:23:50.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:50.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786512+0000", 2023-11-25T04:23:50.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4889999999999997e-06 2023-11-25T04:23:50.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:50.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788438+0000", 2023-11-25T04:23:50.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019257059999999999 2023-11-25T04:23:50.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:50.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788449+0000", 2023-11-25T04:23:50.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1419e-05 2023-11-25T04:23:50.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.250 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:50.250 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791122+0000", 2023-11-25T04:23:50.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002672822 2023-11-25T04:23:50.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.250 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:50.250 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791145+0000", 2023-11-25T04:23:50.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2589e-05 2023-11-25T04:23:50.250 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:50.250 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:50.251 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:50.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.251 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:206 1.0 e12/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 12'152, mlcod=12'148)", 2023-11-25T04:23:50.251 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:50.251 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 71.92913197, 2023-11-25T04:23:50.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046728189999999999, 2023-11-25T04:23:50.251 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:50.251 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:50.251 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:50.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:50.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:50.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:50.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:50.252 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:50.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:50.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:50.252 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298248+0000", 2023-11-25T04:23:50.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3600000000000003e-07 2023-11-25T04:23:50.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:50.252 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298278+0000", 2023-11-25T04:23:50.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0474e-05 2023-11-25T04:23:50.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:50.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298288+0000", 2023-11-25T04:23:50.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0156000000000001e-05 2023-11-25T04:23:50.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:50.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298293+0000", 2023-11-25T04:23:50.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7400000000000004e-06 2023-11-25T04:23:50.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:50.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298333+0000", 2023-11-25T04:23:50.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9666000000000003e-05 2023-11-25T04:23:50.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:50.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298351+0000", 2023-11-25T04:23:50.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8544999999999999e-05 2023-11-25T04:23:50.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:50.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302891+0000", 2023-11-25T04:23:50.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045398519999999996 2023-11-25T04:23:50.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:50.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302920+0000", 2023-11-25T04:23:50.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8750000000000001e-05 2023-11-25T04:23:50.255 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:50.255 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:50.255 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:50.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.255 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:14 3.15 3:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:50.255 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:50.255 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 63.193997613000001, 2023-11-25T04:23:50.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010990437, 2023-11-25T04:23:50.256 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:50.256 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:50.256 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:50.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:50.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:50.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:50.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:50.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:50.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:50.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:50.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033383+0000", 2023-11-25T04:23:50.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9870000000000002e-06 2023-11-25T04:23:50.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:50.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033398+0000", 2023-11-25T04:23:50.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4569999999999999e-05 2023-11-25T04:23:50.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:50.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033411+0000", 2023-11-25T04:23:50.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3543999999999999e-05 2023-11-25T04:23:50.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:50.258 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033423+0000", 2023-11-25T04:23:50.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1905999999999999e-05 2023-11-25T04:23:50.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:50.258 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034433+0000", 2023-11-25T04:23:50.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00101022 2023-11-25T04:23:50.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:50.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034539+0000", 2023-11-25T04:23:50.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010597199999999999 2023-11-25T04:23:50.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:50.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034667+0000", 2023-11-25T04:23:50.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012765500000000001 2023-11-25T04:23:50.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:50.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037803+0000", 2023-11-25T04:23:50.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031362220000000001 2023-11-25T04:23:50.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:50.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:23:50.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0064964669999999997 2023-11-25T04:23:50.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:50.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044331+0000", 2023-11-25T04:23:50.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1544999999999997e-05 2023-11-25T04:23:50.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:50.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044372+0000", 2023-11-25T04:23:50.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0349000000000003e-05 2023-11-25T04:23:50.260 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:50.261 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:50.261 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:50.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.261 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:15 3.15 3:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:50.261 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:50.261 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 63.193337962999998, 2023-11-25T04:23:50.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010464362, 2023-11-25T04:23:50.261 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:50.261 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:50.261 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:50.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:50.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:50.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:50.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:50.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:50.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:50.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:50.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034042+0000", 2023-11-25T04:23:50.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0999999999999997e-07 2023-11-25T04:23:50.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:50.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034048+0000", 2023-11-25T04:23:50.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0390000000000001e-06 2023-11-25T04:23:50.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:50.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034059+0000", 2023-11-25T04:23:50.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1182999999999999e-05 2023-11-25T04:23:50.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:50.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034067+0000", 2023-11-25T04:23:50.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1559999999999995e-06 2023-11-25T04:23:50.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:50.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035005+0000", 2023-11-25T04:23:50.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00093747399999999999 2023-11-25T04:23:50.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:50.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035077+0000", 2023-11-25T04:23:50.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2747e-05 2023-11-25T04:23:50.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:50.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035219+0000", 2023-11-25T04:23:50.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000141877 2023-11-25T04:23:50.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:50.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037810+0000", 2023-11-25T04:23:50.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002590752 2023-11-25T04:23:50.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:50.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044437+0000", 2023-11-25T04:23:50.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066270110000000004 2023-11-25T04:23:50.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:50.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044481+0000", 2023-11-25T04:23:50.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4106000000000001e-05 2023-11-25T04:23:50.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.267 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:50.267 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044505+0000", 2023-11-25T04:23:50.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4207e-05 2023-11-25T04:23:50.267 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:50.267 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:50.267 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:50.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.267 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:16 3.1d 3:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:50.267 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:50.268 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 63.193296115000003, 2023-11-25T04:23:50.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010413923, 2023-11-25T04:23:50.268 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:50.268 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:50.268 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:50.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:50.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:50.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:50.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:50.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:50.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:50.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:50.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:50.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4499999999999997e-07 2023-11-25T04:23:50.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:50.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034087+0000", 2023-11-25T04:23:50.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9770000000000002e-06 2023-11-25T04:23:50.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:50.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034094+0000", 2023-11-25T04:23:50.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8809999999999998e-06 2023-11-25T04:23:50.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:50.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034100+0000", 2023-11-25T04:23:50.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8440000000000003e-06 2023-11-25T04:23:50.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:50.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034993+0000", 2023-11-25T04:23:50.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00089336799999999996 2023-11-25T04:23:50.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:50.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035099+0000", 2023-11-25T04:23:50.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106037 2023-11-25T04:23:50.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:50.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035276+0000", 2023-11-25T04:23:50.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017631799999999999 2023-11-25T04:23:50.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:50.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037815+0000", 2023-11-25T04:23:50.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025395959999999999 2023-11-25T04:23:50.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:50.272 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044420+0000", 2023-11-25T04:23:50.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066045749999999997 2023-11-25T04:23:50.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:50.272 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044463+0000", 2023-11-25T04:23:50.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3205999999999999e-05 2023-11-25T04:23:50.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:50.272 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044497+0000", 2023-11-25T04:23:50.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3476000000000003e-05 2023-11-25T04:23:50.272 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:50.273 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:50.273 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:50.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.273 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:17 3.1d 3:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:50.273 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:50.273 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 63.193270906000002, 2023-11-25T04:23:50.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01127356, 2023-11-25T04:23:50.273 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:50.273 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:50.273 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:50.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:50.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:50.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:50.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:50.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:50.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:50.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:50.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:50.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1100000000000001e-07 2023-11-25T04:23:50.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:50.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034112+0000", 2023-11-25T04:23:50.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8600000000000003e-06 2023-11-25T04:23:50.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:50.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034120+0000", 2023-11-25T04:23:50.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7700000000000001e-06 2023-11-25T04:23:50.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:50.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034127+0000", 2023-11-25T04:23:50.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0999999999999998e-06 2023-11-25T04:23:50.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:50.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035601+0000", 2023-11-25T04:23:50.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014739410000000001 2023-11-25T04:23:50.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:50.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035685+0000", 2023-11-25T04:23:50.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3677999999999994e-05 2023-11-25T04:23:50.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:50.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035827+0000", 2023-11-25T04:23:50.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014232599999999999 2023-11-25T04:23:50.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:50.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038725+0000", 2023-11-25T04:23:50.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028982779999999998 2023-11-25T04:23:50.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:50.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045323+0000", 2023-11-25T04:23:50.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0065980780000000003 2023-11-25T04:23:50.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:50.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045354+0000", 2023-11-25T04:23:50.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0463e-05 2023-11-25T04:23:50.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:50.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045381+0000", 2023-11-25T04:23:50.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7654999999999999e-05 2023-11-25T04:23:50.278 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:50.278 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:50.278 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:50.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.278 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:22 3.e 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:50.279 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:50.279 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 63.193244493000002, 2023-11-25T04:23:50.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010133422, 2023-11-25T04:23:50.279 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:50.279 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:50.279 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:50.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:50.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:50.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:50.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:50.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:50.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:50.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:50.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034135+0000", 2023-11-25T04:23:50.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-07 2023-11-25T04:23:50.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:50.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034138+0000", 2023-11-25T04:23:50.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6009999999999999e-06 2023-11-25T04:23:50.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:50.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034145+0000", 2023-11-25T04:23:50.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1169999999999998e-06 2023-11-25T04:23:50.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:50.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034150+0000", 2023-11-25T04:23:50.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9080000000000003e-06 2023-11-25T04:23:50.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:50.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034209+0000", 2023-11-25T04:23:50.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8149000000000002e-05 2023-11-25T04:23:50.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:50.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034415+0000", 2023-11-25T04:23:50.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020637699999999999 2023-11-25T04:23:50.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:50.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034584+0000", 2023-11-25T04:23:50.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016870399999999999 2023-11-25T04:23:50.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:50.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.036979+0000", 2023-11-25T04:23:50.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023957509999999998 2023-11-25T04:23:50.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:50.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044161+0000", 2023-11-25T04:23:50.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0071811590000000003 2023-11-25T04:23:50.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:50.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044224+0000", 2023-11-25T04:23:50.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3454000000000006e-05 2023-11-25T04:23:50.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:50.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044268+0000", 2023-11-25T04:23:50.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3761999999999999e-05 2023-11-25T04:23:50.284 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:50.284 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:50.284 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:50.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.284 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:23 3.e 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:50.284 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:50.284 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 63.193213084, 2023-11-25T04:23:50.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010281676, 2023-11-25T04:23:50.284 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:50.284 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:50.284 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:50.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:50.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:50.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:50.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:50.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:50.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:50.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:50.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:50.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5799999999999999e-07 2023-11-25T04:23:50.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:50.286 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034169+0000", 2023-11-25T04:23:50.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1879999999999998e-06 2023-11-25T04:23:50.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:50.286 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034177+0000", 2023-11-25T04:23:50.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7020000000000001e-06 2023-11-25T04:23:50.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:50.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034184+0000", 2023-11-25T04:23:50.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3910000000000003e-06 2023-11-25T04:23:50.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:50.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034924+0000", 2023-11-25T04:23:50.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00074032999999999998 2023-11-25T04:23:50.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:50.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034991+0000", 2023-11-25T04:23:50.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7347000000000004e-05 2023-11-25T04:23:50.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:50.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035096+0000", 2023-11-25T04:23:50.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010488 2023-11-25T04:23:50.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:50.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037588+0000", 2023-11-25T04:23:50.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024918309999999999 2023-11-25T04:23:50.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:50.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044367+0000", 2023-11-25T04:23:50.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067787589999999997 2023-11-25T04:23:50.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:50.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044415+0000", 2023-11-25T04:23:50.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7803000000000003e-05 2023-11-25T04:23:50.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:50.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044447+0000", 2023-11-25T04:23:50.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2886999999999999e-05 2023-11-25T04:23:50.289 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:50.289 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:50.289 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:50.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.290 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:30 3.15 3:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:50.290 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:50.290 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 63.193154644000003, 2023-11-25T04:23:50.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010365070000000001, 2023-11-25T04:23:50.290 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:50.290 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:50.290 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:50.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.290 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:50.290 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:50.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:50.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.291 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:50.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:50.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:50.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.291 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:50.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034225+0000", 2023-11-25T04:23:50.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7400000000000002e-07 2023-11-25T04:23:50.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.291 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:50.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034229+0000", 2023-11-25T04:23:50.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3100000000000002e-06 2023-11-25T04:23:50.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:50.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034239+0000", 2023-11-25T04:23:50.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9140000000000003e-06 2023-11-25T04:23:50.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:50.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034246+0000", 2023-11-25T04:23:50.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0149999999999998e-06 2023-11-25T04:23:50.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.293 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:50.293 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035572+0000", 2023-11-25T04:23:50.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001325655 2023-11-25T04:23:50.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.293 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:50.293 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035730+0000", 2023-11-25T04:23:50.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015827899999999999 2023-11-25T04:23:50.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.293 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:50.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035857+0000", 2023-11-25T04:23:50.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012644699999999999 2023-11-25T04:23:50.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:50.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038737+0000", 2023-11-25T04:23:50.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002880077 2023-11-25T04:23:50.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:50.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044540+0000", 2023-11-25T04:23:50.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058034239999999997 2023-11-25T04:23:50.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:50.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044569+0000", 2023-11-25T04:23:50.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8388999999999999e-05 2023-11-25T04:23:50.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:50.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044589+0000", 2023-11-25T04:23:50.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0686000000000002e-05 2023-11-25T04:23:50.295 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:50.295 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:50.296 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:50.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.296 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:33 3.1b 3:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:50.296 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:50.296 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 63.193109145999998, 2023-11-25T04:23:50.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099754049999999997, 2023-11-25T04:23:50.296 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:50.296 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:23:50.296 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:50.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.296 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:50.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:50.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:50.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:50.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:50.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:50.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:50.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:50.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4799999999999998e-07 2023-11-25T04:23:50.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:50.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034273+0000", 2023-11-25T04:23:50.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7819999999999998e-06 2023-11-25T04:23:50.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.298 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:50.298 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034285+0000", 2023-11-25T04:23:50.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2058e-05 2023-11-25T04:23:50.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.298 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:50.298 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034291+0000", 2023-11-25T04:23:50.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0680000000000002e-06 2023-11-25T04:23:50.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:50.299 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034370+0000", 2023-11-25T04:23:50.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8452e-05 2023-11-25T04:23:50.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:50.299 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034477+0000", 2023-11-25T04:23:50.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107471 2023-11-25T04:23:50.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:50.299 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044245+0000", 2023-11-25T04:23:50.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0097679259999999997 2023-11-25T04:23:50.299 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:50.300 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:50.300 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:50.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.300 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:36 3.1a 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:50.300 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:50.300 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 63.193062472999998, 2023-11-25T04:23:50.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058179099999999999, 2023-11-25T04:23:50.300 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:50.300 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:23:50.300 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:50.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.300 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:50.300 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:50.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:50.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.301 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:50.301 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:50.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:50.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.301 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:50.301 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034317+0000", 2023-11-25T04:23:50.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1999999999999999e-07 2023-11-25T04:23:50.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.301 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:50.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034321+0000", 2023-11-25T04:23:50.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.233e-06 2023-11-25T04:23:50.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:50.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034337+0000", 2023-11-25T04:23:50.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5308999999999999e-05 2023-11-25T04:23:50.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:50.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034354+0000", 2023-11-25T04:23:50.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7411999999999999e-05 2023-11-25T04:23:50.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.303 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:50.303 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034465+0000", 2023-11-25T04:23:50.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011154699999999999 2023-11-25T04:23:50.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.303 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:50.303 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034560+0000", 2023-11-25T04:23:50.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4186999999999998e-05 2023-11-25T04:23:50.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.303 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:50.303 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.040134+0000", 2023-11-25T04:23:50.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055746019999999997 2023-11-25T04:23:50.303 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:50.304 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:50.304 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:50.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.304 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:9 3.10 e18/16 3:09401a81:::601.00000000:head v 18'4, mlcod=0'0)", 2023-11-25T04:23:50.304 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:50.304 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 63.184168165000003, 2023-11-25T04:23:50.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061641020000000003, 2023-11-25T04:23:50.304 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:50.304 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:50.304 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:50.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.304 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:50.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:50.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:50.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:50.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:50.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:50.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:50.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:50.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5300000000000001e-07 2023-11-25T04:23:50.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:50.306 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043222+0000", 2023-11-25T04:23:50.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.082e-05 2023-11-25T04:23:50.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:50.306 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043228+0000", 2023-11-25T04:23:50.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6699999999999997e-06 2023-11-25T04:23:50.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:50.306 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043236+0000", 2023-11-25T04:23:50.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8189999999999992e-06 2023-11-25T04:23:50.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:50.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043507+0000", 2023-11-25T04:23:50.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000271012 2023-11-25T04:23:50.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:50.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043530+0000", 2023-11-25T04:23:50.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2659000000000001e-05 2023-11-25T04:23:50.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:50.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049321+0000", 2023-11-25T04:23:50.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057915129999999999 2023-11-25T04:23:50.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.308 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:50.308 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049375+0000", 2023-11-25T04:23:50.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3455999999999998e-05 2023-11-25T04:23:50.308 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:50.308 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:50.308 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:50.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.308 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:1 3.14 e18/16 3:292cf221:::200.00000000:head v 18'3, mlcod=0'0)", 2023-11-25T04:23:50.308 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:50.308 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 63.184131452000003, 2023-11-25T04:23:50.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0060983599999999997, 2023-11-25T04:23:50.309 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:50.309 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:50.309 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:50.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.309 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:50.309 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:50.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:50.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.309 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:50.309 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:50.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:50.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:50.310 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043248+0000", 2023-11-25T04:23:50.310 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9399999999999995e-07 2023-11-25T04:23:50.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:50.310 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043252+0000", 2023-11-25T04:23:50.310 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1799999999999998e-06 2023-11-25T04:23:50.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:50.310 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043259+0000", 2023-11-25T04:23:50.310 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0049999999999998e-06 2023-11-25T04:23:50.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:50.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043265+0000", 2023-11-25T04:23:50.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0859999999999998e-06 2023-11-25T04:23:50.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:50.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043311+0000", 2023-11-25T04:23:50.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6082999999999999e-05 2023-11-25T04:23:50.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:50.312 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043330+0000", 2023-11-25T04:23:50.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8556e-05 2023-11-25T04:23:50.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.312 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:50.312 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049303+0000", 2023-11-25T04:23:50.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059733470000000004 2023-11-25T04:23:50.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.312 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:50.312 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049346+0000", 2023-11-25T04:23:50.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2608999999999998e-05 2023-11-25T04:23:50.312 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:50.312 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:50.313 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:50.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.313 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:3 3.14 e18/16 3:287034c4:::1.00000000.inode:head v 18'4, mlcod=0'0)", 2023-11-25T04:23:50.313 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:50.313 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 63.183614945999999, 2023-11-25T04:23:50.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057040090000000003, 2023-11-25T04:23:50.313 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:50.313 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:50.313 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:50.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.313 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:50.313 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:50.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:50.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.314 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:50.314 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:50.314 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:50.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.314 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:50.314 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:50.314 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9799999999999999e-07 2023-11-25T04:23:50.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.314 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:50.314 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043785+0000", 2023-11-25T04:23:50.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0721999999999999e-05 2023-11-25T04:23:50.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.315 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:50.315 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043793+0000", 2023-11-25T04:23:50.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5839999999999995e-06 2023-11-25T04:23:50.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.315 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:50.315 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043803+0000", 2023-11-25T04:23:50.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0729999999999994e-06 2023-11-25T04:23:50.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.315 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:50.316 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043843+0000", 2023-11-25T04:23:50.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0747999999999997e-05 2023-11-25T04:23:50.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.316 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:50.316 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043856+0000", 2023-11-25T04:23:50.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2697000000000001e-05 2023-11-25T04:23:50.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.316 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:50.316 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049426+0000", 2023-11-25T04:23:50.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055704409999999998 2023-11-25T04:23:50.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.317 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:50.317 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049468+0000", 2023-11-25T04:23:50.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1446000000000003e-05 2023-11-25T04:23:50.317 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:50.317 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:50.317 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:50.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.317 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "pg_update_log_missing(3.14 epoch 18/16 rep_tid 171 entries 18'5 (0'0) error 3:2e2fa760:::200.00000001:head by mds.0.3:32 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:23:50.317 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:50.317 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 63.183453983, 2023-11-25T04:23:50.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005556938, 2023-11-25T04:23:50.317 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:50.318 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2023-11-25T04:23:50.318 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:50.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.318 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:50.318 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:50.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:50.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.318 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:50.318 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:50.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:50.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.319 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:50.319 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:50.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.03e-07 2023-11-25T04:23:50.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.319 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:50.319 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043931+0000", 2023-11-25T04:23:50.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1079999999999998e-06 2023-11-25T04:23:50.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.319 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:50.319 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043944+0000", 2023-11-25T04:23:50.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2852e-05 2023-11-25T04:23:50.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:50.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043955+0000", 2023-11-25T04:23:50.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1037e-05 2023-11-25T04:23:50.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:50.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044088+0000", 2023-11-25T04:23:50.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013269599999999999 2023-11-25T04:23:50.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:50.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049482+0000", 2023-11-25T04:23:50.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053940419999999999 2023-11-25T04:23:50.321 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:50.321 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:50.321 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:50.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.321 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:10 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'5, mlcod=0'0)", 2023-11-25T04:23:50.321 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:50.321 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 63.183363954999997, 2023-11-25T04:23:50.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054493099999999997, 2023-11-25T04:23:50.321 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:50.321 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:50.321 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:50.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:50.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:50.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:50.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:50.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:50.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:50.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:50.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:50.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6100000000000002e-07 2023-11-25T04:23:50.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.323 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:50.323 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044019+0000", 2023-11-25T04:23:50.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.839e-06 2023-11-25T04:23:50.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.323 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:50.323 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044026+0000", 2023-11-25T04:23:50.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5869999999999998e-06 2023-11-25T04:23:50.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.323 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:50.323 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044031+0000", 2023-11-25T04:23:50.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4909999999999996e-06 2023-11-25T04:23:50.324 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.324 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.324 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:50.324 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044070+0000", 2023-11-25T04:23:50.324 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8829999999999999e-05 2023-11-25T04:23:50.324 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.324 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.324 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:50.324 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044081+0000", 2023-11-25T04:23:50.324 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1447999999999999e-05 2023-11-25T04:23:50.324 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.324 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.324 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:50.325 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049406+0000", 2023-11-25T04:23:50.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053246480000000004 2023-11-25T04:23:50.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.325 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:50.325 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049464+0000", 2023-11-25T04:23:50.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8205999999999998e-05 2023-11-25T04:23:50.325 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:50.325 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:50.325 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:50.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.325 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:11 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'6, mlcod=0'0)", 2023-11-25T04:23:50.325 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:50.326 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 63.183162691, 2023-11-25T04:23:50.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052673499999999996, 2023-11-25T04:23:50.326 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:50.326 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:50.326 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:50.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:50.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:50.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:50.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:50.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:50.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:50.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.327 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:50.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:50.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0199999999999998e-07 2023-11-25T04:23:50.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.327 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:50.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044220+0000", 2023-11-25T04:23:50.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7720000000000001e-06 2023-11-25T04:23:50.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.327 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:50.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044226+0000", 2023-11-25T04:23:50.328 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8579999999999996e-06 2023-11-25T04:23:50.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.328 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.328 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:50.328 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044232+0000", 2023-11-25T04:23:50.328 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1700000000000002e-06 2023-11-25T04:23:50.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.328 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.328 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:50.328 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044274+0000", 2023-11-25T04:23:50.328 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1254999999999997e-05 2023-11-25T04:23:50.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.328 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.328 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:50.329 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:23:50.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6880999999999999e-05 2023-11-25T04:23:50.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.329 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.329 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:50.329 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049466+0000", 2023-11-25T04:23:50.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051659219999999999 2023-11-25T04:23:50.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:50.329 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:50.329 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:50.329 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049483+0000", 2023-11-25T04:23:50.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.719e-05 2023-11-25T04:23:50.329 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:50.329 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:50.330 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:50.330 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:50.330 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:50.330 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:50.331 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:23:50.332 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:50.330+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:50.432 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:50.431+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:50.438 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:50.438 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:23:50.438 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2023-11-25T04:23:50.438 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:50.447 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:23:50.533 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:50.532+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:50.552 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:50.552 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:23:50.552 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2023-11-25T04:23:50.552 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2023-11-25T04:23:50.552 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:50.561 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:23:50.635 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:50.633+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:50.710 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:50.711 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2023-11-25T04:23:50.711 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2023-11-25T04:23:50.711 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2023-11-25T04:23:50.711 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.712 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:50.712 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:50.712 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 63.674286991999999, 2023-11-25T04:23:50.712 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01072943, 2023-11-25T04:23:50.712 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:50.712 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:50.712 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:50.712 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.712 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:50.713 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:50.713 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:50.713 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.713 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.713 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:50.713 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:50.713 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:50.713 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.713 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.713 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:50.713 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031829+0000", 2023-11-25T04:23:50.713 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.666e-06 2023-11-25T04:23:50.714 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.714 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.714 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:50.714 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031839+0000", 2023-11-25T04:23:50.714 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9490000000000008e-06 2023-11-25T04:23:50.714 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.714 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.714 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:50.714 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031857+0000", 2023-11-25T04:23:50.714 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8093000000000001e-05 2023-11-25T04:23:50.714 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.715 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.715 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:50.715 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031882+0000", 2023-11-25T04:23:50.715 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4294000000000002e-05 2023-11-25T04:23:50.715 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.715 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.715 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:50.715 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033130+0000", 2023-11-25T04:23:50.715 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001248264 2023-11-25T04:23:50.715 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.715 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.715 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:50.716 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033426+0000", 2023-11-25T04:23:50.716 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029620999999999998 2023-11-25T04:23:50.716 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.716 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.716 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:50.716 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033814+0000", 2023-11-25T04:23:50.716 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000387742 2023-11-25T04:23:50.716 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.716 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.716 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:50.716 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036793+0000", 2023-11-25T04:23:50.717 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002979598 2023-11-25T04:23:50.717 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.717 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.717 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:50.717 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042455+0000", 2023-11-25T04:23:50.717 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0056619260000000003 2023-11-25T04:23:50.717 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.717 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.717 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:50.717 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042502+0000", 2023-11-25T04:23:50.717 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7139999999999999e-05 2023-11-25T04:23:50.717 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.718 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.718 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:50.718 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042557+0000", 2023-11-25T04:23:50.718 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4548e-05 2023-11-25T04:23:50.718 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:50.718 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:50.718 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:50.718 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.718 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.718 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:50.718 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:50.718 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 63.673088456999999, 2023-11-25T04:23:50.719 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011505202000000001, 2023-11-25T04:23:50.719 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:50.719 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:50.719 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:50.719 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.719 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:50.719 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:50.719 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:50.719 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.719 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.719 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:50.719 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:50.720 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:50.720 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.720 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.720 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:50.720 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033027+0000", 2023-11-25T04:23:50.720 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0699999999999999e-06 2023-11-25T04:23:50.720 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.720 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.720 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:50.720 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033035+0000", 2023-11-25T04:23:50.720 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0830000000000004e-06 2023-11-25T04:23:50.720 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.721 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.721 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:50.721 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033059+0000", 2023-11-25T04:23:50.721 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3975e-05 2023-11-25T04:23:50.721 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.721 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.721 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:50.721 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033482+0000", 2023-11-25T04:23:50.721 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042253700000000001 2023-11-25T04:23:50.721 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.721 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.721 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:50.722 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034520+0000", 2023-11-25T04:23:50.722 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001037991 2023-11-25T04:23:50.722 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.722 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.722 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:50.722 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034611+0000", 2023-11-25T04:23:50.722 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1113999999999995e-05 2023-11-25T04:23:50.722 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.722 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.722 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:50.722 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034769+0000", 2023-11-25T04:23:50.722 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000157641 2023-11-25T04:23:50.723 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.723 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.723 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:50.723 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038292+0000", 2023-11-25T04:23:50.723 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035231160000000002 2023-11-25T04:23:50.723 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.723 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.723 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:50.723 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044440+0000", 2023-11-25T04:23:50.723 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0061486980000000002 2023-11-25T04:23:50.723 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.723 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.724 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:50.724 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044495+0000", 2023-11-25T04:23:50.724 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4339000000000003e-05 2023-11-25T04:23:50.724 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.724 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.724 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:50.724 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044531+0000", 2023-11-25T04:23:50.724 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6637999999999997e-05 2023-11-25T04:23:50.724 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:50.724 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:50.725 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:50.725 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.725 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.725 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:50.725 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:50.725 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 63.672614103000001, 2023-11-25T04:23:50.725 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0057634039999999997, 2023-11-25T04:23:50.725 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:50.725 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:50.725 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:50.725 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.725 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:50.725 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:50.726 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:50.726 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.726 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.726 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:50.726 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:50.726 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:50.726 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.726 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.726 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:50.726 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:50.727 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7900000000000007e-07 2023-11-25T04:23:50.727 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.727 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.727 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:50.727 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033507+0000", 2023-11-25T04:23:50.727 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8900000000000004e-06 2023-11-25T04:23:50.727 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.727 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.727 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:50.727 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033519+0000", 2023-11-25T04:23:50.727 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1756000000000001e-05 2023-11-25T04:23:50.727 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.728 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.728 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:50.728 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033527+0000", 2023-11-25T04:23:50.728 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3610000000000001e-06 2023-11-25T04:23:50.728 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.728 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.728 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:50.728 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034451+0000", 2023-11-25T04:23:50.728 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00092372400000000001 2023-11-25T04:23:50.728 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.728 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.728 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:50.729 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034809+0000", 2023-11-25T04:23:50.729 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00035757699999999998 2023-11-25T04:23:50.729 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.729 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.729 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:23:50.729 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034965+0000", 2023-11-25T04:23:50.729 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015638699999999999 2023-11-25T04:23:50.729 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.729 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.729 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:50.729 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038299+0000", 2023-11-25T04:23:50.729 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0033336559999999999 2023-11-25T04:23:50.729 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.730 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.730 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:50.730 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039169+0000", 2023-11-25T04:23:50.730 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00087066400000000005 2023-11-25T04:23:50.730 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.730 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.730 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:50.730 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039226+0000", 2023-11-25T04:23:50.730 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6511999999999997e-05 2023-11-25T04:23:50.730 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.731 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.731 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:50.731 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039264+0000", 2023-11-25T04:23:50.731 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7898000000000002e-05 2023-11-25T04:23:50.731 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:50.731 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:50.731 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:50.731 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.731 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.731 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:25 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:50.732 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:50.732 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 63.672535519, 2023-11-25T04:23:50.732 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096656169999999996, 2023-11-25T04:23:50.732 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:50.732 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:50.732 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:50.732 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.732 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:50.732 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:50.732 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:50.732 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.732 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.732 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:50.733 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:50.733 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:50.733 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.733 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.733 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:50.733 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033580+0000", 2023-11-25T04:23:50.733 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8199999999999999e-07 2023-11-25T04:23:50.733 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.733 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.733 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:50.733 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033585+0000", 2023-11-25T04:23:50.733 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3680000000000001e-06 2023-11-25T04:23:50.733 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.734 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.734 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:50.734 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033600+0000", 2023-11-25T04:23:50.734 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4331000000000001e-05 2023-11-25T04:23:50.734 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.734 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.734 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:50.734 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033606+0000", 2023-11-25T04:23:50.734 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5780000000000004e-06 2023-11-25T04:23:50.734 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.734 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.735 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:50.735 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035747+0000", 2023-11-25T04:23:50.735 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0021406530000000002 2023-11-25T04:23:50.735 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.735 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.735 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:50.735 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035901+0000", 2023-11-25T04:23:50.735 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000153931 2023-11-25T04:23:50.735 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.735 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.735 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:23:50.735 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036193+0000", 2023-11-25T04:23:50.735 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029211299999999999 2023-11-25T04:23:50.736 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.736 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.736 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:50.736 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038655+0000", 2023-11-25T04:23:50.736 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0024621640000000002 2023-11-25T04:23:50.736 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.736 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.736 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:50.736 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043130+0000", 2023-11-25T04:23:50.736 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.004474698 2023-11-25T04:23:50.736 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.736 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.736 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:50.737 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043204+0000", 2023-11-25T04:23:50.737 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4091999999999995e-05 2023-11-25T04:23:50.737 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.737 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.737 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:50.737 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043245+0000", 2023-11-25T04:23:50.737 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0907e-05 2023-11-25T04:23:50.737 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:50.737 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:50.737 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:50.737 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.737 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.737 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:26 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:50.738 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:50.738 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 63.672499694999999, 2023-11-25T04:23:50.738 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011086945000000001, 2023-11-25T04:23:50.738 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:50.738 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:50.738 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:50.738 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.738 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:50.738 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:50.738 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:50.738 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.738 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.738 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:50.739 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:50.739 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:50.739 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.739 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.739 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:50.739 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:50.739 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1699999999999998e-07 2023-11-25T04:23:50.739 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.739 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.739 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:50.739 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033627+0000", 2023-11-25T04:23:50.739 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.163e-05 2023-11-25T04:23:50.739 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.740 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.740 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:50.740 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033636+0000", 2023-11-25T04:23:50.740 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6859999999999995e-06 2023-11-25T04:23:50.740 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.740 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.740 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:50.741 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033642+0000", 2023-11-25T04:23:50.741 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3149999999999997e-06 2023-11-25T04:23:50.741 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.741 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.741 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:50.741 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035414+0000", 2023-11-25T04:23:50.741 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0017716380000000001 2023-11-25T04:23:50.742 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.742 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.742 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:50.742 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035539+0000", 2023-11-25T04:23:50.742 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000125477 2023-11-25T04:23:50.742 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.742 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.742 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:50.742 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035672+0000", 2023-11-25T04:23:50.742 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000133316 2023-11-25T04:23:50.742 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.742 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.742 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:50.743 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038649+0000", 2023-11-25T04:23:50.743 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029764679999999999 2023-11-25T04:23:50.743 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.743 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.743 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:50.743 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044578+0000", 2023-11-25T04:23:50.743 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0059293510000000002 2023-11-25T04:23:50.743 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.743 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.743 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:50.743 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044651+0000", 2023-11-25T04:23:50.743 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2513000000000002e-05 2023-11-25T04:23:50.743 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.744 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.744 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:50.744 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044702+0000", 2023-11-25T04:23:50.744 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1034e-05 2023-11-25T04:23:50.744 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:50.744 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:50.744 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:50.744 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.744 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.744 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:29 3.b 3:d0630e4c:::mds0_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:50.744 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:50.744 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 63.672461775000002, 2023-11-25T04:23:50.745 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010980227, 2023-11-25T04:23:50.745 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:50.745 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:50.745 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:50.745 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.745 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:50.745 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:50.745 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:50.745 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.745 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.745 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:50.745 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:50.745 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:50.746 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.746 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.746 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:50.746 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:50.746 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3499999999999996e-07 2023-11-25T04:23:50.746 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.746 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.746 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:50.746 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033658+0000", 2023-11-25T04:23:50.746 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1280000000000001e-06 2023-11-25T04:23:50.746 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.746 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.746 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:50.747 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033667+0000", 2023-11-25T04:23:50.747 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1430000000000004e-06 2023-11-25T04:23:50.747 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.747 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.747 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:50.747 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033678+0000", 2023-11-25T04:23:50.747 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0906e-05 2023-11-25T04:23:50.747 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.747 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.747 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:50.747 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034456+0000", 2023-11-25T04:23:50.747 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00077833899999999996 2023-11-25T04:23:50.747 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.748 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.748 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:50.748 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034575+0000", 2023-11-25T04:23:50.748 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011926000000000001 2023-11-25T04:23:50.748 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.748 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.748 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:50.748 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035001+0000", 2023-11-25T04:23:50.748 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042550999999999998 2023-11-25T04:23:50.748 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.748 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.748 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:50.748 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.037923+0000", 2023-11-25T04:23:50.748 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029219670000000001 2023-11-25T04:23:50.749 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.749 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.749 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:50.749 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044538+0000", 2023-11-25T04:23:50.749 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0066152349999999997 2023-11-25T04:23:50.749 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.749 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.749 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:50.749 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044594+0000", 2023-11-25T04:23:50.749 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5593000000000001e-05 2023-11-25T04:23:50.749 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.749 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.749 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:50.749 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044633+0000", 2023-11-25T04:23:50.750 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9511000000000002e-05 2023-11-25T04:23:50.750 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:50.750 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:50.750 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:50.750 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.750 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.750 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:34 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:50.750 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:50.750 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 63.672211662000002, 2023-11-25T04:23:50.750 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010920445000000001, 2023-11-25T04:23:50.750 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:50.750 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:50.750 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:50.751 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.751 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:50.751 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:50.751 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:50.751 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.751 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.751 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:50.751 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:50.751 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:50.751 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.751 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.751 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:50.751 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033904+0000", 2023-11-25T04:23:50.751 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6599999999999994e-07 2023-11-25T04:23:50.752 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.752 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.752 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:50.752 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033909+0000", 2023-11-25T04:23:50.752 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5639999999999996e-06 2023-11-25T04:23:50.752 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.752 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.752 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:50.752 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033923+0000", 2023-11-25T04:23:50.752 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3028e-05 2023-11-25T04:23:50.752 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.752 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.752 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:50.753 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034043+0000", 2023-11-25T04:23:50.753 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000120328 2023-11-25T04:23:50.753 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.753 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.753 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:50.753 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035576+0000", 2023-11-25T04:23:50.753 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015333720000000001 2023-11-25T04:23:50.753 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.753 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.753 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:50.753 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035699+0000", 2023-11-25T04:23:50.753 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000123001 2023-11-25T04:23:50.753 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.753 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.754 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:50.754 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044823+0000", 2023-11-25T04:23:50.754 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0091241859999999994 2023-11-25T04:23:50.754 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:50.754 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:50.754 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:50.754 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.754 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.754 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:35 3.1f 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:50.754 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:50.754 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 63.670907456999998, 2023-11-25T04:23:50.754 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096164779999999995, 2023-11-25T04:23:50.754 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:50.755 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:50.755 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:50.755 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.755 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:50.755 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:50.755 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:50.755 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.755 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.755 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:50.755 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:50.755 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:50.755 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.755 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.755 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:50.756 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035208+0000", 2023-11-25T04:23:50.756 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2699999999999998e-07 2023-11-25T04:23:50.756 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.756 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.756 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:50.756 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035213+0000", 2023-11-25T04:23:50.756 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.6070000000000001e-06 2023-11-25T04:23:50.756 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.756 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.756 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:50.756 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035350+0000", 2023-11-25T04:23:50.756 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013693599999999999 2023-11-25T04:23:50.756 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.756 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.757 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:50.757 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035358+0000", 2023-11-25T04:23:50.757 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0099999999999995e-06 2023-11-25T04:23:50.757 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.757 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.757 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:50.757 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036260+0000", 2023-11-25T04:23:50.757 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00090240100000000005 2023-11-25T04:23:50.757 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.757 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.757 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:50.757 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036517+0000", 2023-11-25T04:23:50.757 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00025736299999999998 2023-11-25T04:23:50.757 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.758 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.758 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:50.758 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044824+0000", 2023-11-25T04:23:50.758 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0083063340000000003 2023-11-25T04:23:50.758 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:50.758 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:50.758 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:50.758 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.758 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.758 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:37 3.1e 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:50.758 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:50.758 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 63.670740770999998, 2023-11-25T04:23:50.758 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0078322640000000002, 2023-11-25T04:23:50.759 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:50.759 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:50.759 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:50.759 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.759 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:50.759 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:50.759 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:50.759 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.759 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.759 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:50.759 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:50.759 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:50.759 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.760 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.760 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:50.760 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035375+0000", 2023-11-25T04:23:50.760 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9100000000000003e-07 2023-11-25T04:23:50.760 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.760 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.760 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:50.760 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035379+0000", 2023-11-25T04:23:50.760 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9430000000000002e-06 2023-11-25T04:23:50.760 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.760 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.760 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:50.760 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035388+0000", 2023-11-25T04:23:50.761 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2760000000000006e-06 2023-11-25T04:23:50.761 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.761 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.761 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:50.761 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035396+0000", 2023-11-25T04:23:50.761 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8480000000000001e-06 2023-11-25T04:23:50.761 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.761 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.761 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:50.761 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035452+0000", 2023-11-25T04:23:50.761 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5831000000000001e-05 2023-11-25T04:23:50.761 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.761 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.762 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:50.762 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035934+0000", 2023-11-25T04:23:50.762 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00048265599999999998 2023-11-25T04:23:50.762 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.762 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.762 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:50.762 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043206+0000", 2023-11-25T04:23:50.762 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.007271919 2023-11-25T04:23:50.762 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:50.762 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:50.762 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:50.762 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.762 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.763 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing(3.1a epoch 18/16 rep_tid 3 entries 18'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:36 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:23:50.763 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:50.763 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 63.669465185, 2023-11-25T04:23:50.763 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002073414, 2023-11-25T04:23:50.763 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:50.763 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:50.763 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:50.763 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.763 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:50.763 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:50.763 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:50.763 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.763 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.764 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:50.764 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:50.764 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:50.764 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.764 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.764 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:50.764 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036650+0000", 2023-11-25T04:23:50.764 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7300000000000002e-07 2023-11-25T04:23:50.764 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.764 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.764 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:50.764 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036659+0000", 2023-11-25T04:23:50.764 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.7499999999999992e-06 2023-11-25T04:23:50.765 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.765 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.765 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:50.765 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036673+0000", 2023-11-25T04:23:50.765 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4917e-05 2023-11-25T04:23:50.765 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.765 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.765 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:50.765 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036687+0000", 2023-11-25T04:23:50.765 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3883000000000001e-05 2023-11-25T04:23:50.765 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.765 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.765 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:50.766 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036781+0000", 2023-11-25T04:23:50.766 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3219000000000003e-05 2023-11-25T04:23:50.766 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.766 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.766 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:50.766 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038723+0000", 2023-11-25T04:23:50.766 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019422720000000001 2023-11-25T04:23:50.766 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:50.766 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:50.766 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:50.766 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.766 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.766 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:28 3.13 e18/16 3:cd5a64a3:::100.00000000:head v 18'1, mlcod=0'0)", 2023-11-25T04:23:50.767 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:50.767 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 63.665804027, 2023-11-25T04:23:50.767 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001850546, 2023-11-25T04:23:50.767 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:50.767 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:50.767 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:50.767 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.767 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:50.767 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:50.767 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:50.767 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.767 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.767 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:50.767 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:50.768 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:50.768 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.768 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.768 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:50.768 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:50.768 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7800000000000002e-07 2023-11-25T04:23:50.768 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.768 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.768 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:50.768 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040331+0000", 2023-11-25T04:23:50.768 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0449999999999999e-05 2023-11-25T04:23:50.768 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.768 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.769 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:50.769 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040339+0000", 2023-11-25T04:23:50.769 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0709999999999995e-06 2023-11-25T04:23:50.769 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.769 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.769 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:50.769 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040347+0000", 2023-11-25T04:23:50.769 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7079999999999997e-06 2023-11-25T04:23:50.769 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.769 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.769 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:50.770 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040410+0000", 2023-11-25T04:23:50.770 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2453000000000007e-05 2023-11-25T04:23:50.770 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.770 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.770 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:50.770 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040422+0000", 2023-11-25T04:23:50.770 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2429000000000001e-05 2023-11-25T04:23:50.770 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.770 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.770 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:50.770 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042129+0000", 2023-11-25T04:23:50.770 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001707266 2023-11-25T04:23:50.770 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.771 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.771 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:50.771 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042161+0000", 2023-11-25T04:23:50.771 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1791000000000001e-05 2023-11-25T04:23:50.771 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:50.771 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:50.771 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:50.771 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.771 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.771 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:27 3.13 e18/16 3:c91d4a1d:::mds0_inotable:head v 18'2, mlcod=0'0)", 2023-11-25T04:23:50.771 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:50.771 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 63.665600503999997, 2023-11-25T04:23:50.771 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0016886430000000001, 2023-11-25T04:23:50.771 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:50.772 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:50.772 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:50.772 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.772 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:50.772 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:50.772 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:50.772 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.772 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.772 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:50.772 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:50.772 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:50.772 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.772 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.772 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:50.773 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:50.773 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3999999999999997e-07 2023-11-25T04:23:50.773 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.773 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.773 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:50.773 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040520+0000", 2023-11-25T04:23:50.773 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2989999999999996e-06 2023-11-25T04:23:50.773 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.773 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.773 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:50.773 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040527+0000", 2023-11-25T04:23:50.773 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5730000000000002e-06 2023-11-25T04:23:50.773 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.774 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.774 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:50.774 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040535+0000", 2023-11-25T04:23:50.774 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8630000000000008e-06 2023-11-25T04:23:50.774 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.774 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.774 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:50.774 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040683+0000", 2023-11-25T04:23:50.774 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000148002 2023-11-25T04:23:50.774 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.774 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.774 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:50.774 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040700+0000", 2023-11-25T04:23:50.775 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7065e-05 2023-11-25T04:23:50.775 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.775 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.775 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:50.775 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042163+0000", 2023-11-25T04:23:50.775 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001462767 2023-11-25T04:23:50.775 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.775 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.775 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:50.775 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042203+0000", 2023-11-25T04:23:50.775 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9734000000000003e-05 2023-11-25T04:23:50.775 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:50.775 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:50.775 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:50.776 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.776 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.776 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:20 3.1c e18/16 3:3ed09add:::607.00000000:head v 18'1, mlcod=0'0)", 2023-11-25T04:23:50.776 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:50.776 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 63.665565463, 2023-11-25T04:23:50.776 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015271, 2023-11-25T04:23:50.776 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:50.776 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:50.776 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:50.776 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.776 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:50.776 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:50.776 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:50.777 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.777 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.777 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:50.777 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:50.777 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:50.777 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.777 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.777 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:50.777 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:50.777 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4799999999999999e-07 2023-11-25T04:23:50.777 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.778 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.778 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:50.778 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040561+0000", 2023-11-25T04:23:50.778 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1800000000000001e-05 2023-11-25T04:23:50.778 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.778 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.778 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:50.778 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040568+0000", 2023-11-25T04:23:50.778 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7730000000000002e-06 2023-11-25T04:23:50.778 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.778 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.778 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:50.778 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040587+0000", 2023-11-25T04:23:50.779 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8691000000000001e-05 2023-11-25T04:23:50.779 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.779 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.779 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:50.779 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040638+0000", 2023-11-25T04:23:50.779 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1068e-05 2023-11-25T04:23:50.779 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.779 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.779 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:50.779 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040670+0000", 2023-11-25T04:23:50.779 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2125999999999997e-05 2023-11-25T04:23:50.779 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.779 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.780 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:50.780 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042032+0000", 2023-11-25T04:23:50.780 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001361821 2023-11-25T04:23:50.780 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.780 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.780 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:50.780 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042076+0000", 2023-11-25T04:23:50.780 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4472999999999999e-05 2023-11-25T04:23:50.780 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:50.780 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:50.780 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:50.780 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.780 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.780 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:21 3.1c e18/16 3:3ed09add:::607.00000000:head v 18'2, mlcod=0'0)", 2023-11-25T04:23:50.781 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:50.781 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 63.665330738000002, 2023-11-25T04:23:50.781 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0013228160000000001, 2023-11-25T04:23:50.781 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:50.781 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:50.781 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:50.781 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.781 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:50.781 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:50.781 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:50.781 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.781 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.781 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:50.782 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:50.782 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:50.782 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.782 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.782 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:50.782 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:50.782 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9299999999999999e-07 2023-11-25T04:23:50.782 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.782 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.782 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:50.782 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040790+0000", 2023-11-25T04:23:50.782 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5770000000000001e-06 2023-11-25T04:23:50.782 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.782 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.783 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:50.783 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040799+0000", 2023-11-25T04:23:50.783 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6439999999999995e-06 2023-11-25T04:23:50.783 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.783 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.783 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:50.783 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040806+0000", 2023-11-25T04:23:50.783 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0180000000000004e-06 2023-11-25T04:23:50.783 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.783 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.783 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:50.783 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040907+0000", 2023-11-25T04:23:50.783 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000100096 2023-11-25T04:23:50.784 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.784 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.784 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:50.784 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040925+0000", 2023-11-25T04:23:50.784 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8536999999999999e-05 2023-11-25T04:23:50.784 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.784 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.784 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:50.784 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042079+0000", 2023-11-25T04:23:50.784 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00115358 2023-11-25T04:23:50.784 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.784 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.784 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:50.784 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042107+0000", 2023-11-25T04:23:50.785 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7971e-05 2023-11-25T04:23:50.785 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:50.785 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:50.785 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:50.785 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.785 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.785 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop_reply(mds.0.3:4 3.1f e18/16 ondisk, result = 0)", 2023-11-25T04:23:50.785 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:50.785 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 63.663991348000003, 2023-11-25T04:23:50.785 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00044092900000000001, 2023-11-25T04:23:50.785 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:50.786 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:50.786 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:50.786 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.786 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:50.786 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:50.786 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:50.786 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.786 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.786 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:50.786 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:50.786 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:50.786 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.786 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.786 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:50.787 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042124+0000", 2023-11-25T04:23:50.787 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1800000000000005e-07 2023-11-25T04:23:50.787 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.787 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.787 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:50.787 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042128+0000", 2023-11-25T04:23:50.787 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2289999999999997e-06 2023-11-25T04:23:50.787 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.787 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.787 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:50.787 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042138+0000", 2023-11-25T04:23:50.788 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9729999999999992e-06 2023-11-25T04:23:50.788 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.788 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.788 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:50.788 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042145+0000", 2023-11-25T04:23:50.788 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.336e-06 2023-11-25T04:23:50.788 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.788 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.788 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:50.788 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042441+0000", 2023-11-25T04:23:50.788 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029650500000000002 2023-11-25T04:23:50.788 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.789 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.789 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:50.789 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042448+0000", 2023-11-25T04:23:50.789 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4100000000000002e-06 2023-11-25T04:23:50.789 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.789 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.789 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:50.789 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042564+0000", 2023-11-25T04:23:50.789 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000115758 2023-11-25T04:23:50.789 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:50.789 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:50.789 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:50.789 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.790 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.790 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing_reply(3.1f epoch 18/16 rep_tid 8 lcod 18'4)", 2023-11-25T04:23:50.790 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:50.790 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 63.661665935999999, 2023-11-25T04:23:50.790 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00038154699999999998, 2023-11-25T04:23:50.790 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:50.790 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:50.790 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:50.790 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.790 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:50.790 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:50.790 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:50.790 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.790 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.791 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:50.791 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:50.791 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:50.791 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.791 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.791 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:50.791 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:50.791 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2700000000000001e-07 2023-11-25T04:23:50.791 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.791 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.791 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:50.791 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044451+0000", 2023-11-25T04:23:50.791 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7460000000000001e-06 2023-11-25T04:23:50.792 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.792 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.792 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:50.792 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044455+0000", 2023-11-25T04:23:50.792 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5809999999999998e-06 2023-11-25T04:23:50.792 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.792 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.792 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:50.792 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044458+0000", 2023-11-25T04:23:50.792 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0170000000000001e-06 2023-11-25T04:23:50.792 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.792 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.792 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:50.792 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044734+0000", 2023-11-25T04:23:50.793 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00027582799999999999 2023-11-25T04:23:50.793 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.793 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.793 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:50.793 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044830+0000", 2023-11-25T04:23:50.793 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6248000000000004e-05 2023-11-25T04:23:50.793 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:50.793 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:50.793 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:50.793 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.793 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.793 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing_reply(3.b epoch 18/16 rep_tid 5 lcod 18'2)", 2023-11-25T04:23:50.793 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:50.794 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 63.661629275000003, 2023-11-25T04:23:50.794 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000344488, 2023-11-25T04:23:50.794 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:50.794 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:50.794 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:50.794 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.794 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:50.794 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:50.794 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:50.794 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.794 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.794 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:50.794 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:50.794 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:50.795 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.795 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.795 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:50.795 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:50.795 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.55e-07 2023-11-25T04:23:50.796 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.796 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.796 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:50.796 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044487+0000", 2023-11-25T04:23:50.796 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2470000000000001e-06 2023-11-25T04:23:50.796 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.796 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.796 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:50.796 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044491+0000", 2023-11-25T04:23:50.796 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9659999999999998e-06 2023-11-25T04:23:50.796 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.796 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.796 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:50.796 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044494+0000", 2023-11-25T04:23:50.797 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2310000000000001e-06 2023-11-25T04:23:50.797 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.797 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.797 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:50.797 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044756+0000", 2023-11-25T04:23:50.797 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00026172800000000003 2023-11-25T04:23:50.797 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.797 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.797 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:50.797 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044830+0000", 2023-11-25T04:23:50.797 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4160999999999997e-05 2023-11-25T04:23:50.797 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:50.797 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:50.798 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:50.798 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.798 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.798 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:39 3.6 3:654134d2:::mds0_openfiles.0:head [omap-clear,omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2023-11-25T04:23:50.798 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:50.798 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 58.601803232999998, 2023-11-25T04:23:50.798 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001788658, 2023-11-25T04:23:50.798 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:50.798 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:50.798 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:50.798 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.798 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:50.798 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:50.799 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:50.799 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.799 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.799 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:50.799 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:50.799 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:50.799 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.799 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.799 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:50.799 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104314+0000", 2023-11-25T04:23:50.799 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3099999999999999e-06 2023-11-25T04:23:50.799 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.800 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.800 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:50.800 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104325+0000", 2023-11-25T04:23:50.800 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1511999999999999e-05 2023-11-25T04:23:50.800 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.800 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.800 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:50.800 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104362+0000", 2023-11-25T04:23:50.800 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6903000000000002e-05 2023-11-25T04:23:50.800 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.800 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.800 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:50.800 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104387+0000", 2023-11-25T04:23:50.801 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4766e-05 2023-11-25T04:23:50.801 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.801 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.801 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:50.801 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104465+0000", 2023-11-25T04:23:50.801 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8119000000000002e-05 2023-11-25T04:23:50.801 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.801 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.801 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:50.801 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104557+0000", 2023-11-25T04:23:50.801 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2496000000000003e-05 2023-11-25T04:23:50.801 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.801 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.801 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:50.802 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104646+0000", 2023-11-25T04:23:50.802 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8818000000000002e-05 2023-11-25T04:23:50.802 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.802 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.802 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:50.802 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105201+0000", 2023-11-25T04:23:50.802 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00055506000000000002 2023-11-25T04:23:50.802 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.802 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.802 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:50.802 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105994+0000", 2023-11-25T04:23:50.802 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00079294700000000001 2023-11-25T04:23:50.802 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.803 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.803 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:50.803 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.106055+0000", 2023-11-25T04:23:50.803 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0566000000000003e-05 2023-11-25T04:23:50.803 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.803 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.803 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:50.803 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.106100+0000", 2023-11-25T04:23:50.803 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5160999999999997e-05 2023-11-25T04:23:50.803 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:50.803 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:50.803 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:50.803 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.803 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.804 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:43 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2023-11-25T04:23:50.804 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:50.804 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 43.601578236999998, 2023-11-25T04:23:50.804 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001751166, 2023-11-25T04:23:50.804 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:50.804 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:50.804 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:50.804 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.804 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:50.804 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:50.804 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:50.804 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.804 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.805 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:50.805 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:50.805 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:50.805 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.805 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.805 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:50.805 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104539+0000", 2023-11-25T04:23:50.805 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9529999999999999e-06 2023-11-25T04:23:50.805 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.805 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.805 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:50.805 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104557+0000", 2023-11-25T04:23:50.805 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7464000000000001e-05 2023-11-25T04:23:50.805 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.806 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.806 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:50.806 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104590+0000", 2023-11-25T04:23:50.806 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2966000000000003e-05 2023-11-25T04:23:50.806 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.806 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.806 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:50.806 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104608+0000", 2023-11-25T04:23:50.806 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8193e-05 2023-11-25T04:23:50.806 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.806 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.806 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:50.806 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104767+0000", 2023-11-25T04:23:50.807 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015882 2023-11-25T04:23:50.807 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.807 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.807 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:50.807 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104909+0000", 2023-11-25T04:23:50.807 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014244999999999999 2023-11-25T04:23:50.807 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.807 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.807 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:50.807 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.105059+0000", 2023-11-25T04:23:50.807 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014964399999999999 2023-11-25T04:23:50.807 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.807 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.807 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:50.808 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.105701+0000", 2023-11-25T04:23:50.808 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00064201099999999995 2023-11-25T04:23:50.808 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.808 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.808 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:50.808 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106233+0000", 2023-11-25T04:23:50.808 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00053178500000000001 2023-11-25T04:23:50.808 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.808 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.808 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:50.808 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106265+0000", 2023-11-25T04:23:50.808 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1986000000000002e-05 2023-11-25T04:23:50.808 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.808 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.809 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:50.809 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106288+0000", 2023-11-25T04:23:50.809 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2894000000000001e-05 2023-11-25T04:23:50.809 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:50.809 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:50.809 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:50.809 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.809 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.809 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:45 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e23)", 2023-11-25T04:23:50.809 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:23:50.809 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 38.601380196999997, 2023-11-25T04:23:50.809 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00098423299999999994, 2023-11-25T04:23:50.809 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:50.810 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:50.810 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:50.810 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.810 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:50.810 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:23:50.810 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:50.810 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.810 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.810 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:50.810 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:23:50.810 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:50.810 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.810 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.811 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:50.811 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104737+0000", 2023-11-25T04:23:50.811 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3309999999999998e-06 2023-11-25T04:23:50.811 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.811 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.811 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:50.811 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104748+0000", 2023-11-25T04:23:50.811 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1080999999999999e-05 2023-11-25T04:23:50.811 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.811 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.811 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:50.811 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104771+0000", 2023-11-25T04:23:50.811 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2707000000000001e-05 2023-11-25T04:23:50.811 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.812 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.812 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:50.812 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104800+0000", 2023-11-25T04:23:50.812 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9595e-05 2023-11-25T04:23:50.812 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.812 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.812 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:50.812 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104890+0000", 2023-11-25T04:23:50.812 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.9808999999999993e-05 2023-11-25T04:23:50.812 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.812 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.812 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:50.812 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105004+0000", 2023-11-25T04:23:50.812 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000114508 2023-11-25T04:23:50.813 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.813 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.813 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:50.813 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105634+0000", 2023-11-25T04:23:50.813 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00062985799999999996 2023-11-25T04:23:50.813 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.813 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.813 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:50.813 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105671+0000", 2023-11-25T04:23:50.813 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6640000000000002e-05 2023-11-25T04:23:50.813 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:50.813 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:50.813 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:50.813 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105719+0000", 2023-11-25T04:23:50.814 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7704000000000003e-05 2023-11-25T04:23:50.814 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:50.814 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:50.814 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:50.814 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:50.814 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:50.814 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:50.815 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:23:50.816 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:50.814+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:50.916 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:50.914+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:50.924 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:50.924 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:23:50.924 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2023-11-25T04:23:50.925 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:50.933 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:23:51.016 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:51.015+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:51.038 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:51.038 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:23:51.038 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2023-11-25T04:23:51.038 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2023-11-25T04:23:51.039 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:51.047 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:23:51.131 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:51.130+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:51.194 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:51.195 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2023-11-25T04:23:51.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2023-11-25T04:23:51.195 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2023-11-25T04:23:51.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.195 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:155 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'109, mlcod=11'105)", 2023-11-25T04:23:51.195 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:51.195 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.407600685999995, 2023-11-25T04:23:51.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061719019999999999, 2023-11-25T04:23:51.195 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:51.195 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:51.196 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:51.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.196 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:51.196 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:51.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:51.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.196 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:51.196 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:51.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:51.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:51.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784884+0000", 2023-11-25T04:23:51.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3099999999999999e-07 2023-11-25T04:23:51.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:51.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784942+0000", 2023-11-25T04:23:51.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8570000000000003e-05 2023-11-25T04:23:51.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:51.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784949+0000", 2023-11-25T04:23:51.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0310000000000001e-06 2023-11-25T04:23:51.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:51.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784953+0000", 2023-11-25T04:23:51.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6320000000000001e-06 2023-11-25T04:23:51.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:51.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784993+0000", 2023-11-25T04:23:51.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9737000000000003e-05 2023-11-25T04:23:51.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:51.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785011+0000", 2023-11-25T04:23:51.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8349999999999999e-05 2023-11-25T04:23:51.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:51.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791024+0000", 2023-11-25T04:23:51.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006012997 2023-11-25T04:23:51.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:51.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791055+0000", 2023-11-25T04:23:51.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1254000000000003e-05 2023-11-25T04:23:51.200 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:51.200 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:51.200 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:51.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.200 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:156 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'110, mlcod=11'105)", 2023-11-25T04:23:51.200 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:51.200 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.407014637000003, 2023-11-25T04:23:51.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005616116, 2023-11-25T04:23:51.200 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:51.200 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:51.200 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:51.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:51.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:51.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:51.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:51.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:51.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:51.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:51.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785470+0000", 2023-11-25T04:23:51.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0699999999999998e-07 2023-11-25T04:23:51.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.202 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:51.202 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785474+0000", 2023-11-25T04:23:51.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.048e-06 2023-11-25T04:23:51.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.202 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:51.202 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785480+0000", 2023-11-25T04:23:51.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7729999999999996e-06 2023-11-25T04:23:51.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:51.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785483+0000", 2023-11-25T04:23:51.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1970000000000001e-06 2023-11-25T04:23:51.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:51.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787936+0000", 2023-11-25T04:23:51.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024529040000000001 2023-11-25T04:23:51.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:51.204 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787947+0000", 2023-11-25T04:23:51.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1669e-05 2023-11-25T04:23:51.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.204 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:51.204 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791057+0000", 2023-11-25T04:23:51.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031095200000000002 2023-11-25T04:23:51.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.204 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:51.204 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791086+0000", 2023-11-25T04:23:51.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8697999999999999e-05 2023-11-25T04:23:51.204 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:51.205 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:51.205 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:51.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.205 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:157 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'111, mlcod=11'106)", 2023-11-25T04:23:51.205 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:51.205 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.406422078999995, 2023-11-25T04:23:51.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050590280000000001, 2023-11-25T04:23:51.205 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:51.205 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:51.205 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:51.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.205 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:51.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:51.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:51.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:51.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:51.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:51.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:51.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786063+0000", 2023-11-25T04:23:51.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2600000000000002e-07 2023-11-25T04:23:51.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.207 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:51.207 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786069+0000", 2023-11-25T04:23:51.207 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8839999999999999e-06 2023-11-25T04:23:51.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.207 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:51.207 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786079+0000", 2023-11-25T04:23:51.207 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0124e-05 2023-11-25T04:23:51.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.207 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:51.208 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786085+0000", 2023-11-25T04:23:51.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6660000000000002e-06 2023-11-25T04:23:51.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.208 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:51.208 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788160+0000", 2023-11-25T04:23:51.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020747669999999999 2023-11-25T04:23:51.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.208 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:51.208 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788180+0000", 2023-11-25T04:23:51.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0208999999999999e-05 2023-11-25T04:23:51.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.209 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.209 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:51.209 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791087+0000", 2023-11-25T04:23:51.209 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029064630000000002 2023-11-25T04:23:51.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.209 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.209 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:51.209 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791121+0000", 2023-11-25T04:23:51.209 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4289000000000003e-05 2023-11-25T04:23:51.209 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:51.209 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:51.209 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:51.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.210 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:158 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'112, mlcod=11'106)", 2023-11-25T04:23:51.210 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:51.210 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.406005686, 2023-11-25T04:23:51.210 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046663030000000001, 2023-11-25T04:23:51.210 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:51.210 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:51.210 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:51.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.210 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:51.210 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:51.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:51.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.211 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:51.211 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:51.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:51.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.211 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:51.211 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786479+0000", 2023-11-25T04:23:51.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2799999999999996e-07 2023-11-25T04:23:51.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.212 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:51.212 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786498+0000", 2023-11-25T04:23:51.212 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9069e-05 2023-11-25T04:23:51.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.212 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:51.212 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786508+0000", 2023-11-25T04:23:51.212 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5810000000000003e-06 2023-11-25T04:23:51.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.212 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:51.212 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786512+0000", 2023-11-25T04:23:51.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4889999999999997e-06 2023-11-25T04:23:51.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.213 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:51.213 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788438+0000", 2023-11-25T04:23:51.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019257059999999999 2023-11-25T04:23:51.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.213 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:51.213 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788449+0000", 2023-11-25T04:23:51.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1419e-05 2023-11-25T04:23:51.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.214 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.214 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:51.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791122+0000", 2023-11-25T04:23:51.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002672822 2023-11-25T04:23:51.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.214 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.214 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:51.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791145+0000", 2023-11-25T04:23:51.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2589e-05 2023-11-25T04:23:51.214 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:51.214 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:51.214 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:51.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.215 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:206 1.0 e12/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 12'152, mlcod=12'148)", 2023-11-25T04:23:51.215 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:51.215 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 72.894237215999993, 2023-11-25T04:23:51.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046728189999999999, 2023-11-25T04:23:51.215 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:51.215 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:51.215 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:51.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:51.215 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:51.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:51.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:51.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:51.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:51.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:51.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298248+0000", 2023-11-25T04:23:51.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3600000000000003e-07 2023-11-25T04:23:51.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:51.217 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298278+0000", 2023-11-25T04:23:51.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0474e-05 2023-11-25T04:23:51.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.217 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:51.217 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298288+0000", 2023-11-25T04:23:51.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0156000000000001e-05 2023-11-25T04:23:51.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.217 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:51.217 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298293+0000", 2023-11-25T04:23:51.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7400000000000004e-06 2023-11-25T04:23:51.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.218 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:51.218 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298333+0000", 2023-11-25T04:23:51.218 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9666000000000003e-05 2023-11-25T04:23:51.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.218 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:51.218 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298351+0000", 2023-11-25T04:23:51.218 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8544999999999999e-05 2023-11-25T04:23:51.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.218 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:51.218 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302891+0000", 2023-11-25T04:23:51.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045398519999999996 2023-11-25T04:23:51.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.219 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:51.219 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302920+0000", 2023-11-25T04:23:51.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8750000000000001e-05 2023-11-25T04:23:51.219 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:51.219 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:51.219 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:51.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.219 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:14 3.15 3:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:51.220 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:51.220 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 64.159102859000001, 2023-11-25T04:23:51.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010990437, 2023-11-25T04:23:51.220 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:51.220 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:51.220 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:51.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:51.220 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:51.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:51.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:51.221 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:51.221 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:51.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.221 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.221 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:51.221 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033383+0000", 2023-11-25T04:23:51.221 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9870000000000002e-06 2023-11-25T04:23:51.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.221 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.221 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:51.221 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033398+0000", 2023-11-25T04:23:51.221 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4569999999999999e-05 2023-11-25T04:23:51.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.222 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:51.222 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033411+0000", 2023-11-25T04:23:51.222 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3543999999999999e-05 2023-11-25T04:23:51.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.222 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:51.222 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033423+0000", 2023-11-25T04:23:51.222 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1905999999999999e-05 2023-11-25T04:23:51.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.222 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:51.223 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034433+0000", 2023-11-25T04:23:51.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00101022 2023-11-25T04:23:51.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.223 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.223 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:51.223 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034539+0000", 2023-11-25T04:23:51.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010597199999999999 2023-11-25T04:23:51.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.223 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.223 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:51.223 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034667+0000", 2023-11-25T04:23:51.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012765500000000001 2023-11-25T04:23:51.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:51.224 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037803+0000", 2023-11-25T04:23:51.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031362220000000001 2023-11-25T04:23:51.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:51.224 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:23:51.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0064964669999999997 2023-11-25T04:23:51.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:51.224 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044331+0000", 2023-11-25T04:23:51.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1544999999999997e-05 2023-11-25T04:23:51.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.225 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:51.225 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044372+0000", 2023-11-25T04:23:51.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0349000000000003e-05 2023-11-25T04:23:51.225 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:51.225 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:51.225 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:51.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.225 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:15 3.15 3:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:51.226 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:51.226 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 64.158443208999998, 2023-11-25T04:23:51.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010464362, 2023-11-25T04:23:51.226 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:51.226 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:51.226 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:51.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.226 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:51.226 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:51.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:51.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:51.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:51.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:51.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:51.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034042+0000", 2023-11-25T04:23:51.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0999999999999997e-07 2023-11-25T04:23:51.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:51.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034048+0000", 2023-11-25T04:23:51.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0390000000000001e-06 2023-11-25T04:23:51.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.228 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:51.228 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034059+0000", 2023-11-25T04:23:51.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1182999999999999e-05 2023-11-25T04:23:51.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.228 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:51.228 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034067+0000", 2023-11-25T04:23:51.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1559999999999995e-06 2023-11-25T04:23:51.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.229 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:51.229 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035005+0000", 2023-11-25T04:23:51.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00093747399999999999 2023-11-25T04:23:51.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.229 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:51.229 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035077+0000", 2023-11-25T04:23:51.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2747e-05 2023-11-25T04:23:51.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.229 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:51.229 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035219+0000", 2023-11-25T04:23:51.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000141877 2023-11-25T04:23:51.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:51.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037810+0000", 2023-11-25T04:23:51.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002590752 2023-11-25T04:23:51.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:51.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044437+0000", 2023-11-25T04:23:51.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066270110000000004 2023-11-25T04:23:51.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:51.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044481+0000", 2023-11-25T04:23:51.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4106000000000001e-05 2023-11-25T04:23:51.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:51.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044505+0000", 2023-11-25T04:23:51.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4207e-05 2023-11-25T04:23:51.231 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:51.231 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:51.231 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:51.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.232 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:16 3.1d 3:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:51.232 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:51.232 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 64.158401361000003, 2023-11-25T04:23:51.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010413923, 2023-11-25T04:23:51.232 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:51.232 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:51.232 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:51.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:51.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:51.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:51.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.233 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:51.233 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:51.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:51.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.233 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:51.233 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:51.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4499999999999997e-07 2023-11-25T04:23:51.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.233 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:51.233 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034087+0000", 2023-11-25T04:23:51.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9770000000000002e-06 2023-11-25T04:23:51.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:51.234 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034094+0000", 2023-11-25T04:23:51.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8809999999999998e-06 2023-11-25T04:23:51.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:51.234 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034100+0000", 2023-11-25T04:23:51.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8440000000000003e-06 2023-11-25T04:23:51.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.235 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:51.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034993+0000", 2023-11-25T04:23:51.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00089336799999999996 2023-11-25T04:23:51.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.235 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:51.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035099+0000", 2023-11-25T04:23:51.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106037 2023-11-25T04:23:51.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:51.236 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035276+0000", 2023-11-25T04:23:51.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017631799999999999 2023-11-25T04:23:51.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:51.236 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037815+0000", 2023-11-25T04:23:51.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025395959999999999 2023-11-25T04:23:51.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:51.236 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044420+0000", 2023-11-25T04:23:51.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066045749999999997 2023-11-25T04:23:51.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:51.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044463+0000", 2023-11-25T04:23:51.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3205999999999999e-05 2023-11-25T04:23:51.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:51.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044497+0000", 2023-11-25T04:23:51.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3476000000000003e-05 2023-11-25T04:23:51.237 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:51.237 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:51.238 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:51.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.238 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:17 3.1d 3:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:51.238 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:51.238 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 64.158376152000002, 2023-11-25T04:23:51.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01127356, 2023-11-25T04:23:51.238 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:51.238 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:51.238 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:51.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:51.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:51.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:51.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.239 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:51.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:51.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:51.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.239 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:51.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:51.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1100000000000001e-07 2023-11-25T04:23:51.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:51.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034112+0000", 2023-11-25T04:23:51.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8600000000000003e-06 2023-11-25T04:23:51.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:51.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034120+0000", 2023-11-25T04:23:51.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7700000000000001e-06 2023-11-25T04:23:51.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:51.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034127+0000", 2023-11-25T04:23:51.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0999999999999998e-06 2023-11-25T04:23:51.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:51.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035601+0000", 2023-11-25T04:23:51.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014739410000000001 2023-11-25T04:23:51.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:51.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035685+0000", 2023-11-25T04:23:51.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3677999999999994e-05 2023-11-25T04:23:51.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:51.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035827+0000", 2023-11-25T04:23:51.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014232599999999999 2023-11-25T04:23:51.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:51.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038725+0000", 2023-11-25T04:23:51.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028982779999999998 2023-11-25T04:23:51.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:51.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045323+0000", 2023-11-25T04:23:51.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0065980780000000003 2023-11-25T04:23:51.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.244 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:51.244 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045354+0000", 2023-11-25T04:23:51.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0463e-05 2023-11-25T04:23:51.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.244 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:51.244 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045381+0000", 2023-11-25T04:23:51.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7654999999999999e-05 2023-11-25T04:23:51.244 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:51.245 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:51.245 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:51.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.245 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:22 3.e 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:51.245 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:51.245 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 64.158349739000002, 2023-11-25T04:23:51.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010133422, 2023-11-25T04:23:51.245 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:51.245 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:51.245 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:51.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.245 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:51.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:51.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:51.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:51.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:51.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:51.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:51.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034135+0000", 2023-11-25T04:23:51.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-07 2023-11-25T04:23:51.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:51.247 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034138+0000", 2023-11-25T04:23:51.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6009999999999999e-06 2023-11-25T04:23:51.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:51.247 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034145+0000", 2023-11-25T04:23:51.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1169999999999998e-06 2023-11-25T04:23:51.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:51.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034150+0000", 2023-11-25T04:23:51.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9080000000000003e-06 2023-11-25T04:23:51.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:51.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034209+0000", 2023-11-25T04:23:51.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8149000000000002e-05 2023-11-25T04:23:51.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:51.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034415+0000", 2023-11-25T04:23:51.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020637699999999999 2023-11-25T04:23:51.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:51.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034584+0000", 2023-11-25T04:23:51.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016870399999999999 2023-11-25T04:23:51.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:51.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.036979+0000", 2023-11-25T04:23:51.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023957509999999998 2023-11-25T04:23:51.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:51.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044161+0000", 2023-11-25T04:23:51.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0071811590000000003 2023-11-25T04:23:51.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.250 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:51.250 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044224+0000", 2023-11-25T04:23:51.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3454000000000006e-05 2023-11-25T04:23:51.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.250 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:51.250 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044268+0000", 2023-11-25T04:23:51.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3761999999999999e-05 2023-11-25T04:23:51.250 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:51.251 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:51.251 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:51.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.251 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:23 3.e 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:51.251 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:51.251 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 64.15831833, 2023-11-25T04:23:51.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010281676, 2023-11-25T04:23:51.251 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:51.251 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:51.251 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:51.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:51.252 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:51.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:51.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:51.252 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:51.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:51.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:51.252 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:51.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5799999999999999e-07 2023-11-25T04:23:51.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:51.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034169+0000", 2023-11-25T04:23:51.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1879999999999998e-06 2023-11-25T04:23:51.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:51.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034177+0000", 2023-11-25T04:23:51.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7020000000000001e-06 2023-11-25T04:23:51.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:51.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034184+0000", 2023-11-25T04:23:51.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3910000000000003e-06 2023-11-25T04:23:51.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:51.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034924+0000", 2023-11-25T04:23:51.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00074032999999999998 2023-11-25T04:23:51.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:51.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034991+0000", 2023-11-25T04:23:51.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7347000000000004e-05 2023-11-25T04:23:51.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:51.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035096+0000", 2023-11-25T04:23:51.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010488 2023-11-25T04:23:51.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:51.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037588+0000", 2023-11-25T04:23:51.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024918309999999999 2023-11-25T04:23:51.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:51.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044367+0000", 2023-11-25T04:23:51.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067787589999999997 2023-11-25T04:23:51.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:51.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044415+0000", 2023-11-25T04:23:51.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7803000000000003e-05 2023-11-25T04:23:51.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:51.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044447+0000", 2023-11-25T04:23:51.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2886999999999999e-05 2023-11-25T04:23:51.256 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:51.256 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:51.257 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:51.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.257 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:30 3.15 3:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:51.257 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:51.257 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 64.158259889999997, 2023-11-25T04:23:51.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010365070000000001, 2023-11-25T04:23:51.257 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:51.257 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:51.257 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:51.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:51.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:51.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:51.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:51.258 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:51.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:51.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:51.258 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034225+0000", 2023-11-25T04:23:51.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7400000000000002e-07 2023-11-25T04:23:51.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:51.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034229+0000", 2023-11-25T04:23:51.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3100000000000002e-06 2023-11-25T04:23:51.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:51.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034239+0000", 2023-11-25T04:23:51.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9140000000000003e-06 2023-11-25T04:23:51.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:51.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034246+0000", 2023-11-25T04:23:51.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0149999999999998e-06 2023-11-25T04:23:51.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:51.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035572+0000", 2023-11-25T04:23:51.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001325655 2023-11-25T04:23:51.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:51.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035730+0000", 2023-11-25T04:23:51.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015827899999999999 2023-11-25T04:23:51.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:51.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035857+0000", 2023-11-25T04:23:51.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012644699999999999 2023-11-25T04:23:51.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:51.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038737+0000", 2023-11-25T04:23:51.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002880077 2023-11-25T04:23:51.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:51.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044540+0000", 2023-11-25T04:23:51.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058034239999999997 2023-11-25T04:23:51.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:51.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044569+0000", 2023-11-25T04:23:51.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8388999999999999e-05 2023-11-25T04:23:51.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:51.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044589+0000", 2023-11-25T04:23:51.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0686000000000002e-05 2023-11-25T04:23:51.262 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:51.263 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:51.263 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:51.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.263 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:33 3.1b 3:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:51.263 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:51.263 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 64.158214392000005, 2023-11-25T04:23:51.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099754049999999997, 2023-11-25T04:23:51.263 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:51.263 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:23:51.263 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:51.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:51.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:51.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:51.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:51.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:51.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:51.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:51.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:51.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4799999999999998e-07 2023-11-25T04:23:51.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:51.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034273+0000", 2023-11-25T04:23:51.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7819999999999998e-06 2023-11-25T04:23:51.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:51.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034285+0000", 2023-11-25T04:23:51.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2058e-05 2023-11-25T04:23:51.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:51.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034291+0000", 2023-11-25T04:23:51.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0680000000000002e-06 2023-11-25T04:23:51.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:51.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034370+0000", 2023-11-25T04:23:51.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8452e-05 2023-11-25T04:23:51.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:51.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034477+0000", 2023-11-25T04:23:51.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107471 2023-11-25T04:23:51.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.267 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:51.267 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044245+0000", 2023-11-25T04:23:51.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0097679259999999997 2023-11-25T04:23:51.267 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:51.267 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:51.267 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:51.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.267 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:36 3.1a 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:51.267 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:51.267 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 64.158167719000005, 2023-11-25T04:23:51.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058179099999999999, 2023-11-25T04:23:51.268 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:51.268 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:23:51.268 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:51.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:51.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:51.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:51.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:51.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:51.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:51.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:51.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034317+0000", 2023-11-25T04:23:51.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1999999999999999e-07 2023-11-25T04:23:51.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:51.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034321+0000", 2023-11-25T04:23:51.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.233e-06 2023-11-25T04:23:51.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:51.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034337+0000", 2023-11-25T04:23:51.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5308999999999999e-05 2023-11-25T04:23:51.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:51.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034354+0000", 2023-11-25T04:23:51.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7411999999999999e-05 2023-11-25T04:23:51.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:51.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034465+0000", 2023-11-25T04:23:51.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011154699999999999 2023-11-25T04:23:51.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:51.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034560+0000", 2023-11-25T04:23:51.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4186999999999998e-05 2023-11-25T04:23:51.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:51.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.040134+0000", 2023-11-25T04:23:51.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055746019999999997 2023-11-25T04:23:51.271 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:51.271 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:51.271 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:51.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.272 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:9 3.10 e18/16 3:09401a81:::601.00000000:head v 18'4, mlcod=0'0)", 2023-11-25T04:23:51.272 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:51.272 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 64.149273410999996, 2023-11-25T04:23:51.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061641020000000003, 2023-11-25T04:23:51.272 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:51.272 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:51.272 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:51.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:51.272 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:51.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:51.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:51.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:51.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:51.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:51.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:51.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5300000000000001e-07 2023-11-25T04:23:51.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:51.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043222+0000", 2023-11-25T04:23:51.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.082e-05 2023-11-25T04:23:51.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:51.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043228+0000", 2023-11-25T04:23:51.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6699999999999997e-06 2023-11-25T04:23:51.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:51.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043236+0000", 2023-11-25T04:23:51.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8189999999999992e-06 2023-11-25T04:23:51.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:51.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043507+0000", 2023-11-25T04:23:51.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000271012 2023-11-25T04:23:51.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:51.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043530+0000", 2023-11-25T04:23:51.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2659000000000001e-05 2023-11-25T04:23:51.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:51.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049321+0000", 2023-11-25T04:23:51.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057915129999999999 2023-11-25T04:23:51.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:51.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049375+0000", 2023-11-25T04:23:51.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3455999999999998e-05 2023-11-25T04:23:51.276 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:51.276 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:51.276 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:51.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.276 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:1 3.14 e18/16 3:292cf221:::200.00000000:head v 18'3, mlcod=0'0)", 2023-11-25T04:23:51.276 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:51.276 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 64.149236697999996, 2023-11-25T04:23:51.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0060983599999999997, 2023-11-25T04:23:51.277 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:51.277 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:51.277 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:51.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:51.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:51.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:51.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:51.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:51.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:51.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:51.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043248+0000", 2023-11-25T04:23:51.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9399999999999995e-07 2023-11-25T04:23:51.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:51.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043252+0000", 2023-11-25T04:23:51.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1799999999999998e-06 2023-11-25T04:23:51.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:51.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043259+0000", 2023-11-25T04:23:51.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0049999999999998e-06 2023-11-25T04:23:51.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:51.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043265+0000", 2023-11-25T04:23:51.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0859999999999998e-06 2023-11-25T04:23:51.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:51.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043311+0000", 2023-11-25T04:23:51.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6082999999999999e-05 2023-11-25T04:23:51.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:51.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043330+0000", 2023-11-25T04:23:51.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8556e-05 2023-11-25T04:23:51.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:51.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049303+0000", 2023-11-25T04:23:51.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059733470000000004 2023-11-25T04:23:51.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:51.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049346+0000", 2023-11-25T04:23:51.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2608999999999998e-05 2023-11-25T04:23:51.281 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:51.281 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:51.281 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:51.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.281 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:3 3.14 e18/16 3:287034c4:::1.00000000.inode:head v 18'4, mlcod=0'0)", 2023-11-25T04:23:51.281 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:51.281 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 64.148720191999999, 2023-11-25T04:23:51.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057040090000000003, 2023-11-25T04:23:51.282 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:51.282 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:51.282 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:51.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:51.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:51.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:51.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:51.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:51.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:51.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:51.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:51.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9799999999999999e-07 2023-11-25T04:23:51.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:51.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043785+0000", 2023-11-25T04:23:51.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0721999999999999e-05 2023-11-25T04:23:51.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:51.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043793+0000", 2023-11-25T04:23:51.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5839999999999995e-06 2023-11-25T04:23:51.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:51.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043803+0000", 2023-11-25T04:23:51.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0729999999999994e-06 2023-11-25T04:23:51.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:51.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043843+0000", 2023-11-25T04:23:51.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0747999999999997e-05 2023-11-25T04:23:51.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:51.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043856+0000", 2023-11-25T04:23:51.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2697000000000001e-05 2023-11-25T04:23:51.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:51.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049426+0000", 2023-11-25T04:23:51.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055704409999999998 2023-11-25T04:23:51.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:51.286 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049468+0000", 2023-11-25T04:23:51.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1446000000000003e-05 2023-11-25T04:23:51.286 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:51.286 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:51.286 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:51.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.286 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "pg_update_log_missing(3.14 epoch 18/16 rep_tid 171 entries 18'5 (0'0) error 3:2e2fa760:::200.00000001:head by mds.0.3:32 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:23:51.286 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:51.286 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 64.148559229, 2023-11-25T04:23:51.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005556938, 2023-11-25T04:23:51.286 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:51.287 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2023-11-25T04:23:51.287 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:51.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:51.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:51.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:51.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:51.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:51.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:51.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:51.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:51.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.03e-07 2023-11-25T04:23:51.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:51.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043931+0000", 2023-11-25T04:23:51.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1079999999999998e-06 2023-11-25T04:23:51.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:51.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043944+0000", 2023-11-25T04:23:51.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2852e-05 2023-11-25T04:23:51.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:51.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043955+0000", 2023-11-25T04:23:51.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1037e-05 2023-11-25T04:23:51.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:51.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044088+0000", 2023-11-25T04:23:51.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013269599999999999 2023-11-25T04:23:51.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.290 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:51.290 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049482+0000", 2023-11-25T04:23:51.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053940419999999999 2023-11-25T04:23:51.290 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:51.290 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:51.290 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:51.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.290 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:10 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'5, mlcod=0'0)", 2023-11-25T04:23:51.290 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:51.290 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 64.148469200999997, 2023-11-25T04:23:51.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054493099999999997, 2023-11-25T04:23:51.291 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:51.291 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:51.291 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:51.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.291 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:51.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:51.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:51.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.291 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:51.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:51.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:51.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:51.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:51.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6100000000000002e-07 2023-11-25T04:23:51.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:51.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044019+0000", 2023-11-25T04:23:51.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.839e-06 2023-11-25T04:23:51.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:51.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044026+0000", 2023-11-25T04:23:51.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5869999999999998e-06 2023-11-25T04:23:51.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.293 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:51.293 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044031+0000", 2023-11-25T04:23:51.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4909999999999996e-06 2023-11-25T04:23:51.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.293 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:51.293 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044070+0000", 2023-11-25T04:23:51.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8829999999999999e-05 2023-11-25T04:23:51.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:51.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044081+0000", 2023-11-25T04:23:51.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1447999999999999e-05 2023-11-25T04:23:51.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:51.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049406+0000", 2023-11-25T04:23:51.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053246480000000004 2023-11-25T04:23:51.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:51.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049464+0000", 2023-11-25T04:23:51.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8205999999999998e-05 2023-11-25T04:23:51.295 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:51.295 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:51.295 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:51.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.295 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:11 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'6, mlcod=0'0)", 2023-11-25T04:23:51.295 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:51.295 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 64.148267937, 2023-11-25T04:23:51.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052673499999999996, 2023-11-25T04:23:51.295 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:51.295 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:51.295 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:51.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.296 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:51.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:51.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:51.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.296 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:51.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:51.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:51.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.296 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:51.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:51.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0199999999999998e-07 2023-11-25T04:23:51.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:51.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044220+0000", 2023-11-25T04:23:51.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7720000000000001e-06 2023-11-25T04:23:51.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:51.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044226+0000", 2023-11-25T04:23:51.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8579999999999996e-06 2023-11-25T04:23:51.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.298 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:51.298 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044232+0000", 2023-11-25T04:23:51.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1700000000000002e-06 2023-11-25T04:23:51.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.298 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:51.298 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044274+0000", 2023-11-25T04:23:51.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1254999999999997e-05 2023-11-25T04:23:51.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.298 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:51.298 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:23:51.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6880999999999999e-05 2023-11-25T04:23:51.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:51.299 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049466+0000", 2023-11-25T04:23:51.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051659219999999999 2023-11-25T04:23:51.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:51.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:51.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:51.299 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049483+0000", 2023-11-25T04:23:51.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.719e-05 2023-11-25T04:23:51.299 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:51.299 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:51.300 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:51.300 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:51.300 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:51.300 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:51.301 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:23:51.302 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:51.300+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:51.402 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:51.401+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:51.416 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:51.416 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:23:51.416 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2023-11-25T04:23:51.416 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:51.425 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:23:51.503 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:51.501+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:51.532 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:51.532 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:23:51.532 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2023-11-25T04:23:51.532 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2023-11-25T04:23:51.533 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:51.541 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:23:51.625 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:51.624+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:51.688 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:51.689 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2023-11-25T04:23:51.689 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2023-11-25T04:23:51.689 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2023-11-25T04:23:51.689 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.689 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:51.689 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:51.689 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 64.654629564000004, 2023-11-25T04:23:51.689 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01072943, 2023-11-25T04:23:51.689 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:51.689 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:51.689 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:51.690 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.690 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:51.690 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:51.690 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:51.690 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.690 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.690 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:51.690 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:51.690 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:51.690 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.690 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.690 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:51.690 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031829+0000", 2023-11-25T04:23:51.691 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.666e-06 2023-11-25T04:23:51.691 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.691 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.691 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:51.691 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031839+0000", 2023-11-25T04:23:51.691 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9490000000000008e-06 2023-11-25T04:23:51.691 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.691 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.691 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:51.691 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031857+0000", 2023-11-25T04:23:51.691 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8093000000000001e-05 2023-11-25T04:23:51.691 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.691 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.692 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:51.692 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031882+0000", 2023-11-25T04:23:51.692 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4294000000000002e-05 2023-11-25T04:23:51.692 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.692 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.692 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:51.692 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033130+0000", 2023-11-25T04:23:51.692 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001248264 2023-11-25T04:23:51.692 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.692 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.692 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:51.692 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033426+0000", 2023-11-25T04:23:51.692 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029620999999999998 2023-11-25T04:23:51.693 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.693 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.693 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:51.693 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033814+0000", 2023-11-25T04:23:51.693 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000387742 2023-11-25T04:23:51.693 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.693 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.693 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:51.693 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036793+0000", 2023-11-25T04:23:51.693 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002979598 2023-11-25T04:23:51.693 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.693 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.693 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:51.693 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042455+0000", 2023-11-25T04:23:51.694 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0056619260000000003 2023-11-25T04:23:51.694 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.694 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.694 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:51.694 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042502+0000", 2023-11-25T04:23:51.694 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7139999999999999e-05 2023-11-25T04:23:51.694 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.694 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.694 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:51.694 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042557+0000", 2023-11-25T04:23:51.694 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4548e-05 2023-11-25T04:23:51.694 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:51.694 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:51.695 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:51.695 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.695 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.695 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:51.695 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:51.695 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 64.653431029000004, 2023-11-25T04:23:51.695 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011505202000000001, 2023-11-25T04:23:51.695 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:51.695 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:51.695 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:51.695 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.696 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:51.696 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:51.696 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:51.696 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.696 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.697 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:51.697 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:51.697 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:51.697 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.697 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.697 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:51.697 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033027+0000", 2023-11-25T04:23:51.697 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0699999999999999e-06 2023-11-25T04:23:51.697 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.697 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.698 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:51.698 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033035+0000", 2023-11-25T04:23:51.698 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0830000000000004e-06 2023-11-25T04:23:51.698 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.698 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.698 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:51.698 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033059+0000", 2023-11-25T04:23:51.698 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3975e-05 2023-11-25T04:23:51.698 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.698 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.698 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:51.698 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033482+0000", 2023-11-25T04:23:51.699 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042253700000000001 2023-11-25T04:23:51.699 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.700 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.700 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:51.700 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034520+0000", 2023-11-25T04:23:51.700 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001037991 2023-11-25T04:23:51.700 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.700 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.700 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:51.700 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034611+0000", 2023-11-25T04:23:51.700 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1113999999999995e-05 2023-11-25T04:23:51.700 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.701 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.701 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:51.701 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034769+0000", 2023-11-25T04:23:51.701 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000157641 2023-11-25T04:23:51.701 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.701 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.701 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:51.701 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038292+0000", 2023-11-25T04:23:51.701 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035231160000000002 2023-11-25T04:23:51.701 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.701 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.701 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:51.701 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044440+0000", 2023-11-25T04:23:51.702 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0061486980000000002 2023-11-25T04:23:51.702 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.702 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.702 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:51.702 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044495+0000", 2023-11-25T04:23:51.702 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4339000000000003e-05 2023-11-25T04:23:51.702 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.702 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.702 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:51.702 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044531+0000", 2023-11-25T04:23:51.702 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6637999999999997e-05 2023-11-25T04:23:51.702 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:51.703 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:51.703 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:51.703 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.703 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.703 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:51.703 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:51.703 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 64.652956674999999, 2023-11-25T04:23:51.703 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0057634039999999997, 2023-11-25T04:23:51.703 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:51.703 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:51.703 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:51.703 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.703 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:51.704 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:51.704 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:51.704 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.704 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.704 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:51.704 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:51.704 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:51.704 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.704 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.704 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:51.704 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:51.704 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7900000000000007e-07 2023-11-25T04:23:51.704 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.705 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.705 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:51.705 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033507+0000", 2023-11-25T04:23:51.705 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8900000000000004e-06 2023-11-25T04:23:51.705 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.705 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.705 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:51.705 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033519+0000", 2023-11-25T04:23:51.705 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1756000000000001e-05 2023-11-25T04:23:51.705 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.705 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.705 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:51.706 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033527+0000", 2023-11-25T04:23:51.706 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3610000000000001e-06 2023-11-25T04:23:51.706 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.706 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.706 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:51.706 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034451+0000", 2023-11-25T04:23:51.706 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00092372400000000001 2023-11-25T04:23:51.706 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.706 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.707 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:51.707 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034809+0000", 2023-11-25T04:23:51.707 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00035757699999999998 2023-11-25T04:23:51.707 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.707 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.707 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:23:51.707 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034965+0000", 2023-11-25T04:23:51.709 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015638699999999999 2023-11-25T04:23:51.709 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.709 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.709 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:51.709 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038299+0000", 2023-11-25T04:23:51.709 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0033336559999999999 2023-11-25T04:23:51.709 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.709 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.710 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:51.710 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039169+0000", 2023-11-25T04:23:51.710 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00087066400000000005 2023-11-25T04:23:51.710 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.710 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.710 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:51.710 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039226+0000", 2023-11-25T04:23:51.710 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6511999999999997e-05 2023-11-25T04:23:51.710 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.710 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.710 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:51.711 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039264+0000", 2023-11-25T04:23:51.711 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7898000000000002e-05 2023-11-25T04:23:51.711 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:51.711 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:51.711 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:51.711 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.711 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.711 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:25 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:51.711 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:51.711 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 64.652878091000005, 2023-11-25T04:23:51.712 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096656169999999996, 2023-11-25T04:23:51.712 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:51.712 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:51.712 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:51.712 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.712 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:51.712 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:51.712 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:51.712 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.712 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.712 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:51.712 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:51.712 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:51.713 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.713 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.713 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:51.713 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033580+0000", 2023-11-25T04:23:51.713 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8199999999999999e-07 2023-11-25T04:23:51.713 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.713 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.713 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:51.713 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033585+0000", 2023-11-25T04:23:51.713 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3680000000000001e-06 2023-11-25T04:23:51.713 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.713 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.713 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:51.714 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033600+0000", 2023-11-25T04:23:51.714 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4331000000000001e-05 2023-11-25T04:23:51.714 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.714 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.714 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:51.714 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033606+0000", 2023-11-25T04:23:51.714 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5780000000000004e-06 2023-11-25T04:23:51.714 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.714 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.714 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:51.714 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035747+0000", 2023-11-25T04:23:51.714 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0021406530000000002 2023-11-25T04:23:51.715 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.715 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.715 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:51.715 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035901+0000", 2023-11-25T04:23:51.715 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000153931 2023-11-25T04:23:51.715 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.715 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.715 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:23:51.715 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036193+0000", 2023-11-25T04:23:51.715 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029211299999999999 2023-11-25T04:23:51.715 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.715 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.716 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:51.716 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038655+0000", 2023-11-25T04:23:51.716 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0024621640000000002 2023-11-25T04:23:51.717 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.717 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.717 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:51.717 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043130+0000", 2023-11-25T04:23:51.718 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.004474698 2023-11-25T04:23:51.718 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.718 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.718 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:51.718 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043204+0000", 2023-11-25T04:23:51.718 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4091999999999995e-05 2023-11-25T04:23:51.718 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.718 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.718 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:51.718 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043245+0000", 2023-11-25T04:23:51.718 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0907e-05 2023-11-25T04:23:51.718 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:51.718 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:51.719 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:51.719 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.719 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.719 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:26 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:51.719 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:51.719 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 64.652842266999997, 2023-11-25T04:23:51.719 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011086945000000001, 2023-11-25T04:23:51.719 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:51.719 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:51.719 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:51.719 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.719 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:51.719 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:51.720 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:51.720 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.720 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.720 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:51.720 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:51.720 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:51.720 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.720 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.720 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:51.720 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:51.720 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1699999999999998e-07 2023-11-25T04:23:51.720 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.720 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.721 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:51.721 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033627+0000", 2023-11-25T04:23:51.721 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.163e-05 2023-11-25T04:23:51.721 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.721 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.721 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:51.721 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033636+0000", 2023-11-25T04:23:51.721 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6859999999999995e-06 2023-11-25T04:23:51.721 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.721 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.721 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:51.721 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033642+0000", 2023-11-25T04:23:51.721 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3149999999999997e-06 2023-11-25T04:23:51.722 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.722 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.722 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:51.722 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035414+0000", 2023-11-25T04:23:51.722 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0017716380000000001 2023-11-25T04:23:51.722 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.722 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.722 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:51.722 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035539+0000", 2023-11-25T04:23:51.722 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000125477 2023-11-25T04:23:51.722 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.722 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.723 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:51.723 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035672+0000", 2023-11-25T04:23:51.723 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000133316 2023-11-25T04:23:51.723 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.723 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.723 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:51.723 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038649+0000", 2023-11-25T04:23:51.723 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029764679999999999 2023-11-25T04:23:51.723 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.723 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.724 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:51.724 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044578+0000", 2023-11-25T04:23:51.724 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0059293510000000002 2023-11-25T04:23:51.724 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.724 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.724 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:51.724 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044651+0000", 2023-11-25T04:23:51.724 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2513000000000002e-05 2023-11-25T04:23:51.724 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.724 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.724 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:51.724 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044702+0000", 2023-11-25T04:23:51.724 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1034e-05 2023-11-25T04:23:51.725 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:51.725 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:51.725 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:51.725 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.725 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.725 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:29 3.b 3:d0630e4c:::mds0_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:51.725 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:51.725 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 64.652804347, 2023-11-25T04:23:51.725 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010980227, 2023-11-25T04:23:51.725 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:51.725 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:51.726 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:51.726 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.726 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:51.726 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:51.726 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:51.726 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.726 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.726 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:51.726 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:51.727 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:51.727 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.727 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.727 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:51.727 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:51.727 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3499999999999996e-07 2023-11-25T04:23:51.727 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.727 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.727 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:51.727 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033658+0000", 2023-11-25T04:23:51.727 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1280000000000001e-06 2023-11-25T04:23:51.727 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.727 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.728 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:51.728 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033667+0000", 2023-11-25T04:23:51.728 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1430000000000004e-06 2023-11-25T04:23:51.728 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.728 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.728 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:51.728 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033678+0000", 2023-11-25T04:23:51.728 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0906e-05 2023-11-25T04:23:51.728 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.728 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.728 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:51.728 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034456+0000", 2023-11-25T04:23:51.728 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00077833899999999996 2023-11-25T04:23:51.729 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.729 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.729 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:51.729 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034575+0000", 2023-11-25T04:23:51.729 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011926000000000001 2023-11-25T04:23:51.729 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.729 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.729 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:51.729 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035001+0000", 2023-11-25T04:23:51.729 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042550999999999998 2023-11-25T04:23:51.729 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.729 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.729 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:51.730 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.037923+0000", 2023-11-25T04:23:51.730 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029219670000000001 2023-11-25T04:23:51.730 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.730 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.730 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:51.730 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044538+0000", 2023-11-25T04:23:51.730 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0066152349999999997 2023-11-25T04:23:51.730 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.730 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.730 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:51.730 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044594+0000", 2023-11-25T04:23:51.730 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5593000000000001e-05 2023-11-25T04:23:51.730 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.731 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.731 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:51.731 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044633+0000", 2023-11-25T04:23:51.731 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9511000000000002e-05 2023-11-25T04:23:51.731 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:51.731 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:51.731 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:51.731 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.731 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.731 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:34 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:51.731 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:51.731 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 64.652554233999993, 2023-11-25T04:23:51.731 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010920445000000001, 2023-11-25T04:23:51.732 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:51.732 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:51.732 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:51.732 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.732 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:51.732 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:51.732 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:51.732 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.732 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.732 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:51.732 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:51.732 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:51.732 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.733 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.733 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:51.733 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033904+0000", 2023-11-25T04:23:51.733 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6599999999999994e-07 2023-11-25T04:23:51.733 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.733 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.733 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:51.733 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033909+0000", 2023-11-25T04:23:51.733 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5639999999999996e-06 2023-11-25T04:23:51.733 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.733 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.733 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:51.734 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033923+0000", 2023-11-25T04:23:51.734 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3028e-05 2023-11-25T04:23:51.734 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.734 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.734 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:51.734 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034043+0000", 2023-11-25T04:23:51.734 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000120328 2023-11-25T04:23:51.734 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.734 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.734 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:51.734 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035576+0000", 2023-11-25T04:23:51.734 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015333720000000001 2023-11-25T04:23:51.735 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.735 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.735 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:51.735 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035699+0000", 2023-11-25T04:23:51.735 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000123001 2023-11-25T04:23:51.735 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.735 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.735 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:51.735 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044823+0000", 2023-11-25T04:23:51.735 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0091241859999999994 2023-11-25T04:23:51.735 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:51.736 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:51.736 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:51.736 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.736 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.736 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:35 3.1f 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:51.736 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:51.736 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 64.651250028999996, 2023-11-25T04:23:51.736 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096164779999999995, 2023-11-25T04:23:51.736 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:51.736 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:51.736 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:51.736 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.737 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:51.737 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:51.737 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:51.737 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.737 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.737 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:51.737 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:51.737 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:51.737 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.737 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.737 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:51.737 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035208+0000", 2023-11-25T04:23:51.737 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2699999999999998e-07 2023-11-25T04:23:51.738 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.738 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.738 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:51.738 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035213+0000", 2023-11-25T04:23:51.738 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.6070000000000001e-06 2023-11-25T04:23:51.738 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.738 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.738 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:51.738 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035350+0000", 2023-11-25T04:23:51.738 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013693599999999999 2023-11-25T04:23:51.738 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.738 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.739 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:51.739 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035358+0000", 2023-11-25T04:23:51.739 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0099999999999995e-06 2023-11-25T04:23:51.739 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.739 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.739 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:51.739 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036260+0000", 2023-11-25T04:23:51.739 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00090240100000000005 2023-11-25T04:23:51.739 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.739 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.739 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:51.739 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036517+0000", 2023-11-25T04:23:51.739 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00025736299999999998 2023-11-25T04:23:51.740 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.740 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.740 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:51.740 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044824+0000", 2023-11-25T04:23:51.740 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0083063340000000003 2023-11-25T04:23:51.740 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:51.740 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:51.740 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:51.740 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.740 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.740 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:37 3.1e 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:51.740 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:51.740 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 64.651083342999996, 2023-11-25T04:23:51.741 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0078322640000000002, 2023-11-25T04:23:51.741 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:51.741 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:51.741 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:51.741 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.741 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:51.741 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:51.741 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:51.741 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.741 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.741 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:51.741 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:51.741 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:51.742 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.742 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.742 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:51.742 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035375+0000", 2023-11-25T04:23:51.742 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9100000000000003e-07 2023-11-25T04:23:51.742 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.742 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.742 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:51.742 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035379+0000", 2023-11-25T04:23:51.742 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9430000000000002e-06 2023-11-25T04:23:51.742 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.742 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.743 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:51.743 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035388+0000", 2023-11-25T04:23:51.743 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2760000000000006e-06 2023-11-25T04:23:51.743 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.743 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.743 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:51.743 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035396+0000", 2023-11-25T04:23:51.743 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8480000000000001e-06 2023-11-25T04:23:51.743 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.743 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.743 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:51.743 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035452+0000", 2023-11-25T04:23:51.743 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5831000000000001e-05 2023-11-25T04:23:51.743 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.744 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.744 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:51.744 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035934+0000", 2023-11-25T04:23:51.744 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00048265599999999998 2023-11-25T04:23:51.744 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.744 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.744 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:51.744 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043206+0000", 2023-11-25T04:23:51.744 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.007271919 2023-11-25T04:23:51.744 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:51.744 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:51.744 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:51.744 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.745 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.745 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing(3.1a epoch 18/16 rep_tid 3 entries 18'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:36 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:23:51.745 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:51.745 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 64.649807757000005, 2023-11-25T04:23:51.745 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002073414, 2023-11-25T04:23:51.745 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:51.745 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:51.745 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:51.745 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.745 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:51.745 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:51.745 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:51.746 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.746 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.746 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:51.746 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:51.746 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:51.746 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.746 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.746 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:51.746 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036650+0000", 2023-11-25T04:23:51.746 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7300000000000002e-07 2023-11-25T04:23:51.750 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.750 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.750 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:51.750 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036659+0000", 2023-11-25T04:23:51.750 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.7499999999999992e-06 2023-11-25T04:23:51.750 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.751 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.751 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:51.751 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036673+0000", 2023-11-25T04:23:51.751 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4917e-05 2023-11-25T04:23:51.751 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.751 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.751 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:51.751 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036687+0000", 2023-11-25T04:23:51.751 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3883000000000001e-05 2023-11-25T04:23:51.751 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.751 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.751 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:51.751 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036781+0000", 2023-11-25T04:23:51.752 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3219000000000003e-05 2023-11-25T04:23:51.752 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.752 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.752 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:51.752 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038723+0000", 2023-11-25T04:23:51.752 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019422720000000001 2023-11-25T04:23:51.752 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:51.752 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:51.752 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:51.752 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.752 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.752 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:28 3.13 e18/16 3:cd5a64a3:::100.00000000:head v 18'1, mlcod=0'0)", 2023-11-25T04:23:51.752 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:51.753 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 64.646146599000005, 2023-11-25T04:23:51.753 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001850546, 2023-11-25T04:23:51.753 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:51.753 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:51.753 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:51.753 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.753 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:51.753 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:51.753 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:51.754 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.754 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.754 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:51.754 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:51.754 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:51.754 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.754 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.754 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:51.754 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:51.754 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7800000000000002e-07 2023-11-25T04:23:51.754 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.754 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.755 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:51.755 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040331+0000", 2023-11-25T04:23:51.755 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0449999999999999e-05 2023-11-25T04:23:51.755 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.755 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.755 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:51.755 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040339+0000", 2023-11-25T04:23:51.755 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0709999999999995e-06 2023-11-25T04:23:51.755 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.755 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.755 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:51.755 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040347+0000", 2023-11-25T04:23:51.756 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7079999999999997e-06 2023-11-25T04:23:51.756 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.756 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.756 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:51.756 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040410+0000", 2023-11-25T04:23:51.756 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2453000000000007e-05 2023-11-25T04:23:51.756 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.756 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.756 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:51.756 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040422+0000", 2023-11-25T04:23:51.756 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2429000000000001e-05 2023-11-25T04:23:51.756 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.756 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.757 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:51.757 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042129+0000", 2023-11-25T04:23:51.757 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001707266 2023-11-25T04:23:51.757 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.757 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.757 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:51.757 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042161+0000", 2023-11-25T04:23:51.757 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1791000000000001e-05 2023-11-25T04:23:51.757 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:51.757 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:51.757 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:51.757 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.758 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.758 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:27 3.13 e18/16 3:c91d4a1d:::mds0_inotable:head v 18'2, mlcod=0'0)", 2023-11-25T04:23:51.758 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:51.758 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 64.645943075999995, 2023-11-25T04:23:51.758 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0016886430000000001, 2023-11-25T04:23:51.758 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:51.758 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:51.758 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:51.758 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.758 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:51.758 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:51.758 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:51.759 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.759 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.759 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:51.759 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:51.759 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:51.759 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.759 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.759 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:51.759 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:51.759 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3999999999999997e-07 2023-11-25T04:23:51.759 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.759 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.759 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:51.760 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040520+0000", 2023-11-25T04:23:51.760 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2989999999999996e-06 2023-11-25T04:23:51.760 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.760 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.760 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:51.760 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040527+0000", 2023-11-25T04:23:51.760 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5730000000000002e-06 2023-11-25T04:23:51.760 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.760 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.760 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:51.760 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040535+0000", 2023-11-25T04:23:51.761 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8630000000000008e-06 2023-11-25T04:23:51.761 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.761 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.761 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:51.761 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040683+0000", 2023-11-25T04:23:51.761 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000148002 2023-11-25T04:23:51.761 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.761 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.761 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:51.761 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040700+0000", 2023-11-25T04:23:51.761 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7065e-05 2023-11-25T04:23:51.761 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.761 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.762 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:51.762 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042163+0000", 2023-11-25T04:23:51.762 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001462767 2023-11-25T04:23:51.762 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.762 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.762 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:51.762 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042203+0000", 2023-11-25T04:23:51.762 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9734000000000003e-05 2023-11-25T04:23:51.762 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:51.762 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:51.762 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:51.762 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.763 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.763 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:20 3.1c e18/16 3:3ed09add:::607.00000000:head v 18'1, mlcod=0'0)", 2023-11-25T04:23:51.763 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:51.763 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 64.645908035000005, 2023-11-25T04:23:51.763 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015271, 2023-11-25T04:23:51.763 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:51.763 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:51.763 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:51.763 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.763 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:51.763 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:51.763 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:51.763 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.764 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.764 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:51.764 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:51.764 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:51.764 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.764 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.764 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:51.764 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:51.764 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4799999999999999e-07 2023-11-25T04:23:51.764 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.764 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.764 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:51.764 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040561+0000", 2023-11-25T04:23:51.764 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1800000000000001e-05 2023-11-25T04:23:51.765 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.765 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.765 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:51.765 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040568+0000", 2023-11-25T04:23:51.765 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7730000000000002e-06 2023-11-25T04:23:51.765 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.765 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.765 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:51.765 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040587+0000", 2023-11-25T04:23:51.765 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8691000000000001e-05 2023-11-25T04:23:51.765 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.765 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.766 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:51.766 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040638+0000", 2023-11-25T04:23:51.766 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1068e-05 2023-11-25T04:23:51.766 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.766 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.766 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:51.766 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040670+0000", 2023-11-25T04:23:51.766 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2125999999999997e-05 2023-11-25T04:23:51.766 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.766 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.767 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:51.767 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042032+0000", 2023-11-25T04:23:51.767 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001361821 2023-11-25T04:23:51.767 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.767 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.767 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:51.767 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042076+0000", 2023-11-25T04:23:51.767 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4472999999999999e-05 2023-11-25T04:23:51.767 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:51.767 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:51.767 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:51.767 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.768 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.768 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:21 3.1c e18/16 3:3ed09add:::607.00000000:head v 18'2, mlcod=0'0)", 2023-11-25T04:23:51.768 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:51.768 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 64.645673310000006, 2023-11-25T04:23:51.768 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0013228160000000001, 2023-11-25T04:23:51.768 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:51.768 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:51.768 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:51.768 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.768 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:51.768 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:51.768 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:51.769 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.769 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.769 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:51.769 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:51.769 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:51.769 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.769 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.769 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:51.769 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:51.769 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9299999999999999e-07 2023-11-25T04:23:51.769 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.769 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.769 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:51.770 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040790+0000", 2023-11-25T04:23:51.770 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5770000000000001e-06 2023-11-25T04:23:51.770 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.770 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.770 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:51.770 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040799+0000", 2023-11-25T04:23:51.770 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6439999999999995e-06 2023-11-25T04:23:51.770 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.770 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.770 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:51.770 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040806+0000", 2023-11-25T04:23:51.770 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0180000000000004e-06 2023-11-25T04:23:51.771 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.771 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.771 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:51.771 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040907+0000", 2023-11-25T04:23:51.771 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000100096 2023-11-25T04:23:51.771 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.771 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.771 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:51.771 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040925+0000", 2023-11-25T04:23:51.771 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8536999999999999e-05 2023-11-25T04:23:51.771 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.771 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.771 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:51.772 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042079+0000", 2023-11-25T04:23:51.772 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00115358 2023-11-25T04:23:51.772 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.772 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.772 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:51.772 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042107+0000", 2023-11-25T04:23:51.772 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7971e-05 2023-11-25T04:23:51.772 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:51.772 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:51.772 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:51.772 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.772 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.772 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop_reply(mds.0.3:4 3.1f e18/16 ondisk, result = 0)", 2023-11-25T04:23:51.773 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:51.773 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 64.644333919999994, 2023-11-25T04:23:51.773 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00044092900000000001, 2023-11-25T04:23:51.773 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:51.773 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:51.773 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:51.773 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.773 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:51.773 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:51.773 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:51.773 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.773 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.774 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:51.774 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:51.774 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:51.774 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.774 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.774 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:51.774 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042124+0000", 2023-11-25T04:23:51.774 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1800000000000005e-07 2023-11-25T04:23:51.774 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.774 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.774 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:51.774 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042128+0000", 2023-11-25T04:23:51.774 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2289999999999997e-06 2023-11-25T04:23:51.775 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.775 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.775 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:51.775 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042138+0000", 2023-11-25T04:23:51.775 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9729999999999992e-06 2023-11-25T04:23:51.775 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.775 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.775 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:51.775 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042145+0000", 2023-11-25T04:23:51.775 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.336e-06 2023-11-25T04:23:51.775 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.775 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.775 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:51.776 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042441+0000", 2023-11-25T04:23:51.776 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029650500000000002 2023-11-25T04:23:51.776 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.776 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.776 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:51.776 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042448+0000", 2023-11-25T04:23:51.776 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4100000000000002e-06 2023-11-25T04:23:51.776 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.776 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.776 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:51.776 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042564+0000", 2023-11-25T04:23:51.776 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000115758 2023-11-25T04:23:51.776 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:51.777 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:51.777 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:51.777 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.777 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.777 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing_reply(3.1f epoch 18/16 rep_tid 8 lcod 18'4)", 2023-11-25T04:23:51.777 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:51.777 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 64.642008508000004, 2023-11-25T04:23:51.777 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00038154699999999998, 2023-11-25T04:23:51.777 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:51.777 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:51.777 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:51.777 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.777 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:51.778 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:51.778 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:51.778 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.778 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.778 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:51.778 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:51.778 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:51.778 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.778 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.778 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:51.778 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:51.779 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2700000000000001e-07 2023-11-25T04:23:51.779 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.779 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.779 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:51.779 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044451+0000", 2023-11-25T04:23:51.779 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7460000000000001e-06 2023-11-25T04:23:51.779 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.779 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.779 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:51.779 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044455+0000", 2023-11-25T04:23:51.779 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5809999999999998e-06 2023-11-25T04:23:51.779 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.779 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.779 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:51.780 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044458+0000", 2023-11-25T04:23:51.780 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0170000000000001e-06 2023-11-25T04:23:51.780 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.780 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.780 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:51.780 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044734+0000", 2023-11-25T04:23:51.780 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00027582799999999999 2023-11-25T04:23:51.780 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.780 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.780 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:51.780 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044830+0000", 2023-11-25T04:23:51.780 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6248000000000004e-05 2023-11-25T04:23:51.781 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:51.781 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:51.781 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:51.781 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.781 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.781 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing_reply(3.b epoch 18/16 rep_tid 5 lcod 18'2)", 2023-11-25T04:23:51.781 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:51.781 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 64.641971846999994, 2023-11-25T04:23:51.781 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000344488, 2023-11-25T04:23:51.781 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:51.781 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:51.781 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:51.781 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.782 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:51.782 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:51.782 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:51.782 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.782 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.782 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:51.782 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:51.782 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:51.782 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.782 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.782 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:51.782 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:51.782 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.55e-07 2023-11-25T04:23:51.783 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.783 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.783 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:51.783 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044487+0000", 2023-11-25T04:23:51.783 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2470000000000001e-06 2023-11-25T04:23:51.783 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.783 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.783 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:51.783 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044491+0000", 2023-11-25T04:23:51.783 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9659999999999998e-06 2023-11-25T04:23:51.783 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.783 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.784 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:51.784 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044494+0000", 2023-11-25T04:23:51.784 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2310000000000001e-06 2023-11-25T04:23:51.784 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.784 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.784 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:51.784 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044756+0000", 2023-11-25T04:23:51.784 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00026172800000000003 2023-11-25T04:23:51.784 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.784 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.784 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:51.784 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044830+0000", 2023-11-25T04:23:51.784 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4160999999999997e-05 2023-11-25T04:23:51.785 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:51.785 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:51.785 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:51.785 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.785 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.785 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:39 3.6 3:654134d2:::mds0_openfiles.0:head [omap-clear,omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2023-11-25T04:23:51.785 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:51.785 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 59.582145805000003, 2023-11-25T04:23:51.785 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001788658, 2023-11-25T04:23:51.785 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:51.785 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:51.785 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:51.786 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.786 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:51.786 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:51.786 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:51.786 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.786 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.786 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:51.786 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:51.786 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:51.786 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.786 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.786 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:51.786 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104314+0000", 2023-11-25T04:23:51.787 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3099999999999999e-06 2023-11-25T04:23:51.787 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.787 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.787 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:51.787 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104325+0000", 2023-11-25T04:23:51.787 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1511999999999999e-05 2023-11-25T04:23:51.787 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.787 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.787 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:51.787 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104362+0000", 2023-11-25T04:23:51.787 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6903000000000002e-05 2023-11-25T04:23:51.787 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.788 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.788 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:51.788 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104387+0000", 2023-11-25T04:23:51.788 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4766e-05 2023-11-25T04:23:51.788 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.788 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.788 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:51.788 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104465+0000", 2023-11-25T04:23:51.788 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8119000000000002e-05 2023-11-25T04:23:51.788 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.788 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.788 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:51.788 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104557+0000", 2023-11-25T04:23:51.788 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2496000000000003e-05 2023-11-25T04:23:51.789 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.789 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.789 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:51.789 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104646+0000", 2023-11-25T04:23:51.789 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8818000000000002e-05 2023-11-25T04:23:51.789 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.789 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.789 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:51.789 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105201+0000", 2023-11-25T04:23:51.789 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00055506000000000002 2023-11-25T04:23:51.789 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.789 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.789 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:51.790 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105994+0000", 2023-11-25T04:23:51.790 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00079294700000000001 2023-11-25T04:23:51.790 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.790 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.790 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:51.790 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.106055+0000", 2023-11-25T04:23:51.790 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0566000000000003e-05 2023-11-25T04:23:51.790 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.790 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.790 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:51.790 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.106100+0000", 2023-11-25T04:23:51.790 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5160999999999997e-05 2023-11-25T04:23:51.791 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:51.791 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:51.791 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:51.791 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.791 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.791 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:43 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2023-11-25T04:23:51.791 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:51.791 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 44.581920809000003, 2023-11-25T04:23:51.791 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001751166, 2023-11-25T04:23:51.791 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:51.791 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:51.791 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:51.791 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.792 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:51.792 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:51.792 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:51.792 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.792 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.792 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:51.792 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:51.792 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:51.792 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.792 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.792 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:51.792 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104539+0000", 2023-11-25T04:23:51.792 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9529999999999999e-06 2023-11-25T04:23:51.793 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.793 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.793 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:51.793 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104557+0000", 2023-11-25T04:23:51.793 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7464000000000001e-05 2023-11-25T04:23:51.793 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.793 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.793 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:51.793 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104590+0000", 2023-11-25T04:23:51.793 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2966000000000003e-05 2023-11-25T04:23:51.793 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.793 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.793 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:51.794 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104608+0000", 2023-11-25T04:23:51.794 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8193e-05 2023-11-25T04:23:51.794 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.794 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.794 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:51.794 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104767+0000", 2023-11-25T04:23:51.794 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015882 2023-11-25T04:23:51.794 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.794 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.794 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:51.795 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104909+0000", 2023-11-25T04:23:51.795 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014244999999999999 2023-11-25T04:23:51.795 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.795 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.795 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:51.795 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.105059+0000", 2023-11-25T04:23:51.795 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014964399999999999 2023-11-25T04:23:51.795 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.795 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.795 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:51.795 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.105701+0000", 2023-11-25T04:23:51.795 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00064201099999999995 2023-11-25T04:23:51.796 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.796 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.796 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:51.796 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106233+0000", 2023-11-25T04:23:51.796 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00053178500000000001 2023-11-25T04:23:51.796 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.796 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.796 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:51.796 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106265+0000", 2023-11-25T04:23:51.796 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1986000000000002e-05 2023-11-25T04:23:51.796 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.796 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.796 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:51.797 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106288+0000", 2023-11-25T04:23:51.797 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2894000000000001e-05 2023-11-25T04:23:51.797 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:51.797 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:51.797 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:51.797 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.797 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.797 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:45 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e23)", 2023-11-25T04:23:51.797 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:23:51.797 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 39.581722769000002, 2023-11-25T04:23:51.797 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00098423299999999994, 2023-11-25T04:23:51.797 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:51.797 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:51.798 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:51.798 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.798 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:51.798 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:23:51.798 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:51.798 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.798 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.798 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:51.798 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:23:51.798 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:51.798 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.798 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.799 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:51.799 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104737+0000", 2023-11-25T04:23:51.799 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3309999999999998e-06 2023-11-25T04:23:51.799 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.799 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.799 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:51.799 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104748+0000", 2023-11-25T04:23:51.799 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1080999999999999e-05 2023-11-25T04:23:51.799 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.799 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.799 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:51.799 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104771+0000", 2023-11-25T04:23:51.799 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2707000000000001e-05 2023-11-25T04:23:51.800 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.800 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.800 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:51.800 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104800+0000", 2023-11-25T04:23:51.800 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9595e-05 2023-11-25T04:23:51.800 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.800 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.800 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:51.800 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104890+0000", 2023-11-25T04:23:51.800 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.9808999999999993e-05 2023-11-25T04:23:51.800 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.800 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.800 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:51.801 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105004+0000", 2023-11-25T04:23:51.801 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000114508 2023-11-25T04:23:51.801 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.801 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.801 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:51.801 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105634+0000", 2023-11-25T04:23:51.801 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00062985799999999996 2023-11-25T04:23:51.801 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.801 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.801 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:51.801 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105671+0000", 2023-11-25T04:23:51.801 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6640000000000002e-05 2023-11-25T04:23:51.802 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:51.802 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:51.802 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:51.802 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105719+0000", 2023-11-25T04:23:51.802 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7704000000000003e-05 2023-11-25T04:23:51.802 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:51.802 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:51.802 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:51.802 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:51.802 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:51.802 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:51.804 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:51.802+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:51.804 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:23:51.904 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:51.903+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:51.914 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:51.914 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:23:51.914 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2023-11-25T04:23:51.914 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:51.923 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:23:52.005 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2023-11-25T04:23:52.006 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:52.004+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:52.058 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-11-25T04:23:52.069 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:52.069 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:23:52.069 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2023-11-25T04:23:52.069 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2023-11-25T04:23:52.069 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:52.078 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:23:52.107 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:52.105+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:52.185 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:52.185 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2023-11-25T04:23:52.185 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2023-11-25T04:23:52.185 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2023-11-25T04:23:52.185 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.185 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:155 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'109, mlcod=11'105)", 2023-11-25T04:23:52.185 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:52.185 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 74.397870112999996, 2023-11-25T04:23:52.186 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061719019999999999, 2023-11-25T04:23:52.186 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:52.186 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:52.186 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:52.186 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.186 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:52.186 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:52.186 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:52.186 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.186 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.186 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:52.186 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:52.187 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:52.187 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.187 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.187 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:52.187 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784884+0000", 2023-11-25T04:23:52.187 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3099999999999999e-07 2023-11-25T04:23:52.187 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.187 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.187 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:52.187 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784942+0000", 2023-11-25T04:23:52.187 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8570000000000003e-05 2023-11-25T04:23:52.187 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.188 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:52.188 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784949+0000", 2023-11-25T04:23:52.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0310000000000001e-06 2023-11-25T04:23:52.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.188 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:52.188 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784953+0000", 2023-11-25T04:23:52.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6320000000000001e-06 2023-11-25T04:23:52.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.188 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:52.188 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784993+0000", 2023-11-25T04:23:52.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9737000000000003e-05 2023-11-25T04:23:52.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:52.189 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785011+0000", 2023-11-25T04:23:52.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8349999999999999e-05 2023-11-25T04:23:52.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:52.189 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791024+0000", 2023-11-25T04:23:52.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006012997 2023-11-25T04:23:52.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.190 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:52.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791055+0000", 2023-11-25T04:23:52.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1254000000000003e-05 2023-11-25T04:23:52.190 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:52.190 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:52.190 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:52.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.190 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:156 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'110, mlcod=11'105)", 2023-11-25T04:23:52.191 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:52.191 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 74.397284064000004, 2023-11-25T04:23:52.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005616116, 2023-11-25T04:23:52.191 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:52.191 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:52.191 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:52.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.191 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:52.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:52.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:52.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.192 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:52.192 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:52.192 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:52.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.193 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:52.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785470+0000", 2023-11-25T04:23:52.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0699999999999998e-07 2023-11-25T04:23:52.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:52.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785474+0000", 2023-11-25T04:23:52.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.048e-06 2023-11-25T04:23:52.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:52.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785480+0000", 2023-11-25T04:23:52.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7729999999999996e-06 2023-11-25T04:23:52.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.195 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:52.195 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785483+0000", 2023-11-25T04:23:52.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1970000000000001e-06 2023-11-25T04:23:52.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.195 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:52.195 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787936+0000", 2023-11-25T04:23:52.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024529040000000001 2023-11-25T04:23:52.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.195 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:52.195 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787947+0000", 2023-11-25T04:23:52.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1669e-05 2023-11-25T04:23:52.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.196 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:52.196 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791057+0000", 2023-11-25T04:23:52.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031095200000000002 2023-11-25T04:23:52.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.196 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:52.196 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791086+0000", 2023-11-25T04:23:52.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8697999999999999e-05 2023-11-25T04:23:52.196 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:52.196 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:52.197 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:52.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.197 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:157 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'111, mlcod=11'106)", 2023-11-25T04:23:52.197 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:52.197 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 74.396691505999996, 2023-11-25T04:23:52.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050590280000000001, 2023-11-25T04:23:52.197 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:52.197 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:52.197 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:52.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:52.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:52.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:52.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:52.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:52.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:52.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:52.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786063+0000", 2023-11-25T04:23:52.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2600000000000002e-07 2023-11-25T04:23:52.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:52.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786069+0000", 2023-11-25T04:23:52.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8839999999999999e-06 2023-11-25T04:23:52.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:52.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786079+0000", 2023-11-25T04:23:52.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0124e-05 2023-11-25T04:23:52.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.200 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:52.200 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786085+0000", 2023-11-25T04:23:52.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6660000000000002e-06 2023-11-25T04:23:52.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.200 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:52.200 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788160+0000", 2023-11-25T04:23:52.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020747669999999999 2023-11-25T04:23:52.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:52.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788180+0000", 2023-11-25T04:23:52.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0208999999999999e-05 2023-11-25T04:23:52.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:52.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791087+0000", 2023-11-25T04:23:52.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029064630000000002 2023-11-25T04:23:52.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:52.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791121+0000", 2023-11-25T04:23:52.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4289000000000003e-05 2023-11-25T04:23:52.202 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:52.202 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:52.202 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:52.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.202 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:158 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'112, mlcod=11'106)", 2023-11-25T04:23:52.202 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:52.202 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 74.396275113000002, 2023-11-25T04:23:52.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046663030000000001, 2023-11-25T04:23:52.202 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:52.202 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:52.202 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:52.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:52.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:52.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:52.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:52.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:52.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:52.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:52.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786479+0000", 2023-11-25T04:23:52.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2799999999999996e-07 2023-11-25T04:23:52.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.204 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:52.204 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786498+0000", 2023-11-25T04:23:52.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9069e-05 2023-11-25T04:23:52.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.204 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:52.204 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786508+0000", 2023-11-25T04:23:52.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5810000000000003e-06 2023-11-25T04:23:52.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.205 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:52.205 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786512+0000", 2023-11-25T04:23:52.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4889999999999997e-06 2023-11-25T04:23:52.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.205 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:52.205 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788438+0000", 2023-11-25T04:23:52.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019257059999999999 2023-11-25T04:23:52.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.205 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:52.205 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788449+0000", 2023-11-25T04:23:52.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1419e-05 2023-11-25T04:23:52.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:52.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791122+0000", 2023-11-25T04:23:52.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002672822 2023-11-25T04:23:52.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:52.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791145+0000", 2023-11-25T04:23:52.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2589e-05 2023-11-25T04:23:52.206 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:52.206 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:52.207 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:52.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.207 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:206 1.0 e12/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 12'152, mlcod=12'148)", 2023-11-25T04:23:52.207 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:52.207 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.884506642999995, 2023-11-25T04:23:52.207 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046728189999999999, 2023-11-25T04:23:52.207 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:52.207 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:52.207 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:52.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.207 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:52.207 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:52.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:52.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.208 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:52.208 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:52.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:52.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.208 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:52.208 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298248+0000", 2023-11-25T04:23:52.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3600000000000003e-07 2023-11-25T04:23:52.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.209 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:52.209 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298278+0000", 2023-11-25T04:23:52.209 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0474e-05 2023-11-25T04:23:52.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.209 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.209 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:52.209 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298288+0000", 2023-11-25T04:23:52.209 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0156000000000001e-05 2023-11-25T04:23:52.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.209 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.209 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:52.209 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298293+0000", 2023-11-25T04:23:52.210 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7400000000000004e-06 2023-11-25T04:23:52.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.210 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:52.210 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298333+0000", 2023-11-25T04:23:52.210 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9666000000000003e-05 2023-11-25T04:23:52.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.210 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:52.210 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298351+0000", 2023-11-25T04:23:52.210 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8544999999999999e-05 2023-11-25T04:23:52.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.211 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:52.211 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302891+0000", 2023-11-25T04:23:52.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045398519999999996 2023-11-25T04:23:52.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.211 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:52.211 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302920+0000", 2023-11-25T04:23:52.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8750000000000001e-05 2023-11-25T04:23:52.211 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:52.211 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:52.211 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:52.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.212 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:14 3.15 3:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:52.212 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:52.212 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 65.149372286000002, 2023-11-25T04:23:52.212 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010990437, 2023-11-25T04:23:52.212 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:52.212 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:52.212 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:52.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.212 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:52.212 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:52.212 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:52.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.213 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:52.213 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:52.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:52.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.213 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:52.213 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033383+0000", 2023-11-25T04:23:52.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9870000000000002e-06 2023-11-25T04:23:52.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.213 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:52.213 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033398+0000", 2023-11-25T04:23:52.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4569999999999999e-05 2023-11-25T04:23:52.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.214 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.214 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:52.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033411+0000", 2023-11-25T04:23:52.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3543999999999999e-05 2023-11-25T04:23:52.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.214 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.214 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:52.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033423+0000", 2023-11-25T04:23:52.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1905999999999999e-05 2023-11-25T04:23:52.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.214 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:52.215 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034433+0000", 2023-11-25T04:23:52.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00101022 2023-11-25T04:23:52.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:52.215 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034539+0000", 2023-11-25T04:23:52.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010597199999999999 2023-11-25T04:23:52.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:52.215 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034667+0000", 2023-11-25T04:23:52.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012765500000000001 2023-11-25T04:23:52.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:52.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037803+0000", 2023-11-25T04:23:52.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031362220000000001 2023-11-25T04:23:52.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:52.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:23:52.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0064964669999999997 2023-11-25T04:23:52.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:52.217 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044331+0000", 2023-11-25T04:23:52.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1544999999999997e-05 2023-11-25T04:23:52.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.217 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:52.217 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044372+0000", 2023-11-25T04:23:52.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0349000000000003e-05 2023-11-25T04:23:52.217 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:52.217 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:52.217 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:52.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.217 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:15 3.15 3:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:52.218 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:52.218 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 65.148712635999999, 2023-11-25T04:23:52.218 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010464362, 2023-11-25T04:23:52.218 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:52.218 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:52.218 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:52.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.218 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:52.218 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:52.218 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:52.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.219 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:52.219 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:52.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:52.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.219 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:52.219 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034042+0000", 2023-11-25T04:23:52.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0999999999999997e-07 2023-11-25T04:23:52.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.219 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:52.219 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034048+0000", 2023-11-25T04:23:52.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0390000000000001e-06 2023-11-25T04:23:52.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:52.220 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034059+0000", 2023-11-25T04:23:52.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1182999999999999e-05 2023-11-25T04:23:52.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:52.220 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034067+0000", 2023-11-25T04:23:52.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1559999999999995e-06 2023-11-25T04:23:52.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:52.221 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035005+0000", 2023-11-25T04:23:52.221 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00093747399999999999 2023-11-25T04:23:52.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.221 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.221 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:52.221 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035077+0000", 2023-11-25T04:23:52.221 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2747e-05 2023-11-25T04:23:52.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.221 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.221 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:52.221 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035219+0000", 2023-11-25T04:23:52.221 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000141877 2023-11-25T04:23:52.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.222 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:52.222 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037810+0000", 2023-11-25T04:23:52.222 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002590752 2023-11-25T04:23:52.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.222 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:52.222 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044437+0000", 2023-11-25T04:23:52.222 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066270110000000004 2023-11-25T04:23:52.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.222 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:52.223 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044481+0000", 2023-11-25T04:23:52.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4106000000000001e-05 2023-11-25T04:23:52.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.223 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.223 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:52.223 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044505+0000", 2023-11-25T04:23:52.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4207e-05 2023-11-25T04:23:52.223 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:52.223 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:52.223 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:52.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.223 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.223 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:16 3.1d 3:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:52.224 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:52.224 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 65.148670788000004, 2023-11-25T04:23:52.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010413923, 2023-11-25T04:23:52.224 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:52.224 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:52.224 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:52.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:52.224 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:52.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:52.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:52.225 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:52.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:52.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.225 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:52.225 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:52.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4499999999999997e-07 2023-11-25T04:23:52.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.225 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:52.225 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034087+0000", 2023-11-25T04:23:52.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9770000000000002e-06 2023-11-25T04:23:52.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.226 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:52.226 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034094+0000", 2023-11-25T04:23:52.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8809999999999998e-06 2023-11-25T04:23:52.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.226 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:52.226 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034100+0000", 2023-11-25T04:23:52.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8440000000000003e-06 2023-11-25T04:23:52.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.226 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:52.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034993+0000", 2023-11-25T04:23:52.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00089336799999999996 2023-11-25T04:23:52.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:52.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035099+0000", 2023-11-25T04:23:52.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106037 2023-11-25T04:23:52.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:52.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035276+0000", 2023-11-25T04:23:52.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017631799999999999 2023-11-25T04:23:52.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.228 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:52.228 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037815+0000", 2023-11-25T04:23:52.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025395959999999999 2023-11-25T04:23:52.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.228 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:52.228 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044420+0000", 2023-11-25T04:23:52.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066045749999999997 2023-11-25T04:23:52.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.228 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:52.228 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044463+0000", 2023-11-25T04:23:52.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3205999999999999e-05 2023-11-25T04:23:52.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.229 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:52.229 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044497+0000", 2023-11-25T04:23:52.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3476000000000003e-05 2023-11-25T04:23:52.229 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:52.229 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:52.229 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:52.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.229 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:17 3.1d 3:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:52.229 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:52.230 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 65.148645579000004, 2023-11-25T04:23:52.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01127356, 2023-11-25T04:23:52.230 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:52.230 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:52.230 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:52.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:52.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:52.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:52.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:52.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:52.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:52.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:52.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:52.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1100000000000001e-07 2023-11-25T04:23:52.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:52.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034112+0000", 2023-11-25T04:23:52.231 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8600000000000003e-06 2023-11-25T04:23:52.231 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:52.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034120+0000", 2023-11-25T04:23:52.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7700000000000001e-06 2023-11-25T04:23:52.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:52.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034127+0000", 2023-11-25T04:23:52.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0999999999999998e-06 2023-11-25T04:23:52.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:52.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035601+0000", 2023-11-25T04:23:52.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014739410000000001 2023-11-25T04:23:52.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.233 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:52.233 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035685+0000", 2023-11-25T04:23:52.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3677999999999994e-05 2023-11-25T04:23:52.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.233 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:52.233 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035827+0000", 2023-11-25T04:23:52.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014232599999999999 2023-11-25T04:23:52.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:52.234 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038725+0000", 2023-11-25T04:23:52.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028982779999999998 2023-11-25T04:23:52.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:52.234 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045323+0000", 2023-11-25T04:23:52.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0065980780000000003 2023-11-25T04:23:52.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:52.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045354+0000", 2023-11-25T04:23:52.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0463e-05 2023-11-25T04:23:52.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.235 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:52.235 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045381+0000", 2023-11-25T04:23:52.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7654999999999999e-05 2023-11-25T04:23:52.235 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:52.235 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:52.235 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:52.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.235 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:22 3.e 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:52.236 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:52.236 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 65.148619166000003, 2023-11-25T04:23:52.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010133422, 2023-11-25T04:23:52.236 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:52.236 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:52.236 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:52.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:52.236 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:52.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:52.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:52.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:52.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:52.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:52.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034135+0000", 2023-11-25T04:23:52.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-07 2023-11-25T04:23:52.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.237 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:52.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034138+0000", 2023-11-25T04:23:52.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6009999999999999e-06 2023-11-25T04:23:52.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:52.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034145+0000", 2023-11-25T04:23:52.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1169999999999998e-06 2023-11-25T04:23:52.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:52.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034150+0000", 2023-11-25T04:23:52.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9080000000000003e-06 2023-11-25T04:23:52.238 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.238 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:52.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034209+0000", 2023-11-25T04:23:52.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8149000000000002e-05 2023-11-25T04:23:52.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.239 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:52.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034415+0000", 2023-11-25T04:23:52.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020637699999999999 2023-11-25T04:23:52.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.239 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:52.239 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034584+0000", 2023-11-25T04:23:52.239 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016870399999999999 2023-11-25T04:23:52.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:52.240 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.036979+0000", 2023-11-25T04:23:52.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023957509999999998 2023-11-25T04:23:52.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:52.240 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044161+0000", 2023-11-25T04:23:52.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0071811590000000003 2023-11-25T04:23:52.240 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.240 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.240 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:52.240 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044224+0000", 2023-11-25T04:23:52.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3454000000000006e-05 2023-11-25T04:23:52.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.241 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:52.241 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044268+0000", 2023-11-25T04:23:52.241 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3761999999999999e-05 2023-11-25T04:23:52.241 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:52.241 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:52.241 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:52.241 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.241 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.241 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:23 3.e 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:52.241 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:52.241 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 65.148587757000001, 2023-11-25T04:23:52.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010281676, 2023-11-25T04:23:52.242 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:52.242 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:52.242 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:52.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:52.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:52.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:52.242 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:52.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:52.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:52.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:52.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:52.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5799999999999999e-07 2023-11-25T04:23:52.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:52.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034169+0000", 2023-11-25T04:23:52.243 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1879999999999998e-06 2023-11-25T04:23:52.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:52.244 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034177+0000", 2023-11-25T04:23:52.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7020000000000001e-06 2023-11-25T04:23:52.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.244 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:52.244 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034184+0000", 2023-11-25T04:23:52.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3910000000000003e-06 2023-11-25T04:23:52.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.244 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:52.244 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034924+0000", 2023-11-25T04:23:52.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00074032999999999998 2023-11-25T04:23:52.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.245 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:52.245 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034991+0000", 2023-11-25T04:23:52.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7347000000000004e-05 2023-11-25T04:23:52.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.245 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:52.245 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035096+0000", 2023-11-25T04:23:52.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010488 2023-11-25T04:23:52.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.245 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:52.245 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037588+0000", 2023-11-25T04:23:52.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024918309999999999 2023-11-25T04:23:52.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:52.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044367+0000", 2023-11-25T04:23:52.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067787589999999997 2023-11-25T04:23:52.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:52.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044415+0000", 2023-11-25T04:23:52.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7803000000000003e-05 2023-11-25T04:23:52.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:52.247 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044447+0000", 2023-11-25T04:23:52.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2886999999999999e-05 2023-11-25T04:23:52.247 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:52.247 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:52.247 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:52.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.247 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:30 3.15 3:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:52.247 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:52.247 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 65.148529316999998, 2023-11-25T04:23:52.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010365070000000001, 2023-11-25T04:23:52.248 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:52.248 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:52.248 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:52.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:52.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:52.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:52.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:52.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:52.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:52.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:52.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034225+0000", 2023-11-25T04:23:52.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7400000000000002e-07 2023-11-25T04:23:52.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:52.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034229+0000", 2023-11-25T04:23:52.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3100000000000002e-06 2023-11-25T04:23:52.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:52.250 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034239+0000", 2023-11-25T04:23:52.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9140000000000003e-06 2023-11-25T04:23:52.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.250 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:52.250 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034246+0000", 2023-11-25T04:23:52.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0149999999999998e-06 2023-11-25T04:23:52.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.250 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:52.250 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035572+0000", 2023-11-25T04:23:52.250 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001325655 2023-11-25T04:23:52.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:52.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035730+0000", 2023-11-25T04:23:52.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015827899999999999 2023-11-25T04:23:52.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:52.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035857+0000", 2023-11-25T04:23:52.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012644699999999999 2023-11-25T04:23:52.251 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.251 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.251 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:52.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038737+0000", 2023-11-25T04:23:52.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002880077 2023-11-25T04:23:52.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:52.252 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044540+0000", 2023-11-25T04:23:52.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058034239999999997 2023-11-25T04:23:52.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.252 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:52.252 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044569+0000", 2023-11-25T04:23:52.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8388999999999999e-05 2023-11-25T04:23:52.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.253 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:52.253 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044589+0000", 2023-11-25T04:23:52.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0686000000000002e-05 2023-11-25T04:23:52.253 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:52.253 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:52.253 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:52.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.253 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:33 3.1b 3:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:52.253 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:52.253 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 65.148483819000006, 2023-11-25T04:23:52.253 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099754049999999997, 2023-11-25T04:23:52.253 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:52.254 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:23:52.254 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:52.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:52.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:52.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:52.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:52.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:52.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:52.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:52.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:52.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4799999999999998e-07 2023-11-25T04:23:52.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:52.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034273+0000", 2023-11-25T04:23:52.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7819999999999998e-06 2023-11-25T04:23:52.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:52.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034285+0000", 2023-11-25T04:23:52.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2058e-05 2023-11-25T04:23:52.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:52.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034291+0000", 2023-11-25T04:23:52.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0680000000000002e-06 2023-11-25T04:23:52.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:52.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034370+0000", 2023-11-25T04:23:52.256 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8452e-05 2023-11-25T04:23:52.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:52.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034477+0000", 2023-11-25T04:23:52.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107471 2023-11-25T04:23:52.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:52.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044245+0000", 2023-11-25T04:23:52.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0097679259999999997 2023-11-25T04:23:52.257 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:52.257 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:52.257 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:52.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.258 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:36 3.1a 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:52.258 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:52.258 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 65.148437146000006, 2023-11-25T04:23:52.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058179099999999999, 2023-11-25T04:23:52.258 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:52.258 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:23:52.258 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:52.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:52.258 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:52.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:52.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:52.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:52.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:52.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:52.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034317+0000", 2023-11-25T04:23:52.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1999999999999999e-07 2023-11-25T04:23:52.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.259 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:52.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034321+0000", 2023-11-25T04:23:52.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.233e-06 2023-11-25T04:23:52.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:52.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034337+0000", 2023-11-25T04:23:52.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5308999999999999e-05 2023-11-25T04:23:52.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:52.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034354+0000", 2023-11-25T04:23:52.260 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7411999999999999e-05 2023-11-25T04:23:52.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.260 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:52.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034465+0000", 2023-11-25T04:23:52.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011154699999999999 2023-11-25T04:23:52.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:52.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034560+0000", 2023-11-25T04:23:52.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4186999999999998e-05 2023-11-25T04:23:52.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.261 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:52.261 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.040134+0000", 2023-11-25T04:23:52.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055746019999999997 2023-11-25T04:23:52.261 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:52.262 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:52.262 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:52.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.262 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:9 3.10 e18/16 3:09401a81:::601.00000000:head v 18'4, mlcod=0'0)", 2023-11-25T04:23:52.262 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:52.262 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 65.139542837999997, 2023-11-25T04:23:52.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061641020000000003, 2023-11-25T04:23:52.262 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:52.262 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:52.262 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:52.262 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:52.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:52.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:52.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:52.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:52.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:52.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:52.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:52.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5300000000000001e-07 2023-11-25T04:23:52.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:52.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043222+0000", 2023-11-25T04:23:52.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.082e-05 2023-11-25T04:23:52.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:52.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043228+0000", 2023-11-25T04:23:52.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6699999999999997e-06 2023-11-25T04:23:52.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:52.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043236+0000", 2023-11-25T04:23:52.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8189999999999992e-06 2023-11-25T04:23:52.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:52.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043507+0000", 2023-11-25T04:23:52.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000271012 2023-11-25T04:23:52.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.265 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:52.265 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043530+0000", 2023-11-25T04:23:52.265 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2659000000000001e-05 2023-11-25T04:23:52.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:52.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049321+0000", 2023-11-25T04:23:52.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057915129999999999 2023-11-25T04:23:52.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.266 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.266 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:52.266 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049375+0000", 2023-11-25T04:23:52.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3455999999999998e-05 2023-11-25T04:23:52.266 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:52.266 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:52.266 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:52.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.267 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:1 3.14 e18/16 3:292cf221:::200.00000000:head v 18'3, mlcod=0'0)", 2023-11-25T04:23:52.267 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:52.267 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 65.139506124999997, 2023-11-25T04:23:52.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0060983599999999997, 2023-11-25T04:23:52.267 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:52.267 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:52.267 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:52.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.267 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:52.267 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:52.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:52.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:52.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:52.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:52.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:52.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043248+0000", 2023-11-25T04:23:52.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9399999999999995e-07 2023-11-25T04:23:52.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:52.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043252+0000", 2023-11-25T04:23:52.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1799999999999998e-06 2023-11-25T04:23:52.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:52.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043259+0000", 2023-11-25T04:23:52.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0049999999999998e-06 2023-11-25T04:23:52.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:52.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043265+0000", 2023-11-25T04:23:52.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0859999999999998e-06 2023-11-25T04:23:52.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:52.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043311+0000", 2023-11-25T04:23:52.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6082999999999999e-05 2023-11-25T04:23:52.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:52.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043330+0000", 2023-11-25T04:23:52.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8556e-05 2023-11-25T04:23:52.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:52.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049303+0000", 2023-11-25T04:23:52.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059733470000000004 2023-11-25T04:23:52.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:52.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049346+0000", 2023-11-25T04:23:52.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2608999999999998e-05 2023-11-25T04:23:52.271 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:52.271 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:52.271 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:52.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.271 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:3 3.14 e18/16 3:287034c4:::1.00000000.inode:head v 18'4, mlcod=0'0)", 2023-11-25T04:23:52.272 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:52.272 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 65.138989619, 2023-11-25T04:23:52.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057040090000000003, 2023-11-25T04:23:52.272 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:52.272 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:52.272 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:52.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:52.272 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:52.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:52.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:52.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:52.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:52.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:52.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:52.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9799999999999999e-07 2023-11-25T04:23:52.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:52.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043785+0000", 2023-11-25T04:23:52.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0721999999999999e-05 2023-11-25T04:23:52.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:52.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043793+0000", 2023-11-25T04:23:52.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5839999999999995e-06 2023-11-25T04:23:52.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:52.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043803+0000", 2023-11-25T04:23:52.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0729999999999994e-06 2023-11-25T04:23:52.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:52.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043843+0000", 2023-11-25T04:23:52.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0747999999999997e-05 2023-11-25T04:23:52.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:52.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043856+0000", 2023-11-25T04:23:52.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2697000000000001e-05 2023-11-25T04:23:52.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:52.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049426+0000", 2023-11-25T04:23:52.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055704409999999998 2023-11-25T04:23:52.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:52.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049468+0000", 2023-11-25T04:23:52.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1446000000000003e-05 2023-11-25T04:23:52.276 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:52.276 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:52.276 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:52.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.276 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "pg_update_log_missing(3.14 epoch 18/16 rep_tid 171 entries 18'5 (0'0) error 3:2e2fa760:::200.00000001:head by mds.0.3:32 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:23:52.276 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:52.276 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 65.138828656000001, 2023-11-25T04:23:52.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005556938, 2023-11-25T04:23:52.276 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:52.277 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2023-11-25T04:23:52.277 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:52.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:52.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:52.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:52.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:52.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:52.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:52.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:52.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:52.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.03e-07 2023-11-25T04:23:52.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:52.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043931+0000", 2023-11-25T04:23:52.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1079999999999998e-06 2023-11-25T04:23:52.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:52.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043944+0000", 2023-11-25T04:23:52.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2852e-05 2023-11-25T04:23:52.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:52.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043955+0000", 2023-11-25T04:23:52.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1037e-05 2023-11-25T04:23:52.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:52.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044088+0000", 2023-11-25T04:23:52.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013269599999999999 2023-11-25T04:23:52.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:52.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049482+0000", 2023-11-25T04:23:52.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053940419999999999 2023-11-25T04:23:52.280 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:52.280 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:52.280 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:52.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.280 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:10 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'5, mlcod=0'0)", 2023-11-25T04:23:52.280 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:52.280 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 65.138738627999999, 2023-11-25T04:23:52.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054493099999999997, 2023-11-25T04:23:52.280 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:52.280 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:52.281 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:52.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:52.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:52.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:52.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:52.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:52.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:52.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.281 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:52.281 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:52.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6100000000000002e-07 2023-11-25T04:23:52.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:52.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044019+0000", 2023-11-25T04:23:52.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.839e-06 2023-11-25T04:23:52.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:52.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044026+0000", 2023-11-25T04:23:52.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5869999999999998e-06 2023-11-25T04:23:52.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:52.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044031+0000", 2023-11-25T04:23:52.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4909999999999996e-06 2023-11-25T04:23:52.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:52.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044070+0000", 2023-11-25T04:23:52.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8829999999999999e-05 2023-11-25T04:23:52.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:52.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044081+0000", 2023-11-25T04:23:52.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1447999999999999e-05 2023-11-25T04:23:52.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:52.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049406+0000", 2023-11-25T04:23:52.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053246480000000004 2023-11-25T04:23:52.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:52.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049464+0000", 2023-11-25T04:23:52.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8205999999999998e-05 2023-11-25T04:23:52.284 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:52.284 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:52.284 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:52.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.285 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:11 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'6, mlcod=0'0)", 2023-11-25T04:23:52.285 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:52.285 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 65.138537364000001, 2023-11-25T04:23:52.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052673499999999996, 2023-11-25T04:23:52.285 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:52.285 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:52.285 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:52.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:52.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:52.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:52.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:52.286 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:52.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:52.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:52.286 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:52.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0199999999999998e-07 2023-11-25T04:23:52.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:52.286 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044220+0000", 2023-11-25T04:23:52.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7720000000000001e-06 2023-11-25T04:23:52.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:52.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044226+0000", 2023-11-25T04:23:52.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8579999999999996e-06 2023-11-25T04:23:52.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:52.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044232+0000", 2023-11-25T04:23:52.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1700000000000002e-06 2023-11-25T04:23:52.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:52.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044274+0000", 2023-11-25T04:23:52.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1254999999999997e-05 2023-11-25T04:23:52.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:52.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:23:52.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6880999999999999e-05 2023-11-25T04:23:52.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:52.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049466+0000", 2023-11-25T04:23:52.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051659219999999999 2023-11-25T04:23:52.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:52.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:52.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:52.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049483+0000", 2023-11-25T04:23:52.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.719e-05 2023-11-25T04:23:52.289 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:52.289 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:52.289 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:52.289 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:52.289 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:52.290 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:52.290 INFO:teuthology.orchestra.run.smithi067.stderr:Error ENXIO: problem getting command descriptions from osd.0 2023-11-25T04:23:52.291 INFO:teuthology.orchestra.run.smithi067.stdout:osd.1: {} 2023-11-25T04:23:52.291 INFO:teuthology.orchestra.run.smithi067.stderr:osd.1: osd_enable_op_tracker = '' 2023-11-25T04:23:52.292 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:23:52.292 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:52.289+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:52.302 INFO:teuthology.orchestra.run.smithi067.stdout:osd.2: {} 2023-11-25T04:23:52.302 INFO:teuthology.orchestra.run.smithi067.stderr:osd.2: osd_enable_op_tracker = '' 2023-11-25T04:23:52.305 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-11-25T04:23:52.305 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-11-25T04:23:52.392 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:52.390+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:52.403 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:52.403 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:52.411 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:23:52.493 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:52.492+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:52.558 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:52.558 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:52.567 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:23:52.594 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:52.593+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:52.672 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:52.673 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:52.682 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:23:52.695 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:52.693+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:52.792 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:52.792 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:52.795 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:52.794+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:52.801 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:23:52.896 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:52.894+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:52.946 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:52.946 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:52.954 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:23:52.986 INFO:teuthology.orchestra.run.smithi067.stderr:noscrub is set 2023-11-25T04:23:53.000 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:52.997+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:53.061 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:53.061 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:53.072 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:23:53.112 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:53.111+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:53.182 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:53.183 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:53.191 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:23:53.199 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:53.198+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:53.297 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:53.297 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:53.300 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:53.298+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:53.306 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:23:53.401 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:53.400+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:53.412 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:53.412 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:53.421 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:23:53.502 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:53.500+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:53.527 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:53.527 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:53.535 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:23:53.602 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:53.601+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:53.641 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:53.641 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:53.650 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:23:53.703 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:53.701+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:53.796 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:53.796 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:53.804 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:53.802+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:53.805 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:23:53.905 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:53.903+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:53.912 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:53.913 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:53.922 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:23:54.006 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:54.005+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:54.067 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:54.068 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:54.077 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:23:54.120 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:54.119+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:54.182 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:54.182 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:54.191 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:23:54.235 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:54.234+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:54.300 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:54.300 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:54.306 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-11-25T04:23:54.308 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:23:54.310 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:54.308+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:54.411 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:54.409+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:54.419 INFO:teuthology.orchestra.run.smithi067.stderr:Error ENXIO: problem getting command descriptions from osd.0 2023-11-25T04:23:54.454 INFO:teuthology.orchestra.run.smithi067.stdout:osd.1: {} 2023-11-25T04:23:54.454 INFO:teuthology.orchestra.run.smithi067.stderr:osd.1: osd_enable_op_tracker = '' 2023-11-25T04:23:54.454 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:54.455 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:23:54.455 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2023-11-25T04:23:54.455 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2023-11-25T04:23:54.455 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:54.463 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:23:54.478 INFO:teuthology.orchestra.run.smithi067.stdout:osd.2: {} 2023-11-25T04:23:54.478 INFO:teuthology.orchestra.run.smithi067.stderr:osd.2: osd_enable_op_tracker = '' 2023-11-25T04:23:54.480 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-11-25T04:23:54.481 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-11-25T04:23:54.547 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:54.546+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:54.609 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:54.609 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2023-11-25T04:23:54.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2023-11-25T04:23:54.609 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2023-11-25T04:23:54.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.609 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:155 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'109, mlcod=11'105)", 2023-11-25T04:23:54.609 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:54.609 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 76.821686, 2023-11-25T04:23:54.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061719019999999999, 2023-11-25T04:23:54.609 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:54.610 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:54.610 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:54.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.610 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:54.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:54.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:54.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.610 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:54.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:54.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:54.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:54.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784884+0000", 2023-11-25T04:23:54.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3099999999999999e-07 2023-11-25T04:23:54.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:54.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784942+0000", 2023-11-25T04:23:54.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8570000000000003e-05 2023-11-25T04:23:54.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:54.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784949+0000", 2023-11-25T04:23:54.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0310000000000001e-06 2023-11-25T04:23:54.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:54.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784953+0000", 2023-11-25T04:23:54.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6320000000000001e-06 2023-11-25T04:23:54.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:54.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784993+0000", 2023-11-25T04:23:54.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9737000000000003e-05 2023-11-25T04:23:54.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.613 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:54.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785011+0000", 2023-11-25T04:23:54.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8349999999999999e-05 2023-11-25T04:23:54.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.613 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:54.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791024+0000", 2023-11-25T04:23:54.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006012997 2023-11-25T04:23:54.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:54.614 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791055+0000", 2023-11-25T04:23:54.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1254000000000003e-05 2023-11-25T04:23:54.614 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:54.614 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:54.614 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:54.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.615 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:156 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'110, mlcod=11'105)", 2023-11-25T04:23:54.615 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:54.615 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 76.821099950999994, 2023-11-25T04:23:54.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005616116, 2023-11-25T04:23:54.615 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:54.615 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:54.615 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:54.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:54.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:54.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:54.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:54.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:54.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:54.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:54.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785470+0000", 2023-11-25T04:23:54.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0699999999999998e-07 2023-11-25T04:23:54.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:54.617 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785474+0000", 2023-11-25T04:23:54.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.048e-06 2023-11-25T04:23:54.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.617 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:54.617 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785480+0000", 2023-11-25T04:23:54.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7729999999999996e-06 2023-11-25T04:23:54.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.617 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:54.618 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785483+0000", 2023-11-25T04:23:54.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1970000000000001e-06 2023-11-25T04:23:54.618 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.618 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:54.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787936+0000", 2023-11-25T04:23:54.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024529040000000001 2023-11-25T04:23:54.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:54.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787947+0000", 2023-11-25T04:23:54.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1669e-05 2023-11-25T04:23:54.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:54.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791057+0000", 2023-11-25T04:23:54.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031095200000000002 2023-11-25T04:23:54.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:54.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791086+0000", 2023-11-25T04:23:54.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8697999999999999e-05 2023-11-25T04:23:54.620 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:54.620 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:54.620 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:54.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.620 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:157 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'111, mlcod=11'106)", 2023-11-25T04:23:54.620 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:54.621 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 76.820507393, 2023-11-25T04:23:54.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050590280000000001, 2023-11-25T04:23:54.621 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:54.621 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:54.621 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:54.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:54.621 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:54.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:54.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:54.622 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:54.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:54.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.622 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:54.622 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786063+0000", 2023-11-25T04:23:54.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2600000000000002e-07 2023-11-25T04:23:54.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.622 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:54.622 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786069+0000", 2023-11-25T04:23:54.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8839999999999999e-06 2023-11-25T04:23:54.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.623 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:54.623 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786079+0000", 2023-11-25T04:23:54.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0124e-05 2023-11-25T04:23:54.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.623 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:54.623 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786085+0000", 2023-11-25T04:23:54.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6660000000000002e-06 2023-11-25T04:23:54.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:54.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788160+0000", 2023-11-25T04:23:54.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020747669999999999 2023-11-25T04:23:54.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:54.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788180+0000", 2023-11-25T04:23:54.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0208999999999999e-05 2023-11-25T04:23:54.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:54.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791087+0000", 2023-11-25T04:23:54.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029064630000000002 2023-11-25T04:23:54.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:54.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791121+0000", 2023-11-25T04:23:54.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4289000000000003e-05 2023-11-25T04:23:54.625 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:54.625 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:54.625 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:54.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.625 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:158 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'112, mlcod=11'106)", 2023-11-25T04:23:54.625 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:54.626 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 76.820091000000005, 2023-11-25T04:23:54.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046663030000000001, 2023-11-25T04:23:54.626 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:54.626 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:54.626 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:54.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:54.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:54.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:54.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.627 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:54.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:54.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:54.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.627 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:54.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786479+0000", 2023-11-25T04:23:54.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2799999999999996e-07 2023-11-25T04:23:54.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.627 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:54.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786498+0000", 2023-11-25T04:23:54.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9069e-05 2023-11-25T04:23:54.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:54.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786508+0000", 2023-11-25T04:23:54.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5810000000000003e-06 2023-11-25T04:23:54.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:54.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786512+0000", 2023-11-25T04:23:54.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4889999999999997e-06 2023-11-25T04:23:54.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:54.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788438+0000", 2023-11-25T04:23:54.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019257059999999999 2023-11-25T04:23:54.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:54.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788449+0000", 2023-11-25T04:23:54.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1419e-05 2023-11-25T04:23:54.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:54.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791122+0000", 2023-11-25T04:23:54.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002672822 2023-11-25T04:23:54.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:54.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791145+0000", 2023-11-25T04:23:54.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2589e-05 2023-11-25T04:23:54.630 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:54.630 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:54.630 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:54.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.631 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:206 1.0 e12/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 12'152, mlcod=12'148)", 2023-11-25T04:23:54.631 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:54.631 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 76.308322529999998, 2023-11-25T04:23:54.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046728189999999999, 2023-11-25T04:23:54.631 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:54.631 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:54.631 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:54.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.631 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:54.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:54.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:54.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:54.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:54.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:54.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:54.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298248+0000", 2023-11-25T04:23:54.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3600000000000003e-07 2023-11-25T04:23:54.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:54.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298278+0000", 2023-11-25T04:23:54.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0474e-05 2023-11-25T04:23:54.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:54.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298288+0000", 2023-11-25T04:23:54.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0156000000000001e-05 2023-11-25T04:23:54.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:54.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298293+0000", 2023-11-25T04:23:54.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7400000000000004e-06 2023-11-25T04:23:54.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:54.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298333+0000", 2023-11-25T04:23:54.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9666000000000003e-05 2023-11-25T04:23:54.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:54.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298351+0000", 2023-11-25T04:23:54.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8544999999999999e-05 2023-11-25T04:23:54.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:54.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302891+0000", 2023-11-25T04:23:54.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045398519999999996 2023-11-25T04:23:54.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:54.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302920+0000", 2023-11-25T04:23:54.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8750000000000001e-05 2023-11-25T04:23:54.635 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:54.635 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:54.636 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:54.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.636 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:14 3.15 3:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:54.636 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:54.636 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 67.573188173000005, 2023-11-25T04:23:54.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010990437, 2023-11-25T04:23:54.636 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:54.636 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:54.636 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:54.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:54.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:54.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:54.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:54.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:54.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:54.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:54.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033383+0000", 2023-11-25T04:23:54.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9870000000000002e-06 2023-11-25T04:23:54.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:54.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033398+0000", 2023-11-25T04:23:54.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4569999999999999e-05 2023-11-25T04:23:54.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:54.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033411+0000", 2023-11-25T04:23:54.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3543999999999999e-05 2023-11-25T04:23:54.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:54.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033423+0000", 2023-11-25T04:23:54.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1905999999999999e-05 2023-11-25T04:23:54.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:54.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034433+0000", 2023-11-25T04:23:54.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00101022 2023-11-25T04:23:54.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:54.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034539+0000", 2023-11-25T04:23:54.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010597199999999999 2023-11-25T04:23:54.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:54.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034667+0000", 2023-11-25T04:23:54.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012765500000000001 2023-11-25T04:23:54.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:54.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037803+0000", 2023-11-25T04:23:54.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031362220000000001 2023-11-25T04:23:54.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:54.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:23:54.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0064964669999999997 2023-11-25T04:23:54.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:54.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044331+0000", 2023-11-25T04:23:54.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1544999999999997e-05 2023-11-25T04:23:54.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:54.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044372+0000", 2023-11-25T04:23:54.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0349000000000003e-05 2023-11-25T04:23:54.642 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:54.642 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:54.642 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:54.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.642 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:15 3.15 3:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:54.642 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:54.642 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 67.572528523000003, 2023-11-25T04:23:54.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010464362, 2023-11-25T04:23:54.642 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:54.642 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:54.643 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:54.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:54.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:54.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:54.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:54.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:54.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:54.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:54.644 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034042+0000", 2023-11-25T04:23:54.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0999999999999997e-07 2023-11-25T04:23:54.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.644 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:54.644 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034048+0000", 2023-11-25T04:23:54.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0390000000000001e-06 2023-11-25T04:23:54.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.644 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:54.644 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034059+0000", 2023-11-25T04:23:54.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1182999999999999e-05 2023-11-25T04:23:54.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:54.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034067+0000", 2023-11-25T04:23:54.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1559999999999995e-06 2023-11-25T04:23:54.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:54.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035005+0000", 2023-11-25T04:23:54.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00093747399999999999 2023-11-25T04:23:54.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:54.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035077+0000", 2023-11-25T04:23:54.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2747e-05 2023-11-25T04:23:54.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:54.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035219+0000", 2023-11-25T04:23:54.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000141877 2023-11-25T04:23:54.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:54.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037810+0000", 2023-11-25T04:23:54.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002590752 2023-11-25T04:23:54.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.647 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:54.647 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044437+0000", 2023-11-25T04:23:54.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066270110000000004 2023-11-25T04:23:54.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.647 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:54.647 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044481+0000", 2023-11-25T04:23:54.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4106000000000001e-05 2023-11-25T04:23:54.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.648 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:54.648 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044505+0000", 2023-11-25T04:23:54.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4207e-05 2023-11-25T04:23:54.648 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:54.648 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:54.648 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:54.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.648 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:16 3.1d 3:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:54.648 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:54.648 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 67.572486674999993, 2023-11-25T04:23:54.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010413923, 2023-11-25T04:23:54.649 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:54.649 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:54.649 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:54.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.649 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:54.649 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:54.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:54.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.649 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:54.649 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:54.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:54.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:54.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:54.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4499999999999997e-07 2023-11-25T04:23:54.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:54.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034087+0000", 2023-11-25T04:23:54.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9770000000000002e-06 2023-11-25T04:23:54.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:54.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034094+0000", 2023-11-25T04:23:54.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8809999999999998e-06 2023-11-25T04:23:54.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:54.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034100+0000", 2023-11-25T04:23:54.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8440000000000003e-06 2023-11-25T04:23:54.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:54.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034993+0000", 2023-11-25T04:23:54.652 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00089336799999999996 2023-11-25T04:23:54.652 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.652 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.652 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:54.652 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035099+0000", 2023-11-25T04:23:54.652 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106037 2023-11-25T04:23:54.652 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.652 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.652 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:54.652 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035276+0000", 2023-11-25T04:23:54.652 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017631799999999999 2023-11-25T04:23:54.652 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.653 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:54.653 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037815+0000", 2023-11-25T04:23:54.653 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025395959999999999 2023-11-25T04:23:54.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.653 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:54.653 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044420+0000", 2023-11-25T04:23:54.653 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066045749999999997 2023-11-25T04:23:54.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.653 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:54.653 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044463+0000", 2023-11-25T04:23:54.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3205999999999999e-05 2023-11-25T04:23:54.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:54.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044497+0000", 2023-11-25T04:23:54.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3476000000000003e-05 2023-11-25T04:23:54.654 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:54.654 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:54.654 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:54.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.654 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:17 3.1d 3:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:54.655 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:54.655 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 67.572461465999993, 2023-11-25T04:23:54.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01127356, 2023-11-25T04:23:54.655 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:54.655 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:54.655 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:54.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:54.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:54.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:54.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.656 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:54.656 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:54.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:54.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.656 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:54.656 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:54.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1100000000000001e-07 2023-11-25T04:23:54.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.656 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:54.656 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034112+0000", 2023-11-25T04:23:54.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8600000000000003e-06 2023-11-25T04:23:54.657 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.657 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.657 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:54.657 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034120+0000", 2023-11-25T04:23:54.657 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7700000000000001e-06 2023-11-25T04:23:54.657 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.657 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.657 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:54.657 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034127+0000", 2023-11-25T04:23:54.657 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0999999999999998e-06 2023-11-25T04:23:54.657 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.657 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.658 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:54.658 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035601+0000", 2023-11-25T04:23:54.658 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014739410000000001 2023-11-25T04:23:54.658 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.658 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.658 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:54.658 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035685+0000", 2023-11-25T04:23:54.658 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3677999999999994e-05 2023-11-25T04:23:54.658 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.658 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.658 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:54.659 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035827+0000", 2023-11-25T04:23:54.659 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014232599999999999 2023-11-25T04:23:54.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.659 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.659 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:54.659 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038725+0000", 2023-11-25T04:23:54.659 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028982779999999998 2023-11-25T04:23:54.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.659 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.659 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:54.659 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045323+0000", 2023-11-25T04:23:54.659 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0065980780000000003 2023-11-25T04:23:54.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.660 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.660 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:54.660 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045354+0000", 2023-11-25T04:23:54.660 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0463e-05 2023-11-25T04:23:54.660 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.660 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.660 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:54.660 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045381+0000", 2023-11-25T04:23:54.660 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7654999999999999e-05 2023-11-25T04:23:54.660 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:54.660 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:54.660 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:54.661 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.661 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.661 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:22 3.e 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:54.661 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:54.661 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 67.572435053000007, 2023-11-25T04:23:54.661 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010133422, 2023-11-25T04:23:54.661 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:54.661 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:54.661 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:54.661 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.661 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:54.661 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:54.662 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:54.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.662 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.662 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:54.662 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:54.662 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:54.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.662 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.662 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:54.662 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034135+0000", 2023-11-25T04:23:54.662 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-07 2023-11-25T04:23:54.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:54.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034138+0000", 2023-11-25T04:23:54.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6009999999999999e-06 2023-11-25T04:23:54.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:54.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034145+0000", 2023-11-25T04:23:54.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1169999999999998e-06 2023-11-25T04:23:54.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:54.664 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034150+0000", 2023-11-25T04:23:54.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9080000000000003e-06 2023-11-25T04:23:54.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:54.664 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034209+0000", 2023-11-25T04:23:54.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8149000000000002e-05 2023-11-25T04:23:54.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:54.664 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034415+0000", 2023-11-25T04:23:54.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020637699999999999 2023-11-25T04:23:54.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.665 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.665 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:54.665 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034584+0000", 2023-11-25T04:23:54.665 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016870399999999999 2023-11-25T04:23:54.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.665 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.665 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:54.665 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.036979+0000", 2023-11-25T04:23:54.665 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023957509999999998 2023-11-25T04:23:54.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.665 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.665 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:54.665 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044161+0000", 2023-11-25T04:23:54.666 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0071811590000000003 2023-11-25T04:23:54.666 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.666 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.666 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:54.666 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044224+0000", 2023-11-25T04:23:54.666 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3454000000000006e-05 2023-11-25T04:23:54.666 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.666 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.666 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:54.666 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044268+0000", 2023-11-25T04:23:54.666 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3761999999999999e-05 2023-11-25T04:23:54.667 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:54.667 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:54.667 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:54.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.667 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:23 3.e 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:54.667 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:54.667 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 67.572403644000005, 2023-11-25T04:23:54.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010281676, 2023-11-25T04:23:54.667 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:54.667 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:54.667 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:54.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.668 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:54.668 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:54.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:54.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.668 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:54.668 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:54.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:54.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.668 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:54.668 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:54.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5799999999999999e-07 2023-11-25T04:23:54.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.669 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:54.669 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034169+0000", 2023-11-25T04:23:54.669 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1879999999999998e-06 2023-11-25T04:23:54.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.669 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:54.669 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034177+0000", 2023-11-25T04:23:54.669 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7020000000000001e-06 2023-11-25T04:23:54.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.670 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:54.670 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034184+0000", 2023-11-25T04:23:54.670 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3910000000000003e-06 2023-11-25T04:23:54.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.670 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:54.670 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034924+0000", 2023-11-25T04:23:54.670 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00074032999999999998 2023-11-25T04:23:54.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.670 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:54.670 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034991+0000", 2023-11-25T04:23:54.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7347000000000004e-05 2023-11-25T04:23:54.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:54.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035096+0000", 2023-11-25T04:23:54.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010488 2023-11-25T04:23:54.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:54.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037588+0000", 2023-11-25T04:23:54.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024918309999999999 2023-11-25T04:23:54.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.672 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:54.672 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044367+0000", 2023-11-25T04:23:54.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067787589999999997 2023-11-25T04:23:54.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.672 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:54.672 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044415+0000", 2023-11-25T04:23:54.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7803000000000003e-05 2023-11-25T04:23:54.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.672 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:54.672 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044447+0000", 2023-11-25T04:23:54.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2886999999999999e-05 2023-11-25T04:23:54.673 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:54.673 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:54.673 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:54.673 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.673 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.673 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:30 3.15 3:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:54.673 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:54.673 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 67.572345204000001, 2023-11-25T04:23:54.673 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010365070000000001, 2023-11-25T04:23:54.673 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:54.673 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:54.673 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:54.674 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.674 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:54.674 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:54.674 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:54.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.674 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.674 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:54.674 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:54.674 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:54.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.674 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.674 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:54.674 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034225+0000", 2023-11-25T04:23:54.675 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7400000000000002e-07 2023-11-25T04:23:54.675 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.675 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.675 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:54.675 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034229+0000", 2023-11-25T04:23:54.675 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3100000000000002e-06 2023-11-25T04:23:54.675 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.675 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.675 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:54.675 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034239+0000", 2023-11-25T04:23:54.675 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9140000000000003e-06 2023-11-25T04:23:54.675 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.676 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:54.676 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034246+0000", 2023-11-25T04:23:54.676 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0149999999999998e-06 2023-11-25T04:23:54.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.676 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:54.676 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035572+0000", 2023-11-25T04:23:54.676 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001325655 2023-11-25T04:23:54.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.676 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:54.676 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035730+0000", 2023-11-25T04:23:54.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015827899999999999 2023-11-25T04:23:54.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.677 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:54.677 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035857+0000", 2023-11-25T04:23:54.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012644699999999999 2023-11-25T04:23:54.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.677 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:54.677 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038737+0000", 2023-11-25T04:23:54.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002880077 2023-11-25T04:23:54.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.678 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.678 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:54.678 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044540+0000", 2023-11-25T04:23:54.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058034239999999997 2023-11-25T04:23:54.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.678 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.678 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:54.678 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044569+0000", 2023-11-25T04:23:54.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8388999999999999e-05 2023-11-25T04:23:54.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.678 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.678 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:54.678 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044589+0000", 2023-11-25T04:23:54.679 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0686000000000002e-05 2023-11-25T04:23:54.679 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:54.679 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:54.679 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:54.679 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.679 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.679 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:33 3.1b 3:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:54.679 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:54.679 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 67.572299705999995, 2023-11-25T04:23:54.679 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099754049999999997, 2023-11-25T04:23:54.679 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:54.679 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:23:54.680 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:54.680 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.680 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:54.680 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:54.680 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:54.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.680 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.680 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:54.680 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:54.680 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:54.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.680 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.680 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:54.681 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:54.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4799999999999998e-07 2023-11-25T04:23:54.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.681 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:54.681 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034273+0000", 2023-11-25T04:23:54.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7819999999999998e-06 2023-11-25T04:23:54.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.681 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:54.681 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034285+0000", 2023-11-25T04:23:54.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2058e-05 2023-11-25T04:23:54.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:54.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034291+0000", 2023-11-25T04:23:54.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0680000000000002e-06 2023-11-25T04:23:54.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:54.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034370+0000", 2023-11-25T04:23:54.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8452e-05 2023-11-25T04:23:54.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:54.683 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034477+0000", 2023-11-25T04:23:54.683 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107471 2023-11-25T04:23:54.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.683 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:54.683 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044245+0000", 2023-11-25T04:23:54.683 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0097679259999999997 2023-11-25T04:23:54.683 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:54.683 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:54.683 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:54.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.683 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:36 3.1a 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:54.684 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:54.684 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 67.572253032999996, 2023-11-25T04:23:54.684 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058179099999999999, 2023-11-25T04:23:54.684 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:54.684 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:23:54.684 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:54.684 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.684 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:54.684 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:54.684 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:54.684 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.684 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.685 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:54.685 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:54.685 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:54.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.685 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.685 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:54.685 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034317+0000", 2023-11-25T04:23:54.685 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1999999999999999e-07 2023-11-25T04:23:54.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.685 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.685 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:54.685 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034321+0000", 2023-11-25T04:23:54.685 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.233e-06 2023-11-25T04:23:54.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:54.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034337+0000", 2023-11-25T04:23:54.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5308999999999999e-05 2023-11-25T04:23:54.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:54.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034354+0000", 2023-11-25T04:23:54.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7411999999999999e-05 2023-11-25T04:23:54.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.687 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:54.687 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034465+0000", 2023-11-25T04:23:54.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011154699999999999 2023-11-25T04:23:54.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.687 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:54.687 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034560+0000", 2023-11-25T04:23:54.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4186999999999998e-05 2023-11-25T04:23:54.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.687 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:54.687 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.040134+0000", 2023-11-25T04:23:54.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055746019999999997 2023-11-25T04:23:54.688 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:54.688 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:54.688 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:54.688 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.688 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.688 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:9 3.10 e18/16 3:09401a81:::601.00000000:head v 18'4, mlcod=0'0)", 2023-11-25T04:23:54.688 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:54.688 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 67.563358725000001, 2023-11-25T04:23:54.688 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061641020000000003, 2023-11-25T04:23:54.688 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:54.688 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:54.688 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:54.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.689 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:54.689 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:54.689 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:54.689 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.689 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:54.689 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:54.689 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:54.689 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.689 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:54.689 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:54.690 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5300000000000001e-07 2023-11-25T04:23:54.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.690 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.690 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:54.690 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043222+0000", 2023-11-25T04:23:54.690 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.082e-05 2023-11-25T04:23:54.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.690 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.690 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:54.690 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043228+0000", 2023-11-25T04:23:54.690 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6699999999999997e-06 2023-11-25T04:23:54.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.691 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:54.691 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043236+0000", 2023-11-25T04:23:54.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8189999999999992e-06 2023-11-25T04:23:54.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.691 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:54.691 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043507+0000", 2023-11-25T04:23:54.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000271012 2023-11-25T04:23:54.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.691 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:54.691 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043530+0000", 2023-11-25T04:23:54.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2659000000000001e-05 2023-11-25T04:23:54.692 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.692 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.692 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:54.692 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049321+0000", 2023-11-25T04:23:54.692 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057915129999999999 2023-11-25T04:23:54.692 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.692 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.692 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:54.692 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049375+0000", 2023-11-25T04:23:54.692 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3455999999999998e-05 2023-11-25T04:23:54.692 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:54.692 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:54.693 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:54.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.693 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:1 3.14 e18/16 3:292cf221:::200.00000000:head v 18'3, mlcod=0'0)", 2023-11-25T04:23:54.693 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:54.693 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 67.563322012, 2023-11-25T04:23:54.693 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0060983599999999997, 2023-11-25T04:23:54.693 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:54.693 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:54.693 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:54.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.693 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:54.693 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:54.694 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:54.694 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.694 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.694 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:54.694 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:54.694 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:54.694 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.694 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.694 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:54.694 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043248+0000", 2023-11-25T04:23:54.694 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9399999999999995e-07 2023-11-25T04:23:54.694 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.695 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.695 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:54.695 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043252+0000", 2023-11-25T04:23:54.695 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1799999999999998e-06 2023-11-25T04:23:54.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.695 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.695 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:54.695 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043259+0000", 2023-11-25T04:23:54.695 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0049999999999998e-06 2023-11-25T04:23:54.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.695 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.695 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:54.696 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043265+0000", 2023-11-25T04:23:54.696 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0859999999999998e-06 2023-11-25T04:23:54.696 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.696 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.696 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:54.696 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043311+0000", 2023-11-25T04:23:54.696 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6082999999999999e-05 2023-11-25T04:23:54.696 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.696 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.696 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:54.696 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043330+0000", 2023-11-25T04:23:54.696 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8556e-05 2023-11-25T04:23:54.696 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.697 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:54.697 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049303+0000", 2023-11-25T04:23:54.697 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059733470000000004 2023-11-25T04:23:54.697 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.697 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:54.697 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049346+0000", 2023-11-25T04:23:54.697 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2608999999999998e-05 2023-11-25T04:23:54.697 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:54.697 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:54.697 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:54.698 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.698 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.698 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:3 3.14 e18/16 3:287034c4:::1.00000000.inode:head v 18'4, mlcod=0'0)", 2023-11-25T04:23:54.698 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:54.698 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 67.562805506000004, 2023-11-25T04:23:54.698 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057040090000000003, 2023-11-25T04:23:54.698 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:54.698 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:54.698 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:54.698 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.698 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:54.698 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:54.699 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:54.699 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.699 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.699 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:54.699 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:54.699 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:54.699 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.699 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.699 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:54.699 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:54.699 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9799999999999999e-07 2023-11-25T04:23:54.699 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.700 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.700 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:54.700 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043785+0000", 2023-11-25T04:23:54.700 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0721999999999999e-05 2023-11-25T04:23:54.700 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.700 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.700 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:54.700 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043793+0000", 2023-11-25T04:23:54.700 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5839999999999995e-06 2023-11-25T04:23:54.700 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.700 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.700 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:54.700 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043803+0000", 2023-11-25T04:23:54.701 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0729999999999994e-06 2023-11-25T04:23:54.701 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.701 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.701 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:54.701 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043843+0000", 2023-11-25T04:23:54.701 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0747999999999997e-05 2023-11-25T04:23:54.701 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.701 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.701 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:54.701 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043856+0000", 2023-11-25T04:23:54.701 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2697000000000001e-05 2023-11-25T04:23:54.701 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.702 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:54.702 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049426+0000", 2023-11-25T04:23:54.702 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055704409999999998 2023-11-25T04:23:54.702 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.702 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:54.702 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049468+0000", 2023-11-25T04:23:54.702 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1446000000000003e-05 2023-11-25T04:23:54.702 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:54.702 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:54.702 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:54.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.703 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "pg_update_log_missing(3.14 epoch 18/16 rep_tid 171 entries 18'5 (0'0) error 3:2e2fa760:::200.00000001:head by mds.0.3:32 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:23:54.703 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:54.703 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 67.562644543000005, 2023-11-25T04:23:54.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005556938, 2023-11-25T04:23:54.703 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:54.703 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2023-11-25T04:23:54.703 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:54.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.703 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:54.703 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:54.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:54.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.704 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.704 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:54.704 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:54.704 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:54.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.704 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.704 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:54.704 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:54.704 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.03e-07 2023-11-25T04:23:54.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.704 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.705 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:54.705 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043931+0000", 2023-11-25T04:23:54.705 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1079999999999998e-06 2023-11-25T04:23:54.705 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.705 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.705 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:54.705 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043944+0000", 2023-11-25T04:23:54.705 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2852e-05 2023-11-25T04:23:54.705 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.705 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.705 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:54.705 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043955+0000", 2023-11-25T04:23:54.706 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1037e-05 2023-11-25T04:23:54.706 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.706 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.706 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:54.706 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044088+0000", 2023-11-25T04:23:54.706 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013269599999999999 2023-11-25T04:23:54.706 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.706 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.706 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:54.706 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049482+0000", 2023-11-25T04:23:54.706 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053940419999999999 2023-11-25T04:23:54.706 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:54.707 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:54.707 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:54.707 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.707 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.707 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:10 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'5, mlcod=0'0)", 2023-11-25T04:23:54.707 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:54.707 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 67.562554515000002, 2023-11-25T04:23:54.707 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054493099999999997, 2023-11-25T04:23:54.707 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:54.707 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:54.707 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:54.707 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.707 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:54.708 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:54.708 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:54.708 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.708 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.708 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:54.708 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:54.708 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:54.708 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.708 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.708 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:54.708 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:54.708 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6100000000000002e-07 2023-11-25T04:23:54.708 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.709 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.709 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:54.709 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044019+0000", 2023-11-25T04:23:54.709 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.839e-06 2023-11-25T04:23:54.709 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.709 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.709 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:54.709 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044026+0000", 2023-11-25T04:23:54.709 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5869999999999998e-06 2023-11-25T04:23:54.709 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.709 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.709 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:54.709 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044031+0000", 2023-11-25T04:23:54.710 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4909999999999996e-06 2023-11-25T04:23:54.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.710 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.710 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:54.710 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044070+0000", 2023-11-25T04:23:54.710 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8829999999999999e-05 2023-11-25T04:23:54.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.710 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.710 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:54.710 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044081+0000", 2023-11-25T04:23:54.710 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1447999999999999e-05 2023-11-25T04:23:54.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.711 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:54.711 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049406+0000", 2023-11-25T04:23:54.711 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053246480000000004 2023-11-25T04:23:54.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.711 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:54.711 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049464+0000", 2023-11-25T04:23:54.711 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8205999999999998e-05 2023-11-25T04:23:54.711 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:54.711 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:54.711 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:54.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.712 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.712 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:11 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'6, mlcod=0'0)", 2023-11-25T04:23:54.712 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:54.712 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 67.562353251000005, 2023-11-25T04:23:54.712 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052673499999999996, 2023-11-25T04:23:54.712 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:54.712 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:54.712 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:54.712 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.712 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:54.712 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:54.712 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:54.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.713 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:54.713 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:54.713 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:54.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.713 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:54.713 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:54.713 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0199999999999998e-07 2023-11-25T04:23:54.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.713 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:54.714 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044220+0000", 2023-11-25T04:23:54.714 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7720000000000001e-06 2023-11-25T04:23:54.714 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.714 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.714 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:54.714 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044226+0000", 2023-11-25T04:23:54.714 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8579999999999996e-06 2023-11-25T04:23:54.714 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.714 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.714 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:54.714 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044232+0000", 2023-11-25T04:23:54.714 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1700000000000002e-06 2023-11-25T04:23:54.715 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.715 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.715 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:54.715 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044274+0000", 2023-11-25T04:23:54.715 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1254999999999997e-05 2023-11-25T04:23:54.715 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.715 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.715 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:54.715 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:23:54.715 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6880999999999999e-05 2023-11-25T04:23:54.715 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.715 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.716 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:54.716 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049466+0000", 2023-11-25T04:23:54.716 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051659219999999999 2023-11-25T04:23:54.716 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:54.716 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:54.716 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:54.716 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049483+0000", 2023-11-25T04:23:54.716 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.719e-05 2023-11-25T04:23:54.716 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:54.716 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:54.716 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:54.716 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:54.716 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:54.717 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:54.718 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:23:54.718 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:54.716+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:54.818 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:54.818+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:54.827 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:54.827 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:23:54.827 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2023-11-25T04:23:54.827 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:54.836 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:23:54.919 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:54.918+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:54.941 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:54.941 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:23:54.941 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2023-11-25T04:23:54.941 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2023-11-25T04:23:54.941 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:54.951 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:23:55.000 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2023-11-25T04:23:55.035 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:55.034+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:55.100 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:55.100 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2023-11-25T04:23:55.100 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2023-11-25T04:23:55.100 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2023-11-25T04:23:55.100 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.100 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:55.101 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:55.101 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 68.066008529000001, 2023-11-25T04:23:55.101 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01072943, 2023-11-25T04:23:55.101 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:55.101 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:55.101 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:55.101 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.101 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:55.101 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:55.101 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:55.101 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.101 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.101 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:55.102 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:55.102 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:55.102 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.102 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.102 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:55.102 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031829+0000", 2023-11-25T04:23:55.102 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.666e-06 2023-11-25T04:23:55.102 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.102 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.102 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:55.102 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031839+0000", 2023-11-25T04:23:55.102 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9490000000000008e-06 2023-11-25T04:23:55.103 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.103 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.103 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:55.103 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031857+0000", 2023-11-25T04:23:55.103 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8093000000000001e-05 2023-11-25T04:23:55.103 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.103 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.103 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:55.103 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031882+0000", 2023-11-25T04:23:55.103 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4294000000000002e-05 2023-11-25T04:23:55.103 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.103 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.104 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:55.104 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033130+0000", 2023-11-25T04:23:55.104 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001248264 2023-11-25T04:23:55.104 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.104 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.104 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:55.104 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033426+0000", 2023-11-25T04:23:55.104 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029620999999999998 2023-11-25T04:23:55.104 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.104 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.105 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:55.105 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033814+0000", 2023-11-25T04:23:55.105 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000387742 2023-11-25T04:23:55.105 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.105 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.105 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:55.105 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036793+0000", 2023-11-25T04:23:55.105 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002979598 2023-11-25T04:23:55.105 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.105 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.105 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:55.105 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042455+0000", 2023-11-25T04:23:55.106 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0056619260000000003 2023-11-25T04:23:55.106 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.106 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.106 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:55.106 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042502+0000", 2023-11-25T04:23:55.106 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7139999999999999e-05 2023-11-25T04:23:55.106 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.106 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.106 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:55.106 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042557+0000", 2023-11-25T04:23:55.106 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4548e-05 2023-11-25T04:23:55.107 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:55.107 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:55.107 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:55.107 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.107 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.107 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:55.107 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:55.107 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 68.064809994000001, 2023-11-25T04:23:55.107 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011505202000000001, 2023-11-25T04:23:55.107 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:55.107 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:55.107 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:55.108 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.108 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:55.108 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:55.108 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:55.108 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.108 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.108 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:55.108 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:55.108 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:55.108 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.108 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.108 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:55.108 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033027+0000", 2023-11-25T04:23:55.109 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0699999999999999e-06 2023-11-25T04:23:55.109 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.109 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.109 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:55.109 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033035+0000", 2023-11-25T04:23:55.109 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0830000000000004e-06 2023-11-25T04:23:55.109 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.109 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.109 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:55.109 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033059+0000", 2023-11-25T04:23:55.109 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3975e-05 2023-11-25T04:23:55.109 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.110 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.110 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:55.110 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033482+0000", 2023-11-25T04:23:55.110 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042253700000000001 2023-11-25T04:23:55.110 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.110 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.110 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:55.110 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034520+0000", 2023-11-25T04:23:55.110 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001037991 2023-11-25T04:23:55.110 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.110 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.110 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:55.111 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034611+0000", 2023-11-25T04:23:55.111 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1113999999999995e-05 2023-11-25T04:23:55.111 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.111 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.111 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:55.111 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034769+0000", 2023-11-25T04:23:55.111 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000157641 2023-11-25T04:23:55.111 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.111 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.111 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:55.111 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038292+0000", 2023-11-25T04:23:55.111 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035231160000000002 2023-11-25T04:23:55.112 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.112 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.112 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:55.112 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044440+0000", 2023-11-25T04:23:55.112 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0061486980000000002 2023-11-25T04:23:55.112 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.112 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.112 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:55.112 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044495+0000", 2023-11-25T04:23:55.112 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4339000000000003e-05 2023-11-25T04:23:55.112 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.112 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.112 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:55.113 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044531+0000", 2023-11-25T04:23:55.113 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6637999999999997e-05 2023-11-25T04:23:55.113 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:55.113 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:55.113 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:55.113 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.113 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.113 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:55.113 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:55.113 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 68.064335639999996, 2023-11-25T04:23:55.113 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0057634039999999997, 2023-11-25T04:23:55.113 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:55.114 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:55.114 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:55.114 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.114 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:55.114 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:55.114 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:55.114 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.114 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.114 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:55.114 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:55.114 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:55.115 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.115 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.115 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:55.115 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:55.115 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7900000000000007e-07 2023-11-25T04:23:55.115 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.115 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.115 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:55.115 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033507+0000", 2023-11-25T04:23:55.115 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8900000000000004e-06 2023-11-25T04:23:55.115 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.115 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.115 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:55.116 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033519+0000", 2023-11-25T04:23:55.116 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1756000000000001e-05 2023-11-25T04:23:55.116 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.116 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.116 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:55.116 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033527+0000", 2023-11-25T04:23:55.116 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3610000000000001e-06 2023-11-25T04:23:55.116 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.116 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.116 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:55.116 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034451+0000", 2023-11-25T04:23:55.116 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00092372400000000001 2023-11-25T04:23:55.117 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.117 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.117 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:55.117 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034809+0000", 2023-11-25T04:23:55.117 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00035757699999999998 2023-11-25T04:23:55.117 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.117 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.117 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:23:55.117 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034965+0000", 2023-11-25T04:23:55.117 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015638699999999999 2023-11-25T04:23:55.117 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.117 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.118 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:55.118 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038299+0000", 2023-11-25T04:23:55.118 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0033336559999999999 2023-11-25T04:23:55.118 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.118 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.118 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:55.118 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039169+0000", 2023-11-25T04:23:55.118 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00087066400000000005 2023-11-25T04:23:55.118 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.118 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.118 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:55.118 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039226+0000", 2023-11-25T04:23:55.119 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6511999999999997e-05 2023-11-25T04:23:55.119 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.119 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.119 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:55.119 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039264+0000", 2023-11-25T04:23:55.119 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7898000000000002e-05 2023-11-25T04:23:55.119 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:55.119 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:55.119 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:55.119 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.119 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.119 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:25 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:55.120 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:55.120 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 68.064257056000002, 2023-11-25T04:23:55.120 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096656169999999996, 2023-11-25T04:23:55.120 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:55.120 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:55.120 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:55.120 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.120 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:55.120 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:55.120 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:55.120 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.120 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.121 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:55.121 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:55.121 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:55.121 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.121 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.121 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:55.121 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033580+0000", 2023-11-25T04:23:55.121 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8199999999999999e-07 2023-11-25T04:23:55.121 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.121 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.121 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:55.121 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033585+0000", 2023-11-25T04:23:55.122 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3680000000000001e-06 2023-11-25T04:23:55.122 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.122 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.122 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:55.122 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033600+0000", 2023-11-25T04:23:55.122 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4331000000000001e-05 2023-11-25T04:23:55.122 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.122 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.122 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:55.122 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033606+0000", 2023-11-25T04:23:55.122 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5780000000000004e-06 2023-11-25T04:23:55.122 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.122 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.123 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:55.123 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035747+0000", 2023-11-25T04:23:55.123 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0021406530000000002 2023-11-25T04:23:55.123 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.123 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.123 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:55.123 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035901+0000", 2023-11-25T04:23:55.123 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000153931 2023-11-25T04:23:55.123 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.123 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.123 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:23:55.123 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036193+0000", 2023-11-25T04:23:55.123 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029211299999999999 2023-11-25T04:23:55.124 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.124 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.124 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:55.124 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038655+0000", 2023-11-25T04:23:55.124 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0024621640000000002 2023-11-25T04:23:55.124 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.124 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.124 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:55.124 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043130+0000", 2023-11-25T04:23:55.124 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.004474698 2023-11-25T04:23:55.125 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.125 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.125 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:55.125 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043204+0000", 2023-11-25T04:23:55.125 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4091999999999995e-05 2023-11-25T04:23:55.125 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.125 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.125 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:55.125 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043245+0000", 2023-11-25T04:23:55.125 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0907e-05 2023-11-25T04:23:55.125 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:55.125 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:55.125 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:55.126 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.126 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.126 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:26 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:55.126 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:55.126 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 68.064221231999994, 2023-11-25T04:23:55.126 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011086945000000001, 2023-11-25T04:23:55.126 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:55.126 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:55.126 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:55.126 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.126 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:55.127 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:55.127 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:55.127 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.127 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.127 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:55.127 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:55.127 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:55.127 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.127 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.127 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:55.129 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:55.130 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1699999999999998e-07 2023-11-25T04:23:55.130 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.130 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.130 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:55.130 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033627+0000", 2023-11-25T04:23:55.130 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.163e-05 2023-11-25T04:23:55.130 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.131 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.131 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:55.131 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033636+0000", 2023-11-25T04:23:55.131 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6859999999999995e-06 2023-11-25T04:23:55.131 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.131 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.131 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:55.131 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033642+0000", 2023-11-25T04:23:55.131 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3149999999999997e-06 2023-11-25T04:23:55.131 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.131 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.131 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:55.131 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035414+0000", 2023-11-25T04:23:55.132 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0017716380000000001 2023-11-25T04:23:55.132 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.132 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.132 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:55.132 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035539+0000", 2023-11-25T04:23:55.132 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000125477 2023-11-25T04:23:55.132 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.132 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.132 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:55.132 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035672+0000", 2023-11-25T04:23:55.132 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000133316 2023-11-25T04:23:55.132 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.133 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.133 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:55.133 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038649+0000", 2023-11-25T04:23:55.133 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029764679999999999 2023-11-25T04:23:55.133 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.133 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.133 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:55.133 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044578+0000", 2023-11-25T04:23:55.133 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0059293510000000002 2023-11-25T04:23:55.133 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.133 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.133 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:55.134 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044651+0000", 2023-11-25T04:23:55.134 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2513000000000002e-05 2023-11-25T04:23:55.134 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.134 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.134 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:55.134 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044702+0000", 2023-11-25T04:23:55.134 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1034e-05 2023-11-25T04:23:55.134 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:55.134 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:55.134 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:55.134 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.135 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.135 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:29 3.b 3:d0630e4c:::mds0_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:55.135 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:55.135 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 68.064183311999997, 2023-11-25T04:23:55.135 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010980227, 2023-11-25T04:23:55.135 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:55.135 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:55.135 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:55.135 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.135 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:55.135 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:55.135 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:55.136 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.136 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.136 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:55.136 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:55.136 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:55.136 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.136 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.136 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:55.136 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:55.136 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3499999999999996e-07 2023-11-25T04:23:55.136 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.136 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.137 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:55.137 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033658+0000", 2023-11-25T04:23:55.137 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1280000000000001e-06 2023-11-25T04:23:55.137 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.137 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.137 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:55.137 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033667+0000", 2023-11-25T04:23:55.137 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1430000000000004e-06 2023-11-25T04:23:55.137 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.137 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.137 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:55.137 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033678+0000", 2023-11-25T04:23:55.138 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0906e-05 2023-11-25T04:23:55.138 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.138 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.138 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:55.138 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034456+0000", 2023-11-25T04:23:55.138 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00077833899999999996 2023-11-25T04:23:55.138 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.138 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.138 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:55.138 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034575+0000", 2023-11-25T04:23:55.139 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011926000000000001 2023-11-25T04:23:55.139 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.139 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.139 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:55.139 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035001+0000", 2023-11-25T04:23:55.139 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042550999999999998 2023-11-25T04:23:55.139 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.139 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.139 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:55.139 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.037923+0000", 2023-11-25T04:23:55.139 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029219670000000001 2023-11-25T04:23:55.139 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.140 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.140 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:55.140 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044538+0000", 2023-11-25T04:23:55.140 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0066152349999999997 2023-11-25T04:23:55.140 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.140 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.141 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:55.141 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044594+0000", 2023-11-25T04:23:55.141 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5593000000000001e-05 2023-11-25T04:23:55.141 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.141 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.141 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:55.141 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044633+0000", 2023-11-25T04:23:55.141 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9511000000000002e-05 2023-11-25T04:23:55.141 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:55.141 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:55.141 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:55.141 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.141 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.142 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:34 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:55.142 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:55.142 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 68.063933199000004, 2023-11-25T04:23:55.142 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010920445000000001, 2023-11-25T04:23:55.142 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:55.142 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:55.142 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:55.142 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.142 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:55.142 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:55.142 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:55.143 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.143 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.143 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:55.143 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:55.143 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:55.143 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.143 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.143 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:55.143 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033904+0000", 2023-11-25T04:23:55.143 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6599999999999994e-07 2023-11-25T04:23:55.143 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.144 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.144 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:55.144 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033909+0000", 2023-11-25T04:23:55.144 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5639999999999996e-06 2023-11-25T04:23:55.144 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.144 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.144 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:55.144 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033923+0000", 2023-11-25T04:23:55.144 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3028e-05 2023-11-25T04:23:55.144 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.144 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.145 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:55.145 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034043+0000", 2023-11-25T04:23:55.145 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000120328 2023-11-25T04:23:55.145 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.145 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.145 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:55.145 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035576+0000", 2023-11-25T04:23:55.145 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015333720000000001 2023-11-25T04:23:55.145 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.145 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.145 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:55.145 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035699+0000", 2023-11-25T04:23:55.146 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000123001 2023-11-25T04:23:55.146 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.146 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.146 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:55.146 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044823+0000", 2023-11-25T04:23:55.146 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0091241859999999994 2023-11-25T04:23:55.146 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:55.146 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:55.146 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:55.146 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.146 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.147 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:35 3.1f 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:55.147 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:55.147 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 68.062628993999994, 2023-11-25T04:23:55.147 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096164779999999995, 2023-11-25T04:23:55.147 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:55.147 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:55.147 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:55.147 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.147 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:55.147 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:55.147 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:55.147 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.148 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.148 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:55.148 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:55.148 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:55.148 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.148 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.148 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:55.148 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035208+0000", 2023-11-25T04:23:55.148 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2699999999999998e-07 2023-11-25T04:23:55.148 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.148 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.148 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:55.148 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035213+0000", 2023-11-25T04:23:55.149 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.6070000000000001e-06 2023-11-25T04:23:55.149 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.149 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.149 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:55.149 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035350+0000", 2023-11-25T04:23:55.149 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013693599999999999 2023-11-25T04:23:55.149 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.149 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.149 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:55.149 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035358+0000", 2023-11-25T04:23:55.149 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0099999999999995e-06 2023-11-25T04:23:55.150 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.150 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.150 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:55.150 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036260+0000", 2023-11-25T04:23:55.150 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00090240100000000005 2023-11-25T04:23:55.150 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.150 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.150 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:55.150 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036517+0000", 2023-11-25T04:23:55.150 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00025736299999999998 2023-11-25T04:23:55.150 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.150 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.151 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:55.151 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044824+0000", 2023-11-25T04:23:55.151 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0083063340000000003 2023-11-25T04:23:55.151 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:55.151 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:55.151 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:55.151 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.151 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.151 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:37 3.1e 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:55.151 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:55.151 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 68.062462307999994, 2023-11-25T04:23:55.151 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0078322640000000002, 2023-11-25T04:23:55.152 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:55.152 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:55.152 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:55.152 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.152 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:55.152 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:55.152 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:55.152 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.152 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.152 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:55.152 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:55.152 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:55.153 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.153 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.153 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:55.153 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035375+0000", 2023-11-25T04:23:55.153 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9100000000000003e-07 2023-11-25T04:23:55.153 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.153 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.153 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:55.153 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035379+0000", 2023-11-25T04:23:55.153 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9430000000000002e-06 2023-11-25T04:23:55.153 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.153 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.153 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:55.154 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035388+0000", 2023-11-25T04:23:55.154 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2760000000000006e-06 2023-11-25T04:23:55.154 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.154 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.154 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:55.154 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035396+0000", 2023-11-25T04:23:55.154 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8480000000000001e-06 2023-11-25T04:23:55.154 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.154 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.154 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:55.154 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035452+0000", 2023-11-25T04:23:55.155 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5831000000000001e-05 2023-11-25T04:23:55.155 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.155 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.155 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:55.155 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035934+0000", 2023-11-25T04:23:55.155 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00048265599999999998 2023-11-25T04:23:55.155 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.155 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.155 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:55.155 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043206+0000", 2023-11-25T04:23:55.155 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.007271919 2023-11-25T04:23:55.155 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:55.156 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:55.156 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:55.156 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.156 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.156 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing(3.1a epoch 18/16 rep_tid 3 entries 18'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:36 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:23:55.156 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:55.156 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 68.061186722000002, 2023-11-25T04:23:55.156 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002073414, 2023-11-25T04:23:55.156 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:55.156 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:55.156 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:55.157 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.157 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:55.157 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:55.157 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:55.157 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.157 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.157 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:55.157 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:55.157 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:55.157 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.157 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.157 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:55.157 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036650+0000", 2023-11-25T04:23:55.158 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7300000000000002e-07 2023-11-25T04:23:55.158 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.158 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.158 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:55.158 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036659+0000", 2023-11-25T04:23:55.158 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.7499999999999992e-06 2023-11-25T04:23:55.158 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.158 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.158 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:55.158 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036673+0000", 2023-11-25T04:23:55.158 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4917e-05 2023-11-25T04:23:55.158 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.159 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.159 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:55.159 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036687+0000", 2023-11-25T04:23:55.159 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3883000000000001e-05 2023-11-25T04:23:55.159 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.159 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.159 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:55.159 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036781+0000", 2023-11-25T04:23:55.159 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3219000000000003e-05 2023-11-25T04:23:55.159 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.159 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.159 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:55.159 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038723+0000", 2023-11-25T04:23:55.160 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019422720000000001 2023-11-25T04:23:55.160 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:55.160 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:55.160 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:55.160 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.160 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.160 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:28 3.13 e18/16 3:cd5a64a3:::100.00000000:head v 18'1, mlcod=0'0)", 2023-11-25T04:23:55.160 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:55.160 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 68.057525564000002, 2023-11-25T04:23:55.160 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001850546, 2023-11-25T04:23:55.160 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:55.160 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:55.161 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:55.161 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.161 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:55.161 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:55.161 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:55.161 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.161 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.161 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:55.161 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:55.161 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:55.161 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.161 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.162 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:55.162 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:55.162 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7800000000000002e-07 2023-11-25T04:23:55.162 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.162 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.162 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:55.162 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040331+0000", 2023-11-25T04:23:55.162 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0449999999999999e-05 2023-11-25T04:23:55.162 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.162 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.162 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:55.162 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040339+0000", 2023-11-25T04:23:55.163 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0709999999999995e-06 2023-11-25T04:23:55.163 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.163 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.163 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:55.163 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040347+0000", 2023-11-25T04:23:55.163 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7079999999999997e-06 2023-11-25T04:23:55.163 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.163 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.163 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:55.163 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040410+0000", 2023-11-25T04:23:55.163 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2453000000000007e-05 2023-11-25T04:23:55.163 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.164 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.164 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:55.164 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040422+0000", 2023-11-25T04:23:55.164 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2429000000000001e-05 2023-11-25T04:23:55.164 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.164 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.164 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:55.164 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042129+0000", 2023-11-25T04:23:55.164 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001707266 2023-11-25T04:23:55.164 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.164 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.164 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:55.165 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042161+0000", 2023-11-25T04:23:55.165 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1791000000000001e-05 2023-11-25T04:23:55.165 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:55.165 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:55.165 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:55.165 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.165 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.165 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:27 3.13 e18/16 3:c91d4a1d:::mds0_inotable:head v 18'2, mlcod=0'0)", 2023-11-25T04:23:55.165 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:55.165 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 68.057322041000006, 2023-11-25T04:23:55.165 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0016886430000000001, 2023-11-25T04:23:55.165 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:55.166 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:55.166 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:55.166 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.166 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:55.166 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:55.166 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:55.166 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.166 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.166 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:55.166 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:55.166 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:55.166 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.166 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.167 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:55.167 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:55.167 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3999999999999997e-07 2023-11-25T04:23:55.167 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.167 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.167 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:55.167 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040520+0000", 2023-11-25T04:23:55.167 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2989999999999996e-06 2023-11-25T04:23:55.167 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.167 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.167 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:55.168 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040527+0000", 2023-11-25T04:23:55.168 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5730000000000002e-06 2023-11-25T04:23:55.168 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.168 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.168 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:55.168 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040535+0000", 2023-11-25T04:23:55.168 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8630000000000008e-06 2023-11-25T04:23:55.168 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.168 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.168 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:55.168 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040683+0000", 2023-11-25T04:23:55.168 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000148002 2023-11-25T04:23:55.168 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.169 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.169 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:55.169 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040700+0000", 2023-11-25T04:23:55.169 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7065e-05 2023-11-25T04:23:55.169 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.169 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.169 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:55.169 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042163+0000", 2023-11-25T04:23:55.169 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001462767 2023-11-25T04:23:55.169 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.169 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.169 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:55.170 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042203+0000", 2023-11-25T04:23:55.170 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9734000000000003e-05 2023-11-25T04:23:55.170 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:55.170 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:55.170 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:55.170 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.170 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.170 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:20 3.1c e18/16 3:3ed09add:::607.00000000:head v 18'1, mlcod=0'0)", 2023-11-25T04:23:55.170 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:55.170 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 68.057287000000002, 2023-11-25T04:23:55.170 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015271, 2023-11-25T04:23:55.170 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:55.171 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:55.171 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:55.171 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.171 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:55.171 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:55.171 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:55.171 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.171 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.171 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:55.171 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:55.171 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:55.171 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.172 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.172 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:55.172 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:55.172 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4799999999999999e-07 2023-11-25T04:23:55.172 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.172 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.172 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:55.172 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040561+0000", 2023-11-25T04:23:55.172 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1800000000000001e-05 2023-11-25T04:23:55.172 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.172 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.172 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:55.173 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040568+0000", 2023-11-25T04:23:55.173 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7730000000000002e-06 2023-11-25T04:23:55.173 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.173 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.173 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:55.173 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040587+0000", 2023-11-25T04:23:55.173 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8691000000000001e-05 2023-11-25T04:23:55.173 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.173 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.173 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:55.173 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040638+0000", 2023-11-25T04:23:55.173 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1068e-05 2023-11-25T04:23:55.174 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.174 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.174 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:55.174 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040670+0000", 2023-11-25T04:23:55.174 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2125999999999997e-05 2023-11-25T04:23:55.174 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.174 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.174 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:55.174 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042032+0000", 2023-11-25T04:23:55.174 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001361821 2023-11-25T04:23:55.174 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.174 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.175 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:55.175 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042076+0000", 2023-11-25T04:23:55.175 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4472999999999999e-05 2023-11-25T04:23:55.175 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:55.175 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:55.175 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:55.175 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.175 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.175 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:21 3.1c e18/16 3:3ed09add:::607.00000000:head v 18'2, mlcod=0'0)", 2023-11-25T04:23:55.175 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:55.175 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 68.057052275000004, 2023-11-25T04:23:55.175 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0013228160000000001, 2023-11-25T04:23:55.175 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:55.176 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:55.176 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:55.176 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.176 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:55.176 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:55.176 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:55.176 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.176 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.176 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:55.176 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:55.176 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:55.176 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.177 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.177 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:55.177 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:55.177 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9299999999999999e-07 2023-11-25T04:23:55.177 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.177 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.177 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:55.177 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040790+0000", 2023-11-25T04:23:55.177 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5770000000000001e-06 2023-11-25T04:23:55.177 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.177 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.177 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:55.178 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040799+0000", 2023-11-25T04:23:55.178 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6439999999999995e-06 2023-11-25T04:23:55.178 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.178 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.178 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:55.178 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040806+0000", 2023-11-25T04:23:55.178 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0180000000000004e-06 2023-11-25T04:23:55.178 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.178 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.178 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:55.178 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040907+0000", 2023-11-25T04:23:55.178 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000100096 2023-11-25T04:23:55.179 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.179 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.179 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:55.179 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040925+0000", 2023-11-25T04:23:55.179 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8536999999999999e-05 2023-11-25T04:23:55.179 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.179 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.179 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:55.179 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042079+0000", 2023-11-25T04:23:55.179 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00115358 2023-11-25T04:23:55.179 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.179 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.180 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:55.180 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042107+0000", 2023-11-25T04:23:55.180 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7971e-05 2023-11-25T04:23:55.180 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:55.180 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:55.180 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:55.180 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.180 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.180 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop_reply(mds.0.3:4 3.1f e18/16 ondisk, result = 0)", 2023-11-25T04:23:55.180 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:55.180 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 68.055712885000005, 2023-11-25T04:23:55.180 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00044092900000000001, 2023-11-25T04:23:55.180 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:55.181 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:55.181 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:55.181 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.181 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:55.181 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:55.181 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:55.181 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.181 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.181 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:55.181 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:55.181 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:55.181 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.182 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.182 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:55.182 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042124+0000", 2023-11-25T04:23:55.182 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1800000000000005e-07 2023-11-25T04:23:55.182 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.182 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.182 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:55.182 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042128+0000", 2023-11-25T04:23:55.182 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2289999999999997e-06 2023-11-25T04:23:55.182 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.182 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.183 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:55.183 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042138+0000", 2023-11-25T04:23:55.183 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9729999999999992e-06 2023-11-25T04:23:55.183 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.183 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.183 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:55.183 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042145+0000", 2023-11-25T04:23:55.183 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.336e-06 2023-11-25T04:23:55.183 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.183 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.183 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:55.183 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042441+0000", 2023-11-25T04:23:55.184 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029650500000000002 2023-11-25T04:23:55.184 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.184 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.184 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:55.184 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042448+0000", 2023-11-25T04:23:55.184 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4100000000000002e-06 2023-11-25T04:23:55.184 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.184 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.184 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:55.184 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042564+0000", 2023-11-25T04:23:55.184 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000115758 2023-11-25T04:23:55.185 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:55.185 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:55.185 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:55.185 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.185 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.185 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing_reply(3.1f epoch 18/16 rep_tid 8 lcod 18'4)", 2023-11-25T04:23:55.185 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:55.185 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 68.053387473000001, 2023-11-25T04:23:55.185 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00038154699999999998, 2023-11-25T04:23:55.185 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:55.185 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:55.186 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:55.186 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.186 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:55.186 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:55.186 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:55.186 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.186 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.186 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:55.186 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:55.186 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:55.186 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.186 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.187 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:55.187 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:55.187 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2700000000000001e-07 2023-11-25T04:23:55.187 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.187 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.187 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:55.187 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044451+0000", 2023-11-25T04:23:55.187 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7460000000000001e-06 2023-11-25T04:23:55.187 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.187 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.187 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:55.188 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044455+0000", 2023-11-25T04:23:55.188 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5809999999999998e-06 2023-11-25T04:23:55.188 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.188 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.188 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:55.188 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044458+0000", 2023-11-25T04:23:55.188 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0170000000000001e-06 2023-11-25T04:23:55.188 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.188 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.188 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:55.188 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044734+0000", 2023-11-25T04:23:55.188 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00027582799999999999 2023-11-25T04:23:55.188 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.189 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.189 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:55.189 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044830+0000", 2023-11-25T04:23:55.189 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6248000000000004e-05 2023-11-25T04:23:55.189 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:55.189 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:55.189 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:55.189 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.189 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.189 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing_reply(3.b epoch 18/16 rep_tid 5 lcod 18'2)", 2023-11-25T04:23:55.189 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:55.189 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 68.053350812000005, 2023-11-25T04:23:55.190 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000344488, 2023-11-25T04:23:55.190 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:55.190 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:55.190 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:55.190 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.190 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:55.190 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:55.190 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:55.190 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.190 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.190 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:55.191 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:55.191 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:55.191 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.191 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.191 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:55.191 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:55.191 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.55e-07 2023-11-25T04:23:55.191 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.191 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.191 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:55.191 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044487+0000", 2023-11-25T04:23:55.192 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2470000000000001e-06 2023-11-25T04:23:55.192 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.192 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.192 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:55.192 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044491+0000", 2023-11-25T04:23:55.192 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9659999999999998e-06 2023-11-25T04:23:55.192 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.192 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.192 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:55.192 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044494+0000", 2023-11-25T04:23:55.192 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2310000000000001e-06 2023-11-25T04:23:55.192 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.193 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.193 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:55.193 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044756+0000", 2023-11-25T04:23:55.193 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00026172800000000003 2023-11-25T04:23:55.193 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.193 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.193 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:55.193 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044830+0000", 2023-11-25T04:23:55.193 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4160999999999997e-05 2023-11-25T04:23:55.193 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:55.193 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:55.193 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:55.194 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.194 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.194 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:39 3.6 3:654134d2:::mds0_openfiles.0:head [omap-clear,omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2023-11-25T04:23:55.194 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:55.194 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 62.993524770000001, 2023-11-25T04:23:55.194 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001788658, 2023-11-25T04:23:55.194 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:55.194 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:55.194 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:55.194 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.194 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:55.194 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:55.195 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:55.195 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.195 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.195 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:55.195 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:55.195 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:55.195 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.195 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.195 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:55.195 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104314+0000", 2023-11-25T04:23:55.195 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3099999999999999e-06 2023-11-25T04:23:55.196 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.196 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.196 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:55.196 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104325+0000", 2023-11-25T04:23:55.196 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1511999999999999e-05 2023-11-25T04:23:55.196 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.196 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.196 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:55.196 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104362+0000", 2023-11-25T04:23:55.196 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6903000000000002e-05 2023-11-25T04:23:55.196 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.196 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.197 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:55.197 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104387+0000", 2023-11-25T04:23:55.197 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4766e-05 2023-11-25T04:23:55.197 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.197 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.197 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:55.197 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104465+0000", 2023-11-25T04:23:55.197 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8119000000000002e-05 2023-11-25T04:23:55.197 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.197 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.197 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:55.197 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104557+0000", 2023-11-25T04:23:55.198 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2496000000000003e-05 2023-11-25T04:23:55.198 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.198 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.198 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:55.198 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104646+0000", 2023-11-25T04:23:55.198 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8818000000000002e-05 2023-11-25T04:23:55.198 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.198 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.198 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:55.198 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105201+0000", 2023-11-25T04:23:55.199 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00055506000000000002 2023-11-25T04:23:55.199 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.199 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.199 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:55.199 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105994+0000", 2023-11-25T04:23:55.199 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00079294700000000001 2023-11-25T04:23:55.199 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.199 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.199 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:55.199 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.106055+0000", 2023-11-25T04:23:55.199 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0566000000000003e-05 2023-11-25T04:23:55.199 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.199 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.200 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:55.200 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.106100+0000", 2023-11-25T04:23:55.200 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5160999999999997e-05 2023-11-25T04:23:55.200 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:55.200 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:55.200 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:55.200 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.200 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.200 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:43 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2023-11-25T04:23:55.200 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:55.200 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.993299774, 2023-11-25T04:23:55.200 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001751166, 2023-11-25T04:23:55.201 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:55.201 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:55.201 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:55.201 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.201 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:55.201 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:55.201 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:55.201 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.201 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.201 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:55.201 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:55.201 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:55.202 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.202 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.202 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:55.202 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104539+0000", 2023-11-25T04:23:55.202 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9529999999999999e-06 2023-11-25T04:23:55.202 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.202 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.202 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:55.202 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104557+0000", 2023-11-25T04:23:55.202 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7464000000000001e-05 2023-11-25T04:23:55.202 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.202 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.203 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:55.203 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104590+0000", 2023-11-25T04:23:55.203 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2966000000000003e-05 2023-11-25T04:23:55.203 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.203 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.203 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:55.203 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104608+0000", 2023-11-25T04:23:55.203 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8193e-05 2023-11-25T04:23:55.203 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.203 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.203 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:55.203 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104767+0000", 2023-11-25T04:23:55.203 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015882 2023-11-25T04:23:55.204 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.204 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.204 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:55.204 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104909+0000", 2023-11-25T04:23:55.204 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014244999999999999 2023-11-25T04:23:55.204 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.204 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.204 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:55.204 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.105059+0000", 2023-11-25T04:23:55.204 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014964399999999999 2023-11-25T04:23:55.204 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.205 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.205 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:55.205 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.105701+0000", 2023-11-25T04:23:55.205 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00064201099999999995 2023-11-25T04:23:55.205 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.205 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.205 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:55.205 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106233+0000", 2023-11-25T04:23:55.205 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00053178500000000001 2023-11-25T04:23:55.205 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.205 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.205 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:55.205 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106265+0000", 2023-11-25T04:23:55.206 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1986000000000002e-05 2023-11-25T04:23:55.206 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.206 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.206 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:55.206 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106288+0000", 2023-11-25T04:23:55.206 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2894000000000001e-05 2023-11-25T04:23:55.206 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:55.206 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:55.206 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:55.206 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.207 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.207 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:45 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e23)", 2023-11-25T04:23:55.207 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:23:55.207 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 42.993101734, 2023-11-25T04:23:55.207 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00098423299999999994, 2023-11-25T04:23:55.207 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:55.207 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:55.207 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:55.207 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.207 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:55.207 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:23:55.207 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:55.208 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.208 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.208 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:55.208 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:23:55.208 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:55.208 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.208 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.208 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:55.208 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104737+0000", 2023-11-25T04:23:55.208 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3309999999999998e-06 2023-11-25T04:23:55.208 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.208 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.209 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:55.209 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104748+0000", 2023-11-25T04:23:55.209 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1080999999999999e-05 2023-11-25T04:23:55.209 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.209 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.209 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:55.209 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104771+0000", 2023-11-25T04:23:55.209 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2707000000000001e-05 2023-11-25T04:23:55.209 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.209 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.209 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:55.209 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104800+0000", 2023-11-25T04:23:55.210 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9595e-05 2023-11-25T04:23:55.210 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.210 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.210 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:55.210 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104890+0000", 2023-11-25T04:23:55.210 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.9808999999999993e-05 2023-11-25T04:23:55.210 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.210 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.210 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:55.210 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105004+0000", 2023-11-25T04:23:55.210 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000114508 2023-11-25T04:23:55.210 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.211 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.211 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:55.211 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105634+0000", 2023-11-25T04:23:55.211 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00062985799999999996 2023-11-25T04:23:55.211 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.211 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.211 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:55.211 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105671+0000", 2023-11-25T04:23:55.211 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6640000000000002e-05 2023-11-25T04:23:55.211 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:55.211 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:55.211 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:55.212 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105719+0000", 2023-11-25T04:23:55.212 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7704000000000003e-05 2023-11-25T04:23:55.212 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:55.212 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:55.212 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:55.212 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:55.212 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:55.212 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:55.214 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:23:55.214 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:55.212+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:55.261 INFO:tasks.workunit.client.0.smithi067.stderr:Note: switching to 'ec928368edcc5fc71884a56fd9fd821b148f4e30'. 2023-11-25T04:23:55.262 INFO:tasks.workunit.client.0.smithi067.stderr: 2023-11-25T04:23:55.262 INFO:tasks.workunit.client.0.smithi067.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2023-11-25T04:23:55.262 INFO:tasks.workunit.client.0.smithi067.stderr:changes and commit them, and you can discard any commits you make in this 2023-11-25T04:23:55.262 INFO:tasks.workunit.client.0.smithi067.stderr:state without impacting any branches by switching back to a branch. 2023-11-25T04:23:55.262 INFO:tasks.workunit.client.0.smithi067.stderr: 2023-11-25T04:23:55.262 INFO:tasks.workunit.client.0.smithi067.stderr:If you want to create a new branch to retain commits you create, you may 2023-11-25T04:23:55.262 INFO:tasks.workunit.client.0.smithi067.stderr:do so (now or later) by using -c with the switch command. Example: 2023-11-25T04:23:55.262 INFO:tasks.workunit.client.0.smithi067.stderr: 2023-11-25T04:23:55.262 INFO:tasks.workunit.client.0.smithi067.stderr: git switch -c 2023-11-25T04:23:55.262 INFO:tasks.workunit.client.0.smithi067.stderr: 2023-11-25T04:23:55.262 INFO:tasks.workunit.client.0.smithi067.stderr:Or undo this operation with: 2023-11-25T04:23:55.262 INFO:tasks.workunit.client.0.smithi067.stderr: 2023-11-25T04:23:55.263 INFO:tasks.workunit.client.0.smithi067.stderr: git switch - 2023-11-25T04:23:55.263 INFO:tasks.workunit.client.0.smithi067.stderr: 2023-11-25T04:23:55.263 INFO:tasks.workunit.client.0.smithi067.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2023-11-25T04:23:55.263 INFO:tasks.workunit.client.0.smithi067.stderr: 2023-11-25T04:23:55.263 INFO:tasks.workunit.client.0.smithi067.stderr:HEAD is now at ec928368edc Merge pull request #52931 from yuvalif/wip-yuval-lua-teuthology 2023-11-25T04:23:55.266 DEBUG:teuthology.orchestra.run.smithi067:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2023-11-25T04:23:55.300 INFO:tasks.workunit.client.0.smithi067.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2023-11-25T04:23:55.302 INFO:tasks.workunit.client.0.smithi067.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-11-25T04:23:55.302 INFO:tasks.workunit.client.0.smithi067.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2023-11-25T04:23:55.315 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:55.313+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:55.325 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:55.325 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:23:55.325 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2023-11-25T04:23:55.326 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:55.334 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:23:55.416 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:55.414+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:55.439 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:55.440 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:23:55.442 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2023-11-25T04:23:55.442 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2023-11-25T04:23:55.442 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:55.448 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:23:55.517 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:55.515+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:55.598 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:55.598 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2023-11-25T04:23:55.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2023-11-25T04:23:55.598 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2023-11-25T04:23:55.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.599 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:155 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'109, mlcod=11'105)", 2023-11-25T04:23:55.599 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:55.599 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 77.811035029999999, 2023-11-25T04:23:55.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061719019999999999, 2023-11-25T04:23:55.599 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:55.599 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:55.599 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:55.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:55.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:55.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:55.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.600 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:55.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:55.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:55.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.600 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:55.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784884+0000", 2023-11-25T04:23:55.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3099999999999999e-07 2023-11-25T04:23:55.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.600 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:55.601 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784942+0000", 2023-11-25T04:23:55.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8570000000000003e-05 2023-11-25T04:23:55.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:55.601 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784949+0000", 2023-11-25T04:23:55.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0310000000000001e-06 2023-11-25T04:23:55.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:55.601 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784953+0000", 2023-11-25T04:23:55.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6320000000000001e-06 2023-11-25T04:23:55.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:55.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784993+0000", 2023-11-25T04:23:55.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9737000000000003e-05 2023-11-25T04:23:55.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:55.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785011+0000", 2023-11-25T04:23:55.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8349999999999999e-05 2023-11-25T04:23:55.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:55.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791024+0000", 2023-11-25T04:23:55.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006012997 2023-11-25T04:23:55.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:55.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791055+0000", 2023-11-25T04:23:55.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1254000000000003e-05 2023-11-25T04:23:55.603 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:55.603 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:55.603 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:55.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.603 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:156 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'110, mlcod=11'105)", 2023-11-25T04:23:55.603 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:55.604 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 77.810448980999993, 2023-11-25T04:23:55.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005616116, 2023-11-25T04:23:55.604 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:55.604 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:55.604 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:55.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:55.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:55.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:55.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:55.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:55.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:55.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.605 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:55.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785470+0000", 2023-11-25T04:23:55.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0699999999999998e-07 2023-11-25T04:23:55.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.605 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:55.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785474+0000", 2023-11-25T04:23:55.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.048e-06 2023-11-25T04:23:55.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:55.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785480+0000", 2023-11-25T04:23:55.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7729999999999996e-06 2023-11-25T04:23:55.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:55.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785483+0000", 2023-11-25T04:23:55.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1970000000000001e-06 2023-11-25T04:23:55.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:55.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787936+0000", 2023-11-25T04:23:55.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024529040000000001 2023-11-25T04:23:55.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:55.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787947+0000", 2023-11-25T04:23:55.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1669e-05 2023-11-25T04:23:55.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:55.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791057+0000", 2023-11-25T04:23:55.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031095200000000002 2023-11-25T04:23:55.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:55.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791086+0000", 2023-11-25T04:23:55.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8697999999999999e-05 2023-11-25T04:23:55.608 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:55.608 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:55.608 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:55.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.608 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:157 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'111, mlcod=11'106)", 2023-11-25T04:23:55.608 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:55.608 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 77.809856422999999, 2023-11-25T04:23:55.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050590280000000001, 2023-11-25T04:23:55.608 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:55.608 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:55.608 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:55.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:55.609 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:55.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:55.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:55.609 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:55.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:55.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:55.609 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786063+0000", 2023-11-25T04:23:55.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2600000000000002e-07 2023-11-25T04:23:55.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.610 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:55.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786069+0000", 2023-11-25T04:23:55.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8839999999999999e-06 2023-11-25T04:23:55.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.610 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:55.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786079+0000", 2023-11-25T04:23:55.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0124e-05 2023-11-25T04:23:55.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.610 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:55.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786085+0000", 2023-11-25T04:23:55.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6660000000000002e-06 2023-11-25T04:23:55.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:55.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788160+0000", 2023-11-25T04:23:55.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020747669999999999 2023-11-25T04:23:55.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:55.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788180+0000", 2023-11-25T04:23:55.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0208999999999999e-05 2023-11-25T04:23:55.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:55.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791087+0000", 2023-11-25T04:23:55.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029064630000000002 2023-11-25T04:23:55.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:55.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791121+0000", 2023-11-25T04:23:55.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4289000000000003e-05 2023-11-25T04:23:55.612 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:55.612 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:55.612 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:55.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.613 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:158 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'112, mlcod=11'106)", 2023-11-25T04:23:55.613 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:55.613 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 77.809440030000005, 2023-11-25T04:23:55.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046663030000000001, 2023-11-25T04:23:55.613 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:55.613 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:55.613 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:55.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.613 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:55.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:55.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:55.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:55.614 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:55.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:55.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:55.614 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786479+0000", 2023-11-25T04:23:55.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2799999999999996e-07 2023-11-25T04:23:55.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:55.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786498+0000", 2023-11-25T04:23:55.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9069e-05 2023-11-25T04:23:55.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:55.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786508+0000", 2023-11-25T04:23:55.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5810000000000003e-06 2023-11-25T04:23:55.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:55.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786512+0000", 2023-11-25T04:23:55.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4889999999999997e-06 2023-11-25T04:23:55.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:55.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788438+0000", 2023-11-25T04:23:55.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019257059999999999 2023-11-25T04:23:55.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:55.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788449+0000", 2023-11-25T04:23:55.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1419e-05 2023-11-25T04:23:55.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.617 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:55.617 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791122+0000", 2023-11-25T04:23:55.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002672822 2023-11-25T04:23:55.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.617 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:55.617 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791145+0000", 2023-11-25T04:23:55.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2589e-05 2023-11-25T04:23:55.617 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:55.617 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:55.617 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:55.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.618 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:206 1.0 e12/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 12'152, mlcod=12'148)", 2023-11-25T04:23:55.618 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:55.618 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 77.297671559999998, 2023-11-25T04:23:55.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046728189999999999, 2023-11-25T04:23:55.618 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:55.618 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:55.618 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:55.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.618 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:55.618 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:55.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:55.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:55.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:55.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:55.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:55.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298248+0000", 2023-11-25T04:23:55.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3600000000000003e-07 2023-11-25T04:23:55.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:55.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298278+0000", 2023-11-25T04:23:55.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0474e-05 2023-11-25T04:23:55.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:55.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298288+0000", 2023-11-25T04:23:55.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0156000000000001e-05 2023-11-25T04:23:55.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:55.621 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298293+0000", 2023-11-25T04:23:55.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7400000000000004e-06 2023-11-25T04:23:55.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:55.621 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298333+0000", 2023-11-25T04:23:55.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9666000000000003e-05 2023-11-25T04:23:55.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:55.622 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298351+0000", 2023-11-25T04:23:55.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8544999999999999e-05 2023-11-25T04:23:55.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.622 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:55.622 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302891+0000", 2023-11-25T04:23:55.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045398519999999996 2023-11-25T04:23:55.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.622 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:55.622 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302920+0000", 2023-11-25T04:23:55.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8750000000000001e-05 2023-11-25T04:23:55.623 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:55.623 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:55.623 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:55.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.623 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:14 3.15 3:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:55.623 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:55.623 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 68.562537203000005, 2023-11-25T04:23:55.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010990437, 2023-11-25T04:23:55.623 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:55.623 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:55.624 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:55.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:55.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:55.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:55.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:55.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:55.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:55.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:55.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033383+0000", 2023-11-25T04:23:55.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9870000000000002e-06 2023-11-25T04:23:55.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:55.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033398+0000", 2023-11-25T04:23:55.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4569999999999999e-05 2023-11-25T04:23:55.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:55.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033411+0000", 2023-11-25T04:23:55.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3543999999999999e-05 2023-11-25T04:23:55.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:55.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033423+0000", 2023-11-25T04:23:55.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1905999999999999e-05 2023-11-25T04:23:55.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:55.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034433+0000", 2023-11-25T04:23:55.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00101022 2023-11-25T04:23:55.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.627 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:55.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034539+0000", 2023-11-25T04:23:55.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010597199999999999 2023-11-25T04:23:55.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.627 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:55.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034667+0000", 2023-11-25T04:23:55.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012765500000000001 2023-11-25T04:23:55.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:55.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037803+0000", 2023-11-25T04:23:55.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031362220000000001 2023-11-25T04:23:55.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:55.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:23:55.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0064964669999999997 2023-11-25T04:23:55.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:55.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044331+0000", 2023-11-25T04:23:55.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1544999999999997e-05 2023-11-25T04:23:55.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:55.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044372+0000", 2023-11-25T04:23:55.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0349000000000003e-05 2023-11-25T04:23:55.629 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:55.629 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:55.629 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:55.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.629 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:15 3.15 3:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:55.630 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:55.630 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 68.561877553000002, 2023-11-25T04:23:55.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010464362, 2023-11-25T04:23:55.630 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:55.630 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:55.630 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:55.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:55.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:55.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:55.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.631 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:55.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:55.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:55.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.631 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:55.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034042+0000", 2023-11-25T04:23:55.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0999999999999997e-07 2023-11-25T04:23:55.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.631 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:55.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034048+0000", 2023-11-25T04:23:55.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0390000000000001e-06 2023-11-25T04:23:55.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:55.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034059+0000", 2023-11-25T04:23:55.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1182999999999999e-05 2023-11-25T04:23:55.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:55.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034067+0000", 2023-11-25T04:23:55.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1559999999999995e-06 2023-11-25T04:23:55.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:55.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035005+0000", 2023-11-25T04:23:55.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00093747399999999999 2023-11-25T04:23:55.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:55.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035077+0000", 2023-11-25T04:23:55.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2747e-05 2023-11-25T04:23:55.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:55.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035219+0000", 2023-11-25T04:23:55.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000141877 2023-11-25T04:23:55.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:55.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037810+0000", 2023-11-25T04:23:55.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002590752 2023-11-25T04:23:55.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:55.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044437+0000", 2023-11-25T04:23:55.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066270110000000004 2023-11-25T04:23:55.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:55.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044481+0000", 2023-11-25T04:23:55.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4106000000000001e-05 2023-11-25T04:23:55.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:55.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044505+0000", 2023-11-25T04:23:55.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4207e-05 2023-11-25T04:23:55.635 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:55.635 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:55.635 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:55.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.636 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:16 3.1d 3:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:55.636 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:55.636 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 68.561835704999993, 2023-11-25T04:23:55.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010413923, 2023-11-25T04:23:55.636 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:55.636 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:55.636 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:55.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:55.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:55.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:55.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:55.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:55.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:55.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:55.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:55.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4499999999999997e-07 2023-11-25T04:23:55.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:55.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034087+0000", 2023-11-25T04:23:55.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9770000000000002e-06 2023-11-25T04:23:55.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:55.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034094+0000", 2023-11-25T04:23:55.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8809999999999998e-06 2023-11-25T04:23:55.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:55.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034100+0000", 2023-11-25T04:23:55.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8440000000000003e-06 2023-11-25T04:23:55.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:55.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034993+0000", 2023-11-25T04:23:55.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00089336799999999996 2023-11-25T04:23:55.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:55.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035099+0000", 2023-11-25T04:23:55.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106037 2023-11-25T04:23:55.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:55.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035276+0000", 2023-11-25T04:23:55.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017631799999999999 2023-11-25T04:23:55.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:55.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037815+0000", 2023-11-25T04:23:55.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025395959999999999 2023-11-25T04:23:55.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:55.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044420+0000", 2023-11-25T04:23:55.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066045749999999997 2023-11-25T04:23:55.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:55.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044463+0000", 2023-11-25T04:23:55.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3205999999999999e-05 2023-11-25T04:23:55.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:55.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044497+0000", 2023-11-25T04:23:55.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3476000000000003e-05 2023-11-25T04:23:55.641 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:55.642 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:55.642 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:55.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.642 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:17 3.1d 3:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:55.642 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:55.642 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 68.561810496000007, 2023-11-25T04:23:55.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01127356, 2023-11-25T04:23:55.642 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:55.642 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:55.642 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:55.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:55.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:55.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:55.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:55.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:55.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:55.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:55.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:55.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1100000000000001e-07 2023-11-25T04:23:55.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.644 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:55.644 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034112+0000", 2023-11-25T04:23:55.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8600000000000003e-06 2023-11-25T04:23:55.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.644 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:55.644 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034120+0000", 2023-11-25T04:23:55.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7700000000000001e-06 2023-11-25T04:23:55.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:55.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034127+0000", 2023-11-25T04:23:55.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0999999999999998e-06 2023-11-25T04:23:55.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:55.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035601+0000", 2023-11-25T04:23:55.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014739410000000001 2023-11-25T04:23:55.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:55.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035685+0000", 2023-11-25T04:23:55.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3677999999999994e-05 2023-11-25T04:23:55.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:55.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035827+0000", 2023-11-25T04:23:55.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014232599999999999 2023-11-25T04:23:55.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:55.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038725+0000", 2023-11-25T04:23:55.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028982779999999998 2023-11-25T04:23:55.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.647 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:55.647 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045323+0000", 2023-11-25T04:23:55.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0065980780000000003 2023-11-25T04:23:55.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.647 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:55.647 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045354+0000", 2023-11-25T04:23:55.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0463e-05 2023-11-25T04:23:55.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.648 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:55.648 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045381+0000", 2023-11-25T04:23:55.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7654999999999999e-05 2023-11-25T04:23:55.648 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:55.648 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:55.648 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:55.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.648 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:22 3.e 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:55.649 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:55.649 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 68.561784083000006, 2023-11-25T04:23:55.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010133422, 2023-11-25T04:23:55.649 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:55.649 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:55.649 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:55.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.649 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:55.649 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:55.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:55.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:55.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:55.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:55.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:55.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034135+0000", 2023-11-25T04:23:55.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-07 2023-11-25T04:23:55.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:55.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034138+0000", 2023-11-25T04:23:55.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6009999999999999e-06 2023-11-25T04:23:55.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:55.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034145+0000", 2023-11-25T04:23:55.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1169999999999998e-06 2023-11-25T04:23:55.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:55.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034150+0000", 2023-11-25T04:23:55.652 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9080000000000003e-06 2023-11-25T04:23:55.652 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.652 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.652 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:55.652 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034209+0000", 2023-11-25T04:23:55.652 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8149000000000002e-05 2023-11-25T04:23:55.652 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.652 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.652 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:55.652 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034415+0000", 2023-11-25T04:23:55.652 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020637699999999999 2023-11-25T04:23:55.652 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.652 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.653 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:55.653 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034584+0000", 2023-11-25T04:23:55.653 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016870399999999999 2023-11-25T04:23:55.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.653 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:55.653 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.036979+0000", 2023-11-25T04:23:55.653 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023957509999999998 2023-11-25T04:23:55.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.653 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:55.653 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044161+0000", 2023-11-25T04:23:55.653 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0071811590000000003 2023-11-25T04:23:55.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:55.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044224+0000", 2023-11-25T04:23:55.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3454000000000006e-05 2023-11-25T04:23:55.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:55.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044268+0000", 2023-11-25T04:23:55.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3761999999999999e-05 2023-11-25T04:23:55.654 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:55.654 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:55.654 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:55.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.655 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:23 3.e 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:55.655 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:55.655 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 68.561752674000005, 2023-11-25T04:23:55.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010281676, 2023-11-25T04:23:55.655 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:55.655 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:55.655 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:55.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:55.656 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:55.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:55.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.656 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:55.656 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:55.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:55.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.656 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:55.656 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:55.657 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5799999999999999e-07 2023-11-25T04:23:55.657 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.657 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.657 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:55.657 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034169+0000", 2023-11-25T04:23:55.657 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1879999999999998e-06 2023-11-25T04:23:55.657 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.657 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.657 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:55.657 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034177+0000", 2023-11-25T04:23:55.657 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7020000000000001e-06 2023-11-25T04:23:55.657 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.657 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.658 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:55.658 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034184+0000", 2023-11-25T04:23:55.658 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3910000000000003e-06 2023-11-25T04:23:55.658 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.658 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.658 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:55.658 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034924+0000", 2023-11-25T04:23:55.658 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00074032999999999998 2023-11-25T04:23:55.658 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.658 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.658 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:55.658 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034991+0000", 2023-11-25T04:23:55.659 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7347000000000004e-05 2023-11-25T04:23:55.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.659 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.659 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:55.659 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035096+0000", 2023-11-25T04:23:55.659 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010488 2023-11-25T04:23:55.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.659 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.659 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:55.659 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037588+0000", 2023-11-25T04:23:55.659 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024918309999999999 2023-11-25T04:23:55.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.660 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.660 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:55.660 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044367+0000", 2023-11-25T04:23:55.660 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067787589999999997 2023-11-25T04:23:55.660 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.660 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.660 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:55.660 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044415+0000", 2023-11-25T04:23:55.660 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7803000000000003e-05 2023-11-25T04:23:55.660 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.660 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.660 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:55.660 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044447+0000", 2023-11-25T04:23:55.661 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2886999999999999e-05 2023-11-25T04:23:55.661 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:55.661 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:55.661 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:55.661 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.661 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.661 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:30 3.15 3:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:55.661 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:55.661 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 68.561694234000001, 2023-11-25T04:23:55.661 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010365070000000001, 2023-11-25T04:23:55.661 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:55.661 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:55.661 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:55.662 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.662 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:55.662 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:55.662 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:55.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.662 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.662 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:55.662 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:55.662 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:55.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.662 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:55.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034225+0000", 2023-11-25T04:23:55.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7400000000000002e-07 2023-11-25T04:23:55.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:55.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034229+0000", 2023-11-25T04:23:55.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3100000000000002e-06 2023-11-25T04:23:55.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:55.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034239+0000", 2023-11-25T04:23:55.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9140000000000003e-06 2023-11-25T04:23:55.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:55.664 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034246+0000", 2023-11-25T04:23:55.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0149999999999998e-06 2023-11-25T04:23:55.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:55.664 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035572+0000", 2023-11-25T04:23:55.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001325655 2023-11-25T04:23:55.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:55.665 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035730+0000", 2023-11-25T04:23:55.665 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015827899999999999 2023-11-25T04:23:55.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.665 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.665 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:55.665 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035857+0000", 2023-11-25T04:23:55.665 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012644699999999999 2023-11-25T04:23:55.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.665 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.665 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:55.665 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038737+0000", 2023-11-25T04:23:55.665 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002880077 2023-11-25T04:23:55.666 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.666 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.666 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:55.666 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044540+0000", 2023-11-25T04:23:55.666 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058034239999999997 2023-11-25T04:23:55.666 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.666 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.666 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:55.666 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044569+0000", 2023-11-25T04:23:55.666 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8388999999999999e-05 2023-11-25T04:23:55.666 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.666 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.667 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:55.667 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044589+0000", 2023-11-25T04:23:55.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0686000000000002e-05 2023-11-25T04:23:55.667 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:55.667 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:55.667 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:55.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.667 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:33 3.1b 3:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:55.667 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:55.667 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 68.561648735999995, 2023-11-25T04:23:55.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099754049999999997, 2023-11-25T04:23:55.668 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:55.668 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:23:55.668 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:55.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.668 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:55.668 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:55.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:55.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.668 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:55.668 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:55.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:55.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.669 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:55.669 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:55.669 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4799999999999998e-07 2023-11-25T04:23:55.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.669 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:55.669 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034273+0000", 2023-11-25T04:23:55.669 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7819999999999998e-06 2023-11-25T04:23:55.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.669 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:55.670 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034285+0000", 2023-11-25T04:23:55.670 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2058e-05 2023-11-25T04:23:55.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.670 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:55.670 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034291+0000", 2023-11-25T04:23:55.670 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0680000000000002e-06 2023-11-25T04:23:55.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.670 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:55.670 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034370+0000", 2023-11-25T04:23:55.670 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8452e-05 2023-11-25T04:23:55.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:55.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034477+0000", 2023-11-25T04:23:55.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107471 2023-11-25T04:23:55.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:55.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044245+0000", 2023-11-25T04:23:55.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0097679259999999997 2023-11-25T04:23:55.671 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:55.671 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:55.671 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:55.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.672 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:36 3.1a 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:55.672 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:55.672 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 68.561602062999995, 2023-11-25T04:23:55.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058179099999999999, 2023-11-25T04:23:55.672 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:55.672 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:23:55.672 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:55.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.672 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:55.672 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:55.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:55.673 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.673 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.673 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:55.673 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:55.673 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:55.673 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.673 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.673 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:55.673 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034317+0000", 2023-11-25T04:23:55.673 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1999999999999999e-07 2023-11-25T04:23:55.673 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.673 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.673 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:55.674 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034321+0000", 2023-11-25T04:23:55.674 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.233e-06 2023-11-25T04:23:55.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.674 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.674 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:55.674 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034337+0000", 2023-11-25T04:23:55.674 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5308999999999999e-05 2023-11-25T04:23:55.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.674 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.674 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:55.674 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034354+0000", 2023-11-25T04:23:55.674 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7411999999999999e-05 2023-11-25T04:23:55.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.675 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.675 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:55.675 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034465+0000", 2023-11-25T04:23:55.675 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011154699999999999 2023-11-25T04:23:55.675 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.675 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.675 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:55.675 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034560+0000", 2023-11-25T04:23:55.675 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4186999999999998e-05 2023-11-25T04:23:55.675 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.675 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.675 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:55.676 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.040134+0000", 2023-11-25T04:23:55.676 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055746019999999997 2023-11-25T04:23:55.676 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:55.676 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:55.676 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:55.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.676 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:9 3.10 e18/16 3:09401a81:::601.00000000:head v 18'4, mlcod=0'0)", 2023-11-25T04:23:55.676 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:55.676 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 68.552707755, 2023-11-25T04:23:55.676 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061641020000000003, 2023-11-25T04:23:55.676 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:55.677 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:55.677 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:55.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.677 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:55.677 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:55.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:55.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.677 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:55.677 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:55.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:55.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.678 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:55.678 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:55.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5300000000000001e-07 2023-11-25T04:23:55.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.678 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.678 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:55.678 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043222+0000", 2023-11-25T04:23:55.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.082e-05 2023-11-25T04:23:55.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.678 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.678 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:55.678 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043228+0000", 2023-11-25T04:23:55.679 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6699999999999997e-06 2023-11-25T04:23:55.679 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.679 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.679 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:55.679 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043236+0000", 2023-11-25T04:23:55.679 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8189999999999992e-06 2023-11-25T04:23:55.679 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.679 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.679 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:55.679 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043507+0000", 2023-11-25T04:23:55.679 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000271012 2023-11-25T04:23:55.679 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.680 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.680 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:55.680 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043530+0000", 2023-11-25T04:23:55.680 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2659000000000001e-05 2023-11-25T04:23:55.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.680 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.680 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:55.680 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049321+0000", 2023-11-25T04:23:55.680 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057915129999999999 2023-11-25T04:23:55.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.680 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.680 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:55.680 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049375+0000", 2023-11-25T04:23:55.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3455999999999998e-05 2023-11-25T04:23:55.681 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:55.681 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:55.681 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:55.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.681 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:1 3.14 e18/16 3:292cf221:::200.00000000:head v 18'3, mlcod=0'0)", 2023-11-25T04:23:55.681 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:55.681 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 68.552671042, 2023-11-25T04:23:55.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0060983599999999997, 2023-11-25T04:23:55.681 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:55.681 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:55.681 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:55.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:55.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:55.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:55.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:55.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:55.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:55.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:55.683 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043248+0000", 2023-11-25T04:23:55.683 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9399999999999995e-07 2023-11-25T04:23:55.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.683 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:55.683 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043252+0000", 2023-11-25T04:23:55.683 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1799999999999998e-06 2023-11-25T04:23:55.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.683 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:55.683 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043259+0000", 2023-11-25T04:23:55.683 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0049999999999998e-06 2023-11-25T04:23:55.684 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.684 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.684 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:55.684 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043265+0000", 2023-11-25T04:23:55.684 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0859999999999998e-06 2023-11-25T04:23:55.684 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.684 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.684 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:55.684 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043311+0000", 2023-11-25T04:23:55.684 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6082999999999999e-05 2023-11-25T04:23:55.684 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.684 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.684 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:55.685 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043330+0000", 2023-11-25T04:23:55.685 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8556e-05 2023-11-25T04:23:55.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.685 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.685 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:55.685 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049303+0000", 2023-11-25T04:23:55.685 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059733470000000004 2023-11-25T04:23:55.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.685 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.685 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:55.685 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049346+0000", 2023-11-25T04:23:55.685 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2608999999999998e-05 2023-11-25T04:23:55.685 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:55.686 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:55.686 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:55.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.686 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:3 3.14 e18/16 3:287034c4:::1.00000000.inode:head v 18'4, mlcod=0'0)", 2023-11-25T04:23:55.686 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:55.686 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 68.552154536000003, 2023-11-25T04:23:55.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057040090000000003, 2023-11-25T04:23:55.686 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:55.686 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:55.686 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:55.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:55.687 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:55.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:55.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.687 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:55.687 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:55.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:55.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.687 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:55.687 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:55.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9799999999999999e-07 2023-11-25T04:23:55.688 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.688 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.688 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:55.688 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043785+0000", 2023-11-25T04:23:55.688 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0721999999999999e-05 2023-11-25T04:23:55.688 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.688 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.688 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:55.688 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043793+0000", 2023-11-25T04:23:55.688 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5839999999999995e-06 2023-11-25T04:23:55.688 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.688 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.688 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:55.689 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043803+0000", 2023-11-25T04:23:55.689 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0729999999999994e-06 2023-11-25T04:23:55.689 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.689 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:55.689 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043843+0000", 2023-11-25T04:23:55.689 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0747999999999997e-05 2023-11-25T04:23:55.689 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.689 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:55.689 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043856+0000", 2023-11-25T04:23:55.689 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2697000000000001e-05 2023-11-25T04:23:55.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.690 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.690 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:55.690 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049426+0000", 2023-11-25T04:23:55.690 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055704409999999998 2023-11-25T04:23:55.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.690 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.690 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:55.690 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049468+0000", 2023-11-25T04:23:55.690 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1446000000000003e-05 2023-11-25T04:23:55.690 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:55.690 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:55.690 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:55.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.691 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "pg_update_log_missing(3.14 epoch 18/16 rep_tid 171 entries 18'5 (0'0) error 3:2e2fa760:::200.00000001:head by mds.0.3:32 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:23:55.691 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:55.691 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 68.551993573000004, 2023-11-25T04:23:55.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005556938, 2023-11-25T04:23:55.691 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:55.691 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2023-11-25T04:23:55.691 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:55.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.691 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:55.691 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:55.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:55.692 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.692 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.692 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:55.692 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:55.692 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:55.692 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.692 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.692 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:55.692 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:55.692 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.03e-07 2023-11-25T04:23:55.692 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.692 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.693 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:55.693 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043931+0000", 2023-11-25T04:23:55.693 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1079999999999998e-06 2023-11-25T04:23:55.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.693 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:55.693 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043944+0000", 2023-11-25T04:23:55.693 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2852e-05 2023-11-25T04:23:55.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.693 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:55.693 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043955+0000", 2023-11-25T04:23:55.693 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1037e-05 2023-11-25T04:23:55.694 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.694 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.694 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:55.694 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044088+0000", 2023-11-25T04:23:55.694 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013269599999999999 2023-11-25T04:23:55.694 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.694 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.694 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:55.694 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049482+0000", 2023-11-25T04:23:55.694 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053940419999999999 2023-11-25T04:23:55.694 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:55.694 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:55.695 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:55.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.695 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.695 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:10 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'5, mlcod=0'0)", 2023-11-25T04:23:55.695 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:55.695 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 68.551903545000002, 2023-11-25T04:23:55.695 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054493099999999997, 2023-11-25T04:23:55.695 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:55.695 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:55.695 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:55.695 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.695 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:55.695 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:55.696 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:55.696 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.696 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.696 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:55.696 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:55.696 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:55.696 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.696 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.696 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:55.696 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:55.696 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6100000000000002e-07 2023-11-25T04:23:55.696 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.697 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:55.697 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044019+0000", 2023-11-25T04:23:55.697 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.839e-06 2023-11-25T04:23:55.697 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.697 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:55.697 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044026+0000", 2023-11-25T04:23:55.697 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5869999999999998e-06 2023-11-25T04:23:55.697 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.697 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:55.697 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044031+0000", 2023-11-25T04:23:55.698 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4909999999999996e-06 2023-11-25T04:23:55.698 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.698 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.698 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:55.698 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044070+0000", 2023-11-25T04:23:55.698 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8829999999999999e-05 2023-11-25T04:23:55.698 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.698 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.698 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:55.698 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044081+0000", 2023-11-25T04:23:55.698 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1447999999999999e-05 2023-11-25T04:23:55.698 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.698 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.699 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:55.699 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049406+0000", 2023-11-25T04:23:55.699 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053246480000000004 2023-11-25T04:23:55.699 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.699 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.699 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:55.699 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049464+0000", 2023-11-25T04:23:55.699 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8205999999999998e-05 2023-11-25T04:23:55.699 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:55.699 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:55.699 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:55.699 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.699 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.700 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:11 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'6, mlcod=0'0)", 2023-11-25T04:23:55.700 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:55.700 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 68.551702281000004, 2023-11-25T04:23:55.700 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052673499999999996, 2023-11-25T04:23:55.700 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:55.700 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:55.700 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:55.700 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.700 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:55.700 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:55.700 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:55.700 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.700 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.701 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:55.701 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:55.701 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:55.701 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.701 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.701 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:55.701 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:55.701 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0199999999999998e-07 2023-11-25T04:23:55.701 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.701 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.701 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:55.701 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044220+0000", 2023-11-25T04:23:55.702 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7720000000000001e-06 2023-11-25T04:23:55.702 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.702 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:55.702 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044226+0000", 2023-11-25T04:23:55.702 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8579999999999996e-06 2023-11-25T04:23:55.702 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.702 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:55.702 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044232+0000", 2023-11-25T04:23:55.702 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1700000000000002e-06 2023-11-25T04:23:55.702 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.703 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:55.703 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044274+0000", 2023-11-25T04:23:55.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1254999999999997e-05 2023-11-25T04:23:55.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.703 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:55.703 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:23:55.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6880999999999999e-05 2023-11-25T04:23:55.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.703 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:55.703 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049466+0000", 2023-11-25T04:23:55.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051659219999999999 2023-11-25T04:23:55.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:55.704 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:55.704 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:55.704 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049483+0000", 2023-11-25T04:23:55.704 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.719e-05 2023-11-25T04:23:55.704 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:55.704 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:55.704 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:55.704 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:55.704 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:55.704 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:55.706 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:23:55.706 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:55.705+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:55.807 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:55.806+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:55.816 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:55.816 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:23:55.816 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2023-11-25T04:23:55.817 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:55.825 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:23:55.907 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~47s 2023-11-25T04:23:55.907 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:55.906+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:55.930 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:55.930 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:23:55.930 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2023-11-25T04:23:55.930 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2023-11-25T04:23:55.930 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:55.939 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:23:55.958 INFO:tasks.workunit.client.0.smithi067.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2023-11-25T04:23:56.002 INFO:teuthology.orchestra.run.smithi067.stderr:nodeep-scrub is set 2023-11-25T04:23:56.008 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:56.007+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:56.023 INFO:tasks.workunit.client.0.smithi067.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2023-11-25T04:23:56.067 INFO:tasks.workunit.client.0.smithi067.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-11-25T04:23:56.069 INFO:tasks.workunit.client.0.smithi067.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-11-25T04:23:56.069 INFO:tasks.workunit.client.0.smithi067.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2023-11-25T04:23:56.086 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:56.087 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2023-11-25T04:23:56.087 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2023-11-25T04:23:56.087 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2023-11-25T04:23:56.087 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.087 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:56.087 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:56.087 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 69.052540796000002, 2023-11-25T04:23:56.087 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01072943, 2023-11-25T04:23:56.087 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:56.087 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:56.087 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:56.087 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.088 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:56.088 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:56.088 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:56.088 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.088 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.088 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:56.088 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:56.088 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:56.088 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.088 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.088 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:56.088 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031829+0000", 2023-11-25T04:23:56.088 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.666e-06 2023-11-25T04:23:56.089 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.089 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.089 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:56.089 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031839+0000", 2023-11-25T04:23:56.089 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9490000000000008e-06 2023-11-25T04:23:56.089 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.089 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.089 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:56.089 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031857+0000", 2023-11-25T04:23:56.089 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8093000000000001e-05 2023-11-25T04:23:56.089 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.089 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.089 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:56.090 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031882+0000", 2023-11-25T04:23:56.090 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4294000000000002e-05 2023-11-25T04:23:56.090 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.090 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.090 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:56.090 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033130+0000", 2023-11-25T04:23:56.090 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001248264 2023-11-25T04:23:56.090 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.090 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.090 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:56.090 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033426+0000", 2023-11-25T04:23:56.090 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029620999999999998 2023-11-25T04:23:56.090 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.091 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.091 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:56.091 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033814+0000", 2023-11-25T04:23:56.091 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000387742 2023-11-25T04:23:56.091 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.091 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.091 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:56.091 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036793+0000", 2023-11-25T04:23:56.091 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002979598 2023-11-25T04:23:56.091 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.091 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.091 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:56.091 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042455+0000", 2023-11-25T04:23:56.092 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0056619260000000003 2023-11-25T04:23:56.092 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.092 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.092 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:56.092 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042502+0000", 2023-11-25T04:23:56.092 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7139999999999999e-05 2023-11-25T04:23:56.092 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.092 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.092 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:56.092 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042557+0000", 2023-11-25T04:23:56.092 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4548e-05 2023-11-25T04:23:56.092 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:56.092 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:56.093 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:56.093 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.093 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.093 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:56.093 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:56.093 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 69.051342261000002, 2023-11-25T04:23:56.093 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011505202000000001, 2023-11-25T04:23:56.093 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:56.093 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:56.093 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:56.093 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.093 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:56.093 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:56.094 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:56.094 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.094 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.094 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:56.094 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:56.094 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:56.094 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.094 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.094 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:56.094 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033027+0000", 2023-11-25T04:23:56.094 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0699999999999999e-06 2023-11-25T04:23:56.094 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.095 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.095 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:56.095 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033035+0000", 2023-11-25T04:23:56.095 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0830000000000004e-06 2023-11-25T04:23:56.095 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.095 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.095 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:56.095 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033059+0000", 2023-11-25T04:23:56.095 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3975e-05 2023-11-25T04:23:56.095 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.095 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.095 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:56.095 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033482+0000", 2023-11-25T04:23:56.096 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042253700000000001 2023-11-25T04:23:56.096 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.096 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.096 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:56.096 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034520+0000", 2023-11-25T04:23:56.096 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001037991 2023-11-25T04:23:56.096 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.096 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.096 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:56.096 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034611+0000", 2023-11-25T04:23:56.096 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1113999999999995e-05 2023-11-25T04:23:56.096 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.096 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.097 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:56.097 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034769+0000", 2023-11-25T04:23:56.097 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000157641 2023-11-25T04:23:56.097 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.097 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.097 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:56.097 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038292+0000", 2023-11-25T04:23:56.097 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035231160000000002 2023-11-25T04:23:56.097 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.097 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.097 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:56.099 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044440+0000", 2023-11-25T04:23:56.099 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0061486980000000002 2023-11-25T04:23:56.099 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.099 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.099 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:56.099 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044495+0000", 2023-11-25T04:23:56.099 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4339000000000003e-05 2023-11-25T04:23:56.099 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.099 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.099 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:56.100 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044531+0000", 2023-11-25T04:23:56.100 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6637999999999997e-05 2023-11-25T04:23:56.100 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:56.100 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:56.100 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:56.100 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.100 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.100 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:56.100 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:56.100 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 69.050867906999997, 2023-11-25T04:23:56.100 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0057634039999999997, 2023-11-25T04:23:56.100 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:56.100 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:56.101 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:56.101 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.101 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:56.101 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:56.101 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:56.101 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.101 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.101 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:56.101 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:56.101 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:56.101 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.101 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.101 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:56.102 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:56.102 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7900000000000007e-07 2023-11-25T04:23:56.102 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.102 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.102 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:56.102 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033507+0000", 2023-11-25T04:23:56.102 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8900000000000004e-06 2023-11-25T04:23:56.102 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.102 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.102 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:56.102 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033519+0000", 2023-11-25T04:23:56.102 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1756000000000001e-05 2023-11-25T04:23:56.102 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.103 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.103 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:56.103 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033527+0000", 2023-11-25T04:23:56.103 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3610000000000001e-06 2023-11-25T04:23:56.103 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.103 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.103 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:56.103 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034451+0000", 2023-11-25T04:23:56.103 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00092372400000000001 2023-11-25T04:23:56.103 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.103 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.103 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:56.103 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034809+0000", 2023-11-25T04:23:56.104 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00035757699999999998 2023-11-25T04:23:56.104 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.104 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.104 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:23:56.104 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034965+0000", 2023-11-25T04:23:56.104 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015638699999999999 2023-11-25T04:23:56.104 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.104 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.104 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:56.104 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038299+0000", 2023-11-25T04:23:56.104 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0033336559999999999 2023-11-25T04:23:56.104 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.104 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.104 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:56.105 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039169+0000", 2023-11-25T04:23:56.105 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00087066400000000005 2023-11-25T04:23:56.105 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.105 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.105 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:56.105 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039226+0000", 2023-11-25T04:23:56.106 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6511999999999997e-05 2023-11-25T04:23:56.106 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.106 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.106 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:56.106 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039264+0000", 2023-11-25T04:23:56.106 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7898000000000002e-05 2023-11-25T04:23:56.106 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:56.106 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:56.106 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:56.106 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.106 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.106 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:25 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:56.106 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:56.107 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 69.050789323000004, 2023-11-25T04:23:56.107 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096656169999999996, 2023-11-25T04:23:56.107 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:56.107 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:56.107 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:56.107 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.107 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:56.107 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:56.107 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:56.107 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.107 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.107 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:56.107 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:56.107 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:56.108 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.108 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.108 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:56.108 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033580+0000", 2023-11-25T04:23:56.108 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8199999999999999e-07 2023-11-25T04:23:56.108 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.108 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.108 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:56.108 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033585+0000", 2023-11-25T04:23:56.108 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3680000000000001e-06 2023-11-25T04:23:56.108 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.108 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.108 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:56.109 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033600+0000", 2023-11-25T04:23:56.109 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4331000000000001e-05 2023-11-25T04:23:56.109 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.109 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.109 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:56.109 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033606+0000", 2023-11-25T04:23:56.109 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5780000000000004e-06 2023-11-25T04:23:56.109 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.109 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.109 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:56.109 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035747+0000", 2023-11-25T04:23:56.109 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0021406530000000002 2023-11-25T04:23:56.109 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.109 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.110 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:56.110 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035901+0000", 2023-11-25T04:23:56.110 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000153931 2023-11-25T04:23:56.110 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.110 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.110 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:23:56.110 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036193+0000", 2023-11-25T04:23:56.110 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029211299999999999 2023-11-25T04:23:56.110 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.110 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.110 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:56.110 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038655+0000", 2023-11-25T04:23:56.110 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0024621640000000002 2023-11-25T04:23:56.111 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.111 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.111 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:56.111 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043130+0000", 2023-11-25T04:23:56.111 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.004474698 2023-11-25T04:23:56.111 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.111 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.111 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:56.111 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043204+0000", 2023-11-25T04:23:56.111 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4091999999999995e-05 2023-11-25T04:23:56.111 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.111 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.111 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:56.112 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043245+0000", 2023-11-25T04:23:56.112 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0907e-05 2023-11-25T04:23:56.112 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:56.112 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:56.112 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:56.112 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.112 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.112 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:26 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:56.112 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:56.112 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 69.050753498999995, 2023-11-25T04:23:56.112 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011086945000000001, 2023-11-25T04:23:56.112 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:56.113 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:56.113 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:56.113 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.113 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:56.113 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:56.113 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:56.113 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.113 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.113 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:56.113 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:56.113 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:56.113 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.113 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.114 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:56.114 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:56.114 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1699999999999998e-07 2023-11-25T04:23:56.114 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.114 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.114 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:56.114 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033627+0000", 2023-11-25T04:23:56.114 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.163e-05 2023-11-25T04:23:56.114 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.114 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.114 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:56.114 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033636+0000", 2023-11-25T04:23:56.115 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6859999999999995e-06 2023-11-25T04:23:56.115 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.116 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.116 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:56.116 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033642+0000", 2023-11-25T04:23:56.117 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3149999999999997e-06 2023-11-25T04:23:56.117 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.117 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.117 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:56.117 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035414+0000", 2023-11-25T04:23:56.117 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0017716380000000001 2023-11-25T04:23:56.117 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.117 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.117 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:56.117 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035539+0000", 2023-11-25T04:23:56.117 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000125477 2023-11-25T04:23:56.117 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.117 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.117 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:56.118 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035672+0000", 2023-11-25T04:23:56.118 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000133316 2023-11-25T04:23:56.118 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.118 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.118 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:56.118 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038649+0000", 2023-11-25T04:23:56.118 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029764679999999999 2023-11-25T04:23:56.118 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.118 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.118 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:56.118 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044578+0000", 2023-11-25T04:23:56.118 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0059293510000000002 2023-11-25T04:23:56.118 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.119 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.119 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:56.119 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044651+0000", 2023-11-25T04:23:56.119 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2513000000000002e-05 2023-11-25T04:23:56.119 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.119 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.119 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:56.119 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044702+0000", 2023-11-25T04:23:56.119 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1034e-05 2023-11-25T04:23:56.119 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:56.119 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:56.119 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:56.119 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.120 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.120 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:29 3.b 3:d0630e4c:::mds0_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:56.120 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:56.120 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 69.050715578999998, 2023-11-25T04:23:56.120 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010980227, 2023-11-25T04:23:56.120 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:56.120 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:56.120 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:56.120 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.120 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:56.120 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:56.120 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:56.120 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.120 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.121 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:56.121 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:56.121 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:56.121 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.121 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.121 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:56.121 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:56.121 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3499999999999996e-07 2023-11-25T04:23:56.121 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.121 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.121 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:56.121 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033658+0000", 2023-11-25T04:23:56.121 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1280000000000001e-06 2023-11-25T04:23:56.121 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.122 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.122 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:56.122 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033667+0000", 2023-11-25T04:23:56.122 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1430000000000004e-06 2023-11-25T04:23:56.122 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.122 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.122 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:56.122 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033678+0000", 2023-11-25T04:23:56.122 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0906e-05 2023-11-25T04:23:56.122 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.122 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.122 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:56.122 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034456+0000", 2023-11-25T04:23:56.123 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00077833899999999996 2023-11-25T04:23:56.123 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.123 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.123 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:56.123 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034575+0000", 2023-11-25T04:23:56.123 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011926000000000001 2023-11-25T04:23:56.124 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.124 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.124 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:56.124 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035001+0000", 2023-11-25T04:23:56.124 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042550999999999998 2023-11-25T04:23:56.124 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.124 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.124 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:56.125 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.037923+0000", 2023-11-25T04:23:56.125 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029219670000000001 2023-11-25T04:23:56.125 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.125 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.125 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:56.125 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044538+0000", 2023-11-25T04:23:56.125 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0066152349999999997 2023-11-25T04:23:56.125 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.125 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.125 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:56.125 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044594+0000", 2023-11-25T04:23:56.126 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5593000000000001e-05 2023-11-25T04:23:56.126 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.126 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.126 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:56.126 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044633+0000", 2023-11-25T04:23:56.126 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9511000000000002e-05 2023-11-25T04:23:56.127 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:56.127 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:56.127 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:56.127 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.127 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.127 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:34 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:56.127 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:56.127 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 69.050465466000006, 2023-11-25T04:23:56.127 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010920445000000001, 2023-11-25T04:23:56.127 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:56.127 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:56.127 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:56.127 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.127 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:56.128 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:56.128 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:56.128 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.128 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.128 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:56.128 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:56.128 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:56.128 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.128 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.128 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:56.128 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033904+0000", 2023-11-25T04:23:56.128 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6599999999999994e-07 2023-11-25T04:23:56.128 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.128 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.129 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:56.129 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033909+0000", 2023-11-25T04:23:56.129 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5639999999999996e-06 2023-11-25T04:23:56.129 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.129 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.129 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:56.129 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033923+0000", 2023-11-25T04:23:56.129 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3028e-05 2023-11-25T04:23:56.129 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.129 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.129 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:56.129 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034043+0000", 2023-11-25T04:23:56.129 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000120328 2023-11-25T04:23:56.130 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.130 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.130 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:56.130 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035576+0000", 2023-11-25T04:23:56.130 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015333720000000001 2023-11-25T04:23:56.130 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.130 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.130 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:56.130 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035699+0000", 2023-11-25T04:23:56.130 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000123001 2023-11-25T04:23:56.130 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.130 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.130 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:56.130 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044823+0000", 2023-11-25T04:23:56.131 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0091241859999999994 2023-11-25T04:23:56.131 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:56.131 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:56.131 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:56.131 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.131 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.131 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:35 3.1f 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:56.131 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:56.131 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 69.049161260999995, 2023-11-25T04:23:56.131 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096164779999999995, 2023-11-25T04:23:56.131 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:56.131 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:56.131 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:56.131 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.132 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:56.132 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:56.132 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:56.132 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.132 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.132 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:56.132 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:56.132 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:56.132 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.132 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.132 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:56.132 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035208+0000", 2023-11-25T04:23:56.132 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2699999999999998e-07 2023-11-25T04:23:56.133 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.133 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.133 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:56.133 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035213+0000", 2023-11-25T04:23:56.133 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.6070000000000001e-06 2023-11-25T04:23:56.133 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.133 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.133 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:56.133 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035350+0000", 2023-11-25T04:23:56.133 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013693599999999999 2023-11-25T04:23:56.133 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.133 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.133 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:56.133 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035358+0000", 2023-11-25T04:23:56.134 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0099999999999995e-06 2023-11-25T04:23:56.134 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.134 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.134 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:56.134 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036260+0000", 2023-11-25T04:23:56.134 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00090240100000000005 2023-11-25T04:23:56.134 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.134 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.134 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:56.134 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036517+0000", 2023-11-25T04:23:56.134 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00025736299999999998 2023-11-25T04:23:56.134 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.134 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.135 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:56.135 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044824+0000", 2023-11-25T04:23:56.135 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0083063340000000003 2023-11-25T04:23:56.135 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:56.135 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:56.135 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:56.135 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.135 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.135 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:37 3.1e 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:56.135 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:56.135 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 69.048994574999995, 2023-11-25T04:23:56.135 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0078322640000000002, 2023-11-25T04:23:56.135 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:56.136 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:56.136 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:56.136 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.136 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:56.136 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:56.136 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:56.136 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.136 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.136 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:56.137 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:56.137 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:56.137 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.137 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.137 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:56.137 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035375+0000", 2023-11-25T04:23:56.137 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9100000000000003e-07 2023-11-25T04:23:56.137 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.137 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.137 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:56.137 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035379+0000", 2023-11-25T04:23:56.138 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9430000000000002e-06 2023-11-25T04:23:56.138 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.138 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.138 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:56.138 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035388+0000", 2023-11-25T04:23:56.138 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2760000000000006e-06 2023-11-25T04:23:56.138 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.138 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.138 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:56.138 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035396+0000", 2023-11-25T04:23:56.138 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8480000000000001e-06 2023-11-25T04:23:56.138 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.138 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.139 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:56.139 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035452+0000", 2023-11-25T04:23:56.139 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5831000000000001e-05 2023-11-25T04:23:56.139 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.139 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.139 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:56.139 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035934+0000", 2023-11-25T04:23:56.139 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00048265599999999998 2023-11-25T04:23:56.139 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.139 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.139 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:56.139 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043206+0000", 2023-11-25T04:23:56.139 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.007271919 2023-11-25T04:23:56.139 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:56.140 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:56.140 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:56.140 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.140 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.140 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing(3.1a epoch 18/16 rep_tid 3 entries 18'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:36 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:23:56.140 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:56.140 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 69.047718989000003, 2023-11-25T04:23:56.140 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002073414, 2023-11-25T04:23:56.140 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:56.140 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:56.140 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:56.140 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.140 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:56.141 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:56.141 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:56.141 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.141 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.141 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:56.141 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:56.141 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:56.141 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.141 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.141 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:56.141 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036650+0000", 2023-11-25T04:23:56.141 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7300000000000002e-07 2023-11-25T04:23:56.141 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.141 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.142 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:56.142 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036659+0000", 2023-11-25T04:23:56.142 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.7499999999999992e-06 2023-11-25T04:23:56.142 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.142 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.142 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:56.142 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036673+0000", 2023-11-25T04:23:56.142 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4917e-05 2023-11-25T04:23:56.142 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.142 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.142 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:56.142 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036687+0000", 2023-11-25T04:23:56.142 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3883000000000001e-05 2023-11-25T04:23:56.143 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.143 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.143 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:56.143 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036781+0000", 2023-11-25T04:23:56.143 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3219000000000003e-05 2023-11-25T04:23:56.143 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.143 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.143 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:56.143 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038723+0000", 2023-11-25T04:23:56.143 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019422720000000001 2023-11-25T04:23:56.143 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:56.143 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:56.143 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:56.143 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.144 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.144 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:28 3.13 e18/16 3:cd5a64a3:::100.00000000:head v 18'1, mlcod=0'0)", 2023-11-25T04:23:56.144 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:56.144 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 69.044057831000003, 2023-11-25T04:23:56.144 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001850546, 2023-11-25T04:23:56.144 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:56.144 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:56.144 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:56.144 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.144 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:56.144 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:56.144 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:56.144 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.145 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.145 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:56.145 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:56.145 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:56.145 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.145 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.145 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:56.145 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:56.145 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7800000000000002e-07 2023-11-25T04:23:56.145 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.145 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.145 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:56.145 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040331+0000", 2023-11-25T04:23:56.145 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0449999999999999e-05 2023-11-25T04:23:56.146 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.146 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.146 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:56.146 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040339+0000", 2023-11-25T04:23:56.146 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0709999999999995e-06 2023-11-25T04:23:56.146 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.146 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.146 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:56.146 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040347+0000", 2023-11-25T04:23:56.146 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7079999999999997e-06 2023-11-25T04:23:56.147 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.147 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.147 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:56.147 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040410+0000", 2023-11-25T04:23:56.147 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2453000000000007e-05 2023-11-25T04:23:56.147 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.147 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.147 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:56.147 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040422+0000", 2023-11-25T04:23:56.147 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2429000000000001e-05 2023-11-25T04:23:56.147 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.147 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.147 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:56.148 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042129+0000", 2023-11-25T04:23:56.148 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001707266 2023-11-25T04:23:56.148 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.148 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.148 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:56.148 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042161+0000", 2023-11-25T04:23:56.148 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1791000000000001e-05 2023-11-25T04:23:56.148 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:56.148 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:56.148 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:56.148 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.148 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.148 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:27 3.13 e18/16 3:c91d4a1d:::mds0_inotable:head v 18'2, mlcod=0'0)", 2023-11-25T04:23:56.148 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:56.149 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 69.043854307999993, 2023-11-25T04:23:56.149 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0016886430000000001, 2023-11-25T04:23:56.149 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:56.149 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:56.149 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:56.149 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.149 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:56.149 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:56.149 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:56.149 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.149 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.149 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:56.149 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:56.150 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:56.150 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.150 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.150 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:56.150 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:56.150 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3999999999999997e-07 2023-11-25T04:23:56.150 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.150 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.150 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:56.150 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040520+0000", 2023-11-25T04:23:56.150 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2989999999999996e-06 2023-11-25T04:23:56.150 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.151 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.151 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:56.151 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040527+0000", 2023-11-25T04:23:56.151 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5730000000000002e-06 2023-11-25T04:23:56.151 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.151 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.164 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:56.164 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040535+0000", 2023-11-25T04:23:56.164 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8630000000000008e-06 2023-11-25T04:23:56.164 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.164 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.164 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:56.164 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040683+0000", 2023-11-25T04:23:56.164 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000148002 2023-11-25T04:23:56.164 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.165 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.165 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:56.165 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040700+0000", 2023-11-25T04:23:56.165 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7065e-05 2023-11-25T04:23:56.165 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.165 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.165 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:56.165 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042163+0000", 2023-11-25T04:23:56.165 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001462767 2023-11-25T04:23:56.165 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.165 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.165 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:56.165 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042203+0000", 2023-11-25T04:23:56.166 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9734000000000003e-05 2023-11-25T04:23:56.166 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:56.166 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:56.166 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:56.166 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.166 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.166 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:20 3.1c e18/16 3:3ed09add:::607.00000000:head v 18'1, mlcod=0'0)", 2023-11-25T04:23:56.166 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:56.166 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 69.043819267000003, 2023-11-25T04:23:56.166 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015271, 2023-11-25T04:23:56.166 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:56.166 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:56.166 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:56.166 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.167 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:56.167 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:56.167 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:56.167 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.167 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.167 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:56.167 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:56.167 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:56.167 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.167 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.167 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:56.167 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:56.167 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4799999999999999e-07 2023-11-25T04:23:56.168 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.168 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.168 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:56.168 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040561+0000", 2023-11-25T04:23:56.168 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1800000000000001e-05 2023-11-25T04:23:56.168 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.168 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.168 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:56.168 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040568+0000", 2023-11-25T04:23:56.168 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7730000000000002e-06 2023-11-25T04:23:56.168 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.168 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.168 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:56.169 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040587+0000", 2023-11-25T04:23:56.169 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8691000000000001e-05 2023-11-25T04:23:56.169 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.169 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.169 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:56.169 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040638+0000", 2023-11-25T04:23:56.169 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1068e-05 2023-11-25T04:23:56.169 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.169 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.169 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:56.169 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040670+0000", 2023-11-25T04:23:56.169 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2125999999999997e-05 2023-11-25T04:23:56.169 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.170 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.170 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:56.170 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042032+0000", 2023-11-25T04:23:56.170 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001361821 2023-11-25T04:23:56.170 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.170 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.170 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:56.170 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042076+0000", 2023-11-25T04:23:56.170 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4472999999999999e-05 2023-11-25T04:23:56.170 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:56.170 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:56.170 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:56.170 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.171 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.171 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:21 3.1c e18/16 3:3ed09add:::607.00000000:head v 18'2, mlcod=0'0)", 2023-11-25T04:23:56.171 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:56.171 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 69.043584542000005, 2023-11-25T04:23:56.171 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0013228160000000001, 2023-11-25T04:23:56.171 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:56.171 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:56.171 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:56.171 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.171 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:56.171 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:56.171 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:56.171 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.171 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.172 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:56.172 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:56.172 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:56.172 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.172 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.172 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:56.172 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:56.172 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9299999999999999e-07 2023-11-25T04:23:56.172 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.172 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.172 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:56.172 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040790+0000", 2023-11-25T04:23:56.172 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5770000000000001e-06 2023-11-25T04:23:56.172 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.173 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.173 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:56.173 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040799+0000", 2023-11-25T04:23:56.173 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6439999999999995e-06 2023-11-25T04:23:56.173 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.173 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.173 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:56.173 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040806+0000", 2023-11-25T04:23:56.173 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0180000000000004e-06 2023-11-25T04:23:56.173 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.173 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.173 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:56.173 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040907+0000", 2023-11-25T04:23:56.174 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000100096 2023-11-25T04:23:56.174 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.174 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.174 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:56.174 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040925+0000", 2023-11-25T04:23:56.174 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8536999999999999e-05 2023-11-25T04:23:56.174 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.174 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.174 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:56.174 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042079+0000", 2023-11-25T04:23:56.174 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00115358 2023-11-25T04:23:56.174 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.174 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.175 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:56.175 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042107+0000", 2023-11-25T04:23:56.175 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7971e-05 2023-11-25T04:23:56.175 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:56.175 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:56.175 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:56.175 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.175 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.175 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop_reply(mds.0.3:4 3.1f e18/16 ondisk, result = 0)", 2023-11-25T04:23:56.175 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:56.175 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 69.042245152000007, 2023-11-25T04:23:56.175 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00044092900000000001, 2023-11-25T04:23:56.176 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:56.176 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:56.176 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:56.176 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.176 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:56.176 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:56.176 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:56.176 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.176 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.176 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:56.176 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:56.176 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:56.176 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.176 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.177 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:56.177 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042124+0000", 2023-11-25T04:23:56.177 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1800000000000005e-07 2023-11-25T04:23:56.177 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.177 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.177 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:56.177 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042128+0000", 2023-11-25T04:23:56.177 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2289999999999997e-06 2023-11-25T04:23:56.177 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.177 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.177 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:56.177 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042138+0000", 2023-11-25T04:23:56.177 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9729999999999992e-06 2023-11-25T04:23:56.178 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.178 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.178 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:56.178 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042145+0000", 2023-11-25T04:23:56.178 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.336e-06 2023-11-25T04:23:56.178 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.178 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.178 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:56.178 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042441+0000", 2023-11-25T04:23:56.178 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029650500000000002 2023-11-25T04:23:56.178 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.178 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.178 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:56.179 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042448+0000", 2023-11-25T04:23:56.179 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4100000000000002e-06 2023-11-25T04:23:56.179 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.179 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.179 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:56.179 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042564+0000", 2023-11-25T04:23:56.179 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000115758 2023-11-25T04:23:56.179 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:56.179 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:56.179 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:56.179 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.179 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.179 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing_reply(3.1f epoch 18/16 rep_tid 8 lcod 18'4)", 2023-11-25T04:23:56.180 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:56.180 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 69.039919740000002, 2023-11-25T04:23:56.180 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00038154699999999998, 2023-11-25T04:23:56.180 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:56.180 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:56.180 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:56.180 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.180 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:56.180 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:56.180 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:56.180 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.180 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.180 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:56.181 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:56.181 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:56.181 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.181 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.181 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:56.181 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:56.181 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2700000000000001e-07 2023-11-25T04:23:56.181 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.181 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.181 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:56.181 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044451+0000", 2023-11-25T04:23:56.181 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7460000000000001e-06 2023-11-25T04:23:56.181 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.182 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.182 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:56.182 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044455+0000", 2023-11-25T04:23:56.182 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5809999999999998e-06 2023-11-25T04:23:56.182 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.182 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.182 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:56.182 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044458+0000", 2023-11-25T04:23:56.182 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0170000000000001e-06 2023-11-25T04:23:56.182 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.182 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.182 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:56.182 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044734+0000", 2023-11-25T04:23:56.183 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00027582799999999999 2023-11-25T04:23:56.183 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.183 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.183 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:56.183 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044830+0000", 2023-11-25T04:23:56.183 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6248000000000004e-05 2023-11-25T04:23:56.183 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:56.183 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:56.183 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:56.183 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.183 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.183 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing_reply(3.b epoch 18/16 rep_tid 5 lcod 18'2)", 2023-11-25T04:23:56.183 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:56.184 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 69.039883079000006, 2023-11-25T04:23:56.184 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000344488, 2023-11-25T04:23:56.184 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:56.184 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:56.184 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:56.184 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.184 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:56.184 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:56.184 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:56.184 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.184 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.184 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:56.184 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:56.185 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:56.185 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.185 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.185 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:56.185 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:56.185 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.55e-07 2023-11-25T04:23:56.185 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.185 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.185 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:56.185 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044487+0000", 2023-11-25T04:23:56.185 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2470000000000001e-06 2023-11-25T04:23:56.185 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.185 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.186 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:56.186 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044491+0000", 2023-11-25T04:23:56.186 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9659999999999998e-06 2023-11-25T04:23:56.186 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.186 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.186 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:56.186 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044494+0000", 2023-11-25T04:23:56.186 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2310000000000001e-06 2023-11-25T04:23:56.186 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.186 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.186 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:56.186 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044756+0000", 2023-11-25T04:23:56.186 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00026172800000000003 2023-11-25T04:23:56.187 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.187 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.187 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:56.187 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044830+0000", 2023-11-25T04:23:56.187 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4160999999999997e-05 2023-11-25T04:23:56.187 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:56.187 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:56.187 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:56.187 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.187 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.187 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:39 3.6 3:654134d2:::mds0_openfiles.0:head [omap-clear,omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2023-11-25T04:23:56.187 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:56.188 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 63.980057037000002, 2023-11-25T04:23:56.188 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001788658, 2023-11-25T04:23:56.188 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:56.188 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:56.188 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:56.188 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.188 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:56.188 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:56.188 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:56.188 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.188 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.188 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:56.188 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:56.188 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:56.189 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.189 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.189 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:56.189 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104314+0000", 2023-11-25T04:23:56.189 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3099999999999999e-06 2023-11-25T04:23:56.189 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.189 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.189 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:56.189 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104325+0000", 2023-11-25T04:23:56.189 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1511999999999999e-05 2023-11-25T04:23:56.189 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.189 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.190 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:56.190 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104362+0000", 2023-11-25T04:23:56.190 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6903000000000002e-05 2023-11-25T04:23:56.190 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.190 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.190 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:56.190 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104387+0000", 2023-11-25T04:23:56.190 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4766e-05 2023-11-25T04:23:56.190 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.190 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.190 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:56.190 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104465+0000", 2023-11-25T04:23:56.191 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8119000000000002e-05 2023-11-25T04:23:56.191 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.191 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.191 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:56.191 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104557+0000", 2023-11-25T04:23:56.191 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2496000000000003e-05 2023-11-25T04:23:56.191 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.191 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.191 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:56.191 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104646+0000", 2023-11-25T04:23:56.191 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8818000000000002e-05 2023-11-25T04:23:56.191 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.191 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.192 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:56.192 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105201+0000", 2023-11-25T04:23:56.192 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00055506000000000002 2023-11-25T04:23:56.192 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.192 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.192 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:56.192 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105994+0000", 2023-11-25T04:23:56.192 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00079294700000000001 2023-11-25T04:23:56.192 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.192 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.192 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:56.192 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.106055+0000", 2023-11-25T04:23:56.192 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0566000000000003e-05 2023-11-25T04:23:56.193 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.193 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.193 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:56.193 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.106100+0000", 2023-11-25T04:23:56.193 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5160999999999997e-05 2023-11-25T04:23:56.193 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:56.193 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:56.193 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:56.193 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.193 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.193 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:43 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2023-11-25T04:23:56.193 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:56.194 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 48.979832041000002, 2023-11-25T04:23:56.194 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001751166, 2023-11-25T04:23:56.194 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:56.194 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:56.194 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:56.194 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.194 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:56.194 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:56.194 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:56.194 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.194 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.194 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:56.195 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:56.195 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:56.195 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.195 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.195 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:56.195 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104539+0000", 2023-11-25T04:23:56.195 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9529999999999999e-06 2023-11-25T04:23:56.195 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.195 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.195 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:56.195 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104557+0000", 2023-11-25T04:23:56.195 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7464000000000001e-05 2023-11-25T04:23:56.195 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.196 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.196 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:56.196 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104590+0000", 2023-11-25T04:23:56.196 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2966000000000003e-05 2023-11-25T04:23:56.196 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.196 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.196 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:56.196 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104608+0000", 2023-11-25T04:23:56.196 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8193e-05 2023-11-25T04:23:56.196 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.196 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.197 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:56.197 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104767+0000", 2023-11-25T04:23:56.197 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015882 2023-11-25T04:23:56.197 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.197 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.197 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:56.197 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104909+0000", 2023-11-25T04:23:56.197 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014244999999999999 2023-11-25T04:23:56.197 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.197 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.197 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:56.197 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.105059+0000", 2023-11-25T04:23:56.197 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014964399999999999 2023-11-25T04:23:56.198 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.198 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.198 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:56.198 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.105701+0000", 2023-11-25T04:23:56.198 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00064201099999999995 2023-11-25T04:23:56.198 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.198 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.198 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:56.198 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106233+0000", 2023-11-25T04:23:56.198 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00053178500000000001 2023-11-25T04:23:56.198 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.198 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.198 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:56.199 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106265+0000", 2023-11-25T04:23:56.199 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1986000000000002e-05 2023-11-25T04:23:56.199 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.199 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.199 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:56.199 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106288+0000", 2023-11-25T04:23:56.199 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2894000000000001e-05 2023-11-25T04:23:56.199 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:56.199 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:56.199 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:56.199 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.199 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.199 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:45 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e23)", 2023-11-25T04:23:56.199 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:23:56.200 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 43.979634001000001, 2023-11-25T04:23:56.200 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00098423299999999994, 2023-11-25T04:23:56.200 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:56.200 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:56.200 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:56.200 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.200 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:56.200 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:23:56.200 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:56.200 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.200 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.200 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:56.200 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:23:56.200 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:56.201 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.201 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.201 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:56.201 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104737+0000", 2023-11-25T04:23:56.201 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3309999999999998e-06 2023-11-25T04:23:56.201 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.201 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.201 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:56.201 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104748+0000", 2023-11-25T04:23:56.201 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1080999999999999e-05 2023-11-25T04:23:56.201 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.201 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.201 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:56.201 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104771+0000", 2023-11-25T04:23:56.202 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2707000000000001e-05 2023-11-25T04:23:56.202 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.202 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.202 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:56.202 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104800+0000", 2023-11-25T04:23:56.202 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9595e-05 2023-11-25T04:23:56.202 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.202 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.202 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:56.202 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104890+0000", 2023-11-25T04:23:56.202 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.9808999999999993e-05 2023-11-25T04:23:56.202 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.202 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.202 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:56.203 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105004+0000", 2023-11-25T04:23:56.203 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000114508 2023-11-25T04:23:56.203 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.203 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.203 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:56.203 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105634+0000", 2023-11-25T04:23:56.203 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00062985799999999996 2023-11-25T04:23:56.203 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.203 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.203 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:56.203 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105671+0000", 2023-11-25T04:23:56.203 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6640000000000002e-05 2023-11-25T04:23:56.203 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:56.204 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:56.204 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:56.204 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105719+0000", 2023-11-25T04:23:56.204 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7704000000000003e-05 2023-11-25T04:23:56.204 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:56.204 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:56.204 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:56.204 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:56.204 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:56.204 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:56.206 INFO:tasks.workunit.client.0.smithi067.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-11-25T04:23:56.206 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-11-25T04:23:56.206 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2023-11-25T04:23:56.207 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:23:56.207 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:56.204+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:56.262 INFO:tasks.workunit:Running workunits matching suites/fsx.sh on client.0... 2023-11-25T04:23:56.263 INFO:tasks.workunit:Running workunit suites/fsx.sh... 2023-11-25T04:23:56.264 DEBUG:teuthology.orchestra.run.smithi067:workunit test suites/fsx.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ec928368edcc5fc71884a56fd9fd821b148f4e30 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh 2023-11-25T04:23:56.306 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:56.305+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:56.317 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:56.317 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:23:56.317 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2023-11-25T04:23:56.318 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:56.326 INFO:tasks.workunit.client.0.smithi067.stderr:+ set -e 2023-11-25T04:23:56.326 INFO:tasks.workunit.client.0.smithi067.stderr:+ git clone https://git.ceph.com/xfstests-dev.git 2023-11-25T04:23:56.327 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:23:56.328 INFO:tasks.workunit.client.0.smithi067.stderr:Cloning into 'xfstests-dev'... 2023-11-25T04:23:56.408 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:56.406+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:56.431 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:56.431 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:23:56.431 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2023-11-25T04:23:56.431 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2023-11-25T04:23:56.431 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:56.440 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:23:56.483 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-11-25T04:23:56.508 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:56.507+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:56.546 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:56.547 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2023-11-25T04:23:56.547 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2023-11-25T04:23:56.547 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2023-11-25T04:23:56.547 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.547 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:155 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'109, mlcod=11'105)", 2023-11-25T04:23:56.547 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:56.547 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 78.759537711999997, 2023-11-25T04:23:56.547 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061719019999999999, 2023-11-25T04:23:56.547 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:56.548 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:56.548 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:56.548 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.548 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:56.548 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:56.548 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:56.548 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.548 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.548 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:56.548 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:56.548 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:56.548 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.548 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.549 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:56.549 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784884+0000", 2023-11-25T04:23:56.549 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3099999999999999e-07 2023-11-25T04:23:56.549 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.549 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.549 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:56.549 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784942+0000", 2023-11-25T04:23:56.549 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8570000000000003e-05 2023-11-25T04:23:56.549 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.549 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.549 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:56.549 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784949+0000", 2023-11-25T04:23:56.549 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0310000000000001e-06 2023-11-25T04:23:56.550 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.550 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.550 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:56.550 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784953+0000", 2023-11-25T04:23:56.550 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6320000000000001e-06 2023-11-25T04:23:56.550 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.550 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.550 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:56.550 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784993+0000", 2023-11-25T04:23:56.550 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9737000000000003e-05 2023-11-25T04:23:56.550 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.550 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.551 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:56.551 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785011+0000", 2023-11-25T04:23:56.551 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8349999999999999e-05 2023-11-25T04:23:56.551 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.551 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.551 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:56.551 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791024+0000", 2023-11-25T04:23:56.551 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006012997 2023-11-25T04:23:56.551 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.551 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.551 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:56.551 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791055+0000", 2023-11-25T04:23:56.551 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1254000000000003e-05 2023-11-25T04:23:56.552 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:56.552 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:56.552 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:56.552 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.552 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.552 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:156 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'110, mlcod=11'105)", 2023-11-25T04:23:56.552 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:56.552 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 78.758951663000005, 2023-11-25T04:23:56.552 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005616116, 2023-11-25T04:23:56.552 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:56.552 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:56.552 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:56.553 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.553 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:56.553 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:56.553 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:56.553 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.553 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.553 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:56.553 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:56.553 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:56.553 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.553 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.553 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:56.554 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785470+0000", 2023-11-25T04:23:56.554 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0699999999999998e-07 2023-11-25T04:23:56.554 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.554 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.554 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:56.554 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785474+0000", 2023-11-25T04:23:56.554 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.048e-06 2023-11-25T04:23:56.554 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.554 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.555 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:56.555 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785480+0000", 2023-11-25T04:23:56.555 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7729999999999996e-06 2023-11-25T04:23:56.555 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.555 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.555 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:56.555 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785483+0000", 2023-11-25T04:23:56.555 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1970000000000001e-06 2023-11-25T04:23:56.555 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.556 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.556 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:56.557 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787936+0000", 2023-11-25T04:23:56.557 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024529040000000001 2023-11-25T04:23:56.557 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.557 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.557 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:56.557 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787947+0000", 2023-11-25T04:23:56.557 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1669e-05 2023-11-25T04:23:56.557 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.557 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.557 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:56.558 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791057+0000", 2023-11-25T04:23:56.558 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031095200000000002 2023-11-25T04:23:56.558 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.558 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.558 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:56.558 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791086+0000", 2023-11-25T04:23:56.558 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8697999999999999e-05 2023-11-25T04:23:56.558 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:56.558 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:56.558 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:56.558 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.558 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.559 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:157 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'111, mlcod=11'106)", 2023-11-25T04:23:56.559 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:56.559 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 78.758359104999997, 2023-11-25T04:23:56.559 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050590280000000001, 2023-11-25T04:23:56.559 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:56.559 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:56.559 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:56.559 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.559 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:56.559 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:56.559 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:56.559 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.559 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.560 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:56.560 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:56.560 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:56.560 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.560 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.560 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:56.560 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786063+0000", 2023-11-25T04:23:56.560 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2600000000000002e-07 2023-11-25T04:23:56.560 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.560 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.560 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:56.560 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786069+0000", 2023-11-25T04:23:56.560 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8839999999999999e-06 2023-11-25T04:23:56.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.561 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:56.561 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786079+0000", 2023-11-25T04:23:56.561 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0124e-05 2023-11-25T04:23:56.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.561 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:56.561 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786085+0000", 2023-11-25T04:23:56.561 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6660000000000002e-06 2023-11-25T04:23:56.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.562 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.562 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:56.562 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788160+0000", 2023-11-25T04:23:56.562 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020747669999999999 2023-11-25T04:23:56.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.562 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.562 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:56.562 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788180+0000", 2023-11-25T04:23:56.562 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0208999999999999e-05 2023-11-25T04:23:56.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.562 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.563 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:56.563 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791087+0000", 2023-11-25T04:23:56.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029064630000000002 2023-11-25T04:23:56.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.563 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:56.563 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791121+0000", 2023-11-25T04:23:56.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4289000000000003e-05 2023-11-25T04:23:56.563 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:56.563 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:56.563 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:56.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.564 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:158 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'112, mlcod=11'106)", 2023-11-25T04:23:56.564 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:56.564 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 78.757942712000002, 2023-11-25T04:23:56.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046663030000000001, 2023-11-25T04:23:56.564 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:56.564 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:56.564 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:56.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:56.564 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:56.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:56.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.565 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:56.565 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:56.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:56.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.565 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:56.565 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786479+0000", 2023-11-25T04:23:56.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2799999999999996e-07 2023-11-25T04:23:56.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.565 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:56.565 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786498+0000", 2023-11-25T04:23:56.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9069e-05 2023-11-25T04:23:56.566 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.566 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.566 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:56.566 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786508+0000", 2023-11-25T04:23:56.566 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5810000000000003e-06 2023-11-25T04:23:56.566 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.566 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.566 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:56.566 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786512+0000", 2023-11-25T04:23:56.566 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4889999999999997e-06 2023-11-25T04:23:56.566 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.566 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:56.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788438+0000", 2023-11-25T04:23:56.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019257059999999999 2023-11-25T04:23:56.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:56.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788449+0000", 2023-11-25T04:23:56.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1419e-05 2023-11-25T04:23:56.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:56.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791122+0000", 2023-11-25T04:23:56.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002672822 2023-11-25T04:23:56.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:56.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791145+0000", 2023-11-25T04:23:56.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2589e-05 2023-11-25T04:23:56.568 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:56.568 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:56.568 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:56.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.568 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:206 1.0 e12/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 12'152, mlcod=12'148)", 2023-11-25T04:23:56.568 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:56.569 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 78.246174241999995, 2023-11-25T04:23:56.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046728189999999999, 2023-11-25T04:23:56.569 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:56.569 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:56.569 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:56.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:56.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:56.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:56.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:56.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:56.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:56.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.570 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:56.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298248+0000", 2023-11-25T04:23:56.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3600000000000003e-07 2023-11-25T04:23:56.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.570 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:56.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298278+0000", 2023-11-25T04:23:56.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0474e-05 2023-11-25T04:23:56.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:56.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298288+0000", 2023-11-25T04:23:56.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0156000000000001e-05 2023-11-25T04:23:56.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:56.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298293+0000", 2023-11-25T04:23:56.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7400000000000004e-06 2023-11-25T04:23:56.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:56.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298333+0000", 2023-11-25T04:23:56.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9666000000000003e-05 2023-11-25T04:23:56.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.573 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:56.573 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298351+0000", 2023-11-25T04:23:56.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8544999999999999e-05 2023-11-25T04:23:56.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:56.574 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302891+0000", 2023-11-25T04:23:56.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045398519999999996 2023-11-25T04:23:56.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:56.574 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302920+0000", 2023-11-25T04:23:56.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8750000000000001e-05 2023-11-25T04:23:56.574 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:56.574 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:56.574 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:56.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.575 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:14 3.15 3:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:56.575 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:56.575 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 69.511039885000002, 2023-11-25T04:23:56.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010990437, 2023-11-25T04:23:56.575 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:56.575 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:56.575 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:56.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:56.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:56.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:56.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.576 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:56.576 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:56.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:56.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.576 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:56.576 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033383+0000", 2023-11-25T04:23:56.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9870000000000002e-06 2023-11-25T04:23:56.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.576 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:56.576 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033398+0000", 2023-11-25T04:23:56.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4569999999999999e-05 2023-11-25T04:23:56.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.577 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:56.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033411+0000", 2023-11-25T04:23:56.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3543999999999999e-05 2023-11-25T04:23:56.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.577 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:56.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033423+0000", 2023-11-25T04:23:56.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1905999999999999e-05 2023-11-25T04:23:56.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.578 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:56.578 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034433+0000", 2023-11-25T04:23:56.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00101022 2023-11-25T04:23:56.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.578 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:56.578 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034539+0000", 2023-11-25T04:23:56.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010597199999999999 2023-11-25T04:23:56.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.578 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:56.578 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034667+0000", 2023-11-25T04:23:56.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012765500000000001 2023-11-25T04:23:56.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:56.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037803+0000", 2023-11-25T04:23:56.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031362220000000001 2023-11-25T04:23:56.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:56.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:23:56.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0064964669999999997 2023-11-25T04:23:56.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:56.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044331+0000", 2023-11-25T04:23:56.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1544999999999997e-05 2023-11-25T04:23:56.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:56.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044372+0000", 2023-11-25T04:23:56.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0349000000000003e-05 2023-11-25T04:23:56.580 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:56.580 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:56.580 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:56.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.581 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:15 3.15 3:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:56.581 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:56.581 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 69.510380235, 2023-11-25T04:23:56.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010464362, 2023-11-25T04:23:56.581 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:56.581 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:56.581 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:56.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.581 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:56.581 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:56.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:56.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:56.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:56.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:56.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:56.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034042+0000", 2023-11-25T04:23:56.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0999999999999997e-07 2023-11-25T04:23:56.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:56.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034048+0000", 2023-11-25T04:23:56.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0390000000000001e-06 2023-11-25T04:23:56.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.583 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:56.583 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034059+0000", 2023-11-25T04:23:56.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1182999999999999e-05 2023-11-25T04:23:56.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.583 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:56.583 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034067+0000", 2023-11-25T04:23:56.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1559999999999995e-06 2023-11-25T04:23:56.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.584 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:56.584 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035005+0000", 2023-11-25T04:23:56.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00093747399999999999 2023-11-25T04:23:56.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.584 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:56.584 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035077+0000", 2023-11-25T04:23:56.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2747e-05 2023-11-25T04:23:56.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.584 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:56.584 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035219+0000", 2023-11-25T04:23:56.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000141877 2023-11-25T04:23:56.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:56.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037810+0000", 2023-11-25T04:23:56.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002590752 2023-11-25T04:23:56.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:56.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044437+0000", 2023-11-25T04:23:56.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066270110000000004 2023-11-25T04:23:56.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:56.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044481+0000", 2023-11-25T04:23:56.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4106000000000001e-05 2023-11-25T04:23:56.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:56.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044505+0000", 2023-11-25T04:23:56.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4207e-05 2023-11-25T04:23:56.586 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:56.586 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:56.586 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:56.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.587 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:16 3.1d 3:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:56.587 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:56.587 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 69.510338387000004, 2023-11-25T04:23:56.587 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010413923, 2023-11-25T04:23:56.587 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:56.587 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:56.587 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:56.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.587 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:56.587 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:56.587 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:56.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.588 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:56.588 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:56.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:56.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.588 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:56.588 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:56.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4499999999999997e-07 2023-11-25T04:23:56.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.588 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:56.588 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034087+0000", 2023-11-25T04:23:56.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9770000000000002e-06 2023-11-25T04:23:56.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:56.589 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034094+0000", 2023-11-25T04:23:56.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8809999999999998e-06 2023-11-25T04:23:56.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:56.589 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034100+0000", 2023-11-25T04:23:56.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8440000000000003e-06 2023-11-25T04:23:56.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:56.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034993+0000", 2023-11-25T04:23:56.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00089336799999999996 2023-11-25T04:23:56.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:56.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035099+0000", 2023-11-25T04:23:56.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106037 2023-11-25T04:23:56.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:56.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035276+0000", 2023-11-25T04:23:56.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017631799999999999 2023-11-25T04:23:56.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.591 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:56.591 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037815+0000", 2023-11-25T04:23:56.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025395959999999999 2023-11-25T04:23:56.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.591 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:56.591 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044420+0000", 2023-11-25T04:23:56.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066045749999999997 2023-11-25T04:23:56.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.591 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:56.592 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044463+0000", 2023-11-25T04:23:56.592 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3205999999999999e-05 2023-11-25T04:23:56.592 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.592 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.592 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:56.592 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044497+0000", 2023-11-25T04:23:56.592 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3476000000000003e-05 2023-11-25T04:23:56.592 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:56.592 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:56.592 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:56.592 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.592 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.592 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:17 3.1d 3:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:56.593 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:56.593 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 69.510313178000004, 2023-11-25T04:23:56.593 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01127356, 2023-11-25T04:23:56.593 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:56.593 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:56.593 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:56.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:56.593 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:56.593 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:56.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:56.594 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:56.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:56.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.594 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:56.594 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:56.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1100000000000001e-07 2023-11-25T04:23:56.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.594 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:56.594 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034112+0000", 2023-11-25T04:23:56.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8600000000000003e-06 2023-11-25T04:23:56.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:56.595 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034120+0000", 2023-11-25T04:23:56.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7700000000000001e-06 2023-11-25T04:23:56.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:56.595 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034127+0000", 2023-11-25T04:23:56.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0999999999999998e-06 2023-11-25T04:23:56.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:56.596 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035601+0000", 2023-11-25T04:23:56.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014739410000000001 2023-11-25T04:23:56.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.596 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.596 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:56.596 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035685+0000", 2023-11-25T04:23:56.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3677999999999994e-05 2023-11-25T04:23:56.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.596 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.596 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:56.596 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035827+0000", 2023-11-25T04:23:56.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014232599999999999 2023-11-25T04:23:56.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.597 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.597 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:56.597 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038725+0000", 2023-11-25T04:23:56.597 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028982779999999998 2023-11-25T04:23:56.597 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.597 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.597 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:56.597 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045323+0000", 2023-11-25T04:23:56.597 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0065980780000000003 2023-11-25T04:23:56.597 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.597 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.597 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:56.597 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045354+0000", 2023-11-25T04:23:56.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0463e-05 2023-11-25T04:23:56.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:56.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045381+0000", 2023-11-25T04:23:56.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7654999999999999e-05 2023-11-25T04:23:56.598 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:56.598 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:56.598 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:56.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.598 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:22 3.e 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:56.598 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:56.599 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 69.510286765000004, 2023-11-25T04:23:56.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010133422, 2023-11-25T04:23:56.599 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:56.599 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:56.599 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:56.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:56.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:56.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:56.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:56.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:56.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:56.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.600 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:56.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034135+0000", 2023-11-25T04:23:56.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-07 2023-11-25T04:23:56.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.600 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:56.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034138+0000", 2023-11-25T04:23:56.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6009999999999999e-06 2023-11-25T04:23:56.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:56.601 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034145+0000", 2023-11-25T04:23:56.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1169999999999998e-06 2023-11-25T04:23:56.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:56.601 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034150+0000", 2023-11-25T04:23:56.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9080000000000003e-06 2023-11-25T04:23:56.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:56.601 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034209+0000", 2023-11-25T04:23:56.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8149000000000002e-05 2023-11-25T04:23:56.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:56.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034415+0000", 2023-11-25T04:23:56.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020637699999999999 2023-11-25T04:23:56.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:56.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034584+0000", 2023-11-25T04:23:56.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016870399999999999 2023-11-25T04:23:56.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:56.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.036979+0000", 2023-11-25T04:23:56.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023957509999999998 2023-11-25T04:23:56.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:56.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044161+0000", 2023-11-25T04:23:56.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0071811590000000003 2023-11-25T04:23:56.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:56.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044224+0000", 2023-11-25T04:23:56.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3454000000000006e-05 2023-11-25T04:23:56.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:56.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044268+0000", 2023-11-25T04:23:56.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3761999999999999e-05 2023-11-25T04:23:56.604 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:56.604 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:56.604 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:56.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.604 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:23 3.e 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:56.605 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:56.605 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 69.510255356000002, 2023-11-25T04:23:56.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010281676, 2023-11-25T04:23:56.605 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:56.605 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:56.605 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:56.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.605 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:56.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:56.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:56.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.605 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:56.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:56.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:56.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:56.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:56.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5799999999999999e-07 2023-11-25T04:23:56.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:56.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034169+0000", 2023-11-25T04:23:56.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1879999999999998e-06 2023-11-25T04:23:56.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:56.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034177+0000", 2023-11-25T04:23:56.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7020000000000001e-06 2023-11-25T04:23:56.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:56.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034184+0000", 2023-11-25T04:23:56.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3910000000000003e-06 2023-11-25T04:23:56.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:56.608 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034924+0000", 2023-11-25T04:23:56.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00074032999999999998 2023-11-25T04:23:56.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:56.608 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034991+0000", 2023-11-25T04:23:56.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7347000000000004e-05 2023-11-25T04:23:56.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:56.608 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035096+0000", 2023-11-25T04:23:56.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010488 2023-11-25T04:23:56.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:56.609 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037588+0000", 2023-11-25T04:23:56.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024918309999999999 2023-11-25T04:23:56.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:56.609 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044367+0000", 2023-11-25T04:23:56.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067787589999999997 2023-11-25T04:23:56.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:56.609 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044415+0000", 2023-11-25T04:23:56.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7803000000000003e-05 2023-11-25T04:23:56.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.610 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:56.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044447+0000", 2023-11-25T04:23:56.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2886999999999999e-05 2023-11-25T04:23:56.610 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:56.610 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:56.610 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:56.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.610 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:30 3.15 3:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:56.611 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:56.611 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 69.510196915999998, 2023-11-25T04:23:56.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010365070000000001, 2023-11-25T04:23:56.611 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:56.611 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:56.611 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:56.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:56.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:56.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:56.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:56.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:56.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:56.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:56.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034225+0000", 2023-11-25T04:23:56.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7400000000000002e-07 2023-11-25T04:23:56.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:56.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034229+0000", 2023-11-25T04:23:56.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3100000000000002e-06 2023-11-25T04:23:56.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.613 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:56.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034239+0000", 2023-11-25T04:23:56.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9140000000000003e-06 2023-11-25T04:23:56.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.613 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:56.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034246+0000", 2023-11-25T04:23:56.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0149999999999998e-06 2023-11-25T04:23:56.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:56.614 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035572+0000", 2023-11-25T04:23:56.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001325655 2023-11-25T04:23:56.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:56.614 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035730+0000", 2023-11-25T04:23:56.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015827899999999999 2023-11-25T04:23:56.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:56.614 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035857+0000", 2023-11-25T04:23:56.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012644699999999999 2023-11-25T04:23:56.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:56.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038737+0000", 2023-11-25T04:23:56.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002880077 2023-11-25T04:23:56.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:56.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044540+0000", 2023-11-25T04:23:56.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058034239999999997 2023-11-25T04:23:56.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:56.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044569+0000", 2023-11-25T04:23:56.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8388999999999999e-05 2023-11-25T04:23:56.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:56.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044589+0000", 2023-11-25T04:23:56.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0686000000000002e-05 2023-11-25T04:23:56.616 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:56.616 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:56.616 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:56.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.617 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:33 3.1b 3:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:56.617 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:56.617 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 69.510151418000007, 2023-11-25T04:23:56.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099754049999999997, 2023-11-25T04:23:56.617 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:56.617 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:23:56.617 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:56.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.617 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:56.617 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:56.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:56.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.618 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:56.618 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:56.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:56.618 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.618 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:56.618 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:56.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4799999999999998e-07 2023-11-25T04:23:56.618 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.618 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:56.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034273+0000", 2023-11-25T04:23:56.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7819999999999998e-06 2023-11-25T04:23:56.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:56.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034285+0000", 2023-11-25T04:23:56.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2058e-05 2023-11-25T04:23:56.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:56.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034291+0000", 2023-11-25T04:23:56.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0680000000000002e-06 2023-11-25T04:23:56.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:56.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034370+0000", 2023-11-25T04:23:56.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8452e-05 2023-11-25T04:23:56.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:56.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034477+0000", 2023-11-25T04:23:56.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107471 2023-11-25T04:23:56.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:56.621 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044245+0000", 2023-11-25T04:23:56.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0097679259999999997 2023-11-25T04:23:56.621 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:56.621 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:56.621 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:56.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.621 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:36 3.1a 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:56.621 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:56.622 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 69.510104745000007, 2023-11-25T04:23:56.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058179099999999999, 2023-11-25T04:23:56.622 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:56.622 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:23:56.622 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:56.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.622 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:56.622 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:56.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:56.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.623 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:56.623 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:56.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:56.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.623 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:56.623 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034317+0000", 2023-11-25T04:23:56.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1999999999999999e-07 2023-11-25T04:23:56.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:56.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034321+0000", 2023-11-25T04:23:56.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.233e-06 2023-11-25T04:23:56.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:56.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034337+0000", 2023-11-25T04:23:56.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5308999999999999e-05 2023-11-25T04:23:56.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:56.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034354+0000", 2023-11-25T04:23:56.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7411999999999999e-05 2023-11-25T04:23:56.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:56.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034465+0000", 2023-11-25T04:23:56.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011154699999999999 2023-11-25T04:23:56.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:56.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034560+0000", 2023-11-25T04:23:56.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4186999999999998e-05 2023-11-25T04:23:56.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:56.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.040134+0000", 2023-11-25T04:23:56.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055746019999999997 2023-11-25T04:23:56.626 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:56.626 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:56.626 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:56.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.626 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:9 3.10 e18/16 3:09401a81:::601.00000000:head v 18'4, mlcod=0'0)", 2023-11-25T04:23:56.626 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:56.627 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 69.501210436999997, 2023-11-25T04:23:56.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061641020000000003, 2023-11-25T04:23:56.627 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:56.627 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:56.627 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:56.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.627 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:56.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:56.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:56.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.627 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:56.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:56.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:56.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:56.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:56.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5300000000000001e-07 2023-11-25T04:23:56.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:56.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043222+0000", 2023-11-25T04:23:56.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.082e-05 2023-11-25T04:23:56.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:56.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043228+0000", 2023-11-25T04:23:56.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6699999999999997e-06 2023-11-25T04:23:56.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:56.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043236+0000", 2023-11-25T04:23:56.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8189999999999992e-06 2023-11-25T04:23:56.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:56.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043507+0000", 2023-11-25T04:23:56.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000271012 2023-11-25T04:23:56.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:56.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043530+0000", 2023-11-25T04:23:56.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2659000000000001e-05 2023-11-25T04:23:56.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:56.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049321+0000", 2023-11-25T04:23:56.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057915129999999999 2023-11-25T04:23:56.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:56.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049375+0000", 2023-11-25T04:23:56.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3455999999999998e-05 2023-11-25T04:23:56.631 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:56.631 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:56.631 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:56.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.631 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:1 3.14 e18/16 3:292cf221:::200.00000000:head v 18'3, mlcod=0'0)", 2023-11-25T04:23:56.631 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:56.631 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 69.501173723999997, 2023-11-25T04:23:56.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0060983599999999997, 2023-11-25T04:23:56.631 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:56.631 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:56.632 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:56.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:56.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:56.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:56.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:56.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:56.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:56.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:56.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043248+0000", 2023-11-25T04:23:56.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9399999999999995e-07 2023-11-25T04:23:56.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:56.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043252+0000", 2023-11-25T04:23:56.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1799999999999998e-06 2023-11-25T04:23:56.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:56.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043259+0000", 2023-11-25T04:23:56.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0049999999999998e-06 2023-11-25T04:23:56.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:56.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043265+0000", 2023-11-25T04:23:56.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0859999999999998e-06 2023-11-25T04:23:56.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:56.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043311+0000", 2023-11-25T04:23:56.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6082999999999999e-05 2023-11-25T04:23:56.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:56.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043330+0000", 2023-11-25T04:23:56.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8556e-05 2023-11-25T04:23:56.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:56.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049303+0000", 2023-11-25T04:23:56.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059733470000000004 2023-11-25T04:23:56.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:56.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049346+0000", 2023-11-25T04:23:56.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2608999999999998e-05 2023-11-25T04:23:56.636 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:56.636 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:56.636 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:56.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.636 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:3 3.14 e18/16 3:287034c4:::1.00000000.inode:head v 18'4, mlcod=0'0)", 2023-11-25T04:23:56.636 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:56.636 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 69.500657218000001, 2023-11-25T04:23:56.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057040090000000003, 2023-11-25T04:23:56.636 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:56.636 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:56.636 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:56.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:56.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:56.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:56.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:56.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:56.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:56.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:56.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:56.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9799999999999999e-07 2023-11-25T04:23:56.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:56.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043785+0000", 2023-11-25T04:23:56.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0721999999999999e-05 2023-11-25T04:23:56.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:56.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043793+0000", 2023-11-25T04:23:56.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5839999999999995e-06 2023-11-25T04:23:56.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:56.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043803+0000", 2023-11-25T04:23:56.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0729999999999994e-06 2023-11-25T04:23:56.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:56.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043843+0000", 2023-11-25T04:23:56.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0747999999999997e-05 2023-11-25T04:23:56.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:56.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043856+0000", 2023-11-25T04:23:56.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2697000000000001e-05 2023-11-25T04:23:56.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:56.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049426+0000", 2023-11-25T04:23:56.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055704409999999998 2023-11-25T04:23:56.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:56.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049468+0000", 2023-11-25T04:23:56.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1446000000000003e-05 2023-11-25T04:23:56.640 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:56.640 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:56.640 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:56.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.641 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "pg_update_log_missing(3.14 epoch 18/16 rep_tid 171 entries 18'5 (0'0) error 3:2e2fa760:::200.00000001:head by mds.0.3:32 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:23:56.641 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:56.641 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 69.500496255000002, 2023-11-25T04:23:56.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005556938, 2023-11-25T04:23:56.641 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:56.641 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2023-11-25T04:23:56.641 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:56.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:56.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:56.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:56.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:56.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:56.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:56.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:56.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:56.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.03e-07 2023-11-25T04:23:56.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:56.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043931+0000", 2023-11-25T04:23:56.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1079999999999998e-06 2023-11-25T04:23:56.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:56.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043944+0000", 2023-11-25T04:23:56.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2852e-05 2023-11-25T04:23:56.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.644 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:56.644 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043955+0000", 2023-11-25T04:23:56.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1037e-05 2023-11-25T04:23:56.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.644 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:56.644 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044088+0000", 2023-11-25T04:23:56.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013269599999999999 2023-11-25T04:23:56.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.644 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:56.644 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049482+0000", 2023-11-25T04:23:56.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053940419999999999 2023-11-25T04:23:56.645 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:56.645 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:56.645 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:56.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.645 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:10 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'5, mlcod=0'0)", 2023-11-25T04:23:56.645 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:56.645 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 69.500406226999999, 2023-11-25T04:23:56.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054493099999999997, 2023-11-25T04:23:56.645 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:56.645 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:56.646 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:56.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:56.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:56.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:56.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:56.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:56.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:56.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:56.647 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:56.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6100000000000002e-07 2023-11-25T04:23:56.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.647 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:56.647 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044019+0000", 2023-11-25T04:23:56.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.839e-06 2023-11-25T04:23:56.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.647 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:56.647 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044026+0000", 2023-11-25T04:23:56.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5869999999999998e-06 2023-11-25T04:23:56.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.648 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:56.648 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044031+0000", 2023-11-25T04:23:56.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4909999999999996e-06 2023-11-25T04:23:56.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.648 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:56.648 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044070+0000", 2023-11-25T04:23:56.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8829999999999999e-05 2023-11-25T04:23:56.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.648 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:56.649 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044081+0000", 2023-11-25T04:23:56.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1447999999999999e-05 2023-11-25T04:23:56.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.649 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:56.649 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049406+0000", 2023-11-25T04:23:56.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053246480000000004 2023-11-25T04:23:56.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.649 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:56.649 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049464+0000", 2023-11-25T04:23:56.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8205999999999998e-05 2023-11-25T04:23:56.649 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:56.650 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:56.650 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:56.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.650 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:11 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'6, mlcod=0'0)", 2023-11-25T04:23:56.650 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:56.650 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 69.500204963000002, 2023-11-25T04:23:56.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052673499999999996, 2023-11-25T04:23:56.650 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:56.650 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:56.650 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:56.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:56.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:56.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:56.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:56.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:56.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:56.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:56.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:56.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0199999999999998e-07 2023-11-25T04:23:56.652 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.652 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.652 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:56.652 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044220+0000", 2023-11-25T04:23:56.652 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7720000000000001e-06 2023-11-25T04:23:56.652 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.652 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.652 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:56.652 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044226+0000", 2023-11-25T04:23:56.652 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8579999999999996e-06 2023-11-25T04:23:56.652 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.652 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.652 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:56.652 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044232+0000", 2023-11-25T04:23:56.653 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1700000000000002e-06 2023-11-25T04:23:56.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.653 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:56.653 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044274+0000", 2023-11-25T04:23:56.653 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1254999999999997e-05 2023-11-25T04:23:56.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.653 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:56.653 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:23:56.653 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6880999999999999e-05 2023-11-25T04:23:56.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:56.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049466+0000", 2023-11-25T04:23:56.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051659219999999999 2023-11-25T04:23:56.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:56.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:56.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:56.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049483+0000", 2023-11-25T04:23:56.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.719e-05 2023-11-25T04:23:56.654 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:56.654 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:56.654 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:56.654 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:56.654 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:56.655 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:56.655 INFO:teuthology.orchestra.run.smithi067.stderr:Error ENXIO: problem getting command descriptions from osd.0 2023-11-25T04:23:56.656 INFO:teuthology.orchestra.run.smithi067.stdout:osd.1: {} 2023-11-25T04:23:56.657 INFO:teuthology.orchestra.run.smithi067.stderr:osd.1: osd_enable_op_tracker = '' 2023-11-25T04:23:56.657 INFO:teuthology.orchestra.run.smithi067.stdout:osd.2: {} 2023-11-25T04:23:56.657 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:23:56.658 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-11-25T04:23:56.658 INFO:teuthology.orchestra.run.smithi067.stderr:osd.2: osd_enable_op_tracker = '' 2023-11-25T04:23:56.658 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-11-25T04:23:56.659 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:56.654+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:56.757 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:56.756+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:56.765 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:56.765 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:56.774 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:23:56.859 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:56.857+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:56.879 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:56.879 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:56.888 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:23:56.959 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:56.958+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:57.033 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:57.034 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:57.043 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:23:57.087 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:57.086+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:57.151 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:57.151 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:57.160 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:23:57.161 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:57.160+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:57.262 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:57.261+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:57.306 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:57.307 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:57.316 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:23:57.399 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:57.398+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:57.461 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:57.461 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:57.463 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:57.462+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:57.470 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:23:57.565 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:57.562+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:57.577 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:57.577 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:57.591 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:23:57.665 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:57.664+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:57.737 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:57.737 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:57.747 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:23:57.766 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:57.764+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:57.853 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:57.853 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:57.862 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:23:57.906 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:57.905+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:57.967 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:57.966+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:57.967 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:57.967 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:57.976 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:23:58.017 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2023-11-25T04:23:58.068 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:58.067+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:58.122 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:58.122 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:58.130 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:23:58.171 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:58.170+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:58.270 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:58.269+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:58.275 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:58.275 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:58.284 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:23:58.370 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:58.369+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:58.393 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:58.393 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:58.402 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:23:58.485 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:58.484+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:58.547 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:58.547 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:58.556 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:23:58.598 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:58.596+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:58.660 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-11-25T04:23:58.672 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:58.672+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:58.702 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:23:58.702 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:23:58.711 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:23:58.773 INFO:teuthology.orchestra.run.smithi067.stderr:Error ENXIO: problem getting command descriptions from osd.0 2023-11-25T04:23:58.773 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:58.772+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:58.791 INFO:teuthology.orchestra.run.smithi067.stdout:osd.1: {} 2023-11-25T04:23:58.792 INFO:teuthology.orchestra.run.smithi067.stderr:osd.1: osd_enable_op_tracker = '' 2023-11-25T04:23:58.810 INFO:teuthology.orchestra.run.smithi067.stdout:osd.2: {} 2023-11-25T04:23:58.810 INFO:teuthology.orchestra.run.smithi067.stderr:osd.2: osd_enable_op_tracker = '' 2023-11-25T04:23:58.813 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-11-25T04:23:58.813 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-11-25T04:23:58.818 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:58.818 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:23:58.818 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2023-11-25T04:23:58.819 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:58.827 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:23:58.875 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:58.873+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:58.931 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:58.931 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:23:58.932 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2023-11-25T04:23:58.932 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2023-11-25T04:23:58.932 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:58.940 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:23:58.981 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:58.980+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:58.999 INFO:teuthology.orchestra.run.smithi067.stderr:noscrub is unset 2023-11-25T04:23:59.076 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:59.075+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:59.088 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:59.088 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2023-11-25T04:23:59.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2023-11-25T04:23:59.088 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2023-11-25T04:23:59.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.089 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:155 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'109, mlcod=11'105)", 2023-11-25T04:23:59.089 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:59.089 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 81.301265709000006, 2023-11-25T04:23:59.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061719019999999999, 2023-11-25T04:23:59.089 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:59.089 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:59.089 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:59.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:59.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:59.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:59.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:59.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:23:59.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:59.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:59.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784884+0000", 2023-11-25T04:23:59.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3099999999999999e-07 2023-11-25T04:23:59.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:59.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784942+0000", 2023-11-25T04:23:59.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8570000000000003e-05 2023-11-25T04:23:59.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:59.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784949+0000", 2023-11-25T04:23:59.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0310000000000001e-06 2023-11-25T04:23:59.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:59.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784953+0000", 2023-11-25T04:23:59.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6320000000000001e-06 2023-11-25T04:23:59.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.092 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:59.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784993+0000", 2023-11-25T04:23:59.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9737000000000003e-05 2023-11-25T04:23:59.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.092 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:59.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785011+0000", 2023-11-25T04:23:59.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8349999999999999e-05 2023-11-25T04:23:59.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:59.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791024+0000", 2023-11-25T04:23:59.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006012997 2023-11-25T04:23:59.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:59.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791055+0000", 2023-11-25T04:23:59.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1254000000000003e-05 2023-11-25T04:23:59.093 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:59.093 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:59.093 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:59.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.094 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:156 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'110, mlcod=11'105)", 2023-11-25T04:23:59.094 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:59.094 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 81.30067966, 2023-11-25T04:23:59.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005616116, 2023-11-25T04:23:59.094 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:59.094 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:59.094 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:59.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:59.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:59.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:59.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:59.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:23:59.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:59.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:59.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785470+0000", 2023-11-25T04:23:59.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0699999999999998e-07 2023-11-25T04:23:59.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:59.096 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785474+0000", 2023-11-25T04:23:59.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.048e-06 2023-11-25T04:23:59.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:59.096 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785480+0000", 2023-11-25T04:23:59.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7729999999999996e-06 2023-11-25T04:23:59.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:59.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785483+0000", 2023-11-25T04:23:59.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1970000000000001e-06 2023-11-25T04:23:59.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:59.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787936+0000", 2023-11-25T04:23:59.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024529040000000001 2023-11-25T04:23:59.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:59.097 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787947+0000", 2023-11-25T04:23:59.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1669e-05 2023-11-25T04:23:59.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.098 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:59.098 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791057+0000", 2023-11-25T04:23:59.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031095200000000002 2023-11-25T04:23:59.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.098 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:59.098 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791086+0000", 2023-11-25T04:23:59.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8697999999999999e-05 2023-11-25T04:23:59.098 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:59.099 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:59.099 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:59.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.099 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:157 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'111, mlcod=11'106)", 2023-11-25T04:23:59.099 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:59.099 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 81.300087102000006, 2023-11-25T04:23:59.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050590280000000001, 2023-11-25T04:23:59.099 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:59.099 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:59.099 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:59.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.099 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:59.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:59.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:59.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:59.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:23:59.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:59.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:59.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786063+0000", 2023-11-25T04:23:59.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2600000000000002e-07 2023-11-25T04:23:59.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:59.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786069+0000", 2023-11-25T04:23:59.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8839999999999999e-06 2023-11-25T04:23:59.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:59.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786079+0000", 2023-11-25T04:23:59.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0124e-05 2023-11-25T04:23:59.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.102 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:59.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786085+0000", 2023-11-25T04:23:59.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6660000000000002e-06 2023-11-25T04:23:59.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.102 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:59.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788160+0000", 2023-11-25T04:23:59.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020747669999999999 2023-11-25T04:23:59.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.103 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:59.103 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788180+0000", 2023-11-25T04:23:59.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0208999999999999e-05 2023-11-25T04:23:59.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.103 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:59.103 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791087+0000", 2023-11-25T04:23:59.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029064630000000002 2023-11-25T04:23:59.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.104 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:59.104 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791121+0000", 2023-11-25T04:23:59.104 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4289000000000003e-05 2023-11-25T04:23:59.104 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:59.104 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:59.104 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:59.104 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.104 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:158 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'112, mlcod=11'106)", 2023-11-25T04:23:59.104 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:59.104 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 81.299670708999997, 2023-11-25T04:23:59.104 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046663030000000001, 2023-11-25T04:23:59.105 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:59.105 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:59.105 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:59.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.105 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:59.105 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:59.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:59.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.105 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:59.105 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:23:59.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:59.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.106 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:59.106 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786479+0000", 2023-11-25T04:23:59.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2799999999999996e-07 2023-11-25T04:23:59.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.106 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:59.106 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786498+0000", 2023-11-25T04:23:59.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9069e-05 2023-11-25T04:23:59.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.107 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:59.107 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786508+0000", 2023-11-25T04:23:59.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5810000000000003e-06 2023-11-25T04:23:59.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.107 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:59.107 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786512+0000", 2023-11-25T04:23:59.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4889999999999997e-06 2023-11-25T04:23:59.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.110 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:59.110 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788438+0000", 2023-11-25T04:23:59.110 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019257059999999999 2023-11-25T04:23:59.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.110 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.110 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:59.110 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788449+0000", 2023-11-25T04:23:59.110 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1419e-05 2023-11-25T04:23:59.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.111 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.111 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:59.111 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791122+0000", 2023-11-25T04:23:59.111 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002672822 2023-11-25T04:23:59.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.111 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.111 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:59.111 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791145+0000", 2023-11-25T04:23:59.111 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2589e-05 2023-11-25T04:23:59.112 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:59.112 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:59.112 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:59.112 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.112 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.112 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:206 1.0 e12/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 12'152, mlcod=12'148)", 2023-11-25T04:23:59.112 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:59.112 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 80.787902239000005, 2023-11-25T04:23:59.112 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046728189999999999, 2023-11-25T04:23:59.112 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:59.112 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:59.112 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:59.112 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.113 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:59.113 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:59.113 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:59.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.113 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:59.113 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:23:59.113 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:59.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.113 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:59.113 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298248+0000", 2023-11-25T04:23:59.114 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3600000000000003e-07 2023-11-25T04:23:59.114 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.114 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.115 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:59.115 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298278+0000", 2023-11-25T04:23:59.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0474e-05 2023-11-25T04:23:59.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.115 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:59.115 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298288+0000", 2023-11-25T04:23:59.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0156000000000001e-05 2023-11-25T04:23:59.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.115 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:59.115 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298293+0000", 2023-11-25T04:23:59.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7400000000000004e-06 2023-11-25T04:23:59.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.116 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:59.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298333+0000", 2023-11-25T04:23:59.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9666000000000003e-05 2023-11-25T04:23:59.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.116 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:59.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298351+0000", 2023-11-25T04:23:59.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8544999999999999e-05 2023-11-25T04:23:59.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.117 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:59.117 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302891+0000", 2023-11-25T04:23:59.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045398519999999996 2023-11-25T04:23:59.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.117 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:59.117 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302920+0000", 2023-11-25T04:23:59.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8750000000000001e-05 2023-11-25T04:23:59.117 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:59.117 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:59.117 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:59.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.118 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:14 3.15 3:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:59.118 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:59.118 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 72.052767881999998, 2023-11-25T04:23:59.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010990437, 2023-11-25T04:23:59.118 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:59.118 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:59.118 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:59.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.118 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:59.118 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:59.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:59.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:59.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:23:59.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:59.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:59.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033383+0000", 2023-11-25T04:23:59.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9870000000000002e-06 2023-11-25T04:23:59.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:59.120 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033398+0000", 2023-11-25T04:23:59.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4569999999999999e-05 2023-11-25T04:23:59.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.120 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:59.120 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033411+0000", 2023-11-25T04:23:59.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3543999999999999e-05 2023-11-25T04:23:59.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.120 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:59.120 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033423+0000", 2023-11-25T04:23:59.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1905999999999999e-05 2023-11-25T04:23:59.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.121 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:59.121 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034433+0000", 2023-11-25T04:23:59.121 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00101022 2023-11-25T04:23:59.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.121 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:59.121 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034539+0000", 2023-11-25T04:23:59.121 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010597199999999999 2023-11-25T04:23:59.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.121 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:59.121 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034667+0000", 2023-11-25T04:23:59.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012765500000000001 2023-11-25T04:23:59.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.122 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:59.122 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037803+0000", 2023-11-25T04:23:59.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031362220000000001 2023-11-25T04:23:59.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.122 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:59.122 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:23:59.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0064964669999999997 2023-11-25T04:23:59.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.123 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:59.123 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044331+0000", 2023-11-25T04:23:59.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1544999999999997e-05 2023-11-25T04:23:59.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.123 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:59.123 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044372+0000", 2023-11-25T04:23:59.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0349000000000003e-05 2023-11-25T04:23:59.123 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:59.123 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:59.123 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:59.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.124 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:15 3.15 3:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:59.124 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:59.124 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 72.052108231999995, 2023-11-25T04:23:59.125 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010464362, 2023-11-25T04:23:59.125 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:59.125 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:59.125 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:59.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.125 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:59.125 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:59.125 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:59.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.125 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:59.125 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:23:59.125 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:59.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.126 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:59.126 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034042+0000", 2023-11-25T04:23:59.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0999999999999997e-07 2023-11-25T04:23:59.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.126 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:59.126 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034048+0000", 2023-11-25T04:23:59.127 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0390000000000001e-06 2023-11-25T04:23:59.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.127 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:59.127 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034059+0000", 2023-11-25T04:23:59.127 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1182999999999999e-05 2023-11-25T04:23:59.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.127 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:59.127 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034067+0000", 2023-11-25T04:23:59.127 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1559999999999995e-06 2023-11-25T04:23:59.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.128 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:59.128 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035005+0000", 2023-11-25T04:23:59.128 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00093747399999999999 2023-11-25T04:23:59.128 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.128 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.128 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:59.128 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035077+0000", 2023-11-25T04:23:59.128 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2747e-05 2023-11-25T04:23:59.128 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.128 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.128 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:59.128 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035219+0000", 2023-11-25T04:23:59.128 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000141877 2023-11-25T04:23:59.129 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.129 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.129 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:59.129 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037810+0000", 2023-11-25T04:23:59.129 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002590752 2023-11-25T04:23:59.129 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.129 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.129 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:59.129 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044437+0000", 2023-11-25T04:23:59.129 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066270110000000004 2023-11-25T04:23:59.129 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.129 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.129 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:59.130 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044481+0000", 2023-11-25T04:23:59.130 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4106000000000001e-05 2023-11-25T04:23:59.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.130 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.130 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:59.130 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044505+0000", 2023-11-25T04:23:59.130 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4207e-05 2023-11-25T04:23:59.130 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:59.130 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:59.130 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:59.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.130 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.131 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:16 3.1d 3:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:59.131 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:59.131 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 72.052066384, 2023-11-25T04:23:59.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010413923, 2023-11-25T04:23:59.131 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:59.131 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:59.131 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:59.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:59.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:59.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:59.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.132 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:59.132 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:59.132 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:59.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.132 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:59.132 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:23:59.132 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4499999999999997e-07 2023-11-25T04:23:59.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.132 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:59.132 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034087+0000", 2023-11-25T04:23:59.133 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9770000000000002e-06 2023-11-25T04:23:59.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.133 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.133 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:59.133 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034094+0000", 2023-11-25T04:23:59.133 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8809999999999998e-06 2023-11-25T04:23:59.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.133 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.133 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:59.133 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034100+0000", 2023-11-25T04:23:59.133 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8440000000000003e-06 2023-11-25T04:23:59.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.133 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.134 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:59.134 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034993+0000", 2023-11-25T04:23:59.134 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00089336799999999996 2023-11-25T04:23:59.134 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.134 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.134 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:59.134 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035099+0000", 2023-11-25T04:23:59.134 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106037 2023-11-25T04:23:59.134 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.134 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.134 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:59.134 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035276+0000", 2023-11-25T04:23:59.134 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017631799999999999 2023-11-25T04:23:59.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.135 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:59.135 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037815+0000", 2023-11-25T04:23:59.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025395959999999999 2023-11-25T04:23:59.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.135 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:59.135 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044420+0000", 2023-11-25T04:23:59.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066045749999999997 2023-11-25T04:23:59.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.135 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:59.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044463+0000", 2023-11-25T04:23:59.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3205999999999999e-05 2023-11-25T04:23:59.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:59.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044497+0000", 2023-11-25T04:23:59.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3476000000000003e-05 2023-11-25T04:23:59.136 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:59.136 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:59.136 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:59.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.136 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:17 3.1d 3:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:59.137 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:59.137 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 72.052041174999999, 2023-11-25T04:23:59.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01127356, 2023-11-25T04:23:59.137 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:59.137 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:59.137 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:59.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:59.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:59.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:59.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.138 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:59.138 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:59.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:59.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.138 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:59.138 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:23:59.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1100000000000001e-07 2023-11-25T04:23:59.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.138 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:59.139 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034112+0000", 2023-11-25T04:23:59.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8600000000000003e-06 2023-11-25T04:23:59.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.139 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:59.139 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034120+0000", 2023-11-25T04:23:59.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7700000000000001e-06 2023-11-25T04:23:59.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.139 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:59.139 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034127+0000", 2023-11-25T04:23:59.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0999999999999998e-06 2023-11-25T04:23:59.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.140 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:59.140 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035601+0000", 2023-11-25T04:23:59.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014739410000000001 2023-11-25T04:23:59.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.140 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:59.140 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035685+0000", 2023-11-25T04:23:59.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3677999999999994e-05 2023-11-25T04:23:59.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.140 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:59.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035827+0000", 2023-11-25T04:23:59.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014232599999999999 2023-11-25T04:23:59.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.141 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.141 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:59.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038725+0000", 2023-11-25T04:23:59.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028982779999999998 2023-11-25T04:23:59.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.141 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.141 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:59.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045323+0000", 2023-11-25T04:23:59.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0065980780000000003 2023-11-25T04:23:59.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.142 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:59.142 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045354+0000", 2023-11-25T04:23:59.142 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0463e-05 2023-11-25T04:23:59.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.142 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:59.142 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045381+0000", 2023-11-25T04:23:59.142 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7654999999999999e-05 2023-11-25T04:23:59.142 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:59.142 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:59.142 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:59.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.143 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:22 3.e 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:59.143 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:59.143 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 72.052014761999999, 2023-11-25T04:23:59.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010133422, 2023-11-25T04:23:59.143 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:59.143 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:59.143 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:59.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:59.143 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:59.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:59.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:59.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:23:59.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:59.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:59.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034135+0000", 2023-11-25T04:23:59.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-07 2023-11-25T04:23:59.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:59.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034138+0000", 2023-11-25T04:23:59.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6009999999999999e-06 2023-11-25T04:23:59.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:59.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034145+0000", 2023-11-25T04:23:59.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1169999999999998e-06 2023-11-25T04:23:59.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:59.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034150+0000", 2023-11-25T04:23:59.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9080000000000003e-06 2023-11-25T04:23:59.146 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.146 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:59.146 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034209+0000", 2023-11-25T04:23:59.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8149000000000002e-05 2023-11-25T04:23:59.146 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.146 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:59.146 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034415+0000", 2023-11-25T04:23:59.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020637699999999999 2023-11-25T04:23:59.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.147 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:59.147 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034584+0000", 2023-11-25T04:23:59.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016870399999999999 2023-11-25T04:23:59.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.147 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:59.147 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.036979+0000", 2023-11-25T04:23:59.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023957509999999998 2023-11-25T04:23:59.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.148 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:59.148 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044161+0000", 2023-11-25T04:23:59.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0071811590000000003 2023-11-25T04:23:59.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.148 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:59.148 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044224+0000", 2023-11-25T04:23:59.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3454000000000006e-05 2023-11-25T04:23:59.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.148 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:59.148 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044268+0000", 2023-11-25T04:23:59.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3761999999999999e-05 2023-11-25T04:23:59.149 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:59.149 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:59.149 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:59.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.149 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:23 3.e 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:59.149 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:59.149 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 72.051983352999997, 2023-11-25T04:23:59.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010281676, 2023-11-25T04:23:59.149 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:59.149 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:59.149 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:59.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:59.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:59.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:59.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:59.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:59.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:59.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:59.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:23:59.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5799999999999999e-07 2023-11-25T04:23:59.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.151 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:59.151 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034169+0000", 2023-11-25T04:23:59.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1879999999999998e-06 2023-11-25T04:23:59.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.151 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:59.151 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034177+0000", 2023-11-25T04:23:59.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7020000000000001e-06 2023-11-25T04:23:59.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.151 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:59.152 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034184+0000", 2023-11-25T04:23:59.152 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3910000000000003e-06 2023-11-25T04:23:59.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.152 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:59.152 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034924+0000", 2023-11-25T04:23:59.152 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00074032999999999998 2023-11-25T04:23:59.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.152 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:59.152 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034991+0000", 2023-11-25T04:23:59.152 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7347000000000004e-05 2023-11-25T04:23:59.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:59.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035096+0000", 2023-11-25T04:23:59.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010488 2023-11-25T04:23:59.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:59.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037588+0000", 2023-11-25T04:23:59.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024918309999999999 2023-11-25T04:23:59.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:59.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044367+0000", 2023-11-25T04:23:59.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067787589999999997 2023-11-25T04:23:59.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:59.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044415+0000", 2023-11-25T04:23:59.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7803000000000003e-05 2023-11-25T04:23:59.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:59.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044447+0000", 2023-11-25T04:23:59.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2886999999999999e-05 2023-11-25T04:23:59.154 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:59.154 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:59.154 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:59.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.155 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:30 3.15 3:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:59.155 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:59.155 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 72.051924912999993, 2023-11-25T04:23:59.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010365070000000001, 2023-11-25T04:23:59.155 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:59.155 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:59.155 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:59.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.155 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:59.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:59.156 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:59.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.156 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:59.156 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:23:59.156 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:59.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.156 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:59.156 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034225+0000", 2023-11-25T04:23:59.156 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7400000000000002e-07 2023-11-25T04:23:59.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.157 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:59.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034229+0000", 2023-11-25T04:23:59.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3100000000000002e-06 2023-11-25T04:23:59.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.157 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:59.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034239+0000", 2023-11-25T04:23:59.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9140000000000003e-06 2023-11-25T04:23:59.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.157 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:59.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034246+0000", 2023-11-25T04:23:59.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0149999999999998e-06 2023-11-25T04:23:59.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:59.158 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035572+0000", 2023-11-25T04:23:59.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001325655 2023-11-25T04:23:59.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:59.158 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035730+0000", 2023-11-25T04:23:59.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015827899999999999 2023-11-25T04:23:59.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:59.159 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035857+0000", 2023-11-25T04:23:59.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012644699999999999 2023-11-25T04:23:59.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.159 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:23:59.159 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038737+0000", 2023-11-25T04:23:59.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002880077 2023-11-25T04:23:59.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.159 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:59.159 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044540+0000", 2023-11-25T04:23:59.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058034239999999997 2023-11-25T04:23:59.160 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.160 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.160 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:59.160 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044569+0000", 2023-11-25T04:23:59.160 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8388999999999999e-05 2023-11-25T04:23:59.160 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.160 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.160 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:59.160 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044589+0000", 2023-11-25T04:23:59.160 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0686000000000002e-05 2023-11-25T04:23:59.160 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:59.160 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:59.160 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:59.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.161 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.161 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:33 3.1b 3:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:59.161 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:59.161 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 72.051879415000002, 2023-11-25T04:23:59.161 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099754049999999997, 2023-11-25T04:23:59.161 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:59.161 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:23:59.161 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:59.161 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.161 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:59.161 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:59.161 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:59.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.162 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:59.162 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:59.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:59.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.162 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:59.162 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:23:59.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4799999999999998e-07 2023-11-25T04:23:59.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.162 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:59.163 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034273+0000", 2023-11-25T04:23:59.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7819999999999998e-06 2023-11-25T04:23:59.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.163 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.163 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:59.163 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034285+0000", 2023-11-25T04:23:59.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2058e-05 2023-11-25T04:23:59.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.163 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.163 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:59.163 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034291+0000", 2023-11-25T04:23:59.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0680000000000002e-06 2023-11-25T04:23:59.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.163 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.164 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:59.164 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034370+0000", 2023-11-25T04:23:59.164 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8452e-05 2023-11-25T04:23:59.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.164 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.164 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:59.164 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034477+0000", 2023-11-25T04:23:59.164 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107471 2023-11-25T04:23:59.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.164 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.164 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:59.164 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044245+0000", 2023-11-25T04:23:59.165 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0097679259999999997 2023-11-25T04:23:59.165 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:59.165 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:59.165 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:59.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.165 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.165 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:36 3.1a 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:59.165 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:59.165 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 72.051832742000002, 2023-11-25T04:23:59.165 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058179099999999999, 2023-11-25T04:23:59.165 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:59.165 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:23:59.165 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:59.165 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.166 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:59.166 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:59.166 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:59.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.166 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:59.166 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:23:59.166 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:59.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.167 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:59.167 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034317+0000", 2023-11-25T04:23:59.167 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1999999999999999e-07 2023-11-25T04:23:59.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.167 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:59.167 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034321+0000", 2023-11-25T04:23:59.167 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.233e-06 2023-11-25T04:23:59.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.167 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:59.167 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034337+0000", 2023-11-25T04:23:59.168 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5308999999999999e-05 2023-11-25T04:23:59.168 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.168 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.168 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:59.168 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034354+0000", 2023-11-25T04:23:59.168 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7411999999999999e-05 2023-11-25T04:23:59.168 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.168 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.168 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:59.168 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034465+0000", 2023-11-25T04:23:59.168 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011154699999999999 2023-11-25T04:23:59.168 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.168 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.169 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:59.169 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034560+0000", 2023-11-25T04:23:59.169 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4186999999999998e-05 2023-11-25T04:23:59.169 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.169 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.169 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:59.169 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.040134+0000", 2023-11-25T04:23:59.169 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055746019999999997 2023-11-25T04:23:59.169 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:59.169 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:59.169 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:59.169 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.169 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.169 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:9 3.10 e18/16 3:09401a81:::601.00000000:head v 18'4, mlcod=0'0)", 2023-11-25T04:23:59.170 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:59.170 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 72.042938434000007, 2023-11-25T04:23:59.170 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061641020000000003, 2023-11-25T04:23:59.170 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:59.170 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:59.170 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:59.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.170 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:59.170 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:59.170 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:59.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.171 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:59.171 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:59.171 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:59.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.171 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:59.171 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:23:59.171 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5300000000000001e-07 2023-11-25T04:23:59.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.171 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:59.171 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043222+0000", 2023-11-25T04:23:59.172 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.082e-05 2023-11-25T04:23:59.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.172 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:59.172 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043228+0000", 2023-11-25T04:23:59.172 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6699999999999997e-06 2023-11-25T04:23:59.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.172 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:59.172 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043236+0000", 2023-11-25T04:23:59.172 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8189999999999992e-06 2023-11-25T04:23:59.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.173 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:59.173 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043507+0000", 2023-11-25T04:23:59.173 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000271012 2023-11-25T04:23:59.173 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.173 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.173 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:59.173 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043530+0000", 2023-11-25T04:23:59.173 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2659000000000001e-05 2023-11-25T04:23:59.173 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.173 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.173 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:59.173 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049321+0000", 2023-11-25T04:23:59.174 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057915129999999999 2023-11-25T04:23:59.174 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.174 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.174 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:59.174 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049375+0000", 2023-11-25T04:23:59.174 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3455999999999998e-05 2023-11-25T04:23:59.174 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:59.174 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:59.174 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:59.174 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.174 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.174 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:1 3.14 e18/16 3:292cf221:::200.00000000:head v 18'3, mlcod=0'0)", 2023-11-25T04:23:59.175 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:59.175 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 72.042901721000007, 2023-11-25T04:23:59.175 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0060983599999999997, 2023-11-25T04:23:59.175 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:59.175 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:59.175 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:59.175 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.175 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:59.175 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:59.175 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:59.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.175 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.175 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:59.176 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:23:59.176 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:59.176 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.176 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.176 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:59.176 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043248+0000", 2023-11-25T04:23:59.176 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9399999999999995e-07 2023-11-25T04:23:59.176 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.176 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.176 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:59.176 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043252+0000", 2023-11-25T04:23:59.176 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1799999999999998e-06 2023-11-25T04:23:59.177 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.177 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.177 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:59.177 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043259+0000", 2023-11-25T04:23:59.177 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0049999999999998e-06 2023-11-25T04:23:59.177 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.177 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.177 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:59.177 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043265+0000", 2023-11-25T04:23:59.177 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0859999999999998e-06 2023-11-25T04:23:59.177 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.177 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.177 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:59.178 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043311+0000", 2023-11-25T04:23:59.178 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6082999999999999e-05 2023-11-25T04:23:59.178 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.178 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.178 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:59.178 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043330+0000", 2023-11-25T04:23:59.178 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8556e-05 2023-11-25T04:23:59.178 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.178 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.178 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:59.178 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049303+0000", 2023-11-25T04:23:59.178 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059733470000000004 2023-11-25T04:23:59.178 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.179 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.179 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:59.179 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049346+0000", 2023-11-25T04:23:59.179 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2608999999999998e-05 2023-11-25T04:23:59.179 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:59.179 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:59.179 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:59.179 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.179 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.179 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:3 3.14 e18/16 3:287034c4:::1.00000000.inode:head v 18'4, mlcod=0'0)", 2023-11-25T04:23:59.179 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:59.179 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 72.042385214999996, 2023-11-25T04:23:59.179 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057040090000000003, 2023-11-25T04:23:59.180 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:59.180 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:59.180 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:59.180 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.180 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:59.180 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:59.180 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:59.180 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.180 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.180 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:59.180 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:59.180 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:59.180 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.181 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.181 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:59.181 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:23:59.181 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9799999999999999e-07 2023-11-25T04:23:59.181 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.181 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.181 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:59.181 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043785+0000", 2023-11-25T04:23:59.181 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0721999999999999e-05 2023-11-25T04:23:59.181 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.181 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.181 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:59.181 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043793+0000", 2023-11-25T04:23:59.182 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5839999999999995e-06 2023-11-25T04:23:59.182 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.182 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.182 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:59.182 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043803+0000", 2023-11-25T04:23:59.182 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0729999999999994e-06 2023-11-25T04:23:59.182 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.182 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.182 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:59.182 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043843+0000", 2023-11-25T04:23:59.183 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0747999999999997e-05 2023-11-25T04:23:59.183 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.183 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.183 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:59.183 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043856+0000", 2023-11-25T04:23:59.183 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2697000000000001e-05 2023-11-25T04:23:59.183 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.183 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.183 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:59.183 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049426+0000", 2023-11-25T04:23:59.183 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055704409999999998 2023-11-25T04:23:59.183 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.184 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.184 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:59.184 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049468+0000", 2023-11-25T04:23:59.184 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1446000000000003e-05 2023-11-25T04:23:59.184 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:59.184 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:59.184 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:59.184 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.184 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.184 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "pg_update_log_missing(3.14 epoch 18/16 rep_tid 171 entries 18'5 (0'0) error 3:2e2fa760:::200.00000001:head by mds.0.3:32 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:23:59.184 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:59.184 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 72.042224251999997, 2023-11-25T04:23:59.184 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005556938, 2023-11-25T04:23:59.185 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:59.185 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2023-11-25T04:23:59.185 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:59.185 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.185 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:59.185 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:59.185 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:59.185 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.185 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.185 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:59.185 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:59.185 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:59.186 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.186 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.186 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:59.186 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:23:59.186 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.03e-07 2023-11-25T04:23:59.186 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.186 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.186 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:59.186 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043931+0000", 2023-11-25T04:23:59.186 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1079999999999998e-06 2023-11-25T04:23:59.186 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.187 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.187 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:59.187 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043944+0000", 2023-11-25T04:23:59.187 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2852e-05 2023-11-25T04:23:59.187 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.187 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.187 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:59.187 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043955+0000", 2023-11-25T04:23:59.187 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1037e-05 2023-11-25T04:23:59.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.188 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:59.188 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044088+0000", 2023-11-25T04:23:59.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013269599999999999 2023-11-25T04:23:59.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.188 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:59.188 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049482+0000", 2023-11-25T04:23:59.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053940419999999999 2023-11-25T04:23:59.188 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:59.188 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:59.188 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:59.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.189 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:10 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'5, mlcod=0'0)", 2023-11-25T04:23:59.189 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:59.189 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 72.042134223999994, 2023-11-25T04:23:59.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054493099999999997, 2023-11-25T04:23:59.189 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:59.189 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:59.189 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:59.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:59.189 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:59.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:59.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.190 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:59.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:59.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:59.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.190 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:59.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:23:59.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6100000000000002e-07 2023-11-25T04:23:59.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.191 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:59.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044019+0000", 2023-11-25T04:23:59.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.839e-06 2023-11-25T04:23:59.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.191 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:59.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044026+0000", 2023-11-25T04:23:59.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5869999999999998e-06 2023-11-25T04:23:59.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.191 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:59.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044031+0000", 2023-11-25T04:23:59.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4909999999999996e-06 2023-11-25T04:23:59.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.192 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:59.192 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044070+0000", 2023-11-25T04:23:59.192 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8829999999999999e-05 2023-11-25T04:23:59.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.192 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:59.192 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044081+0000", 2023-11-25T04:23:59.192 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1447999999999999e-05 2023-11-25T04:23:59.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.192 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:59.193 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049406+0000", 2023-11-25T04:23:59.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053246480000000004 2023-11-25T04:23:59.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.193 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:59.193 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049464+0000", 2023-11-25T04:23:59.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8205999999999998e-05 2023-11-25T04:23:59.193 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:59.193 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:59.193 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:59.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.193 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:11 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'6, mlcod=0'0)", 2023-11-25T04:23:59.194 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:59.194 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 72.041932959999997, 2023-11-25T04:23:59.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052673499999999996, 2023-11-25T04:23:59.194 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:23:59.195 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:59.195 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:23:59.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.195 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:23:59.195 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:59.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:59.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.195 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:23:59.195 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:59.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:23:59.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.196 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:23:59.196 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:23:59.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0199999999999998e-07 2023-11-25T04:23:59.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.196 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:23:59.196 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044220+0000", 2023-11-25T04:23:59.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7720000000000001e-06 2023-11-25T04:23:59.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:23:59.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044226+0000", 2023-11-25T04:23:59.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8579999999999996e-06 2023-11-25T04:23:59.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:23:59.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044232+0000", 2023-11-25T04:23:59.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1700000000000002e-06 2023-11-25T04:23:59.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:23:59.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044274+0000", 2023-11-25T04:23:59.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1254999999999997e-05 2023-11-25T04:23:59.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:23:59.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:23:59.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6880999999999999e-05 2023-11-25T04:23:59.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:23:59.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049466+0000", 2023-11-25T04:23:59.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051659219999999999 2023-11-25T04:23:59.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:23:59.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:23:59.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:23:59.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049483+0000", 2023-11-25T04:23:59.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.719e-05 2023-11-25T04:23:59.199 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:59.199 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:59.199 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:59.199 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:23:59.199 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:23:59.199 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:59.201 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:23:59.201 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:59.200+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:59.302 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:59.300+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:59.308 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:59.309 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:23:59.309 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2023-11-25T04:23:59.309 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:59.317 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:23:59.402 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:59.401+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:59.463 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:59.463 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:23:59.463 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2023-11-25T04:23:59.463 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2023-11-25T04:23:59.463 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:59.473 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:23:59.517 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:59.516+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:23:59.580 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:23:59.581 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2023-11-25T04:23:59.581 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2023-11-25T04:23:59.581 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2023-11-25T04:23:59.581 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.581 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:59.581 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:59.581 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 72.546563250000005, 2023-11-25T04:23:59.581 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01072943, 2023-11-25T04:23:59.581 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:59.581 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:59.581 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:59.581 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.581 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:59.582 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:59.582 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:59.582 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.582 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.582 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:59.582 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:23:59.582 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:59.582 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.582 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.582 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:59.582 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031829+0000", 2023-11-25T04:23:59.582 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.666e-06 2023-11-25T04:23:59.583 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.583 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.583 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:59.583 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031839+0000", 2023-11-25T04:23:59.583 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9490000000000008e-06 2023-11-25T04:23:59.583 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.583 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.583 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:59.583 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031857+0000", 2023-11-25T04:23:59.583 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8093000000000001e-05 2023-11-25T04:23:59.583 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.583 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.583 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:59.584 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031882+0000", 2023-11-25T04:23:59.584 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4294000000000002e-05 2023-11-25T04:23:59.584 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.584 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.585 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:59.585 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033130+0000", 2023-11-25T04:23:59.585 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001248264 2023-11-25T04:23:59.585 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.585 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.585 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:59.586 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033426+0000", 2023-11-25T04:23:59.586 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029620999999999998 2023-11-25T04:23:59.586 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.586 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.587 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:59.587 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033814+0000", 2023-11-25T04:23:59.587 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000387742 2023-11-25T04:23:59.587 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.587 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.587 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:59.587 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036793+0000", 2023-11-25T04:23:59.587 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002979598 2023-11-25T04:23:59.587 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.587 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.588 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:59.588 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042455+0000", 2023-11-25T04:23:59.588 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0056619260000000003 2023-11-25T04:23:59.588 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.588 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.588 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:59.588 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042502+0000", 2023-11-25T04:23:59.588 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7139999999999999e-05 2023-11-25T04:23:59.588 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.589 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.589 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:59.589 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042557+0000", 2023-11-25T04:23:59.589 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4548e-05 2023-11-25T04:23:59.589 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:59.589 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:59.589 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:59.589 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.589 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.589 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:59.589 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:59.589 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 72.545364715000005, 2023-11-25T04:23:59.590 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011505202000000001, 2023-11-25T04:23:59.590 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:59.590 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:59.590 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:59.590 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.590 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:59.590 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:59.590 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:59.590 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.590 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.590 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:59.590 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:23:59.590 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:59.590 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.591 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.591 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:59.591 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033027+0000", 2023-11-25T04:23:59.591 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0699999999999999e-06 2023-11-25T04:23:59.591 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.591 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.591 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:59.591 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033035+0000", 2023-11-25T04:23:59.591 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0830000000000004e-06 2023-11-25T04:23:59.591 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.591 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.591 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:59.591 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033059+0000", 2023-11-25T04:23:59.591 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3975e-05 2023-11-25T04:23:59.592 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.592 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.592 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:59.592 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033482+0000", 2023-11-25T04:23:59.592 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042253700000000001 2023-11-25T04:23:59.592 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.592 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.592 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:59.593 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034520+0000", 2023-11-25T04:23:59.593 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001037991 2023-11-25T04:23:59.593 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.594 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.594 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:59.594 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034611+0000", 2023-11-25T04:23:59.594 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1113999999999995e-05 2023-11-25T04:23:59.594 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.594 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.594 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:59.594 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034769+0000", 2023-11-25T04:23:59.594 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000157641 2023-11-25T04:23:59.594 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.594 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.594 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:59.594 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038292+0000", 2023-11-25T04:23:59.595 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035231160000000002 2023-11-25T04:23:59.595 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.595 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.595 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:59.595 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044440+0000", 2023-11-25T04:23:59.595 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0061486980000000002 2023-11-25T04:23:59.595 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.595 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.595 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:59.595 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044495+0000", 2023-11-25T04:23:59.595 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4339000000000003e-05 2023-11-25T04:23:59.595 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.595 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.596 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:59.596 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044531+0000", 2023-11-25T04:23:59.596 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6637999999999997e-05 2023-11-25T04:23:59.596 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:59.596 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:59.596 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:59.596 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.596 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.597 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:59.597 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:59.597 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 72.544890361, 2023-11-25T04:23:59.597 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0057634039999999997, 2023-11-25T04:23:59.597 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:59.597 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:59.597 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:59.597 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.597 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:59.597 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:59.597 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:59.598 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.598 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.598 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:59.598 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:59.599 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:59.599 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.599 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.599 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:59.599 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:23:59.599 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7900000000000007e-07 2023-11-25T04:23:59.599 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.599 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.599 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:59.599 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033507+0000", 2023-11-25T04:23:59.599 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8900000000000004e-06 2023-11-25T04:23:59.599 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.599 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.599 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:59.600 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033519+0000", 2023-11-25T04:23:59.600 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1756000000000001e-05 2023-11-25T04:23:59.600 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.600 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.600 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:59.600 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033527+0000", 2023-11-25T04:23:59.600 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3610000000000001e-06 2023-11-25T04:23:59.600 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.600 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.600 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:59.600 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034451+0000", 2023-11-25T04:23:59.601 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00092372400000000001 2023-11-25T04:23:59.601 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.601 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.601 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:59.601 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034809+0000", 2023-11-25T04:23:59.601 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00035757699999999998 2023-11-25T04:23:59.601 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.601 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.602 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:23:59.602 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034965+0000", 2023-11-25T04:23:59.602 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015638699999999999 2023-11-25T04:23:59.602 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.602 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.602 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:59.602 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038299+0000", 2023-11-25T04:23:59.602 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0033336559999999999 2023-11-25T04:23:59.602 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.602 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.602 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:59.603 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039169+0000", 2023-11-25T04:23:59.603 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00087066400000000005 2023-11-25T04:23:59.603 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.603 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.603 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:59.603 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039226+0000", 2023-11-25T04:23:59.603 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6511999999999997e-05 2023-11-25T04:23:59.603 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.603 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.603 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:59.603 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039264+0000", 2023-11-25T04:23:59.603 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7898000000000002e-05 2023-11-25T04:23:59.604 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:59.604 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:59.604 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:59.604 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.604 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.604 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:25 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:59.604 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:59.604 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 72.544811777000007, 2023-11-25T04:23:59.604 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096656169999999996, 2023-11-25T04:23:59.604 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:59.604 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:59.604 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:59.604 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.604 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:59.605 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:59.605 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:59.605 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.605 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.605 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:59.605 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:23:59.605 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:59.605 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.605 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.605 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:59.605 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033580+0000", 2023-11-25T04:23:59.605 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8199999999999999e-07 2023-11-25T04:23:59.605 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.606 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.606 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:59.606 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033585+0000", 2023-11-25T04:23:59.606 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3680000000000001e-06 2023-11-25T04:23:59.606 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.606 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.606 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:59.606 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033600+0000", 2023-11-25T04:23:59.606 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4331000000000001e-05 2023-11-25T04:23:59.606 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.606 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.607 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:59.607 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033606+0000", 2023-11-25T04:23:59.607 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5780000000000004e-06 2023-11-25T04:23:59.607 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.607 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.607 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:59.607 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035747+0000", 2023-11-25T04:23:59.607 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0021406530000000002 2023-11-25T04:23:59.608 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.608 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.608 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:59.608 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035901+0000", 2023-11-25T04:23:59.608 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000153931 2023-11-25T04:23:59.608 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.608 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.608 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:23:59.608 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036193+0000", 2023-11-25T04:23:59.608 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029211299999999999 2023-11-25T04:23:59.608 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.608 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.608 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:59.609 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038655+0000", 2023-11-25T04:23:59.609 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0024621640000000002 2023-11-25T04:23:59.609 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.609 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.609 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:59.609 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043130+0000", 2023-11-25T04:23:59.609 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.004474698 2023-11-25T04:23:59.609 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.609 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.609 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:59.609 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043204+0000", 2023-11-25T04:23:59.609 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4091999999999995e-05 2023-11-25T04:23:59.609 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.609 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.610 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:59.610 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043245+0000", 2023-11-25T04:23:59.610 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0907e-05 2023-11-25T04:23:59.610 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:59.610 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:59.610 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:59.610 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.610 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.610 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:26 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:59.611 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:59.611 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 72.544775952999998, 2023-11-25T04:23:59.611 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011086945000000001, 2023-11-25T04:23:59.611 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:59.611 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:59.611 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:59.611 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.611 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:59.611 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:59.611 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:59.611 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.611 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.612 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:59.612 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:59.612 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:59.612 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.612 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.612 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:59.612 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:23:59.612 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1699999999999998e-07 2023-11-25T04:23:59.612 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.612 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.612 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:59.612 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033627+0000", 2023-11-25T04:23:59.613 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.163e-05 2023-11-25T04:23:59.613 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.613 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.613 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:59.613 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033636+0000", 2023-11-25T04:23:59.613 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6859999999999995e-06 2023-11-25T04:23:59.613 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.613 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.613 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:59.613 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033642+0000", 2023-11-25T04:23:59.613 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3149999999999997e-06 2023-11-25T04:23:59.613 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.613 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.613 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:59.614 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035414+0000", 2023-11-25T04:23:59.614 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0017716380000000001 2023-11-25T04:23:59.614 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.614 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.614 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:59.614 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035539+0000", 2023-11-25T04:23:59.614 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000125477 2023-11-25T04:23:59.614 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.614 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.614 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:59.614 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035672+0000", 2023-11-25T04:23:59.614 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000133316 2023-11-25T04:23:59.614 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.615 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.615 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:59.615 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038649+0000", 2023-11-25T04:23:59.615 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029764679999999999 2023-11-25T04:23:59.615 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.615 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.615 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:59.615 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044578+0000", 2023-11-25T04:23:59.615 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0059293510000000002 2023-11-25T04:23:59.615 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.615 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.615 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:59.615 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044651+0000", 2023-11-25T04:23:59.615 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2513000000000002e-05 2023-11-25T04:23:59.616 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.616 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.616 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:59.616 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044702+0000", 2023-11-25T04:23:59.616 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1034e-05 2023-11-25T04:23:59.616 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:59.616 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:59.616 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:59.616 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.616 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.617 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:29 3.b 3:d0630e4c:::mds0_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:59.617 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:59.617 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 72.544738033000002, 2023-11-25T04:23:59.617 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010980227, 2023-11-25T04:23:59.617 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:59.617 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:59.618 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:59.618 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.618 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:59.620 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:59.620 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:59.620 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.620 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.621 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:59.621 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:59.621 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:59.621 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.621 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.621 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:59.621 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:23:59.621 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3499999999999996e-07 2023-11-25T04:23:59.621 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.621 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.621 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:59.621 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033658+0000", 2023-11-25T04:23:59.622 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1280000000000001e-06 2023-11-25T04:23:59.622 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.622 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.622 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:59.622 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033667+0000", 2023-11-25T04:23:59.622 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1430000000000004e-06 2023-11-25T04:23:59.622 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.623 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.623 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:59.623 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033678+0000", 2023-11-25T04:23:59.623 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0906e-05 2023-11-25T04:23:59.623 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.623 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.623 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:59.623 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034456+0000", 2023-11-25T04:23:59.623 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00077833899999999996 2023-11-25T04:23:59.623 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.623 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.623 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:59.623 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034575+0000", 2023-11-25T04:23:59.624 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011926000000000001 2023-11-25T04:23:59.624 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.624 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.624 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:59.624 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035001+0000", 2023-11-25T04:23:59.624 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042550999999999998 2023-11-25T04:23:59.624 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.624 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.624 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:59.624 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.037923+0000", 2023-11-25T04:23:59.624 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029219670000000001 2023-11-25T04:23:59.624 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.624 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.625 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:59.625 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044538+0000", 2023-11-25T04:23:59.625 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0066152349999999997 2023-11-25T04:23:59.625 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.625 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.625 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:59.625 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044594+0000", 2023-11-25T04:23:59.625 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5593000000000001e-05 2023-11-25T04:23:59.625 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.625 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.625 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:59.625 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044633+0000", 2023-11-25T04:23:59.626 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9511000000000002e-05 2023-11-25T04:23:59.626 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:59.626 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:59.626 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:59.626 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.626 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.626 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:34 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:59.626 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:59.626 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 72.544487919999995, 2023-11-25T04:23:59.626 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010920445000000001, 2023-11-25T04:23:59.626 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:59.626 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:59.627 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:59.627 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.627 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:59.627 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:59.627 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:59.627 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.627 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.627 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:59.627 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:23:59.628 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:59.628 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.628 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.628 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:59.628 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033904+0000", 2023-11-25T04:23:59.628 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6599999999999994e-07 2023-11-25T04:23:59.628 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.628 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.629 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:59.629 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033909+0000", 2023-11-25T04:23:59.629 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5639999999999996e-06 2023-11-25T04:23:59.629 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.629 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.630 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:59.630 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033923+0000", 2023-11-25T04:23:59.630 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3028e-05 2023-11-25T04:23:59.630 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.630 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.630 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:59.630 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034043+0000", 2023-11-25T04:23:59.630 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000120328 2023-11-25T04:23:59.630 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.630 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.630 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:59.631 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035576+0000", 2023-11-25T04:23:59.631 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015333720000000001 2023-11-25T04:23:59.631 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.631 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.631 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:59.631 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035699+0000", 2023-11-25T04:23:59.631 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000123001 2023-11-25T04:23:59.632 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.632 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.632 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:59.632 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044823+0000", 2023-11-25T04:23:59.632 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0091241859999999994 2023-11-25T04:23:59.632 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:59.632 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:59.632 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:59.632 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.632 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.632 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:35 3.1f 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:59.632 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:59.632 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 72.543183714999998, 2023-11-25T04:23:59.632 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096164779999999995, 2023-11-25T04:23:59.633 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:59.633 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:59.633 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:59.633 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.633 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:59.633 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:59.633 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:59.633 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.633 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.633 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:59.633 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:23:59.633 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:59.633 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.634 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.634 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:59.634 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035208+0000", 2023-11-25T04:23:59.634 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2699999999999998e-07 2023-11-25T04:23:59.634 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.634 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.634 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:59.634 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035213+0000", 2023-11-25T04:23:59.634 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.6070000000000001e-06 2023-11-25T04:23:59.634 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.634 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.634 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:59.634 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035350+0000", 2023-11-25T04:23:59.634 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013693599999999999 2023-11-25T04:23:59.635 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.635 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.635 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:59.635 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035358+0000", 2023-11-25T04:23:59.635 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0099999999999995e-06 2023-11-25T04:23:59.635 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.635 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.635 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:59.635 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036260+0000", 2023-11-25T04:23:59.635 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00090240100000000005 2023-11-25T04:23:59.635 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.635 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.635 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:59.636 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036517+0000", 2023-11-25T04:23:59.636 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00025736299999999998 2023-11-25T04:23:59.636 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.636 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.636 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:59.636 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044824+0000", 2023-11-25T04:23:59.636 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0083063340000000003 2023-11-25T04:23:59.636 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:59.636 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:59.636 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:59.636 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.636 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.637 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:37 3.1e 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:23:59.637 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:59.637 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 72.543017028999998, 2023-11-25T04:23:59.637 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0078322640000000002, 2023-11-25T04:23:59.637 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:59.637 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:59.637 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:59.637 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.637 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:59.637 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:59.637 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:59.637 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.637 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.638 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:59.638 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:23:59.638 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:59.638 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.638 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.638 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:59.638 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035375+0000", 2023-11-25T04:23:59.638 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9100000000000003e-07 2023-11-25T04:23:59.638 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.638 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.638 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:59.638 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035379+0000", 2023-11-25T04:23:59.639 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9430000000000002e-06 2023-11-25T04:23:59.639 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.639 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.639 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:59.639 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035388+0000", 2023-11-25T04:23:59.639 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2760000000000006e-06 2023-11-25T04:23:59.639 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.639 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.639 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:59.639 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035396+0000", 2023-11-25T04:23:59.640 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8480000000000001e-06 2023-11-25T04:23:59.640 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.640 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.640 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:59.640 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035452+0000", 2023-11-25T04:23:59.640 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5831000000000001e-05 2023-11-25T04:23:59.640 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.640 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.640 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:59.640 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035934+0000", 2023-11-25T04:23:59.640 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00048265599999999998 2023-11-25T04:23:59.640 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.640 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.641 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:59.641 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043206+0000", 2023-11-25T04:23:59.641 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.007271919 2023-11-25T04:23:59.641 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:59.641 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:59.641 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:59.641 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.641 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.641 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing(3.1a epoch 18/16 rep_tid 3 entries 18'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:36 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:23:59.641 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:59.641 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 72.541741443000006, 2023-11-25T04:23:59.641 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002073414, 2023-11-25T04:23:59.642 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:59.642 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:59.642 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:59.642 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.642 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:59.642 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:59.642 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:59.642 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.642 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.642 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:59.642 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:23:59.642 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:59.643 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.643 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.643 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:59.643 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036650+0000", 2023-11-25T04:23:59.643 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7300000000000002e-07 2023-11-25T04:23:59.643 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.643 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.643 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:59.643 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036659+0000", 2023-11-25T04:23:59.643 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.7499999999999992e-06 2023-11-25T04:23:59.643 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.643 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.643 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:59.644 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036673+0000", 2023-11-25T04:23:59.644 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4917e-05 2023-11-25T04:23:59.644 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.644 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.644 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:59.644 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036687+0000", 2023-11-25T04:23:59.644 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3883000000000001e-05 2023-11-25T04:23:59.644 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.644 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.644 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:59.644 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036781+0000", 2023-11-25T04:23:59.644 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3219000000000003e-05 2023-11-25T04:23:59.645 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.645 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.645 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:59.645 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038723+0000", 2023-11-25T04:23:59.645 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019422720000000001 2023-11-25T04:23:59.645 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:59.645 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:59.645 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:59.645 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.645 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.645 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:28 3.13 e18/16 3:cd5a64a3:::100.00000000:head v 18'1, mlcod=0'0)", 2023-11-25T04:23:59.645 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:59.646 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 72.538080285000007, 2023-11-25T04:23:59.646 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001850546, 2023-11-25T04:23:59.646 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:59.646 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:59.646 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:59.646 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.646 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:59.646 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:59.646 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:59.646 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.646 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.646 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:59.646 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:59.646 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:59.647 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.647 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.647 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:59.647 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:23:59.647 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7800000000000002e-07 2023-11-25T04:23:59.647 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.647 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.647 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:59.647 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040331+0000", 2023-11-25T04:23:59.647 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0449999999999999e-05 2023-11-25T04:23:59.647 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.647 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.647 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:59.648 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040339+0000", 2023-11-25T04:23:59.648 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0709999999999995e-06 2023-11-25T04:23:59.648 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.648 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.648 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:59.648 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040347+0000", 2023-11-25T04:23:59.648 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7079999999999997e-06 2023-11-25T04:23:59.648 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.648 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.648 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:59.648 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040410+0000", 2023-11-25T04:23:59.648 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2453000000000007e-05 2023-11-25T04:23:59.648 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.649 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.649 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:59.649 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040422+0000", 2023-11-25T04:23:59.649 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2429000000000001e-05 2023-11-25T04:23:59.649 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.649 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.649 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:59.649 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042129+0000", 2023-11-25T04:23:59.649 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001707266 2023-11-25T04:23:59.649 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.649 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.649 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:59.649 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042161+0000", 2023-11-25T04:23:59.650 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1791000000000001e-05 2023-11-25T04:23:59.650 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:59.650 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:59.650 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:59.650 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.650 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.650 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:27 3.13 e18/16 3:c91d4a1d:::mds0_inotable:head v 18'2, mlcod=0'0)", 2023-11-25T04:23:59.650 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:59.650 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 72.537876761999996, 2023-11-25T04:23:59.650 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0016886430000000001, 2023-11-25T04:23:59.650 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:59.650 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:59.651 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:59.651 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.651 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:59.651 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:59.651 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:59.651 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.651 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.651 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:59.651 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:59.651 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:59.651 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.652 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.652 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:59.652 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:23:59.652 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3999999999999997e-07 2023-11-25T04:23:59.652 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.652 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.652 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:59.652 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040520+0000", 2023-11-25T04:23:59.652 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2989999999999996e-06 2023-11-25T04:23:59.652 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.652 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.652 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:59.652 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040527+0000", 2023-11-25T04:23:59.653 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5730000000000002e-06 2023-11-25T04:23:59.653 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.653 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.653 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:59.653 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040535+0000", 2023-11-25T04:23:59.653 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8630000000000008e-06 2023-11-25T04:23:59.653 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.653 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.653 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:59.653 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040683+0000", 2023-11-25T04:23:59.653 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000148002 2023-11-25T04:23:59.653 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.653 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.654 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:59.654 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040700+0000", 2023-11-25T04:23:59.654 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7065e-05 2023-11-25T04:23:59.654 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.654 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.654 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:59.654 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042163+0000", 2023-11-25T04:23:59.654 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001462767 2023-11-25T04:23:59.654 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.654 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.654 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:59.654 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042203+0000", 2023-11-25T04:23:59.654 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9734000000000003e-05 2023-11-25T04:23:59.655 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:59.655 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:59.655 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:59.655 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.655 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.655 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:20 3.1c e18/16 3:3ed09add:::607.00000000:head v 18'1, mlcod=0'0)", 2023-11-25T04:23:59.655 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:59.655 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 72.537841721000007, 2023-11-25T04:23:59.655 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015271, 2023-11-25T04:23:59.655 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:59.655 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:59.655 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:59.655 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.656 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:59.656 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:59.656 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:59.656 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.656 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.656 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:59.656 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:59.656 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:59.656 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.656 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.656 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:59.656 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:23:59.656 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4799999999999999e-07 2023-11-25T04:23:59.657 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.657 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.657 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:59.657 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040561+0000", 2023-11-25T04:23:59.657 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1800000000000001e-05 2023-11-25T04:23:59.657 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.657 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.657 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:59.657 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040568+0000", 2023-11-25T04:23:59.657 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7730000000000002e-06 2023-11-25T04:23:59.657 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.657 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.657 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:59.657 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040587+0000", 2023-11-25T04:23:59.658 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8691000000000001e-05 2023-11-25T04:23:59.658 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.658 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.658 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:59.658 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040638+0000", 2023-11-25T04:23:59.658 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1068e-05 2023-11-25T04:23:59.658 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.658 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.658 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:59.658 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040670+0000", 2023-11-25T04:23:59.658 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2125999999999997e-05 2023-11-25T04:23:59.658 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.658 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.659 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:59.659 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042032+0000", 2023-11-25T04:23:59.659 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001361821 2023-11-25T04:23:59.659 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.659 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.659 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:59.659 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042076+0000", 2023-11-25T04:23:59.659 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4472999999999999e-05 2023-11-25T04:23:59.659 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:59.659 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:59.659 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:59.659 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.660 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.660 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:21 3.1c e18/16 3:3ed09add:::607.00000000:head v 18'2, mlcod=0'0)", 2023-11-25T04:23:59.660 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:59.660 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 72.537606995999994, 2023-11-25T04:23:59.660 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0013228160000000001, 2023-11-25T04:23:59.660 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:59.660 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:59.660 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:59.660 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.660 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:59.660 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:59.660 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:59.660 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.660 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.661 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:59.661 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:59.661 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:59.661 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.661 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.661 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:59.661 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:23:59.661 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9299999999999999e-07 2023-11-25T04:23:59.661 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.661 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.661 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:59.661 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040790+0000", 2023-11-25T04:23:59.661 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5770000000000001e-06 2023-11-25T04:23:59.662 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.662 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.662 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:59.662 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040799+0000", 2023-11-25T04:23:59.662 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6439999999999995e-06 2023-11-25T04:23:59.662 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.662 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.662 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:59.662 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040806+0000", 2023-11-25T04:23:59.662 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0180000000000004e-06 2023-11-25T04:23:59.662 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.663 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.663 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:59.663 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040907+0000", 2023-11-25T04:23:59.663 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000100096 2023-11-25T04:23:59.663 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.663 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.663 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:59.663 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040925+0000", 2023-11-25T04:23:59.663 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8536999999999999e-05 2023-11-25T04:23:59.663 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.663 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.663 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:59.664 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042079+0000", 2023-11-25T04:23:59.664 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00115358 2023-11-25T04:23:59.664 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.664 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.664 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:59.664 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042107+0000", 2023-11-25T04:23:59.664 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7971e-05 2023-11-25T04:23:59.664 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:59.664 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:59.664 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:59.664 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.664 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.664 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop_reply(mds.0.3:4 3.1f e18/16 ondisk, result = 0)", 2023-11-25T04:23:59.665 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:59.665 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 72.536267605999996, 2023-11-25T04:23:59.665 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00044092900000000001, 2023-11-25T04:23:59.665 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:59.665 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:23:59.665 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:59.665 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.665 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:59.665 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:59.665 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:59.665 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.665 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.665 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:59.666 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:23:59.666 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:59.666 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.666 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.666 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:59.666 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042124+0000", 2023-11-25T04:23:59.666 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1800000000000005e-07 2023-11-25T04:23:59.666 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.666 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.666 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:59.666 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042128+0000", 2023-11-25T04:23:59.666 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2289999999999997e-06 2023-11-25T04:23:59.667 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.667 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.667 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:59.667 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042138+0000", 2023-11-25T04:23:59.667 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9729999999999992e-06 2023-11-25T04:23:59.667 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.667 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.667 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:59.667 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042145+0000", 2023-11-25T04:23:59.667 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.336e-06 2023-11-25T04:23:59.667 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.667 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.667 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:59.668 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042441+0000", 2023-11-25T04:23:59.668 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029650500000000002 2023-11-25T04:23:59.668 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.668 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.668 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:59.668 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042448+0000", 2023-11-25T04:23:59.668 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4100000000000002e-06 2023-11-25T04:23:59.668 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.668 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.668 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:59.668 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042564+0000", 2023-11-25T04:23:59.669 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000115758 2023-11-25T04:23:59.669 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:59.669 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:59.669 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:59.669 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.669 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.669 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing_reply(3.1f epoch 18/16 rep_tid 8 lcod 18'4)", 2023-11-25T04:23:59.669 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:59.669 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 72.533942194000005, 2023-11-25T04:23:59.669 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00038154699999999998, 2023-11-25T04:23:59.669 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:59.669 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:59.669 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:59.670 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.670 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:59.670 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:59.670 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:59.670 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.670 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.670 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:59.670 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:59.670 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:59.670 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.670 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.670 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:59.671 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:23:59.671 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2700000000000001e-07 2023-11-25T04:23:59.671 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.671 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.671 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:59.671 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044451+0000", 2023-11-25T04:23:59.671 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7460000000000001e-06 2023-11-25T04:23:59.671 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.671 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.671 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:59.671 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044455+0000", 2023-11-25T04:23:59.671 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5809999999999998e-06 2023-11-25T04:23:59.671 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.672 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.672 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:59.672 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044458+0000", 2023-11-25T04:23:59.672 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0170000000000001e-06 2023-11-25T04:23:59.672 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.672 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.672 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:59.672 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044734+0000", 2023-11-25T04:23:59.672 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00027582799999999999 2023-11-25T04:23:59.672 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.672 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.672 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:59.672 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044830+0000", 2023-11-25T04:23:59.673 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6248000000000004e-05 2023-11-25T04:23:59.673 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:59.673 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:59.673 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:59.673 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.673 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.673 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing_reply(3.b epoch 18/16 rep_tid 5 lcod 18'2)", 2023-11-25T04:23:59.673 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:59.673 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 72.533905532999995, 2023-11-25T04:23:59.673 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000344488, 2023-11-25T04:23:59.673 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:59.673 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:23:59.673 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:59.674 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.674 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:59.674 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:59.674 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:59.674 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.674 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.674 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:59.674 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:59.674 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:59.674 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.674 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.674 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:59.675 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:23:59.675 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.55e-07 2023-11-25T04:23:59.675 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.675 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.675 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:59.675 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044487+0000", 2023-11-25T04:23:59.675 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2470000000000001e-06 2023-11-25T04:23:59.675 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.675 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.675 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:59.675 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044491+0000", 2023-11-25T04:23:59.675 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9659999999999998e-06 2023-11-25T04:23:59.676 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.676 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.676 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:59.676 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044494+0000", 2023-11-25T04:23:59.676 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2310000000000001e-06 2023-11-25T04:23:59.676 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.676 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.676 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:59.676 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044756+0000", 2023-11-25T04:23:59.676 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00026172800000000003 2023-11-25T04:23:59.676 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.676 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.676 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:59.676 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044830+0000", 2023-11-25T04:23:59.677 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4160999999999997e-05 2023-11-25T04:23:59.677 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:59.677 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:59.677 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:59.677 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.677 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.677 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:39 3.6 3:654134d2:::mds0_openfiles.0:head [omap-clear,omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2023-11-25T04:23:59.677 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:59.677 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 67.474079490999998, 2023-11-25T04:23:59.677 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001788658, 2023-11-25T04:23:59.677 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:59.677 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:59.677 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:59.678 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.678 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:59.678 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:59.678 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:59.678 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.678 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.678 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:59.678 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:23:59.678 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:59.678 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.678 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.678 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:59.679 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104314+0000", 2023-11-25T04:23:59.679 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3099999999999999e-06 2023-11-25T04:23:59.679 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.679 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.679 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:59.679 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104325+0000", 2023-11-25T04:23:59.679 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1511999999999999e-05 2023-11-25T04:23:59.679 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.679 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.679 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:59.679 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104362+0000", 2023-11-25T04:23:59.679 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6903000000000002e-05 2023-11-25T04:23:59.680 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.680 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.680 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:59.680 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104387+0000", 2023-11-25T04:23:59.680 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4766e-05 2023-11-25T04:23:59.680 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.680 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.680 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:59.680 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104465+0000", 2023-11-25T04:23:59.680 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8119000000000002e-05 2023-11-25T04:23:59.680 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.680 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.680 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:59.681 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104557+0000", 2023-11-25T04:23:59.681 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2496000000000003e-05 2023-11-25T04:23:59.681 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.681 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.681 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:59.681 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104646+0000", 2023-11-25T04:23:59.681 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8818000000000002e-05 2023-11-25T04:23:59.681 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.681 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.681 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:59.681 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105201+0000", 2023-11-25T04:23:59.681 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00055506000000000002 2023-11-25T04:23:59.681 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.682 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.682 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:59.682 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105994+0000", 2023-11-25T04:23:59.682 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00079294700000000001 2023-11-25T04:23:59.682 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.682 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.682 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:59.682 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.106055+0000", 2023-11-25T04:23:59.682 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0566000000000003e-05 2023-11-25T04:23:59.682 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.682 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.682 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:59.683 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.106100+0000", 2023-11-25T04:23:59.683 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5160999999999997e-05 2023-11-25T04:23:59.683 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:59.683 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:59.683 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:59.683 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.683 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.683 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:43 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2023-11-25T04:23:59.683 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:59.683 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 52.473854494999998, 2023-11-25T04:23:59.683 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001751166, 2023-11-25T04:23:59.683 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:59.684 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:59.684 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:59.684 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.684 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:59.684 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:59.684 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:59.684 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.684 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.684 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:59.684 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:23:59.684 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:59.684 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.684 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.685 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:59.685 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104539+0000", 2023-11-25T04:23:59.685 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9529999999999999e-06 2023-11-25T04:23:59.685 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.685 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.685 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:59.685 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104557+0000", 2023-11-25T04:23:59.685 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7464000000000001e-05 2023-11-25T04:23:59.685 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.685 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.685 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:59.685 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104590+0000", 2023-11-25T04:23:59.686 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2966000000000003e-05 2023-11-25T04:23:59.686 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.686 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.686 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:59.686 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104608+0000", 2023-11-25T04:23:59.686 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8193e-05 2023-11-25T04:23:59.686 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.686 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.686 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:59.686 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104767+0000", 2023-11-25T04:23:59.686 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015882 2023-11-25T04:23:59.686 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.687 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.687 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:59.687 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104909+0000", 2023-11-25T04:23:59.687 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014244999999999999 2023-11-25T04:23:59.687 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.687 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.687 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:23:59.687 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.105059+0000", 2023-11-25T04:23:59.687 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014964399999999999 2023-11-25T04:23:59.687 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.687 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.687 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:59.687 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.105701+0000", 2023-11-25T04:23:59.688 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00064201099999999995 2023-11-25T04:23:59.688 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.688 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.688 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:23:59.688 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106233+0000", 2023-11-25T04:23:59.688 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00053178500000000001 2023-11-25T04:23:59.688 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.688 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.688 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:59.688 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106265+0000", 2023-11-25T04:23:59.688 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1986000000000002e-05 2023-11-25T04:23:59.688 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.689 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.689 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:59.689 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106288+0000", 2023-11-25T04:23:59.689 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2894000000000001e-05 2023-11-25T04:23:59.689 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:59.689 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:59.689 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:59.689 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.689 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.689 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:45 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e23)", 2023-11-25T04:23:59.689 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:23:59.689 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.473656454999997, 2023-11-25T04:23:59.689 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00098423299999999994, 2023-11-25T04:23:59.690 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:23:59.690 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:23:59.690 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:23:59.690 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.690 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:23:59.690 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:23:59.690 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:59.690 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.690 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.690 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:23:59.690 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:23:59.690 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:23:59.691 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.691 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.691 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:23:59.691 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104737+0000", 2023-11-25T04:23:59.691 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3309999999999998e-06 2023-11-25T04:23:59.691 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.691 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.691 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:23:59.691 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104748+0000", 2023-11-25T04:23:59.691 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1080999999999999e-05 2023-11-25T04:23:59.691 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.691 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.692 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:23:59.692 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104771+0000", 2023-11-25T04:23:59.692 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2707000000000001e-05 2023-11-25T04:23:59.692 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.692 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.692 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:23:59.692 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104800+0000", 2023-11-25T04:23:59.692 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9595e-05 2023-11-25T04:23:59.692 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.692 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.692 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:23:59.692 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104890+0000", 2023-11-25T04:23:59.692 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.9808999999999993e-05 2023-11-25T04:23:59.693 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.693 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.693 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:23:59.693 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105004+0000", 2023-11-25T04:23:59.693 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000114508 2023-11-25T04:23:59.693 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.693 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.693 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:23:59.693 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105634+0000", 2023-11-25T04:23:59.693 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00062985799999999996 2023-11-25T04:23:59.693 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.693 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.693 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:23:59.694 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105671+0000", 2023-11-25T04:23:59.694 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6640000000000002e-05 2023-11-25T04:23:59.694 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:23:59.694 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:23:59.694 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:23:59.694 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105719+0000", 2023-11-25T04:23:59.694 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7704000000000003e-05 2023-11-25T04:23:59.694 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:59.694 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:59.694 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:59.694 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:23:59.694 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:23:59.695 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:23:59.696 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:23:59.696 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:59.695+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:59.797 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:59.795+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:59.807 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:59.807 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:23:59.807 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2023-11-25T04:23:59.807 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:59.815 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:23:59.900 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:23:59.899+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:23:59.960 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:23:59.960 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:23:59.960 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2023-11-25T04:23:59.960 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2023-11-25T04:23:59.960 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:23:59.969 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:23:59.999 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:23:59.998+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:00.100 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:00.099+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:00.114 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:00.114 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2023-11-25T04:24:00.114 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2023-11-25T04:24:00.115 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2023-11-25T04:24:00.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.115 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:155 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'109, mlcod=11'105)", 2023-11-25T04:24:00.115 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:24:00.115 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 82.327526204999998, 2023-11-25T04:24:00.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061719019999999999, 2023-11-25T04:24:00.115 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:00.115 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:00.115 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:00.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.115 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:00.115 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:24:00.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:00.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.116 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:00.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:24:00.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:00.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.116 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:00.116 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784884+0000", 2023-11-25T04:24:00.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3099999999999999e-07 2023-11-25T04:24:00.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.117 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:00.117 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784942+0000", 2023-11-25T04:24:00.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8570000000000003e-05 2023-11-25T04:24:00.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.117 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:00.117 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784949+0000", 2023-11-25T04:24:00.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0310000000000001e-06 2023-11-25T04:24:00.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.117 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:00.118 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784953+0000", 2023-11-25T04:24:00.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6320000000000001e-06 2023-11-25T04:24:00.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.118 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:00.118 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784993+0000", 2023-11-25T04:24:00.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9737000000000003e-05 2023-11-25T04:24:00.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.118 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:00.118 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785011+0000", 2023-11-25T04:24:00.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8349999999999999e-05 2023-11-25T04:24:00.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:00.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791024+0000", 2023-11-25T04:24:00.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006012997 2023-11-25T04:24:00.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:00.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791055+0000", 2023-11-25T04:24:00.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1254000000000003e-05 2023-11-25T04:24:00.119 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:00.119 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:00.119 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:00.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.120 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:156 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'110, mlcod=11'105)", 2023-11-25T04:24:00.120 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:24:00.120 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 82.326940156000006, 2023-11-25T04:24:00.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005616116, 2023-11-25T04:24:00.120 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:00.120 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:00.120 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:00.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.120 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:00.120 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:24:00.121 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:00.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.121 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:00.121 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:24:00.121 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:00.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.121 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:00.121 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785470+0000", 2023-11-25T04:24:00.121 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0699999999999998e-07 2023-11-25T04:24:00.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.122 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:00.122 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785474+0000", 2023-11-25T04:24:00.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.048e-06 2023-11-25T04:24:00.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.122 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:00.122 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785480+0000", 2023-11-25T04:24:00.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7729999999999996e-06 2023-11-25T04:24:00.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.122 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:00.123 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785483+0000", 2023-11-25T04:24:00.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1970000000000001e-06 2023-11-25T04:24:00.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.123 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:00.123 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787936+0000", 2023-11-25T04:24:00.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024529040000000001 2023-11-25T04:24:00.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.123 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:00.123 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787947+0000", 2023-11-25T04:24:00.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1669e-05 2023-11-25T04:24:00.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.124 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:00.124 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791057+0000", 2023-11-25T04:24:00.124 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031095200000000002 2023-11-25T04:24:00.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.124 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:00.124 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791086+0000", 2023-11-25T04:24:00.124 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8697999999999999e-05 2023-11-25T04:24:00.124 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:00.124 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:00.125 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:00.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.125 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:157 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'111, mlcod=11'106)", 2023-11-25T04:24:00.125 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:24:00.125 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 82.326347597999998, 2023-11-25T04:24:00.125 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050590280000000001, 2023-11-25T04:24:00.125 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:00.125 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:00.125 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:00.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.125 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:00.125 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:24:00.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:00.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.126 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:00.126 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:24:00.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:00.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.126 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:00.126 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786063+0000", 2023-11-25T04:24:00.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2600000000000002e-07 2023-11-25T04:24:00.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.127 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:00.127 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786069+0000", 2023-11-25T04:24:00.127 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8839999999999999e-06 2023-11-25T04:24:00.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.127 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:00.127 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786079+0000", 2023-11-25T04:24:00.127 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0124e-05 2023-11-25T04:24:00.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.127 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:00.128 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786085+0000", 2023-11-25T04:24:00.128 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6660000000000002e-06 2023-11-25T04:24:00.128 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.128 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.128 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:00.128 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788160+0000", 2023-11-25T04:24:00.128 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020747669999999999 2023-11-25T04:24:00.128 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.128 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.128 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:00.128 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788180+0000", 2023-11-25T04:24:00.129 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0208999999999999e-05 2023-11-25T04:24:00.129 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.129 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.129 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:00.129 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791087+0000", 2023-11-25T04:24:00.129 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029064630000000002 2023-11-25T04:24:00.129 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.129 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.129 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:00.129 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791121+0000", 2023-11-25T04:24:00.129 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4289000000000003e-05 2023-11-25T04:24:00.129 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:00.130 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:00.130 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:00.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.130 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.130 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:158 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'112, mlcod=11'106)", 2023-11-25T04:24:00.130 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:24:00.130 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 82.325931205000003, 2023-11-25T04:24:00.130 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046663030000000001, 2023-11-25T04:24:00.130 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:00.130 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:00.130 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:00.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:00.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:24:00.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:00.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:00.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:24:00.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:00.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.132 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:00.132 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786479+0000", 2023-11-25T04:24:00.132 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2799999999999996e-07 2023-11-25T04:24:00.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.132 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:00.132 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786498+0000", 2023-11-25T04:24:00.132 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9069e-05 2023-11-25T04:24:00.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.132 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:00.132 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786508+0000", 2023-11-25T04:24:00.132 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5810000000000003e-06 2023-11-25T04:24:00.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.133 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.133 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:00.133 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786512+0000", 2023-11-25T04:24:00.133 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4889999999999997e-06 2023-11-25T04:24:00.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.133 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.133 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:00.133 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788438+0000", 2023-11-25T04:24:00.133 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019257059999999999 2023-11-25T04:24:00.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.133 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.134 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:00.134 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788449+0000", 2023-11-25T04:24:00.134 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1419e-05 2023-11-25T04:24:00.134 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.134 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.134 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:00.134 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791122+0000", 2023-11-25T04:24:00.134 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002672822 2023-11-25T04:24:00.134 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.134 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.134 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:00.134 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791145+0000", 2023-11-25T04:24:00.134 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2589e-05 2023-11-25T04:24:00.135 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:00.135 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:00.135 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:00.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.135 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:206 1.0 e12/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 12'152, mlcod=12'148)", 2023-11-25T04:24:00.135 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:24:00.135 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 81.814162734999996, 2023-11-25T04:24:00.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046728189999999999, 2023-11-25T04:24:00.135 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:00.135 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:00.135 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:00.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:00.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:24:00.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:00.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:00.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:24:00.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:00.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:00.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298248+0000", 2023-11-25T04:24:00.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3600000000000003e-07 2023-11-25T04:24:00.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:00.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298278+0000", 2023-11-25T04:24:00.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0474e-05 2023-11-25T04:24:00.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:00.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298288+0000", 2023-11-25T04:24:00.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0156000000000001e-05 2023-11-25T04:24:00.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.138 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:00.138 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298293+0000", 2023-11-25T04:24:00.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7400000000000004e-06 2023-11-25T04:24:00.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.138 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:00.138 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298333+0000", 2023-11-25T04:24:00.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9666000000000003e-05 2023-11-25T04:24:00.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.139 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:00.139 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298351+0000", 2023-11-25T04:24:00.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8544999999999999e-05 2023-11-25T04:24:00.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.139 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:00.139 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302891+0000", 2023-11-25T04:24:00.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045398519999999996 2023-11-25T04:24:00.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.139 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:00.139 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302920+0000", 2023-11-25T04:24:00.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8750000000000001e-05 2023-11-25T04:24:00.140 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:00.140 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:00.140 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:00.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.140 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:14 3.15 3:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:00.140 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:24:00.140 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.079028378000004, 2023-11-25T04:24:00.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010990437, 2023-11-25T04:24:00.140 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:00.140 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:00.140 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:00.141 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.141 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:00.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:24:00.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:00.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.141 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.141 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:00.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:24:00.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:00.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.141 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.141 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:00.142 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033383+0000", 2023-11-25T04:24:00.142 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9870000000000002e-06 2023-11-25T04:24:00.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.142 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:00.142 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033398+0000", 2023-11-25T04:24:00.142 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4569999999999999e-05 2023-11-25T04:24:00.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.142 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:00.142 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033411+0000", 2023-11-25T04:24:00.142 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3543999999999999e-05 2023-11-25T04:24:00.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:00.143 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033423+0000", 2023-11-25T04:24:00.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1905999999999999e-05 2023-11-25T04:24:00.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:00.143 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034433+0000", 2023-11-25T04:24:00.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00101022 2023-11-25T04:24:00.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:00.143 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034539+0000", 2023-11-25T04:24:00.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010597199999999999 2023-11-25T04:24:00.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:00.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034667+0000", 2023-11-25T04:24:00.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012765500000000001 2023-11-25T04:24:00.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:00.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037803+0000", 2023-11-25T04:24:00.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031362220000000001 2023-11-25T04:24:00.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:00.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:24:00.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0064964669999999997 2023-11-25T04:24:00.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:00.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044331+0000", 2023-11-25T04:24:00.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1544999999999997e-05 2023-11-25T04:24:00.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:00.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044372+0000", 2023-11-25T04:24:00.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0349000000000003e-05 2023-11-25T04:24:00.146 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:00.146 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:00.146 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:00.146 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.146 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:15 3.15 3:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:00.146 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:24:00.146 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.078368728000001, 2023-11-25T04:24:00.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010464362, 2023-11-25T04:24:00.146 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:00.146 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:00.147 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:00.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.147 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:00.147 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:24:00.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:00.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.147 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:00.147 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:24:00.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:00.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.148 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:00.148 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034042+0000", 2023-11-25T04:24:00.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0999999999999997e-07 2023-11-25T04:24:00.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.148 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:00.148 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034048+0000", 2023-11-25T04:24:00.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0390000000000001e-06 2023-11-25T04:24:00.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.148 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:00.148 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034059+0000", 2023-11-25T04:24:00.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1182999999999999e-05 2023-11-25T04:24:00.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:00.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034067+0000", 2023-11-25T04:24:00.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1559999999999995e-06 2023-11-25T04:24:00.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:00.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035005+0000", 2023-11-25T04:24:00.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00093747399999999999 2023-11-25T04:24:00.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:00.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035077+0000", 2023-11-25T04:24:00.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2747e-05 2023-11-25T04:24:00.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:00.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035219+0000", 2023-11-25T04:24:00.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000141877 2023-11-25T04:24:00.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:00.151 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037810+0000", 2023-11-25T04:24:00.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002590752 2023-11-25T04:24:00.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.151 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:00.151 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044437+0000", 2023-11-25T04:24:00.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066270110000000004 2023-11-25T04:24:00.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.151 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:00.151 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044481+0000", 2023-11-25T04:24:00.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4106000000000001e-05 2023-11-25T04:24:00.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.152 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:00.152 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044505+0000", 2023-11-25T04:24:00.152 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4207e-05 2023-11-25T04:24:00.152 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:00.152 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:00.152 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:00.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.152 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:16 3.1d 3:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:00.152 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:24:00.152 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.078326880000006, 2023-11-25T04:24:00.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010413923, 2023-11-25T04:24:00.153 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:00.153 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:00.153 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:00.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:00.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:24:00.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:00.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:00.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:24:00.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:00.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:00.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:24:00.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4499999999999997e-07 2023-11-25T04:24:00.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:00.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034087+0000", 2023-11-25T04:24:00.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9770000000000002e-06 2023-11-25T04:24:00.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.155 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:00.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034094+0000", 2023-11-25T04:24:00.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8809999999999998e-06 2023-11-25T04:24:00.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.155 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:00.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034100+0000", 2023-11-25T04:24:00.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8440000000000003e-06 2023-11-25T04:24:00.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.156 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:00.156 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034993+0000", 2023-11-25T04:24:00.156 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00089336799999999996 2023-11-25T04:24:00.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.156 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:00.156 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035099+0000", 2023-11-25T04:24:00.156 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106037 2023-11-25T04:24:00.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.156 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:00.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035276+0000", 2023-11-25T04:24:00.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017631799999999999 2023-11-25T04:24:00.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.157 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:00.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037815+0000", 2023-11-25T04:24:00.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025395959999999999 2023-11-25T04:24:00.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.157 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:00.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044420+0000", 2023-11-25T04:24:00.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066045749999999997 2023-11-25T04:24:00.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:00.158 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044463+0000", 2023-11-25T04:24:00.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3205999999999999e-05 2023-11-25T04:24:00.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:00.158 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044497+0000", 2023-11-25T04:24:00.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3476000000000003e-05 2023-11-25T04:24:00.158 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:00.158 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:00.159 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:00.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.159 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:17 3.1d 3:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:00.159 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:24:00.159 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.078301671000006, 2023-11-25T04:24:00.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01127356, 2023-11-25T04:24:00.159 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:00.159 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:00.159 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:00.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.159 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:00.160 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:24:00.160 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:00.160 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.160 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.160 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:00.160 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:24:00.160 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:00.160 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.160 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.160 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:00.160 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:24:00.160 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1100000000000001e-07 2023-11-25T04:24:00.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.161 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.161 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:00.161 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034112+0000", 2023-11-25T04:24:00.161 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8600000000000003e-06 2023-11-25T04:24:00.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.161 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.161 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:00.161 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034120+0000", 2023-11-25T04:24:00.161 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7700000000000001e-06 2023-11-25T04:24:00.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.161 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.162 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:00.162 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034127+0000", 2023-11-25T04:24:00.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0999999999999998e-06 2023-11-25T04:24:00.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.162 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:00.162 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035601+0000", 2023-11-25T04:24:00.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014739410000000001 2023-11-25T04:24:00.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.162 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:00.163 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035685+0000", 2023-11-25T04:24:00.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3677999999999994e-05 2023-11-25T04:24:00.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.163 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.163 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:00.163 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035827+0000", 2023-11-25T04:24:00.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014232599999999999 2023-11-25T04:24:00.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.163 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.163 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:00.163 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038725+0000", 2023-11-25T04:24:00.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028982779999999998 2023-11-25T04:24:00.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.164 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.164 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:00.164 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045323+0000", 2023-11-25T04:24:00.164 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0065980780000000003 2023-11-25T04:24:00.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.164 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.164 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:00.164 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045354+0000", 2023-11-25T04:24:00.164 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0463e-05 2023-11-25T04:24:00.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.164 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.165 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:00.165 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045381+0000", 2023-11-25T04:24:00.165 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7654999999999999e-05 2023-11-25T04:24:00.165 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:00.165 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:00.165 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:00.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.165 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.165 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:22 3.e 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:00.165 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:24:00.165 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.078275258000005, 2023-11-25T04:24:00.165 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010133422, 2023-11-25T04:24:00.165 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:00.166 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:00.166 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:00.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.166 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:00.166 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:24:00.166 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:00.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.166 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:00.166 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:24:00.166 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:00.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.167 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:00.167 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034135+0000", 2023-11-25T04:24:00.167 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-07 2023-11-25T04:24:00.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.167 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:00.167 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034138+0000", 2023-11-25T04:24:00.167 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6009999999999999e-06 2023-11-25T04:24:00.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.168 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:00.168 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034145+0000", 2023-11-25T04:24:00.168 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1169999999999998e-06 2023-11-25T04:24:00.168 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.168 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.168 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:00.168 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034150+0000", 2023-11-25T04:24:00.168 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9080000000000003e-06 2023-11-25T04:24:00.168 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.168 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.168 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:00.168 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034209+0000", 2023-11-25T04:24:00.168 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8149000000000002e-05 2023-11-25T04:24:00.169 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.169 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.169 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:00.169 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034415+0000", 2023-11-25T04:24:00.169 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020637699999999999 2023-11-25T04:24:00.169 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.169 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.169 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:00.169 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034584+0000", 2023-11-25T04:24:00.169 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016870399999999999 2023-11-25T04:24:00.169 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.169 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.169 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:00.170 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.036979+0000", 2023-11-25T04:24:00.170 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023957509999999998 2023-11-25T04:24:00.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.170 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:00.170 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044161+0000", 2023-11-25T04:24:00.170 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0071811590000000003 2023-11-25T04:24:00.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.170 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:00.170 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044224+0000", 2023-11-25T04:24:00.170 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3454000000000006e-05 2023-11-25T04:24:00.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.171 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:00.171 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044268+0000", 2023-11-25T04:24:00.171 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3761999999999999e-05 2023-11-25T04:24:00.171 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:00.171 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:00.171 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:00.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.172 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:23 3.e 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:00.173 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:24:00.173 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.078243849000003, 2023-11-25T04:24:00.173 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010281676, 2023-11-25T04:24:00.173 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:00.173 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:00.173 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:00.173 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.173 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:00.173 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:24:00.173 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:00.173 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.173 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.174 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:00.174 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:24:00.174 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:00.174 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.174 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.174 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:00.174 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:24:00.174 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5799999999999999e-07 2023-11-25T04:24:00.174 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.174 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.174 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:00.174 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034169+0000", 2023-11-25T04:24:00.174 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1879999999999998e-06 2023-11-25T04:24:00.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.175 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.175 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:00.175 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034177+0000", 2023-11-25T04:24:00.175 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7020000000000001e-06 2023-11-25T04:24:00.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.175 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.175 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:00.175 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034184+0000", 2023-11-25T04:24:00.175 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3910000000000003e-06 2023-11-25T04:24:00.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.175 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.176 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:00.176 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034924+0000", 2023-11-25T04:24:00.176 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00074032999999999998 2023-11-25T04:24:00.176 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.176 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.176 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:00.176 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034991+0000", 2023-11-25T04:24:00.176 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7347000000000004e-05 2023-11-25T04:24:00.176 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.176 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.176 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:00.176 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035096+0000", 2023-11-25T04:24:00.176 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010488 2023-11-25T04:24:00.177 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.177 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.177 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:00.177 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037588+0000", 2023-11-25T04:24:00.177 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024918309999999999 2023-11-25T04:24:00.177 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.177 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.177 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:00.177 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044367+0000", 2023-11-25T04:24:00.177 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067787589999999997 2023-11-25T04:24:00.177 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.177 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.177 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:00.177 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044415+0000", 2023-11-25T04:24:00.178 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7803000000000003e-05 2023-11-25T04:24:00.178 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.178 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.178 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:00.178 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044447+0000", 2023-11-25T04:24:00.178 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2886999999999999e-05 2023-11-25T04:24:00.178 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:00.178 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:00.178 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:00.178 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.178 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.178 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:30 3.15 3:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:00.179 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:24:00.179 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.078185409, 2023-11-25T04:24:00.179 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010365070000000001, 2023-11-25T04:24:00.179 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:00.179 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:00.179 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:00.179 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.179 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:00.179 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:24:00.179 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:00.179 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.179 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.180 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:00.180 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:24:00.180 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:00.180 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.180 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.180 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:00.180 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034225+0000", 2023-11-25T04:24:00.180 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7400000000000002e-07 2023-11-25T04:24:00.180 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.180 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.180 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:00.180 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034229+0000", 2023-11-25T04:24:00.180 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3100000000000002e-06 2023-11-25T04:24:00.181 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.181 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.181 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:00.181 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034239+0000", 2023-11-25T04:24:00.181 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9140000000000003e-06 2023-11-25T04:24:00.181 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.181 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.181 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:00.181 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034246+0000", 2023-11-25T04:24:00.181 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0149999999999998e-06 2023-11-25T04:24:00.181 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.181 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.181 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:00.182 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035572+0000", 2023-11-25T04:24:00.182 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001325655 2023-11-25T04:24:00.182 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.182 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.182 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:00.182 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035730+0000", 2023-11-25T04:24:00.182 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015827899999999999 2023-11-25T04:24:00.182 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.182 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.182 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:00.182 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035857+0000", 2023-11-25T04:24:00.182 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012644699999999999 2023-11-25T04:24:00.182 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.183 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.183 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:00.183 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038737+0000", 2023-11-25T04:24:00.183 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002880077 2023-11-25T04:24:00.183 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.183 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.183 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:00.183 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044540+0000", 2023-11-25T04:24:00.183 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058034239999999997 2023-11-25T04:24:00.183 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.183 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.183 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:00.184 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044569+0000", 2023-11-25T04:24:00.184 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8388999999999999e-05 2023-11-25T04:24:00.184 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.184 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.184 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:00.184 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044589+0000", 2023-11-25T04:24:00.184 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0686000000000002e-05 2023-11-25T04:24:00.184 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:00.184 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:00.184 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:00.184 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.184 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.185 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:33 3.1b 3:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:00.185 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:24:00.185 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.078139910999994, 2023-11-25T04:24:00.185 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099754049999999997, 2023-11-25T04:24:00.185 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:00.185 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:00.185 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:00.185 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.185 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:00.185 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:24:00.185 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:00.185 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.185 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.186 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:00.186 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:24:00.186 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:00.186 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.186 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.186 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:00.186 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:24:00.186 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4799999999999998e-07 2023-11-25T04:24:00.186 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.186 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.186 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:00.187 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034273+0000", 2023-11-25T04:24:00.187 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7819999999999998e-06 2023-11-25T04:24:00.187 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.187 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.187 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:00.187 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034285+0000", 2023-11-25T04:24:00.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2058e-05 2023-11-25T04:24:00.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.188 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:00.188 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034291+0000", 2023-11-25T04:24:00.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0680000000000002e-06 2023-11-25T04:24:00.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.188 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:00.188 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034370+0000", 2023-11-25T04:24:00.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8452e-05 2023-11-25T04:24:00.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:00.189 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034477+0000", 2023-11-25T04:24:00.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107471 2023-11-25T04:24:00.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:00.189 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044245+0000", 2023-11-25T04:24:00.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0097679259999999997 2023-11-25T04:24:00.189 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:00.189 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:00.189 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:00.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.189 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:36 3.1a 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:00.190 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:24:00.190 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.078093237999994, 2023-11-25T04:24:00.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058179099999999999, 2023-11-25T04:24:00.190 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:00.190 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:00.190 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:00.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.190 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:00.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:24:00.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:00.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.191 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:00.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:24:00.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:00.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.191 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:00.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034317+0000", 2023-11-25T04:24:00.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1999999999999999e-07 2023-11-25T04:24:00.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.191 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:00.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034321+0000", 2023-11-25T04:24:00.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.233e-06 2023-11-25T04:24:00.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.192 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:00.192 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034337+0000", 2023-11-25T04:24:00.192 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5308999999999999e-05 2023-11-25T04:24:00.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.192 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:00.192 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034354+0000", 2023-11-25T04:24:00.192 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7411999999999999e-05 2023-11-25T04:24:00.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.192 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:00.192 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034465+0000", 2023-11-25T04:24:00.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011154699999999999 2023-11-25T04:24:00.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.193 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:00.193 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034560+0000", 2023-11-25T04:24:00.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4186999999999998e-05 2023-11-25T04:24:00.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.193 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:00.193 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.040134+0000", 2023-11-25T04:24:00.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055746019999999997 2023-11-25T04:24:00.193 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:00.193 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:00.194 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:00.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.194 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:9 3.10 e18/16 3:09401a81:::601.00000000:head v 18'4, mlcod=0'0)", 2023-11-25T04:24:00.194 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:24:00.194 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.069198929999999, 2023-11-25T04:24:00.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061641020000000003, 2023-11-25T04:24:00.194 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:00.194 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:00.194 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:00.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:00.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:24:00.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:00.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.195 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:00.195 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:24:00.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:00.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.195 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:00.195 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:24:00.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5300000000000001e-07 2023-11-25T04:24:00.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.196 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:00.196 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043222+0000", 2023-11-25T04:24:00.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.082e-05 2023-11-25T04:24:00.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.196 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:00.196 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043228+0000", 2023-11-25T04:24:00.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6699999999999997e-06 2023-11-25T04:24:00.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.196 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:00.196 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043236+0000", 2023-11-25T04:24:00.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8189999999999992e-06 2023-11-25T04:24:00.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:00.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043507+0000", 2023-11-25T04:24:00.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000271012 2023-11-25T04:24:00.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:00.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043530+0000", 2023-11-25T04:24:00.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2659000000000001e-05 2023-11-25T04:24:00.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:00.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049321+0000", 2023-11-25T04:24:00.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057915129999999999 2023-11-25T04:24:00.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:00.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049375+0000", 2023-11-25T04:24:00.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3455999999999998e-05 2023-11-25T04:24:00.198 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:00.198 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:00.198 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:00.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.199 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:1 3.14 e18/16 3:292cf221:::200.00000000:head v 18'3, mlcod=0'0)", 2023-11-25T04:24:00.199 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:24:00.199 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.069162216999999, 2023-11-25T04:24:00.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0060983599999999997, 2023-11-25T04:24:00.199 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:00.199 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:00.199 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:00.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:00.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:24:00.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:00.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.200 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:00.200 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:24:00.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:00.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.200 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:00.200 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043248+0000", 2023-11-25T04:24:00.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9399999999999995e-07 2023-11-25T04:24:00.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.200 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:00.200 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043252+0000", 2023-11-25T04:24:00.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1799999999999998e-06 2023-11-25T04:24:00.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:00.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043259+0000", 2023-11-25T04:24:00.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0049999999999998e-06 2023-11-25T04:24:00.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:00.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043265+0000", 2023-11-25T04:24:00.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0859999999999998e-06 2023-11-25T04:24:00.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.202 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:00.202 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043311+0000", 2023-11-25T04:24:00.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6082999999999999e-05 2023-11-25T04:24:00.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.202 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:00.202 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043330+0000", 2023-11-25T04:24:00.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8556e-05 2023-11-25T04:24:00.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:00.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049303+0000", 2023-11-25T04:24:00.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059733470000000004 2023-11-25T04:24:00.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:00.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049346+0000", 2023-11-25T04:24:00.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2608999999999998e-05 2023-11-25T04:24:00.203 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:00.203 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:00.203 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:00.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.204 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:3 3.14 e18/16 3:287034c4:::1.00000000.inode:head v 18'4, mlcod=0'0)", 2023-11-25T04:24:00.204 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:24:00.204 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.068645711000002, 2023-11-25T04:24:00.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057040090000000003, 2023-11-25T04:24:00.204 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:00.204 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:00.204 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:00.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.204 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:00.204 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:24:00.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:00.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.205 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:00.205 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:24:00.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:00.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.205 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:00.205 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:24:00.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9799999999999999e-07 2023-11-25T04:24:00.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.205 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:00.205 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043785+0000", 2023-11-25T04:24:00.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0721999999999999e-05 2023-11-25T04:24:00.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:00.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043793+0000", 2023-11-25T04:24:00.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5839999999999995e-06 2023-11-25T04:24:00.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:00.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043803+0000", 2023-11-25T04:24:00.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0729999999999994e-06 2023-11-25T04:24:00.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:00.207 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043843+0000", 2023-11-25T04:24:00.207 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0747999999999997e-05 2023-11-25T04:24:00.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.207 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:00.207 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043856+0000", 2023-11-25T04:24:00.207 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2697000000000001e-05 2023-11-25T04:24:00.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.207 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:00.207 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049426+0000", 2023-11-25T04:24:00.207 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055704409999999998 2023-11-25T04:24:00.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.208 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:00.208 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049468+0000", 2023-11-25T04:24:00.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1446000000000003e-05 2023-11-25T04:24:00.208 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:00.208 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:00.208 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:00.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.208 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "pg_update_log_missing(3.14 epoch 18/16 rep_tid 171 entries 18'5 (0'0) error 3:2e2fa760:::200.00000001:head by mds.0.3:32 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:24:00.208 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:24:00.208 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.068484748000003, 2023-11-25T04:24:00.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005556938, 2023-11-25T04:24:00.209 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:00.209 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2023-11-25T04:24:00.209 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:00.209 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.209 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:00.209 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:24:00.209 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:00.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.209 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.209 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:00.209 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:24:00.209 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:00.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.210 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:00.210 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:24:00.210 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.03e-07 2023-11-25T04:24:00.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.210 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:00.210 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043931+0000", 2023-11-25T04:24:00.210 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1079999999999998e-06 2023-11-25T04:24:00.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.210 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:00.210 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043944+0000", 2023-11-25T04:24:00.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2852e-05 2023-11-25T04:24:00.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.211 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:00.211 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043955+0000", 2023-11-25T04:24:00.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1037e-05 2023-11-25T04:24:00.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.211 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:00.211 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044088+0000", 2023-11-25T04:24:00.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013269599999999999 2023-11-25T04:24:00.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.211 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:00.212 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049482+0000", 2023-11-25T04:24:00.212 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053940419999999999 2023-11-25T04:24:00.212 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:00.212 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:00.212 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:00.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.212 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:10 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'5, mlcod=0'0)", 2023-11-25T04:24:00.212 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:24:00.212 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.068394720000001, 2023-11-25T04:24:00.212 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054493099999999997, 2023-11-25T04:24:00.212 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:00.212 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:00.213 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:00.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.213 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:00.213 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:24:00.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:00.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.213 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:00.213 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:24:00.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:00.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.213 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:00.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:24:00.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6100000000000002e-07 2023-11-25T04:24:00.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.214 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.214 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:00.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044019+0000", 2023-11-25T04:24:00.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.839e-06 2023-11-25T04:24:00.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.214 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.214 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:00.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044026+0000", 2023-11-25T04:24:00.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5869999999999998e-06 2023-11-25T04:24:00.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.214 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:00.215 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044031+0000", 2023-11-25T04:24:00.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4909999999999996e-06 2023-11-25T04:24:00.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:00.215 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044070+0000", 2023-11-25T04:24:00.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8829999999999999e-05 2023-11-25T04:24:00.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:00.215 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044081+0000", 2023-11-25T04:24:00.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1447999999999999e-05 2023-11-25T04:24:00.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:00.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049406+0000", 2023-11-25T04:24:00.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053246480000000004 2023-11-25T04:24:00.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:00.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049464+0000", 2023-11-25T04:24:00.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8205999999999998e-05 2023-11-25T04:24:00.216 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:00.216 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:00.216 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:00.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.217 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:11 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'6, mlcod=0'0)", 2023-11-25T04:24:00.217 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:24:00.217 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 73.068193456000003, 2023-11-25T04:24:00.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052673499999999996, 2023-11-25T04:24:00.217 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:00.217 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:00.217 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:00.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.217 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:00.217 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:24:00.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:00.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.218 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:00.218 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:24:00.218 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:00.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.218 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:00.218 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:24:00.218 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0199999999999998e-07 2023-11-25T04:24:00.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.219 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:00.219 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044220+0000", 2023-11-25T04:24:00.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7720000000000001e-06 2023-11-25T04:24:00.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.219 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:00.219 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044226+0000", 2023-11-25T04:24:00.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8579999999999996e-06 2023-11-25T04:24:00.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.219 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:00.219 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044232+0000", 2023-11-25T04:24:00.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1700000000000002e-06 2023-11-25T04:24:00.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:00.220 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044274+0000", 2023-11-25T04:24:00.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1254999999999997e-05 2023-11-25T04:24:00.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:00.220 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:24:00.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6880999999999999e-05 2023-11-25T04:24:00.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:00.220 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049466+0000", 2023-11-25T04:24:00.221 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051659219999999999 2023-11-25T04:24:00.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:00.221 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:00.221 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:00.221 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049483+0000", 2023-11-25T04:24:00.221 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.719e-05 2023-11-25T04:24:00.221 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:00.221 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:00.221 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:00.221 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:00.221 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:00.221 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:00.224 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:00.221+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:00.225 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:24:00.324 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:00.322+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:00.341 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:00.342 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:24:00.342 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2023-11-25T04:24:00.343 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:00.350 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:24:00.425 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:00.424+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:00.495 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:00.495 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:24:00.495 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2023-11-25T04:24:00.496 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2023-11-25T04:24:00.496 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:00.504 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:24:00.526 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:00.525+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:00.611 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:00.612 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2023-11-25T04:24:00.612 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2023-11-25T04:24:00.612 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2023-11-25T04:24:00.612 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.612 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:00.612 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:24:00.612 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 73.577837403000004, 2023-11-25T04:24:00.612 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01072943, 2023-11-25T04:24:00.612 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:00.612 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:00.613 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:00.613 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.613 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:00.613 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:24:00.613 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:00.613 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.613 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.613 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:00.613 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:24:00.613 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:00.613 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.613 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.613 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:00.614 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031829+0000", 2023-11-25T04:24:00.614 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.666e-06 2023-11-25T04:24:00.614 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.614 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.614 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:00.614 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031839+0000", 2023-11-25T04:24:00.614 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9490000000000008e-06 2023-11-25T04:24:00.614 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.614 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.614 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:00.614 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031857+0000", 2023-11-25T04:24:00.614 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8093000000000001e-05 2023-11-25T04:24:00.615 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.615 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.615 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:00.615 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031882+0000", 2023-11-25T04:24:00.615 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4294000000000002e-05 2023-11-25T04:24:00.615 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.615 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.615 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:00.615 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033130+0000", 2023-11-25T04:24:00.615 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001248264 2023-11-25T04:24:00.615 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.615 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.615 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:00.615 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033426+0000", 2023-11-25T04:24:00.616 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029620999999999998 2023-11-25T04:24:00.616 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.616 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.616 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:00.616 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033814+0000", 2023-11-25T04:24:00.616 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000387742 2023-11-25T04:24:00.616 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.616 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.616 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:00.616 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036793+0000", 2023-11-25T04:24:00.616 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002979598 2023-11-25T04:24:00.616 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.616 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.617 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:00.617 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042455+0000", 2023-11-25T04:24:00.617 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0056619260000000003 2023-11-25T04:24:00.617 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.617 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.617 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:00.617 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042502+0000", 2023-11-25T04:24:00.617 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7139999999999999e-05 2023-11-25T04:24:00.617 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.617 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.617 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:00.617 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042557+0000", 2023-11-25T04:24:00.617 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4548e-05 2023-11-25T04:24:00.617 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:00.618 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:00.618 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:00.618 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.618 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.618 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:00.619 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:24:00.619 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 73.576638868000003, 2023-11-25T04:24:00.620 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011505202000000001, 2023-11-25T04:24:00.620 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:00.620 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:00.620 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:00.620 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.620 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:00.620 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:24:00.620 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:00.620 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.620 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.620 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:00.620 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:24:00.621 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:00.621 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.621 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.621 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:00.621 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033027+0000", 2023-11-25T04:24:00.621 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0699999999999999e-06 2023-11-25T04:24:00.621 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.621 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.621 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:00.621 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033035+0000", 2023-11-25T04:24:00.621 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0830000000000004e-06 2023-11-25T04:24:00.621 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.621 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.621 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:00.622 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033059+0000", 2023-11-25T04:24:00.622 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3975e-05 2023-11-25T04:24:00.622 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.622 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.622 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:00.622 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033482+0000", 2023-11-25T04:24:00.622 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042253700000000001 2023-11-25T04:24:00.622 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.622 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.622 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:00.622 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034520+0000", 2023-11-25T04:24:00.622 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001037991 2023-11-25T04:24:00.623 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.623 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.623 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:00.623 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034611+0000", 2023-11-25T04:24:00.623 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1113999999999995e-05 2023-11-25T04:24:00.623 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.623 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.623 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:00.623 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034769+0000", 2023-11-25T04:24:00.623 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000157641 2023-11-25T04:24:00.623 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.623 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.623 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:00.624 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038292+0000", 2023-11-25T04:24:00.624 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035231160000000002 2023-11-25T04:24:00.624 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.624 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.624 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:00.624 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044440+0000", 2023-11-25T04:24:00.624 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0061486980000000002 2023-11-25T04:24:00.624 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.624 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.624 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:00.624 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044495+0000", 2023-11-25T04:24:00.624 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4339000000000003e-05 2023-11-25T04:24:00.624 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.625 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.625 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:00.625 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044531+0000", 2023-11-25T04:24:00.625 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6637999999999997e-05 2023-11-25T04:24:00.625 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:00.625 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:00.625 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:00.625 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.625 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.625 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:00.625 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:24:00.625 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 73.576164513999998, 2023-11-25T04:24:00.625 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0057634039999999997, 2023-11-25T04:24:00.626 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:00.626 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:00.626 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:00.626 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.626 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:00.626 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:24:00.626 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:00.626 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.626 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.626 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:00.626 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:24:00.627 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:00.627 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.627 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.627 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:00.627 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:24:00.627 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7900000000000007e-07 2023-11-25T04:24:00.627 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.627 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.627 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:00.627 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033507+0000", 2023-11-25T04:24:00.627 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8900000000000004e-06 2023-11-25T04:24:00.627 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.627 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.627 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:00.627 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033519+0000", 2023-11-25T04:24:00.628 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1756000000000001e-05 2023-11-25T04:24:00.628 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.628 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.628 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:00.628 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033527+0000", 2023-11-25T04:24:00.628 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3610000000000001e-06 2023-11-25T04:24:00.628 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.628 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.628 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:00.628 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034451+0000", 2023-11-25T04:24:00.628 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00092372400000000001 2023-11-25T04:24:00.628 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.629 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.629 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:00.629 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034809+0000", 2023-11-25T04:24:00.629 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00035757699999999998 2023-11-25T04:24:00.629 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.629 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.629 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:00.629 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034965+0000", 2023-11-25T04:24:00.629 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015638699999999999 2023-11-25T04:24:00.629 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.629 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.629 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:00.629 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038299+0000", 2023-11-25T04:24:00.629 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0033336559999999999 2023-11-25T04:24:00.630 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.630 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.630 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:00.630 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039169+0000", 2023-11-25T04:24:00.630 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00087066400000000005 2023-11-25T04:24:00.630 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.630 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.630 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:00.630 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039226+0000", 2023-11-25T04:24:00.630 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6511999999999997e-05 2023-11-25T04:24:00.630 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.630 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.631 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:00.631 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039264+0000", 2023-11-25T04:24:00.631 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7898000000000002e-05 2023-11-25T04:24:00.631 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:00.631 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:00.631 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:00.631 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.631 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.631 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:25 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:00.631 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:24:00.631 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 73.576085930000005, 2023-11-25T04:24:00.631 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096656169999999996, 2023-11-25T04:24:00.632 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:00.632 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:00.632 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:00.632 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.632 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:00.632 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:24:00.632 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:00.632 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.632 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.632 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:00.632 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:24:00.632 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:00.632 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.633 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.633 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:00.633 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033580+0000", 2023-11-25T04:24:00.633 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8199999999999999e-07 2023-11-25T04:24:00.633 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.633 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.633 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:00.633 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033585+0000", 2023-11-25T04:24:00.633 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3680000000000001e-06 2023-11-25T04:24:00.633 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.633 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.633 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:00.633 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033600+0000", 2023-11-25T04:24:00.634 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4331000000000001e-05 2023-11-25T04:24:00.634 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.634 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.634 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:00.634 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033606+0000", 2023-11-25T04:24:00.634 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5780000000000004e-06 2023-11-25T04:24:00.634 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.634 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.634 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:00.634 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035747+0000", 2023-11-25T04:24:00.634 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0021406530000000002 2023-11-25T04:24:00.634 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.634 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.634 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:00.635 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035901+0000", 2023-11-25T04:24:00.635 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000153931 2023-11-25T04:24:00.635 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.635 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.635 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:00.635 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036193+0000", 2023-11-25T04:24:00.635 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029211299999999999 2023-11-25T04:24:00.635 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.635 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.635 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:00.635 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038655+0000", 2023-11-25T04:24:00.635 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0024621640000000002 2023-11-25T04:24:00.635 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.636 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.636 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:00.636 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043130+0000", 2023-11-25T04:24:00.636 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.004474698 2023-11-25T04:24:00.636 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.636 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.636 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:00.636 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043204+0000", 2023-11-25T04:24:00.636 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4091999999999995e-05 2023-11-25T04:24:00.636 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.636 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.636 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:00.636 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043245+0000", 2023-11-25T04:24:00.637 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0907e-05 2023-11-25T04:24:00.637 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:00.637 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:00.637 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:00.637 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.637 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.637 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:26 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:00.637 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:00.637 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 73.576050105999997, 2023-11-25T04:24:00.637 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011086945000000001, 2023-11-25T04:24:00.637 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:00.637 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:00.637 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:00.637 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.638 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:00.638 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:00.638 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:00.638 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.638 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.638 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:00.638 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:00.638 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:00.638 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.638 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.638 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:00.638 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:00.638 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1699999999999998e-07 2023-11-25T04:24:00.638 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.639 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.639 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:00.639 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033627+0000", 2023-11-25T04:24:00.639 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.163e-05 2023-11-25T04:24:00.639 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.639 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.639 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:00.639 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033636+0000", 2023-11-25T04:24:00.639 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6859999999999995e-06 2023-11-25T04:24:00.639 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.639 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.639 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:00.639 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033642+0000", 2023-11-25T04:24:00.640 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3149999999999997e-06 2023-11-25T04:24:00.640 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.640 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.640 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:00.640 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035414+0000", 2023-11-25T04:24:00.640 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0017716380000000001 2023-11-25T04:24:00.640 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.640 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.640 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:00.640 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035539+0000", 2023-11-25T04:24:00.640 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000125477 2023-11-25T04:24:00.640 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.640 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.640 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:00.641 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035672+0000", 2023-11-25T04:24:00.641 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000133316 2023-11-25T04:24:00.641 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.641 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.641 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:00.641 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038649+0000", 2023-11-25T04:24:00.641 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029764679999999999 2023-11-25T04:24:00.641 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.641 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.641 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:00.641 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044578+0000", 2023-11-25T04:24:00.641 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0059293510000000002 2023-11-25T04:24:00.641 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.641 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.641 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:00.642 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044651+0000", 2023-11-25T04:24:00.642 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2513000000000002e-05 2023-11-25T04:24:00.642 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.642 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.642 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:00.642 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044702+0000", 2023-11-25T04:24:00.642 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1034e-05 2023-11-25T04:24:00.642 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:00.642 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:00.642 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:00.642 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.642 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.642 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:29 3.b 3:d0630e4c:::mds0_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:00.643 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:00.643 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 73.576012186, 2023-11-25T04:24:00.643 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010980227, 2023-11-25T04:24:00.643 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:00.643 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:00.643 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:00.643 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.643 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:00.643 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:00.643 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:00.643 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.643 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.643 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:00.643 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:00.644 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:00.644 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.644 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.644 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:00.644 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:00.645 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3499999999999996e-07 2023-11-25T04:24:00.645 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.645 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.645 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:00.645 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033658+0000", 2023-11-25T04:24:00.645 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1280000000000001e-06 2023-11-25T04:24:00.645 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.645 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.645 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:00.645 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033667+0000", 2023-11-25T04:24:00.646 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1430000000000004e-06 2023-11-25T04:24:00.646 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.646 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.646 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:00.646 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033678+0000", 2023-11-25T04:24:00.646 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0906e-05 2023-11-25T04:24:00.646 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.646 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.646 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:00.646 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034456+0000", 2023-11-25T04:24:00.646 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00077833899999999996 2023-11-25T04:24:00.646 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.646 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.646 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:00.647 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034575+0000", 2023-11-25T04:24:00.647 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011926000000000001 2023-11-25T04:24:00.647 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.647 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.647 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:00.647 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035001+0000", 2023-11-25T04:24:00.647 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042550999999999998 2023-11-25T04:24:00.647 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.647 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.647 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:00.647 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.037923+0000", 2023-11-25T04:24:00.647 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029219670000000001 2023-11-25T04:24:00.647 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.647 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.648 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:00.648 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044538+0000", 2023-11-25T04:24:00.648 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0066152349999999997 2023-11-25T04:24:00.648 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.648 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.648 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:00.648 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044594+0000", 2023-11-25T04:24:00.648 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5593000000000001e-05 2023-11-25T04:24:00.648 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.648 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.648 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:00.648 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044633+0000", 2023-11-25T04:24:00.648 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9511000000000002e-05 2023-11-25T04:24:00.648 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:00.649 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:00.649 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:00.649 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.649 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.649 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:34 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:00.649 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:24:00.649 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 73.575762073000007, 2023-11-25T04:24:00.649 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010920445000000001, 2023-11-25T04:24:00.649 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:00.649 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:00.649 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:00.649 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.649 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:00.650 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:24:00.650 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:00.650 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.650 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.650 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:00.650 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:24:00.650 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:00.650 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.650 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.650 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:00.650 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033904+0000", 2023-11-25T04:24:00.650 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6599999999999994e-07 2023-11-25T04:24:00.650 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.651 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.651 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:00.651 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033909+0000", 2023-11-25T04:24:00.651 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5639999999999996e-06 2023-11-25T04:24:00.651 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.651 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.651 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:00.651 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033923+0000", 2023-11-25T04:24:00.651 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3028e-05 2023-11-25T04:24:00.651 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.651 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.651 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:00.651 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034043+0000", 2023-11-25T04:24:00.652 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000120328 2023-11-25T04:24:00.652 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.652 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.652 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:00.652 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035576+0000", 2023-11-25T04:24:00.652 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015333720000000001 2023-11-25T04:24:00.652 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.652 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.652 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:00.652 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035699+0000", 2023-11-25T04:24:00.652 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000123001 2023-11-25T04:24:00.652 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.652 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.652 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:00.653 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044823+0000", 2023-11-25T04:24:00.653 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0091241859999999994 2023-11-25T04:24:00.653 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:00.653 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:00.653 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:00.653 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.653 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.653 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:35 3.1f 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:00.653 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:24:00.653 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 73.574457867999996, 2023-11-25T04:24:00.653 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096164779999999995, 2023-11-25T04:24:00.653 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:00.653 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:00.653 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:00.654 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.654 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:00.654 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:24:00.654 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:00.654 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.654 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.654 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:00.654 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:24:00.654 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:00.654 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.654 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.654 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:00.654 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035208+0000", 2023-11-25T04:24:00.655 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2699999999999998e-07 2023-11-25T04:24:00.655 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.655 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.655 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:00.655 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035213+0000", 2023-11-25T04:24:00.655 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.6070000000000001e-06 2023-11-25T04:24:00.655 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.655 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.655 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:00.655 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035350+0000", 2023-11-25T04:24:00.655 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013693599999999999 2023-11-25T04:24:00.655 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.655 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.655 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:00.655 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035358+0000", 2023-11-25T04:24:00.656 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0099999999999995e-06 2023-11-25T04:24:00.656 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.656 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.656 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:00.656 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036260+0000", 2023-11-25T04:24:00.656 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00090240100000000005 2023-11-25T04:24:00.656 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.656 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.656 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:00.656 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036517+0000", 2023-11-25T04:24:00.656 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00025736299999999998 2023-11-25T04:24:00.656 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.656 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.656 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:00.657 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044824+0000", 2023-11-25T04:24:00.657 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0083063340000000003 2023-11-25T04:24:00.657 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:00.657 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:00.657 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:00.658 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.658 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.658 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:37 3.1e 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:00.658 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:24:00.658 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 73.574291181999996, 2023-11-25T04:24:00.658 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0078322640000000002, 2023-11-25T04:24:00.658 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:00.658 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:00.658 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:00.658 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.658 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:00.659 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:24:00.659 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:00.659 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.659 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.659 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:00.659 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:24:00.659 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:00.659 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.659 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.659 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:00.659 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035375+0000", 2023-11-25T04:24:00.660 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9100000000000003e-07 2023-11-25T04:24:00.660 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.660 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.660 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:00.660 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035379+0000", 2023-11-25T04:24:00.660 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9430000000000002e-06 2023-11-25T04:24:00.660 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.660 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.660 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:00.660 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035388+0000", 2023-11-25T04:24:00.660 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2760000000000006e-06 2023-11-25T04:24:00.660 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.660 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.661 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:00.661 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035396+0000", 2023-11-25T04:24:00.661 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8480000000000001e-06 2023-11-25T04:24:00.661 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.661 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.661 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:00.661 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035452+0000", 2023-11-25T04:24:00.661 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5831000000000001e-05 2023-11-25T04:24:00.661 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.661 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.661 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:00.661 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035934+0000", 2023-11-25T04:24:00.661 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00048265599999999998 2023-11-25T04:24:00.661 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.662 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.662 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:00.662 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043206+0000", 2023-11-25T04:24:00.662 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.007271919 2023-11-25T04:24:00.662 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:00.662 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:00.662 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:00.662 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.662 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.662 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing(3.1a epoch 18/16 rep_tid 3 entries 18'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:36 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:24:00.662 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:24:00.662 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 73.573015596000005, 2023-11-25T04:24:00.662 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002073414, 2023-11-25T04:24:00.663 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:00.663 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:24:00.663 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:00.663 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.663 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:00.663 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:24:00.663 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:00.663 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.663 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.663 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:00.663 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:24:00.663 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:00.663 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.663 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.664 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:00.664 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036650+0000", 2023-11-25T04:24:00.664 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7300000000000002e-07 2023-11-25T04:24:00.664 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.664 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.664 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:00.664 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036659+0000", 2023-11-25T04:24:00.664 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.7499999999999992e-06 2023-11-25T04:24:00.664 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.664 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.664 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:00.664 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036673+0000", 2023-11-25T04:24:00.664 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4917e-05 2023-11-25T04:24:00.664 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.665 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.665 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:00.665 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036687+0000", 2023-11-25T04:24:00.665 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3883000000000001e-05 2023-11-25T04:24:00.665 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.665 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.665 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:00.665 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036781+0000", 2023-11-25T04:24:00.665 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3219000000000003e-05 2023-11-25T04:24:00.665 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.665 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.665 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:00.665 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038723+0000", 2023-11-25T04:24:00.665 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019422720000000001 2023-11-25T04:24:00.666 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:00.666 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:00.666 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:00.666 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.666 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.666 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:28 3.13 e18/16 3:cd5a64a3:::100.00000000:head v 18'1, mlcod=0'0)", 2023-11-25T04:24:00.666 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:24:00.666 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 73.569354438000005, 2023-11-25T04:24:00.666 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001850546, 2023-11-25T04:24:00.666 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:00.666 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:00.666 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:00.666 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.666 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:00.667 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:24:00.667 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:00.667 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.667 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.667 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:00.667 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:24:00.667 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:00.667 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.667 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.667 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:00.667 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:24:00.668 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7800000000000002e-07 2023-11-25T04:24:00.668 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.668 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.668 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:00.668 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040331+0000", 2023-11-25T04:24:00.668 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0449999999999999e-05 2023-11-25T04:24:00.668 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.668 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.668 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:00.668 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040339+0000", 2023-11-25T04:24:00.668 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0709999999999995e-06 2023-11-25T04:24:00.668 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.668 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.668 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:00.669 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040347+0000", 2023-11-25T04:24:00.669 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7079999999999997e-06 2023-11-25T04:24:00.669 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.669 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.669 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:00.669 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040410+0000", 2023-11-25T04:24:00.669 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2453000000000007e-05 2023-11-25T04:24:00.669 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.669 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.670 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:00.670 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040422+0000", 2023-11-25T04:24:00.670 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2429000000000001e-05 2023-11-25T04:24:00.670 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.670 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.670 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:00.670 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042129+0000", 2023-11-25T04:24:00.670 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001707266 2023-11-25T04:24:00.670 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.670 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.670 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:00.670 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042161+0000", 2023-11-25T04:24:00.671 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1791000000000001e-05 2023-11-25T04:24:00.671 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:00.671 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:00.671 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:00.671 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.671 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.671 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:27 3.13 e18/16 3:c91d4a1d:::mds0_inotable:head v 18'2, mlcod=0'0)", 2023-11-25T04:24:00.671 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:24:00.671 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 73.569150914999994, 2023-11-25T04:24:00.671 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0016886430000000001, 2023-11-25T04:24:00.671 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:00.671 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:00.673 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:00.673 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.674 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:00.674 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:24:00.674 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:00.674 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.674 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.674 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:00.674 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:24:00.674 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:00.674 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.674 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.674 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:00.674 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:24:00.674 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3999999999999997e-07 2023-11-25T04:24:00.675 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.675 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.675 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:00.675 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040520+0000", 2023-11-25T04:24:00.675 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2989999999999996e-06 2023-11-25T04:24:00.675 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.675 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.675 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:00.675 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040527+0000", 2023-11-25T04:24:00.675 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5730000000000002e-06 2023-11-25T04:24:00.675 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.675 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.675 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:00.676 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040535+0000", 2023-11-25T04:24:00.676 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8630000000000008e-06 2023-11-25T04:24:00.676 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.676 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.676 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:00.676 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040683+0000", 2023-11-25T04:24:00.676 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000148002 2023-11-25T04:24:00.676 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.676 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.676 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:00.676 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040700+0000", 2023-11-25T04:24:00.676 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7065e-05 2023-11-25T04:24:00.676 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.676 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.677 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:00.677 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042163+0000", 2023-11-25T04:24:00.677 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001462767 2023-11-25T04:24:00.677 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.677 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.677 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:00.677 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042203+0000", 2023-11-25T04:24:00.677 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9734000000000003e-05 2023-11-25T04:24:00.677 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:00.677 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:00.677 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:00.677 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.677 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.677 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:20 3.1c e18/16 3:3ed09add:::607.00000000:head v 18'1, mlcod=0'0)", 2023-11-25T04:24:00.678 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:24:00.678 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 73.569115874000005, 2023-11-25T04:24:00.678 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015271, 2023-11-25T04:24:00.678 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:00.678 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:00.678 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:00.678 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.678 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:00.678 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:24:00.678 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:00.678 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.678 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.678 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:00.679 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:24:00.679 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:00.679 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.679 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.679 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:00.679 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:24:00.679 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4799999999999999e-07 2023-11-25T04:24:00.679 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.679 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.679 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:00.679 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040561+0000", 2023-11-25T04:24:00.679 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1800000000000001e-05 2023-11-25T04:24:00.679 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.679 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.680 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:00.680 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040568+0000", 2023-11-25T04:24:00.680 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7730000000000002e-06 2023-11-25T04:24:00.680 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.680 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.680 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:00.680 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040587+0000", 2023-11-25T04:24:00.680 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8691000000000001e-05 2023-11-25T04:24:00.680 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.680 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.680 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:00.680 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040638+0000", 2023-11-25T04:24:00.680 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1068e-05 2023-11-25T04:24:00.680 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.681 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.681 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:00.681 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040670+0000", 2023-11-25T04:24:00.681 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2125999999999997e-05 2023-11-25T04:24:00.681 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.681 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.681 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:00.681 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042032+0000", 2023-11-25T04:24:00.681 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001361821 2023-11-25T04:24:00.681 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.681 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.681 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:00.681 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042076+0000", 2023-11-25T04:24:00.681 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4472999999999999e-05 2023-11-25T04:24:00.682 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:00.682 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:00.682 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:00.682 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.682 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.682 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:21 3.1c e18/16 3:3ed09add:::607.00000000:head v 18'2, mlcod=0'0)", 2023-11-25T04:24:00.682 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:24:00.682 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 73.568881149000006, 2023-11-25T04:24:00.682 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0013228160000000001, 2023-11-25T04:24:00.682 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:00.682 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:00.682 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:00.682 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.682 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:00.683 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:24:00.683 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:00.683 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.683 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.683 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:00.683 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:24:00.683 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:00.683 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.683 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.683 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:00.683 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:24:00.683 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9299999999999999e-07 2023-11-25T04:24:00.683 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.683 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.684 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:00.684 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040790+0000", 2023-11-25T04:24:00.684 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5770000000000001e-06 2023-11-25T04:24:00.684 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.684 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.684 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:00.684 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040799+0000", 2023-11-25T04:24:00.684 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6439999999999995e-06 2023-11-25T04:24:00.684 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.684 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.684 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:00.684 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040806+0000", 2023-11-25T04:24:00.684 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0180000000000004e-06 2023-11-25T04:24:00.684 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.685 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.685 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:00.685 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040907+0000", 2023-11-25T04:24:00.685 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000100096 2023-11-25T04:24:00.685 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.685 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.685 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:00.685 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040925+0000", 2023-11-25T04:24:00.685 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8536999999999999e-05 2023-11-25T04:24:00.685 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.685 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.685 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:00.685 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042079+0000", 2023-11-25T04:24:00.685 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00115358 2023-11-25T04:24:00.686 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.686 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.686 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:00.686 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042107+0000", 2023-11-25T04:24:00.686 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7971e-05 2023-11-25T04:24:00.686 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:00.686 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:00.686 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:00.686 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.686 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.686 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop_reply(mds.0.3:4 3.1f e18/16 ondisk, result = 0)", 2023-11-25T04:24:00.686 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:24:00.686 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 73.567541758999994, 2023-11-25T04:24:00.687 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00044092900000000001, 2023-11-25T04:24:00.687 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:00.687 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:00.687 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:00.687 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.687 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:00.687 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:24:00.687 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:00.687 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.687 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.687 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:00.687 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:24:00.687 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:00.687 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.687 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.688 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:00.688 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042124+0000", 2023-11-25T04:24:00.688 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1800000000000005e-07 2023-11-25T04:24:00.688 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.688 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.688 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:00.688 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042128+0000", 2023-11-25T04:24:00.688 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2289999999999997e-06 2023-11-25T04:24:00.688 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.688 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.688 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:00.688 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042138+0000", 2023-11-25T04:24:00.688 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9729999999999992e-06 2023-11-25T04:24:00.689 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.689 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.689 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:00.689 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042145+0000", 2023-11-25T04:24:00.689 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.336e-06 2023-11-25T04:24:00.689 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.689 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.689 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:00.689 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042441+0000", 2023-11-25T04:24:00.689 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029650500000000002 2023-11-25T04:24:00.689 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.689 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.689 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:00.689 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042448+0000", 2023-11-25T04:24:00.690 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4100000000000002e-06 2023-11-25T04:24:00.690 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.690 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.690 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:00.690 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042564+0000", 2023-11-25T04:24:00.690 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000115758 2023-11-25T04:24:00.690 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:00.690 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:00.690 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:00.690 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.690 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.690 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing_reply(3.1f epoch 18/16 rep_tid 8 lcod 18'4)", 2023-11-25T04:24:00.690 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:24:00.691 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 73.565216347000003, 2023-11-25T04:24:00.691 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00038154699999999998, 2023-11-25T04:24:00.691 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:00.691 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:24:00.691 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:00.691 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.691 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:00.691 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:24:00.691 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:00.691 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.691 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.691 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:00.691 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:24:00.691 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:00.692 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.692 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.692 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:00.692 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:24:00.692 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2700000000000001e-07 2023-11-25T04:24:00.692 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.692 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.692 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:00.692 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044451+0000", 2023-11-25T04:24:00.692 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7460000000000001e-06 2023-11-25T04:24:00.692 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.692 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.692 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:00.692 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044455+0000", 2023-11-25T04:24:00.693 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5809999999999998e-06 2023-11-25T04:24:00.693 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.693 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.693 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:00.693 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044458+0000", 2023-11-25T04:24:00.693 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0170000000000001e-06 2023-11-25T04:24:00.693 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.693 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.693 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:00.693 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044734+0000", 2023-11-25T04:24:00.693 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00027582799999999999 2023-11-25T04:24:00.693 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.693 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.693 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:00.694 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044830+0000", 2023-11-25T04:24:00.694 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6248000000000004e-05 2023-11-25T04:24:00.694 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:00.694 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:00.694 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:00.694 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.694 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.694 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing_reply(3.b epoch 18/16 rep_tid 5 lcod 18'2)", 2023-11-25T04:24:00.694 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:24:00.694 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 73.565179685999993, 2023-11-25T04:24:00.694 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000344488, 2023-11-25T04:24:00.694 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:00.694 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:24:00.694 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:00.695 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.695 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:00.695 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:24:00.695 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:00.695 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.695 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.695 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:00.695 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:24:00.695 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:00.695 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.695 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.695 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:00.695 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:24:00.695 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.55e-07 2023-11-25T04:24:00.696 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.696 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.696 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:00.696 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044487+0000", 2023-11-25T04:24:00.696 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2470000000000001e-06 2023-11-25T04:24:00.696 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.696 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.696 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:00.696 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044491+0000", 2023-11-25T04:24:00.696 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9659999999999998e-06 2023-11-25T04:24:00.696 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.696 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.696 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:00.696 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044494+0000", 2023-11-25T04:24:00.697 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2310000000000001e-06 2023-11-25T04:24:00.697 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.697 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.697 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:00.697 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044756+0000", 2023-11-25T04:24:00.697 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00026172800000000003 2023-11-25T04:24:00.697 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.697 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.697 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:00.697 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044830+0000", 2023-11-25T04:24:00.697 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4160999999999997e-05 2023-11-25T04:24:00.697 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:00.697 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:00.697 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:00.698 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.698 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.698 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:39 3.6 3:654134d2:::mds0_openfiles.0:head [omap-clear,omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2023-11-25T04:24:00.698 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:24:00.698 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 68.505353643999996, 2023-11-25T04:24:00.698 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001788658, 2023-11-25T04:24:00.698 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:00.698 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:00.698 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:00.698 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.698 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:00.698 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:24:00.698 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:00.698 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.699 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.699 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:00.699 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:24:00.699 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:00.699 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.699 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.699 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:00.699 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104314+0000", 2023-11-25T04:24:00.699 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3099999999999999e-06 2023-11-25T04:24:00.699 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.699 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.699 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:00.699 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104325+0000", 2023-11-25T04:24:00.699 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1511999999999999e-05 2023-11-25T04:24:00.700 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.700 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.700 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:00.700 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104362+0000", 2023-11-25T04:24:00.700 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6903000000000002e-05 2023-11-25T04:24:00.700 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.700 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.700 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:00.700 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104387+0000", 2023-11-25T04:24:00.700 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4766e-05 2023-11-25T04:24:00.700 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.700 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.700 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:00.700 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104465+0000", 2023-11-25T04:24:00.701 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8119000000000002e-05 2023-11-25T04:24:00.701 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.701 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.701 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:00.701 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104557+0000", 2023-11-25T04:24:00.701 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2496000000000003e-05 2023-11-25T04:24:00.701 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.701 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.701 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:00.701 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104646+0000", 2023-11-25T04:24:00.701 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8818000000000002e-05 2023-11-25T04:24:00.701 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.701 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.701 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:00.702 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105201+0000", 2023-11-25T04:24:00.702 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00055506000000000002 2023-11-25T04:24:00.702 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.702 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.702 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:00.702 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105994+0000", 2023-11-25T04:24:00.702 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00079294700000000001 2023-11-25T04:24:00.702 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.702 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.702 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:00.702 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.106055+0000", 2023-11-25T04:24:00.702 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0566000000000003e-05 2023-11-25T04:24:00.702 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.702 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.703 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:00.703 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.106100+0000", 2023-11-25T04:24:00.703 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5160999999999997e-05 2023-11-25T04:24:00.703 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:00.703 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:00.703 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:00.703 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.703 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.703 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:43 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2023-11-25T04:24:00.703 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:24:00.703 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 53.505128648000003, 2023-11-25T04:24:00.703 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001751166, 2023-11-25T04:24:00.703 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:00.704 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:00.704 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:00.704 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.704 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:00.704 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:24:00.704 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:00.704 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.704 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.704 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:00.704 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:24:00.704 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:00.704 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.704 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.704 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:00.705 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104539+0000", 2023-11-25T04:24:00.705 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9529999999999999e-06 2023-11-25T04:24:00.705 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.705 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.705 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:00.705 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104557+0000", 2023-11-25T04:24:00.705 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7464000000000001e-05 2023-11-25T04:24:00.705 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.705 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.705 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:00.705 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104590+0000", 2023-11-25T04:24:00.705 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2966000000000003e-05 2023-11-25T04:24:00.705 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.705 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.705 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:00.706 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104608+0000", 2023-11-25T04:24:00.706 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8193e-05 2023-11-25T04:24:00.706 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.706 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.706 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:00.706 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104767+0000", 2023-11-25T04:24:00.706 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015882 2023-11-25T04:24:00.706 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.707 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.707 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:00.707 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104909+0000", 2023-11-25T04:24:00.707 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014244999999999999 2023-11-25T04:24:00.707 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.707 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.707 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:00.707 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.105059+0000", 2023-11-25T04:24:00.708 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014964399999999999 2023-11-25T04:24:00.708 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.708 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.708 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:00.708 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.105701+0000", 2023-11-25T04:24:00.708 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00064201099999999995 2023-11-25T04:24:00.708 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.708 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.708 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:00.708 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106233+0000", 2023-11-25T04:24:00.708 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00053178500000000001 2023-11-25T04:24:00.708 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.708 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.709 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:00.709 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106265+0000", 2023-11-25T04:24:00.709 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1986000000000002e-05 2023-11-25T04:24:00.709 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.709 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.709 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:00.709 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106288+0000", 2023-11-25T04:24:00.709 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2894000000000001e-05 2023-11-25T04:24:00.709 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:00.709 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:00.709 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:00.709 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.709 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.709 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:45 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e23)", 2023-11-25T04:24:00.709 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:24:00.710 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 48.504930608000002, 2023-11-25T04:24:00.710 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00098423299999999994, 2023-11-25T04:24:00.710 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:00.710 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:00.710 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:00.710 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.710 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:00.710 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:24:00.710 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:00.710 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.710 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.710 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:00.710 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:24:00.711 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:00.711 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.711 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.711 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:00.711 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104737+0000", 2023-11-25T04:24:00.711 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3309999999999998e-06 2023-11-25T04:24:00.711 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.711 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.711 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:00.711 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104748+0000", 2023-11-25T04:24:00.711 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1080999999999999e-05 2023-11-25T04:24:00.711 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.711 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.712 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:00.712 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104771+0000", 2023-11-25T04:24:00.712 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2707000000000001e-05 2023-11-25T04:24:00.712 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.712 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.712 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:00.712 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104800+0000", 2023-11-25T04:24:00.712 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9595e-05 2023-11-25T04:24:00.712 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.712 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.712 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:00.712 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104890+0000", 2023-11-25T04:24:00.712 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.9808999999999993e-05 2023-11-25T04:24:00.713 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.713 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.713 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:00.713 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105004+0000", 2023-11-25T04:24:00.713 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000114508 2023-11-25T04:24:00.713 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.713 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.713 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:00.713 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105634+0000", 2023-11-25T04:24:00.713 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00062985799999999996 2023-11-25T04:24:00.713 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.713 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.713 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:00.714 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105671+0000", 2023-11-25T04:24:00.714 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6640000000000002e-05 2023-11-25T04:24:00.714 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:00.714 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:00.714 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:00.714 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105719+0000", 2023-11-25T04:24:00.714 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7704000000000003e-05 2023-11-25T04:24:00.714 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:00.714 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:00.714 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:00.714 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:00.714 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:00.715 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:00.716 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:24:00.716 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:00.714+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:00.814 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-11-25T04:24:00.816 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:00.814+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:00.829 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:00.829 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:24:00.829 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2023-11-25T04:24:00.829 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:00.839 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:24:00.917 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:00.916+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:00.939 INFO:teuthology.orchestra.run.smithi067.stderr:Error ENXIO: problem getting command descriptions from osd.0 2023-11-25T04:24:00.958 INFO:teuthology.orchestra.run.smithi067.stdout:osd.1: {} 2023-11-25T04:24:00.959 INFO:teuthology.orchestra.run.smithi067.stderr:osd.1: osd_enable_op_tracker = '' 2023-11-25T04:24:00.977 INFO:teuthology.orchestra.run.smithi067.stdout:osd.2: {} 2023-11-25T04:24:00.977 INFO:teuthology.orchestra.run.smithi067.stderr:osd.2: osd_enable_op_tracker = '' 2023-11-25T04:24:00.979 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-11-25T04:24:00.980 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-11-25T04:24:00.987 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:00.987 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:00.996 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:24:01.014 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2023-11-25T04:24:01.039 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:01.037+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:01.100 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:01.101 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:01.109 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:24:01.155 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:01.153+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:01.218 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:01.218 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:01.220 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:01.218+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:01.226 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:24:01.321 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:01.319+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:01.371 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:01.371 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:01.380 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:24:01.422 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:01.420+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:01.523 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:01.521+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:01.525 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:01.525 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:01.534 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:24:01.624 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:01.622+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:01.643 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:01.643 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:01.652 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:24:01.712 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~53s 2023-11-25T04:24:01.726 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:01.724+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:01.797 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:01.797 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:01.807 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:24:01.827 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:01.825+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:01.913 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:01.913 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:01.921 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:24:01.928 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:01.926+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:02.017 INFO:teuthology.orchestra.run.smithi067.stderr:nodeep-scrub is unset 2023-11-25T04:24:02.029 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:02.027+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:02.031 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:02.031 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:02.040 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:24:02.130 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:02.128+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:02.146 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:02.146 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:02.155 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:24:02.231 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:02.229+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:02.301 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:02.301 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:02.310 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:24:02.332 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:02.331+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:02.416 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:02.416 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:02.425 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:24:02.433 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:02.432+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:02.529 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:02.529 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:02.534 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:02.532+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:02.538 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:24:02.635 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:02.634+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:02.683 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:02.684 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:02.692 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:24:02.736 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:02.734+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:02.799 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:02.799 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:02.808 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:24:02.852 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:02.851+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:02.912 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:02.913 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:02.921 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:24:02.937 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:02.935+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:02.982 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-11-25T04:24:03.039 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:03.037+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:03.068 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:03.068 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:03.077 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:24:03.107 INFO:teuthology.orchestra.run.smithi067.stderr:Error ENXIO: problem getting command descriptions from osd.0 2023-11-25T04:24:03.125 INFO:teuthology.orchestra.run.smithi067.stdout:osd.1: {} 2023-11-25T04:24:03.125 INFO:teuthology.orchestra.run.smithi067.stderr:osd.1: osd_enable_op_tracker = '' 2023-11-25T04:24:03.140 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:03.137+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:03.144 INFO:teuthology.orchestra.run.smithi067.stdout:osd.2: {} 2023-11-25T04:24:03.144 INFO:teuthology.orchestra.run.smithi067.stderr:osd.2: osd_enable_op_tracker = '' 2023-11-25T04:24:03.147 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-11-25T04:24:03.147 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-11-25T04:24:03.187 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:03.187 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:24:03.187 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2023-11-25T04:24:03.187 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:03.195 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:24:03.240 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:03.239+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:03.341 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:03.342 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:24:03.342 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2023-11-25T04:24:03.342 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2023-11-25T04:24:03.342 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:03.342 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:03.339+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:03.350 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:24:03.443 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:03.441+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:03.497 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:03.497 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2023-11-25T04:24:03.497 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2023-11-25T04:24:03.498 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2023-11-25T04:24:03.498 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.498 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:155 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'109, mlcod=11'105)", 2023-11-25T04:24:03.498 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:24:03.498 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 85.709487703999997, 2023-11-25T04:24:03.498 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061719019999999999, 2023-11-25T04:24:03.499 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:03.499 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:03.499 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:03.499 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.499 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:03.499 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:24:03.499 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:03.500 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.500 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.500 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:03.500 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:24:03.500 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:03.500 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.501 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.501 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:03.501 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784884+0000", 2023-11-25T04:24:03.501 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3099999999999999e-07 2023-11-25T04:24:03.501 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.501 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.502 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:03.502 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784942+0000", 2023-11-25T04:24:03.502 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8570000000000003e-05 2023-11-25T04:24:03.502 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.502 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.502 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:03.503 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784949+0000", 2023-11-25T04:24:03.503 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0310000000000001e-06 2023-11-25T04:24:03.503 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.503 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.503 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:03.504 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784953+0000", 2023-11-25T04:24:03.504 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6320000000000001e-06 2023-11-25T04:24:03.504 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.504 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.504 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:03.504 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784993+0000", 2023-11-25T04:24:03.505 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9737000000000003e-05 2023-11-25T04:24:03.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.505 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.505 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:03.505 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785011+0000", 2023-11-25T04:24:03.505 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8349999999999999e-05 2023-11-25T04:24:03.506 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.506 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.506 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:03.506 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791024+0000", 2023-11-25T04:24:03.506 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006012997 2023-11-25T04:24:03.507 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.507 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.507 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:03.507 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791055+0000", 2023-11-25T04:24:03.507 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1254000000000003e-05 2023-11-25T04:24:03.508 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:03.508 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:03.508 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:03.508 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.508 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.508 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:156 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'110, mlcod=11'105)", 2023-11-25T04:24:03.508 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:24:03.509 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 85.708901655000005, 2023-11-25T04:24:03.509 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005616116, 2023-11-25T04:24:03.509 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:03.509 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:03.509 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:03.509 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.510 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:03.510 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:24:03.510 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:03.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.510 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.511 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:03.511 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:24:03.511 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:03.511 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.511 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.511 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:03.512 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785470+0000", 2023-11-25T04:24:03.512 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0699999999999998e-07 2023-11-25T04:24:03.512 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.512 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.512 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:03.512 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785474+0000", 2023-11-25T04:24:03.512 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.048e-06 2023-11-25T04:24:03.512 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.513 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.513 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:03.513 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785480+0000", 2023-11-25T04:24:03.513 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7729999999999996e-06 2023-11-25T04:24:03.513 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.513 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.513 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:03.514 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785483+0000", 2023-11-25T04:24:03.514 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1970000000000001e-06 2023-11-25T04:24:03.514 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.514 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.514 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:03.514 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787936+0000", 2023-11-25T04:24:03.514 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024529040000000001 2023-11-25T04:24:03.515 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.515 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.515 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:03.515 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787947+0000", 2023-11-25T04:24:03.515 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1669e-05 2023-11-25T04:24:03.515 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.515 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.516 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:03.516 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791057+0000", 2023-11-25T04:24:03.516 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031095200000000002 2023-11-25T04:24:03.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.516 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.516 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:03.517 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791086+0000", 2023-11-25T04:24:03.517 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8697999999999999e-05 2023-11-25T04:24:03.517 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:03.517 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:03.517 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:03.518 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.518 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.519 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:157 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'111, mlcod=11'106)", 2023-11-25T04:24:03.519 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:24:03.519 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 85.708309096999997, 2023-11-25T04:24:03.519 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050590280000000001, 2023-11-25T04:24:03.519 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:03.519 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:03.519 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:03.519 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.520 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:03.520 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:24:03.520 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:03.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.520 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.520 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:03.520 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:24:03.520 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:03.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.520 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.520 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:03.520 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786063+0000", 2023-11-25T04:24:03.521 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2600000000000002e-07 2023-11-25T04:24:03.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.521 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.521 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:03.521 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786069+0000", 2023-11-25T04:24:03.521 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8839999999999999e-06 2023-11-25T04:24:03.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.521 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.521 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:03.521 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786079+0000", 2023-11-25T04:24:03.522 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0124e-05 2023-11-25T04:24:03.522 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.522 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.522 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:03.522 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786085+0000", 2023-11-25T04:24:03.522 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6660000000000002e-06 2023-11-25T04:24:03.522 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.522 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.522 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:03.522 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788160+0000", 2023-11-25T04:24:03.522 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020747669999999999 2023-11-25T04:24:03.522 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.523 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.523 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:03.523 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788180+0000", 2023-11-25T04:24:03.523 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0208999999999999e-05 2023-11-25T04:24:03.523 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.523 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.523 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:03.523 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791087+0000", 2023-11-25T04:24:03.523 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029064630000000002 2023-11-25T04:24:03.523 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.523 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.523 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:03.524 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791121+0000", 2023-11-25T04:24:03.524 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4289000000000003e-05 2023-11-25T04:24:03.524 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:03.524 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:03.524 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:03.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.524 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.524 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:158 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'112, mlcod=11'106)", 2023-11-25T04:24:03.524 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:24:03.524 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 85.707892704000002, 2023-11-25T04:24:03.524 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046663030000000001, 2023-11-25T04:24:03.524 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:03.525 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:03.525 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:03.525 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.525 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:03.525 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:24:03.525 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:03.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.525 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.525 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:03.525 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:24:03.525 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:03.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.526 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.526 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:03.526 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786479+0000", 2023-11-25T04:24:03.526 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2799999999999996e-07 2023-11-25T04:24:03.526 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.526 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.526 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:03.526 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786498+0000", 2023-11-25T04:24:03.526 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9069e-05 2023-11-25T04:24:03.526 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.527 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.527 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:03.527 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786508+0000", 2023-11-25T04:24:03.527 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5810000000000003e-06 2023-11-25T04:24:03.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.527 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.527 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:03.527 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786512+0000", 2023-11-25T04:24:03.527 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4889999999999997e-06 2023-11-25T04:24:03.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.527 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.528 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:03.528 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788438+0000", 2023-11-25T04:24:03.528 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019257059999999999 2023-11-25T04:24:03.528 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.528 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.528 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:03.528 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788449+0000", 2023-11-25T04:24:03.528 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1419e-05 2023-11-25T04:24:03.528 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.528 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.528 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:03.528 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791122+0000", 2023-11-25T04:24:03.528 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002672822 2023-11-25T04:24:03.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.529 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.529 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:03.529 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791145+0000", 2023-11-25T04:24:03.529 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2589e-05 2023-11-25T04:24:03.529 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:03.529 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:03.529 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:03.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.529 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.529 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:206 1.0 e12/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 12'152, mlcod=12'148)", 2023-11-25T04:24:03.530 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:24:03.530 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 85.196124233999996, 2023-11-25T04:24:03.530 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046728189999999999, 2023-11-25T04:24:03.530 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:03.530 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:03.530 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:03.530 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.530 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:03.530 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:24:03.530 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:03.530 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.531 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.531 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:03.531 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:24:03.531 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:03.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.531 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.531 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:03.531 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298248+0000", 2023-11-25T04:24:03.531 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3600000000000003e-07 2023-11-25T04:24:03.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.531 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.531 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:03.532 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298278+0000", 2023-11-25T04:24:03.532 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0474e-05 2023-11-25T04:24:03.532 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.532 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.532 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:03.532 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298288+0000", 2023-11-25T04:24:03.532 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0156000000000001e-05 2023-11-25T04:24:03.532 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.532 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.532 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:03.532 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298293+0000", 2023-11-25T04:24:03.532 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7400000000000004e-06 2023-11-25T04:24:03.533 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.533 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.533 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:03.533 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298333+0000", 2023-11-25T04:24:03.533 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9666000000000003e-05 2023-11-25T04:24:03.533 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.533 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.533 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:03.533 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298351+0000", 2023-11-25T04:24:03.533 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8544999999999999e-05 2023-11-25T04:24:03.533 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.534 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.534 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:03.534 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302891+0000", 2023-11-25T04:24:03.534 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045398519999999996 2023-11-25T04:24:03.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.534 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.534 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:03.534 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302920+0000", 2023-11-25T04:24:03.534 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8750000000000001e-05 2023-11-25T04:24:03.534 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:03.534 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:03.535 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:03.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.535 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:14 3.15 3:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:03.535 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:24:03.535 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 76.460989877000003, 2023-11-25T04:24:03.535 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010990437, 2023-11-25T04:24:03.535 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:03.535 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:03.535 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:03.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.535 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:03.535 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:24:03.536 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:03.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.536 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.536 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:03.536 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:24:03.536 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:03.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.536 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.536 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:03.536 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033383+0000", 2023-11-25T04:24:03.536 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9870000000000002e-06 2023-11-25T04:24:03.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.537 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.537 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:03.537 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033398+0000", 2023-11-25T04:24:03.537 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4569999999999999e-05 2023-11-25T04:24:03.537 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.537 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.537 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:03.537 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033411+0000", 2023-11-25T04:24:03.537 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3543999999999999e-05 2023-11-25T04:24:03.537 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.537 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.537 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:03.538 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033423+0000", 2023-11-25T04:24:03.538 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1905999999999999e-05 2023-11-25T04:24:03.538 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.538 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.538 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:03.538 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034433+0000", 2023-11-25T04:24:03.538 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00101022 2023-11-25T04:24:03.538 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.538 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.538 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:03.538 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034539+0000", 2023-11-25T04:24:03.538 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010597199999999999 2023-11-25T04:24:03.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.539 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:03.539 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034667+0000", 2023-11-25T04:24:03.539 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012765500000000001 2023-11-25T04:24:03.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.539 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:03.539 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037803+0000", 2023-11-25T04:24:03.539 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031362220000000001 2023-11-25T04:24:03.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.539 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:03.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:24:03.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0064964669999999997 2023-11-25T04:24:03.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.540 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.540 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:03.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044331+0000", 2023-11-25T04:24:03.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1544999999999997e-05 2023-11-25T04:24:03.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.540 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.540 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:03.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044372+0000", 2023-11-25T04:24:03.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0349000000000003e-05 2023-11-25T04:24:03.541 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:03.541 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:03.541 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:03.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.541 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.541 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:15 3.15 3:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:03.541 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:24:03.541 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 76.460330227, 2023-11-25T04:24:03.541 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010464362, 2023-11-25T04:24:03.541 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:03.541 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:03.541 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:03.542 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.542 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:03.542 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:24:03.542 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:03.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.542 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.542 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:03.542 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:24:03.542 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:03.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.543 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.543 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:03.543 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034042+0000", 2023-11-25T04:24:03.543 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0999999999999997e-07 2023-11-25T04:24:03.543 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.543 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.545 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:03.545 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034048+0000", 2023-11-25T04:24:03.545 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0390000000000001e-06 2023-11-25T04:24:03.545 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.545 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.545 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:03.545 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034059+0000", 2023-11-25T04:24:03.545 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1182999999999999e-05 2023-11-25T04:24:03.546 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.546 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.546 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:03.546 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034067+0000", 2023-11-25T04:24:03.546 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1559999999999995e-06 2023-11-25T04:24:03.546 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.546 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.546 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:03.546 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035005+0000", 2023-11-25T04:24:03.546 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00093747399999999999 2023-11-25T04:24:03.546 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.546 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.547 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:03.547 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035077+0000", 2023-11-25T04:24:03.547 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2747e-05 2023-11-25T04:24:03.547 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.547 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.547 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:03.547 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035219+0000", 2023-11-25T04:24:03.547 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000141877 2023-11-25T04:24:03.547 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.547 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.547 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:03.548 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037810+0000", 2023-11-25T04:24:03.548 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002590752 2023-11-25T04:24:03.548 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.548 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.548 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:03.548 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044437+0000", 2023-11-25T04:24:03.548 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066270110000000004 2023-11-25T04:24:03.548 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.548 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.548 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:03.549 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044481+0000", 2023-11-25T04:24:03.549 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4106000000000001e-05 2023-11-25T04:24:03.549 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.549 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.549 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:03.549 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044505+0000", 2023-11-25T04:24:03.549 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4207e-05 2023-11-25T04:24:03.549 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:03.549 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:03.549 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:03.551 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.551 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.551 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:16 3.1d 3:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:03.551 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:24:03.551 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 76.460288379000005, 2023-11-25T04:24:03.551 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010413923, 2023-11-25T04:24:03.551 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:03.551 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:03.551 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:03.551 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.551 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:03.552 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:24:03.552 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:03.552 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.552 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.552 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:03.552 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:24:03.552 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:03.552 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.552 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.552 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:03.552 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:24:03.553 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4499999999999997e-07 2023-11-25T04:24:03.553 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.553 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.553 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:03.553 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034087+0000", 2023-11-25T04:24:03.553 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9770000000000002e-06 2023-11-25T04:24:03.553 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.553 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.553 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:03.553 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034094+0000", 2023-11-25T04:24:03.553 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8809999999999998e-06 2023-11-25T04:24:03.553 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.554 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.554 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:03.554 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034100+0000", 2023-11-25T04:24:03.554 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8440000000000003e-06 2023-11-25T04:24:03.554 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.554 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.554 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:03.554 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034993+0000", 2023-11-25T04:24:03.554 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00089336799999999996 2023-11-25T04:24:03.554 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.554 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.554 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:03.555 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035099+0000", 2023-11-25T04:24:03.555 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106037 2023-11-25T04:24:03.555 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.555 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.555 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:03.555 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035276+0000", 2023-11-25T04:24:03.555 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017631799999999999 2023-11-25T04:24:03.555 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.555 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.555 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:03.555 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037815+0000", 2023-11-25T04:24:03.555 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025395959999999999 2023-11-25T04:24:03.556 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.556 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.556 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:03.556 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044420+0000", 2023-11-25T04:24:03.556 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066045749999999997 2023-11-25T04:24:03.556 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.556 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.556 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:03.556 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044463+0000", 2023-11-25T04:24:03.556 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3205999999999999e-05 2023-11-25T04:24:03.556 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.556 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.556 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:03.557 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044497+0000", 2023-11-25T04:24:03.557 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3476000000000003e-05 2023-11-25T04:24:03.557 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:03.557 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:03.557 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:03.557 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.557 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.557 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:17 3.1d 3:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:03.557 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:24:03.557 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 76.460263170000005, 2023-11-25T04:24:03.557 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01127356, 2023-11-25T04:24:03.557 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:03.557 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:03.558 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:03.558 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.558 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:03.558 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:24:03.558 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:03.558 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.558 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.558 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:03.558 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:24:03.558 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:03.558 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.559 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.559 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:03.559 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:24:03.559 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1100000000000001e-07 2023-11-25T04:24:03.559 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.559 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.559 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:03.559 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034112+0000", 2023-11-25T04:24:03.559 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8600000000000003e-06 2023-11-25T04:24:03.559 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.559 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.559 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:03.559 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034120+0000", 2023-11-25T04:24:03.560 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7700000000000001e-06 2023-11-25T04:24:03.560 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.560 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.560 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:03.560 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034127+0000", 2023-11-25T04:24:03.560 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0999999999999998e-06 2023-11-25T04:24:03.560 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.560 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.560 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:03.560 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035601+0000", 2023-11-25T04:24:03.560 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014739410000000001 2023-11-25T04:24:03.560 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.561 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:03.561 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035685+0000", 2023-11-25T04:24:03.561 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3677999999999994e-05 2023-11-25T04:24:03.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.561 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:03.561 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035827+0000", 2023-11-25T04:24:03.561 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014232599999999999 2023-11-25T04:24:03.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.561 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:03.562 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038725+0000", 2023-11-25T04:24:03.562 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028982779999999998 2023-11-25T04:24:03.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.562 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.562 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:03.562 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045323+0000", 2023-11-25T04:24:03.562 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0065980780000000003 2023-11-25T04:24:03.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.562 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.562 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:03.562 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045354+0000", 2023-11-25T04:24:03.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0463e-05 2023-11-25T04:24:03.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.563 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:03.563 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045381+0000", 2023-11-25T04:24:03.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7654999999999999e-05 2023-11-25T04:24:03.563 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:03.563 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:03.563 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:03.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.563 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:22 3.e 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:03.564 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:24:03.564 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 76.460236757000004, 2023-11-25T04:24:03.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010133422, 2023-11-25T04:24:03.564 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:03.564 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:03.564 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:03.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:03.564 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:24:03.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:03.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:03.565 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:24:03.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:03.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.565 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:03.565 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034135+0000", 2023-11-25T04:24:03.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-07 2023-11-25T04:24:03.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.565 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:03.565 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034138+0000", 2023-11-25T04:24:03.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6009999999999999e-06 2023-11-25T04:24:03.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.566 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.566 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:03.566 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034145+0000", 2023-11-25T04:24:03.566 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1169999999999998e-06 2023-11-25T04:24:03.566 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.566 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.566 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:03.566 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034150+0000", 2023-11-25T04:24:03.566 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9080000000000003e-06 2023-11-25T04:24:03.566 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.566 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:03.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034209+0000", 2023-11-25T04:24:03.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8149000000000002e-05 2023-11-25T04:24:03.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:03.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034415+0000", 2023-11-25T04:24:03.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020637699999999999 2023-11-25T04:24:03.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:03.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034584+0000", 2023-11-25T04:24:03.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016870399999999999 2023-11-25T04:24:03.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:03.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.036979+0000", 2023-11-25T04:24:03.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023957509999999998 2023-11-25T04:24:03.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:03.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044161+0000", 2023-11-25T04:24:03.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0071811590000000003 2023-11-25T04:24:03.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:03.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044224+0000", 2023-11-25T04:24:03.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3454000000000006e-05 2023-11-25T04:24:03.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:03.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044268+0000", 2023-11-25T04:24:03.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3761999999999999e-05 2023-11-25T04:24:03.569 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:03.569 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:03.569 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:03.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.570 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:23 3.e 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:03.570 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:24:03.570 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 76.460205348000002, 2023-11-25T04:24:03.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010281676, 2023-11-25T04:24:03.570 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:03.570 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:03.570 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:03.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.570 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:03.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:24:03.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:03.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:03.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:24:03.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:03.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:03.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:24:03.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5799999999999999e-07 2023-11-25T04:24:03.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.572 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:03.572 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034169+0000", 2023-11-25T04:24:03.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1879999999999998e-06 2023-11-25T04:24:03.572 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.572 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.572 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:03.572 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034177+0000", 2023-11-25T04:24:03.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7020000000000001e-06 2023-11-25T04:24:03.572 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.572 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.572 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:03.572 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034184+0000", 2023-11-25T04:24:03.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3910000000000003e-06 2023-11-25T04:24:03.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.573 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:03.573 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034924+0000", 2023-11-25T04:24:03.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00074032999999999998 2023-11-25T04:24:03.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.573 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:03.573 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034991+0000", 2023-11-25T04:24:03.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7347000000000004e-05 2023-11-25T04:24:03.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:03.574 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035096+0000", 2023-11-25T04:24:03.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010488 2023-11-25T04:24:03.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:03.574 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037588+0000", 2023-11-25T04:24:03.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024918309999999999 2023-11-25T04:24:03.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:03.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044367+0000", 2023-11-25T04:24:03.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067787589999999997 2023-11-25T04:24:03.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:03.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044415+0000", 2023-11-25T04:24:03.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7803000000000003e-05 2023-11-25T04:24:03.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:03.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044447+0000", 2023-11-25T04:24:03.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2886999999999999e-05 2023-11-25T04:24:03.576 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:03.576 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:03.576 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:03.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.576 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:30 3.15 3:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:03.576 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:24:03.576 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 76.460146907999999, 2023-11-25T04:24:03.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010365070000000001, 2023-11-25T04:24:03.576 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:03.576 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:03.577 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:03.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.577 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:03.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:24:03.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:03.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.577 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:03.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:24:03.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:03.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.578 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:03.578 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034225+0000", 2023-11-25T04:24:03.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7400000000000002e-07 2023-11-25T04:24:03.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.578 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:03.578 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034229+0000", 2023-11-25T04:24:03.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3100000000000002e-06 2023-11-25T04:24:03.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.578 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:03.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034239+0000", 2023-11-25T04:24:03.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9140000000000003e-06 2023-11-25T04:24:03.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:03.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034246+0000", 2023-11-25T04:24:03.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0149999999999998e-06 2023-11-25T04:24:03.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:03.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035572+0000", 2023-11-25T04:24:03.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001325655 2023-11-25T04:24:03.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:03.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035730+0000", 2023-11-25T04:24:03.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015827899999999999 2023-11-25T04:24:03.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:03.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035857+0000", 2023-11-25T04:24:03.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012644699999999999 2023-11-25T04:24:03.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:03.581 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038737+0000", 2023-11-25T04:24:03.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002880077 2023-11-25T04:24:03.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.581 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:03.581 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044540+0000", 2023-11-25T04:24:03.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058034239999999997 2023-11-25T04:24:03.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.581 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:03.581 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044569+0000", 2023-11-25T04:24:03.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8388999999999999e-05 2023-11-25T04:24:03.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:03.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044589+0000", 2023-11-25T04:24:03.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0686000000000002e-05 2023-11-25T04:24:03.582 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:03.582 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:03.582 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:03.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.582 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:33 3.1b 3:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:03.582 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:24:03.583 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 76.460101409999993, 2023-11-25T04:24:03.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099754049999999997, 2023-11-25T04:24:03.583 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:03.583 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:03.583 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:03.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.583 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:03.583 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:24:03.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:03.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.583 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:03.584 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:24:03.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:03.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.584 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:03.584 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:24:03.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4799999999999998e-07 2023-11-25T04:24:03.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.584 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:03.584 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034273+0000", 2023-11-25T04:24:03.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7819999999999998e-06 2023-11-25T04:24:03.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:03.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034285+0000", 2023-11-25T04:24:03.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2058e-05 2023-11-25T04:24:03.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:03.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034291+0000", 2023-11-25T04:24:03.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0680000000000002e-06 2023-11-25T04:24:03.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:03.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034370+0000", 2023-11-25T04:24:03.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8452e-05 2023-11-25T04:24:03.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:03.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034477+0000", 2023-11-25T04:24:03.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107471 2023-11-25T04:24:03.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:03.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044245+0000", 2023-11-25T04:24:03.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0097679259999999997 2023-11-25T04:24:03.587 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:03.587 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:03.587 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:03.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.587 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:36 3.1a 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:03.587 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:24:03.587 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 76.460054736999993, 2023-11-25T04:24:03.587 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058179099999999999, 2023-11-25T04:24:03.587 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:03.587 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:03.588 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:03.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.588 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:03.588 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:24:03.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:03.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.588 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:03.588 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:24:03.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:03.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:03.589 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034317+0000", 2023-11-25T04:24:03.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1999999999999999e-07 2023-11-25T04:24:03.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:03.589 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034321+0000", 2023-11-25T04:24:03.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.233e-06 2023-11-25T04:24:03.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:03.589 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034337+0000", 2023-11-25T04:24:03.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5308999999999999e-05 2023-11-25T04:24:03.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:03.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034354+0000", 2023-11-25T04:24:03.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7411999999999999e-05 2023-11-25T04:24:03.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:03.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034465+0000", 2023-11-25T04:24:03.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011154699999999999 2023-11-25T04:24:03.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:03.591 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034560+0000", 2023-11-25T04:24:03.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4186999999999998e-05 2023-11-25T04:24:03.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.591 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:03.591 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.040134+0000", 2023-11-25T04:24:03.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055746019999999997 2023-11-25T04:24:03.591 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:03.591 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:03.591 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:03.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.592 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:9 3.10 e18/16 3:09401a81:::601.00000000:head v 18'4, mlcod=0'0)", 2023-11-25T04:24:03.592 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:24:03.592 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 76.451160428999998, 2023-11-25T04:24:03.592 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061641020000000003, 2023-11-25T04:24:03.592 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:03.592 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:03.592 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:03.592 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.592 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:03.592 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:24:03.592 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:03.592 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:03.593 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:24:03.593 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:03.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:03.593 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:24:03.593 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5300000000000001e-07 2023-11-25T04:24:03.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:03.594 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043222+0000", 2023-11-25T04:24:03.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.082e-05 2023-11-25T04:24:03.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.594 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:03.594 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043228+0000", 2023-11-25T04:24:03.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6699999999999997e-06 2023-11-25T04:24:03.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.594 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:03.594 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043236+0000", 2023-11-25T04:24:03.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8189999999999992e-06 2023-11-25T04:24:03.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:03.595 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043507+0000", 2023-11-25T04:24:03.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000271012 2023-11-25T04:24:03.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:03.595 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043530+0000", 2023-11-25T04:24:03.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2659000000000001e-05 2023-11-25T04:24:03.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:03.596 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049321+0000", 2023-11-25T04:24:03.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057915129999999999 2023-11-25T04:24:03.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.596 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.596 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:03.596 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049375+0000", 2023-11-25T04:24:03.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3455999999999998e-05 2023-11-25T04:24:03.596 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:03.596 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:03.596 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:03.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.596 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.597 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:1 3.14 e18/16 3:292cf221:::200.00000000:head v 18'3, mlcod=0'0)", 2023-11-25T04:24:03.597 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:24:03.597 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 76.451123715999998, 2023-11-25T04:24:03.597 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0060983599999999997, 2023-11-25T04:24:03.597 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:03.597 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:03.597 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:03.597 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.597 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:03.597 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:24:03.597 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:03.597 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.597 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.597 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:03.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:24:03.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:03.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:03.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043248+0000", 2023-11-25T04:24:03.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9399999999999995e-07 2023-11-25T04:24:03.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:03.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043252+0000", 2023-11-25T04:24:03.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1799999999999998e-06 2023-11-25T04:24:03.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:03.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043259+0000", 2023-11-25T04:24:03.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0049999999999998e-06 2023-11-25T04:24:03.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:03.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043265+0000", 2023-11-25T04:24:03.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0859999999999998e-06 2023-11-25T04:24:03.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:03.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043311+0000", 2023-11-25T04:24:03.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6082999999999999e-05 2023-11-25T04:24:03.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.600 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:03.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043330+0000", 2023-11-25T04:24:03.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8556e-05 2023-11-25T04:24:03.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.600 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:03.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049303+0000", 2023-11-25T04:24:03.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059733470000000004 2023-11-25T04:24:03.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:03.601 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049346+0000", 2023-11-25T04:24:03.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2608999999999998e-05 2023-11-25T04:24:03.601 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:03.601 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:03.601 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:03.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.601 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:3 3.14 e18/16 3:287034c4:::1.00000000.inode:head v 18'4, mlcod=0'0)", 2023-11-25T04:24:03.601 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:24:03.601 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 76.450607210000001, 2023-11-25T04:24:03.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057040090000000003, 2023-11-25T04:24:03.602 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:03.602 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:03.602 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:03.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:03.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:24:03.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:03.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:03.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:24:03.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:03.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:03.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:24:03.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9799999999999999e-07 2023-11-25T04:24:03.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:03.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043785+0000", 2023-11-25T04:24:03.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0721999999999999e-05 2023-11-25T04:24:03.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:03.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043793+0000", 2023-11-25T04:24:03.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5839999999999995e-06 2023-11-25T04:24:03.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:03.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043803+0000", 2023-11-25T04:24:03.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0729999999999994e-06 2023-11-25T04:24:03.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:03.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043843+0000", 2023-11-25T04:24:03.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0747999999999997e-05 2023-11-25T04:24:03.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.605 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:03.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043856+0000", 2023-11-25T04:24:03.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2697000000000001e-05 2023-11-25T04:24:03.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.605 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:03.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049426+0000", 2023-11-25T04:24:03.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055704409999999998 2023-11-25T04:24:03.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:03.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049468+0000", 2023-11-25T04:24:03.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1446000000000003e-05 2023-11-25T04:24:03.606 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:03.606 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:03.606 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:03.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.606 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "pg_update_log_missing(3.14 epoch 18/16 rep_tid 171 entries 18'5 (0'0) error 3:2e2fa760:::200.00000001:head by mds.0.3:32 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:24:03.606 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:24:03.606 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 76.450446247000002, 2023-11-25T04:24:03.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005556938, 2023-11-25T04:24:03.607 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:03.607 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2023-11-25T04:24:03.607 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:03.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:03.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:24:03.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:03.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:03.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:24:03.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:03.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:03.608 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:24:03.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.03e-07 2023-11-25T04:24:03.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:03.608 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043931+0000", 2023-11-25T04:24:03.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1079999999999998e-06 2023-11-25T04:24:03.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:03.609 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043944+0000", 2023-11-25T04:24:03.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2852e-05 2023-11-25T04:24:03.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:03.609 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043955+0000", 2023-11-25T04:24:03.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1037e-05 2023-11-25T04:24:03.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.610 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:03.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044088+0000", 2023-11-25T04:24:03.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013269599999999999 2023-11-25T04:24:03.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.610 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:03.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049482+0000", 2023-11-25T04:24:03.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053940419999999999 2023-11-25T04:24:03.610 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:03.610 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:03.611 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:03.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.611 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:10 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'5, mlcod=0'0)", 2023-11-25T04:24:03.611 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:24:03.611 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 76.450356219, 2023-11-25T04:24:03.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054493099999999997, 2023-11-25T04:24:03.611 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:03.611 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:03.611 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:03.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:03.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:24:03.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:03.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:03.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:24:03.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:03.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:03.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:24:03.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6100000000000002e-07 2023-11-25T04:24:03.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.613 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:03.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044019+0000", 2023-11-25T04:24:03.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.839e-06 2023-11-25T04:24:03.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.613 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:03.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044026+0000", 2023-11-25T04:24:03.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5869999999999998e-06 2023-11-25T04:24:03.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:03.614 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044031+0000", 2023-11-25T04:24:03.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4909999999999996e-06 2023-11-25T04:24:03.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:03.614 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044070+0000", 2023-11-25T04:24:03.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8829999999999999e-05 2023-11-25T04:24:03.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:03.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044081+0000", 2023-11-25T04:24:03.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1447999999999999e-05 2023-11-25T04:24:03.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:03.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049406+0000", 2023-11-25T04:24:03.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053246480000000004 2023-11-25T04:24:03.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:03.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049464+0000", 2023-11-25T04:24:03.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8205999999999998e-05 2023-11-25T04:24:03.616 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:03.616 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:03.616 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:03.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.616 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:11 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'6, mlcod=0'0)", 2023-11-25T04:24:03.616 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:24:03.616 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 76.450154955000002, 2023-11-25T04:24:03.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052673499999999996, 2023-11-25T04:24:03.616 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:03.616 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:03.617 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:03.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.617 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:03.617 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:24:03.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:03.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.617 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:03.617 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:24:03.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:03.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.617 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:03.618 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:24:03.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0199999999999998e-07 2023-11-25T04:24:03.618 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.618 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:03.618 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044220+0000", 2023-11-25T04:24:03.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7720000000000001e-06 2023-11-25T04:24:03.618 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.618 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:03.618 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044226+0000", 2023-11-25T04:24:03.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8579999999999996e-06 2023-11-25T04:24:03.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:03.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044232+0000", 2023-11-25T04:24:03.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1700000000000002e-06 2023-11-25T04:24:03.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:03.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044274+0000", 2023-11-25T04:24:03.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1254999999999997e-05 2023-11-25T04:24:03.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:03.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:24:03.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6880999999999999e-05 2023-11-25T04:24:03.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:03.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049466+0000", 2023-11-25T04:24:03.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051659219999999999 2023-11-25T04:24:03.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:03.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:03.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:03.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049483+0000", 2023-11-25T04:24:03.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.719e-05 2023-11-25T04:24:03.621 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:03.621 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:03.621 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:03.621 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:03.621 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:03.621 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:03.623 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:24:03.623 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:03.620+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:03.723 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:03.722+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:03.730 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:03.730 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:24:03.730 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2023-11-25T04:24:03.730 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:03.739 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:24:03.824 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:03.822+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:03.884 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:03.884 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:24:03.884 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2023-11-25T04:24:03.884 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2023-11-25T04:24:03.884 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:03.893 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:24:03.925 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:03.924+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:04.026 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:04.024+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:04.033 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2023-11-25T04:24:04.042 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:04.042 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2023-11-25T04:24:04.042 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2023-11-25T04:24:04.042 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2023-11-25T04:24:04.042 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.042 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:04.042 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:24:04.042 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 77.007064811999996, 2023-11-25T04:24:04.042 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01072943, 2023-11-25T04:24:04.043 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:04.043 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:04.043 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:04.043 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.043 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:04.043 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:24:04.043 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:04.043 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.043 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.043 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:04.043 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:24:04.043 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:04.044 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.044 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.044 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:04.044 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031829+0000", 2023-11-25T04:24:04.044 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.666e-06 2023-11-25T04:24:04.044 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.044 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.044 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:04.044 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031839+0000", 2023-11-25T04:24:04.044 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9490000000000008e-06 2023-11-25T04:24:04.044 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.044 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.045 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:04.045 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031857+0000", 2023-11-25T04:24:04.045 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8093000000000001e-05 2023-11-25T04:24:04.045 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.045 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.045 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:04.045 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031882+0000", 2023-11-25T04:24:04.045 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4294000000000002e-05 2023-11-25T04:24:04.045 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.045 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.045 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:04.045 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033130+0000", 2023-11-25T04:24:04.046 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001248264 2023-11-25T04:24:04.046 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.046 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.046 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:04.046 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033426+0000", 2023-11-25T04:24:04.046 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029620999999999998 2023-11-25T04:24:04.046 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.046 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.046 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:04.046 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033814+0000", 2023-11-25T04:24:04.046 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000387742 2023-11-25T04:24:04.046 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.046 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.047 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:04.047 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036793+0000", 2023-11-25T04:24:04.047 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002979598 2023-11-25T04:24:04.047 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.047 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.047 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:04.047 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042455+0000", 2023-11-25T04:24:04.047 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0056619260000000003 2023-11-25T04:24:04.047 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.047 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.047 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:04.047 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042502+0000", 2023-11-25T04:24:04.048 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7139999999999999e-05 2023-11-25T04:24:04.048 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.048 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.048 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:04.048 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042557+0000", 2023-11-25T04:24:04.048 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4548e-05 2023-11-25T04:24:04.048 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:04.048 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:04.048 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:04.048 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.048 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.048 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:04.049 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:24:04.049 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 77.005866276999996, 2023-11-25T04:24:04.049 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011505202000000001, 2023-11-25T04:24:04.049 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:04.049 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:04.049 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:04.049 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.049 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:04.049 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:24:04.049 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:04.049 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.049 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.049 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:04.050 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:24:04.050 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:04.050 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.050 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.050 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:04.050 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033027+0000", 2023-11-25T04:24:04.050 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0699999999999999e-06 2023-11-25T04:24:04.050 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.050 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.050 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:04.050 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033035+0000", 2023-11-25T04:24:04.050 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0830000000000004e-06 2023-11-25T04:24:04.050 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.051 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.051 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:04.051 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033059+0000", 2023-11-25T04:24:04.051 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3975e-05 2023-11-25T04:24:04.051 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.051 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.051 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:04.051 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033482+0000", 2023-11-25T04:24:04.051 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042253700000000001 2023-11-25T04:24:04.051 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.051 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.051 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:04.051 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034520+0000", 2023-11-25T04:24:04.051 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001037991 2023-11-25T04:24:04.052 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.052 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.052 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:04.052 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034611+0000", 2023-11-25T04:24:04.052 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1113999999999995e-05 2023-11-25T04:24:04.052 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.052 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.052 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:04.052 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034769+0000", 2023-11-25T04:24:04.052 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000157641 2023-11-25T04:24:04.052 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.053 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.053 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:04.053 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038292+0000", 2023-11-25T04:24:04.053 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035231160000000002 2023-11-25T04:24:04.053 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.053 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.053 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:04.053 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044440+0000", 2023-11-25T04:24:04.053 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0061486980000000002 2023-11-25T04:24:04.053 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.053 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.053 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:04.054 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044495+0000", 2023-11-25T04:24:04.054 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4339000000000003e-05 2023-11-25T04:24:04.054 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.054 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.054 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:04.054 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044531+0000", 2023-11-25T04:24:04.054 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6637999999999997e-05 2023-11-25T04:24:04.055 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:04.055 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:04.055 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:04.055 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.055 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.056 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:04.056 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:24:04.056 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 77.005391923000005, 2023-11-25T04:24:04.056 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0057634039999999997, 2023-11-25T04:24:04.056 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:04.056 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:04.056 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:04.056 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.056 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:04.056 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:24:04.056 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:04.056 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.056 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.057 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:04.057 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:24:04.057 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:04.057 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.057 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.057 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:04.057 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:24:04.057 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7900000000000007e-07 2023-11-25T04:24:04.057 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.057 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.057 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:04.057 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033507+0000", 2023-11-25T04:24:04.058 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8900000000000004e-06 2023-11-25T04:24:04.058 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.058 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.058 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:04.058 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033519+0000", 2023-11-25T04:24:04.058 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1756000000000001e-05 2023-11-25T04:24:04.058 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.058 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.058 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:04.059 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033527+0000", 2023-11-25T04:24:04.059 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3610000000000001e-06 2023-11-25T04:24:04.059 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.059 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.059 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:04.059 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034451+0000", 2023-11-25T04:24:04.059 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00092372400000000001 2023-11-25T04:24:04.059 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.059 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.059 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:04.059 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034809+0000", 2023-11-25T04:24:04.059 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00035757699999999998 2023-11-25T04:24:04.059 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.060 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.060 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:04.060 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034965+0000", 2023-11-25T04:24:04.060 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015638699999999999 2023-11-25T04:24:04.060 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.060 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.060 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:04.060 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038299+0000", 2023-11-25T04:24:04.060 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0033336559999999999 2023-11-25T04:24:04.060 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.060 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.060 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:04.061 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039169+0000", 2023-11-25T04:24:04.061 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00087066400000000005 2023-11-25T04:24:04.061 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.061 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.061 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:04.061 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039226+0000", 2023-11-25T04:24:04.061 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6511999999999997e-05 2023-11-25T04:24:04.061 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.061 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.061 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:04.061 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039264+0000", 2023-11-25T04:24:04.061 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7898000000000002e-05 2023-11-25T04:24:04.061 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:04.062 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:04.062 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:04.062 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.062 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.062 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:25 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:04.062 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:24:04.062 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 77.005313338999997, 2023-11-25T04:24:04.062 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096656169999999996, 2023-11-25T04:24:04.062 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:04.062 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:04.062 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:04.062 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.063 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:04.063 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:24:04.063 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:04.063 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.063 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.063 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:04.063 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:24:04.063 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:04.063 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.063 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.063 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:04.063 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033580+0000", 2023-11-25T04:24:04.064 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8199999999999999e-07 2023-11-25T04:24:04.064 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.064 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.064 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:04.064 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033585+0000", 2023-11-25T04:24:04.064 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3680000000000001e-06 2023-11-25T04:24:04.064 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.064 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.064 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:04.064 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033600+0000", 2023-11-25T04:24:04.065 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4331000000000001e-05 2023-11-25T04:24:04.065 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.065 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.065 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:04.065 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033606+0000", 2023-11-25T04:24:04.065 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5780000000000004e-06 2023-11-25T04:24:04.065 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.065 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.065 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:04.065 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035747+0000", 2023-11-25T04:24:04.065 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0021406530000000002 2023-11-25T04:24:04.065 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.065 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.066 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:04.066 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035901+0000", 2023-11-25T04:24:04.066 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000153931 2023-11-25T04:24:04.066 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.066 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.066 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:04.066 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036193+0000", 2023-11-25T04:24:04.066 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029211299999999999 2023-11-25T04:24:04.066 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.066 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.066 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:04.066 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038655+0000", 2023-11-25T04:24:04.067 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0024621640000000002 2023-11-25T04:24:04.067 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.067 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.067 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:04.067 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043130+0000", 2023-11-25T04:24:04.067 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.004474698 2023-11-25T04:24:04.067 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.067 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.067 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:04.067 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043204+0000", 2023-11-25T04:24:04.067 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4091999999999995e-05 2023-11-25T04:24:04.067 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.067 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.067 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:04.068 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043245+0000", 2023-11-25T04:24:04.068 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0907e-05 2023-11-25T04:24:04.068 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:04.068 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:04.068 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:04.068 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.068 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.068 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:26 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:04.068 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:04.068 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 77.005277515000003, 2023-11-25T04:24:04.068 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011086945000000001, 2023-11-25T04:24:04.068 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:04.068 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:04.069 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:04.069 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.069 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:04.069 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:04.069 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:04.069 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.069 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.069 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:04.069 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:04.069 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:04.069 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.069 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.069 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:04.070 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:04.070 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1699999999999998e-07 2023-11-25T04:24:04.070 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.070 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.070 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:04.070 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033627+0000", 2023-11-25T04:24:04.070 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.163e-05 2023-11-25T04:24:04.070 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.070 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.070 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:04.070 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033636+0000", 2023-11-25T04:24:04.070 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6859999999999995e-06 2023-11-25T04:24:04.070 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.071 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.071 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:04.071 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033642+0000", 2023-11-25T04:24:04.071 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3149999999999997e-06 2023-11-25T04:24:04.071 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.071 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.071 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:04.071 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035414+0000", 2023-11-25T04:24:04.071 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0017716380000000001 2023-11-25T04:24:04.071 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.071 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.071 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:04.071 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035539+0000", 2023-11-25T04:24:04.072 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000125477 2023-11-25T04:24:04.072 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.072 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.072 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:04.072 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035672+0000", 2023-11-25T04:24:04.072 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000133316 2023-11-25T04:24:04.072 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.072 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.072 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:04.072 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038649+0000", 2023-11-25T04:24:04.072 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029764679999999999 2023-11-25T04:24:04.072 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.072 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.073 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:04.073 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044578+0000", 2023-11-25T04:24:04.073 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0059293510000000002 2023-11-25T04:24:04.073 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.073 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.073 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:04.073 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044651+0000", 2023-11-25T04:24:04.073 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2513000000000002e-05 2023-11-25T04:24:04.073 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.073 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.073 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:04.073 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044702+0000", 2023-11-25T04:24:04.073 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1034e-05 2023-11-25T04:24:04.074 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:04.074 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:04.074 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:04.074 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.074 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.074 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:29 3.b 3:d0630e4c:::mds0_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:04.074 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:04.074 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 77.005239595000006, 2023-11-25T04:24:04.074 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010980227, 2023-11-25T04:24:04.074 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:04.074 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:04.074 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:04.075 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.075 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:04.075 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:04.075 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:04.075 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.075 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.075 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:04.075 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:04.075 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:04.075 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.075 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.075 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:04.075 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:04.076 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3499999999999996e-07 2023-11-25T04:24:04.076 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.076 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.076 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:04.076 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033658+0000", 2023-11-25T04:24:04.076 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1280000000000001e-06 2023-11-25T04:24:04.076 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.076 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.076 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:04.076 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033667+0000", 2023-11-25T04:24:04.076 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1430000000000004e-06 2023-11-25T04:24:04.076 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.076 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.077 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:04.077 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033678+0000", 2023-11-25T04:24:04.077 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0906e-05 2023-11-25T04:24:04.077 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.077 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.077 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:04.077 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034456+0000", 2023-11-25T04:24:04.077 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00077833899999999996 2023-11-25T04:24:04.077 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.077 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.077 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:04.077 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034575+0000", 2023-11-25T04:24:04.077 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011926000000000001 2023-11-25T04:24:04.078 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.078 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.078 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:04.078 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035001+0000", 2023-11-25T04:24:04.078 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042550999999999998 2023-11-25T04:24:04.078 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.078 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.078 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:04.078 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.037923+0000", 2023-11-25T04:24:04.078 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029219670000000001 2023-11-25T04:24:04.078 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.078 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.078 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:04.078 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044538+0000", 2023-11-25T04:24:04.079 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0066152349999999997 2023-11-25T04:24:04.079 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.079 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.079 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:04.079 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044594+0000", 2023-11-25T04:24:04.079 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5593000000000001e-05 2023-11-25T04:24:04.079 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.079 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.079 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:04.079 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044633+0000", 2023-11-25T04:24:04.079 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9511000000000002e-05 2023-11-25T04:24:04.079 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:04.079 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:04.080 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:04.080 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.080 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.080 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:34 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:04.080 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:24:04.080 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 77.004989481999999, 2023-11-25T04:24:04.080 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010920445000000001, 2023-11-25T04:24:04.080 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:04.080 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:04.080 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:04.080 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.080 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:04.080 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:24:04.081 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:04.081 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.081 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.081 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:04.081 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:24:04.081 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:04.081 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.081 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.081 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:04.081 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033904+0000", 2023-11-25T04:24:04.081 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6599999999999994e-07 2023-11-25T04:24:04.081 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.081 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.082 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:04.082 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033909+0000", 2023-11-25T04:24:04.082 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5639999999999996e-06 2023-11-25T04:24:04.082 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.082 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.082 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:04.082 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033923+0000", 2023-11-25T04:24:04.082 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3028e-05 2023-11-25T04:24:04.082 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.082 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.082 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:04.082 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034043+0000", 2023-11-25T04:24:04.082 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000120328 2023-11-25T04:24:04.083 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.083 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.083 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:04.083 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035576+0000", 2023-11-25T04:24:04.083 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015333720000000001 2023-11-25T04:24:04.083 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.083 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.083 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:04.083 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035699+0000", 2023-11-25T04:24:04.083 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000123001 2023-11-25T04:24:04.084 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.084 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.084 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:04.084 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044823+0000", 2023-11-25T04:24:04.084 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0091241859999999994 2023-11-25T04:24:04.084 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:04.084 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:04.084 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:04.084 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.084 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.084 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:35 3.1f 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:04.084 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:24:04.084 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 77.003685277000002, 2023-11-25T04:24:04.085 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096164779999999995, 2023-11-25T04:24:04.085 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:04.085 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:04.085 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:04.085 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.085 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:04.085 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:24:04.085 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:04.085 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.085 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.085 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:04.085 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:24:04.085 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:04.086 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.086 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.086 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:04.086 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035208+0000", 2023-11-25T04:24:04.086 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2699999999999998e-07 2023-11-25T04:24:04.086 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.086 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.086 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:04.086 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035213+0000", 2023-11-25T04:24:04.086 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.6070000000000001e-06 2023-11-25T04:24:04.086 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.086 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.086 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:04.087 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035350+0000", 2023-11-25T04:24:04.087 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013693599999999999 2023-11-25T04:24:04.087 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.087 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.087 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:04.087 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035358+0000", 2023-11-25T04:24:04.087 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0099999999999995e-06 2023-11-25T04:24:04.087 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.087 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.087 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:04.088 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036260+0000", 2023-11-25T04:24:04.088 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00090240100000000005 2023-11-25T04:24:04.088 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.088 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.088 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:04.088 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036517+0000", 2023-11-25T04:24:04.088 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00025736299999999998 2023-11-25T04:24:04.088 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.088 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.088 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:04.088 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044824+0000", 2023-11-25T04:24:04.088 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0083063340000000003 2023-11-25T04:24:04.088 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:04.089 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:04.089 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:04.089 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.089 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.089 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:37 3.1e 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:04.089 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:24:04.089 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 77.003518591000002, 2023-11-25T04:24:04.089 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0078322640000000002, 2023-11-25T04:24:04.089 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:04.089 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:04.089 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:04.089 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.090 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:04.090 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:24:04.090 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:04.090 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.090 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.090 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:04.090 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:24:04.090 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:04.090 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.090 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.090 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:04.090 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035375+0000", 2023-11-25T04:24:04.090 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9100000000000003e-07 2023-11-25T04:24:04.091 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.091 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.091 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:04.091 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035379+0000", 2023-11-25T04:24:04.091 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9430000000000002e-06 2023-11-25T04:24:04.091 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.091 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.091 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:04.091 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035388+0000", 2023-11-25T04:24:04.091 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2760000000000006e-06 2023-11-25T04:24:04.091 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.091 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.091 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:04.092 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035396+0000", 2023-11-25T04:24:04.092 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8480000000000001e-06 2023-11-25T04:24:04.092 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.092 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.092 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:04.092 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035452+0000", 2023-11-25T04:24:04.092 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5831000000000001e-05 2023-11-25T04:24:04.092 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.092 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.092 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:04.092 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035934+0000", 2023-11-25T04:24:04.092 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00048265599999999998 2023-11-25T04:24:04.093 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.093 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.093 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:04.093 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043206+0000", 2023-11-25T04:24:04.093 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.007271919 2023-11-25T04:24:04.093 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:04.093 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:04.093 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:04.093 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.093 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.093 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing(3.1a epoch 18/16 rep_tid 3 entries 18'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:36 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:24:04.093 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:24:04.093 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 77.002243004999997, 2023-11-25T04:24:04.094 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002073414, 2023-11-25T04:24:04.094 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:04.094 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:24:04.094 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:04.094 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.094 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:04.094 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:24:04.094 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:04.094 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.094 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.094 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:04.094 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:24:04.094 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:04.095 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.095 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.095 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:04.095 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036650+0000", 2023-11-25T04:24:04.095 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7300000000000002e-07 2023-11-25T04:24:04.095 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.095 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.095 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:04.095 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036659+0000", 2023-11-25T04:24:04.095 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.7499999999999992e-06 2023-11-25T04:24:04.095 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.095 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.095 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:04.096 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036673+0000", 2023-11-25T04:24:04.096 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4917e-05 2023-11-25T04:24:04.096 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.096 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.096 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:04.096 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036687+0000", 2023-11-25T04:24:04.096 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3883000000000001e-05 2023-11-25T04:24:04.096 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.096 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.096 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:04.096 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036781+0000", 2023-11-25T04:24:04.096 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3219000000000003e-05 2023-11-25T04:24:04.096 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.097 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.097 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:04.097 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038723+0000", 2023-11-25T04:24:04.097 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019422720000000001 2023-11-25T04:24:04.097 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:04.097 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:04.097 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:04.097 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.097 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.097 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:28 3.13 e18/16 3:cd5a64a3:::100.00000000:head v 18'1, mlcod=0'0)", 2023-11-25T04:24:04.097 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:24:04.097 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 76.998581846999997, 2023-11-25T04:24:04.097 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001850546, 2023-11-25T04:24:04.098 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:04.098 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:04.098 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:04.098 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.098 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:04.098 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:24:04.098 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:04.098 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.098 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.098 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:04.098 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:24:04.098 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:04.098 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.099 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.099 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:04.099 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:24:04.099 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7800000000000002e-07 2023-11-25T04:24:04.099 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.099 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.099 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:04.099 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040331+0000", 2023-11-25T04:24:04.099 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0449999999999999e-05 2023-11-25T04:24:04.099 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.099 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.099 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:04.099 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040339+0000", 2023-11-25T04:24:04.100 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0709999999999995e-06 2023-11-25T04:24:04.100 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.100 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.100 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:04.100 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040347+0000", 2023-11-25T04:24:04.100 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7079999999999997e-06 2023-11-25T04:24:04.100 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.100 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.100 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:04.100 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040410+0000", 2023-11-25T04:24:04.100 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2453000000000007e-05 2023-11-25T04:24:04.100 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.100 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.101 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:04.101 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040422+0000", 2023-11-25T04:24:04.101 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2429000000000001e-05 2023-11-25T04:24:04.101 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.101 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.101 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:04.101 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042129+0000", 2023-11-25T04:24:04.101 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001707266 2023-11-25T04:24:04.101 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.101 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.101 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:04.101 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042161+0000", 2023-11-25T04:24:04.101 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1791000000000001e-05 2023-11-25T04:24:04.102 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:04.102 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:04.102 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:04.102 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.102 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.102 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:27 3.13 e18/16 3:c91d4a1d:::mds0_inotable:head v 18'2, mlcod=0'0)", 2023-11-25T04:24:04.102 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:24:04.102 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 76.998378324000001, 2023-11-25T04:24:04.102 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0016886430000000001, 2023-11-25T04:24:04.102 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:04.102 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:04.102 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:04.102 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.103 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:04.103 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:24:04.103 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:04.103 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.103 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.103 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:04.103 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:24:04.103 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:04.103 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.103 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.103 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:04.103 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:24:04.103 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3999999999999997e-07 2023-11-25T04:24:04.104 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.104 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.104 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:04.104 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040520+0000", 2023-11-25T04:24:04.104 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2989999999999996e-06 2023-11-25T04:24:04.104 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.104 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.104 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:04.104 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040527+0000", 2023-11-25T04:24:04.104 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5730000000000002e-06 2023-11-25T04:24:04.104 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.104 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.105 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:04.105 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040535+0000", 2023-11-25T04:24:04.105 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8630000000000008e-06 2023-11-25T04:24:04.105 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.105 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.105 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:04.105 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040683+0000", 2023-11-25T04:24:04.105 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000148002 2023-11-25T04:24:04.105 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.105 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.105 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:04.105 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040700+0000", 2023-11-25T04:24:04.105 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7065e-05 2023-11-25T04:24:04.106 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.106 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.106 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:04.106 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042163+0000", 2023-11-25T04:24:04.106 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001462767 2023-11-25T04:24:04.106 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.106 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.106 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:04.106 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042203+0000", 2023-11-25T04:24:04.106 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9734000000000003e-05 2023-11-25T04:24:04.106 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:04.106 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:04.106 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:04.107 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.107 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.107 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:20 3.1c e18/16 3:3ed09add:::607.00000000:head v 18'1, mlcod=0'0)", 2023-11-25T04:24:04.107 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:24:04.107 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 76.998343282999997, 2023-11-25T04:24:04.107 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015271, 2023-11-25T04:24:04.107 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:04.107 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:04.107 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:04.107 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.107 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:04.107 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:24:04.107 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:04.108 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.108 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.108 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:04.108 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:24:04.108 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:04.108 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.108 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.108 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:04.108 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:24:04.108 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4799999999999999e-07 2023-11-25T04:24:04.108 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.108 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.108 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:04.109 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040561+0000", 2023-11-25T04:24:04.109 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1800000000000001e-05 2023-11-25T04:24:04.109 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.109 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.109 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:04.109 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040568+0000", 2023-11-25T04:24:04.109 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7730000000000002e-06 2023-11-25T04:24:04.109 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.109 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.109 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:04.109 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040587+0000", 2023-11-25T04:24:04.109 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8691000000000001e-05 2023-11-25T04:24:04.109 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.110 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.110 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:04.110 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040638+0000", 2023-11-25T04:24:04.110 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1068e-05 2023-11-25T04:24:04.110 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.110 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.110 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:04.110 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040670+0000", 2023-11-25T04:24:04.110 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2125999999999997e-05 2023-11-25T04:24:04.110 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.110 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.110 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:04.111 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042032+0000", 2023-11-25T04:24:04.111 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001361821 2023-11-25T04:24:04.111 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.111 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.111 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:04.111 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042076+0000", 2023-11-25T04:24:04.111 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4472999999999999e-05 2023-11-25T04:24:04.111 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:04.111 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:04.111 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:04.111 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.111 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.111 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:21 3.1c e18/16 3:3ed09add:::607.00000000:head v 18'2, mlcod=0'0)", 2023-11-25T04:24:04.112 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:24:04.112 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 76.998108557999998, 2023-11-25T04:24:04.112 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0013228160000000001, 2023-11-25T04:24:04.112 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:04.112 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:04.112 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:04.112 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.112 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:04.112 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:24:04.112 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:04.112 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.112 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.112 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:04.113 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:24:04.113 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:04.113 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.113 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.113 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:04.113 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:24:04.113 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9299999999999999e-07 2023-11-25T04:24:04.113 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.113 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.113 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:04.113 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040790+0000", 2023-11-25T04:24:04.113 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5770000000000001e-06 2023-11-25T04:24:04.114 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.114 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.114 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:04.114 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040799+0000", 2023-11-25T04:24:04.114 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6439999999999995e-06 2023-11-25T04:24:04.114 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.114 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.114 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:04.114 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040806+0000", 2023-11-25T04:24:04.114 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0180000000000004e-06 2023-11-25T04:24:04.114 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.114 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.114 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:04.115 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040907+0000", 2023-11-25T04:24:04.115 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000100096 2023-11-25T04:24:04.115 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.115 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.115 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:04.115 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040925+0000", 2023-11-25T04:24:04.115 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8536999999999999e-05 2023-11-25T04:24:04.115 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.115 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.115 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:04.115 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042079+0000", 2023-11-25T04:24:04.115 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00115358 2023-11-25T04:24:04.116 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.116 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.116 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:04.116 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042107+0000", 2023-11-25T04:24:04.116 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7971e-05 2023-11-25T04:24:04.116 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:04.116 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:04.116 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:04.116 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.116 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.116 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop_reply(mds.0.3:4 3.1f e18/16 ondisk, result = 0)", 2023-11-25T04:24:04.116 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:24:04.116 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 76.996769168, 2023-11-25T04:24:04.117 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00044092900000000001, 2023-11-25T04:24:04.117 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:04.117 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:04.117 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:04.117 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.117 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:04.117 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:24:04.117 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:04.117 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.117 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.117 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:04.117 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:24:04.117 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:04.118 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.118 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.118 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:04.118 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042124+0000", 2023-11-25T04:24:04.118 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1800000000000005e-07 2023-11-25T04:24:04.118 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.118 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.118 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:04.118 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042128+0000", 2023-11-25T04:24:04.118 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2289999999999997e-06 2023-11-25T04:24:04.118 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.118 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.118 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:04.119 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042138+0000", 2023-11-25T04:24:04.119 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9729999999999992e-06 2023-11-25T04:24:04.119 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.119 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.119 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:04.119 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042145+0000", 2023-11-25T04:24:04.119 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.336e-06 2023-11-25T04:24:04.119 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.119 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.119 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:04.119 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042441+0000", 2023-11-25T04:24:04.119 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029650500000000002 2023-11-25T04:24:04.120 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.120 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.120 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:04.120 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042448+0000", 2023-11-25T04:24:04.120 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4100000000000002e-06 2023-11-25T04:24:04.120 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.120 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.120 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:04.120 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042564+0000", 2023-11-25T04:24:04.120 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000115758 2023-11-25T04:24:04.120 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:04.120 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:04.120 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:04.121 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.121 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.121 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing_reply(3.1f epoch 18/16 rep_tid 8 lcod 18'4)", 2023-11-25T04:24:04.121 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:24:04.121 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 76.994443755999995, 2023-11-25T04:24:04.121 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00038154699999999998, 2023-11-25T04:24:04.121 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:04.121 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:24:04.121 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:04.121 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.121 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:04.121 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:24:04.121 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:04.122 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.122 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.122 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:04.122 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:24:04.122 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:04.122 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.122 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.122 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:04.122 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:24:04.122 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2700000000000001e-07 2023-11-25T04:24:04.122 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.122 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.123 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:04.123 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044451+0000", 2023-11-25T04:24:04.123 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7460000000000001e-06 2023-11-25T04:24:04.123 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.123 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.123 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:04.123 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044455+0000", 2023-11-25T04:24:04.123 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5809999999999998e-06 2023-11-25T04:24:04.123 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.123 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.123 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:04.123 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044458+0000", 2023-11-25T04:24:04.123 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0170000000000001e-06 2023-11-25T04:24:04.124 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.124 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.124 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:04.124 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044734+0000", 2023-11-25T04:24:04.124 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00027582799999999999 2023-11-25T04:24:04.124 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.124 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.124 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:04.124 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044830+0000", 2023-11-25T04:24:04.124 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6248000000000004e-05 2023-11-25T04:24:04.124 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:04.124 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:04.124 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:04.125 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.125 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.125 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing_reply(3.b epoch 18/16 rep_tid 5 lcod 18'2)", 2023-11-25T04:24:04.125 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:24:04.125 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 76.994407095, 2023-11-25T04:24:04.125 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000344488, 2023-11-25T04:24:04.125 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:04.125 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:24:04.125 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:04.125 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.125 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:04.125 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:24:04.126 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:04.126 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.126 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.126 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:04.126 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:24:04.126 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:04.126 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.126 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.126 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:04.126 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:24:04.126 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.55e-07 2023-11-25T04:24:04.126 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.126 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.127 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:04.127 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044487+0000", 2023-11-25T04:24:04.127 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2470000000000001e-06 2023-11-25T04:24:04.127 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.127 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.127 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:04.127 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044491+0000", 2023-11-25T04:24:04.127 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9659999999999998e-06 2023-11-25T04:24:04.127 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.127 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.128 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:04.128 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044494+0000", 2023-11-25T04:24:04.128 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2310000000000001e-06 2023-11-25T04:24:04.128 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.128 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.128 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:04.128 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044756+0000", 2023-11-25T04:24:04.128 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00026172800000000003 2023-11-25T04:24:04.128 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.128 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.128 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:04.128 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044830+0000", 2023-11-25T04:24:04.128 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4160999999999997e-05 2023-11-25T04:24:04.129 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:04.129 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:04.129 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:04.129 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.129 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.129 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:39 3.6 3:654134d2:::mds0_openfiles.0:head [omap-clear,omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2023-11-25T04:24:04.129 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:24:04.129 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 71.934581053000002, 2023-11-25T04:24:04.129 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001788658, 2023-11-25T04:24:04.129 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:04.129 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:04.129 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:04.130 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.130 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:04.130 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:24:04.130 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:04.130 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.130 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.130 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:04.130 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:24:04.130 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:04.131 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.131 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.131 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:04.131 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104314+0000", 2023-11-25T04:24:04.131 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3099999999999999e-06 2023-11-25T04:24:04.131 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.131 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.131 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:04.131 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104325+0000", 2023-11-25T04:24:04.131 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1511999999999999e-05 2023-11-25T04:24:04.131 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.131 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.131 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:04.132 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104362+0000", 2023-11-25T04:24:04.132 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6903000000000002e-05 2023-11-25T04:24:04.132 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.132 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.132 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:04.132 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104387+0000", 2023-11-25T04:24:04.132 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4766e-05 2023-11-25T04:24:04.132 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.132 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.132 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:04.132 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104465+0000", 2023-11-25T04:24:04.132 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8119000000000002e-05 2023-11-25T04:24:04.132 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.133 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.133 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:04.133 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104557+0000", 2023-11-25T04:24:04.133 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2496000000000003e-05 2023-11-25T04:24:04.133 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.133 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.133 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:04.133 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104646+0000", 2023-11-25T04:24:04.133 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8818000000000002e-05 2023-11-25T04:24:04.133 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.133 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.133 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:04.134 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105201+0000", 2023-11-25T04:24:04.134 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00055506000000000002 2023-11-25T04:24:04.134 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.134 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.134 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:04.134 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105994+0000", 2023-11-25T04:24:04.134 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00079294700000000001 2023-11-25T04:24:04.134 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.134 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.134 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:04.134 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.106055+0000", 2023-11-25T04:24:04.134 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0566000000000003e-05 2023-11-25T04:24:04.134 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.135 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.135 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:04.135 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.106100+0000", 2023-11-25T04:24:04.135 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5160999999999997e-05 2023-11-25T04:24:04.135 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:04.135 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:04.135 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:04.135 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.135 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.135 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:43 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2023-11-25T04:24:04.135 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:24:04.135 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 56.934356057000002, 2023-11-25T04:24:04.136 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001751166, 2023-11-25T04:24:04.136 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:04.136 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:04.136 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:04.136 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.136 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:04.136 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:24:04.136 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:04.136 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.136 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.136 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:04.136 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:24:04.136 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:04.137 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.137 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.137 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:04.137 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104539+0000", 2023-11-25T04:24:04.137 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9529999999999999e-06 2023-11-25T04:24:04.137 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.137 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.137 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:04.137 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104557+0000", 2023-11-25T04:24:04.137 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7464000000000001e-05 2023-11-25T04:24:04.137 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.137 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.137 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:04.138 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104590+0000", 2023-11-25T04:24:04.138 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2966000000000003e-05 2023-11-25T04:24:04.138 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.138 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.138 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:04.138 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104608+0000", 2023-11-25T04:24:04.138 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8193e-05 2023-11-25T04:24:04.138 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.138 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.138 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:04.138 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104767+0000", 2023-11-25T04:24:04.138 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015882 2023-11-25T04:24:04.139 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.139 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.139 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:04.139 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104909+0000", 2023-11-25T04:24:04.139 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014244999999999999 2023-11-25T04:24:04.139 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.139 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.139 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:04.139 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.105059+0000", 2023-11-25T04:24:04.139 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014964399999999999 2023-11-25T04:24:04.139 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.139 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.140 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:04.140 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.105701+0000", 2023-11-25T04:24:04.140 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00064201099999999995 2023-11-25T04:24:04.140 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.140 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.140 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:04.140 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106233+0000", 2023-11-25T04:24:04.140 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00053178500000000001 2023-11-25T04:24:04.140 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.140 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.140 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:04.140 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106265+0000", 2023-11-25T04:24:04.140 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1986000000000002e-05 2023-11-25T04:24:04.141 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.141 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.141 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:04.141 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106288+0000", 2023-11-25T04:24:04.141 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2894000000000001e-05 2023-11-25T04:24:04.141 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:04.141 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:04.141 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:04.141 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.141 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.141 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:45 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e23)", 2023-11-25T04:24:04.141 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:24:04.141 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 51.934158017000001, 2023-11-25T04:24:04.142 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00098423299999999994, 2023-11-25T04:24:04.142 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:04.142 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:04.142 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:04.142 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.142 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:04.142 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:24:04.142 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:04.142 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.142 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.142 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:04.142 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:24:04.143 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:04.143 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.143 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.143 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:04.143 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104737+0000", 2023-11-25T04:24:04.143 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3309999999999998e-06 2023-11-25T04:24:04.143 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.143 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.143 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:04.143 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104748+0000", 2023-11-25T04:24:04.143 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1080999999999999e-05 2023-11-25T04:24:04.143 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.143 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.143 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:04.144 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104771+0000", 2023-11-25T04:24:04.144 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2707000000000001e-05 2023-11-25T04:24:04.144 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.144 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.144 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:04.144 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104800+0000", 2023-11-25T04:24:04.144 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9595e-05 2023-11-25T04:24:04.144 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.144 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.144 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:04.144 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104890+0000", 2023-11-25T04:24:04.144 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.9808999999999993e-05 2023-11-25T04:24:04.144 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.145 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.145 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:04.145 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105004+0000", 2023-11-25T04:24:04.145 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000114508 2023-11-25T04:24:04.145 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.145 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.145 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:04.145 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105634+0000", 2023-11-25T04:24:04.145 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00062985799999999996 2023-11-25T04:24:04.145 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.145 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.145 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:04.146 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105671+0000", 2023-11-25T04:24:04.146 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6640000000000002e-05 2023-11-25T04:24:04.146 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.146 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.146 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:04.146 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105719+0000", 2023-11-25T04:24:04.146 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7704000000000003e-05 2023-11-25T04:24:04.146 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:04.146 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:04.146 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:04.146 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:04.146 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:04.146 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:04.148 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:24:04.148 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:04.146+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:04.249 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:04.248+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:04.258 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:04.258 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:24:04.258 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2023-11-25T04:24:04.258 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:04.268 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:24:04.350 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:04.348+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:04.374 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:04.374 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:24:04.374 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2023-11-25T04:24:04.374 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2023-11-25T04:24:04.374 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:04.383 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:24:04.450 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:04.449+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:04.532 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:04.533 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2023-11-25T04:24:04.533 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2023-11-25T04:24:04.533 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2023-11-25T04:24:04.533 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.533 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:155 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'109, mlcod=11'105)", 2023-11-25T04:24:04.533 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:24:04.533 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.744891967000001, 2023-11-25T04:24:04.533 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061719019999999999, 2023-11-25T04:24:04.533 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:04.534 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:04.534 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:04.534 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.534 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:04.534 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:24:04.534 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:04.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.534 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.534 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:04.534 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:24:04.534 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:04.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.535 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:04.535 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784884+0000", 2023-11-25T04:24:04.535 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3099999999999999e-07 2023-11-25T04:24:04.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.535 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:04.535 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784942+0000", 2023-11-25T04:24:04.535 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8570000000000003e-05 2023-11-25T04:24:04.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.535 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:04.536 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784949+0000", 2023-11-25T04:24:04.536 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0310000000000001e-06 2023-11-25T04:24:04.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.536 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.536 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:04.536 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784953+0000", 2023-11-25T04:24:04.536 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6320000000000001e-06 2023-11-25T04:24:04.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.536 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.536 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:04.536 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784993+0000", 2023-11-25T04:24:04.536 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9737000000000003e-05 2023-11-25T04:24:04.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.537 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.537 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:04.537 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785011+0000", 2023-11-25T04:24:04.537 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8349999999999999e-05 2023-11-25T04:24:04.537 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.537 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.537 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:04.537 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791024+0000", 2023-11-25T04:24:04.537 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006012997 2023-11-25T04:24:04.537 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.537 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.537 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:04.538 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791055+0000", 2023-11-25T04:24:04.538 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1254000000000003e-05 2023-11-25T04:24:04.538 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:04.538 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:04.538 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:04.538 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.538 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.538 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:156 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'110, mlcod=11'105)", 2023-11-25T04:24:04.538 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:24:04.538 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.744305917999995, 2023-11-25T04:24:04.538 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005616116, 2023-11-25T04:24:04.539 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:04.539 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:04.539 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:04.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.539 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:04.539 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:24:04.539 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:04.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.539 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:04.539 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:24:04.539 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:04.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.540 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.540 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:04.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785470+0000", 2023-11-25T04:24:04.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0699999999999998e-07 2023-11-25T04:24:04.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.540 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.540 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:04.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785474+0000", 2023-11-25T04:24:04.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.048e-06 2023-11-25T04:24:04.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.540 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.540 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:04.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785480+0000", 2023-11-25T04:24:04.541 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7729999999999996e-06 2023-11-25T04:24:04.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.541 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.541 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:04.541 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785483+0000", 2023-11-25T04:24:04.541 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1970000000000001e-06 2023-11-25T04:24:04.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.541 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.541 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:04.541 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787936+0000", 2023-11-25T04:24:04.541 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024529040000000001 2023-11-25T04:24:04.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.542 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.542 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:04.542 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787947+0000", 2023-11-25T04:24:04.542 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1669e-05 2023-11-25T04:24:04.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.542 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.542 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:04.542 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791057+0000", 2023-11-25T04:24:04.542 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031095200000000002 2023-11-25T04:24:04.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.542 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.543 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:04.543 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791086+0000", 2023-11-25T04:24:04.543 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8697999999999999e-05 2023-11-25T04:24:04.543 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:04.543 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:04.543 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:04.543 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.543 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.543 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:157 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'111, mlcod=11'106)", 2023-11-25T04:24:04.543 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:24:04.543 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.743713360000001, 2023-11-25T04:24:04.543 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050590280000000001, 2023-11-25T04:24:04.543 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:04.544 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:04.544 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:04.544 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.544 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:04.544 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:24:04.544 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:04.544 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.544 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.544 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:04.544 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:24:04.544 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:04.544 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.545 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.545 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:04.545 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786063+0000", 2023-11-25T04:24:04.545 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2600000000000002e-07 2023-11-25T04:24:04.545 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.545 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.545 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:04.545 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786069+0000", 2023-11-25T04:24:04.545 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8839999999999999e-06 2023-11-25T04:24:04.545 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.545 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.545 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:04.545 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786079+0000", 2023-11-25T04:24:04.546 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0124e-05 2023-11-25T04:24:04.546 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.546 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.546 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:04.546 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786085+0000", 2023-11-25T04:24:04.546 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6660000000000002e-06 2023-11-25T04:24:04.546 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.546 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.546 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:04.546 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788160+0000", 2023-11-25T04:24:04.546 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020747669999999999 2023-11-25T04:24:04.546 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.547 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.547 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:04.547 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788180+0000", 2023-11-25T04:24:04.547 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0208999999999999e-05 2023-11-25T04:24:04.547 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.547 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.547 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:04.547 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791087+0000", 2023-11-25T04:24:04.547 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029064630000000002 2023-11-25T04:24:04.547 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.547 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.547 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:04.547 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791121+0000", 2023-11-25T04:24:04.548 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4289000000000003e-05 2023-11-25T04:24:04.548 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:04.548 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:04.548 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:04.548 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.548 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.548 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:158 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'112, mlcod=11'106)", 2023-11-25T04:24:04.548 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:24:04.548 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.743296967000006, 2023-11-25T04:24:04.548 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046663030000000001, 2023-11-25T04:24:04.548 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:04.548 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:04.549 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:04.549 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.549 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:04.549 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:24:04.549 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:04.549 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.549 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.549 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:04.549 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:24:04.549 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:04.549 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.549 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.550 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:04.550 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786479+0000", 2023-11-25T04:24:04.550 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2799999999999996e-07 2023-11-25T04:24:04.550 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.550 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.550 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:04.550 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786498+0000", 2023-11-25T04:24:04.550 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9069e-05 2023-11-25T04:24:04.550 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.550 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.550 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:04.551 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786508+0000", 2023-11-25T04:24:04.551 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5810000000000003e-06 2023-11-25T04:24:04.551 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.551 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.551 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:04.551 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786512+0000", 2023-11-25T04:24:04.551 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4889999999999997e-06 2023-11-25T04:24:04.551 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.551 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.551 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:04.551 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788438+0000", 2023-11-25T04:24:04.551 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019257059999999999 2023-11-25T04:24:04.551 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.552 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.552 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:04.552 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788449+0000", 2023-11-25T04:24:04.552 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1419e-05 2023-11-25T04:24:04.552 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.552 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.552 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:04.552 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791122+0000", 2023-11-25T04:24:04.552 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002672822 2023-11-25T04:24:04.552 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.552 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.553 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:04.553 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791145+0000", 2023-11-25T04:24:04.553 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2589e-05 2023-11-25T04:24:04.553 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:04.553 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:04.553 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:04.553 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.553 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.553 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:206 1.0 e12/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 12'152, mlcod=12'148)", 2023-11-25T04:24:04.553 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:24:04.553 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.231528496999999, 2023-11-25T04:24:04.554 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046728189999999999, 2023-11-25T04:24:04.554 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:04.554 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:04.554 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:04.554 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.554 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:04.554 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:24:04.554 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:04.554 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.554 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.554 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:04.554 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:24:04.555 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:04.555 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.555 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.555 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:04.555 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298248+0000", 2023-11-25T04:24:04.555 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3600000000000003e-07 2023-11-25T04:24:04.555 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.555 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.555 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:04.555 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298278+0000", 2023-11-25T04:24:04.555 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0474e-05 2023-11-25T04:24:04.555 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.556 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.556 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:04.556 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298288+0000", 2023-11-25T04:24:04.556 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0156000000000001e-05 2023-11-25T04:24:04.556 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.556 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.556 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:04.556 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298293+0000", 2023-11-25T04:24:04.556 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7400000000000004e-06 2023-11-25T04:24:04.556 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.556 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.557 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:04.557 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298333+0000", 2023-11-25T04:24:04.557 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9666000000000003e-05 2023-11-25T04:24:04.557 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.557 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.557 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:04.557 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298351+0000", 2023-11-25T04:24:04.557 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8544999999999999e-05 2023-11-25T04:24:04.557 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.557 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.557 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:04.558 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302891+0000", 2023-11-25T04:24:04.558 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045398519999999996 2023-11-25T04:24:04.558 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.558 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.558 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:04.558 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302920+0000", 2023-11-25T04:24:04.558 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8750000000000001e-05 2023-11-25T04:24:04.558 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:04.558 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:04.558 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:04.558 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.558 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.559 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:14 3.15 3:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:04.559 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:24:04.559 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 77.496394140000007, 2023-11-25T04:24:04.559 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010990437, 2023-11-25T04:24:04.559 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:04.559 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:04.559 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:04.559 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.559 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:04.559 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:24:04.559 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:04.559 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.560 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.560 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:04.560 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:24:04.560 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:04.560 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.560 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.560 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:04.560 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033383+0000", 2023-11-25T04:24:04.560 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9870000000000002e-06 2023-11-25T04:24:04.560 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.560 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.560 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:04.561 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033398+0000", 2023-11-25T04:24:04.561 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4569999999999999e-05 2023-11-25T04:24:04.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.561 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:04.561 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033411+0000", 2023-11-25T04:24:04.561 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3543999999999999e-05 2023-11-25T04:24:04.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.561 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:04.561 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033423+0000", 2023-11-25T04:24:04.562 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1905999999999999e-05 2023-11-25T04:24:04.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.562 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.562 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:04.562 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034433+0000", 2023-11-25T04:24:04.562 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00101022 2023-11-25T04:24:04.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.562 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.562 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:04.562 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034539+0000", 2023-11-25T04:24:04.562 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010597199999999999 2023-11-25T04:24:04.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.563 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:04.563 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034667+0000", 2023-11-25T04:24:04.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012765500000000001 2023-11-25T04:24:04.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.563 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:04.563 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037803+0000", 2023-11-25T04:24:04.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031362220000000001 2023-11-25T04:24:04.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:04.564 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:24:04.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0064964669999999997 2023-11-25T04:24:04.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:04.564 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044331+0000", 2023-11-25T04:24:04.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1544999999999997e-05 2023-11-25T04:24:04.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.565 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:04.565 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044372+0000", 2023-11-25T04:24:04.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0349000000000003e-05 2023-11-25T04:24:04.565 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:04.565 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:04.565 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:04.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.565 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:15 3.15 3:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:04.565 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:24:04.565 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 77.495734490000004, 2023-11-25T04:24:04.566 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010464362, 2023-11-25T04:24:04.566 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:04.566 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:04.566 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:04.566 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.566 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:04.566 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:24:04.566 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:04.566 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.566 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.566 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:04.566 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:24:04.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:04.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:04.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034042+0000", 2023-11-25T04:24:04.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0999999999999997e-07 2023-11-25T04:24:04.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:04.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034048+0000", 2023-11-25T04:24:04.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0390000000000001e-06 2023-11-25T04:24:04.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:04.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034059+0000", 2023-11-25T04:24:04.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1182999999999999e-05 2023-11-25T04:24:04.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:04.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034067+0000", 2023-11-25T04:24:04.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1559999999999995e-06 2023-11-25T04:24:04.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:04.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035005+0000", 2023-11-25T04:24:04.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00093747399999999999 2023-11-25T04:24:04.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:04.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035077+0000", 2023-11-25T04:24:04.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2747e-05 2023-11-25T04:24:04.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.570 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:04.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035219+0000", 2023-11-25T04:24:04.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000141877 2023-11-25T04:24:04.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.570 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:04.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037810+0000", 2023-11-25T04:24:04.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002590752 2023-11-25T04:24:04.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.570 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:04.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044437+0000", 2023-11-25T04:24:04.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066270110000000004 2023-11-25T04:24:04.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:04.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044481+0000", 2023-11-25T04:24:04.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4106000000000001e-05 2023-11-25T04:24:04.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:04.573 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044505+0000", 2023-11-25T04:24:04.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4207e-05 2023-11-25T04:24:04.573 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:04.573 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:04.573 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:04.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.574 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:16 3.1d 3:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:04.574 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:24:04.574 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 77.495692641999995, 2023-11-25T04:24:04.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010413923, 2023-11-25T04:24:04.575 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:04.575 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:04.575 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:04.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:04.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:24:04.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:04.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:04.576 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:24:04.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:04.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.576 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:04.576 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:24:04.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4499999999999997e-07 2023-11-25T04:24:04.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.576 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:04.576 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034087+0000", 2023-11-25T04:24:04.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9770000000000002e-06 2023-11-25T04:24:04.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.577 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:04.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034094+0000", 2023-11-25T04:24:04.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8809999999999998e-06 2023-11-25T04:24:04.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.577 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:04.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034100+0000", 2023-11-25T04:24:04.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8440000000000003e-06 2023-11-25T04:24:04.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.578 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:04.578 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034993+0000", 2023-11-25T04:24:04.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00089336799999999996 2023-11-25T04:24:04.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.578 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:04.578 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035099+0000", 2023-11-25T04:24:04.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106037 2023-11-25T04:24:04.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:04.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035276+0000", 2023-11-25T04:24:04.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017631799999999999 2023-11-25T04:24:04.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:04.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037815+0000", 2023-11-25T04:24:04.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025395959999999999 2023-11-25T04:24:04.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:04.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044420+0000", 2023-11-25T04:24:04.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066045749999999997 2023-11-25T04:24:04.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:04.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044463+0000", 2023-11-25T04:24:04.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3205999999999999e-05 2023-11-25T04:24:04.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.581 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:04.581 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044497+0000", 2023-11-25T04:24:04.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3476000000000003e-05 2023-11-25T04:24:04.581 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:04.581 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:04.581 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:04.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.581 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:17 3.1d 3:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:04.581 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:24:04.582 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 77.495667432999994, 2023-11-25T04:24:04.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01127356, 2023-11-25T04:24:04.582 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:04.582 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:04.582 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:04.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:04.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:24:04.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:04.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.583 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:04.583 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:24:04.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:04.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.583 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:04.583 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:24:04.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1100000000000001e-07 2023-11-25T04:24:04.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.584 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:04.584 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034112+0000", 2023-11-25T04:24:04.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8600000000000003e-06 2023-11-25T04:24:04.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.584 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:04.584 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034120+0000", 2023-11-25T04:24:04.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7700000000000001e-06 2023-11-25T04:24:04.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:04.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034127+0000", 2023-11-25T04:24:04.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0999999999999998e-06 2023-11-25T04:24:04.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:04.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035601+0000", 2023-11-25T04:24:04.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014739410000000001 2023-11-25T04:24:04.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:04.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035685+0000", 2023-11-25T04:24:04.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3677999999999994e-05 2023-11-25T04:24:04.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:04.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035827+0000", 2023-11-25T04:24:04.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014232599999999999 2023-11-25T04:24:04.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.587 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:04.587 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038725+0000", 2023-11-25T04:24:04.587 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028982779999999998 2023-11-25T04:24:04.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.587 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:04.587 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045323+0000", 2023-11-25T04:24:04.587 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0065980780000000003 2023-11-25T04:24:04.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.587 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:04.587 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045354+0000", 2023-11-25T04:24:04.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0463e-05 2023-11-25T04:24:04.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.588 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:04.588 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045381+0000", 2023-11-25T04:24:04.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7654999999999999e-05 2023-11-25T04:24:04.588 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:04.588 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:04.588 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:04.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.589 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:22 3.e 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:04.589 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:24:04.589 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 77.495641019999994, 2023-11-25T04:24:04.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010133422, 2023-11-25T04:24:04.589 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:04.589 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:04.589 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:04.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:04.589 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:24:04.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:04.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:04.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:24:04.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:04.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:04.591 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034135+0000", 2023-11-25T04:24:04.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-07 2023-11-25T04:24:04.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.591 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:04.591 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034138+0000", 2023-11-25T04:24:04.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6009999999999999e-06 2023-11-25T04:24:04.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.591 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:04.592 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034145+0000", 2023-11-25T04:24:04.592 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1169999999999998e-06 2023-11-25T04:24:04.592 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.592 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.592 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:04.592 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034150+0000", 2023-11-25T04:24:04.592 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9080000000000003e-06 2023-11-25T04:24:04.592 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.592 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.592 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:04.592 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034209+0000", 2023-11-25T04:24:04.593 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8149000000000002e-05 2023-11-25T04:24:04.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:04.593 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034415+0000", 2023-11-25T04:24:04.593 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020637699999999999 2023-11-25T04:24:04.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:04.593 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034584+0000", 2023-11-25T04:24:04.593 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016870399999999999 2023-11-25T04:24:04.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.594 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:04.594 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.036979+0000", 2023-11-25T04:24:04.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023957509999999998 2023-11-25T04:24:04.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.594 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:04.594 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044161+0000", 2023-11-25T04:24:04.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0071811590000000003 2023-11-25T04:24:04.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:04.595 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044224+0000", 2023-11-25T04:24:04.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3454000000000006e-05 2023-11-25T04:24:04.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:04.595 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044268+0000", 2023-11-25T04:24:04.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3761999999999999e-05 2023-11-25T04:24:04.595 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:04.595 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:04.596 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:04.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.596 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.596 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:23 3.e 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:04.596 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:24:04.596 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 77.495609611000006, 2023-11-25T04:24:04.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010281676, 2023-11-25T04:24:04.596 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:04.596 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:04.596 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:04.597 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.597 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:04.597 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:24:04.597 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:04.597 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.597 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.597 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:04.597 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:24:04.597 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:04.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:04.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:24:04.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5799999999999999e-07 2023-11-25T04:24:04.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:04.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034169+0000", 2023-11-25T04:24:04.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1879999999999998e-06 2023-11-25T04:24:04.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:04.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034177+0000", 2023-11-25T04:24:04.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7020000000000001e-06 2023-11-25T04:24:04.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:04.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034184+0000", 2023-11-25T04:24:04.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3910000000000003e-06 2023-11-25T04:24:04.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:04.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034924+0000", 2023-11-25T04:24:04.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00074032999999999998 2023-11-25T04:24:04.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.600 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:04.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034991+0000", 2023-11-25T04:24:04.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7347000000000004e-05 2023-11-25T04:24:04.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.600 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:04.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035096+0000", 2023-11-25T04:24:04.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010488 2023-11-25T04:24:04.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:04.601 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037588+0000", 2023-11-25T04:24:04.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024918309999999999 2023-11-25T04:24:04.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:04.601 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044367+0000", 2023-11-25T04:24:04.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067787589999999997 2023-11-25T04:24:04.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:04.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044415+0000", 2023-11-25T04:24:04.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7803000000000003e-05 2023-11-25T04:24:04.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:04.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044447+0000", 2023-11-25T04:24:04.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2886999999999999e-05 2023-11-25T04:24:04.602 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:04.602 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:04.602 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:04.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.603 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:30 3.15 3:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:04.603 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:24:04.603 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 77.495551171000002, 2023-11-25T04:24:04.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010365070000000001, 2023-11-25T04:24:04.603 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:04.603 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:04.603 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:04.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:04.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:24:04.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:04.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:04.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:24:04.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:04.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:04.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034225+0000", 2023-11-25T04:24:04.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7400000000000002e-07 2023-11-25T04:24:04.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.605 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:04.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034229+0000", 2023-11-25T04:24:04.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3100000000000002e-06 2023-11-25T04:24:04.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.605 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:04.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034239+0000", 2023-11-25T04:24:04.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9140000000000003e-06 2023-11-25T04:24:04.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:04.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034246+0000", 2023-11-25T04:24:04.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0149999999999998e-06 2023-11-25T04:24:04.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:04.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035572+0000", 2023-11-25T04:24:04.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001325655 2023-11-25T04:24:04.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:04.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035730+0000", 2023-11-25T04:24:04.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015827899999999999 2023-11-25T04:24:04.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:04.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035857+0000", 2023-11-25T04:24:04.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012644699999999999 2023-11-25T04:24:04.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:04.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038737+0000", 2023-11-25T04:24:04.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002880077 2023-11-25T04:24:04.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:04.608 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044540+0000", 2023-11-25T04:24:04.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058034239999999997 2023-11-25T04:24:04.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:04.608 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044569+0000", 2023-11-25T04:24:04.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8388999999999999e-05 2023-11-25T04:24:04.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:04.609 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044589+0000", 2023-11-25T04:24:04.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0686000000000002e-05 2023-11-25T04:24:04.609 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:04.609 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:04.609 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:04.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.609 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:33 3.1b 3:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:04.609 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:24:04.609 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 77.495505672999997, 2023-11-25T04:24:04.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099754049999999997, 2023-11-25T04:24:04.610 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:04.610 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:04.610 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:04.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.610 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:04.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:24:04.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:04.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.610 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:04.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:24:04.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:04.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:04.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:24:04.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4799999999999998e-07 2023-11-25T04:24:04.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:04.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034273+0000", 2023-11-25T04:24:04.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7819999999999998e-06 2023-11-25T04:24:04.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:04.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034285+0000", 2023-11-25T04:24:04.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2058e-05 2023-11-25T04:24:04.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:04.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034291+0000", 2023-11-25T04:24:04.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0680000000000002e-06 2023-11-25T04:24:04.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:04.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034370+0000", 2023-11-25T04:24:04.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8452e-05 2023-11-25T04:24:04.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.613 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:04.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034477+0000", 2023-11-25T04:24:04.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107471 2023-11-25T04:24:04.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.613 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:04.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044245+0000", 2023-11-25T04:24:04.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0097679259999999997 2023-11-25T04:24:04.614 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:04.614 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:04.614 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:04.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.614 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:36 3.1a 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:04.614 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:24:04.614 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 77.495458999999997, 2023-11-25T04:24:04.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058179099999999999, 2023-11-25T04:24:04.614 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:04.614 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:04.614 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:04.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:04.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:24:04.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:04.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:04.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:24:04.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:04.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:04.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034317+0000", 2023-11-25T04:24:04.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1999999999999999e-07 2023-11-25T04:24:04.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:04.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034321+0000", 2023-11-25T04:24:04.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.233e-06 2023-11-25T04:24:04.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:04.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034337+0000", 2023-11-25T04:24:04.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5308999999999999e-05 2023-11-25T04:24:04.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.617 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:04.617 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034354+0000", 2023-11-25T04:24:04.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7411999999999999e-05 2023-11-25T04:24:04.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.617 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:04.617 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034465+0000", 2023-11-25T04:24:04.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011154699999999999 2023-11-25T04:24:04.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.617 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:04.617 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034560+0000", 2023-11-25T04:24:04.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4186999999999998e-05 2023-11-25T04:24:04.618 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.618 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:04.618 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.040134+0000", 2023-11-25T04:24:04.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055746019999999997 2023-11-25T04:24:04.618 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:04.618 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:04.618 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:04.618 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.618 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:9 3.10 e18/16 3:09401a81:::601.00000000:head v 18'4, mlcod=0'0)", 2023-11-25T04:24:04.618 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:24:04.618 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 77.486564692000002, 2023-11-25T04:24:04.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061641020000000003, 2023-11-25T04:24:04.619 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:04.619 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:04.619 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:04.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:04.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:24:04.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:04.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:04.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:24:04.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:04.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:04.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:24:04.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5300000000000001e-07 2023-11-25T04:24:04.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:04.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043222+0000", 2023-11-25T04:24:04.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.082e-05 2023-11-25T04:24:04.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:04.621 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043228+0000", 2023-11-25T04:24:04.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6699999999999997e-06 2023-11-25T04:24:04.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:04.621 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043236+0000", 2023-11-25T04:24:04.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8189999999999992e-06 2023-11-25T04:24:04.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:04.621 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043507+0000", 2023-11-25T04:24:04.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000271012 2023-11-25T04:24:04.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.622 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:04.622 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043530+0000", 2023-11-25T04:24:04.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2659000000000001e-05 2023-11-25T04:24:04.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.622 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:04.622 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049321+0000", 2023-11-25T04:24:04.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057915129999999999 2023-11-25T04:24:04.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.623 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:04.623 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049375+0000", 2023-11-25T04:24:04.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3455999999999998e-05 2023-11-25T04:24:04.623 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:04.623 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:04.623 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:04.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.623 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:1 3.14 e18/16 3:292cf221:::200.00000000:head v 18'3, mlcod=0'0)", 2023-11-25T04:24:04.623 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:24:04.623 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 77.486527979000002, 2023-11-25T04:24:04.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0060983599999999997, 2023-11-25T04:24:04.623 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:04.624 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:04.624 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:04.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:04.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:24:04.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:04.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:04.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:24:04.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:04.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:04.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043248+0000", 2023-11-25T04:24:04.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9399999999999995e-07 2023-11-25T04:24:04.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:04.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043252+0000", 2023-11-25T04:24:04.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1799999999999998e-06 2023-11-25T04:24:04.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:04.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043259+0000", 2023-11-25T04:24:04.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0049999999999998e-06 2023-11-25T04:24:04.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:04.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043265+0000", 2023-11-25T04:24:04.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0859999999999998e-06 2023-11-25T04:24:04.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:04.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043311+0000", 2023-11-25T04:24:04.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6082999999999999e-05 2023-11-25T04:24:04.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.627 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:04.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043330+0000", 2023-11-25T04:24:04.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8556e-05 2023-11-25T04:24:04.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:04.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049303+0000", 2023-11-25T04:24:04.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059733470000000004 2023-11-25T04:24:04.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:04.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049346+0000", 2023-11-25T04:24:04.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2608999999999998e-05 2023-11-25T04:24:04.630 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:04.630 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:04.630 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:04.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.630 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:3 3.14 e18/16 3:287034c4:::1.00000000.inode:head v 18'4, mlcod=0'0)", 2023-11-25T04:24:04.630 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:24:04.631 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 77.486011473000005, 2023-11-25T04:24:04.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057040090000000003, 2023-11-25T04:24:04.631 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:04.631 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:04.631 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:04.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.631 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:04.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:24:04.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:04.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:04.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:24:04.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:04.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:04.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:24:04.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9799999999999999e-07 2023-11-25T04:24:04.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:04.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043785+0000", 2023-11-25T04:24:04.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0721999999999999e-05 2023-11-25T04:24:04.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:04.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043793+0000", 2023-11-25T04:24:04.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5839999999999995e-06 2023-11-25T04:24:04.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:04.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043803+0000", 2023-11-25T04:24:04.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0729999999999994e-06 2023-11-25T04:24:04.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:04.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043843+0000", 2023-11-25T04:24:04.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0747999999999997e-05 2023-11-25T04:24:04.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:04.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043856+0000", 2023-11-25T04:24:04.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2697000000000001e-05 2023-11-25T04:24:04.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:04.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049426+0000", 2023-11-25T04:24:04.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055704409999999998 2023-11-25T04:24:04.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:04.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049468+0000", 2023-11-25T04:24:04.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1446000000000003e-05 2023-11-25T04:24:04.636 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:04.636 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:04.636 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:04.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.636 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "pg_update_log_missing(3.14 epoch 18/16 rep_tid 171 entries 18'5 (0'0) error 3:2e2fa760:::200.00000001:head by mds.0.3:32 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:24:04.636 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:24:04.636 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 77.485850510000006, 2023-11-25T04:24:04.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005556938, 2023-11-25T04:24:04.636 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:04.636 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2023-11-25T04:24:04.636 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:04.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:04.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:24:04.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:04.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:04.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:24:04.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:04.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:04.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:24:04.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.03e-07 2023-11-25T04:24:04.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:04.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043931+0000", 2023-11-25T04:24:04.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1079999999999998e-06 2023-11-25T04:24:04.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:04.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043944+0000", 2023-11-25T04:24:04.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2852e-05 2023-11-25T04:24:04.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:04.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043955+0000", 2023-11-25T04:24:04.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1037e-05 2023-11-25T04:24:04.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:04.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044088+0000", 2023-11-25T04:24:04.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013269599999999999 2023-11-25T04:24:04.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:04.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049482+0000", 2023-11-25T04:24:04.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053940419999999999 2023-11-25T04:24:04.640 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:04.640 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:04.640 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:04.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.640 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:10 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'5, mlcod=0'0)", 2023-11-25T04:24:04.640 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:24:04.640 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 77.485760482000003, 2023-11-25T04:24:04.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054493099999999997, 2023-11-25T04:24:04.640 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:04.640 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:04.640 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:04.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:04.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:24:04.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:04.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:04.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:24:04.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:04.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:04.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:24:04.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6100000000000002e-07 2023-11-25T04:24:04.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:04.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044019+0000", 2023-11-25T04:24:04.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.839e-06 2023-11-25T04:24:04.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:04.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044026+0000", 2023-11-25T04:24:04.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5869999999999998e-06 2023-11-25T04:24:04.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:04.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044031+0000", 2023-11-25T04:24:04.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4909999999999996e-06 2023-11-25T04:24:04.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:04.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044070+0000", 2023-11-25T04:24:04.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8829999999999999e-05 2023-11-25T04:24:04.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:04.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044081+0000", 2023-11-25T04:24:04.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1447999999999999e-05 2023-11-25T04:24:04.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.644 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:04.644 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049406+0000", 2023-11-25T04:24:04.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053246480000000004 2023-11-25T04:24:04.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.644 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:04.644 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049464+0000", 2023-11-25T04:24:04.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8205999999999998e-05 2023-11-25T04:24:04.644 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:04.644 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:04.644 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:04.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.645 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:11 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'6, mlcod=0'0)", 2023-11-25T04:24:04.645 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:24:04.645 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 77.485559218000006, 2023-11-25T04:24:04.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052673499999999996, 2023-11-25T04:24:04.645 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:04.646 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:04.646 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:04.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:04.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:24:04.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:04.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.647 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:04.647 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:24:04.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:04.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.647 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:04.648 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:24:04.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0199999999999998e-07 2023-11-25T04:24:04.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.648 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:04.648 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044220+0000", 2023-11-25T04:24:04.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7720000000000001e-06 2023-11-25T04:24:04.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.648 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:04.648 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044226+0000", 2023-11-25T04:24:04.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8579999999999996e-06 2023-11-25T04:24:04.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.649 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:04.649 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044232+0000", 2023-11-25T04:24:04.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1700000000000002e-06 2023-11-25T04:24:04.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.649 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:04.649 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044274+0000", 2023-11-25T04:24:04.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1254999999999997e-05 2023-11-25T04:24:04.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:04.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:24:04.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6880999999999999e-05 2023-11-25T04:24:04.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:04.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049466+0000", 2023-11-25T04:24:04.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051659219999999999 2023-11-25T04:24:04.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:04.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:04.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:04.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049483+0000", 2023-11-25T04:24:04.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.719e-05 2023-11-25T04:24:04.651 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:04.651 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:04.651 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:04.651 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:04.651 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:04.651 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:04.654 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:24:04.655 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:04.651+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:04.753 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:04.751+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:04.765 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:04.765 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:24:04.765 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2023-11-25T04:24:04.765 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:04.774 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:24:04.854 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:04.851+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:04.879 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:04.879 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:24:04.879 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2023-11-25T04:24:04.879 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2023-11-25T04:24:04.879 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:04.889 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:24:04.954 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:04.952+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:04.995 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:04.995 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2023-11-25T04:24:04.996 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2023-11-25T04:24:04.996 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2023-11-25T04:24:04.996 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.996 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:04.996 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:24:04.996 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 77.961515942000005, 2023-11-25T04:24:04.996 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01072943, 2023-11-25T04:24:04.996 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:04.996 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:04.996 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:04.996 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.996 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:04.997 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:24:04.997 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:04.997 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.997 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.997 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:04.997 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:24:04.997 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:04.997 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.997 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.997 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:04.997 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031829+0000", 2023-11-25T04:24:04.997 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.666e-06 2023-11-25T04:24:04.997 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.998 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.998 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:04.998 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031839+0000", 2023-11-25T04:24:04.998 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9490000000000008e-06 2023-11-25T04:24:04.998 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.998 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.998 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:04.998 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031857+0000", 2023-11-25T04:24:04.998 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8093000000000001e-05 2023-11-25T04:24:04.998 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.998 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.998 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:04.999 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031882+0000", 2023-11-25T04:24:04.999 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4294000000000002e-05 2023-11-25T04:24:04.999 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.999 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.999 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:04.999 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033130+0000", 2023-11-25T04:24:04.999 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001248264 2023-11-25T04:24:04.999 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:04.999 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:04.999 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:04.999 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033426+0000", 2023-11-25T04:24:04.999 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029620999999999998 2023-11-25T04:24:04.999 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.000 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.000 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:05.000 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033814+0000", 2023-11-25T04:24:05.000 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000387742 2023-11-25T04:24:05.000 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.000 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.000 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:05.000 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036793+0000", 2023-11-25T04:24:05.000 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002979598 2023-11-25T04:24:05.000 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.000 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.000 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:05.001 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042455+0000", 2023-11-25T04:24:05.001 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0056619260000000003 2023-11-25T04:24:05.001 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.001 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.001 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:05.001 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042502+0000", 2023-11-25T04:24:05.001 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7139999999999999e-05 2023-11-25T04:24:05.001 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.001 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.001 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:05.001 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042557+0000", 2023-11-25T04:24:05.001 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4548e-05 2023-11-25T04:24:05.001 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:05.002 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:05.002 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:05.002 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.002 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.002 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:05.002 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:24:05.002 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 77.960317407000005, 2023-11-25T04:24:05.002 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011505202000000001, 2023-11-25T04:24:05.002 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:05.002 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:05.002 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:05.002 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.003 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:05.003 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:24:05.003 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:05.003 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.003 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.003 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:05.003 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:24:05.003 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:05.003 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.003 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.003 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:05.003 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033027+0000", 2023-11-25T04:24:05.003 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0699999999999999e-06 2023-11-25T04:24:05.004 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.004 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.004 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:05.004 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033035+0000", 2023-11-25T04:24:05.004 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0830000000000004e-06 2023-11-25T04:24:05.004 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.004 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.004 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:05.004 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033059+0000", 2023-11-25T04:24:05.004 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3975e-05 2023-11-25T04:24:05.004 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.004 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.005 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:05.005 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033482+0000", 2023-11-25T04:24:05.005 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042253700000000001 2023-11-25T04:24:05.005 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.005 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.005 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:05.005 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034520+0000", 2023-11-25T04:24:05.005 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001037991 2023-11-25T04:24:05.006 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.006 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.006 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:05.006 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034611+0000", 2023-11-25T04:24:05.006 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1113999999999995e-05 2023-11-25T04:24:05.006 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.006 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.006 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:05.006 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034769+0000", 2023-11-25T04:24:05.006 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000157641 2023-11-25T04:24:05.007 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.007 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.007 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:05.007 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038292+0000", 2023-11-25T04:24:05.007 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035231160000000002 2023-11-25T04:24:05.007 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.007 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.007 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:05.007 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044440+0000", 2023-11-25T04:24:05.007 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0061486980000000002 2023-11-25T04:24:05.007 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.007 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.007 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:05.008 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044495+0000", 2023-11-25T04:24:05.008 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4339000000000003e-05 2023-11-25T04:24:05.008 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.008 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.008 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:05.008 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044531+0000", 2023-11-25T04:24:05.008 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6637999999999997e-05 2023-11-25T04:24:05.008 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:05.008 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:05.008 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:05.008 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.008 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.009 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:05.009 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:24:05.009 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 77.959843053, 2023-11-25T04:24:05.009 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0057634039999999997, 2023-11-25T04:24:05.009 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:05.009 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:05.009 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:05.009 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.009 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:05.009 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:24:05.009 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:05.009 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.010 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.010 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:05.010 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:24:05.010 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:05.010 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.010 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.010 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:05.010 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:24:05.010 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7900000000000007e-07 2023-11-25T04:24:05.010 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.010 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.010 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:05.011 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033507+0000", 2023-11-25T04:24:05.011 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8900000000000004e-06 2023-11-25T04:24:05.011 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.011 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.011 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:05.011 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033519+0000", 2023-11-25T04:24:05.011 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1756000000000001e-05 2023-11-25T04:24:05.011 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.011 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.011 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:05.011 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033527+0000", 2023-11-25T04:24:05.011 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3610000000000001e-06 2023-11-25T04:24:05.012 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.012 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.012 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:05.012 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034451+0000", 2023-11-25T04:24:05.012 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00092372400000000001 2023-11-25T04:24:05.012 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.012 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.012 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:05.012 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034809+0000", 2023-11-25T04:24:05.012 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00035757699999999998 2023-11-25T04:24:05.012 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.012 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.013 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:05.013 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034965+0000", 2023-11-25T04:24:05.013 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015638699999999999 2023-11-25T04:24:05.013 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.013 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.013 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:05.013 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038299+0000", 2023-11-25T04:24:05.013 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0033336559999999999 2023-11-25T04:24:05.013 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.013 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.013 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:05.013 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039169+0000", 2023-11-25T04:24:05.013 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00087066400000000005 2023-11-25T04:24:05.014 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.014 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.014 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:05.014 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039226+0000", 2023-11-25T04:24:05.014 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6511999999999997e-05 2023-11-25T04:24:05.014 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.014 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.014 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:05.014 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039264+0000", 2023-11-25T04:24:05.014 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7898000000000002e-05 2023-11-25T04:24:05.014 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:05.014 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:05.015 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:05.015 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.015 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.015 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:25 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:05.015 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:24:05.015 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 77.959764469000007, 2023-11-25T04:24:05.015 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096656169999999996, 2023-11-25T04:24:05.015 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:05.015 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:05.015 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:05.015 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.015 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:05.015 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:24:05.016 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:05.016 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.016 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.016 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:05.016 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:24:05.016 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:05.016 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.016 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.016 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:05.017 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033580+0000", 2023-11-25T04:24:05.017 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8199999999999999e-07 2023-11-25T04:24:05.017 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.017 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.017 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:05.017 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033585+0000", 2023-11-25T04:24:05.017 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3680000000000001e-06 2023-11-25T04:24:05.017 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.017 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.017 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:05.017 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033600+0000", 2023-11-25T04:24:05.017 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4331000000000001e-05 2023-11-25T04:24:05.017 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.018 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.018 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:05.018 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033606+0000", 2023-11-25T04:24:05.018 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5780000000000004e-06 2023-11-25T04:24:05.018 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.018 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.018 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:05.018 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035747+0000", 2023-11-25T04:24:05.018 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0021406530000000002 2023-11-25T04:24:05.018 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.018 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.019 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:05.019 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035901+0000", 2023-11-25T04:24:05.019 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000153931 2023-11-25T04:24:05.019 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.019 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.019 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:05.019 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036193+0000", 2023-11-25T04:24:05.019 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029211299999999999 2023-11-25T04:24:05.019 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.019 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.019 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:05.019 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038655+0000", 2023-11-25T04:24:05.019 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0024621640000000002 2023-11-25T04:24:05.020 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.020 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.020 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:05.020 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043130+0000", 2023-11-25T04:24:05.020 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.004474698 2023-11-25T04:24:05.020 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.020 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.020 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:05.020 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043204+0000", 2023-11-25T04:24:05.020 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4091999999999995e-05 2023-11-25T04:24:05.020 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.020 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.021 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:05.021 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043245+0000", 2023-11-25T04:24:05.021 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0907e-05 2023-11-25T04:24:05.021 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:05.021 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:05.021 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:05.021 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.021 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.021 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:26 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:05.021 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:05.021 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 77.959728644999998, 2023-11-25T04:24:05.021 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011086945000000001, 2023-11-25T04:24:05.021 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:05.022 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:05.022 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:05.022 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.022 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:05.022 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:05.022 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:05.022 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.022 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.022 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:05.022 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:05.022 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:05.022 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.022 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.023 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:05.023 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:05.023 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1699999999999998e-07 2023-11-25T04:24:05.023 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.023 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.023 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:05.023 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033627+0000", 2023-11-25T04:24:05.023 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.163e-05 2023-11-25T04:24:05.023 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.023 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.023 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:05.023 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033636+0000", 2023-11-25T04:24:05.024 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6859999999999995e-06 2023-11-25T04:24:05.024 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.024 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.024 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:05.024 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033642+0000", 2023-11-25T04:24:05.024 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3149999999999997e-06 2023-11-25T04:24:05.024 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.024 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.024 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:05.024 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035414+0000", 2023-11-25T04:24:05.024 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0017716380000000001 2023-11-25T04:24:05.024 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.024 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.025 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:05.025 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035539+0000", 2023-11-25T04:24:05.025 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000125477 2023-11-25T04:24:05.025 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.025 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.025 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:05.025 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035672+0000", 2023-11-25T04:24:05.025 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000133316 2023-11-25T04:24:05.025 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.025 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.025 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:05.025 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038649+0000", 2023-11-25T04:24:05.025 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029764679999999999 2023-11-25T04:24:05.026 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.026 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.026 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:05.026 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044578+0000", 2023-11-25T04:24:05.026 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0059293510000000002 2023-11-25T04:24:05.026 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.026 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.026 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:05.026 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044651+0000", 2023-11-25T04:24:05.026 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2513000000000002e-05 2023-11-25T04:24:05.026 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.026 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.027 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:05.027 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044702+0000", 2023-11-25T04:24:05.027 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1034e-05 2023-11-25T04:24:05.027 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:05.027 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:05.027 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:05.027 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.027 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.027 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:29 3.b 3:d0630e4c:::mds0_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:05.027 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:05.027 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 77.959690725000002, 2023-11-25T04:24:05.027 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010980227, 2023-11-25T04:24:05.027 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:05.028 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:05.028 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:05.028 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.028 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:05.028 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:05.028 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:05.028 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.028 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.028 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:05.028 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:05.028 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:05.028 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.029 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.029 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:05.029 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:05.029 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3499999999999996e-07 2023-11-25T04:24:05.029 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.029 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.029 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:05.029 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033658+0000", 2023-11-25T04:24:05.029 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1280000000000001e-06 2023-11-25T04:24:05.029 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.029 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.029 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:05.029 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033667+0000", 2023-11-25T04:24:05.030 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1430000000000004e-06 2023-11-25T04:24:05.030 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.030 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.030 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:05.030 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033678+0000", 2023-11-25T04:24:05.030 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0906e-05 2023-11-25T04:24:05.030 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.030 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.030 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:05.030 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034456+0000", 2023-11-25T04:24:05.030 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00077833899999999996 2023-11-25T04:24:05.030 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.030 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.031 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:05.031 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034575+0000", 2023-11-25T04:24:05.031 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011926000000000001 2023-11-25T04:24:05.031 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.031 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.031 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:05.031 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035001+0000", 2023-11-25T04:24:05.031 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042550999999999998 2023-11-25T04:24:05.031 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.031 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.032 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:05.032 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.037923+0000", 2023-11-25T04:24:05.032 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029219670000000001 2023-11-25T04:24:05.032 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.032 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.032 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:05.032 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044538+0000", 2023-11-25T04:24:05.032 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0066152349999999997 2023-11-25T04:24:05.032 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.032 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.032 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:05.032 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044594+0000", 2023-11-25T04:24:05.033 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5593000000000001e-05 2023-11-25T04:24:05.033 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.033 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.033 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:05.033 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044633+0000", 2023-11-25T04:24:05.033 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9511000000000002e-05 2023-11-25T04:24:05.033 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:05.033 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:05.033 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:05.033 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.033 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.033 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:34 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:05.033 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:24:05.033 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 77.959440611999995, 2023-11-25T04:24:05.034 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010920445000000001, 2023-11-25T04:24:05.034 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:05.034 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:05.034 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:05.034 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.034 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:05.034 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:24:05.034 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:05.034 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.034 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.034 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:05.034 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:24:05.034 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:05.035 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.035 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.035 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:05.035 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033904+0000", 2023-11-25T04:24:05.035 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6599999999999994e-07 2023-11-25T04:24:05.035 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.035 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.035 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:05.035 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033909+0000", 2023-11-25T04:24:05.035 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5639999999999996e-06 2023-11-25T04:24:05.035 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.035 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.035 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:05.036 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033923+0000", 2023-11-25T04:24:05.036 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3028e-05 2023-11-25T04:24:05.036 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.036 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.036 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:05.036 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034043+0000", 2023-11-25T04:24:05.036 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000120328 2023-11-25T04:24:05.036 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.036 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.036 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:05.036 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035576+0000", 2023-11-25T04:24:05.036 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015333720000000001 2023-11-25T04:24:05.036 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.037 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.037 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:05.037 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035699+0000", 2023-11-25T04:24:05.037 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000123001 2023-11-25T04:24:05.037 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.037 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.037 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:05.037 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044823+0000", 2023-11-25T04:24:05.037 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0091241859999999994 2023-11-25T04:24:05.037 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:05.037 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:05.037 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:05.037 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.038 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.038 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:35 3.1f 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:05.038 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:24:05.038 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 77.958136406999998, 2023-11-25T04:24:05.038 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096164779999999995, 2023-11-25T04:24:05.038 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:05.038 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:05.038 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:05.038 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.038 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:05.038 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:24:05.038 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:05.038 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.039 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.039 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:05.039 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:24:05.039 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:05.039 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.039 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.039 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:05.039 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035208+0000", 2023-11-25T04:24:05.039 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2699999999999998e-07 2023-11-25T04:24:05.039 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.039 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.039 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:05.039 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035213+0000", 2023-11-25T04:24:05.040 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.6070000000000001e-06 2023-11-25T04:24:05.040 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.040 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.040 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:05.040 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035350+0000", 2023-11-25T04:24:05.040 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013693599999999999 2023-11-25T04:24:05.040 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.040 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.040 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:05.040 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035358+0000", 2023-11-25T04:24:05.040 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0099999999999995e-06 2023-11-25T04:24:05.040 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.040 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.041 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:05.041 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036260+0000", 2023-11-25T04:24:05.041 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00090240100000000005 2023-11-25T04:24:05.041 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.041 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.041 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:05.041 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036517+0000", 2023-11-25T04:24:05.041 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00025736299999999998 2023-11-25T04:24:05.041 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.041 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.041 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:05.041 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044824+0000", 2023-11-25T04:24:05.041 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0083063340000000003 2023-11-25T04:24:05.042 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:05.042 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:05.042 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:05.042 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.042 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.042 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:37 3.1e 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:05.042 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:24:05.042 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 77.957969720999998, 2023-11-25T04:24:05.042 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0078322640000000002, 2023-11-25T04:24:05.042 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:05.042 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:05.042 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:05.043 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.043 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:05.043 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:24:05.043 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:05.043 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.043 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.043 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:05.043 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:24:05.043 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:05.043 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.043 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.043 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:05.043 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035375+0000", 2023-11-25T04:24:05.044 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9100000000000003e-07 2023-11-25T04:24:05.044 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.044 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.044 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:05.044 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035379+0000", 2023-11-25T04:24:05.044 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9430000000000002e-06 2023-11-25T04:24:05.044 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.044 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.044 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:05.044 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035388+0000", 2023-11-25T04:24:05.044 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2760000000000006e-06 2023-11-25T04:24:05.044 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.044 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.045 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:05.045 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035396+0000", 2023-11-25T04:24:05.045 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8480000000000001e-06 2023-11-25T04:24:05.045 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.045 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.045 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:05.045 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035452+0000", 2023-11-25T04:24:05.045 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5831000000000001e-05 2023-11-25T04:24:05.045 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.045 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.045 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:05.045 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035934+0000", 2023-11-25T04:24:05.045 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00048265599999999998 2023-11-25T04:24:05.046 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.046 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.046 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:05.046 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043206+0000", 2023-11-25T04:24:05.046 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.007271919 2023-11-25T04:24:05.046 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:05.046 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:05.046 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:05.046 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.046 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.046 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing(3.1a epoch 18/16 rep_tid 3 entries 18'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:36 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:24:05.046 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:24:05.046 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 77.956694135000006, 2023-11-25T04:24:05.047 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002073414, 2023-11-25T04:24:05.047 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:05.047 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:24:05.047 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:05.047 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.047 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:05.047 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:24:05.047 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:05.047 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.047 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.047 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:05.047 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:24:05.048 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:05.048 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.048 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.048 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:05.048 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036650+0000", 2023-11-25T04:24:05.048 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7300000000000002e-07 2023-11-25T04:24:05.048 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.048 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.048 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:05.048 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036659+0000", 2023-11-25T04:24:05.048 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.7499999999999992e-06 2023-11-25T04:24:05.048 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.048 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.049 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:05.049 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036673+0000", 2023-11-25T04:24:05.049 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4917e-05 2023-11-25T04:24:05.049 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.049 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.049 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:05.049 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036687+0000", 2023-11-25T04:24:05.049 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3883000000000001e-05 2023-11-25T04:24:05.049 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.050 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.050 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:05.050 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036781+0000", 2023-11-25T04:24:05.050 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3219000000000003e-05 2023-11-25T04:24:05.050 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.050 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.050 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:05.050 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038723+0000", 2023-11-25T04:24:05.050 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019422720000000001 2023-11-25T04:24:05.050 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:05.051 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:05.051 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:05.051 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.051 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.051 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:28 3.13 e18/16 3:cd5a64a3:::100.00000000:head v 18'1, mlcod=0'0)", 2023-11-25T04:24:05.051 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:24:05.051 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 77.953032977000007, 2023-11-25T04:24:05.051 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001850546, 2023-11-25T04:24:05.051 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:05.051 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:05.051 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:05.051 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.051 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:05.052 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:24:05.052 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:05.052 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.052 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.052 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:05.052 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:24:05.052 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:05.052 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.052 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.052 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:05.052 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:24:05.052 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7800000000000002e-07 2023-11-25T04:24:05.053 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.053 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.053 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:05.053 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040331+0000", 2023-11-25T04:24:05.053 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0449999999999999e-05 2023-11-25T04:24:05.053 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.053 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.053 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:05.053 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040339+0000", 2023-11-25T04:24:05.053 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0709999999999995e-06 2023-11-25T04:24:05.053 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.053 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.053 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:05.054 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040347+0000", 2023-11-25T04:24:05.054 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7079999999999997e-06 2023-11-25T04:24:05.054 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.054 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.054 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:05.054 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040410+0000", 2023-11-25T04:24:05.054 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2453000000000007e-05 2023-11-25T04:24:05.054 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.054 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.054 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:05.055 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040422+0000", 2023-11-25T04:24:05.055 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2429000000000001e-05 2023-11-25T04:24:05.055 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.055 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.055 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:05.055 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042129+0000", 2023-11-25T04:24:05.055 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001707266 2023-11-25T04:24:05.055 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.055 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.055 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:05.055 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042161+0000", 2023-11-25T04:24:05.055 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1791000000000001e-05 2023-11-25T04:24:05.055 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:05.056 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:05.056 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:05.056 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.056 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.056 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:27 3.13 e18/16 3:c91d4a1d:::mds0_inotable:head v 18'2, mlcod=0'0)", 2023-11-25T04:24:05.056 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:24:05.056 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 77.952829453999996, 2023-11-25T04:24:05.056 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0016886430000000001, 2023-11-25T04:24:05.056 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:05.056 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:05.056 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:05.056 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.056 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:05.057 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:24:05.057 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:05.057 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.057 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.057 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:05.057 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:24:05.057 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:05.057 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.057 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.057 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:05.057 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:24:05.057 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3999999999999997e-07 2023-11-25T04:24:05.057 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.058 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.058 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:05.058 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040520+0000", 2023-11-25T04:24:05.058 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2989999999999996e-06 2023-11-25T04:24:05.058 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.058 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.058 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:05.058 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040527+0000", 2023-11-25T04:24:05.058 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5730000000000002e-06 2023-11-25T04:24:05.058 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.058 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.058 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:05.058 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040535+0000", 2023-11-25T04:24:05.059 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8630000000000008e-06 2023-11-25T04:24:05.059 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.059 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.059 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:05.059 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040683+0000", 2023-11-25T04:24:05.059 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000148002 2023-11-25T04:24:05.059 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.059 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.059 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:05.059 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040700+0000", 2023-11-25T04:24:05.059 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7065e-05 2023-11-25T04:24:05.059 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.060 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.060 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:05.060 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042163+0000", 2023-11-25T04:24:05.060 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001462767 2023-11-25T04:24:05.060 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.060 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.060 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:05.060 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042203+0000", 2023-11-25T04:24:05.060 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9734000000000003e-05 2023-11-25T04:24:05.060 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:05.061 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:05.061 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:05.061 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.061 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.061 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:20 3.1c e18/16 3:3ed09add:::607.00000000:head v 18'1, mlcod=0'0)", 2023-11-25T04:24:05.061 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:24:05.061 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 77.952794413000007, 2023-11-25T04:24:05.061 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015271, 2023-11-25T04:24:05.061 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:05.061 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:05.061 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:05.061 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.061 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:05.062 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:24:05.062 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:05.062 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.062 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.062 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:05.062 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:24:05.062 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:05.062 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.062 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.062 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:05.062 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:24:05.062 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4799999999999999e-07 2023-11-25T04:24:05.063 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.063 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.063 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:05.063 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040561+0000", 2023-11-25T04:24:05.063 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1800000000000001e-05 2023-11-25T04:24:05.063 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.063 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.063 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:05.064 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040568+0000", 2023-11-25T04:24:05.064 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7730000000000002e-06 2023-11-25T04:24:05.064 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.064 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.064 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:05.064 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040587+0000", 2023-11-25T04:24:05.064 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8691000000000001e-05 2023-11-25T04:24:05.064 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.064 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.064 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:05.064 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040638+0000", 2023-11-25T04:24:05.064 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1068e-05 2023-11-25T04:24:05.064 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.065 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.065 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:05.065 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040670+0000", 2023-11-25T04:24:05.065 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2125999999999997e-05 2023-11-25T04:24:05.065 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.065 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.065 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:05.065 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042032+0000", 2023-11-25T04:24:05.065 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001361821 2023-11-25T04:24:05.065 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.065 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.065 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:05.065 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042076+0000", 2023-11-25T04:24:05.066 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4472999999999999e-05 2023-11-25T04:24:05.066 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:05.066 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:05.066 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:05.066 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.066 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.066 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:21 3.1c e18/16 3:3ed09add:::607.00000000:head v 18'2, mlcod=0'0)", 2023-11-25T04:24:05.066 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:24:05.066 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 77.952559687999994, 2023-11-25T04:24:05.066 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0013228160000000001, 2023-11-25T04:24:05.066 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:05.066 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:05.066 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:05.067 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.067 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:05.067 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:24:05.067 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:05.067 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.067 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.067 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:05.067 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:24:05.067 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:05.067 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.067 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.067 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:05.068 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:24:05.068 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9299999999999999e-07 2023-11-25T04:24:05.068 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.068 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.068 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:05.068 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040790+0000", 2023-11-25T04:24:05.068 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5770000000000001e-06 2023-11-25T04:24:05.068 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.068 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.068 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:05.068 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040799+0000", 2023-11-25T04:24:05.068 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6439999999999995e-06 2023-11-25T04:24:05.068 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.069 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.069 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:05.069 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040806+0000", 2023-11-25T04:24:05.069 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0180000000000004e-06 2023-11-25T04:24:05.069 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.069 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.069 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:05.069 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040907+0000", 2023-11-25T04:24:05.069 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000100096 2023-11-25T04:24:05.069 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.069 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.069 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:05.069 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040925+0000", 2023-11-25T04:24:05.070 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8536999999999999e-05 2023-11-25T04:24:05.070 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.070 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.070 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:05.070 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042079+0000", 2023-11-25T04:24:05.070 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00115358 2023-11-25T04:24:05.070 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.070 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.070 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:05.070 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042107+0000", 2023-11-25T04:24:05.070 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7971e-05 2023-11-25T04:24:05.070 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:05.070 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:05.071 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:05.071 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.071 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.071 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop_reply(mds.0.3:4 3.1f e18/16 ondisk, result = 0)", 2023-11-25T04:24:05.071 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:24:05.071 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 77.951220297999996, 2023-11-25T04:24:05.071 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00044092900000000001, 2023-11-25T04:24:05.071 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:05.071 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:05.071 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:05.071 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.071 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:05.072 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:24:05.072 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:05.072 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.072 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.072 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:05.072 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:24:05.072 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:05.072 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.072 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.072 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:05.072 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042124+0000", 2023-11-25T04:24:05.072 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1800000000000005e-07 2023-11-25T04:24:05.073 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.073 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.073 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:05.073 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042128+0000", 2023-11-25T04:24:05.073 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2289999999999997e-06 2023-11-25T04:24:05.073 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.073 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.073 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:05.073 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042138+0000", 2023-11-25T04:24:05.073 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9729999999999992e-06 2023-11-25T04:24:05.073 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.073 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.074 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:05.074 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042145+0000", 2023-11-25T04:24:05.074 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.336e-06 2023-11-25T04:24:05.074 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.074 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.074 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:05.074 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042441+0000", 2023-11-25T04:24:05.074 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029650500000000002 2023-11-25T04:24:05.074 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.074 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.074 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:05.074 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042448+0000", 2023-11-25T04:24:05.075 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4100000000000002e-06 2023-11-25T04:24:05.075 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.075 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.075 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:05.075 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042564+0000", 2023-11-25T04:24:05.075 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000115758 2023-11-25T04:24:05.075 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:05.075 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:05.075 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:05.075 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.075 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.075 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing_reply(3.1f epoch 18/16 rep_tid 8 lcod 18'4)", 2023-11-25T04:24:05.075 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:24:05.076 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 77.948894886000005, 2023-11-25T04:24:05.076 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00038154699999999998, 2023-11-25T04:24:05.076 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:05.076 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:24:05.076 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:05.076 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.076 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:05.076 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:24:05.076 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:05.076 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.076 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.076 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:05.076 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:24:05.077 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:05.077 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.077 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.077 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:05.077 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:24:05.077 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2700000000000001e-07 2023-11-25T04:24:05.077 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.077 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.077 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:05.077 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044451+0000", 2023-11-25T04:24:05.077 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7460000000000001e-06 2023-11-25T04:24:05.077 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.077 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.078 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:05.078 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044455+0000", 2023-11-25T04:24:05.078 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5809999999999998e-06 2023-11-25T04:24:05.078 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.078 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.078 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:05.078 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044458+0000", 2023-11-25T04:24:05.078 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0170000000000001e-06 2023-11-25T04:24:05.078 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.078 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.078 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:05.078 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044734+0000", 2023-11-25T04:24:05.078 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00027582799999999999 2023-11-25T04:24:05.079 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.079 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.079 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:05.079 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044830+0000", 2023-11-25T04:24:05.079 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6248000000000004e-05 2023-11-25T04:24:05.079 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:05.079 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:05.079 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:05.079 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.079 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.079 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing_reply(3.b epoch 18/16 rep_tid 5 lcod 18'2)", 2023-11-25T04:24:05.079 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:24:05.079 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 77.948858224999995, 2023-11-25T04:24:05.080 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000344488, 2023-11-25T04:24:05.080 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:05.080 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:24:05.080 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:05.080 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.080 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:05.080 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:24:05.080 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:05.080 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.080 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.080 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:05.080 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:24:05.080 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:05.081 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.081 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.081 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:05.081 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:24:05.081 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.55e-07 2023-11-25T04:24:05.081 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.081 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.081 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:05.081 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044487+0000", 2023-11-25T04:24:05.081 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2470000000000001e-06 2023-11-25T04:24:05.081 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.081 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.082 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:05.082 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044491+0000", 2023-11-25T04:24:05.082 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9659999999999998e-06 2023-11-25T04:24:05.082 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.082 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.082 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:05.082 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044494+0000", 2023-11-25T04:24:05.082 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2310000000000001e-06 2023-11-25T04:24:05.082 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.082 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.083 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:05.083 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044756+0000", 2023-11-25T04:24:05.083 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00026172800000000003 2023-11-25T04:24:05.083 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.083 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.083 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:05.083 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044830+0000", 2023-11-25T04:24:05.083 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4160999999999997e-05 2023-11-25T04:24:05.083 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:05.083 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:05.083 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:05.083 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.083 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.084 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:39 3.6 3:654134d2:::mds0_openfiles.0:head [omap-clear,omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2023-11-25T04:24:05.084 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:24:05.084 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 72.889032182999998, 2023-11-25T04:24:05.084 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001788658, 2023-11-25T04:24:05.084 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:05.084 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:05.084 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:05.084 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.084 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:05.084 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:24:05.084 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:05.084 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.085 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.085 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:05.085 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:24:05.085 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:05.085 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.085 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.085 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:05.085 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104314+0000", 2023-11-25T04:24:05.085 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3099999999999999e-06 2023-11-25T04:24:05.085 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.085 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.085 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:05.085 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104325+0000", 2023-11-25T04:24:05.086 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1511999999999999e-05 2023-11-25T04:24:05.086 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.086 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.086 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:05.086 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104362+0000", 2023-11-25T04:24:05.086 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6903000000000002e-05 2023-11-25T04:24:05.086 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.086 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.086 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:05.086 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104387+0000", 2023-11-25T04:24:05.086 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4766e-05 2023-11-25T04:24:05.086 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.086 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.087 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:05.087 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104465+0000", 2023-11-25T04:24:05.087 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8119000000000002e-05 2023-11-25T04:24:05.087 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.087 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.087 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:05.087 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104557+0000", 2023-11-25T04:24:05.087 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2496000000000003e-05 2023-11-25T04:24:05.087 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.087 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.087 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:05.087 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104646+0000", 2023-11-25T04:24:05.087 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8818000000000002e-05 2023-11-25T04:24:05.088 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.088 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.088 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:05.088 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105201+0000", 2023-11-25T04:24:05.088 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00055506000000000002 2023-11-25T04:24:05.088 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.088 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.088 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:05.088 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105994+0000", 2023-11-25T04:24:05.088 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00079294700000000001 2023-11-25T04:24:05.088 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.088 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.088 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:05.089 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.106055+0000", 2023-11-25T04:24:05.089 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0566000000000003e-05 2023-11-25T04:24:05.089 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.089 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.089 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:05.089 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.106100+0000", 2023-11-25T04:24:05.089 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5160999999999997e-05 2023-11-25T04:24:05.089 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:05.089 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:05.089 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:05.089 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.089 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.090 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:43 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2023-11-25T04:24:05.090 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:24:05.090 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 57.888807186999998, 2023-11-25T04:24:05.090 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001751166, 2023-11-25T04:24:05.090 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:05.090 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:05.090 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:05.090 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.090 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:05.090 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:24:05.090 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:05.090 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.090 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.091 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:05.091 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:24:05.091 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:05.091 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.091 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.091 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:05.091 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104539+0000", 2023-11-25T04:24:05.091 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9529999999999999e-06 2023-11-25T04:24:05.091 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.091 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.091 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:05.091 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104557+0000", 2023-11-25T04:24:05.091 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7464000000000001e-05 2023-11-25T04:24:05.092 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.092 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.092 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:05.092 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104590+0000", 2023-11-25T04:24:05.092 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2966000000000003e-05 2023-11-25T04:24:05.092 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.092 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.092 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:05.092 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104608+0000", 2023-11-25T04:24:05.092 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8193e-05 2023-11-25T04:24:05.092 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.092 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.092 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:05.093 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104767+0000", 2023-11-25T04:24:05.093 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015882 2023-11-25T04:24:05.093 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.093 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.093 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:05.093 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104909+0000", 2023-11-25T04:24:05.093 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014244999999999999 2023-11-25T04:24:05.093 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.093 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.093 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:05.093 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.105059+0000", 2023-11-25T04:24:05.093 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014964399999999999 2023-11-25T04:24:05.093 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.093 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.094 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:05.094 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.105701+0000", 2023-11-25T04:24:05.094 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00064201099999999995 2023-11-25T04:24:05.094 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.094 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.094 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:05.094 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106233+0000", 2023-11-25T04:24:05.094 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00053178500000000001 2023-11-25T04:24:05.094 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.094 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.094 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:05.094 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106265+0000", 2023-11-25T04:24:05.094 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1986000000000002e-05 2023-11-25T04:24:05.095 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.095 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.095 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:05.095 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106288+0000", 2023-11-25T04:24:05.095 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2894000000000001e-05 2023-11-25T04:24:05.095 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:05.095 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:05.095 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:05.095 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.095 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.095 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:45 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e23)", 2023-11-25T04:24:05.095 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:24:05.095 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 52.888609146999997, 2023-11-25T04:24:05.096 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00098423299999999994, 2023-11-25T04:24:05.096 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:05.096 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:05.096 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:05.096 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.096 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:05.096 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:24:05.096 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:05.096 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.096 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.096 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:05.098 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:24:05.098 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:05.098 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.098 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.098 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:05.098 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104737+0000", 2023-11-25T04:24:05.098 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3309999999999998e-06 2023-11-25T04:24:05.098 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.099 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.099 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:05.099 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104748+0000", 2023-11-25T04:24:05.099 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1080999999999999e-05 2023-11-25T04:24:05.099 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.099 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.099 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:05.099 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104771+0000", 2023-11-25T04:24:05.099 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2707000000000001e-05 2023-11-25T04:24:05.099 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.100 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.100 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:05.100 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104800+0000", 2023-11-25T04:24:05.100 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9595e-05 2023-11-25T04:24:05.100 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.100 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.100 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:05.100 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104890+0000", 2023-11-25T04:24:05.101 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.9808999999999993e-05 2023-11-25T04:24:05.101 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.101 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.101 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:05.101 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105004+0000", 2023-11-25T04:24:05.101 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000114508 2023-11-25T04:24:05.101 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.101 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.101 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:05.102 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105634+0000", 2023-11-25T04:24:05.102 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00062985799999999996 2023-11-25T04:24:05.102 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.102 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.102 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:05.102 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105671+0000", 2023-11-25T04:24:05.102 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6640000000000002e-05 2023-11-25T04:24:05.103 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:05.103 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:05.103 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:05.103 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105719+0000", 2023-11-25T04:24:05.103 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7704000000000003e-05 2023-11-25T04:24:05.103 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:05.103 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:05.103 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:05.103 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:05.103 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:05.103 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:05.106 INFO:teuthology.orchestra.run.smithi067.stderr:noscrub is set 2023-11-25T04:24:05.107 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:05.103+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:05.107 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:24:05.149 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-11-25T04:24:05.206 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:05.203+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:05.216 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:05.216 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:24:05.216 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2023-11-25T04:24:05.216 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:05.226 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:24:05.264 INFO:teuthology.orchestra.run.smithi067.stderr:Error ENXIO: problem getting command descriptions from osd.0 2023-11-25T04:24:05.283 INFO:teuthology.orchestra.run.smithi067.stdout:osd.1: {} 2023-11-25T04:24:05.284 INFO:teuthology.orchestra.run.smithi067.stderr:osd.1: osd_enable_op_tracker = '' 2023-11-25T04:24:05.301 INFO:teuthology.orchestra.run.smithi067.stdout:osd.2: {} 2023-11-25T04:24:05.301 INFO:teuthology.orchestra.run.smithi067.stderr:osd.2: osd_enable_op_tracker = '' 2023-11-25T04:24:05.304 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-11-25T04:24:05.304 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-11-25T04:24:05.310 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:05.308+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:05.371 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:05.371 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:05.380 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:24:05.420 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:05.418+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:05.508 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:05.506+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:05.525 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:05.525 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:05.534 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:24:05.609 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:05.608+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:05.642 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:05.642 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:05.652 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:24:05.711 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:05.709+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:05.756 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:05.757 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:05.765 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:24:05.850 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:05.848+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:05.911 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:05.912 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:05.913 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:05.911+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:05.920 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:24:06.014 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:06.012+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:06.034 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:06.034 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:06.042 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:24:06.114 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:06.112+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:06.147 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:06.147 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:06.156 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:24:06.215 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:06.214+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:06.301 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:06.301 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:06.310 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:24:06.316 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:06.313+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:06.418 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:06.418 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:06.418 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:06.416+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:06.426 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:24:06.530 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:06.528+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:06.530 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:06.531 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:06.540 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:24:06.636 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:06.630+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:06.646 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:06.646 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:06.655 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:24:06.733 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:06.730+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:06.762 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:06.762 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:06.771 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:24:06.855 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:06.853+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:06.916 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:06.916 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:06.924 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:24:06.965 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:06.963+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:07.037 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:07.034+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:07.071 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:07.071 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:07.080 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:24:07.108 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2023-11-25T04:24:07.136 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:07.135+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:07.187 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:07.187 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:07.196 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:24:07.237 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:07.235+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:07.302 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:07.303 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:07.305 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-11-25T04:24:07.311 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:24:07.338 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:07.337+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:07.416 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:07.416 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:07.417 INFO:teuthology.orchestra.run.smithi067.stderr:Error ENXIO: problem getting command descriptions from osd.0 2023-11-25T04:24:07.426 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:24:07.435 INFO:teuthology.orchestra.run.smithi067.stdout:osd.1: {} 2023-11-25T04:24:07.436 INFO:teuthology.orchestra.run.smithi067.stderr:osd.1: osd_enable_op_tracker = '' 2023-11-25T04:24:07.439 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:07.437+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:07.454 INFO:teuthology.orchestra.run.smithi067.stdout:osd.2: {} 2023-11-25T04:24:07.455 INFO:teuthology.orchestra.run.smithi067.stderr:osd.2: osd_enable_op_tracker = '' 2023-11-25T04:24:07.457 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-11-25T04:24:07.457 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-11-25T04:24:07.517 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~59s 2023-11-25T04:24:07.532 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:07.533 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:24:07.533 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2023-11-25T04:24:07.533 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:07.540 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:07.539+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:07.541 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:24:07.641 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:07.638+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:07.646 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:07.646 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:24:07.646 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2023-11-25T04:24:07.646 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2023-11-25T04:24:07.646 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:07.655 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:24:07.742 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:07.741+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:07.801 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:07.801 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2023-11-25T04:24:07.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2023-11-25T04:24:07.801 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2023-11-25T04:24:07.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.802 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:155 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'109, mlcod=11'105)", 2023-11-25T04:24:07.802 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:24:07.802 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 90.013955425999995, 2023-11-25T04:24:07.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061719019999999999, 2023-11-25T04:24:07.802 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:07.802 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:07.802 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:07.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:07.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:24:07.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:07.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:07.803 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:24:07.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:07.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:07.803 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784884+0000", 2023-11-25T04:24:07.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3099999999999999e-07 2023-11-25T04:24:07.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:07.803 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784942+0000", 2023-11-25T04:24:07.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8570000000000003e-05 2023-11-25T04:24:07.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.804 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:07.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784949+0000", 2023-11-25T04:24:07.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0310000000000001e-06 2023-11-25T04:24:07.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.804 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:07.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784953+0000", 2023-11-25T04:24:07.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6320000000000001e-06 2023-11-25T04:24:07.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:07.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784993+0000", 2023-11-25T04:24:07.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9737000000000003e-05 2023-11-25T04:24:07.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:07.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785011+0000", 2023-11-25T04:24:07.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8349999999999999e-05 2023-11-25T04:24:07.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:07.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791024+0000", 2023-11-25T04:24:07.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006012997 2023-11-25T04:24:07.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:07.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791055+0000", 2023-11-25T04:24:07.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1254000000000003e-05 2023-11-25T04:24:07.806 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:07.806 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:07.806 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:07.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.807 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:156 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'110, mlcod=11'105)", 2023-11-25T04:24:07.807 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:24:07.807 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 90.013369377000004, 2023-11-25T04:24:07.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005616116, 2023-11-25T04:24:07.807 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:07.807 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:07.807 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:07.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:07.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:24:07.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:07.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:07.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:24:07.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:07.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:07.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785470+0000", 2023-11-25T04:24:07.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0699999999999998e-07 2023-11-25T04:24:07.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:07.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785474+0000", 2023-11-25T04:24:07.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.048e-06 2023-11-25T04:24:07.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:07.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785480+0000", 2023-11-25T04:24:07.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7729999999999996e-06 2023-11-25T04:24:07.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:07.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785483+0000", 2023-11-25T04:24:07.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1970000000000001e-06 2023-11-25T04:24:07.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:07.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787936+0000", 2023-11-25T04:24:07.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024529040000000001 2023-11-25T04:24:07.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:07.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787947+0000", 2023-11-25T04:24:07.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1669e-05 2023-11-25T04:24:07.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:07.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791057+0000", 2023-11-25T04:24:07.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031095200000000002 2023-11-25T04:24:07.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.811 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:07.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791086+0000", 2023-11-25T04:24:07.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8697999999999999e-05 2023-11-25T04:24:07.811 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:07.811 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:07.811 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:07.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.811 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:157 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'111, mlcod=11'106)", 2023-11-25T04:24:07.812 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:24:07.812 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 90.012776818999995, 2023-11-25T04:24:07.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050590280000000001, 2023-11-25T04:24:07.812 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:07.812 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:07.812 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:07.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.812 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:07.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:24:07.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:07.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.812 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:07.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:24:07.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:07.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:07.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786063+0000", 2023-11-25T04:24:07.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2600000000000002e-07 2023-11-25T04:24:07.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:07.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786069+0000", 2023-11-25T04:24:07.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8839999999999999e-06 2023-11-25T04:24:07.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:07.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786079+0000", 2023-11-25T04:24:07.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0124e-05 2023-11-25T04:24:07.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:07.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786085+0000", 2023-11-25T04:24:07.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6660000000000002e-06 2023-11-25T04:24:07.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:07.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788160+0000", 2023-11-25T04:24:07.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020747669999999999 2023-11-25T04:24:07.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:07.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788180+0000", 2023-11-25T04:24:07.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0208999999999999e-05 2023-11-25T04:24:07.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:07.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791087+0000", 2023-11-25T04:24:07.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029064630000000002 2023-11-25T04:24:07.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:07.816 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791121+0000", 2023-11-25T04:24:07.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4289000000000003e-05 2023-11-25T04:24:07.816 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:07.816 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:07.816 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:07.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.816 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:158 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'112, mlcod=11'106)", 2023-11-25T04:24:07.816 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:24:07.817 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 90.012360426000001, 2023-11-25T04:24:07.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046663030000000001, 2023-11-25T04:24:07.817 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:07.817 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:07.817 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:07.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:07.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:24:07.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:07.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:07.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:24:07.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:07.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:07.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786479+0000", 2023-11-25T04:24:07.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2799999999999996e-07 2023-11-25T04:24:07.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:07.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786498+0000", 2023-11-25T04:24:07.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9069e-05 2023-11-25T04:24:07.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:07.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786508+0000", 2023-11-25T04:24:07.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5810000000000003e-06 2023-11-25T04:24:07.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:07.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786512+0000", 2023-11-25T04:24:07.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4889999999999997e-06 2023-11-25T04:24:07.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:07.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788438+0000", 2023-11-25T04:24:07.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019257059999999999 2023-11-25T04:24:07.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:07.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788449+0000", 2023-11-25T04:24:07.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1419e-05 2023-11-25T04:24:07.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:07.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791122+0000", 2023-11-25T04:24:07.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002672822 2023-11-25T04:24:07.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:07.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791145+0000", 2023-11-25T04:24:07.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2589e-05 2023-11-25T04:24:07.821 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:07.821 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:07.821 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:07.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.821 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:206 1.0 e12/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 12'152, mlcod=12'148)", 2023-11-25T04:24:07.821 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:24:07.821 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 89.500591955999994, 2023-11-25T04:24:07.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046728189999999999, 2023-11-25T04:24:07.822 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:07.822 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:07.822 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:07.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:07.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:24:07.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:07.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:07.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:24:07.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:07.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.823 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:07.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298248+0000", 2023-11-25T04:24:07.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3600000000000003e-07 2023-11-25T04:24:07.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.823 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:07.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298278+0000", 2023-11-25T04:24:07.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0474e-05 2023-11-25T04:24:07.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:07.824 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298288+0000", 2023-11-25T04:24:07.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0156000000000001e-05 2023-11-25T04:24:07.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:07.824 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298293+0000", 2023-11-25T04:24:07.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7400000000000004e-06 2023-11-25T04:24:07.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:07.824 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298333+0000", 2023-11-25T04:24:07.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9666000000000003e-05 2023-11-25T04:24:07.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:07.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298351+0000", 2023-11-25T04:24:07.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8544999999999999e-05 2023-11-25T04:24:07.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:07.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302891+0000", 2023-11-25T04:24:07.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045398519999999996 2023-11-25T04:24:07.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:07.826 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302920+0000", 2023-11-25T04:24:07.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8750000000000001e-05 2023-11-25T04:24:07.826 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:07.826 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:07.826 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:07.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.826 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:14 3.15 3:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:07.826 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:24:07.826 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 80.765457599000001, 2023-11-25T04:24:07.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010990437, 2023-11-25T04:24:07.827 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:07.827 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:07.827 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:07.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:07.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:24:07.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:07.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:07.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:24:07.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:07.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:07.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033383+0000", 2023-11-25T04:24:07.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9870000000000002e-06 2023-11-25T04:24:07.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:07.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033398+0000", 2023-11-25T04:24:07.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4569999999999999e-05 2023-11-25T04:24:07.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:07.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033411+0000", 2023-11-25T04:24:07.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3543999999999999e-05 2023-11-25T04:24:07.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:07.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033423+0000", 2023-11-25T04:24:07.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1905999999999999e-05 2023-11-25T04:24:07.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:07.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034433+0000", 2023-11-25T04:24:07.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00101022 2023-11-25T04:24:07.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:07.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034539+0000", 2023-11-25T04:24:07.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010597199999999999 2023-11-25T04:24:07.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:07.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034667+0000", 2023-11-25T04:24:07.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012765500000000001 2023-11-25T04:24:07.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:07.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037803+0000", 2023-11-25T04:24:07.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031362220000000001 2023-11-25T04:24:07.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:07.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:24:07.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0064964669999999997 2023-11-25T04:24:07.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.832 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:07.832 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044331+0000", 2023-11-25T04:24:07.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1544999999999997e-05 2023-11-25T04:24:07.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.832 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:07.832 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044372+0000", 2023-11-25T04:24:07.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0349000000000003e-05 2023-11-25T04:24:07.832 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:07.832 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:07.832 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:07.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.833 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:15 3.15 3:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:07.833 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:24:07.833 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 80.764797948999998, 2023-11-25T04:24:07.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010464362, 2023-11-25T04:24:07.833 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:07.833 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:07.833 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:07.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:07.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:24:07.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:07.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:07.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:24:07.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:07.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:07.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034042+0000", 2023-11-25T04:24:07.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0999999999999997e-07 2023-11-25T04:24:07.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:07.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034048+0000", 2023-11-25T04:24:07.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0390000000000001e-06 2023-11-25T04:24:07.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.835 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:07.835 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034059+0000", 2023-11-25T04:24:07.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1182999999999999e-05 2023-11-25T04:24:07.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.835 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:07.835 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034067+0000", 2023-11-25T04:24:07.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1559999999999995e-06 2023-11-25T04:24:07.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:07.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035005+0000", 2023-11-25T04:24:07.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00093747399999999999 2023-11-25T04:24:07.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:07.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035077+0000", 2023-11-25T04:24:07.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2747e-05 2023-11-25T04:24:07.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:07.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035219+0000", 2023-11-25T04:24:07.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000141877 2023-11-25T04:24:07.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.837 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:07.837 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037810+0000", 2023-11-25T04:24:07.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002590752 2023-11-25T04:24:07.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.837 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:07.837 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044437+0000", 2023-11-25T04:24:07.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066270110000000004 2023-11-25T04:24:07.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:07.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044481+0000", 2023-11-25T04:24:07.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4106000000000001e-05 2023-11-25T04:24:07.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:07.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044505+0000", 2023-11-25T04:24:07.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4207e-05 2023-11-25T04:24:07.838 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:07.838 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:07.838 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:07.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.839 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:16 3.1d 3:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:07.839 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:24:07.839 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 80.764756101000003, 2023-11-25T04:24:07.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010413923, 2023-11-25T04:24:07.839 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:07.839 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:07.839 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:07.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:07.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:24:07.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:07.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:07.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:24:07.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:07.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:07.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:24:07.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4499999999999997e-07 2023-11-25T04:24:07.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:07.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034087+0000", 2023-11-25T04:24:07.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9770000000000002e-06 2023-11-25T04:24:07.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:07.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034094+0000", 2023-11-25T04:24:07.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8809999999999998e-06 2023-11-25T04:24:07.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:07.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034100+0000", 2023-11-25T04:24:07.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8440000000000003e-06 2023-11-25T04:24:07.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:07.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034993+0000", 2023-11-25T04:24:07.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00089336799999999996 2023-11-25T04:24:07.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:07.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035099+0000", 2023-11-25T04:24:07.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106037 2023-11-25T04:24:07.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:07.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035276+0000", 2023-11-25T04:24:07.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017631799999999999 2023-11-25T04:24:07.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:07.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037815+0000", 2023-11-25T04:24:07.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025395959999999999 2023-11-25T04:24:07.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:07.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044420+0000", 2023-11-25T04:24:07.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066045749999999997 2023-11-25T04:24:07.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:07.844 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044463+0000", 2023-11-25T04:24:07.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3205999999999999e-05 2023-11-25T04:24:07.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.844 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:07.844 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044497+0000", 2023-11-25T04:24:07.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3476000000000003e-05 2023-11-25T04:24:07.844 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:07.844 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:07.844 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:07.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.844 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:17 3.1d 3:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:07.845 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:24:07.845 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 80.764730892000003, 2023-11-25T04:24:07.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01127356, 2023-11-25T04:24:07.845 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:07.845 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:07.845 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:07.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.845 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:07.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:24:07.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:07.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.845 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:07.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:24:07.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:07.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:07.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:24:07.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1100000000000001e-07 2023-11-25T04:24:07.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:07.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034112+0000", 2023-11-25T04:24:07.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8600000000000003e-06 2023-11-25T04:24:07.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:07.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034120+0000", 2023-11-25T04:24:07.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7700000000000001e-06 2023-11-25T04:24:07.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:07.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034127+0000", 2023-11-25T04:24:07.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0999999999999998e-06 2023-11-25T04:24:07.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.848 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:07.848 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035601+0000", 2023-11-25T04:24:07.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014739410000000001 2023-11-25T04:24:07.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.848 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:07.848 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035685+0000", 2023-11-25T04:24:07.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3677999999999994e-05 2023-11-25T04:24:07.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.848 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:07.848 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035827+0000", 2023-11-25T04:24:07.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014232599999999999 2023-11-25T04:24:07.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:07.849 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038725+0000", 2023-11-25T04:24:07.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028982779999999998 2023-11-25T04:24:07.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:07.849 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045323+0000", 2023-11-25T04:24:07.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0065980780000000003 2023-11-25T04:24:07.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:07.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045354+0000", 2023-11-25T04:24:07.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0463e-05 2023-11-25T04:24:07.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.850 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:07.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045381+0000", 2023-11-25T04:24:07.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7654999999999999e-05 2023-11-25T04:24:07.850 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:07.850 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:07.850 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:07.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.850 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:22 3.e 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:07.851 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:24:07.851 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 80.764704479000002, 2023-11-25T04:24:07.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010133422, 2023-11-25T04:24:07.851 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:07.851 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:07.851 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:07.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.851 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:07.851 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:24:07.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:07.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:07.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:24:07.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:07.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:07.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034135+0000", 2023-11-25T04:24:07.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-07 2023-11-25T04:24:07.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:07.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034138+0000", 2023-11-25T04:24:07.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6009999999999999e-06 2023-11-25T04:24:07.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:07.853 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034145+0000", 2023-11-25T04:24:07.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1169999999999998e-06 2023-11-25T04:24:07.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:07.853 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034150+0000", 2023-11-25T04:24:07.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9080000000000003e-06 2023-11-25T04:24:07.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:07.854 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034209+0000", 2023-11-25T04:24:07.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8149000000000002e-05 2023-11-25T04:24:07.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.854 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:07.854 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034415+0000", 2023-11-25T04:24:07.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020637699999999999 2023-11-25T04:24:07.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.854 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:07.854 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034584+0000", 2023-11-25T04:24:07.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016870399999999999 2023-11-25T04:24:07.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.855 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:07.855 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.036979+0000", 2023-11-25T04:24:07.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023957509999999998 2023-11-25T04:24:07.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.855 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:07.855 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044161+0000", 2023-11-25T04:24:07.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0071811590000000003 2023-11-25T04:24:07.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.855 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:07.855 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044224+0000", 2023-11-25T04:24:07.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3454000000000006e-05 2023-11-25T04:24:07.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.856 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.856 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:07.856 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044268+0000", 2023-11-25T04:24:07.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3761999999999999e-05 2023-11-25T04:24:07.856 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:07.856 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:07.856 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:07.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.856 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.856 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:23 3.e 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:07.856 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:24:07.857 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 80.764673070000001, 2023-11-25T04:24:07.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010281676, 2023-11-25T04:24:07.857 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:07.857 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:07.857 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:07.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.857 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:07.857 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:24:07.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:07.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.857 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:07.858 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:24:07.858 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:07.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.858 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.858 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:07.858 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:24:07.858 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5799999999999999e-07 2023-11-25T04:24:07.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.858 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.858 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:07.858 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034169+0000", 2023-11-25T04:24:07.858 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1879999999999998e-06 2023-11-25T04:24:07.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.859 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.859 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:07.859 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034177+0000", 2023-11-25T04:24:07.859 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7020000000000001e-06 2023-11-25T04:24:07.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.859 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.859 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:07.859 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034184+0000", 2023-11-25T04:24:07.859 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3910000000000003e-06 2023-11-25T04:24:07.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.859 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.859 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:07.859 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034924+0000", 2023-11-25T04:24:07.860 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00074032999999999998 2023-11-25T04:24:07.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.860 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.860 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:07.860 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034991+0000", 2023-11-25T04:24:07.860 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7347000000000004e-05 2023-11-25T04:24:07.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.860 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.860 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:07.860 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035096+0000", 2023-11-25T04:24:07.860 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010488 2023-11-25T04:24:07.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.860 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.861 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:07.861 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037588+0000", 2023-11-25T04:24:07.861 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024918309999999999 2023-11-25T04:24:07.861 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.861 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.861 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:07.861 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044367+0000", 2023-11-25T04:24:07.861 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067787589999999997 2023-11-25T04:24:07.861 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.861 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.861 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:07.861 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044415+0000", 2023-11-25T04:24:07.861 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7803000000000003e-05 2023-11-25T04:24:07.862 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.862 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.862 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:07.862 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044447+0000", 2023-11-25T04:24:07.862 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2886999999999999e-05 2023-11-25T04:24:07.862 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:07.862 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:07.862 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:07.862 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.862 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.862 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:30 3.15 3:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:07.862 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:24:07.863 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 80.764614629999997, 2023-11-25T04:24:07.863 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010365070000000001, 2023-11-25T04:24:07.863 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:07.863 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:07.863 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:07.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.863 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:07.863 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:24:07.863 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:07.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.863 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:07.863 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:24:07.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:07.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.864 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.864 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:07.864 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034225+0000", 2023-11-25T04:24:07.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7400000000000002e-07 2023-11-25T04:24:07.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.864 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.864 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:07.864 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034229+0000", 2023-11-25T04:24:07.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3100000000000002e-06 2023-11-25T04:24:07.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.865 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.865 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:07.865 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034239+0000", 2023-11-25T04:24:07.865 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9140000000000003e-06 2023-11-25T04:24:07.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.865 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.865 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:07.865 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034246+0000", 2023-11-25T04:24:07.865 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0149999999999998e-06 2023-11-25T04:24:07.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.865 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.865 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:07.865 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035572+0000", 2023-11-25T04:24:07.866 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001325655 2023-11-25T04:24:07.866 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.866 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.866 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:07.866 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035730+0000", 2023-11-25T04:24:07.866 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015827899999999999 2023-11-25T04:24:07.866 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.866 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.866 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:07.866 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035857+0000", 2023-11-25T04:24:07.866 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012644699999999999 2023-11-25T04:24:07.866 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.867 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.867 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:07.867 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038737+0000", 2023-11-25T04:24:07.867 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002880077 2023-11-25T04:24:07.867 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.867 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.867 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:07.867 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044540+0000", 2023-11-25T04:24:07.867 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058034239999999997 2023-11-25T04:24:07.867 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.867 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.867 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:07.867 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044569+0000", 2023-11-25T04:24:07.868 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8388999999999999e-05 2023-11-25T04:24:07.868 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.868 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.868 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:07.868 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044589+0000", 2023-11-25T04:24:07.868 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0686000000000002e-05 2023-11-25T04:24:07.868 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:07.868 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:07.868 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:07.868 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.868 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.868 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:33 3.1b 3:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:07.868 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:24:07.869 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 80.764569132000005, 2023-11-25T04:24:07.869 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099754049999999997, 2023-11-25T04:24:07.869 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:07.869 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:07.869 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:07.869 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.869 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:07.869 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:24:07.869 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:07.869 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.869 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.869 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:07.870 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:24:07.870 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:07.870 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.870 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.870 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:07.870 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:24:07.870 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4799999999999998e-07 2023-11-25T04:24:07.870 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.870 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.870 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:07.870 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034273+0000", 2023-11-25T04:24:07.870 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7819999999999998e-06 2023-11-25T04:24:07.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.871 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.871 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:07.871 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034285+0000", 2023-11-25T04:24:07.871 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2058e-05 2023-11-25T04:24:07.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.871 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.871 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:07.871 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034291+0000", 2023-11-25T04:24:07.871 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0680000000000002e-06 2023-11-25T04:24:07.871 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.871 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.871 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:07.871 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034370+0000", 2023-11-25T04:24:07.872 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8452e-05 2023-11-25T04:24:07.872 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.872 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.872 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:07.872 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034477+0000", 2023-11-25T04:24:07.872 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107471 2023-11-25T04:24:07.872 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.872 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.872 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:07.872 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044245+0000", 2023-11-25T04:24:07.872 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0097679259999999997 2023-11-25T04:24:07.872 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:07.872 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:07.873 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:07.873 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.873 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.873 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:36 3.1a 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:07.873 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:24:07.873 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 80.764522459000005, 2023-11-25T04:24:07.873 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058179099999999999, 2023-11-25T04:24:07.873 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:07.873 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:07.873 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:07.873 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.873 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:07.873 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:24:07.874 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:07.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.874 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:07.874 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:24:07.874 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:07.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.874 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:07.874 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034317+0000", 2023-11-25T04:24:07.874 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1999999999999999e-07 2023-11-25T04:24:07.874 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.874 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.875 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:07.875 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034321+0000", 2023-11-25T04:24:07.875 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.233e-06 2023-11-25T04:24:07.875 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.875 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.875 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:07.875 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034337+0000", 2023-11-25T04:24:07.875 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5308999999999999e-05 2023-11-25T04:24:07.875 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.875 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.875 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:07.875 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034354+0000", 2023-11-25T04:24:07.875 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7411999999999999e-05 2023-11-25T04:24:07.876 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.876 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.876 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:07.876 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034465+0000", 2023-11-25T04:24:07.876 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011154699999999999 2023-11-25T04:24:07.876 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.876 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.876 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:07.876 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034560+0000", 2023-11-25T04:24:07.876 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4186999999999998e-05 2023-11-25T04:24:07.876 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.876 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.876 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:07.877 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.040134+0000", 2023-11-25T04:24:07.877 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055746019999999997 2023-11-25T04:24:07.877 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:07.877 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:07.877 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:07.877 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.877 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.877 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:9 3.10 e18/16 3:09401a81:::601.00000000:head v 18'4, mlcod=0'0)", 2023-11-25T04:24:07.877 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:24:07.877 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 80.755628150999996, 2023-11-25T04:24:07.877 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061641020000000003, 2023-11-25T04:24:07.877 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:07.878 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:07.878 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:07.878 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.878 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:07.878 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:24:07.878 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:07.878 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.878 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.878 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:07.878 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:24:07.878 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:07.878 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.878 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.879 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:07.879 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:24:07.879 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5300000000000001e-07 2023-11-25T04:24:07.879 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.879 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.879 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:07.879 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043222+0000", 2023-11-25T04:24:07.879 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.082e-05 2023-11-25T04:24:07.879 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.879 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.879 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:07.879 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043228+0000", 2023-11-25T04:24:07.879 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6699999999999997e-06 2023-11-25T04:24:07.880 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.880 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.880 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:07.880 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043236+0000", 2023-11-25T04:24:07.880 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8189999999999992e-06 2023-11-25T04:24:07.880 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.880 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.880 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:07.880 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043507+0000", 2023-11-25T04:24:07.880 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000271012 2023-11-25T04:24:07.880 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.880 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.880 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:07.881 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043530+0000", 2023-11-25T04:24:07.881 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2659000000000001e-05 2023-11-25T04:24:07.881 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.881 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.881 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:07.881 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049321+0000", 2023-11-25T04:24:07.881 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057915129999999999 2023-11-25T04:24:07.881 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.881 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.881 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:07.881 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049375+0000", 2023-11-25T04:24:07.881 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3455999999999998e-05 2023-11-25T04:24:07.881 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:07.882 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:07.882 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:07.882 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.882 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.882 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:1 3.14 e18/16 3:292cf221:::200.00000000:head v 18'3, mlcod=0'0)", 2023-11-25T04:24:07.882 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:24:07.882 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 80.755591437999996, 2023-11-25T04:24:07.882 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0060983599999999997, 2023-11-25T04:24:07.882 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:07.882 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:07.882 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:07.882 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.882 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:07.883 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:24:07.883 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:07.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.883 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.883 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:07.883 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:24:07.883 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:07.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.883 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.883 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:07.883 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043248+0000", 2023-11-25T04:24:07.883 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9399999999999995e-07 2023-11-25T04:24:07.883 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.884 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.884 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:07.884 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043252+0000", 2023-11-25T04:24:07.884 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1799999999999998e-06 2023-11-25T04:24:07.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.884 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.884 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:07.884 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043259+0000", 2023-11-25T04:24:07.884 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0049999999999998e-06 2023-11-25T04:24:07.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.884 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.884 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:07.884 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043265+0000", 2023-11-25T04:24:07.885 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0859999999999998e-06 2023-11-25T04:24:07.885 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.885 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.885 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:07.885 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043311+0000", 2023-11-25T04:24:07.885 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6082999999999999e-05 2023-11-25T04:24:07.885 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.885 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.885 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:07.885 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043330+0000", 2023-11-25T04:24:07.885 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8556e-05 2023-11-25T04:24:07.885 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.886 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.886 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:07.886 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049303+0000", 2023-11-25T04:24:07.886 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059733470000000004 2023-11-25T04:24:07.886 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.886 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.886 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:07.886 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049346+0000", 2023-11-25T04:24:07.886 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2608999999999998e-05 2023-11-25T04:24:07.886 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:07.886 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:07.886 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:07.886 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.887 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.887 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:3 3.14 e18/16 3:287034c4:::1.00000000.inode:head v 18'4, mlcod=0'0)", 2023-11-25T04:24:07.887 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:24:07.887 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 80.755074931999999, 2023-11-25T04:24:07.887 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057040090000000003, 2023-11-25T04:24:07.887 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:07.887 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:07.887 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:07.887 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.887 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:07.887 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:24:07.887 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:07.887 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.888 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.888 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:07.888 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:24:07.888 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:07.888 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.888 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.888 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:07.888 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:24:07.888 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9799999999999999e-07 2023-11-25T04:24:07.888 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.888 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.888 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:07.888 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043785+0000", 2023-11-25T04:24:07.889 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0721999999999999e-05 2023-11-25T04:24:07.889 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.889 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.889 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:07.889 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043793+0000", 2023-11-25T04:24:07.889 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5839999999999995e-06 2023-11-25T04:24:07.889 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.889 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.889 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:07.889 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043803+0000", 2023-11-25T04:24:07.889 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0729999999999994e-06 2023-11-25T04:24:07.889 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.889 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.890 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:07.890 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043843+0000", 2023-11-25T04:24:07.890 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0747999999999997e-05 2023-11-25T04:24:07.890 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.890 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.890 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:07.890 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043856+0000", 2023-11-25T04:24:07.890 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2697000000000001e-05 2023-11-25T04:24:07.890 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.890 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.890 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:07.890 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049426+0000", 2023-11-25T04:24:07.890 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055704409999999998 2023-11-25T04:24:07.891 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.891 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.891 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:07.891 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049468+0000", 2023-11-25T04:24:07.891 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1446000000000003e-05 2023-11-25T04:24:07.891 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:07.891 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:07.891 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:07.891 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.891 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.891 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "pg_update_log_missing(3.14 epoch 18/16 rep_tid 171 entries 18'5 (0'0) error 3:2e2fa760:::200.00000001:head by mds.0.3:32 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:24:07.891 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:24:07.891 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 80.754913969, 2023-11-25T04:24:07.892 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005556938, 2023-11-25T04:24:07.892 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:07.892 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2023-11-25T04:24:07.892 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:07.892 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.892 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:07.892 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:24:07.892 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:07.892 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.892 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.892 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:07.892 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:24:07.893 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:07.893 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.893 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.893 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:07.893 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:24:07.893 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.03e-07 2023-11-25T04:24:07.893 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.893 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.893 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:07.893 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043931+0000", 2023-11-25T04:24:07.893 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1079999999999998e-06 2023-11-25T04:24:07.893 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.893 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.894 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:07.894 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043944+0000", 2023-11-25T04:24:07.894 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2852e-05 2023-11-25T04:24:07.894 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:07.895 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043955+0000", 2023-11-25T04:24:07.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1037e-05 2023-11-25T04:24:07.895 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.895 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:07.895 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044088+0000", 2023-11-25T04:24:07.896 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013269599999999999 2023-11-25T04:24:07.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.896 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:07.896 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049482+0000", 2023-11-25T04:24:07.896 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053940419999999999 2023-11-25T04:24:07.896 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:07.896 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:07.896 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:07.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.896 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:10 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'5, mlcod=0'0)", 2023-11-25T04:24:07.896 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:24:07.897 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 80.754823940999998, 2023-11-25T04:24:07.897 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054493099999999997, 2023-11-25T04:24:07.897 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:07.897 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:07.897 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:07.897 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.897 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:07.897 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:24:07.897 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:07.897 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.897 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.897 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:07.897 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:24:07.898 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:07.898 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.898 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:07.898 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:24:07.898 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6100000000000002e-07 2023-11-25T04:24:07.898 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.898 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:07.898 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044019+0000", 2023-11-25T04:24:07.898 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.839e-06 2023-11-25T04:24:07.898 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.899 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:07.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044026+0000", 2023-11-25T04:24:07.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5869999999999998e-06 2023-11-25T04:24:07.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.899 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:07.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044031+0000", 2023-11-25T04:24:07.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4909999999999996e-06 2023-11-25T04:24:07.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.899 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:07.899 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044070+0000", 2023-11-25T04:24:07.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8829999999999999e-05 2023-11-25T04:24:07.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.900 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:07.900 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044081+0000", 2023-11-25T04:24:07.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1447999999999999e-05 2023-11-25T04:24:07.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.900 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:07.900 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049406+0000", 2023-11-25T04:24:07.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053246480000000004 2023-11-25T04:24:07.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.900 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:07.901 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049464+0000", 2023-11-25T04:24:07.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8205999999999998e-05 2023-11-25T04:24:07.901 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:07.901 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:07.901 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:07.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.901 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:11 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'6, mlcod=0'0)", 2023-11-25T04:24:07.901 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:24:07.901 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 80.754622677, 2023-11-25T04:24:07.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052673499999999996, 2023-11-25T04:24:07.901 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:07.901 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:07.902 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:07.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.902 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:07.902 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:24:07.902 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:07.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.902 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:07.902 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:24:07.902 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:07.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.902 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:07.903 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:24:07.903 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0199999999999998e-07 2023-11-25T04:24:07.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.903 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.903 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:07.903 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044220+0000", 2023-11-25T04:24:07.903 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7720000000000001e-06 2023-11-25T04:24:07.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.903 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.903 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:07.903 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044226+0000", 2023-11-25T04:24:07.903 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8579999999999996e-06 2023-11-25T04:24:07.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.904 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:07.904 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044232+0000", 2023-11-25T04:24:07.904 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1700000000000002e-06 2023-11-25T04:24:07.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.904 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:07.904 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044274+0000", 2023-11-25T04:24:07.904 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1254999999999997e-05 2023-11-25T04:24:07.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.904 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:07.904 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:24:07.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6880999999999999e-05 2023-11-25T04:24:07.905 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.905 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.905 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:07.905 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049466+0000", 2023-11-25T04:24:07.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051659219999999999 2023-11-25T04:24:07.905 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:07.905 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:07.905 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:07.905 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049483+0000", 2023-11-25T04:24:07.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.719e-05 2023-11-25T04:24:07.905 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:07.905 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:07.906 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:07.906 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:07.906 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:07.906 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:07.907 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:24:07.908 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:07.905+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:08.008 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:08.006+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:08.016 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:08.016 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:24:08.016 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2023-11-25T04:24:08.016 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:08.025 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:24:08.033 INFO:teuthology.orchestra.run.smithi067.stderr:nodeep-scrub is set 2023-11-25T04:24:08.109 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:08.107+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:08.131 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:08.131 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:24:08.131 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2023-11-25T04:24:08.131 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2023-11-25T04:24:08.131 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:08.140 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:24:08.209 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:08.208+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:08.248 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:08.248 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2023-11-25T04:24:08.248 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2023-11-25T04:24:08.248 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2023-11-25T04:24:08.248 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.248 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:08.248 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:24:08.248 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 81.213775557000005, 2023-11-25T04:24:08.248 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01072943, 2023-11-25T04:24:08.248 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:08.249 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:08.249 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:08.249 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.249 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:08.249 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:24:08.249 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:08.249 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.249 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.249 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:08.249 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:24:08.249 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:08.249 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.249 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.250 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:08.250 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031829+0000", 2023-11-25T04:24:08.250 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.666e-06 2023-11-25T04:24:08.250 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.250 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.250 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:08.250 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031839+0000", 2023-11-25T04:24:08.250 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9490000000000008e-06 2023-11-25T04:24:08.250 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.250 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.250 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:08.250 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031857+0000", 2023-11-25T04:24:08.250 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8093000000000001e-05 2023-11-25T04:24:08.251 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.251 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.251 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:08.251 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031882+0000", 2023-11-25T04:24:08.251 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4294000000000002e-05 2023-11-25T04:24:08.251 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.251 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.251 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:08.251 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033130+0000", 2023-11-25T04:24:08.251 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001248264 2023-11-25T04:24:08.251 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.251 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.251 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:08.252 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033426+0000", 2023-11-25T04:24:08.252 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029620999999999998 2023-11-25T04:24:08.252 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.252 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.252 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:08.252 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033814+0000", 2023-11-25T04:24:08.252 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000387742 2023-11-25T04:24:08.252 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.252 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.252 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:08.252 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036793+0000", 2023-11-25T04:24:08.253 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002979598 2023-11-25T04:24:08.253 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.253 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.253 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:08.253 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042455+0000", 2023-11-25T04:24:08.253 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0056619260000000003 2023-11-25T04:24:08.253 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.253 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.253 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:08.254 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042502+0000", 2023-11-25T04:24:08.254 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7139999999999999e-05 2023-11-25T04:24:08.254 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.254 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.254 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:08.254 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042557+0000", 2023-11-25T04:24:08.254 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4548e-05 2023-11-25T04:24:08.254 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:08.254 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:08.254 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:08.254 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.254 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.255 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:08.255 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:24:08.255 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 81.212577022000005, 2023-11-25T04:24:08.255 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011505202000000001, 2023-11-25T04:24:08.255 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:08.255 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:08.255 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:08.255 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.255 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:08.255 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:24:08.255 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:08.255 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.256 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.256 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:08.256 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:24:08.256 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:08.256 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.256 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.256 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:08.256 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033027+0000", 2023-11-25T04:24:08.256 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0699999999999999e-06 2023-11-25T04:24:08.256 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.256 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.257 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:08.257 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033035+0000", 2023-11-25T04:24:08.257 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0830000000000004e-06 2023-11-25T04:24:08.257 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.257 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.257 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:08.257 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033059+0000", 2023-11-25T04:24:08.257 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3975e-05 2023-11-25T04:24:08.257 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.257 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.257 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:08.257 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033482+0000", 2023-11-25T04:24:08.258 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042253700000000001 2023-11-25T04:24:08.258 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.258 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.258 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:08.258 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034520+0000", 2023-11-25T04:24:08.258 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001037991 2023-11-25T04:24:08.258 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.258 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.258 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:08.258 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034611+0000", 2023-11-25T04:24:08.258 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1113999999999995e-05 2023-11-25T04:24:08.258 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.259 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.259 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:08.259 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034769+0000", 2023-11-25T04:24:08.259 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000157641 2023-11-25T04:24:08.259 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.259 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.259 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:08.259 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038292+0000", 2023-11-25T04:24:08.259 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035231160000000002 2023-11-25T04:24:08.259 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.259 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.259 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:08.260 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044440+0000", 2023-11-25T04:24:08.260 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0061486980000000002 2023-11-25T04:24:08.260 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.260 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.260 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:08.260 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044495+0000", 2023-11-25T04:24:08.260 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4339000000000003e-05 2023-11-25T04:24:08.260 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.260 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.260 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:08.260 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044531+0000", 2023-11-25T04:24:08.260 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6637999999999997e-05 2023-11-25T04:24:08.260 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:08.261 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:08.261 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:08.261 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.261 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.261 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:08.261 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:24:08.261 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 81.212102668, 2023-11-25T04:24:08.261 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0057634039999999997, 2023-11-25T04:24:08.261 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:08.261 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:08.261 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:08.261 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.262 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:08.262 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:24:08.262 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:08.262 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.262 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.262 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:08.262 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:24:08.262 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:08.262 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.262 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.262 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:08.262 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:24:08.262 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7900000000000007e-07 2023-11-25T04:24:08.263 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.263 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.263 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:08.263 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033507+0000", 2023-11-25T04:24:08.265 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8900000000000004e-06 2023-11-25T04:24:08.265 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.265 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.265 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:08.265 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033519+0000", 2023-11-25T04:24:08.265 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1756000000000001e-05 2023-11-25T04:24:08.265 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.265 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.265 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:08.266 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033527+0000", 2023-11-25T04:24:08.266 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3610000000000001e-06 2023-11-25T04:24:08.266 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.266 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.266 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:08.266 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034451+0000", 2023-11-25T04:24:08.266 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00092372400000000001 2023-11-25T04:24:08.266 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.266 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.266 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:08.266 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034809+0000", 2023-11-25T04:24:08.266 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00035757699999999998 2023-11-25T04:24:08.267 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.267 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.267 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:08.267 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034965+0000", 2023-11-25T04:24:08.267 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015638699999999999 2023-11-25T04:24:08.267 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.267 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.267 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:08.267 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038299+0000", 2023-11-25T04:24:08.267 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0033336559999999999 2023-11-25T04:24:08.267 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.267 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.267 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:08.268 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039169+0000", 2023-11-25T04:24:08.268 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00087066400000000005 2023-11-25T04:24:08.268 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.268 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.268 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:08.268 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039226+0000", 2023-11-25T04:24:08.268 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6511999999999997e-05 2023-11-25T04:24:08.268 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.268 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.268 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:08.268 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039264+0000", 2023-11-25T04:24:08.268 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7898000000000002e-05 2023-11-25T04:24:08.268 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:08.269 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:08.269 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:08.269 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.269 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.269 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:25 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:08.269 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:24:08.269 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 81.212024084000006, 2023-11-25T04:24:08.269 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096656169999999996, 2023-11-25T04:24:08.269 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:08.269 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:08.269 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:08.269 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.269 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:08.270 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:24:08.270 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:08.270 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.270 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.270 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:08.270 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:24:08.270 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:08.270 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.270 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.270 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:08.270 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033580+0000", 2023-11-25T04:24:08.270 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8199999999999999e-07 2023-11-25T04:24:08.271 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.271 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.271 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:08.271 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033585+0000", 2023-11-25T04:24:08.271 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3680000000000001e-06 2023-11-25T04:24:08.271 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.271 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.271 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:08.271 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033600+0000", 2023-11-25T04:24:08.271 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4331000000000001e-05 2023-11-25T04:24:08.271 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.271 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.271 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:08.272 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033606+0000", 2023-11-25T04:24:08.272 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5780000000000004e-06 2023-11-25T04:24:08.272 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.272 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.272 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:08.272 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035747+0000", 2023-11-25T04:24:08.272 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0021406530000000002 2023-11-25T04:24:08.272 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.272 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.272 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:08.272 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035901+0000", 2023-11-25T04:24:08.272 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000153931 2023-11-25T04:24:08.272 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.273 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.273 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:08.273 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036193+0000", 2023-11-25T04:24:08.273 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029211299999999999 2023-11-25T04:24:08.273 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.273 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.273 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:08.273 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038655+0000", 2023-11-25T04:24:08.273 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0024621640000000002 2023-11-25T04:24:08.273 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.274 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.274 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:08.274 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043130+0000", 2023-11-25T04:24:08.274 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.004474698 2023-11-25T04:24:08.274 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.274 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.274 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:08.274 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043204+0000", 2023-11-25T04:24:08.274 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4091999999999995e-05 2023-11-25T04:24:08.274 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.274 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.275 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:08.275 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043245+0000", 2023-11-25T04:24:08.275 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0907e-05 2023-11-25T04:24:08.275 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:08.275 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:08.275 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:08.275 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.275 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.275 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:26 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:08.275 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:08.275 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 81.211988259999998, 2023-11-25T04:24:08.275 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011086945000000001, 2023-11-25T04:24:08.275 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:08.276 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:08.276 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:08.276 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.276 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:08.276 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:08.276 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:08.276 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.276 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.276 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:08.276 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:08.276 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:08.276 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.276 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.277 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:08.277 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:08.277 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1699999999999998e-07 2023-11-25T04:24:08.277 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.277 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.277 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:08.277 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033627+0000", 2023-11-25T04:24:08.277 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.163e-05 2023-11-25T04:24:08.277 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.277 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.277 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:08.277 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033636+0000", 2023-11-25T04:24:08.277 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6859999999999995e-06 2023-11-25T04:24:08.277 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.278 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.278 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:08.278 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033642+0000", 2023-11-25T04:24:08.278 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3149999999999997e-06 2023-11-25T04:24:08.278 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.278 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.278 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:08.278 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035414+0000", 2023-11-25T04:24:08.278 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0017716380000000001 2023-11-25T04:24:08.278 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.278 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.278 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:08.278 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035539+0000", 2023-11-25T04:24:08.279 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000125477 2023-11-25T04:24:08.279 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.279 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.279 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:08.279 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035672+0000", 2023-11-25T04:24:08.279 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000133316 2023-11-25T04:24:08.279 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.279 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.279 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:08.279 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038649+0000", 2023-11-25T04:24:08.279 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029764679999999999 2023-11-25T04:24:08.279 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.279 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.279 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:08.280 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044578+0000", 2023-11-25T04:24:08.280 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0059293510000000002 2023-11-25T04:24:08.280 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.280 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.280 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:08.280 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044651+0000", 2023-11-25T04:24:08.280 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2513000000000002e-05 2023-11-25T04:24:08.280 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.280 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.280 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:08.280 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044702+0000", 2023-11-25T04:24:08.280 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1034e-05 2023-11-25T04:24:08.280 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:08.281 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:08.281 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:08.281 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.281 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.281 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:29 3.b 3:d0630e4c:::mds0_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:08.281 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:08.281 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 81.211950340000001, 2023-11-25T04:24:08.281 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010980227, 2023-11-25T04:24:08.281 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:08.281 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:08.281 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:08.281 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.281 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:08.282 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:08.282 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:08.282 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.282 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.282 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:08.282 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:08.282 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:08.282 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.282 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.282 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:08.282 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:08.282 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3499999999999996e-07 2023-11-25T04:24:08.283 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.283 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.283 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:08.283 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033658+0000", 2023-11-25T04:24:08.283 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1280000000000001e-06 2023-11-25T04:24:08.283 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.283 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.283 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:08.283 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033667+0000", 2023-11-25T04:24:08.283 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1430000000000004e-06 2023-11-25T04:24:08.283 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.283 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.284 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:08.284 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033678+0000", 2023-11-25T04:24:08.284 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0906e-05 2023-11-25T04:24:08.284 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.284 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.284 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:08.284 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034456+0000", 2023-11-25T04:24:08.284 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00077833899999999996 2023-11-25T04:24:08.284 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.284 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.284 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:08.284 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034575+0000", 2023-11-25T04:24:08.284 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011926000000000001 2023-11-25T04:24:08.285 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.285 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.285 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:08.285 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035001+0000", 2023-11-25T04:24:08.285 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042550999999999998 2023-11-25T04:24:08.285 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.285 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.285 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:08.285 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.037923+0000", 2023-11-25T04:24:08.285 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029219670000000001 2023-11-25T04:24:08.285 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.286 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.286 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:08.286 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044538+0000", 2023-11-25T04:24:08.286 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0066152349999999997 2023-11-25T04:24:08.286 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.286 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.286 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:08.286 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044594+0000", 2023-11-25T04:24:08.286 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5593000000000001e-05 2023-11-25T04:24:08.286 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.286 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.286 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:08.287 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044633+0000", 2023-11-25T04:24:08.287 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9511000000000002e-05 2023-11-25T04:24:08.287 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:08.287 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:08.287 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:08.287 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.287 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.287 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:34 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:08.287 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:24:08.287 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 81.211700226999994, 2023-11-25T04:24:08.287 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010920445000000001, 2023-11-25T04:24:08.287 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:08.288 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:08.288 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:08.288 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.288 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:08.288 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:24:08.288 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:08.288 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.288 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.288 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:08.288 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:24:08.288 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:08.288 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.288 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.289 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:08.289 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033904+0000", 2023-11-25T04:24:08.289 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6599999999999994e-07 2023-11-25T04:24:08.289 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.289 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.289 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:08.289 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033909+0000", 2023-11-25T04:24:08.289 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5639999999999996e-06 2023-11-25T04:24:08.289 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.289 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.289 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:08.289 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033923+0000", 2023-11-25T04:24:08.290 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3028e-05 2023-11-25T04:24:08.290 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.290 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.290 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:08.290 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034043+0000", 2023-11-25T04:24:08.290 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000120328 2023-11-25T04:24:08.290 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.290 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.290 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:08.290 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035576+0000", 2023-11-25T04:24:08.290 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015333720000000001 2023-11-25T04:24:08.290 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.291 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.291 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:08.291 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035699+0000", 2023-11-25T04:24:08.291 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000123001 2023-11-25T04:24:08.291 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.291 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.291 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:08.291 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044823+0000", 2023-11-25T04:24:08.292 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0091241859999999994 2023-11-25T04:24:08.292 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:08.292 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:08.292 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:08.292 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.292 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.292 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:35 3.1f 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:08.292 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:24:08.292 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 81.210396021999998, 2023-11-25T04:24:08.292 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096164779999999995, 2023-11-25T04:24:08.292 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:08.292 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:08.293 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:08.293 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.293 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:08.293 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:24:08.293 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:08.293 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.293 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.293 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:08.293 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:24:08.293 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:08.293 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.293 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.294 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:08.294 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035208+0000", 2023-11-25T04:24:08.294 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2699999999999998e-07 2023-11-25T04:24:08.294 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.294 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.294 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:08.294 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035213+0000", 2023-11-25T04:24:08.294 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.6070000000000001e-06 2023-11-25T04:24:08.294 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.294 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.294 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:08.294 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035350+0000", 2023-11-25T04:24:08.295 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013693599999999999 2023-11-25T04:24:08.295 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.295 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.295 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:08.295 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035358+0000", 2023-11-25T04:24:08.295 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0099999999999995e-06 2023-11-25T04:24:08.295 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.295 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.295 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:08.295 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036260+0000", 2023-11-25T04:24:08.295 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00090240100000000005 2023-11-25T04:24:08.295 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.295 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.296 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:08.296 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036517+0000", 2023-11-25T04:24:08.296 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00025736299999999998 2023-11-25T04:24:08.296 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.296 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.296 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:08.296 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044824+0000", 2023-11-25T04:24:08.296 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0083063340000000003 2023-11-25T04:24:08.296 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:08.296 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:08.296 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:08.296 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.297 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.297 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:37 3.1e 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:08.297 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:24:08.297 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 81.210229335999998, 2023-11-25T04:24:08.297 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0078322640000000002, 2023-11-25T04:24:08.297 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:08.297 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:08.297 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:08.297 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.297 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:08.297 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:24:08.297 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:08.298 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.298 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.298 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:08.298 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:24:08.298 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:08.298 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.298 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.298 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:08.298 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035375+0000", 2023-11-25T04:24:08.298 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9100000000000003e-07 2023-11-25T04:24:08.298 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.299 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.299 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:08.299 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035379+0000", 2023-11-25T04:24:08.299 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9430000000000002e-06 2023-11-25T04:24:08.299 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.299 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.299 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:08.299 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035388+0000", 2023-11-25T04:24:08.299 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2760000000000006e-06 2023-11-25T04:24:08.299 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.299 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.299 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:08.300 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035396+0000", 2023-11-25T04:24:08.300 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8480000000000001e-06 2023-11-25T04:24:08.300 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.300 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.300 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:08.300 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035452+0000", 2023-11-25T04:24:08.300 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5831000000000001e-05 2023-11-25T04:24:08.300 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.300 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.300 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:08.300 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035934+0000", 2023-11-25T04:24:08.300 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00048265599999999998 2023-11-25T04:24:08.300 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.301 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.301 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:08.301 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043206+0000", 2023-11-25T04:24:08.301 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.007271919 2023-11-25T04:24:08.301 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:08.301 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:08.301 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:08.301 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.301 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.301 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing(3.1a epoch 18/16 rep_tid 3 entries 18'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:36 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:24:08.301 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:24:08.301 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 81.208953750000006, 2023-11-25T04:24:08.302 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002073414, 2023-11-25T04:24:08.302 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:08.302 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:24:08.302 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:08.302 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.302 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:08.302 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:24:08.302 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:08.302 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.302 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.302 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:08.302 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:24:08.302 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:08.303 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.303 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.303 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:08.303 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036650+0000", 2023-11-25T04:24:08.303 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7300000000000002e-07 2023-11-25T04:24:08.303 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.303 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.303 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:08.303 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036659+0000", 2023-11-25T04:24:08.303 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.7499999999999992e-06 2023-11-25T04:24:08.303 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.303 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.303 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:08.304 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036673+0000", 2023-11-25T04:24:08.304 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4917e-05 2023-11-25T04:24:08.304 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.304 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.304 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:08.304 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036687+0000", 2023-11-25T04:24:08.304 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3883000000000001e-05 2023-11-25T04:24:08.304 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.304 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.304 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:08.304 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036781+0000", 2023-11-25T04:24:08.304 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3219000000000003e-05 2023-11-25T04:24:08.304 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.304 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.305 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:08.305 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038723+0000", 2023-11-25T04:24:08.305 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019422720000000001 2023-11-25T04:24:08.305 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:08.305 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:08.305 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:08.305 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.305 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.305 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:28 3.13 e18/16 3:cd5a64a3:::100.00000000:head v 18'1, mlcod=0'0)", 2023-11-25T04:24:08.305 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:24:08.305 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 81.205292592000006, 2023-11-25T04:24:08.305 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001850546, 2023-11-25T04:24:08.306 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:08.306 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:08.306 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:08.306 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.306 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:08.306 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:24:08.306 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:08.306 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.306 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.306 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:08.306 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:24:08.306 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:08.306 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.307 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.307 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:08.307 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:24:08.307 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7800000000000002e-07 2023-11-25T04:24:08.307 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.307 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.307 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:08.307 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040331+0000", 2023-11-25T04:24:08.307 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0449999999999999e-05 2023-11-25T04:24:08.307 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.307 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.307 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:08.308 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040339+0000", 2023-11-25T04:24:08.308 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0709999999999995e-06 2023-11-25T04:24:08.308 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.308 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.308 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:08.308 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040347+0000", 2023-11-25T04:24:08.308 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7079999999999997e-06 2023-11-25T04:24:08.308 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.308 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.308 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:08.308 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040410+0000", 2023-11-25T04:24:08.308 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2453000000000007e-05 2023-11-25T04:24:08.308 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.309 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.309 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:08.309 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040422+0000", 2023-11-25T04:24:08.309 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2429000000000001e-05 2023-11-25T04:24:08.309 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.309 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.309 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:08.309 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042129+0000", 2023-11-25T04:24:08.309 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001707266 2023-11-25T04:24:08.309 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.309 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.309 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:08.310 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042161+0000", 2023-11-25T04:24:08.310 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1791000000000001e-05 2023-11-25T04:24:08.310 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:08.310 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:08.310 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:08.310 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.310 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.310 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:27 3.13 e18/16 3:c91d4a1d:::mds0_inotable:head v 18'2, mlcod=0'0)", 2023-11-25T04:24:08.310 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:24:08.310 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 81.205089068999996, 2023-11-25T04:24:08.310 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0016886430000000001, 2023-11-25T04:24:08.310 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:08.311 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:08.311 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:08.311 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.311 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:08.311 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:24:08.311 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:08.311 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.311 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.311 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:08.311 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:24:08.311 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:08.311 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.312 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.312 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:08.312 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:24:08.312 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3999999999999997e-07 2023-11-25T04:24:08.312 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.312 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.312 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:08.312 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040520+0000", 2023-11-25T04:24:08.312 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2989999999999996e-06 2023-11-25T04:24:08.312 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.312 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.312 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:08.313 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040527+0000", 2023-11-25T04:24:08.313 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5730000000000002e-06 2023-11-25T04:24:08.313 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.313 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.313 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:08.313 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040535+0000", 2023-11-25T04:24:08.313 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8630000000000008e-06 2023-11-25T04:24:08.313 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.313 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.313 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:08.313 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040683+0000", 2023-11-25T04:24:08.313 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000148002 2023-11-25T04:24:08.314 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.314 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.314 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:08.314 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040700+0000", 2023-11-25T04:24:08.314 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7065e-05 2023-11-25T04:24:08.314 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.314 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.314 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:08.314 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042163+0000", 2023-11-25T04:24:08.314 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001462767 2023-11-25T04:24:08.314 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.314 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.315 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:08.315 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042203+0000", 2023-11-25T04:24:08.315 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9734000000000003e-05 2023-11-25T04:24:08.315 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:08.315 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:08.315 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:08.315 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.315 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.315 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:20 3.1c e18/16 3:3ed09add:::607.00000000:head v 18'1, mlcod=0'0)", 2023-11-25T04:24:08.315 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:24:08.315 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 81.205054028000006, 2023-11-25T04:24:08.315 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015271, 2023-11-25T04:24:08.315 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:08.316 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:08.316 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:08.316 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.316 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:08.316 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:24:08.316 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:08.316 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.316 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.316 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:08.316 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:24:08.317 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:08.317 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.317 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.317 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:08.317 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:24:08.317 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4799999999999999e-07 2023-11-25T04:24:08.317 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.317 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.317 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:08.317 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040561+0000", 2023-11-25T04:24:08.317 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1800000000000001e-05 2023-11-25T04:24:08.317 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.317 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.318 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:08.318 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040568+0000", 2023-11-25T04:24:08.318 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7730000000000002e-06 2023-11-25T04:24:08.318 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.318 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.318 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:08.318 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040587+0000", 2023-11-25T04:24:08.318 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8691000000000001e-05 2023-11-25T04:24:08.318 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.318 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.318 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:08.318 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040638+0000", 2023-11-25T04:24:08.319 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1068e-05 2023-11-25T04:24:08.319 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.319 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.319 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:08.319 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040670+0000", 2023-11-25T04:24:08.319 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2125999999999997e-05 2023-11-25T04:24:08.319 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.319 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.319 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:08.319 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042032+0000", 2023-11-25T04:24:08.319 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001361821 2023-11-25T04:24:08.319 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.320 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.320 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:08.320 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042076+0000", 2023-11-25T04:24:08.320 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4472999999999999e-05 2023-11-25T04:24:08.320 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:08.320 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:08.320 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:08.320 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.320 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.320 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:21 3.1c e18/16 3:3ed09add:::607.00000000:head v 18'2, mlcod=0'0)", 2023-11-25T04:24:08.320 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:24:08.320 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 81.204819302999994, 2023-11-25T04:24:08.321 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0013228160000000001, 2023-11-25T04:24:08.321 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:08.321 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:08.321 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:08.321 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.321 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:08.321 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:24:08.321 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:08.321 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.321 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.321 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:08.321 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:24:08.322 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:08.322 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.322 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.322 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:08.322 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:24:08.322 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9299999999999999e-07 2023-11-25T04:24:08.322 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.322 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.322 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:08.322 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040790+0000", 2023-11-25T04:24:08.322 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5770000000000001e-06 2023-11-25T04:24:08.322 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.323 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.323 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:08.323 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040799+0000", 2023-11-25T04:24:08.323 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6439999999999995e-06 2023-11-25T04:24:08.323 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.323 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.323 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:08.323 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040806+0000", 2023-11-25T04:24:08.323 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0180000000000004e-06 2023-11-25T04:24:08.323 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.323 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.323 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:08.323 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040907+0000", 2023-11-25T04:24:08.324 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000100096 2023-11-25T04:24:08.324 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.324 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.324 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:08.324 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040925+0000", 2023-11-25T04:24:08.324 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8536999999999999e-05 2023-11-25T04:24:08.324 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.324 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.324 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:08.324 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042079+0000", 2023-11-25T04:24:08.324 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00115358 2023-11-25T04:24:08.324 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.324 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.325 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:08.325 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042107+0000", 2023-11-25T04:24:08.325 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7971e-05 2023-11-25T04:24:08.325 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:08.325 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:08.325 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:08.325 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.325 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.325 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop_reply(mds.0.3:4 3.1f e18/16 ondisk, result = 0)", 2023-11-25T04:24:08.325 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:24:08.325 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 81.203479912999995, 2023-11-25T04:24:08.325 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00044092900000000001, 2023-11-25T04:24:08.326 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:08.326 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:08.326 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:08.326 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.326 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:08.326 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:24:08.326 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:08.326 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.326 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.326 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:08.326 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:24:08.326 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:08.327 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.327 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.327 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:08.327 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042124+0000", 2023-11-25T04:24:08.327 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1800000000000005e-07 2023-11-25T04:24:08.327 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.327 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.327 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:08.327 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042128+0000", 2023-11-25T04:24:08.327 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2289999999999997e-06 2023-11-25T04:24:08.327 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.327 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.328 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:08.328 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042138+0000", 2023-11-25T04:24:08.328 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9729999999999992e-06 2023-11-25T04:24:08.328 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.328 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.328 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:08.328 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042145+0000", 2023-11-25T04:24:08.328 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.336e-06 2023-11-25T04:24:08.328 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.328 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.328 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:08.328 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042441+0000", 2023-11-25T04:24:08.329 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029650500000000002 2023-11-25T04:24:08.329 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.329 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.329 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:08.329 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042448+0000", 2023-11-25T04:24:08.329 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4100000000000002e-06 2023-11-25T04:24:08.329 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.329 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.329 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:08.329 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042564+0000", 2023-11-25T04:24:08.329 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000115758 2023-11-25T04:24:08.329 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:08.330 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:08.330 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:08.330 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.331 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.331 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing_reply(3.1f epoch 18/16 rep_tid 8 lcod 18'4)", 2023-11-25T04:24:08.331 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:24:08.331 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 81.201154501000005, 2023-11-25T04:24:08.331 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00038154699999999998, 2023-11-25T04:24:08.331 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:08.331 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:24:08.331 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:08.331 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.332 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:08.332 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:24:08.332 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:08.332 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.332 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.332 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:08.332 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:24:08.332 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:08.332 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.332 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.332 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:08.332 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:24:08.332 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2700000000000001e-07 2023-11-25T04:24:08.333 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.333 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.333 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:08.333 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044451+0000", 2023-11-25T04:24:08.333 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7460000000000001e-06 2023-11-25T04:24:08.333 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.333 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.333 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:08.333 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044455+0000", 2023-11-25T04:24:08.333 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5809999999999998e-06 2023-11-25T04:24:08.333 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.333 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.333 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:08.334 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044458+0000", 2023-11-25T04:24:08.334 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0170000000000001e-06 2023-11-25T04:24:08.334 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.334 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.334 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:08.334 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044734+0000", 2023-11-25T04:24:08.334 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00027582799999999999 2023-11-25T04:24:08.334 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.334 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.334 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:08.334 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044830+0000", 2023-11-25T04:24:08.334 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6248000000000004e-05 2023-11-25T04:24:08.335 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:08.335 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:08.335 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:08.335 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.335 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.335 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing_reply(3.b epoch 18/16 rep_tid 5 lcod 18'2)", 2023-11-25T04:24:08.335 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:24:08.335 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 81.201117839999995, 2023-11-25T04:24:08.335 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000344488, 2023-11-25T04:24:08.335 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:08.335 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:24:08.335 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:08.336 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.336 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:08.336 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:24:08.336 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:08.336 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.336 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.336 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:08.336 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:24:08.336 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:08.336 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.336 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.336 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:08.337 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:24:08.337 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.55e-07 2023-11-25T04:24:08.337 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.337 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.337 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:08.337 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044487+0000", 2023-11-25T04:24:08.337 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2470000000000001e-06 2023-11-25T04:24:08.337 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.337 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.337 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:08.337 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044491+0000", 2023-11-25T04:24:08.337 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9659999999999998e-06 2023-11-25T04:24:08.337 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.338 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.338 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:08.338 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044494+0000", 2023-11-25T04:24:08.338 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2310000000000001e-06 2023-11-25T04:24:08.338 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.338 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.338 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:08.338 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044756+0000", 2023-11-25T04:24:08.338 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00026172800000000003 2023-11-25T04:24:08.338 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.338 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.339 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:08.339 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044830+0000", 2023-11-25T04:24:08.339 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4160999999999997e-05 2023-11-25T04:24:08.339 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:08.339 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:08.339 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:08.339 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.339 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.339 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:39 3.6 3:654134d2:::mds0_openfiles.0:head [omap-clear,omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2023-11-25T04:24:08.339 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:24:08.339 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 76.141291797999997, 2023-11-25T04:24:08.340 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001788658, 2023-11-25T04:24:08.340 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:08.340 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:08.340 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:08.340 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.340 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:08.340 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:24:08.340 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:08.340 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.340 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.340 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:08.340 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:24:08.341 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:08.341 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.341 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.341 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:08.341 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104314+0000", 2023-11-25T04:24:08.341 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3099999999999999e-06 2023-11-25T04:24:08.341 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.341 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.341 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:08.341 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104325+0000", 2023-11-25T04:24:08.341 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1511999999999999e-05 2023-11-25T04:24:08.341 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.342 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.342 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:08.342 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104362+0000", 2023-11-25T04:24:08.342 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6903000000000002e-05 2023-11-25T04:24:08.342 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.342 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.342 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:08.342 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104387+0000", 2023-11-25T04:24:08.342 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4766e-05 2023-11-25T04:24:08.342 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.342 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.342 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:08.343 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104465+0000", 2023-11-25T04:24:08.343 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8119000000000002e-05 2023-11-25T04:24:08.343 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.343 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.343 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:08.343 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104557+0000", 2023-11-25T04:24:08.343 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2496000000000003e-05 2023-11-25T04:24:08.343 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.343 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.343 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:08.344 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104646+0000", 2023-11-25T04:24:08.344 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8818000000000002e-05 2023-11-25T04:24:08.344 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.344 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.344 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:08.344 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105201+0000", 2023-11-25T04:24:08.344 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00055506000000000002 2023-11-25T04:24:08.344 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.344 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.344 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:08.345 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105994+0000", 2023-11-25T04:24:08.345 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00079294700000000001 2023-11-25T04:24:08.345 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.345 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.345 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:08.345 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.106055+0000", 2023-11-25T04:24:08.345 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0566000000000003e-05 2023-11-25T04:24:08.345 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.345 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.345 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:08.345 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.106100+0000", 2023-11-25T04:24:08.345 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5160999999999997e-05 2023-11-25T04:24:08.346 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:08.346 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:08.346 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:08.346 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.346 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.346 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:43 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2023-11-25T04:24:08.346 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:24:08.346 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 61.141066801999997, 2023-11-25T04:24:08.346 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001751166, 2023-11-25T04:24:08.347 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:08.347 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:08.347 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:08.347 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.347 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:08.347 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:24:08.347 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:08.347 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.347 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.347 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:08.347 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:24:08.347 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:08.348 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.348 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.348 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:08.348 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104539+0000", 2023-11-25T04:24:08.348 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9529999999999999e-06 2023-11-25T04:24:08.348 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.348 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.348 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:08.348 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104557+0000", 2023-11-25T04:24:08.348 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7464000000000001e-05 2023-11-25T04:24:08.348 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.349 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.349 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:08.349 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104590+0000", 2023-11-25T04:24:08.349 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2966000000000003e-05 2023-11-25T04:24:08.349 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.349 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.349 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:08.349 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104608+0000", 2023-11-25T04:24:08.349 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8193e-05 2023-11-25T04:24:08.349 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.349 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.349 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:08.350 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104767+0000", 2023-11-25T04:24:08.350 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015882 2023-11-25T04:24:08.350 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.350 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.350 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:08.350 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104909+0000", 2023-11-25T04:24:08.350 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014244999999999999 2023-11-25T04:24:08.350 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.350 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.351 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:08.351 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.105059+0000", 2023-11-25T04:24:08.351 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014964399999999999 2023-11-25T04:24:08.351 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.351 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.351 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:08.351 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.105701+0000", 2023-11-25T04:24:08.351 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00064201099999999995 2023-11-25T04:24:08.351 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.351 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.351 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:08.352 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106233+0000", 2023-11-25T04:24:08.352 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00053178500000000001 2023-11-25T04:24:08.352 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.352 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.352 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:08.352 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106265+0000", 2023-11-25T04:24:08.352 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1986000000000002e-05 2023-11-25T04:24:08.352 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.352 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.352 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:08.353 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106288+0000", 2023-11-25T04:24:08.353 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2894000000000001e-05 2023-11-25T04:24:08.353 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:08.353 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:08.353 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:08.353 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.353 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.353 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:45 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e23)", 2023-11-25T04:24:08.353 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:24:08.353 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 56.140868761999997, 2023-11-25T04:24:08.353 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00098423299999999994, 2023-11-25T04:24:08.353 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:08.354 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:08.354 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:08.354 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.354 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:08.354 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:24:08.354 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:08.354 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.354 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.354 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:08.354 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:24:08.354 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:08.355 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.355 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.355 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:08.355 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104737+0000", 2023-11-25T04:24:08.355 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3309999999999998e-06 2023-11-25T04:24:08.355 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.355 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.355 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:08.355 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104748+0000", 2023-11-25T04:24:08.355 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1080999999999999e-05 2023-11-25T04:24:08.355 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.355 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.356 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:08.356 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104771+0000", 2023-11-25T04:24:08.356 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2707000000000001e-05 2023-11-25T04:24:08.356 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.356 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.356 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:08.356 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104800+0000", 2023-11-25T04:24:08.356 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9595e-05 2023-11-25T04:24:08.356 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.356 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.356 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:08.356 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104890+0000", 2023-11-25T04:24:08.357 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.9808999999999993e-05 2023-11-25T04:24:08.357 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.357 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.357 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:08.357 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105004+0000", 2023-11-25T04:24:08.357 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000114508 2023-11-25T04:24:08.357 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.357 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.357 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:08.357 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105634+0000", 2023-11-25T04:24:08.357 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00062985799999999996 2023-11-25T04:24:08.357 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.357 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.358 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:08.358 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105671+0000", 2023-11-25T04:24:08.358 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6640000000000002e-05 2023-11-25T04:24:08.358 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:08.358 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:08.358 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:08.358 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105719+0000", 2023-11-25T04:24:08.358 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7704000000000003e-05 2023-11-25T04:24:08.358 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:08.358 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:08.358 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:08.359 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:08.359 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:08.359 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:08.360 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:24:08.361 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:08.358+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:08.461 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:08.459+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:08.469 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:08.469 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:24:08.469 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2023-11-25T04:24:08.469 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:08.477 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:24:08.562 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:08.561+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:08.622 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:08.622 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:24:08.622 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2023-11-25T04:24:08.623 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2023-11-25T04:24:08.623 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:08.631 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:24:08.664 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:08.662+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:08.738 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:08.738 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2023-11-25T04:24:08.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2023-11-25T04:24:08.739 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2023-11-25T04:24:08.739 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.739 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:155 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'109, mlcod=11'105)", 2023-11-25T04:24:08.739 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:24:08.739 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 90.951223670999994, 2023-11-25T04:24:08.739 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061719019999999999, 2023-11-25T04:24:08.739 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:08.739 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:08.739 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:08.739 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.739 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:08.740 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:24:08.740 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:08.740 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.740 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.740 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:08.740 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:24:08.740 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:08.740 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.740 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.740 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:08.740 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784884+0000", 2023-11-25T04:24:08.741 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3099999999999999e-07 2023-11-25T04:24:08.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:08.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784942+0000", 2023-11-25T04:24:08.741 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8570000000000003e-05 2023-11-25T04:24:08.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:08.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784949+0000", 2023-11-25T04:24:08.741 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0310000000000001e-06 2023-11-25T04:24:08.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.742 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:08.742 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784953+0000", 2023-11-25T04:24:08.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6320000000000001e-06 2023-11-25T04:24:08.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.744 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:08.744 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784993+0000", 2023-11-25T04:24:08.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9737000000000003e-05 2023-11-25T04:24:08.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:08.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785011+0000", 2023-11-25T04:24:08.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8349999999999999e-05 2023-11-25T04:24:08.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:08.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791024+0000", 2023-11-25T04:24:08.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006012997 2023-11-25T04:24:08.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.747 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.747 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:08.747 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791055+0000", 2023-11-25T04:24:08.747 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1254000000000003e-05 2023-11-25T04:24:08.748 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:08.748 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:08.748 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:08.748 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.748 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.748 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:156 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'110, mlcod=11'105)", 2023-11-25T04:24:08.749 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:24:08.749 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 90.950637622000002, 2023-11-25T04:24:08.749 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005616116, 2023-11-25T04:24:08.749 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:08.749 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:08.749 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:08.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:08.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:24:08.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:08.750 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:08.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:24:08.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:08.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.752 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:08.752 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785470+0000", 2023-11-25T04:24:08.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0699999999999998e-07 2023-11-25T04:24:08.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.752 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:08.753 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785474+0000", 2023-11-25T04:24:08.753 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.048e-06 2023-11-25T04:24:08.753 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.753 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.753 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:08.753 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785480+0000", 2023-11-25T04:24:08.754 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7729999999999996e-06 2023-11-25T04:24:08.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.754 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:08.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785483+0000", 2023-11-25T04:24:08.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1970000000000001e-06 2023-11-25T04:24:08.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:08.756 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787936+0000", 2023-11-25T04:24:08.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024529040000000001 2023-11-25T04:24:08.756 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.756 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.757 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:08.757 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787947+0000", 2023-11-25T04:24:08.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1669e-05 2023-11-25T04:24:08.757 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.757 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.757 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:08.758 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791057+0000", 2023-11-25T04:24:08.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031095200000000002 2023-11-25T04:24:08.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.758 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:08.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791086+0000", 2023-11-25T04:24:08.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8697999999999999e-05 2023-11-25T04:24:08.759 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:08.759 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:08.759 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:08.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.760 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:157 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'111, mlcod=11'106)", 2023-11-25T04:24:08.760 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:24:08.760 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 90.950045063999994, 2023-11-25T04:24:08.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050590280000000001, 2023-11-25T04:24:08.760 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:08.761 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:08.761 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:08.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:08.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:24:08.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:08.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.762 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:08.762 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:24:08.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:08.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.763 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:08.763 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786063+0000", 2023-11-25T04:24:08.763 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2600000000000002e-07 2023-11-25T04:24:08.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:08.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786069+0000", 2023-11-25T04:24:08.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8839999999999999e-06 2023-11-25T04:24:08.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:08.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786079+0000", 2023-11-25T04:24:08.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0124e-05 2023-11-25T04:24:08.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.766 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:08.766 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786085+0000", 2023-11-25T04:24:08.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6660000000000002e-06 2023-11-25T04:24:08.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.767 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:08.767 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788160+0000", 2023-11-25T04:24:08.767 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020747669999999999 2023-11-25T04:24:08.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.768 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:08.768 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788180+0000", 2023-11-25T04:24:08.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0208999999999999e-05 2023-11-25T04:24:08.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:08.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791087+0000", 2023-11-25T04:24:08.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029064630000000002 2023-11-25T04:24:08.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:08.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791121+0000", 2023-11-25T04:24:08.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4289000000000003e-05 2023-11-25T04:24:08.770 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:08.770 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:08.770 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:08.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.771 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:158 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'112, mlcod=11'106)", 2023-11-25T04:24:08.771 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:24:08.771 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 90.949628670999999, 2023-11-25T04:24:08.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046663030000000001, 2023-11-25T04:24:08.771 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:08.772 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:08.772 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:08.772 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.772 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:08.772 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:24:08.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:08.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.773 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:08.773 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:24:08.773 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:08.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:08.774 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786479+0000", 2023-11-25T04:24:08.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2799999999999996e-07 2023-11-25T04:24:08.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:08.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786498+0000", 2023-11-25T04:24:08.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9069e-05 2023-11-25T04:24:08.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:08.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786508+0000", 2023-11-25T04:24:08.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5810000000000003e-06 2023-11-25T04:24:08.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:08.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786512+0000", 2023-11-25T04:24:08.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4889999999999997e-06 2023-11-25T04:24:08.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.777 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:08.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788438+0000", 2023-11-25T04:24:08.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019257059999999999 2023-11-25T04:24:08.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:08.778 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788449+0000", 2023-11-25T04:24:08.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1419e-05 2023-11-25T04:24:08.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:08.779 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791122+0000", 2023-11-25T04:24:08.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002672822 2023-11-25T04:24:08.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.780 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:08.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791145+0000", 2023-11-25T04:24:08.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2589e-05 2023-11-25T04:24:08.780 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:08.780 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:08.781 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:08.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.781 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:206 1.0 e12/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 12'152, mlcod=12'148)", 2023-11-25T04:24:08.781 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:24:08.781 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 90.437860201000007, 2023-11-25T04:24:08.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046728189999999999, 2023-11-25T04:24:08.782 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:08.782 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:08.782 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:08.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:08.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:24:08.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:08.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:08.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:24:08.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:08.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:08.785 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298248+0000", 2023-11-25T04:24:08.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3600000000000003e-07 2023-11-25T04:24:08.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:08.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298278+0000", 2023-11-25T04:24:08.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0474e-05 2023-11-25T04:24:08.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:08.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298288+0000", 2023-11-25T04:24:08.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0156000000000001e-05 2023-11-25T04:24:08.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:08.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298293+0000", 2023-11-25T04:24:08.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7400000000000004e-06 2023-11-25T04:24:08.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:08.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298333+0000", 2023-11-25T04:24:08.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9666000000000003e-05 2023-11-25T04:24:08.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:08.790 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298351+0000", 2023-11-25T04:24:08.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8544999999999999e-05 2023-11-25T04:24:08.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.790 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:08.790 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302891+0000", 2023-11-25T04:24:08.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045398519999999996 2023-11-25T04:24:08.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:08.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302920+0000", 2023-11-25T04:24:08.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8750000000000001e-05 2023-11-25T04:24:08.791 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:08.791 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:08.791 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:08.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.791 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:14 3.15 3:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:08.791 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:24:08.791 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 81.702725844, 2023-11-25T04:24:08.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010990437, 2023-11-25T04:24:08.791 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:08.791 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:08.792 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:08.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:08.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:24:08.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:08.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:08.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:24:08.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:08.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:08.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033383+0000", 2023-11-25T04:24:08.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9870000000000002e-06 2023-11-25T04:24:08.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:08.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033398+0000", 2023-11-25T04:24:08.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4569999999999999e-05 2023-11-25T04:24:08.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:08.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033411+0000", 2023-11-25T04:24:08.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3543999999999999e-05 2023-11-25T04:24:08.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:08.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033423+0000", 2023-11-25T04:24:08.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1905999999999999e-05 2023-11-25T04:24:08.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.794 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:08.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034433+0000", 2023-11-25T04:24:08.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00101022 2023-11-25T04:24:08.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.794 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:08.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034539+0000", 2023-11-25T04:24:08.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010597199999999999 2023-11-25T04:24:08.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:08.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034667+0000", 2023-11-25T04:24:08.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012765500000000001 2023-11-25T04:24:08.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:08.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037803+0000", 2023-11-25T04:24:08.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031362220000000001 2023-11-25T04:24:08.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:08.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:24:08.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0064964669999999997 2023-11-25T04:24:08.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:08.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044331+0000", 2023-11-25T04:24:08.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1544999999999997e-05 2023-11-25T04:24:08.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:08.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044372+0000", 2023-11-25T04:24:08.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0349000000000003e-05 2023-11-25T04:24:08.797 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:08.797 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:08.797 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:08.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.797 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:15 3.15 3:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:08.797 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:24:08.797 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 81.702066193999997, 2023-11-25T04:24:08.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010464362, 2023-11-25T04:24:08.797 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:08.797 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:08.798 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:08.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.798 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:08.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:24:08.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:08.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.798 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:08.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:24:08.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:08.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:08.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034042+0000", 2023-11-25T04:24:08.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0999999999999997e-07 2023-11-25T04:24:08.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:08.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034048+0000", 2023-11-25T04:24:08.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0390000000000001e-06 2023-11-25T04:24:08.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:08.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034059+0000", 2023-11-25T04:24:08.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1182999999999999e-05 2023-11-25T04:24:08.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:08.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034067+0000", 2023-11-25T04:24:08.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1559999999999995e-06 2023-11-25T04:24:08.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:08.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035005+0000", 2023-11-25T04:24:08.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00093747399999999999 2023-11-25T04:24:08.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:08.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035077+0000", 2023-11-25T04:24:08.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2747e-05 2023-11-25T04:24:08.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:08.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035219+0000", 2023-11-25T04:24:08.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000141877 2023-11-25T04:24:08.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:08.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037810+0000", 2023-11-25T04:24:08.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002590752 2023-11-25T04:24:08.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:08.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044437+0000", 2023-11-25T04:24:08.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066270110000000004 2023-11-25T04:24:08.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:08.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044481+0000", 2023-11-25T04:24:08.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4106000000000001e-05 2023-11-25T04:24:08.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:08.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044505+0000", 2023-11-25T04:24:08.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4207e-05 2023-11-25T04:24:08.803 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:08.803 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:08.803 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:08.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.803 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:16 3.1d 3:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:08.803 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:24:08.803 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 81.702024346000002, 2023-11-25T04:24:08.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010413923, 2023-11-25T04:24:08.803 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:08.803 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:08.803 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:08.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.804 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:08.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:24:08.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:08.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.804 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:08.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:24:08.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:08.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.804 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:08.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:24:08.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4499999999999997e-07 2023-11-25T04:24:08.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:08.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034087+0000", 2023-11-25T04:24:08.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9770000000000002e-06 2023-11-25T04:24:08.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:08.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034094+0000", 2023-11-25T04:24:08.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8809999999999998e-06 2023-11-25T04:24:08.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:08.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034100+0000", 2023-11-25T04:24:08.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8440000000000003e-06 2023-11-25T04:24:08.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:08.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034993+0000", 2023-11-25T04:24:08.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00089336799999999996 2023-11-25T04:24:08.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:08.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035099+0000", 2023-11-25T04:24:08.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106037 2023-11-25T04:24:08.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:08.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035276+0000", 2023-11-25T04:24:08.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017631799999999999 2023-11-25T04:24:08.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:08.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037815+0000", 2023-11-25T04:24:08.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025395959999999999 2023-11-25T04:24:08.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:08.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044420+0000", 2023-11-25T04:24:08.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066045749999999997 2023-11-25T04:24:08.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:08.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044463+0000", 2023-11-25T04:24:08.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3205999999999999e-05 2023-11-25T04:24:08.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:08.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044497+0000", 2023-11-25T04:24:08.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3476000000000003e-05 2023-11-25T04:24:08.808 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:08.808 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:08.808 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:08.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.809 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:17 3.1d 3:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:08.809 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:24:08.809 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 81.701999137000001, 2023-11-25T04:24:08.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01127356, 2023-11-25T04:24:08.809 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:08.809 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:08.809 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:08.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:08.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:24:08.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:08.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:08.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:24:08.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:08.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:08.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:24:08.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1100000000000001e-07 2023-11-25T04:24:08.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:08.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034112+0000", 2023-11-25T04:24:08.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8600000000000003e-06 2023-11-25T04:24:08.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.811 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:08.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034120+0000", 2023-11-25T04:24:08.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7700000000000001e-06 2023-11-25T04:24:08.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.811 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:08.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034127+0000", 2023-11-25T04:24:08.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0999999999999998e-06 2023-11-25T04:24:08.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.812 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:08.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035601+0000", 2023-11-25T04:24:08.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014739410000000001 2023-11-25T04:24:08.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.812 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:08.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035685+0000", 2023-11-25T04:24:08.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3677999999999994e-05 2023-11-25T04:24:08.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:08.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035827+0000", 2023-11-25T04:24:08.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014232599999999999 2023-11-25T04:24:08.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:08.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038725+0000", 2023-11-25T04:24:08.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028982779999999998 2023-11-25T04:24:08.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:08.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045323+0000", 2023-11-25T04:24:08.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0065980780000000003 2023-11-25T04:24:08.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:08.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045354+0000", 2023-11-25T04:24:08.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0463e-05 2023-11-25T04:24:08.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:08.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045381+0000", 2023-11-25T04:24:08.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7654999999999999e-05 2023-11-25T04:24:08.814 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:08.815 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:08.815 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:08.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.815 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:22 3.e 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:08.815 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:24:08.815 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 81.701972724000001, 2023-11-25T04:24:08.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010133422, 2023-11-25T04:24:08.815 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:08.815 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:08.815 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:08.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:08.816 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:24:08.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:08.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:08.816 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:24:08.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:08.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:08.816 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034135+0000", 2023-11-25T04:24:08.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-07 2023-11-25T04:24:08.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:08.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034138+0000", 2023-11-25T04:24:08.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6009999999999999e-06 2023-11-25T04:24:08.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:08.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034145+0000", 2023-11-25T04:24:08.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1169999999999998e-06 2023-11-25T04:24:08.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:08.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034150+0000", 2023-11-25T04:24:08.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9080000000000003e-06 2023-11-25T04:24:08.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:08.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034209+0000", 2023-11-25T04:24:08.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8149000000000002e-05 2023-11-25T04:24:08.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:08.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034415+0000", 2023-11-25T04:24:08.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020637699999999999 2023-11-25T04:24:08.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:08.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034584+0000", 2023-11-25T04:24:08.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016870399999999999 2023-11-25T04:24:08.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:08.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.036979+0000", 2023-11-25T04:24:08.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023957509999999998 2023-11-25T04:24:08.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:08.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044161+0000", 2023-11-25T04:24:08.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0071811590000000003 2023-11-25T04:24:08.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:08.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044224+0000", 2023-11-25T04:24:08.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3454000000000006e-05 2023-11-25T04:24:08.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:08.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044268+0000", 2023-11-25T04:24:08.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3761999999999999e-05 2023-11-25T04:24:08.821 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:08.821 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:08.821 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:08.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.821 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:23 3.e 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:08.821 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:24:08.821 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 81.701941314999999, 2023-11-25T04:24:08.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010281676, 2023-11-25T04:24:08.821 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:08.821 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:08.822 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:08.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:08.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:24:08.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:08.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:08.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:24:08.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:08.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:08.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:24:08.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5799999999999999e-07 2023-11-25T04:24:08.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.823 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:08.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034169+0000", 2023-11-25T04:24:08.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1879999999999998e-06 2023-11-25T04:24:08.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.823 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:08.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034177+0000", 2023-11-25T04:24:08.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7020000000000001e-06 2023-11-25T04:24:08.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:08.824 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034184+0000", 2023-11-25T04:24:08.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3910000000000003e-06 2023-11-25T04:24:08.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:08.824 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034924+0000", 2023-11-25T04:24:08.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00074032999999999998 2023-11-25T04:24:08.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:08.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034991+0000", 2023-11-25T04:24:08.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7347000000000004e-05 2023-11-25T04:24:08.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:08.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035096+0000", 2023-11-25T04:24:08.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010488 2023-11-25T04:24:08.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:08.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037588+0000", 2023-11-25T04:24:08.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024918309999999999 2023-11-25T04:24:08.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:08.826 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044367+0000", 2023-11-25T04:24:08.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067787589999999997 2023-11-25T04:24:08.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:08.826 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044415+0000", 2023-11-25T04:24:08.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7803000000000003e-05 2023-11-25T04:24:08.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:08.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044447+0000", 2023-11-25T04:24:08.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2886999999999999e-05 2023-11-25T04:24:08.827 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:08.827 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:08.827 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:08.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.827 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:30 3.15 3:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:08.827 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:24:08.827 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 81.701882874999995, 2023-11-25T04:24:08.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010365070000000001, 2023-11-25T04:24:08.828 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:08.828 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:08.828 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:08.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:08.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:24:08.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:08.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:08.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:24:08.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:08.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:08.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034225+0000", 2023-11-25T04:24:08.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7400000000000002e-07 2023-11-25T04:24:08.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:08.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034229+0000", 2023-11-25T04:24:08.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3100000000000002e-06 2023-11-25T04:24:08.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:08.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034239+0000", 2023-11-25T04:24:08.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9140000000000003e-06 2023-11-25T04:24:08.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:08.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034246+0000", 2023-11-25T04:24:08.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0149999999999998e-06 2023-11-25T04:24:08.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:08.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035572+0000", 2023-11-25T04:24:08.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001325655 2023-11-25T04:24:08.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:08.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035730+0000", 2023-11-25T04:24:08.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015827899999999999 2023-11-25T04:24:08.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:08.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035857+0000", 2023-11-25T04:24:08.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012644699999999999 2023-11-25T04:24:08.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.832 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:08.832 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038737+0000", 2023-11-25T04:24:08.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002880077 2023-11-25T04:24:08.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.832 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:08.832 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044540+0000", 2023-11-25T04:24:08.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058034239999999997 2023-11-25T04:24:08.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.832 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:08.832 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044569+0000", 2023-11-25T04:24:08.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8388999999999999e-05 2023-11-25T04:24:08.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:08.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044589+0000", 2023-11-25T04:24:08.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0686000000000002e-05 2023-11-25T04:24:08.833 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:08.833 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:08.833 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:08.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.833 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:33 3.1b 3:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:08.833 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:24:08.833 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 81.701837377000004, 2023-11-25T04:24:08.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099754049999999997, 2023-11-25T04:24:08.834 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:08.834 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:08.834 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:08.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:08.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:24:08.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:08.834 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.834 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.834 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:08.834 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:24:08.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:08.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.835 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:08.835 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:24:08.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4799999999999998e-07 2023-11-25T04:24:08.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.835 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:08.835 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034273+0000", 2023-11-25T04:24:08.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7819999999999998e-06 2023-11-25T04:24:08.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:08.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034285+0000", 2023-11-25T04:24:08.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2058e-05 2023-11-25T04:24:08.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:08.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034291+0000", 2023-11-25T04:24:08.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0680000000000002e-06 2023-11-25T04:24:08.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.837 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:08.837 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034370+0000", 2023-11-25T04:24:08.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8452e-05 2023-11-25T04:24:08.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.837 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:08.837 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034477+0000", 2023-11-25T04:24:08.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107471 2023-11-25T04:24:08.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.837 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:08.837 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044245+0000", 2023-11-25T04:24:08.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0097679259999999997 2023-11-25T04:24:08.838 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:08.838 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:08.838 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:08.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.838 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:36 3.1a 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:08.838 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:24:08.838 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 81.701790704000004, 2023-11-25T04:24:08.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058179099999999999, 2023-11-25T04:24:08.838 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:08.838 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:08.838 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:08.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:08.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:24:08.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:08.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:08.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:24:08.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:08.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:08.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034317+0000", 2023-11-25T04:24:08.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1999999999999999e-07 2023-11-25T04:24:08.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:08.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034321+0000", 2023-11-25T04:24:08.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.233e-06 2023-11-25T04:24:08.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:08.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034337+0000", 2023-11-25T04:24:08.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5308999999999999e-05 2023-11-25T04:24:08.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:08.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034354+0000", 2023-11-25T04:24:08.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7411999999999999e-05 2023-11-25T04:24:08.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:08.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034465+0000", 2023-11-25T04:24:08.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011154699999999999 2023-11-25T04:24:08.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:08.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034560+0000", 2023-11-25T04:24:08.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4186999999999998e-05 2023-11-25T04:24:08.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.842 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:08.842 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.040134+0000", 2023-11-25T04:24:08.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055746019999999997 2023-11-25T04:24:08.842 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:08.842 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:08.842 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:08.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.842 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:9 3.10 e18/16 3:09401a81:::601.00000000:head v 18'4, mlcod=0'0)", 2023-11-25T04:24:08.842 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:24:08.843 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 81.692896395999995, 2023-11-25T04:24:08.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061641020000000003, 2023-11-25T04:24:08.843 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:08.843 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:08.843 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:08.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:08.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:24:08.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:08.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:08.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:24:08.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:08.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.844 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:08.844 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:24:08.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5300000000000001e-07 2023-11-25T04:24:08.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.844 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:08.844 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043222+0000", 2023-11-25T04:24:08.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.082e-05 2023-11-25T04:24:08.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.845 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:08.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043228+0000", 2023-11-25T04:24:08.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6699999999999997e-06 2023-11-25T04:24:08.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.845 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:08.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043236+0000", 2023-11-25T04:24:08.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8189999999999992e-06 2023-11-25T04:24:08.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.845 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:08.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043507+0000", 2023-11-25T04:24:08.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000271012 2023-11-25T04:24:08.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:08.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043530+0000", 2023-11-25T04:24:08.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2659000000000001e-05 2023-11-25T04:24:08.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:08.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049321+0000", 2023-11-25T04:24:08.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057915129999999999 2023-11-25T04:24:08.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:08.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049375+0000", 2023-11-25T04:24:08.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3455999999999998e-05 2023-11-25T04:24:08.847 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:08.847 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:08.847 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:08.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.847 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:1 3.14 e18/16 3:292cf221:::200.00000000:head v 18'3, mlcod=0'0)", 2023-11-25T04:24:08.847 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:24:08.847 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 81.692859682999995, 2023-11-25T04:24:08.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0060983599999999997, 2023-11-25T04:24:08.848 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:08.848 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:08.848 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:08.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.848 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:08.848 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:24:08.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:08.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.848 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:08.848 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:24:08.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:08.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:08.849 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043248+0000", 2023-11-25T04:24:08.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9399999999999995e-07 2023-11-25T04:24:08.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:08.849 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043252+0000", 2023-11-25T04:24:08.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1799999999999998e-06 2023-11-25T04:24:08.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:08.849 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043259+0000", 2023-11-25T04:24:08.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0049999999999998e-06 2023-11-25T04:24:08.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.850 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:08.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043265+0000", 2023-11-25T04:24:08.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0859999999999998e-06 2023-11-25T04:24:08.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.850 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:08.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043311+0000", 2023-11-25T04:24:08.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6082999999999999e-05 2023-11-25T04:24:08.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.851 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:08.851 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043330+0000", 2023-11-25T04:24:08.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8556e-05 2023-11-25T04:24:08.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.851 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:08.851 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049303+0000", 2023-11-25T04:24:08.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059733470000000004 2023-11-25T04:24:08.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.851 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:08.851 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049346+0000", 2023-11-25T04:24:08.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2608999999999998e-05 2023-11-25T04:24:08.852 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:08.852 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:08.852 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:08.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.852 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:3 3.14 e18/16 3:287034c4:::1.00000000.inode:head v 18'4, mlcod=0'0)", 2023-11-25T04:24:08.852 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:24:08.852 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 81.692343176999998, 2023-11-25T04:24:08.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057040090000000003, 2023-11-25T04:24:08.852 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:08.852 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:08.852 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:08.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:08.853 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:24:08.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:08.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:08.853 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:24:08.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:08.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:08.853 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:24:08.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9799999999999999e-07 2023-11-25T04:24:08.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.854 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:08.854 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043785+0000", 2023-11-25T04:24:08.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0721999999999999e-05 2023-11-25T04:24:08.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.854 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:08.854 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043793+0000", 2023-11-25T04:24:08.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5839999999999995e-06 2023-11-25T04:24:08.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.855 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:08.855 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043803+0000", 2023-11-25T04:24:08.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0729999999999994e-06 2023-11-25T04:24:08.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.855 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:08.855 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043843+0000", 2023-11-25T04:24:08.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0747999999999997e-05 2023-11-25T04:24:08.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.855 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:08.855 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043856+0000", 2023-11-25T04:24:08.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2697000000000001e-05 2023-11-25T04:24:08.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.856 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.856 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:08.856 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049426+0000", 2023-11-25T04:24:08.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055704409999999998 2023-11-25T04:24:08.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.856 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.856 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:08.856 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049468+0000", 2023-11-25T04:24:08.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1446000000000003e-05 2023-11-25T04:24:08.856 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:08.856 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:08.857 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:08.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.857 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "pg_update_log_missing(3.14 epoch 18/16 rep_tid 171 entries 18'5 (0'0) error 3:2e2fa760:::200.00000001:head by mds.0.3:32 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:24:08.857 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:24:08.857 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 81.692182213999999, 2023-11-25T04:24:08.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005556938, 2023-11-25T04:24:08.857 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:08.857 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2023-11-25T04:24:08.857 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:08.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.857 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:08.857 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:24:08.858 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:08.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.858 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.858 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:08.858 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:24:08.858 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:08.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.858 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.858 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:08.858 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:24:08.858 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.03e-07 2023-11-25T04:24:08.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.858 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.859 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:08.859 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043931+0000", 2023-11-25T04:24:08.859 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1079999999999998e-06 2023-11-25T04:24:08.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.859 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.859 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:08.859 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043944+0000", 2023-11-25T04:24:08.859 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2852e-05 2023-11-25T04:24:08.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.859 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.859 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:08.860 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043955+0000", 2023-11-25T04:24:08.860 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1037e-05 2023-11-25T04:24:08.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.860 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.860 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:08.860 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044088+0000", 2023-11-25T04:24:08.860 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013269599999999999 2023-11-25T04:24:08.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.860 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.860 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:08.860 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049482+0000", 2023-11-25T04:24:08.860 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053940419999999999 2023-11-25T04:24:08.860 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:08.861 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:08.861 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:08.861 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.861 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.861 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:10 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'5, mlcod=0'0)", 2023-11-25T04:24:08.861 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:24:08.861 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 81.692092185999996, 2023-11-25T04:24:08.861 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054493099999999997, 2023-11-25T04:24:08.861 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:08.861 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:08.861 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:08.861 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.861 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:08.862 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:24:08.862 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:08.862 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.862 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.862 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:08.862 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:24:08.862 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:08.862 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.862 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.862 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:08.862 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:24:08.862 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6100000000000002e-07 2023-11-25T04:24:08.862 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.862 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.863 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:08.863 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044019+0000", 2023-11-25T04:24:08.863 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.839e-06 2023-11-25T04:24:08.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.863 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:08.863 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044026+0000", 2023-11-25T04:24:08.863 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5869999999999998e-06 2023-11-25T04:24:08.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.863 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.863 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:08.863 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044031+0000", 2023-11-25T04:24:08.863 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4909999999999996e-06 2023-11-25T04:24:08.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.864 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.864 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:08.864 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044070+0000", 2023-11-25T04:24:08.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8829999999999999e-05 2023-11-25T04:24:08.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.864 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.864 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:08.864 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044081+0000", 2023-11-25T04:24:08.864 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1447999999999999e-05 2023-11-25T04:24:08.864 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.864 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.864 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:08.865 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049406+0000", 2023-11-25T04:24:08.865 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053246480000000004 2023-11-25T04:24:08.865 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.865 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.865 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:08.865 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049464+0000", 2023-11-25T04:24:08.865 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8205999999999998e-05 2023-11-25T04:24:08.865 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:08.865 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:08.866 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:08.866 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.866 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.866 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:11 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'6, mlcod=0'0)", 2023-11-25T04:24:08.866 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:24:08.866 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 81.691890921999999, 2023-11-25T04:24:08.866 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052673499999999996, 2023-11-25T04:24:08.866 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:08.866 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:08.866 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:08.866 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.866 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:08.867 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:24:08.867 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:08.867 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.867 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.867 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:08.867 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:24:08.867 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:08.867 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.867 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.867 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:08.867 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:24:08.867 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0199999999999998e-07 2023-11-25T04:24:08.867 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.868 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.868 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:08.868 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044220+0000", 2023-11-25T04:24:08.868 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7720000000000001e-06 2023-11-25T04:24:08.868 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.868 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.868 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:08.868 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044226+0000", 2023-11-25T04:24:08.868 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8579999999999996e-06 2023-11-25T04:24:08.868 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.868 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.868 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:08.868 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044232+0000", 2023-11-25T04:24:08.869 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1700000000000002e-06 2023-11-25T04:24:08.869 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.869 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.869 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:08.869 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044274+0000", 2023-11-25T04:24:08.869 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1254999999999997e-05 2023-11-25T04:24:08.869 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.869 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.869 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:08.869 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:24:08.869 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6880999999999999e-05 2023-11-25T04:24:08.869 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.870 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.870 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:08.870 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049466+0000", 2023-11-25T04:24:08.870 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051659219999999999 2023-11-25T04:24:08.870 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:08.870 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:08.870 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:08.870 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049483+0000", 2023-11-25T04:24:08.870 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.719e-05 2023-11-25T04:24:08.870 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:08.870 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:08.870 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:08.870 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:08.871 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:08.871 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:08.872 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:24:08.873 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:08.869+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:08.973 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:08.971+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:08.985 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:08.985 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:24:08.985 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2023-11-25T04:24:08.985 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:08.994 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:24:09.074 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:09.072+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:09.140 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:09.140 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:24:09.140 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2023-11-25T04:24:09.140 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2023-11-25T04:24:09.140 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:09.149 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:24:09.175 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:09.174+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:09.256 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:09.256 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2023-11-25T04:24:09.256 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2023-11-25T04:24:09.257 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2023-11-25T04:24:09.257 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.257 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:09.257 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:24:09.257 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 82.221836249000006, 2023-11-25T04:24:09.257 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01072943, 2023-11-25T04:24:09.257 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:09.257 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:09.257 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:09.257 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.257 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:09.257 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:24:09.257 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:09.258 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.258 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.258 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:09.258 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:24:09.258 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:09.258 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.258 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.258 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:09.258 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031829+0000", 2023-11-25T04:24:09.258 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.666e-06 2023-11-25T04:24:09.258 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.258 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.259 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:09.259 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031839+0000", 2023-11-25T04:24:09.259 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9490000000000008e-06 2023-11-25T04:24:09.259 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.259 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.259 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:09.259 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031857+0000", 2023-11-25T04:24:09.259 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8093000000000001e-05 2023-11-25T04:24:09.259 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.259 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.259 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:09.259 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031882+0000", 2023-11-25T04:24:09.259 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4294000000000002e-05 2023-11-25T04:24:09.260 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.260 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.260 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:09.260 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033130+0000", 2023-11-25T04:24:09.260 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001248264 2023-11-25T04:24:09.260 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.260 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.260 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:09.260 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033426+0000", 2023-11-25T04:24:09.260 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029620999999999998 2023-11-25T04:24:09.261 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.261 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.261 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:09.261 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033814+0000", 2023-11-25T04:24:09.261 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000387742 2023-11-25T04:24:09.261 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.261 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.261 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:09.261 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036793+0000", 2023-11-25T04:24:09.261 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002979598 2023-11-25T04:24:09.261 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.261 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.262 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:09.262 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042455+0000", 2023-11-25T04:24:09.262 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0056619260000000003 2023-11-25T04:24:09.262 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.262 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.262 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:09.262 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042502+0000", 2023-11-25T04:24:09.262 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7139999999999999e-05 2023-11-25T04:24:09.262 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.262 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.262 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:09.262 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042557+0000", 2023-11-25T04:24:09.263 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4548e-05 2023-11-25T04:24:09.263 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:09.263 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:09.263 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:09.263 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.263 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.263 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:09.263 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:24:09.263 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 82.220637714000006, 2023-11-25T04:24:09.263 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011505202000000001, 2023-11-25T04:24:09.263 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:09.263 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:09.264 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:09.264 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.264 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:09.264 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:24:09.264 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:09.264 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.264 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.264 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:09.264 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:24:09.264 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:09.264 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.264 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.264 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:09.265 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033027+0000", 2023-11-25T04:24:09.265 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0699999999999999e-06 2023-11-25T04:24:09.265 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.266 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.266 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:09.266 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033035+0000", 2023-11-25T04:24:09.266 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0830000000000004e-06 2023-11-25T04:24:09.266 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.266 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.266 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:09.266 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033059+0000", 2023-11-25T04:24:09.266 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3975e-05 2023-11-25T04:24:09.266 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.266 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.267 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:09.267 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033482+0000", 2023-11-25T04:24:09.267 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042253700000000001 2023-11-25T04:24:09.267 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.267 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.267 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:09.267 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034520+0000", 2023-11-25T04:24:09.267 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001037991 2023-11-25T04:24:09.267 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.267 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.268 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:09.268 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034611+0000", 2023-11-25T04:24:09.268 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1113999999999995e-05 2023-11-25T04:24:09.268 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.268 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.268 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:09.268 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034769+0000", 2023-11-25T04:24:09.268 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000157641 2023-11-25T04:24:09.268 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.268 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.268 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:09.268 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038292+0000", 2023-11-25T04:24:09.268 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035231160000000002 2023-11-25T04:24:09.269 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.269 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.269 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:09.269 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044440+0000", 2023-11-25T04:24:09.269 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0061486980000000002 2023-11-25T04:24:09.269 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.269 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.269 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:09.269 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044495+0000", 2023-11-25T04:24:09.269 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4339000000000003e-05 2023-11-25T04:24:09.269 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.269 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.270 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:09.270 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044531+0000", 2023-11-25T04:24:09.270 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6637999999999997e-05 2023-11-25T04:24:09.270 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:09.270 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:09.270 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:09.270 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.270 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.270 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:09.270 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:24:09.270 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 82.220163360000001, 2023-11-25T04:24:09.270 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0057634039999999997, 2023-11-25T04:24:09.271 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:09.271 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:09.271 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:09.271 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.271 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:09.271 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:24:09.271 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:09.271 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.271 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.271 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:09.271 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:24:09.271 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:09.272 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.272 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.272 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:09.272 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:24:09.272 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7900000000000007e-07 2023-11-25T04:24:09.272 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.272 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.272 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:09.272 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033507+0000", 2023-11-25T04:24:09.272 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8900000000000004e-06 2023-11-25T04:24:09.272 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.272 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.272 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:09.272 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033519+0000", 2023-11-25T04:24:09.273 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1756000000000001e-05 2023-11-25T04:24:09.273 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.273 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.273 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:09.273 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033527+0000", 2023-11-25T04:24:09.273 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3610000000000001e-06 2023-11-25T04:24:09.273 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.273 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.273 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:09.273 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034451+0000", 2023-11-25T04:24:09.273 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00092372400000000001 2023-11-25T04:24:09.273 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.274 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.274 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:09.274 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034809+0000", 2023-11-25T04:24:09.274 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00035757699999999998 2023-11-25T04:24:09.274 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.274 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.274 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:09.274 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034965+0000", 2023-11-25T04:24:09.274 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015638699999999999 2023-11-25T04:24:09.274 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.274 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.274 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:09.274 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038299+0000", 2023-11-25T04:24:09.275 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0033336559999999999 2023-11-25T04:24:09.275 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.275 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.275 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:09.275 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039169+0000", 2023-11-25T04:24:09.275 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00087066400000000005 2023-11-25T04:24:09.275 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.276 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.276 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:09.276 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039226+0000", 2023-11-25T04:24:09.276 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6511999999999997e-05 2023-11-25T04:24:09.276 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.276 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.276 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:09.276 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039264+0000", 2023-11-25T04:24:09.276 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7898000000000002e-05 2023-11-25T04:24:09.277 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:09.277 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:09.277 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:09.277 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.277 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.277 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:25 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:09.277 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:24:09.277 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 82.220084775999993, 2023-11-25T04:24:09.277 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096656169999999996, 2023-11-25T04:24:09.277 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:09.277 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:09.277 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:09.278 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.278 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:09.278 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:24:09.278 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:09.278 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.278 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.278 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:09.278 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:24:09.278 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:09.278 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.278 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.279 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:09.279 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033580+0000", 2023-11-25T04:24:09.279 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8199999999999999e-07 2023-11-25T04:24:09.279 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.279 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.279 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:09.279 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033585+0000", 2023-11-25T04:24:09.279 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3680000000000001e-06 2023-11-25T04:24:09.279 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.279 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.280 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:09.280 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033600+0000", 2023-11-25T04:24:09.280 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4331000000000001e-05 2023-11-25T04:24:09.280 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.281 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.281 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:09.281 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033606+0000", 2023-11-25T04:24:09.281 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5780000000000004e-06 2023-11-25T04:24:09.281 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.281 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.281 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:09.281 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035747+0000", 2023-11-25T04:24:09.281 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0021406530000000002 2023-11-25T04:24:09.281 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.281 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.282 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:09.282 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035901+0000", 2023-11-25T04:24:09.282 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000153931 2023-11-25T04:24:09.282 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.282 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.282 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:09.282 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036193+0000", 2023-11-25T04:24:09.282 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029211299999999999 2023-11-25T04:24:09.282 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.282 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.282 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:09.282 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038655+0000", 2023-11-25T04:24:09.283 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0024621640000000002 2023-11-25T04:24:09.283 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.283 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.283 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:09.283 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043130+0000", 2023-11-25T04:24:09.283 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.004474698 2023-11-25T04:24:09.283 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.283 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.283 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:09.283 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043204+0000", 2023-11-25T04:24:09.284 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4091999999999995e-05 2023-11-25T04:24:09.284 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.284 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.284 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:09.284 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043245+0000", 2023-11-25T04:24:09.285 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0907e-05 2023-11-25T04:24:09.285 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:09.285 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:09.285 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:09.285 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.285 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.285 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:26 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:09.285 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:09.285 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 82.220048951999999, 2023-11-25T04:24:09.286 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011086945000000001, 2023-11-25T04:24:09.286 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:09.286 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:09.286 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:09.287 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.287 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:09.287 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:09.287 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:09.287 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.287 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.287 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:09.287 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:09.287 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:09.287 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.288 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.288 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:09.288 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:09.288 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1699999999999998e-07 2023-11-25T04:24:09.288 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.288 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.288 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:09.288 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033627+0000", 2023-11-25T04:24:09.288 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.163e-05 2023-11-25T04:24:09.288 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.288 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.289 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:09.289 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033636+0000", 2023-11-25T04:24:09.289 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6859999999999995e-06 2023-11-25T04:24:09.289 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.289 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.289 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:09.289 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033642+0000", 2023-11-25T04:24:09.289 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3149999999999997e-06 2023-11-25T04:24:09.290 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.290 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.290 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:09.290 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035414+0000", 2023-11-25T04:24:09.290 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0017716380000000001 2023-11-25T04:24:09.290 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.290 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.290 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:09.290 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035539+0000", 2023-11-25T04:24:09.290 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000125477 2023-11-25T04:24:09.290 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.291 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.291 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:09.291 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035672+0000", 2023-11-25T04:24:09.291 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000133316 2023-11-25T04:24:09.291 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.291 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.291 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:09.291 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038649+0000", 2023-11-25T04:24:09.291 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029764679999999999 2023-11-25T04:24:09.291 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.291 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.291 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:09.291 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044578+0000", 2023-11-25T04:24:09.292 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0059293510000000002 2023-11-25T04:24:09.292 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.292 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.292 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:09.292 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044651+0000", 2023-11-25T04:24:09.292 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2513000000000002e-05 2023-11-25T04:24:09.292 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.292 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.292 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:09.292 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044702+0000", 2023-11-25T04:24:09.292 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1034e-05 2023-11-25T04:24:09.292 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:09.293 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:09.293 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:09.293 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.293 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.293 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:29 3.b 3:d0630e4c:::mds0_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:09.293 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:09.293 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 82.220011032000002, 2023-11-25T04:24:09.293 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010980227, 2023-11-25T04:24:09.293 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:09.293 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:09.293 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:09.293 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.294 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:09.294 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:09.294 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:09.294 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.294 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.294 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:09.294 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:09.294 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:09.294 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.294 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.294 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:09.294 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:09.295 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3499999999999996e-07 2023-11-25T04:24:09.295 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.295 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.295 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:09.295 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033658+0000", 2023-11-25T04:24:09.295 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1280000000000001e-06 2023-11-25T04:24:09.295 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.295 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.295 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:09.296 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033667+0000", 2023-11-25T04:24:09.296 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1430000000000004e-06 2023-11-25T04:24:09.296 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.296 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.296 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:09.297 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033678+0000", 2023-11-25T04:24:09.297 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0906e-05 2023-11-25T04:24:09.297 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.297 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.297 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:09.297 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034456+0000", 2023-11-25T04:24:09.297 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00077833899999999996 2023-11-25T04:24:09.297 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.297 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.297 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:09.297 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034575+0000", 2023-11-25T04:24:09.297 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011926000000000001 2023-11-25T04:24:09.297 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.298 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.298 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:09.298 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035001+0000", 2023-11-25T04:24:09.298 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042550999999999998 2023-11-25T04:24:09.298 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.298 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.298 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:09.298 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.037923+0000", 2023-11-25T04:24:09.298 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029219670000000001 2023-11-25T04:24:09.298 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.298 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.298 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:09.299 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044538+0000", 2023-11-25T04:24:09.299 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0066152349999999997 2023-11-25T04:24:09.299 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.299 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.299 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:09.299 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044594+0000", 2023-11-25T04:24:09.299 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5593000000000001e-05 2023-11-25T04:24:09.299 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.299 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.299 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:09.299 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044633+0000", 2023-11-25T04:24:09.299 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9511000000000002e-05 2023-11-25T04:24:09.300 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:09.300 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:09.300 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:09.300 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.300 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.300 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:34 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:09.300 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:24:09.300 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 82.219760918999995, 2023-11-25T04:24:09.300 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010920445000000001, 2023-11-25T04:24:09.300 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:09.300 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:09.300 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:09.300 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.301 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:09.301 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:24:09.301 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:09.301 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.301 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.301 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:09.301 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:24:09.301 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:09.301 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.301 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.301 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:09.301 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033904+0000", 2023-11-25T04:24:09.301 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6599999999999994e-07 2023-11-25T04:24:09.302 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.302 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.302 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:09.302 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033909+0000", 2023-11-25T04:24:09.302 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5639999999999996e-06 2023-11-25T04:24:09.302 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.302 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.302 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:09.302 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033923+0000", 2023-11-25T04:24:09.302 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3028e-05 2023-11-25T04:24:09.302 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.302 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.303 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:09.303 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034043+0000", 2023-11-25T04:24:09.303 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000120328 2023-11-25T04:24:09.303 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.303 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.303 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:09.303 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035576+0000", 2023-11-25T04:24:09.303 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015333720000000001 2023-11-25T04:24:09.303 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.303 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.303 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:09.303 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035699+0000", 2023-11-25T04:24:09.303 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000123001 2023-11-25T04:24:09.303 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.304 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.304 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:09.304 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044823+0000", 2023-11-25T04:24:09.304 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0091241859999999994 2023-11-25T04:24:09.304 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:09.304 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:09.304 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:09.304 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.304 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.304 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:35 3.1f 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:09.304 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:24:09.304 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 82.218456713999998, 2023-11-25T04:24:09.304 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096164779999999995, 2023-11-25T04:24:09.305 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:09.305 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:09.305 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:09.305 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.305 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:09.305 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:24:09.305 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:09.305 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.305 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.305 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:09.305 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:24:09.305 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:09.305 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.305 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.306 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:09.306 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035208+0000", 2023-11-25T04:24:09.306 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2699999999999998e-07 2023-11-25T04:24:09.306 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.306 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.306 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:09.306 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035213+0000", 2023-11-25T04:24:09.306 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.6070000000000001e-06 2023-11-25T04:24:09.306 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.306 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.306 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:09.306 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035350+0000", 2023-11-25T04:24:09.306 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013693599999999999 2023-11-25T04:24:09.306 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.307 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.307 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:09.307 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035358+0000", 2023-11-25T04:24:09.307 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0099999999999995e-06 2023-11-25T04:24:09.307 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.307 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.307 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:09.307 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036260+0000", 2023-11-25T04:24:09.307 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00090240100000000005 2023-11-25T04:24:09.307 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.307 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.307 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:09.307 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036517+0000", 2023-11-25T04:24:09.308 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00025736299999999998 2023-11-25T04:24:09.308 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.308 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.308 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:09.308 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044824+0000", 2023-11-25T04:24:09.308 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0083063340000000003 2023-11-25T04:24:09.308 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:09.308 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:09.308 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:09.308 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.308 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.308 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:37 3.1e 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:09.308 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:24:09.309 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 82.218290027999998, 2023-11-25T04:24:09.309 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0078322640000000002, 2023-11-25T04:24:09.309 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:09.309 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:09.309 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:09.309 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.309 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:09.309 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:24:09.309 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:09.309 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.309 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.309 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:09.309 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:24:09.310 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:09.310 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.310 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.310 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:09.310 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035375+0000", 2023-11-25T04:24:09.310 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9100000000000003e-07 2023-11-25T04:24:09.310 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.310 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.310 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:09.310 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035379+0000", 2023-11-25T04:24:09.310 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9430000000000002e-06 2023-11-25T04:24:09.310 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.310 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.311 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:09.311 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035388+0000", 2023-11-25T04:24:09.311 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2760000000000006e-06 2023-11-25T04:24:09.311 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.311 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.311 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:09.311 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035396+0000", 2023-11-25T04:24:09.311 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8480000000000001e-06 2023-11-25T04:24:09.311 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.311 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.311 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:09.311 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035452+0000", 2023-11-25T04:24:09.311 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5831000000000001e-05 2023-11-25T04:24:09.312 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.312 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.312 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:09.312 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035934+0000", 2023-11-25T04:24:09.312 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00048265599999999998 2023-11-25T04:24:09.312 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.312 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.312 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:09.312 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043206+0000", 2023-11-25T04:24:09.312 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.007271919 2023-11-25T04:24:09.312 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:09.312 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:09.312 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:09.313 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.313 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.313 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing(3.1a epoch 18/16 rep_tid 3 entries 18'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:36 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:24:09.313 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:24:09.313 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 82.217014442000007, 2023-11-25T04:24:09.313 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002073414, 2023-11-25T04:24:09.313 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:09.313 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:24:09.313 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:09.313 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.313 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:09.313 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:24:09.313 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:09.313 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.314 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.314 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:09.314 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:24:09.314 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:09.314 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.314 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.314 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:09.314 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036650+0000", 2023-11-25T04:24:09.314 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7300000000000002e-07 2023-11-25T04:24:09.314 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.314 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.314 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:09.314 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036659+0000", 2023-11-25T04:24:09.315 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.7499999999999992e-06 2023-11-25T04:24:09.315 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.315 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.315 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:09.315 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036673+0000", 2023-11-25T04:24:09.315 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4917e-05 2023-11-25T04:24:09.315 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.315 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.315 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:09.315 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036687+0000", 2023-11-25T04:24:09.315 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3883000000000001e-05 2023-11-25T04:24:09.315 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.316 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.316 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:09.316 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036781+0000", 2023-11-25T04:24:09.316 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3219000000000003e-05 2023-11-25T04:24:09.316 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.316 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.316 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:09.316 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038723+0000", 2023-11-25T04:24:09.316 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019422720000000001 2023-11-25T04:24:09.316 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:09.316 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:09.316 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:09.316 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.316 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.317 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:28 3.13 e18/16 3:cd5a64a3:::100.00000000:head v 18'1, mlcod=0'0)", 2023-11-25T04:24:09.317 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:24:09.317 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 82.213353283999993, 2023-11-25T04:24:09.317 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001850546, 2023-11-25T04:24:09.317 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:09.317 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:09.317 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:09.317 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.317 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:09.317 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:24:09.317 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:09.317 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.317 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.318 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:09.318 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:24:09.318 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:09.318 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.318 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.318 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:09.318 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:24:09.318 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7800000000000002e-07 2023-11-25T04:24:09.318 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.318 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.318 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:09.318 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040331+0000", 2023-11-25T04:24:09.318 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0449999999999999e-05 2023-11-25T04:24:09.319 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.319 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.319 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:09.319 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040339+0000", 2023-11-25T04:24:09.319 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0709999999999995e-06 2023-11-25T04:24:09.319 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.319 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.319 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:09.319 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040347+0000", 2023-11-25T04:24:09.319 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7079999999999997e-06 2023-11-25T04:24:09.319 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.319 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.319 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:09.319 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040410+0000", 2023-11-25T04:24:09.320 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2453000000000007e-05 2023-11-25T04:24:09.320 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.320 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.320 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:09.320 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040422+0000", 2023-11-25T04:24:09.320 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2429000000000001e-05 2023-11-25T04:24:09.320 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.320 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.320 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:09.320 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042129+0000", 2023-11-25T04:24:09.320 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001707266 2023-11-25T04:24:09.320 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.320 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.321 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:09.321 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042161+0000", 2023-11-25T04:24:09.321 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1791000000000001e-05 2023-11-25T04:24:09.321 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:09.321 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:09.321 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:09.321 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.321 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.321 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:27 3.13 e18/16 3:c91d4a1d:::mds0_inotable:head v 18'2, mlcod=0'0)", 2023-11-25T04:24:09.321 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:24:09.321 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 82.213149760999997, 2023-11-25T04:24:09.321 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0016886430000000001, 2023-11-25T04:24:09.321 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:09.321 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:09.322 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:09.322 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.322 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:09.322 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:24:09.322 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:09.322 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.322 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.322 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:09.322 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:24:09.322 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:09.322 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.322 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.322 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:09.323 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:24:09.323 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3999999999999997e-07 2023-11-25T04:24:09.323 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.323 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.323 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:09.323 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040520+0000", 2023-11-25T04:24:09.323 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2989999999999996e-06 2023-11-25T04:24:09.323 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.323 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.323 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:09.323 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040527+0000", 2023-11-25T04:24:09.323 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5730000000000002e-06 2023-11-25T04:24:09.323 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.324 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.324 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:09.324 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040535+0000", 2023-11-25T04:24:09.324 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8630000000000008e-06 2023-11-25T04:24:09.324 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.324 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.324 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:09.324 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040683+0000", 2023-11-25T04:24:09.324 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000148002 2023-11-25T04:24:09.324 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.324 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.324 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:09.324 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040700+0000", 2023-11-25T04:24:09.325 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7065e-05 2023-11-25T04:24:09.325 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.325 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.325 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:09.325 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042163+0000", 2023-11-25T04:24:09.325 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001462767 2023-11-25T04:24:09.325 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.325 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.325 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:09.325 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042203+0000", 2023-11-25T04:24:09.325 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9734000000000003e-05 2023-11-25T04:24:09.325 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:09.325 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:09.326 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:09.326 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.326 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.326 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:20 3.1c e18/16 3:3ed09add:::607.00000000:head v 18'1, mlcod=0'0)", 2023-11-25T04:24:09.326 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:24:09.326 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 82.213114719999993, 2023-11-25T04:24:09.326 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015271, 2023-11-25T04:24:09.326 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:09.326 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:09.326 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:09.326 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.326 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:09.327 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:24:09.327 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:09.327 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.327 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.327 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:09.327 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:24:09.327 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:09.327 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.327 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.327 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:09.327 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:24:09.327 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4799999999999999e-07 2023-11-25T04:24:09.327 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.328 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.328 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:09.328 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040561+0000", 2023-11-25T04:24:09.328 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1800000000000001e-05 2023-11-25T04:24:09.328 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.328 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.328 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:09.328 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040568+0000", 2023-11-25T04:24:09.328 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7730000000000002e-06 2023-11-25T04:24:09.328 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.328 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.328 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:09.328 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040587+0000", 2023-11-25T04:24:09.329 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8691000000000001e-05 2023-11-25T04:24:09.329 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.329 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.329 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:09.329 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040638+0000", 2023-11-25T04:24:09.329 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1068e-05 2023-11-25T04:24:09.329 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.329 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.329 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:09.329 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040670+0000", 2023-11-25T04:24:09.329 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2125999999999997e-05 2023-11-25T04:24:09.329 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.329 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.330 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:09.330 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042032+0000", 2023-11-25T04:24:09.330 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001361821 2023-11-25T04:24:09.330 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.330 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.330 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:09.330 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042076+0000", 2023-11-25T04:24:09.330 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4472999999999999e-05 2023-11-25T04:24:09.330 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:09.330 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:09.330 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:09.330 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.330 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.330 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:21 3.1c e18/16 3:3ed09add:::607.00000000:head v 18'2, mlcod=0'0)", 2023-11-25T04:24:09.331 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:24:09.331 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 82.212879994999994, 2023-11-25T04:24:09.331 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0013228160000000001, 2023-11-25T04:24:09.331 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:09.331 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:09.331 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:09.331 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.331 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:09.331 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:24:09.331 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:09.331 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.331 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.331 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:09.332 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:24:09.332 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:09.332 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.332 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.332 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:09.332 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:24:09.332 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9299999999999999e-07 2023-11-25T04:24:09.332 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.332 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.332 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:09.332 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040790+0000", 2023-11-25T04:24:09.332 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5770000000000001e-06 2023-11-25T04:24:09.332 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.333 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.333 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:09.333 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040799+0000", 2023-11-25T04:24:09.333 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6439999999999995e-06 2023-11-25T04:24:09.333 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.333 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.333 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:09.333 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040806+0000", 2023-11-25T04:24:09.333 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0180000000000004e-06 2023-11-25T04:24:09.333 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.333 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.333 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:09.333 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040907+0000", 2023-11-25T04:24:09.333 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000100096 2023-11-25T04:24:09.334 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.334 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.334 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:09.334 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040925+0000", 2023-11-25T04:24:09.334 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8536999999999999e-05 2023-11-25T04:24:09.334 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.334 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.334 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:09.334 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042079+0000", 2023-11-25T04:24:09.334 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00115358 2023-11-25T04:24:09.334 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.334 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.335 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:09.335 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042107+0000", 2023-11-25T04:24:09.335 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7971e-05 2023-11-25T04:24:09.335 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:09.335 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:09.335 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:09.335 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.335 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.335 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop_reply(mds.0.3:4 3.1f e18/16 ondisk, result = 0)", 2023-11-25T04:24:09.335 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:24:09.335 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 82.211540604999996, 2023-11-25T04:24:09.335 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00044092900000000001, 2023-11-25T04:24:09.335 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:09.335 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:09.336 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:09.336 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.336 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:09.336 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:24:09.336 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:09.336 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.336 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.336 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:09.336 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:24:09.336 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:09.336 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.336 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.336 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:09.336 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042124+0000", 2023-11-25T04:24:09.337 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1800000000000005e-07 2023-11-25T04:24:09.337 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.337 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.337 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:09.337 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042128+0000", 2023-11-25T04:24:09.337 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2289999999999997e-06 2023-11-25T04:24:09.337 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.337 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.337 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:09.337 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042138+0000", 2023-11-25T04:24:09.337 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9729999999999992e-06 2023-11-25T04:24:09.337 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.337 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.338 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:09.338 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042145+0000", 2023-11-25T04:24:09.338 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.336e-06 2023-11-25T04:24:09.338 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.338 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.338 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:09.338 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042441+0000", 2023-11-25T04:24:09.338 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029650500000000002 2023-11-25T04:24:09.338 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.338 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.338 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:09.338 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042448+0000", 2023-11-25T04:24:09.338 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4100000000000002e-06 2023-11-25T04:24:09.339 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.339 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.339 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:09.339 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042564+0000", 2023-11-25T04:24:09.339 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000115758 2023-11-25T04:24:09.339 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:09.339 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:09.339 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:09.339 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.339 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.339 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing_reply(3.1f epoch 18/16 rep_tid 8 lcod 18'4)", 2023-11-25T04:24:09.339 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:24:09.339 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 82.209215193000006, 2023-11-25T04:24:09.340 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00038154699999999998, 2023-11-25T04:24:09.340 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:09.340 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:24:09.340 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:09.340 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.340 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:09.340 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:24:09.340 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:09.340 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.340 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.340 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:09.340 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:24:09.340 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:09.341 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.341 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.341 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:09.341 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:24:09.341 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2700000000000001e-07 2023-11-25T04:24:09.341 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.341 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.341 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:09.341 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044451+0000", 2023-11-25T04:24:09.341 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7460000000000001e-06 2023-11-25T04:24:09.341 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.341 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.341 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:09.342 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044455+0000", 2023-11-25T04:24:09.342 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5809999999999998e-06 2023-11-25T04:24:09.342 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.342 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.342 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:09.342 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044458+0000", 2023-11-25T04:24:09.342 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0170000000000001e-06 2023-11-25T04:24:09.342 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.342 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.342 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:09.342 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044734+0000", 2023-11-25T04:24:09.342 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00027582799999999999 2023-11-25T04:24:09.342 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.343 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.343 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:09.343 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044830+0000", 2023-11-25T04:24:09.343 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6248000000000004e-05 2023-11-25T04:24:09.343 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:09.343 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:09.343 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:09.343 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.343 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.343 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing_reply(3.b epoch 18/16 rep_tid 5 lcod 18'2)", 2023-11-25T04:24:09.343 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:24:09.343 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 82.209178531999996, 2023-11-25T04:24:09.343 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000344488, 2023-11-25T04:24:09.344 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:09.344 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:24:09.344 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:09.344 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.344 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:09.344 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:24:09.344 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:09.344 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.344 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.344 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:09.344 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:24:09.344 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:09.344 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.344 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.345 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:09.345 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044485+0000", 2023-11-25T04:24:09.345 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.55e-07 2023-11-25T04:24:09.345 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.345 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.345 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:09.345 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044487+0000", 2023-11-25T04:24:09.345 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2470000000000001e-06 2023-11-25T04:24:09.345 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.345 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.345 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:09.345 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044491+0000", 2023-11-25T04:24:09.345 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9659999999999998e-06 2023-11-25T04:24:09.346 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.346 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.346 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:09.346 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044494+0000", 2023-11-25T04:24:09.346 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2310000000000001e-06 2023-11-25T04:24:09.346 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.346 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.346 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:09.346 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044756+0000", 2023-11-25T04:24:09.346 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00026172800000000003 2023-11-25T04:24:09.346 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.346 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.346 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:09.347 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044830+0000", 2023-11-25T04:24:09.347 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4160999999999997e-05 2023-11-25T04:24:09.347 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:09.347 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:09.347 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:09.347 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.347 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.347 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:39 3.6 3:654134d2:::mds0_openfiles.0:head [omap-clear,omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2023-11-25T04:24:09.347 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:24:09.347 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 77.149352489999998, 2023-11-25T04:24:09.347 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001788658, 2023-11-25T04:24:09.347 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:09.347 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:09.347 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:09.348 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.348 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:09.348 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:24:09.348 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:09.348 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.348 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.348 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:09.348 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:24:09.348 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:09.348 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.348 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.348 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:09.348 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104314+0000", 2023-11-25T04:24:09.349 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3099999999999999e-06 2023-11-25T04:24:09.349 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.349 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.349 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:09.349 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104325+0000", 2023-11-25T04:24:09.349 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1511999999999999e-05 2023-11-25T04:24:09.349 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.349 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.349 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:09.349 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104362+0000", 2023-11-25T04:24:09.349 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6903000000000002e-05 2023-11-25T04:24:09.349 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.349 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.349 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:09.350 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104387+0000", 2023-11-25T04:24:09.350 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4766e-05 2023-11-25T04:24:09.350 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.350 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.350 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:09.350 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104465+0000", 2023-11-25T04:24:09.350 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8119000000000002e-05 2023-11-25T04:24:09.350 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.350 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.350 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:09.350 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104557+0000", 2023-11-25T04:24:09.350 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2496000000000003e-05 2023-11-25T04:24:09.351 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.351 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.351 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:09.351 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104646+0000", 2023-11-25T04:24:09.351 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8818000000000002e-05 2023-11-25T04:24:09.351 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.351 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.351 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:09.351 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105201+0000", 2023-11-25T04:24:09.351 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00055506000000000002 2023-11-25T04:24:09.351 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.351 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.351 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:09.352 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105994+0000", 2023-11-25T04:24:09.352 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00079294700000000001 2023-11-25T04:24:09.352 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.352 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.352 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:09.352 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.106055+0000", 2023-11-25T04:24:09.352 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0566000000000003e-05 2023-11-25T04:24:09.352 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.352 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.352 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:09.352 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.106100+0000", 2023-11-25T04:24:09.352 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5160999999999997e-05 2023-11-25T04:24:09.352 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:09.353 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:09.353 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:09.353 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.353 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.353 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:43 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2023-11-25T04:24:09.353 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:24:09.353 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 62.149127493999998, 2023-11-25T04:24:09.353 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001751166, 2023-11-25T04:24:09.353 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:09.353 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:09.353 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:09.353 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.353 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:09.354 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:24:09.354 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:09.354 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.354 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.354 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:09.354 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:24:09.354 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:09.354 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.354 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.354 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:09.354 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104539+0000", 2023-11-25T04:24:09.354 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9529999999999999e-06 2023-11-25T04:24:09.354 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.355 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.355 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:09.355 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104557+0000", 2023-11-25T04:24:09.355 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7464000000000001e-05 2023-11-25T04:24:09.355 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.355 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.355 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:09.355 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104590+0000", 2023-11-25T04:24:09.355 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2966000000000003e-05 2023-11-25T04:24:09.355 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.355 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.355 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:09.356 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104608+0000", 2023-11-25T04:24:09.356 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8193e-05 2023-11-25T04:24:09.356 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.356 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.356 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:09.356 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104767+0000", 2023-11-25T04:24:09.356 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015882 2023-11-25T04:24:09.356 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.356 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.356 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:09.356 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104909+0000", 2023-11-25T04:24:09.356 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014244999999999999 2023-11-25T04:24:09.357 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.357 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.357 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:09.357 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.105059+0000", 2023-11-25T04:24:09.357 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014964399999999999 2023-11-25T04:24:09.357 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.357 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.357 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:09.357 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.105701+0000", 2023-11-25T04:24:09.357 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00064201099999999995 2023-11-25T04:24:09.357 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.357 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.357 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:09.358 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106233+0000", 2023-11-25T04:24:09.358 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00053178500000000001 2023-11-25T04:24:09.358 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.358 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.358 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:09.358 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106265+0000", 2023-11-25T04:24:09.358 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1986000000000002e-05 2023-11-25T04:24:09.358 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.358 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.358 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:09.358 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106288+0000", 2023-11-25T04:24:09.358 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2894000000000001e-05 2023-11-25T04:24:09.359 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:09.359 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:09.359 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:09.359 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.359 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.359 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:45 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e23)", 2023-11-25T04:24:09.359 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:24:09.359 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 57.148929453999997, 2023-11-25T04:24:09.359 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00098423299999999994, 2023-11-25T04:24:09.359 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:09.359 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:09.359 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:09.359 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.360 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:09.360 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:24:09.360 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:09.360 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.360 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.360 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:09.360 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:24:09.360 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:09.360 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.360 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.360 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:09.360 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104737+0000", 2023-11-25T04:24:09.360 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3309999999999998e-06 2023-11-25T04:24:09.361 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.361 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.361 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:09.361 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104748+0000", 2023-11-25T04:24:09.361 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1080999999999999e-05 2023-11-25T04:24:09.361 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.361 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.361 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:09.361 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104771+0000", 2023-11-25T04:24:09.361 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2707000000000001e-05 2023-11-25T04:24:09.361 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.361 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.361 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:09.362 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104800+0000", 2023-11-25T04:24:09.362 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9595e-05 2023-11-25T04:24:09.362 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.362 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.362 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:09.362 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104890+0000", 2023-11-25T04:24:09.362 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.9808999999999993e-05 2023-11-25T04:24:09.362 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.362 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.362 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:09.362 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105004+0000", 2023-11-25T04:24:09.362 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000114508 2023-11-25T04:24:09.362 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.362 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.363 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:09.363 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105634+0000", 2023-11-25T04:24:09.363 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00062985799999999996 2023-11-25T04:24:09.363 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.363 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.363 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:09.363 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105671+0000", 2023-11-25T04:24:09.363 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6640000000000002e-05 2023-11-25T04:24:09.363 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:09.363 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:09.363 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:09.363 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105719+0000", 2023-11-25T04:24:09.363 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7704000000000003e-05 2023-11-25T04:24:09.364 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:09.364 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:09.364 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:09.364 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:09.364 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:09.364 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:09.365 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:24:09.366 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:09.364+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:09.459 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-11-25T04:24:09.466 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:09.463+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:09.486 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:09.486 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:24:09.486 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2023-11-25T04:24:09.486 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:09.497 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:24:09.581 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:09.579+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:09.582 INFO:teuthology.orchestra.run.smithi067.stderr:Error ENXIO: problem getting command descriptions from osd.0 2023-11-25T04:24:09.601 INFO:teuthology.orchestra.run.smithi067.stdout:osd.1: {} 2023-11-25T04:24:09.601 INFO:teuthology.orchestra.run.smithi067.stderr:osd.1: osd_enable_op_tracker = '' 2023-11-25T04:24:09.620 INFO:teuthology.orchestra.run.smithi067.stdout:osd.2: {} 2023-11-25T04:24:09.620 INFO:teuthology.orchestra.run.smithi067.stderr:osd.2: osd_enable_op_tracker = '' 2023-11-25T04:24:09.623 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-11-25T04:24:09.623 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-11-25T04:24:09.647 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:09.647 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:09.655 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:24:09.698 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:09.696+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:09.769 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:09.767+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:09.802 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:09.802 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:09.812 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:24:09.870 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:09.869+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:09.920 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:09.920 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:09.929 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:24:09.971 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:09.970+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:10.034 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:10.034 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:10.043 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:24:10.050 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2023-11-25T04:24:10.087 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:10.086+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:10.150 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:10.150 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:10.159 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:24:10.203 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:10.201+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:10.266 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:10.267 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:10.274 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:10.271+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:10.276 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:24:10.376 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:10.374+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:10.422 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:10.422 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:10.430 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:24:10.476 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:10.475+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:10.576 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:10.576 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:10.577 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:10.575+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:10.585 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:24:10.677 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:10.676+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:10.695 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:10.695 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:10.705 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:24:10.779 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:10.777+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:10.851 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:10.851 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:10.860 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:24:10.904 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:10.903+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:10.967 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:10.967 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:10.976 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:24:11.021 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:11.019+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:11.053 INFO:teuthology.orchestra.run.smithi067.stderr:noscrub is unset 2023-11-25T04:24:11.082 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:11.081+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:11.083 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:11.083 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:11.092 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:24:11.183 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:11.182+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:11.238 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:11.238 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:11.246 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:24:11.290 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:11.289+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:11.352 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:11.352 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:11.360 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:24:11.404 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:11.403+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:11.468 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:11.468 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:11.477 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:24:11.518 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:11.516+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:11.586 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:11.584+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:11.622 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:11.623 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:11.624 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-11-25T04:24:11.631 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:24:11.715 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:11.714+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:11.742 INFO:teuthology.orchestra.run.smithi067.stderr:Error ENXIO: problem getting command descriptions from osd.0 2023-11-25T04:24:11.760 INFO:teuthology.orchestra.run.smithi067.stdout:osd.1: {} 2023-11-25T04:24:11.760 INFO:teuthology.orchestra.run.smithi067.stderr:osd.1: osd_enable_op_tracker = '' 2023-11-25T04:24:11.776 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:11.776 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:11.779 INFO:teuthology.orchestra.run.smithi067.stdout:osd.2: {} 2023-11-25T04:24:11.779 INFO:teuthology.orchestra.run.smithi067.stderr:osd.2: osd_enable_op_tracker = '' 2023-11-25T04:24:11.782 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-11-25T04:24:11.782 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-11-25T04:24:11.785 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:24:11.826 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:11.825+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:11.888 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:11.887+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:11.892 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:11.893 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:24:11.893 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2023-11-25T04:24:11.893 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:11.900 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:24:11.989 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:11.988+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:12.004 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:12.005 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:24:12.005 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2023-11-25T04:24:12.005 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2023-11-25T04:24:12.005 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:12.013 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:24:12.090 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:12.089+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:12.121 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:12.122 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2023-11-25T04:24:12.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2023-11-25T04:24:12.122 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2023-11-25T04:24:12.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.122 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:155 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'109, mlcod=11'105)", 2023-11-25T04:24:12.122 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:24:12.122 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 94.334530091999994, 2023-11-25T04:24:12.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061719019999999999, 2023-11-25T04:24:12.122 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:12.123 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:12.123 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:12.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.123 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:12.123 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:24:12.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:12.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.123 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:12.123 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:24:12.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:12.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.124 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:12.124 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784884+0000", 2023-11-25T04:24:12.124 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3099999999999999e-07 2023-11-25T04:24:12.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.124 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:12.124 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784942+0000", 2023-11-25T04:24:12.124 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8570000000000003e-05 2023-11-25T04:24:12.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.124 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:12.124 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784949+0000", 2023-11-25T04:24:12.125 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0310000000000001e-06 2023-11-25T04:24:12.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.125 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:12.125 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784953+0000", 2023-11-25T04:24:12.125 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6320000000000001e-06 2023-11-25T04:24:12.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.125 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:12.125 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784993+0000", 2023-11-25T04:24:12.125 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9737000000000003e-05 2023-11-25T04:24:12.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.126 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:12.126 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785011+0000", 2023-11-25T04:24:12.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8349999999999999e-05 2023-11-25T04:24:12.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.126 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:12.126 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791024+0000", 2023-11-25T04:24:12.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006012997 2023-11-25T04:24:12.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.126 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:12.126 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791055+0000", 2023-11-25T04:24:12.127 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1254000000000003e-05 2023-11-25T04:24:12.127 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:12.127 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:12.127 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:12.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.127 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:156 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'110, mlcod=11'105)", 2023-11-25T04:24:12.127 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:24:12.127 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 94.333944043000002, 2023-11-25T04:24:12.127 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005616116, 2023-11-25T04:24:12.127 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:12.127 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:12.128 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:12.128 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.128 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:12.128 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:24:12.128 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:12.128 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.128 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.128 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:12.128 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:24:12.128 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:12.128 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.128 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.129 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:12.129 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785470+0000", 2023-11-25T04:24:12.129 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0699999999999998e-07 2023-11-25T04:24:12.129 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.129 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.129 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:12.129 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785474+0000", 2023-11-25T04:24:12.129 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.048e-06 2023-11-25T04:24:12.129 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.129 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.129 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:12.129 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785480+0000", 2023-11-25T04:24:12.129 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7729999999999996e-06 2023-11-25T04:24:12.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.130 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.130 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:12.130 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785483+0000", 2023-11-25T04:24:12.130 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1970000000000001e-06 2023-11-25T04:24:12.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.130 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.130 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:12.130 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787936+0000", 2023-11-25T04:24:12.130 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024529040000000001 2023-11-25T04:24:12.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.130 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:12.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787947+0000", 2023-11-25T04:24:12.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1669e-05 2023-11-25T04:24:12.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:12.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791057+0000", 2023-11-25T04:24:12.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031095200000000002 2023-11-25T04:24:12.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:12.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791086+0000", 2023-11-25T04:24:12.132 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8697999999999999e-05 2023-11-25T04:24:12.132 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:12.132 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:12.132 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:12.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.132 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:157 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'111, mlcod=11'106)", 2023-11-25T04:24:12.132 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:24:12.132 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 94.333351484999994, 2023-11-25T04:24:12.132 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050590280000000001, 2023-11-25T04:24:12.133 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:12.133 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:12.133 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:12.133 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.133 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:12.133 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:24:12.133 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:12.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.133 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.133 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:12.133 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:24:12.133 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:12.134 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.134 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.134 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:12.134 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786063+0000", 2023-11-25T04:24:12.134 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2600000000000002e-07 2023-11-25T04:24:12.134 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.134 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.134 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:12.134 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786069+0000", 2023-11-25T04:24:12.134 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8839999999999999e-06 2023-11-25T04:24:12.134 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.134 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.134 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:12.135 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786079+0000", 2023-11-25T04:24:12.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0124e-05 2023-11-25T04:24:12.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.135 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:12.135 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786085+0000", 2023-11-25T04:24:12.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6660000000000002e-06 2023-11-25T04:24:12.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.135 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:12.135 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788160+0000", 2023-11-25T04:24:12.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020747669999999999 2023-11-25T04:24:12.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:12.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788180+0000", 2023-11-25T04:24:12.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0208999999999999e-05 2023-11-25T04:24:12.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:12.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791087+0000", 2023-11-25T04:24:12.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029064630000000002 2023-11-25T04:24:12.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:12.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791121+0000", 2023-11-25T04:24:12.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4289000000000003e-05 2023-11-25T04:24:12.137 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:12.137 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:12.137 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:12.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.137 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:158 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'112, mlcod=11'106)", 2023-11-25T04:24:12.137 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:24:12.137 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 94.332935092, 2023-11-25T04:24:12.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046663030000000001, 2023-11-25T04:24:12.137 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:12.137 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:12.138 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:12.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.138 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:12.138 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:24:12.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:12.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.138 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:12.138 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:24:12.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:12.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.139 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:12.139 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786479+0000", 2023-11-25T04:24:12.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2799999999999996e-07 2023-11-25T04:24:12.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.139 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:12.139 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786498+0000", 2023-11-25T04:24:12.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9069e-05 2023-11-25T04:24:12.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.140 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:12.140 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786508+0000", 2023-11-25T04:24:12.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5810000000000003e-06 2023-11-25T04:24:12.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.140 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:12.140 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786512+0000", 2023-11-25T04:24:12.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4889999999999997e-06 2023-11-25T04:24:12.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.140 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:12.140 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788438+0000", 2023-11-25T04:24:12.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019257059999999999 2023-11-25T04:24:12.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.141 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.141 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:12.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788449+0000", 2023-11-25T04:24:12.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1419e-05 2023-11-25T04:24:12.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.141 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.141 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:12.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791122+0000", 2023-11-25T04:24:12.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002672822 2023-11-25T04:24:12.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.142 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:12.142 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791145+0000", 2023-11-25T04:24:12.142 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2589e-05 2023-11-25T04:24:12.142 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:12.142 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:12.142 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:12.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.142 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:206 1.0 e12/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 12'152, mlcod=12'148)", 2023-11-25T04:24:12.142 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:24:12.142 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 93.821166622000007, 2023-11-25T04:24:12.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046728189999999999, 2023-11-25T04:24:12.143 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:12.143 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:12.143 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:12.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:12.143 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:24:12.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:12.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:12.143 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:24:12.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:12.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:12.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298248+0000", 2023-11-25T04:24:12.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3600000000000003e-07 2023-11-25T04:24:12.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:12.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298278+0000", 2023-11-25T04:24:12.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0474e-05 2023-11-25T04:24:12.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:12.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298288+0000", 2023-11-25T04:24:12.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0156000000000001e-05 2023-11-25T04:24:12.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:12.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298293+0000", 2023-11-25T04:24:12.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7400000000000004e-06 2023-11-25T04:24:12.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:12.146 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298333+0000", 2023-11-25T04:24:12.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9666000000000003e-05 2023-11-25T04:24:12.146 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.146 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:12.146 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298351+0000", 2023-11-25T04:24:12.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8544999999999999e-05 2023-11-25T04:24:12.146 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.146 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:12.146 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302891+0000", 2023-11-25T04:24:12.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045398519999999996 2023-11-25T04:24:12.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.147 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:12.147 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302920+0000", 2023-11-25T04:24:12.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8750000000000001e-05 2023-11-25T04:24:12.147 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:12.147 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:12.147 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:12.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.147 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:14 3.15 3:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:12.147 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:24:12.148 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 85.086032265, 2023-11-25T04:24:12.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010990437, 2023-11-25T04:24:12.148 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:12.148 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:12.148 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:12.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.148 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:12.148 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:24:12.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:12.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:12.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:24:12.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:12.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:12.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033383+0000", 2023-11-25T04:24:12.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9870000000000002e-06 2023-11-25T04:24:12.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:12.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033398+0000", 2023-11-25T04:24:12.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4569999999999999e-05 2023-11-25T04:24:12.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:12.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033411+0000", 2023-11-25T04:24:12.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3543999999999999e-05 2023-11-25T04:24:12.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:12.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033423+0000", 2023-11-25T04:24:12.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1905999999999999e-05 2023-11-25T04:24:12.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.151 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:12.151 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034433+0000", 2023-11-25T04:24:12.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00101022 2023-11-25T04:24:12.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.151 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:12.151 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034539+0000", 2023-11-25T04:24:12.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010597199999999999 2023-11-25T04:24:12.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.151 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:12.152 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034667+0000", 2023-11-25T04:24:12.152 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012765500000000001 2023-11-25T04:24:12.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.152 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:12.152 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037803+0000", 2023-11-25T04:24:12.152 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031362220000000001 2023-11-25T04:24:12.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.152 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:12.152 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:24:12.152 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0064964669999999997 2023-11-25T04:24:12.152 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:12.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044331+0000", 2023-11-25T04:24:12.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1544999999999997e-05 2023-11-25T04:24:12.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:12.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044372+0000", 2023-11-25T04:24:12.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0349000000000003e-05 2023-11-25T04:24:12.153 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:12.153 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:12.153 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:12.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.154 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:15 3.15 3:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:12.154 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:24:12.154 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 85.085372614999997, 2023-11-25T04:24:12.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010464362, 2023-11-25T04:24:12.154 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:12.154 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:12.154 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:12.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:12.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:24:12.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:12.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.155 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:12.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:24:12.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:12.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.155 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:12.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034042+0000", 2023-11-25T04:24:12.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0999999999999997e-07 2023-11-25T04:24:12.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.155 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:12.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034048+0000", 2023-11-25T04:24:12.156 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0390000000000001e-06 2023-11-25T04:24:12.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.156 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:12.156 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034059+0000", 2023-11-25T04:24:12.156 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1182999999999999e-05 2023-11-25T04:24:12.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.156 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:12.156 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034067+0000", 2023-11-25T04:24:12.156 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1559999999999995e-06 2023-11-25T04:24:12.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.157 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:12.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035005+0000", 2023-11-25T04:24:12.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00093747399999999999 2023-11-25T04:24:12.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.157 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:12.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035077+0000", 2023-11-25T04:24:12.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2747e-05 2023-11-25T04:24:12.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.157 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:12.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035219+0000", 2023-11-25T04:24:12.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000141877 2023-11-25T04:24:12.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:12.158 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037810+0000", 2023-11-25T04:24:12.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002590752 2023-11-25T04:24:12.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:12.158 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044437+0000", 2023-11-25T04:24:12.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066270110000000004 2023-11-25T04:24:12.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.159 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:12.159 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044481+0000", 2023-11-25T04:24:12.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4106000000000001e-05 2023-11-25T04:24:12.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.159 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:12.159 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044505+0000", 2023-11-25T04:24:12.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4207e-05 2023-11-25T04:24:12.159 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:12.159 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:12.159 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:12.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.160 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.160 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:16 3.1d 3:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:12.160 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:24:12.160 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 85.085330767000002, 2023-11-25T04:24:12.160 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010413923, 2023-11-25T04:24:12.160 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:12.160 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:12.160 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:12.160 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.160 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:12.160 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:24:12.160 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:12.160 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.161 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.161 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:12.161 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:24:12.161 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:12.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.161 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.161 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:12.161 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:24:12.161 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4499999999999997e-07 2023-11-25T04:24:12.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.161 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.161 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:12.162 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034087+0000", 2023-11-25T04:24:12.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9770000000000002e-06 2023-11-25T04:24:12.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.162 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:12.162 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034094+0000", 2023-11-25T04:24:12.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8809999999999998e-06 2023-11-25T04:24:12.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.162 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:12.162 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034100+0000", 2023-11-25T04:24:12.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8440000000000003e-06 2023-11-25T04:24:12.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.163 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.163 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:12.163 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034993+0000", 2023-11-25T04:24:12.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00089336799999999996 2023-11-25T04:24:12.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.163 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.163 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:12.163 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035099+0000", 2023-11-25T04:24:12.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106037 2023-11-25T04:24:12.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.163 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.163 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:12.164 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035276+0000", 2023-11-25T04:24:12.164 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017631799999999999 2023-11-25T04:24:12.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.164 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.164 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:12.164 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037815+0000", 2023-11-25T04:24:12.164 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025395959999999999 2023-11-25T04:24:12.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.164 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.164 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:12.164 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044420+0000", 2023-11-25T04:24:12.164 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066045749999999997 2023-11-25T04:24:12.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.165 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.165 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:12.165 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044463+0000", 2023-11-25T04:24:12.165 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3205999999999999e-05 2023-11-25T04:24:12.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.165 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.165 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:12.165 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044497+0000", 2023-11-25T04:24:12.165 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3476000000000003e-05 2023-11-25T04:24:12.165 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:12.165 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:12.165 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:12.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.166 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:17 3.1d 3:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:12.166 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:24:12.166 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 85.085305558000002, 2023-11-25T04:24:12.166 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01127356, 2023-11-25T04:24:12.166 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:12.166 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:12.166 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:12.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.166 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:12.167 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:24:12.167 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:12.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.167 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:12.167 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:24:12.167 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:12.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.167 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:12.167 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:24:12.167 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1100000000000001e-07 2023-11-25T04:24:12.168 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.168 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.168 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:12.168 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034112+0000", 2023-11-25T04:24:12.168 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8600000000000003e-06 2023-11-25T04:24:12.168 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.168 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.168 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:12.168 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034120+0000", 2023-11-25T04:24:12.168 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7700000000000001e-06 2023-11-25T04:24:12.169 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.169 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.169 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:12.169 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034127+0000", 2023-11-25T04:24:12.169 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0999999999999998e-06 2023-11-25T04:24:12.169 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.169 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.169 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:12.169 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035601+0000", 2023-11-25T04:24:12.169 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014739410000000001 2023-11-25T04:24:12.169 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.169 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.170 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:12.170 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035685+0000", 2023-11-25T04:24:12.170 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3677999999999994e-05 2023-11-25T04:24:12.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.170 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:12.170 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035827+0000", 2023-11-25T04:24:12.170 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014232599999999999 2023-11-25T04:24:12.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.170 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:12.170 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038725+0000", 2023-11-25T04:24:12.170 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028982779999999998 2023-11-25T04:24:12.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.171 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:12.171 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045323+0000", 2023-11-25T04:24:12.171 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0065980780000000003 2023-11-25T04:24:12.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.171 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:12.171 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045354+0000", 2023-11-25T04:24:12.171 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0463e-05 2023-11-25T04:24:12.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.171 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:12.172 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045381+0000", 2023-11-25T04:24:12.172 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7654999999999999e-05 2023-11-25T04:24:12.172 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:12.172 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:12.172 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:12.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.173 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:22 3.e 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:12.173 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:24:12.173 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 85.085279145000001, 2023-11-25T04:24:12.173 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010133422, 2023-11-25T04:24:12.173 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:12.173 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:12.173 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:12.173 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.173 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:12.173 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:24:12.173 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:12.173 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.174 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.174 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:12.174 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:24:12.174 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:12.174 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.174 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.174 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:12.174 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034135+0000", 2023-11-25T04:24:12.174 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-07 2023-11-25T04:24:12.174 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.174 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.174 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:12.175 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034138+0000", 2023-11-25T04:24:12.175 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6009999999999999e-06 2023-11-25T04:24:12.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.175 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.175 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:12.175 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034145+0000", 2023-11-25T04:24:12.175 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1169999999999998e-06 2023-11-25T04:24:12.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.175 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.175 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:12.175 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034150+0000", 2023-11-25T04:24:12.175 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9080000000000003e-06 2023-11-25T04:24:12.176 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.176 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.176 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:12.176 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034209+0000", 2023-11-25T04:24:12.176 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8149000000000002e-05 2023-11-25T04:24:12.176 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.176 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.176 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:12.176 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034415+0000", 2023-11-25T04:24:12.177 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020637699999999999 2023-11-25T04:24:12.177 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.177 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.177 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:12.177 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034584+0000", 2023-11-25T04:24:12.177 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016870399999999999 2023-11-25T04:24:12.177 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.177 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.177 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:12.177 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.036979+0000", 2023-11-25T04:24:12.177 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023957509999999998 2023-11-25T04:24:12.177 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.177 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.178 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:12.178 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044161+0000", 2023-11-25T04:24:12.178 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0071811590000000003 2023-11-25T04:24:12.178 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.178 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.178 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:12.178 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044224+0000", 2023-11-25T04:24:12.178 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3454000000000006e-05 2023-11-25T04:24:12.178 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.178 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.178 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:12.178 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044268+0000", 2023-11-25T04:24:12.179 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3761999999999999e-05 2023-11-25T04:24:12.179 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:12.179 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:12.179 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:12.179 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.179 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.179 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:23 3.e 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:12.179 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:24:12.179 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 85.085247735999999, 2023-11-25T04:24:12.179 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010281676, 2023-11-25T04:24:12.179 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:12.179 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:12.179 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:12.180 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.180 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:12.180 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:24:12.180 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:12.180 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.180 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.180 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:12.180 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:24:12.180 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:12.180 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.180 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.180 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:12.180 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:24:12.181 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5799999999999999e-07 2023-11-25T04:24:12.181 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.181 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.181 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:12.181 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034169+0000", 2023-11-25T04:24:12.181 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1879999999999998e-06 2023-11-25T04:24:12.181 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.181 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.181 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:12.181 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034177+0000", 2023-11-25T04:24:12.181 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7020000000000001e-06 2023-11-25T04:24:12.181 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.181 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.182 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:12.182 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034184+0000", 2023-11-25T04:24:12.182 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3910000000000003e-06 2023-11-25T04:24:12.182 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.182 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.182 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:12.182 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034924+0000", 2023-11-25T04:24:12.182 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00074032999999999998 2023-11-25T04:24:12.182 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.182 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.182 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:12.182 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034991+0000", 2023-11-25T04:24:12.182 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7347000000000004e-05 2023-11-25T04:24:12.183 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.183 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.183 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:12.183 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035096+0000", 2023-11-25T04:24:12.183 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010488 2023-11-25T04:24:12.183 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.183 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.183 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:12.183 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037588+0000", 2023-11-25T04:24:12.183 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024918309999999999 2023-11-25T04:24:12.183 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.183 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.183 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:12.183 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044367+0000", 2023-11-25T04:24:12.184 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067787589999999997 2023-11-25T04:24:12.184 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.184 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.184 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:12.184 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044415+0000", 2023-11-25T04:24:12.184 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7803000000000003e-05 2023-11-25T04:24:12.184 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.184 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.184 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:12.184 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044447+0000", 2023-11-25T04:24:12.184 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2886999999999999e-05 2023-11-25T04:24:12.184 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:12.184 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:12.185 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:12.185 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.185 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.185 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:30 3.15 3:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:12.185 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:24:12.185 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 85.085189295999996, 2023-11-25T04:24:12.185 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010365070000000001, 2023-11-25T04:24:12.185 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:12.185 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:12.185 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:12.185 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.185 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:12.185 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:24:12.186 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:12.186 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.186 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.186 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:12.186 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:24:12.186 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:12.186 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.186 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.186 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:12.186 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034225+0000", 2023-11-25T04:24:12.186 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7400000000000002e-07 2023-11-25T04:24:12.186 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.186 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.186 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:12.187 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034229+0000", 2023-11-25T04:24:12.187 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3100000000000002e-06 2023-11-25T04:24:12.187 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.187 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.187 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:12.187 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034239+0000", 2023-11-25T04:24:12.187 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9140000000000003e-06 2023-11-25T04:24:12.187 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.187 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.187 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:12.187 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034246+0000", 2023-11-25T04:24:12.187 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0149999999999998e-06 2023-11-25T04:24:12.187 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.188 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:12.188 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035572+0000", 2023-11-25T04:24:12.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001325655 2023-11-25T04:24:12.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.188 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:12.188 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035730+0000", 2023-11-25T04:24:12.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015827899999999999 2023-11-25T04:24:12.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.188 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:12.188 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035857+0000", 2023-11-25T04:24:12.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012644699999999999 2023-11-25T04:24:12.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:12.189 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038737+0000", 2023-11-25T04:24:12.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002880077 2023-11-25T04:24:12.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:12.189 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044540+0000", 2023-11-25T04:24:12.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058034239999999997 2023-11-25T04:24:12.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.190 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:12.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044569+0000", 2023-11-25T04:24:12.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8388999999999999e-05 2023-11-25T04:24:12.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.190 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:12.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044589+0000", 2023-11-25T04:24:12.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0686000000000002e-05 2023-11-25T04:24:12.190 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:12.191 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:12.191 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:12.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.191 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:33 3.1b 3:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:12.191 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:24:12.191 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 85.085143798000004, 2023-11-25T04:24:12.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099754049999999997, 2023-11-25T04:24:12.191 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:12.191 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:12.191 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:12.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.192 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:12.192 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:24:12.192 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:12.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.192 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:12.192 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:24:12.192 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:12.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.192 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:12.192 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:24:12.192 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4799999999999998e-07 2023-11-25T04:24:12.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.193 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:12.193 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034273+0000", 2023-11-25T04:24:12.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7819999999999998e-06 2023-11-25T04:24:12.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.193 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:12.193 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034285+0000", 2023-11-25T04:24:12.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2058e-05 2023-11-25T04:24:12.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:12.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034291+0000", 2023-11-25T04:24:12.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0680000000000002e-06 2023-11-25T04:24:12.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:12.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034370+0000", 2023-11-25T04:24:12.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8452e-05 2023-11-25T04:24:12.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:12.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034477+0000", 2023-11-25T04:24:12.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107471 2023-11-25T04:24:12.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.195 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:12.195 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044245+0000", 2023-11-25T04:24:12.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0097679259999999997 2023-11-25T04:24:12.195 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:12.195 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:12.195 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:12.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.195 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:36 3.1a 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:12.196 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:24:12.196 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 85.085097125000004, 2023-11-25T04:24:12.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058179099999999999, 2023-11-25T04:24:12.196 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:12.196 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:12.196 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:12.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.196 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:12.196 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:24:12.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:12.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.196 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:12.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:24:12.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:12.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:12.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034317+0000", 2023-11-25T04:24:12.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1999999999999999e-07 2023-11-25T04:24:12.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:12.197 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034321+0000", 2023-11-25T04:24:12.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.233e-06 2023-11-25T04:24:12.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:12.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034337+0000", 2023-11-25T04:24:12.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5308999999999999e-05 2023-11-25T04:24:12.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:12.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034354+0000", 2023-11-25T04:24:12.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7411999999999999e-05 2023-11-25T04:24:12.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:12.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034465+0000", 2023-11-25T04:24:12.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011154699999999999 2023-11-25T04:24:12.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:12.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034560+0000", 2023-11-25T04:24:12.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4186999999999998e-05 2023-11-25T04:24:12.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:12.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.040134+0000", 2023-11-25T04:24:12.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055746019999999997 2023-11-25T04:24:12.199 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:12.200 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:12.200 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:12.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.200 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:9 3.10 e18/16 3:09401a81:::601.00000000:head v 18'4, mlcod=0'0)", 2023-11-25T04:24:12.200 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:24:12.200 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 85.076202816999995, 2023-11-25T04:24:12.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061641020000000003, 2023-11-25T04:24:12.200 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:12.200 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:12.200 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:12.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.200 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:12.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:24:12.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:12.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:12.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:24:12.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:12.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:12.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:24:12.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5300000000000001e-07 2023-11-25T04:24:12.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.202 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:12.202 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043222+0000", 2023-11-25T04:24:12.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.082e-05 2023-11-25T04:24:12.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:12.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043228+0000", 2023-11-25T04:24:12.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6699999999999997e-06 2023-11-25T04:24:12.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.204 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:12.204 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043236+0000", 2023-11-25T04:24:12.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8189999999999992e-06 2023-11-25T04:24:12.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.204 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:12.204 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043507+0000", 2023-11-25T04:24:12.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000271012 2023-11-25T04:24:12.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.204 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:12.204 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043530+0000", 2023-11-25T04:24:12.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2659000000000001e-05 2023-11-25T04:24:12.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.205 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:12.205 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049321+0000", 2023-11-25T04:24:12.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057915129999999999 2023-11-25T04:24:12.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.205 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:12.205 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049375+0000", 2023-11-25T04:24:12.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3455999999999998e-05 2023-11-25T04:24:12.205 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:12.205 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:12.206 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:12.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.206 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:1 3.14 e18/16 3:292cf221:::200.00000000:head v 18'3, mlcod=0'0)", 2023-11-25T04:24:12.206 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:24:12.206 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 85.076166103999995, 2023-11-25T04:24:12.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0060983599999999997, 2023-11-25T04:24:12.206 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:12.206 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:12.206 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:12.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:12.207 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:24:12.207 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:12.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.207 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:12.207 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:24:12.207 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:12.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.207 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:12.207 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043248+0000", 2023-11-25T04:24:12.207 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9399999999999995e-07 2023-11-25T04:24:12.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.208 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:12.208 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043252+0000", 2023-11-25T04:24:12.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1799999999999998e-06 2023-11-25T04:24:12.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.208 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:12.208 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043259+0000", 2023-11-25T04:24:12.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0049999999999998e-06 2023-11-25T04:24:12.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.208 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:12.208 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043265+0000", 2023-11-25T04:24:12.209 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0859999999999998e-06 2023-11-25T04:24:12.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.209 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.209 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:12.209 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043311+0000", 2023-11-25T04:24:12.209 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6082999999999999e-05 2023-11-25T04:24:12.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.209 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.209 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:12.209 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043330+0000", 2023-11-25T04:24:12.209 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8556e-05 2023-11-25T04:24:12.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.210 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:12.210 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049303+0000", 2023-11-25T04:24:12.210 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059733470000000004 2023-11-25T04:24:12.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.210 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:12.210 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049346+0000", 2023-11-25T04:24:12.210 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2608999999999998e-05 2023-11-25T04:24:12.210 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:12.210 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:12.210 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:12.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.211 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:3 3.14 e18/16 3:287034c4:::1.00000000.inode:head v 18'4, mlcod=0'0)", 2023-11-25T04:24:12.211 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:24:12.211 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 85.075649597999998, 2023-11-25T04:24:12.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057040090000000003, 2023-11-25T04:24:12.211 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:12.211 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:12.211 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:12.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.211 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:12.211 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:24:12.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:12.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.212 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:12.212 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:24:12.212 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:12.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.212 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:12.212 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:24:12.212 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9799999999999999e-07 2023-11-25T04:24:12.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.212 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:12.212 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043785+0000", 2023-11-25T04:24:12.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0721999999999999e-05 2023-11-25T04:24:12.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.213 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:12.213 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043793+0000", 2023-11-25T04:24:12.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5839999999999995e-06 2023-11-25T04:24:12.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.213 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:12.213 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043803+0000", 2023-11-25T04:24:12.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0729999999999994e-06 2023-11-25T04:24:12.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.213 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:12.213 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043843+0000", 2023-11-25T04:24:12.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0747999999999997e-05 2023-11-25T04:24:12.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.214 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.214 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:12.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043856+0000", 2023-11-25T04:24:12.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2697000000000001e-05 2023-11-25T04:24:12.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.214 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.214 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:12.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049426+0000", 2023-11-25T04:24:12.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055704409999999998 2023-11-25T04:24:12.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.214 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:12.215 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049468+0000", 2023-11-25T04:24:12.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1446000000000003e-05 2023-11-25T04:24:12.215 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:12.215 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:12.215 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:12.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.215 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "pg_update_log_missing(3.14 epoch 18/16 rep_tid 171 entries 18'5 (0'0) error 3:2e2fa760:::200.00000001:head by mds.0.3:32 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:24:12.215 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:24:12.215 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 85.075488634999999, 2023-11-25T04:24:12.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005556938, 2023-11-25T04:24:12.215 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:12.215 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2023-11-25T04:24:12.216 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:12.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:12.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:24:12.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:12.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:12.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:24:12.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:12.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:12.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:24:12.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.03e-07 2023-11-25T04:24:12.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.217 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:12.217 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043931+0000", 2023-11-25T04:24:12.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1079999999999998e-06 2023-11-25T04:24:12.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.217 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:12.217 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043944+0000", 2023-11-25T04:24:12.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2852e-05 2023-11-25T04:24:12.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.218 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:12.218 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043955+0000", 2023-11-25T04:24:12.218 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1037e-05 2023-11-25T04:24:12.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.218 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:12.218 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044088+0000", 2023-11-25T04:24:12.218 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013269599999999999 2023-11-25T04:24:12.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.218 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:12.218 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049482+0000", 2023-11-25T04:24:12.218 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053940419999999999 2023-11-25T04:24:12.218 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:12.219 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:12.219 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:12.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.219 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:10 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'5, mlcod=0'0)", 2023-11-25T04:24:12.219 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:24:12.219 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 85.075398606999997, 2023-11-25T04:24:12.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054493099999999997, 2023-11-25T04:24:12.219 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:12.219 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:12.219 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:12.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.219 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:12.219 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:24:12.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:12.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:12.220 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:24:12.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:12.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:12.220 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:24:12.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6100000000000002e-07 2023-11-25T04:24:12.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.221 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.221 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:12.221 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044019+0000", 2023-11-25T04:24:12.221 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.839e-06 2023-11-25T04:24:12.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.221 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.221 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:12.221 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044026+0000", 2023-11-25T04:24:12.221 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5869999999999998e-06 2023-11-25T04:24:12.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.221 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.221 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:12.222 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044031+0000", 2023-11-25T04:24:12.222 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4909999999999996e-06 2023-11-25T04:24:12.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.222 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:12.222 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044070+0000", 2023-11-25T04:24:12.222 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8829999999999999e-05 2023-11-25T04:24:12.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.222 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:12.222 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044081+0000", 2023-11-25T04:24:12.222 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1447999999999999e-05 2023-11-25T04:24:12.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.223 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.223 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:12.223 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049406+0000", 2023-11-25T04:24:12.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053246480000000004 2023-11-25T04:24:12.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.223 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.223 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:12.223 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049464+0000", 2023-11-25T04:24:12.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8205999999999998e-05 2023-11-25T04:24:12.223 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:12.223 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:12.223 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:12.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.223 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.224 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:11 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'6, mlcod=0'0)", 2023-11-25T04:24:12.224 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:24:12.224 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 85.075197342999999, 2023-11-25T04:24:12.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052673499999999996, 2023-11-25T04:24:12.224 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:12.224 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:12.224 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:12.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:12.224 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:24:12.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:12.224 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.225 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:12.225 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:24:12.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:12.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.225 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:12.225 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:24:12.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0199999999999998e-07 2023-11-25T04:24:12.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.225 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:12.225 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044220+0000", 2023-11-25T04:24:12.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7720000000000001e-06 2023-11-25T04:24:12.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.226 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:12.226 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044226+0000", 2023-11-25T04:24:12.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8579999999999996e-06 2023-11-25T04:24:12.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.226 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:12.226 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044232+0000", 2023-11-25T04:24:12.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1700000000000002e-06 2023-11-25T04:24:12.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.226 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:12.226 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044274+0000", 2023-11-25T04:24:12.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1254999999999997e-05 2023-11-25T04:24:12.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:12.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:24:12.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6880999999999999e-05 2023-11-25T04:24:12.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:12.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049466+0000", 2023-11-25T04:24:12.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051659219999999999 2023-11-25T04:24:12.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:12.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:12.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:12.228 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049483+0000", 2023-11-25T04:24:12.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.719e-05 2023-11-25T04:24:12.228 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:12.228 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:12.228 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:12.228 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:12.228 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:12.228 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:12.229 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:24:12.230 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:12.228+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:12.330 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:12.328+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:12.337 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:12.337 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:24:12.337 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2023-11-25T04:24:12.337 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:12.347 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:24:12.431 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:12.430+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:12.493 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:12.493 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:24:12.493 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2023-11-25T04:24:12.493 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2023-11-25T04:24:12.493 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:12.503 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:24:12.532 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:12.531+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:12.611 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:12.611 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2023-11-25T04:24:12.611 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2023-11-25T04:24:12.612 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2023-11-25T04:24:12.612 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.612 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:12.612 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:24:12.612 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 85.576599779000006, 2023-11-25T04:24:12.612 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01072943, 2023-11-25T04:24:12.612 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:12.613 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:12.613 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:12.613 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.613 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:12.613 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:24:12.613 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:12.613 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.613 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.613 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:12.613 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:24:12.614 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:12.614 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.614 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.614 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:12.614 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031829+0000", 2023-11-25T04:24:12.614 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.666e-06 2023-11-25T04:24:12.614 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.614 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.614 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:12.614 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031839+0000", 2023-11-25T04:24:12.614 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9490000000000008e-06 2023-11-25T04:24:12.614 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.615 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.615 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:12.615 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031857+0000", 2023-11-25T04:24:12.615 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8093000000000001e-05 2023-11-25T04:24:12.615 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.615 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.615 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:12.615 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031882+0000", 2023-11-25T04:24:12.615 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4294000000000002e-05 2023-11-25T04:24:12.615 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.615 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.615 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:12.615 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033130+0000", 2023-11-25T04:24:12.616 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001248264 2023-11-25T04:24:12.616 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.616 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.616 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:12.616 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033426+0000", 2023-11-25T04:24:12.616 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029620999999999998 2023-11-25T04:24:12.616 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.616 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.616 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:12.616 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033814+0000", 2023-11-25T04:24:12.616 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000387742 2023-11-25T04:24:12.616 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.616 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.617 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:12.617 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036793+0000", 2023-11-25T04:24:12.617 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002979598 2023-11-25T04:24:12.617 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.617 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.617 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:12.617 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042455+0000", 2023-11-25T04:24:12.617 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0056619260000000003 2023-11-25T04:24:12.617 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.617 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.617 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:12.617 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042502+0000", 2023-11-25T04:24:12.617 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7139999999999999e-05 2023-11-25T04:24:12.617 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.618 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.618 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:12.618 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042557+0000", 2023-11-25T04:24:12.618 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4548e-05 2023-11-25T04:24:12.618 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:12.618 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:12.618 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:12.618 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.618 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.618 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:12.618 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:24:12.618 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 85.575401244000005, 2023-11-25T04:24:12.618 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011505202000000001, 2023-11-25T04:24:12.619 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:12.619 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:12.619 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:12.619 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.624 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:12.624 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:24:12.624 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:12.624 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.624 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.624 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:12.624 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:24:12.624 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:12.624 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.624 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.624 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:12.624 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033027+0000", 2023-11-25T04:24:12.625 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0699999999999999e-06 2023-11-25T04:24:12.625 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.625 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.625 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:12.625 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033035+0000", 2023-11-25T04:24:12.625 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0830000000000004e-06 2023-11-25T04:24:12.625 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.625 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.625 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:12.625 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033059+0000", 2023-11-25T04:24:12.625 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3975e-05 2023-11-25T04:24:12.625 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.625 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.626 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:12.626 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033482+0000", 2023-11-25T04:24:12.626 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042253700000000001 2023-11-25T04:24:12.626 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.626 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.626 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:12.626 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034520+0000", 2023-11-25T04:24:12.626 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001037991 2023-11-25T04:24:12.626 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.626 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.626 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:12.626 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034611+0000", 2023-11-25T04:24:12.626 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1113999999999995e-05 2023-11-25T04:24:12.627 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.627 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.627 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:12.627 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034769+0000", 2023-11-25T04:24:12.627 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000157641 2023-11-25T04:24:12.627 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.627 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.627 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:12.627 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038292+0000", 2023-11-25T04:24:12.627 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035231160000000002 2023-11-25T04:24:12.627 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.627 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.627 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:12.628 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044440+0000", 2023-11-25T04:24:12.628 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0061486980000000002 2023-11-25T04:24:12.628 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.628 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.628 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:12.628 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044495+0000", 2023-11-25T04:24:12.628 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4339000000000003e-05 2023-11-25T04:24:12.628 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.628 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.628 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:12.628 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044531+0000", 2023-11-25T04:24:12.628 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6637999999999997e-05 2023-11-25T04:24:12.628 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:12.629 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:12.629 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:12.630 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.630 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.630 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:12.630 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:24:12.630 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 85.57492689, 2023-11-25T04:24:12.630 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0057634039999999997, 2023-11-25T04:24:12.630 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:12.630 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:12.630 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:12.630 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.630 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:12.631 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:24:12.631 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:12.631 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.631 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.631 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:12.631 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:24:12.631 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:12.631 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.631 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.631 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:12.631 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:24:12.631 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7900000000000007e-07 2023-11-25T04:24:12.631 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.632 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.632 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:12.632 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033507+0000", 2023-11-25T04:24:12.632 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8900000000000004e-06 2023-11-25T04:24:12.632 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.632 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.632 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:12.632 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033519+0000", 2023-11-25T04:24:12.632 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1756000000000001e-05 2023-11-25T04:24:12.632 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.632 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.633 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:12.633 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033527+0000", 2023-11-25T04:24:12.633 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3610000000000001e-06 2023-11-25T04:24:12.633 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.633 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.633 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:12.633 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034451+0000", 2023-11-25T04:24:12.633 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00092372400000000001 2023-11-25T04:24:12.633 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.633 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.633 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:12.633 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034809+0000", 2023-11-25T04:24:12.634 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00035757699999999998 2023-11-25T04:24:12.634 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.634 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.634 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:12.634 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034965+0000", 2023-11-25T04:24:12.634 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015638699999999999 2023-11-25T04:24:12.634 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.636 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.636 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:12.636 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038299+0000", 2023-11-25T04:24:12.636 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0033336559999999999 2023-11-25T04:24:12.636 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.637 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.637 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:12.637 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039169+0000", 2023-11-25T04:24:12.637 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00087066400000000005 2023-11-25T04:24:12.637 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.637 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.637 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:12.637 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039226+0000", 2023-11-25T04:24:12.637 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6511999999999997e-05 2023-11-25T04:24:12.637 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.637 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.637 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:12.638 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039264+0000", 2023-11-25T04:24:12.638 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7898000000000002e-05 2023-11-25T04:24:12.638 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:12.638 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:12.638 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:12.638 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.638 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.638 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:25 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:12.638 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:24:12.638 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 85.574848306000007, 2023-11-25T04:24:12.638 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096656169999999996, 2023-11-25T04:24:12.638 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:12.638 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:12.639 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:12.639 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.639 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:12.639 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:24:12.639 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:12.639 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.639 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.639 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:12.639 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:24:12.639 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:12.639 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.639 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.640 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:12.640 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033580+0000", 2023-11-25T04:24:12.640 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8199999999999999e-07 2023-11-25T04:24:12.640 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.640 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.640 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:12.640 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033585+0000", 2023-11-25T04:24:12.640 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3680000000000001e-06 2023-11-25T04:24:12.640 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.640 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.640 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:12.640 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033600+0000", 2023-11-25T04:24:12.640 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4331000000000001e-05 2023-11-25T04:24:12.641 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.641 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.641 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:12.641 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033606+0000", 2023-11-25T04:24:12.641 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5780000000000004e-06 2023-11-25T04:24:12.641 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.641 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.641 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:12.641 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035747+0000", 2023-11-25T04:24:12.641 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0021406530000000002 2023-11-25T04:24:12.641 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.641 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.641 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:12.642 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035901+0000", 2023-11-25T04:24:12.642 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000153931 2023-11-25T04:24:12.642 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.642 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.642 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:12.642 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036193+0000", 2023-11-25T04:24:12.642 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029211299999999999 2023-11-25T04:24:12.642 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.642 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.642 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:12.642 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038655+0000", 2023-11-25T04:24:12.643 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0024621640000000002 2023-11-25T04:24:12.643 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.643 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.643 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:12.643 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043130+0000", 2023-11-25T04:24:12.643 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.004474698 2023-11-25T04:24:12.643 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.643 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.643 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:12.643 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043204+0000", 2023-11-25T04:24:12.643 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4091999999999995e-05 2023-11-25T04:24:12.643 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.643 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.644 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:12.644 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043245+0000", 2023-11-25T04:24:12.644 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0907e-05 2023-11-25T04:24:12.644 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:12.644 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:12.644 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:12.644 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.644 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.644 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:26 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:12.644 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:12.644 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 85.574812481999999, 2023-11-25T04:24:12.644 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011086945000000001, 2023-11-25T04:24:12.644 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:12.645 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:12.645 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:12.645 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.645 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:12.645 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:12.645 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:12.645 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.645 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.645 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:12.645 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:12.645 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:12.646 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.646 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.646 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:12.646 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:12.646 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1699999999999998e-07 2023-11-25T04:24:12.646 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.647 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.647 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:12.647 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033627+0000", 2023-11-25T04:24:12.647 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.163e-05 2023-11-25T04:24:12.647 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.647 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.647 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:12.647 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033636+0000", 2023-11-25T04:24:12.647 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6859999999999995e-06 2023-11-25T04:24:12.647 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.647 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.647 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:12.647 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033642+0000", 2023-11-25T04:24:12.648 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3149999999999997e-06 2023-11-25T04:24:12.648 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.648 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.648 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:12.648 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035414+0000", 2023-11-25T04:24:12.648 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0017716380000000001 2023-11-25T04:24:12.648 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.648 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.648 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:12.648 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035539+0000", 2023-11-25T04:24:12.648 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000125477 2023-11-25T04:24:12.648 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.648 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.649 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:12.649 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035672+0000", 2023-11-25T04:24:12.649 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000133316 2023-11-25T04:24:12.649 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.649 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.649 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:12.649 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038649+0000", 2023-11-25T04:24:12.649 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029764679999999999 2023-11-25T04:24:12.649 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.649 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.649 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:12.649 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044578+0000", 2023-11-25T04:24:12.650 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0059293510000000002 2023-11-25T04:24:12.650 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.650 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.650 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:12.650 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044651+0000", 2023-11-25T04:24:12.650 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2513000000000002e-05 2023-11-25T04:24:12.650 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.650 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.650 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:12.650 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044702+0000", 2023-11-25T04:24:12.650 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1034e-05 2023-11-25T04:24:12.650 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:12.651 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:12.651 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:12.651 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.651 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.651 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:29 3.b 3:d0630e4c:::mds0_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:12.651 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:12.651 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 85.574774562000002, 2023-11-25T04:24:12.651 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010980227, 2023-11-25T04:24:12.651 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:12.651 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:12.651 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:12.651 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.652 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:12.652 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:12.652 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:12.652 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.652 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.652 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:12.652 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:12.652 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:12.652 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.652 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.652 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:12.652 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:12.653 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3499999999999996e-07 2023-11-25T04:24:12.653 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.653 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.653 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:12.653 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033658+0000", 2023-11-25T04:24:12.653 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1280000000000001e-06 2023-11-25T04:24:12.653 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.653 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.653 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:12.653 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033667+0000", 2023-11-25T04:24:12.653 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1430000000000004e-06 2023-11-25T04:24:12.653 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.653 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.654 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:12.654 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033678+0000", 2023-11-25T04:24:12.654 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0906e-05 2023-11-25T04:24:12.654 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.654 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.654 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:12.654 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034456+0000", 2023-11-25T04:24:12.654 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00077833899999999996 2023-11-25T04:24:12.654 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.654 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.654 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:12.654 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034575+0000", 2023-11-25T04:24:12.655 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011926000000000001 2023-11-25T04:24:12.655 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.655 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.655 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:12.655 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035001+0000", 2023-11-25T04:24:12.655 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042550999999999998 2023-11-25T04:24:12.655 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.655 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.655 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:12.655 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.037923+0000", 2023-11-25T04:24:12.655 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029219670000000001 2023-11-25T04:24:12.655 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.656 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.656 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:12.656 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044538+0000", 2023-11-25T04:24:12.656 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0066152349999999997 2023-11-25T04:24:12.656 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.656 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.656 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:12.656 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044594+0000", 2023-11-25T04:24:12.656 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5593000000000001e-05 2023-11-25T04:24:12.656 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.656 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.656 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:12.656 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044633+0000", 2023-11-25T04:24:12.657 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9511000000000002e-05 2023-11-25T04:24:12.657 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:12.657 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:12.657 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:12.657 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.657 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.657 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:34 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:12.657 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:24:12.657 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 85.574524448999995, 2023-11-25T04:24:12.657 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010920445000000001, 2023-11-25T04:24:12.657 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:12.657 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:12.657 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:12.657 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.658 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:12.658 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:24:12.658 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:12.658 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.658 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.658 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:12.658 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:24:12.658 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:12.658 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.658 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.658 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:12.658 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033904+0000", 2023-11-25T04:24:12.659 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6599999999999994e-07 2023-11-25T04:24:12.659 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.659 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.659 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:12.659 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033909+0000", 2023-11-25T04:24:12.659 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5639999999999996e-06 2023-11-25T04:24:12.659 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.659 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.660 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:12.660 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033923+0000", 2023-11-25T04:24:12.660 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3028e-05 2023-11-25T04:24:12.660 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.660 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.660 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:12.660 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034043+0000", 2023-11-25T04:24:12.660 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000120328 2023-11-25T04:24:12.660 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.660 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.660 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:12.660 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035576+0000", 2023-11-25T04:24:12.661 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015333720000000001 2023-11-25T04:24:12.661 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.661 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.661 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:12.661 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035699+0000", 2023-11-25T04:24:12.661 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000123001 2023-11-25T04:24:12.661 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.661 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.661 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:12.661 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044823+0000", 2023-11-25T04:24:12.661 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0091241859999999994 2023-11-25T04:24:12.661 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:12.661 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:12.661 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:12.662 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.662 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.662 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:35 3.1f 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:12.662 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:24:12.662 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 85.573220243999998, 2023-11-25T04:24:12.662 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096164779999999995, 2023-11-25T04:24:12.662 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:12.662 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:12.662 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:12.662 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.662 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:12.662 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:24:12.663 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:12.663 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.663 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.663 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:12.663 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:24:12.663 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:12.663 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.663 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.663 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:12.663 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035208+0000", 2023-11-25T04:24:12.663 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2699999999999998e-07 2023-11-25T04:24:12.663 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.663 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.664 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:12.664 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035213+0000", 2023-11-25T04:24:12.664 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.6070000000000001e-06 2023-11-25T04:24:12.664 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.664 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.664 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:12.664 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035350+0000", 2023-11-25T04:24:12.664 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013693599999999999 2023-11-25T04:24:12.664 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.664 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.664 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:12.664 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035358+0000", 2023-11-25T04:24:12.664 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0099999999999995e-06 2023-11-25T04:24:12.665 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.665 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.665 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:12.665 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036260+0000", 2023-11-25T04:24:12.665 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00090240100000000005 2023-11-25T04:24:12.665 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.665 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.665 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:12.665 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036517+0000", 2023-11-25T04:24:12.665 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00025736299999999998 2023-11-25T04:24:12.665 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.665 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.666 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:12.666 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044824+0000", 2023-11-25T04:24:12.666 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0083063340000000003 2023-11-25T04:24:12.666 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:12.666 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:12.666 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:12.666 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.666 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.666 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:37 3.1e 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:12.666 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:24:12.666 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 85.573053557999998, 2023-11-25T04:24:12.666 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0078322640000000002, 2023-11-25T04:24:12.666 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:12.667 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:12.667 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:12.667 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.667 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:12.667 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:24:12.667 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:12.667 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.667 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.667 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:12.667 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:24:12.667 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:12.667 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.668 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.668 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:12.668 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035375+0000", 2023-11-25T04:24:12.668 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9100000000000003e-07 2023-11-25T04:24:12.668 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.668 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.668 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:12.668 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035379+0000", 2023-11-25T04:24:12.668 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9430000000000002e-06 2023-11-25T04:24:12.668 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.668 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.668 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:12.668 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035388+0000", 2023-11-25T04:24:12.669 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2760000000000006e-06 2023-11-25T04:24:12.669 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.669 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.669 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:12.669 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035396+0000", 2023-11-25T04:24:12.669 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8480000000000001e-06 2023-11-25T04:24:12.669 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.669 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.669 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:12.669 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035452+0000", 2023-11-25T04:24:12.669 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5831000000000001e-05 2023-11-25T04:24:12.669 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.669 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.670 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:12.670 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035934+0000", 2023-11-25T04:24:12.670 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00048265599999999998 2023-11-25T04:24:12.670 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.670 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.670 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:12.670 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043206+0000", 2023-11-25T04:24:12.670 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.007271919 2023-11-25T04:24:12.670 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:12.670 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:12.670 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:12.670 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.671 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.671 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing(3.1a epoch 18/16 rep_tid 3 entries 18'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:36 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:24:12.671 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:24:12.671 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 85.571777972000007, 2023-11-25T04:24:12.671 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002073414, 2023-11-25T04:24:12.671 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:12.671 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:24:12.671 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:12.671 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.671 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:12.671 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:24:12.671 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:12.671 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.672 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.672 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:12.672 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:24:12.672 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:12.672 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.672 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.672 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:12.672 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036650+0000", 2023-11-25T04:24:12.672 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7300000000000002e-07 2023-11-25T04:24:12.672 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.672 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.672 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:12.673 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036659+0000", 2023-11-25T04:24:12.673 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.7499999999999992e-06 2023-11-25T04:24:12.673 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.673 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.673 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:12.673 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036673+0000", 2023-11-25T04:24:12.673 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4917e-05 2023-11-25T04:24:12.673 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.673 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.673 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:12.673 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036687+0000", 2023-11-25T04:24:12.673 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3883000000000001e-05 2023-11-25T04:24:12.673 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.674 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.674 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:12.674 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036781+0000", 2023-11-25T04:24:12.674 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3219000000000003e-05 2023-11-25T04:24:12.674 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.674 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.674 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:12.674 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038723+0000", 2023-11-25T04:24:12.674 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019422720000000001 2023-11-25T04:24:12.674 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:12.674 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:12.674 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:12.675 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.675 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.675 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:28 3.13 e18/16 3:cd5a64a3:::100.00000000:head v 18'1, mlcod=0'0)", 2023-11-25T04:24:12.675 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:24:12.675 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 85.568116814000007, 2023-11-25T04:24:12.675 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001850546, 2023-11-25T04:24:12.675 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:12.675 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:12.675 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:12.675 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.675 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:12.675 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:24:12.675 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:12.676 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.676 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.676 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:12.676 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:24:12.676 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:12.676 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.676 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.676 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:12.676 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:24:12.676 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7800000000000002e-07 2023-11-25T04:24:12.676 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.676 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.677 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:12.677 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040331+0000", 2023-11-25T04:24:12.677 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0449999999999999e-05 2023-11-25T04:24:12.677 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.677 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.677 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:12.677 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040339+0000", 2023-11-25T04:24:12.677 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0709999999999995e-06 2023-11-25T04:24:12.677 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.677 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.677 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:12.677 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040347+0000", 2023-11-25T04:24:12.677 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7079999999999997e-06 2023-11-25T04:24:12.678 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.678 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.678 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:12.678 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040410+0000", 2023-11-25T04:24:12.678 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2453000000000007e-05 2023-11-25T04:24:12.678 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.678 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.678 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:12.678 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040422+0000", 2023-11-25T04:24:12.678 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2429000000000001e-05 2023-11-25T04:24:12.678 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.678 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.679 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:12.679 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042129+0000", 2023-11-25T04:24:12.679 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001707266 2023-11-25T04:24:12.679 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.679 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.679 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:12.679 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042161+0000", 2023-11-25T04:24:12.679 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1791000000000001e-05 2023-11-25T04:24:12.679 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:12.679 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:12.679 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:12.679 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.679 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.679 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:27 3.13 e18/16 3:c91d4a1d:::mds0_inotable:head v 18'2, mlcod=0'0)", 2023-11-25T04:24:12.680 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:24:12.680 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 85.567913290999996, 2023-11-25T04:24:12.680 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0016886430000000001, 2023-11-25T04:24:12.680 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:12.680 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:12.680 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:12.680 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.680 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:12.680 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:24:12.680 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:12.680 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.680 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.680 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:12.681 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:24:12.681 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:12.681 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.681 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.681 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:12.681 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:24:12.681 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3999999999999997e-07 2023-11-25T04:24:12.681 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.681 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.681 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:12.681 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040520+0000", 2023-11-25T04:24:12.681 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2989999999999996e-06 2023-11-25T04:24:12.681 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.682 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.682 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:12.682 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040527+0000", 2023-11-25T04:24:12.682 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5730000000000002e-06 2023-11-25T04:24:12.682 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.682 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.682 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:12.682 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040535+0000", 2023-11-25T04:24:12.682 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8630000000000008e-06 2023-11-25T04:24:12.682 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.682 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.682 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:12.682 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040683+0000", 2023-11-25T04:24:12.683 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000148002 2023-11-25T04:24:12.683 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.683 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.683 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:12.683 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040700+0000", 2023-11-25T04:24:12.683 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7065e-05 2023-11-25T04:24:12.683 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.683 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.683 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:12.683 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042163+0000", 2023-11-25T04:24:12.683 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001462767 2023-11-25T04:24:12.683 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.683 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.684 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:12.684 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042203+0000", 2023-11-25T04:24:12.684 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9734000000000003e-05 2023-11-25T04:24:12.684 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:12.684 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:12.684 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:12.684 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.684 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.684 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:20 3.1c e18/16 3:3ed09add:::607.00000000:head v 18'1, mlcod=0'0)", 2023-11-25T04:24:12.684 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:24:12.684 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 85.567878250000007, 2023-11-25T04:24:12.684 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015271, 2023-11-25T04:24:12.685 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:12.685 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:12.685 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:12.685 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.685 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:12.685 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:24:12.685 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:12.685 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.685 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.685 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:12.685 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:24:12.686 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:12.686 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.686 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.686 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:12.686 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:24:12.686 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4799999999999999e-07 2023-11-25T04:24:12.686 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.686 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.687 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:12.687 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040561+0000", 2023-11-25T04:24:12.687 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1800000000000001e-05 2023-11-25T04:24:12.687 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.687 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.687 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:12.687 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040568+0000", 2023-11-25T04:24:12.687 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7730000000000002e-06 2023-11-25T04:24:12.688 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.688 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.688 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:12.688 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040587+0000", 2023-11-25T04:24:12.688 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8691000000000001e-05 2023-11-25T04:24:12.688 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.688 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.688 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:12.688 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040638+0000", 2023-11-25T04:24:12.688 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1068e-05 2023-11-25T04:24:12.688 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.688 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.689 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:12.689 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040670+0000", 2023-11-25T04:24:12.689 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2125999999999997e-05 2023-11-25T04:24:12.689 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.689 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.689 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:12.689 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042032+0000", 2023-11-25T04:24:12.689 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001361821 2023-11-25T04:24:12.689 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.689 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.689 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:12.689 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042076+0000", 2023-11-25T04:24:12.689 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4472999999999999e-05 2023-11-25T04:24:12.690 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:12.690 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:12.690 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:12.690 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.690 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.690 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:21 3.1c e18/16 3:3ed09add:::607.00000000:head v 18'2, mlcod=0'0)", 2023-11-25T04:24:12.690 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:24:12.690 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 85.567643524999994, 2023-11-25T04:24:12.690 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0013228160000000001, 2023-11-25T04:24:12.690 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:12.690 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:12.690 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:12.691 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.691 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:12.691 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:24:12.691 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:12.691 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.691 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.691 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:12.691 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:24:12.691 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:12.691 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.691 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.691 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:12.691 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:24:12.692 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9299999999999999e-07 2023-11-25T04:24:12.692 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.692 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.692 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:12.692 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040790+0000", 2023-11-25T04:24:12.692 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5770000000000001e-06 2023-11-25T04:24:12.692 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.692 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.692 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:12.692 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040799+0000", 2023-11-25T04:24:12.692 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6439999999999995e-06 2023-11-25T04:24:12.692 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.692 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.693 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:12.693 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040806+0000", 2023-11-25T04:24:12.693 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0180000000000004e-06 2023-11-25T04:24:12.693 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.693 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.693 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:12.693 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040907+0000", 2023-11-25T04:24:12.693 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000100096 2023-11-25T04:24:12.693 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.693 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.693 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:12.693 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040925+0000", 2023-11-25T04:24:12.693 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8536999999999999e-05 2023-11-25T04:24:12.694 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.694 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.694 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:12.694 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042079+0000", 2023-11-25T04:24:12.694 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00115358 2023-11-25T04:24:12.694 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.694 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.694 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:12.694 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042107+0000", 2023-11-25T04:24:12.694 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7971e-05 2023-11-25T04:24:12.694 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:12.694 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:12.695 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:12.695 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.695 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.695 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop_reply(mds.0.3:4 3.1f e18/16 ondisk, result = 0)", 2023-11-25T04:24:12.695 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:24:12.695 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 85.566304134999996, 2023-11-25T04:24:12.695 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00044092900000000001, 2023-11-25T04:24:12.695 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:12.695 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:12.695 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:12.695 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.695 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:12.695 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:24:12.695 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:12.696 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.696 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.696 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:12.696 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042123+0000", 2023-11-25T04:24:12.696 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:12.696 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.696 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.696 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:12.696 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042124+0000", 2023-11-25T04:24:12.696 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1800000000000005e-07 2023-11-25T04:24:12.696 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.696 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.696 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:12.697 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042128+0000", 2023-11-25T04:24:12.697 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2289999999999997e-06 2023-11-25T04:24:12.697 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.697 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.697 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:12.697 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042138+0000", 2023-11-25T04:24:12.697 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9729999999999992e-06 2023-11-25T04:24:12.697 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.697 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.697 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:12.698 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042145+0000", 2023-11-25T04:24:12.698 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.336e-06 2023-11-25T04:24:12.698 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.698 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.698 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:12.698 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042441+0000", 2023-11-25T04:24:12.698 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029650500000000002 2023-11-25T04:24:12.698 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.698 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.698 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:12.698 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042448+0000", 2023-11-25T04:24:12.698 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4100000000000002e-06 2023-11-25T04:24:12.699 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.699 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.699 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:12.699 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042564+0000", 2023-11-25T04:24:12.699 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000115758 2023-11-25T04:24:12.699 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:12.699 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:12.699 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:12.699 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.699 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.699 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing_reply(3.1f epoch 18/16 rep_tid 8 lcod 18'4)", 2023-11-25T04:24:12.699 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:24:12.699 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 85.563978723000005, 2023-11-25T04:24:12.700 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00038154699999999998, 2023-11-25T04:24:12.700 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:12.700 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:24:12.700 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:12.700 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.700 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:12.700 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:24:12.700 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:12.700 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.700 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.700 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:12.700 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:24:12.700 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:12.700 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.701 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.701 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:12.701 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044449+0000", 2023-11-25T04:24:12.701 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2700000000000001e-07 2023-11-25T04:24:12.701 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.701 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.701 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:12.701 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044451+0000", 2023-11-25T04:24:12.701 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7460000000000001e-06 2023-11-25T04:24:12.701 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.701 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.701 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:12.701 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044455+0000", 2023-11-25T04:24:12.701 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5809999999999998e-06 2023-11-25T04:24:12.702 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.702 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.702 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:12.702 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044458+0000", 2023-11-25T04:24:12.702 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0170000000000001e-06 2023-11-25T04:24:12.702 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.702 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.702 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:12.702 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044734+0000", 2023-11-25T04:24:12.702 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00027582799999999999 2023-11-25T04:24:12.702 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.702 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.702 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:12.703 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044830+0000", 2023-11-25T04:24:12.703 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6248000000000004e-05 2023-11-25T04:24:12.703 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:12.703 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:12.703 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:12.703 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.703 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.703 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:39 3.6 3:654134d2:::mds0_openfiles.0:head [omap-clear,omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2023-11-25T04:24:12.703 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:24:12.703 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 80.504116019999998, 2023-11-25T04:24:12.703 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001788658, 2023-11-25T04:24:12.703 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:12.703 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:12.703 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:12.703 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.704 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:12.704 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:24:12.704 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:12.704 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.704 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.704 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:12.704 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:24:12.704 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:12.704 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.704 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.704 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:12.704 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104314+0000", 2023-11-25T04:24:12.704 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3099999999999999e-06 2023-11-25T04:24:12.704 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.705 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.705 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:12.705 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104325+0000", 2023-11-25T04:24:12.705 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1511999999999999e-05 2023-11-25T04:24:12.705 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.705 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.705 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:12.705 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104362+0000", 2023-11-25T04:24:12.705 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6903000000000002e-05 2023-11-25T04:24:12.705 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.705 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.705 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:12.705 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104387+0000", 2023-11-25T04:24:12.705 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4766e-05 2023-11-25T04:24:12.706 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.706 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.706 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:12.706 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104465+0000", 2023-11-25T04:24:12.706 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8119000000000002e-05 2023-11-25T04:24:12.706 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.706 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.706 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:12.706 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104557+0000", 2023-11-25T04:24:12.706 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2496000000000003e-05 2023-11-25T04:24:12.706 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.706 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.706 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:12.706 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104646+0000", 2023-11-25T04:24:12.707 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8818000000000002e-05 2023-11-25T04:24:12.707 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.707 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.707 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:12.707 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105201+0000", 2023-11-25T04:24:12.707 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00055506000000000002 2023-11-25T04:24:12.707 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.707 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.707 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:12.707 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105994+0000", 2023-11-25T04:24:12.707 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00079294700000000001 2023-11-25T04:24:12.707 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.707 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.707 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:12.708 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.106055+0000", 2023-11-25T04:24:12.708 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0566000000000003e-05 2023-11-25T04:24:12.708 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.708 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.708 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:12.708 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.106100+0000", 2023-11-25T04:24:12.708 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5160999999999997e-05 2023-11-25T04:24:12.708 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:12.708 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:12.708 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:12.708 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.708 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.708 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:43 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2023-11-25T04:24:12.708 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:24:12.709 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 65.503891023999998, 2023-11-25T04:24:12.709 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001751166, 2023-11-25T04:24:12.709 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:12.709 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:12.709 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:12.709 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.709 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:12.709 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:24:12.709 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:12.709 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.709 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.709 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:12.709 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:24:12.709 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:12.709 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.710 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.710 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:12.710 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104539+0000", 2023-11-25T04:24:12.710 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9529999999999999e-06 2023-11-25T04:24:12.710 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.710 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.710 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:12.710 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104557+0000", 2023-11-25T04:24:12.710 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7464000000000001e-05 2023-11-25T04:24:12.710 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.710 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.710 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:12.710 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104590+0000", 2023-11-25T04:24:12.711 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2966000000000003e-05 2023-11-25T04:24:12.711 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.711 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.711 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:12.711 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104608+0000", 2023-11-25T04:24:12.711 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8193e-05 2023-11-25T04:24:12.711 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.711 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.711 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:12.711 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104767+0000", 2023-11-25T04:24:12.711 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015882 2023-11-25T04:24:12.711 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.711 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.712 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:12.712 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104909+0000", 2023-11-25T04:24:12.712 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014244999999999999 2023-11-25T04:24:12.712 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.712 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.712 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:12.712 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.105059+0000", 2023-11-25T04:24:12.712 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014964399999999999 2023-11-25T04:24:12.712 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.712 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.712 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:12.713 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.105701+0000", 2023-11-25T04:24:12.713 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00064201099999999995 2023-11-25T04:24:12.713 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.713 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.713 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:12.713 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106233+0000", 2023-11-25T04:24:12.713 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00053178500000000001 2023-11-25T04:24:12.713 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.713 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.713 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:12.713 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106265+0000", 2023-11-25T04:24:12.713 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1986000000000002e-05 2023-11-25T04:24:12.713 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.713 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.714 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:12.714 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106288+0000", 2023-11-25T04:24:12.714 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2894000000000001e-05 2023-11-25T04:24:12.714 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:12.714 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:12.714 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:12.714 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.714 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.714 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:45 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e23)", 2023-11-25T04:24:12.714 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:24:12.714 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 60.503692983999997, 2023-11-25T04:24:12.714 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00098423299999999994, 2023-11-25T04:24:12.714 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:12.714 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:12.715 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:12.715 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.715 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:12.715 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:24:12.715 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:12.715 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.715 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.715 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:12.715 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104734+0000", 2023-11-25T04:24:12.715 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:12.715 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.715 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.715 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:12.715 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104737+0000", 2023-11-25T04:24:12.716 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3309999999999998e-06 2023-11-25T04:24:12.716 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.716 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.716 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:12.716 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104748+0000", 2023-11-25T04:24:12.716 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1080999999999999e-05 2023-11-25T04:24:12.716 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.716 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.716 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:12.716 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104771+0000", 2023-11-25T04:24:12.716 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2707000000000001e-05 2023-11-25T04:24:12.716 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.716 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.717 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:12.717 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104800+0000", 2023-11-25T04:24:12.717 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9595e-05 2023-11-25T04:24:12.717 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.717 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.717 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:12.717 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.104890+0000", 2023-11-25T04:24:12.717 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.9808999999999993e-05 2023-11-25T04:24:12.717 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.717 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.717 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:12.717 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105004+0000", 2023-11-25T04:24:12.717 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000114508 2023-11-25T04:24:12.718 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.718 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.718 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:12.718 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105634+0000", 2023-11-25T04:24:12.718 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00062985799999999996 2023-11-25T04:24:12.718 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.718 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.719 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:12.719 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105671+0000", 2023-11-25T04:24:12.719 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6640000000000002e-05 2023-11-25T04:24:12.719 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.719 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.719 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:12.719 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:12.105719+0000", 2023-11-25T04:24:12.719 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7704000000000003e-05 2023-11-25T04:24:12.719 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:12.719 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:12.719 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:12.719 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.719 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.720 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:64 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e43)", 2023-11-25T04:24:12.720 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:12.105733+0000", 2023-11-25T04:24:12.720 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 0.50269413100000004, 2023-11-25T04:24:12.720 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0010789790000000001, 2023-11-25T04:24:12.720 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:12.720 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:12.720 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:12.720 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.720 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:12.720 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:12.105733+0000", 2023-11-25T04:24:12.720 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:12.720 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.720 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.720 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:12.721 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:12.105733+0000", 2023-11-25T04:24:12.721 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:12.721 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.721 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.721 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:12.721 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:12.105735+0000", 2023-11-25T04:24:12.721 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.632e-06 2023-11-25T04:24:12.721 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.721 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.721 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:12.721 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:12.105750+0000", 2023-11-25T04:24:12.721 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4721e-05 2023-11-25T04:24:12.721 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.722 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.722 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:12.722 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:12.105764+0000", 2023-11-25T04:24:12.722 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4558e-05 2023-11-25T04:24:12.722 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.722 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.722 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:12.722 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:12.105771+0000", 2023-11-25T04:24:12.722 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.041e-06 2023-11-25T04:24:12.722 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.722 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.722 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:12.722 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:12.105843+0000", 2023-11-25T04:24:12.723 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1818999999999998e-05 2023-11-25T04:24:12.723 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.723 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.723 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:12.723 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:12.105910+0000", 2023-11-25T04:24:12.723 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7340000000000002e-05 2023-11-25T04:24:12.723 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.723 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.723 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:12.723 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:12.106731+0000", 2023-11-25T04:24:12.723 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00082021600000000002 2023-11-25T04:24:12.723 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.723 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.723 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:12.724 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:12.106767+0000", 2023-11-25T04:24:12.724 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6028000000000001e-05 2023-11-25T04:24:12.724 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:12.724 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:12.724 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:12.724 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:12.106812+0000", 2023-11-25T04:24:12.724 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5624000000000002e-05 2023-11-25T04:24:12.724 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:12.724 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:12.724 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:12.724 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:12.724 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:12.724 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:12.726 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:24:12.726 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:12.724+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:12.827 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:12.825+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:12.836 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:12.837 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:24:12.837 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2023-11-25T04:24:12.837 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:12.846 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:24:12.930 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:12.927+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:12.989 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:12.989 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:24:12.990 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2023-11-25T04:24:12.990 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2023-11-25T04:24:12.990 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:12.998 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:24:13.041 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:13.040+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:13.067 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2023-11-25T04:24:13.105 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:13.105 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2023-11-25T04:24:13.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2023-11-25T04:24:13.105 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2023-11-25T04:24:13.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.105 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:155 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'109, mlcod=11'105)", 2023-11-25T04:24:13.105 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:24:13.106 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 95.318082079000007, 2023-11-25T04:24:13.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061719019999999999, 2023-11-25T04:24:13.106 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:13.106 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:13.106 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:13.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.106 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:13.106 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:24:13.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:13.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.106 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:13.107 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:24:13.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:13.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.107 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:13.107 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784884+0000", 2023-11-25T04:24:13.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3099999999999999e-07 2023-11-25T04:24:13.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.107 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:13.107 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784942+0000", 2023-11-25T04:24:13.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8570000000000003e-05 2023-11-25T04:24:13.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.108 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:13.108 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784949+0000", 2023-11-25T04:24:13.108 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0310000000000001e-06 2023-11-25T04:24:13.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.108 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:13.108 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784953+0000", 2023-11-25T04:24:13.108 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6320000000000001e-06 2023-11-25T04:24:13.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.108 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:13.108 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784993+0000", 2023-11-25T04:24:13.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9737000000000003e-05 2023-11-25T04:24:13.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.109 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:13.109 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785011+0000", 2023-11-25T04:24:13.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8349999999999999e-05 2023-11-25T04:24:13.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.109 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:13.109 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791024+0000", 2023-11-25T04:24:13.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006012997 2023-11-25T04:24:13.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.110 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:13.110 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791055+0000", 2023-11-25T04:24:13.110 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1254000000000003e-05 2023-11-25T04:24:13.110 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:13.110 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:13.110 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:13.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.110 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.110 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:156 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'110, mlcod=11'105)", 2023-11-25T04:24:13.110 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:24:13.110 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 95.317496030000001, 2023-11-25T04:24:13.110 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005616116, 2023-11-25T04:24:13.110 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:13.111 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:13.111 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:13.111 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.111 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:13.111 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:24:13.111 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:13.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.111 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.111 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:13.111 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:24:13.111 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:13.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.111 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.112 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:13.112 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785470+0000", 2023-11-25T04:24:13.112 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0699999999999998e-07 2023-11-25T04:24:13.112 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.112 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.112 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:13.112 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785474+0000", 2023-11-25T04:24:13.112 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.048e-06 2023-11-25T04:24:13.112 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.112 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.112 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:13.112 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785480+0000", 2023-11-25T04:24:13.113 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7729999999999996e-06 2023-11-25T04:24:13.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.113 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:13.113 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785483+0000", 2023-11-25T04:24:13.113 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1970000000000001e-06 2023-11-25T04:24:13.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.113 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:13.113 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787936+0000", 2023-11-25T04:24:13.113 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024529040000000001 2023-11-25T04:24:13.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.113 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:13.114 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787947+0000", 2023-11-25T04:24:13.114 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1669e-05 2023-11-25T04:24:13.114 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.114 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.114 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:13.114 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791057+0000", 2023-11-25T04:24:13.114 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031095200000000002 2023-11-25T04:24:13.114 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.114 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.114 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:13.114 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791086+0000", 2023-11-25T04:24:13.114 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8697999999999999e-05 2023-11-25T04:24:13.114 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:13.114 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:13.115 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:13.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.115 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:157 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'111, mlcod=11'106)", 2023-11-25T04:24:13.115 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:24:13.116 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 95.316903472000007, 2023-11-25T04:24:13.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050590280000000001, 2023-11-25T04:24:13.116 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:13.116 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:13.117 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:13.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.117 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:13.117 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:24:13.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:13.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.118 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:13.118 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:24:13.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:13.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.118 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:13.118 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786063+0000", 2023-11-25T04:24:13.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2600000000000002e-07 2023-11-25T04:24:13.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:13.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786069+0000", 2023-11-25T04:24:13.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8839999999999999e-06 2023-11-25T04:24:13.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:13.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786079+0000", 2023-11-25T04:24:13.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0124e-05 2023-11-25T04:24:13.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:13.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786085+0000", 2023-11-25T04:24:13.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6660000000000002e-06 2023-11-25T04:24:13.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.120 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:13.120 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788160+0000", 2023-11-25T04:24:13.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020747669999999999 2023-11-25T04:24:13.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.120 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:13.120 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788180+0000", 2023-11-25T04:24:13.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0208999999999999e-05 2023-11-25T04:24:13.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.120 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:13.120 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791087+0000", 2023-11-25T04:24:13.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029064630000000002 2023-11-25T04:24:13.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.121 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:13.121 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791121+0000", 2023-11-25T04:24:13.121 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4289000000000003e-05 2023-11-25T04:24:13.121 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:13.121 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:13.121 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:13.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.121 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:158 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'112, mlcod=11'106)", 2023-11-25T04:24:13.121 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:24:13.122 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 95.316487078999998, 2023-11-25T04:24:13.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046663030000000001, 2023-11-25T04:24:13.122 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:13.122 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:13.122 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:13.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.122 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:13.122 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:24:13.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:13.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.123 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:13.123 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:24:13.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:13.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.123 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:13.123 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786479+0000", 2023-11-25T04:24:13.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2799999999999996e-07 2023-11-25T04:24:13.123 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.123 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.123 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:13.123 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786498+0000", 2023-11-25T04:24:13.123 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9069e-05 2023-11-25T04:24:13.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.124 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:13.124 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786508+0000", 2023-11-25T04:24:13.124 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5810000000000003e-06 2023-11-25T04:24:13.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.124 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:13.124 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786512+0000", 2023-11-25T04:24:13.124 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4889999999999997e-06 2023-11-25T04:24:13.124 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.124 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.124 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:13.124 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788438+0000", 2023-11-25T04:24:13.125 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019257059999999999 2023-11-25T04:24:13.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.125 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:13.125 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788449+0000", 2023-11-25T04:24:13.125 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1419e-05 2023-11-25T04:24:13.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.125 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:13.125 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791122+0000", 2023-11-25T04:24:13.125 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002672822 2023-11-25T04:24:13.125 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.125 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.126 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:13.126 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791145+0000", 2023-11-25T04:24:13.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2589e-05 2023-11-25T04:24:13.126 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:13.126 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:13.126 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:13.126 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.126 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.126 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:206 1.0 e12/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 12'152, mlcod=12'148)", 2023-11-25T04:24:13.126 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:24:13.126 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 94.804718609000005, 2023-11-25T04:24:13.126 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046728189999999999, 2023-11-25T04:24:13.126 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:13.127 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:13.127 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:13.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.127 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:13.127 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:24:13.127 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:13.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.127 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.127 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:13.127 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:24:13.127 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:13.127 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.128 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.129 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:13.129 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298248+0000", 2023-11-25T04:24:13.129 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3600000000000003e-07 2023-11-25T04:24:13.129 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.129 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.129 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:13.129 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298278+0000", 2023-11-25T04:24:13.129 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0474e-05 2023-11-25T04:24:13.129 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.130 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.130 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:13.130 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298288+0000", 2023-11-25T04:24:13.130 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0156000000000001e-05 2023-11-25T04:24:13.130 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:13.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298293+0000", 2023-11-25T04:24:13.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7400000000000004e-06 2023-11-25T04:24:13.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.131 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:13.131 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298333+0000", 2023-11-25T04:24:13.131 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9666000000000003e-05 2023-11-25T04:24:13.131 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.131 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.132 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:13.132 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298351+0000", 2023-11-25T04:24:13.132 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8544999999999999e-05 2023-11-25T04:24:13.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.132 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:13.132 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302891+0000", 2023-11-25T04:24:13.132 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045398519999999996 2023-11-25T04:24:13.132 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.132 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.132 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:13.132 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302920+0000", 2023-11-25T04:24:13.133 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8750000000000001e-05 2023-11-25T04:24:13.133 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:13.133 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:13.133 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:13.133 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.133 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.133 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:14 3.15 3:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:13.133 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:24:13.133 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.069584251999999, 2023-11-25T04:24:13.133 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010990437, 2023-11-25T04:24:13.133 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:13.133 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:13.133 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:13.134 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.134 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:13.134 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:24:13.134 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:13.134 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.134 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.134 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:13.134 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:24:13.134 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:13.134 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.134 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.134 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:13.135 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033383+0000", 2023-11-25T04:24:13.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9870000000000002e-06 2023-11-25T04:24:13.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.135 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:13.135 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033398+0000", 2023-11-25T04:24:13.135 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4569999999999999e-05 2023-11-25T04:24:13.135 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.135 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.135 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:13.135 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033411+0000", 2023-11-25T04:24:13.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3543999999999999e-05 2023-11-25T04:24:13.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:13.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033423+0000", 2023-11-25T04:24:13.136 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1905999999999999e-05 2023-11-25T04:24:13.136 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.136 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.136 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:13.136 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034433+0000", 2023-11-25T04:24:13.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00101022 2023-11-25T04:24:13.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:13.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034539+0000", 2023-11-25T04:24:13.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010597199999999999 2023-11-25T04:24:13.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.137 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.137 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:13.137 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034667+0000", 2023-11-25T04:24:13.137 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012765500000000001 2023-11-25T04:24:13.137 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.138 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:13.138 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037803+0000", 2023-11-25T04:24:13.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031362220000000001 2023-11-25T04:24:13.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.138 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:13.138 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:24:13.138 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0064964669999999997 2023-11-25T04:24:13.138 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.138 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.138 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:13.139 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044331+0000", 2023-11-25T04:24:13.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1544999999999997e-05 2023-11-25T04:24:13.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.139 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:13.139 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044372+0000", 2023-11-25T04:24:13.139 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0349000000000003e-05 2023-11-25T04:24:13.139 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:13.139 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:13.139 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:13.139 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.139 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.140 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:15 3.15 3:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:13.140 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:24:13.140 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.068924601999996, 2023-11-25T04:24:13.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010464362, 2023-11-25T04:24:13.140 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:13.140 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:13.140 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:13.140 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.140 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:13.140 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:24:13.140 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:13.140 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.141 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.141 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:13.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:24:13.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:13.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.141 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.141 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:13.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034042+0000", 2023-11-25T04:24:13.141 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0999999999999997e-07 2023-11-25T04:24:13.141 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.141 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.141 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:13.141 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034048+0000", 2023-11-25T04:24:13.142 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0390000000000001e-06 2023-11-25T04:24:13.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.142 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:13.142 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034059+0000", 2023-11-25T04:24:13.142 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1182999999999999e-05 2023-11-25T04:24:13.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.142 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.142 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:13.142 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034067+0000", 2023-11-25T04:24:13.142 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1559999999999995e-06 2023-11-25T04:24:13.142 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:13.143 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035005+0000", 2023-11-25T04:24:13.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00093747399999999999 2023-11-25T04:24:13.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:13.143 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035077+0000", 2023-11-25T04:24:13.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2747e-05 2023-11-25T04:24:13.143 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.143 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.143 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:13.143 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035219+0000", 2023-11-25T04:24:13.143 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000141877 2023-11-25T04:24:13.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:13.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037810+0000", 2023-11-25T04:24:13.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002590752 2023-11-25T04:24:13.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.144 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.144 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:13.144 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044437+0000", 2023-11-25T04:24:13.144 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066270110000000004 2023-11-25T04:24:13.144 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:13.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044481+0000", 2023-11-25T04:24:13.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4106000000000001e-05 2023-11-25T04:24:13.145 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.145 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.145 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:13.145 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044505+0000", 2023-11-25T04:24:13.145 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4207e-05 2023-11-25T04:24:13.145 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:13.145 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:13.145 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:13.146 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.146 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:16 3.1d 3:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:13.146 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:24:13.146 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.068882754000001, 2023-11-25T04:24:13.146 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010413923, 2023-11-25T04:24:13.146 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:13.146 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:13.146 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:13.146 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.146 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:13.146 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:24:13.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:13.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.147 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:13.147 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:24:13.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:13.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.147 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:13.147 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:24:13.147 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4499999999999997e-07 2023-11-25T04:24:13.147 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.147 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.148 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:13.148 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034087+0000", 2023-11-25T04:24:13.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9770000000000002e-06 2023-11-25T04:24:13.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.148 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:13.148 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034094+0000", 2023-11-25T04:24:13.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8809999999999998e-06 2023-11-25T04:24:13.148 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.148 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.148 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:13.148 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034100+0000", 2023-11-25T04:24:13.148 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8440000000000003e-06 2023-11-25T04:24:13.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:13.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034993+0000", 2023-11-25T04:24:13.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00089336799999999996 2023-11-25T04:24:13.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:13.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035099+0000", 2023-11-25T04:24:13.149 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106037 2023-11-25T04:24:13.149 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.149 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.149 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:13.149 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035276+0000", 2023-11-25T04:24:13.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017631799999999999 2023-11-25T04:24:13.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:13.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037815+0000", 2023-11-25T04:24:13.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025395959999999999 2023-11-25T04:24:13.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.150 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.150 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:13.150 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044420+0000", 2023-11-25T04:24:13.150 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066045749999999997 2023-11-25T04:24:13.150 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.151 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:13.151 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044463+0000", 2023-11-25T04:24:13.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3205999999999999e-05 2023-11-25T04:24:13.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.151 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.151 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:13.151 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044497+0000", 2023-11-25T04:24:13.151 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3476000000000003e-05 2023-11-25T04:24:13.151 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:13.151 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:13.151 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:13.151 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.152 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:17 3.1d 3:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:13.152 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:24:13.152 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.068857545, 2023-11-25T04:24:13.152 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01127356, 2023-11-25T04:24:13.152 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:13.152 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:13.152 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:13.152 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.152 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:13.152 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:24:13.152 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:13.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:13.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:24:13.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:13.153 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.153 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.153 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:13.153 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:24:13.153 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1100000000000001e-07 2023-11-25T04:24:13.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:13.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034112+0000", 2023-11-25T04:24:13.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8600000000000003e-06 2023-11-25T04:24:13.154 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.154 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.154 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:13.154 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034120+0000", 2023-11-25T04:24:13.154 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7700000000000001e-06 2023-11-25T04:24:13.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.155 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:13.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034127+0000", 2023-11-25T04:24:13.155 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0999999999999998e-06 2023-11-25T04:24:13.155 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.155 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.155 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:13.155 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035601+0000", 2023-11-25T04:24:13.156 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014739410000000001 2023-11-25T04:24:13.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.156 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:13.156 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035685+0000", 2023-11-25T04:24:13.156 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3677999999999994e-05 2023-11-25T04:24:13.156 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.156 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.156 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:13.156 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035827+0000", 2023-11-25T04:24:13.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014232599999999999 2023-11-25T04:24:13.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.157 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:13.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038725+0000", 2023-11-25T04:24:13.157 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028982779999999998 2023-11-25T04:24:13.157 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.157 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.157 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:13.157 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045323+0000", 2023-11-25T04:24:13.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0065980780000000003 2023-11-25T04:24:13.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:13.158 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045354+0000", 2023-11-25T04:24:13.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0463e-05 2023-11-25T04:24:13.158 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.158 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.158 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:13.158 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045381+0000", 2023-11-25T04:24:13.158 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7654999999999999e-05 2023-11-25T04:24:13.159 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:13.159 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:13.159 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:13.159 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.159 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.159 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:22 3.e 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:13.159 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:24:13.159 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.068831132, 2023-11-25T04:24:13.159 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010133422, 2023-11-25T04:24:13.159 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:13.159 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:13.160 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:13.160 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.160 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:13.160 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:24:13.160 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:13.160 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.160 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.160 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:13.161 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:24:13.161 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:13.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.161 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.161 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:13.161 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034135+0000", 2023-11-25T04:24:13.161 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-07 2023-11-25T04:24:13.161 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.161 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.161 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:13.162 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034138+0000", 2023-11-25T04:24:13.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6009999999999999e-06 2023-11-25T04:24:13.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.162 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:13.162 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034145+0000", 2023-11-25T04:24:13.162 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1169999999999998e-06 2023-11-25T04:24:13.162 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.162 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.162 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:13.162 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034150+0000", 2023-11-25T04:24:13.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9080000000000003e-06 2023-11-25T04:24:13.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.163 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.163 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:13.163 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034209+0000", 2023-11-25T04:24:13.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8149000000000002e-05 2023-11-25T04:24:13.163 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.163 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.163 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:13.163 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034415+0000", 2023-11-25T04:24:13.163 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020637699999999999 2023-11-25T04:24:13.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.164 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.164 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:13.164 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034584+0000", 2023-11-25T04:24:13.164 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016870399999999999 2023-11-25T04:24:13.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.164 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.164 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:13.164 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.036979+0000", 2023-11-25T04:24:13.164 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023957509999999998 2023-11-25T04:24:13.164 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.164 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.165 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:13.165 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044161+0000", 2023-11-25T04:24:13.165 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0071811590000000003 2023-11-25T04:24:13.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.165 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.165 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:13.165 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044224+0000", 2023-11-25T04:24:13.165 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3454000000000006e-05 2023-11-25T04:24:13.165 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.165 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.165 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:13.165 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044268+0000", 2023-11-25T04:24:13.166 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3761999999999999e-05 2023-11-25T04:24:13.166 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:13.166 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:13.166 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:13.166 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.166 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.166 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:23 3.e 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:13.166 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:24:13.166 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.068799722999998, 2023-11-25T04:24:13.166 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010281676, 2023-11-25T04:24:13.166 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:13.167 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:13.167 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:13.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.167 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:13.167 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:24:13.167 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:13.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.167 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:13.167 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:24:13.167 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:13.167 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.167 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.168 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:13.168 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:24:13.168 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5799999999999999e-07 2023-11-25T04:24:13.168 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.168 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.168 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:13.168 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034169+0000", 2023-11-25T04:24:13.168 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1879999999999998e-06 2023-11-25T04:24:13.168 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.168 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.168 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:13.168 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034177+0000", 2023-11-25T04:24:13.168 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7020000000000001e-06 2023-11-25T04:24:13.169 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.169 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.169 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:13.169 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034184+0000", 2023-11-25T04:24:13.169 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3910000000000003e-06 2023-11-25T04:24:13.169 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.169 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.169 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:13.169 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034924+0000", 2023-11-25T04:24:13.169 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00074032999999999998 2023-11-25T04:24:13.169 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.169 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.170 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:13.170 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034991+0000", 2023-11-25T04:24:13.170 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7347000000000004e-05 2023-11-25T04:24:13.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.170 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:13.170 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035096+0000", 2023-11-25T04:24:13.170 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010488 2023-11-25T04:24:13.170 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.170 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.170 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:13.170 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037588+0000", 2023-11-25T04:24:13.170 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024918309999999999 2023-11-25T04:24:13.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.171 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:13.171 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044367+0000", 2023-11-25T04:24:13.171 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067787589999999997 2023-11-25T04:24:13.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.171 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:13.171 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044415+0000", 2023-11-25T04:24:13.171 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7803000000000003e-05 2023-11-25T04:24:13.171 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.171 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.171 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:13.172 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044447+0000", 2023-11-25T04:24:13.172 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2886999999999999e-05 2023-11-25T04:24:13.172 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:13.172 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:13.172 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:13.172 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.172 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.172 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:30 3.15 3:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:13.172 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:24:13.172 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.068741282999994, 2023-11-25T04:24:13.172 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010365070000000001, 2023-11-25T04:24:13.172 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:13.172 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:13.173 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:13.173 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.173 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:13.173 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:24:13.173 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:13.173 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.173 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.173 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:13.173 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:24:13.173 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:13.174 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.174 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.174 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:13.174 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034225+0000", 2023-11-25T04:24:13.174 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7400000000000002e-07 2023-11-25T04:24:13.174 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.174 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.174 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:13.174 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034229+0000", 2023-11-25T04:24:13.174 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3100000000000002e-06 2023-11-25T04:24:13.174 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.174 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.175 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:13.175 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034239+0000", 2023-11-25T04:24:13.175 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9140000000000003e-06 2023-11-25T04:24:13.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.175 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.175 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:13.175 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034246+0000", 2023-11-25T04:24:13.175 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0149999999999998e-06 2023-11-25T04:24:13.175 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.175 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.175 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:13.175 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035572+0000", 2023-11-25T04:24:13.175 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001325655 2023-11-25T04:24:13.176 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.176 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.176 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:13.176 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035730+0000", 2023-11-25T04:24:13.176 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015827899999999999 2023-11-25T04:24:13.177 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.177 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.177 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:13.177 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035857+0000", 2023-11-25T04:24:13.177 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012644699999999999 2023-11-25T04:24:13.177 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.177 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.177 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:13.178 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038737+0000", 2023-11-25T04:24:13.178 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002880077 2023-11-25T04:24:13.178 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.178 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.178 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:13.178 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044540+0000", 2023-11-25T04:24:13.178 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058034239999999997 2023-11-25T04:24:13.179 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.179 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.179 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:13.179 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044569+0000", 2023-11-25T04:24:13.179 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8388999999999999e-05 2023-11-25T04:24:13.179 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.179 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.179 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:13.179 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044589+0000", 2023-11-25T04:24:13.179 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0686000000000002e-05 2023-11-25T04:24:13.179 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:13.179 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:13.180 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:13.180 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.180 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.180 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:33 3.1b 3:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:13.180 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:24:13.180 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.068695785000003, 2023-11-25T04:24:13.180 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099754049999999997, 2023-11-25T04:24:13.180 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:13.180 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:13.180 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:13.181 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.181 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:13.181 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:24:13.181 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:13.181 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.181 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.181 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:13.181 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:24:13.181 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:13.181 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.181 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.182 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:13.182 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:24:13.182 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4799999999999998e-07 2023-11-25T04:24:13.182 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.182 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.182 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:13.182 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034273+0000", 2023-11-25T04:24:13.182 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7819999999999998e-06 2023-11-25T04:24:13.182 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.183 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.183 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:13.183 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034285+0000", 2023-11-25T04:24:13.184 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2058e-05 2023-11-25T04:24:13.185 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.185 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.185 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:13.185 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034291+0000", 2023-11-25T04:24:13.185 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0680000000000002e-06 2023-11-25T04:24:13.185 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.185 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.185 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:13.185 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034370+0000", 2023-11-25T04:24:13.186 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8452e-05 2023-11-25T04:24:13.186 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.186 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.186 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:13.186 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034477+0000", 2023-11-25T04:24:13.186 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107471 2023-11-25T04:24:13.186 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.186 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.186 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:13.186 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044245+0000", 2023-11-25T04:24:13.186 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0097679259999999997 2023-11-25T04:24:13.187 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:13.187 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:13.187 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:13.187 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.187 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.187 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:36 3.1a 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:13.187 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:24:13.187 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.068649112000003, 2023-11-25T04:24:13.187 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058179099999999999, 2023-11-25T04:24:13.187 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:13.187 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:13.187 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:13.187 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.188 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:13.188 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:24:13.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:13.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.188 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:13.188 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:24:13.188 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:13.188 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.188 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.188 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:13.189 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034317+0000", 2023-11-25T04:24:13.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1999999999999999e-07 2023-11-25T04:24:13.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:13.189 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034321+0000", 2023-11-25T04:24:13.189 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.233e-06 2023-11-25T04:24:13.189 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.189 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.189 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:13.189 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034337+0000", 2023-11-25T04:24:13.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5308999999999999e-05 2023-11-25T04:24:13.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.190 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.190 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:13.190 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034354+0000", 2023-11-25T04:24:13.190 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7411999999999999e-05 2023-11-25T04:24:13.190 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.191 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:13.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034465+0000", 2023-11-25T04:24:13.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011154699999999999 2023-11-25T04:24:13.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.191 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:13.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034560+0000", 2023-11-25T04:24:13.191 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4186999999999998e-05 2023-11-25T04:24:13.191 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.191 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.191 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:13.191 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.040134+0000", 2023-11-25T04:24:13.192 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055746019999999997 2023-11-25T04:24:13.192 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:13.192 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:13.192 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:13.192 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.192 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.192 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:9 3.10 e18/16 3:09401a81:::601.00000000:head v 18'4, mlcod=0'0)", 2023-11-25T04:24:13.192 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:24:13.192 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.059754803999994, 2023-11-25T04:24:13.192 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061641020000000003, 2023-11-25T04:24:13.192 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:13.192 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:13.193 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:13.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.193 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:13.193 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:24:13.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:13.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.193 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:13.193 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:24:13.193 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:13.193 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.193 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.193 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:13.193 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:24:13.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5300000000000001e-07 2023-11-25T04:24:13.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:13.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043222+0000", 2023-11-25T04:24:13.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.082e-05 2023-11-25T04:24:13.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.194 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.194 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:13.194 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043228+0000", 2023-11-25T04:24:13.194 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6699999999999997e-06 2023-11-25T04:24:13.194 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.195 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:13.195 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043236+0000", 2023-11-25T04:24:13.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8189999999999992e-06 2023-11-25T04:24:13.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.195 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:13.195 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043507+0000", 2023-11-25T04:24:13.195 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000271012 2023-11-25T04:24:13.195 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.195 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.195 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:13.196 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043530+0000", 2023-11-25T04:24:13.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2659000000000001e-05 2023-11-25T04:24:13.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.196 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:13.196 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049321+0000", 2023-11-25T04:24:13.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057915129999999999 2023-11-25T04:24:13.196 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.196 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.196 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:13.196 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049375+0000", 2023-11-25T04:24:13.196 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3455999999999998e-05 2023-11-25T04:24:13.196 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:13.196 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:13.197 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:13.197 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.197 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:1 3.14 e18/16 3:292cf221:::200.00000000:head v 18'3, mlcod=0'0)", 2023-11-25T04:24:13.197 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:24:13.197 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.059718090999993, 2023-11-25T04:24:13.197 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0060983599999999997, 2023-11-25T04:24:13.197 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:13.197 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:13.197 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:13.197 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.197 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:13.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:24:13.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:13.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:13.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:24:13.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:13.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.198 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.198 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:13.198 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043248+0000", 2023-11-25T04:24:13.198 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9399999999999995e-07 2023-11-25T04:24:13.198 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:13.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043252+0000", 2023-11-25T04:24:13.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1799999999999998e-06 2023-11-25T04:24:13.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:13.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043259+0000", 2023-11-25T04:24:13.199 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0049999999999998e-06 2023-11-25T04:24:13.199 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.199 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.199 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:13.199 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043265+0000", 2023-11-25T04:24:13.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0859999999999998e-06 2023-11-25T04:24:13.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.200 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.200 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:13.200 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043311+0000", 2023-11-25T04:24:13.200 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6082999999999999e-05 2023-11-25T04:24:13.200 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:13.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043330+0000", 2023-11-25T04:24:13.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8556e-05 2023-11-25T04:24:13.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:13.201 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049303+0000", 2023-11-25T04:24:13.201 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059733470000000004 2023-11-25T04:24:13.201 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.201 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.201 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:13.202 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049346+0000", 2023-11-25T04:24:13.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2608999999999998e-05 2023-11-25T04:24:13.202 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:13.202 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:13.202 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:13.202 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.202 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.202 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:3 3.14 e18/16 3:287034c4:::1.00000000.inode:head v 18'4, mlcod=0'0)", 2023-11-25T04:24:13.202 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:24:13.202 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.059201584999997, 2023-11-25T04:24:13.202 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057040090000000003, 2023-11-25T04:24:13.203 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:13.203 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:13.203 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:13.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:13.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:24:13.203 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:13.203 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.203 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.203 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:13.203 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:24:13.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:13.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.204 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:13.204 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:24:13.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9799999999999999e-07 2023-11-25T04:24:13.204 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.204 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:13.204 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043785+0000", 2023-11-25T04:24:13.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0721999999999999e-05 2023-11-25T04:24:13.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.205 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:13.205 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043793+0000", 2023-11-25T04:24:13.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5839999999999995e-06 2023-11-25T04:24:13.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.205 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:13.205 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043803+0000", 2023-11-25T04:24:13.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0729999999999994e-06 2023-11-25T04:24:13.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:13.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043843+0000", 2023-11-25T04:24:13.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0747999999999997e-05 2023-11-25T04:24:13.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:13.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043856+0000", 2023-11-25T04:24:13.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2697000000000001e-05 2023-11-25T04:24:13.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:13.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049426+0000", 2023-11-25T04:24:13.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055704409999999998 2023-11-25T04:24:13.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.207 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:13.207 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049468+0000", 2023-11-25T04:24:13.207 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1446000000000003e-05 2023-11-25T04:24:13.207 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:13.207 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:13.207 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:13.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.207 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "pg_update_log_missing(3.14 epoch 18/16 rep_tid 171 entries 18'5 (0'0) error 3:2e2fa760:::200.00000001:head by mds.0.3:32 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:24:13.207 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:24:13.208 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.059040621999998, 2023-11-25T04:24:13.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005556938, 2023-11-25T04:24:13.208 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:13.208 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2023-11-25T04:24:13.208 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:13.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.208 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:13.208 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:24:13.208 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:13.208 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.208 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.209 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:13.209 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:24:13.209 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:13.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.209 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.209 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:13.209 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:24:13.209 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.03e-07 2023-11-25T04:24:13.209 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.209 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.209 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:13.209 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043931+0000", 2023-11-25T04:24:13.210 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1079999999999998e-06 2023-11-25T04:24:13.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.210 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:13.210 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043944+0000", 2023-11-25T04:24:13.210 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2852e-05 2023-11-25T04:24:13.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.211 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:13.211 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043955+0000", 2023-11-25T04:24:13.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1037e-05 2023-11-25T04:24:13.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.211 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:13.211 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044088+0000", 2023-11-25T04:24:13.211 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013269599999999999 2023-11-25T04:24:13.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.211 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:13.212 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049482+0000", 2023-11-25T04:24:13.212 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053940419999999999 2023-11-25T04:24:13.212 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:13.212 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:13.212 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:13.212 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.212 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.212 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:10 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'5, mlcod=0'0)", 2023-11-25T04:24:13.212 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:24:13.212 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.058950593999995, 2023-11-25T04:24:13.212 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054493099999999997, 2023-11-25T04:24:13.212 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:13.213 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:13.213 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:13.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.213 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:13.213 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:24:13.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:13.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.213 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:13.213 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:24:13.213 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:13.213 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.213 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.214 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:13.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:24:13.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6100000000000002e-07 2023-11-25T04:24:13.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.214 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.214 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:13.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044019+0000", 2023-11-25T04:24:13.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.839e-06 2023-11-25T04:24:13.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.214 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.214 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:13.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044026+0000", 2023-11-25T04:24:13.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5869999999999998e-06 2023-11-25T04:24:13.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:13.215 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044031+0000", 2023-11-25T04:24:13.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4909999999999996e-06 2023-11-25T04:24:13.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:13.215 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044070+0000", 2023-11-25T04:24:13.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8829999999999999e-05 2023-11-25T04:24:13.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:13.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044081+0000", 2023-11-25T04:24:13.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1447999999999999e-05 2023-11-25T04:24:13.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:13.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049406+0000", 2023-11-25T04:24:13.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053246480000000004 2023-11-25T04:24:13.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:13.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049464+0000", 2023-11-25T04:24:13.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8205999999999998e-05 2023-11-25T04:24:13.217 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:13.217 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:13.217 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:13.217 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.217 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:11 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'6, mlcod=0'0)", 2023-11-25T04:24:13.217 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:24:13.217 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 86.058749329999998, 2023-11-25T04:24:13.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052673499999999996, 2023-11-25T04:24:13.218 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:13.218 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:13.218 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:13.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.218 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:13.218 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:24:13.218 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:13.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.218 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:13.219 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:24:13.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:13.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.219 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:13.219 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:24:13.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0199999999999998e-07 2023-11-25T04:24:13.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.219 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:13.219 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044220+0000", 2023-11-25T04:24:13.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7720000000000001e-06 2023-11-25T04:24:13.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:13.220 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044226+0000", 2023-11-25T04:24:13.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8579999999999996e-06 2023-11-25T04:24:13.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:13.220 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044232+0000", 2023-11-25T04:24:13.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1700000000000002e-06 2023-11-25T04:24:13.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:13.221 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044274+0000", 2023-11-25T04:24:13.221 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1254999999999997e-05 2023-11-25T04:24:13.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.221 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.221 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:13.221 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:24:13.221 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6880999999999999e-05 2023-11-25T04:24:13.221 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.222 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:13.222 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049466+0000", 2023-11-25T04:24:13.222 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051659219999999999 2023-11-25T04:24:13.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:13.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:13.222 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:13.222 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049483+0000", 2023-11-25T04:24:13.222 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.719e-05 2023-11-25T04:24:13.222 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:13.222 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:13.222 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:13.223 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:13.223 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:13.223 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:13.224 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:24:13.225 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:13.223+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:13.325 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:13.324+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:13.333 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:13.334 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:24:13.334 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2023-11-25T04:24:13.334 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:13.342 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:24:13.426 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:13.425+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:13.448 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:13.449 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:24:13.449 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2023-11-25T04:24:13.449 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2023-11-25T04:24:13.449 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:13.457 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:24:13.527 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~64s 2023-11-25T04:24:13.527 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:13.526+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:13.564 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:13.565 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2023-11-25T04:24:13.565 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2023-11-25T04:24:13.565 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2023-11-25T04:24:13.565 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.565 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:13.565 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:24:13.565 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 86.530692501000004, 2023-11-25T04:24:13.565 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01072943, 2023-11-25T04:24:13.565 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:13.565 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:13.565 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:13.566 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.566 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:13.566 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:24:13.566 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:13.566 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.566 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.566 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:13.566 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:24:13.566 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:13.566 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.566 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.566 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:13.566 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031829+0000", 2023-11-25T04:24:13.567 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.666e-06 2023-11-25T04:24:13.567 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.567 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.567 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:13.567 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031839+0000", 2023-11-25T04:24:13.567 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9490000000000008e-06 2023-11-25T04:24:13.567 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.567 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.567 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:13.567 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031857+0000", 2023-11-25T04:24:13.567 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8093000000000001e-05 2023-11-25T04:24:13.567 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.567 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.568 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:13.568 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031882+0000", 2023-11-25T04:24:13.568 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4294000000000002e-05 2023-11-25T04:24:13.568 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.568 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.568 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:13.568 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033130+0000", 2023-11-25T04:24:13.568 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001248264 2023-11-25T04:24:13.568 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.568 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.568 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:13.568 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033426+0000", 2023-11-25T04:24:13.568 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029620999999999998 2023-11-25T04:24:13.569 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.569 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.569 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:13.569 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033814+0000", 2023-11-25T04:24:13.569 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000387742 2023-11-25T04:24:13.569 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.569 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.569 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:13.569 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036793+0000", 2023-11-25T04:24:13.569 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002979598 2023-11-25T04:24:13.569 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.569 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.570 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:13.570 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042455+0000", 2023-11-25T04:24:13.570 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0056619260000000003 2023-11-25T04:24:13.570 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.570 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.570 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:13.570 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042502+0000", 2023-11-25T04:24:13.570 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7139999999999999e-05 2023-11-25T04:24:13.570 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.570 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.570 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:13.570 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042557+0000", 2023-11-25T04:24:13.571 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4548e-05 2023-11-25T04:24:13.571 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:13.571 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:13.571 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:13.571 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.571 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.571 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:13.571 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:24:13.571 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 86.529493966000004, 2023-11-25T04:24:13.571 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011505202000000001, 2023-11-25T04:24:13.571 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:13.571 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:13.571 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:13.572 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.572 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:13.572 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:24:13.572 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:13.572 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.572 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.572 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:13.572 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:24:13.572 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:13.572 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.572 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.572 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:13.573 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033027+0000", 2023-11-25T04:24:13.573 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0699999999999999e-06 2023-11-25T04:24:13.573 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.573 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.573 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:13.573 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033035+0000", 2023-11-25T04:24:13.573 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0830000000000004e-06 2023-11-25T04:24:13.573 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.573 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.573 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:13.573 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033059+0000", 2023-11-25T04:24:13.573 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3975e-05 2023-11-25T04:24:13.573 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.573 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.574 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:13.574 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033482+0000", 2023-11-25T04:24:13.574 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042253700000000001 2023-11-25T04:24:13.574 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.574 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.574 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:13.574 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034520+0000", 2023-11-25T04:24:13.574 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001037991 2023-11-25T04:24:13.574 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.574 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.574 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:13.574 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034611+0000", 2023-11-25T04:24:13.575 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1113999999999995e-05 2023-11-25T04:24:13.575 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.575 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.575 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:13.575 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034769+0000", 2023-11-25T04:24:13.575 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000157641 2023-11-25T04:24:13.575 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.575 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.575 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:13.575 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038292+0000", 2023-11-25T04:24:13.575 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035231160000000002 2023-11-25T04:24:13.575 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.576 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.576 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:13.576 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044440+0000", 2023-11-25T04:24:13.576 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0061486980000000002 2023-11-25T04:24:13.576 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.576 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.576 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:13.576 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044495+0000", 2023-11-25T04:24:13.576 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4339000000000003e-05 2023-11-25T04:24:13.576 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.576 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.576 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:13.576 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044531+0000", 2023-11-25T04:24:13.576 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6637999999999997e-05 2023-11-25T04:24:13.577 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:13.577 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:13.577 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:13.577 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.577 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.577 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:13.577 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:24:13.578 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 86.529019611999999, 2023-11-25T04:24:13.578 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0057634039999999997, 2023-11-25T04:24:13.578 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:13.578 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:13.578 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:13.578 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.578 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:13.579 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:24:13.579 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:13.579 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.579 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.579 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:13.579 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:24:13.579 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:13.579 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.579 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.579 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:13.579 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:24:13.579 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7900000000000007e-07 2023-11-25T04:24:13.579 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.579 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.580 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:13.580 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033507+0000", 2023-11-25T04:24:13.580 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8900000000000004e-06 2023-11-25T04:24:13.580 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.580 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.580 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:13.580 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033519+0000", 2023-11-25T04:24:13.580 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1756000000000001e-05 2023-11-25T04:24:13.580 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.580 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.580 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:13.580 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033527+0000", 2023-11-25T04:24:13.580 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3610000000000001e-06 2023-11-25T04:24:13.581 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.581 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.581 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:13.581 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034451+0000", 2023-11-25T04:24:13.581 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00092372400000000001 2023-11-25T04:24:13.581 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.581 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.581 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:13.581 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034809+0000", 2023-11-25T04:24:13.581 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00035757699999999998 2023-11-25T04:24:13.581 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.581 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.582 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:13.582 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034965+0000", 2023-11-25T04:24:13.582 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015638699999999999 2023-11-25T04:24:13.582 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.582 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.582 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:13.582 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038299+0000", 2023-11-25T04:24:13.582 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0033336559999999999 2023-11-25T04:24:13.582 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.582 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.582 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:13.582 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039169+0000", 2023-11-25T04:24:13.583 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00087066400000000005 2023-11-25T04:24:13.583 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.583 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.583 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:13.583 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039226+0000", 2023-11-25T04:24:13.583 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6511999999999997e-05 2023-11-25T04:24:13.583 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.583 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.583 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:13.583 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039264+0000", 2023-11-25T04:24:13.583 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7898000000000002e-05 2023-11-25T04:24:13.583 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:13.584 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:13.584 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:13.584 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.584 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.584 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:25 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:13.584 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:24:13.584 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 86.528941028000006, 2023-11-25T04:24:13.584 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096656169999999996, 2023-11-25T04:24:13.584 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:13.584 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:13.584 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:13.584 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.585 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:13.585 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:24:13.585 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:13.585 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.585 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.585 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:13.585 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:24:13.585 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:13.585 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.585 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.585 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:13.585 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033580+0000", 2023-11-25T04:24:13.585 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8199999999999999e-07 2023-11-25T04:24:13.586 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.586 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.586 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:13.586 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033585+0000", 2023-11-25T04:24:13.586 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3680000000000001e-06 2023-11-25T04:24:13.586 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.586 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.586 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:13.586 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033600+0000", 2023-11-25T04:24:13.586 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4331000000000001e-05 2023-11-25T04:24:13.586 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.587 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.587 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:13.587 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033606+0000", 2023-11-25T04:24:13.587 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5780000000000004e-06 2023-11-25T04:24:13.587 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.587 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.587 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:13.587 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035747+0000", 2023-11-25T04:24:13.588 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0021406530000000002 2023-11-25T04:24:13.588 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.588 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.588 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:13.588 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035901+0000", 2023-11-25T04:24:13.588 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000153931 2023-11-25T04:24:13.588 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.588 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.588 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:13.588 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036193+0000", 2023-11-25T04:24:13.588 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029211299999999999 2023-11-25T04:24:13.588 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.589 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.589 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:13.589 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038655+0000", 2023-11-25T04:24:13.589 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0024621640000000002 2023-11-25T04:24:13.589 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.589 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.589 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:13.589 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043130+0000", 2023-11-25T04:24:13.589 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.004474698 2023-11-25T04:24:13.589 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.589 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.589 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:13.590 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043204+0000", 2023-11-25T04:24:13.590 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4091999999999995e-05 2023-11-25T04:24:13.590 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.590 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.590 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:13.590 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043245+0000", 2023-11-25T04:24:13.590 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0907e-05 2023-11-25T04:24:13.590 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:13.590 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:13.590 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:13.590 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.590 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.591 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:26 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:13.591 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:13.591 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 86.528905203999997, 2023-11-25T04:24:13.591 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011086945000000001, 2023-11-25T04:24:13.591 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:13.591 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:13.591 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:13.591 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.591 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:13.591 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:13.591 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:13.591 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.591 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.592 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:13.592 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:13.592 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:13.592 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.592 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.592 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:13.592 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:13.592 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1699999999999998e-07 2023-11-25T04:24:13.592 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.592 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.592 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:13.592 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033627+0000", 2023-11-25T04:24:13.593 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.163e-05 2023-11-25T04:24:13.593 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.593 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.593 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:13.593 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033636+0000", 2023-11-25T04:24:13.593 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6859999999999995e-06 2023-11-25T04:24:13.593 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.593 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.593 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:13.593 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033642+0000", 2023-11-25T04:24:13.593 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3149999999999997e-06 2023-11-25T04:24:13.594 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.594 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.594 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:13.594 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035414+0000", 2023-11-25T04:24:13.594 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0017716380000000001 2023-11-25T04:24:13.594 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.594 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.594 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:13.594 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035539+0000", 2023-11-25T04:24:13.594 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000125477 2023-11-25T04:24:13.594 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.594 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.594 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:13.595 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035672+0000", 2023-11-25T04:24:13.595 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000133316 2023-11-25T04:24:13.595 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.595 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.595 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:13.595 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038649+0000", 2023-11-25T04:24:13.595 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029764679999999999 2023-11-25T04:24:13.595 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.595 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.595 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:13.595 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044578+0000", 2023-11-25T04:24:13.595 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0059293510000000002 2023-11-25T04:24:13.596 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.596 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.596 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:13.596 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044651+0000", 2023-11-25T04:24:13.596 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2513000000000002e-05 2023-11-25T04:24:13.596 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.596 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.596 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:13.596 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044702+0000", 2023-11-25T04:24:13.596 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1034e-05 2023-11-25T04:24:13.596 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:13.596 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:13.597 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:13.597 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.597 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.597 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:29 3.b 3:d0630e4c:::mds0_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:13.597 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:13.597 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 86.528867284, 2023-11-25T04:24:13.597 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010980227, 2023-11-25T04:24:13.597 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:13.597 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:13.597 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:13.597 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.597 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:13.598 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:13.598 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:13.598 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.598 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.598 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:13.598 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:13.598 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:13.598 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.598 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.598 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:13.598 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:13.598 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3499999999999996e-07 2023-11-25T04:24:13.599 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.599 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.599 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:13.599 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033658+0000", 2023-11-25T04:24:13.599 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1280000000000001e-06 2023-11-25T04:24:13.599 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.599 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.599 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:13.600 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033667+0000", 2023-11-25T04:24:13.600 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1430000000000004e-06 2023-11-25T04:24:13.600 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.600 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.600 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:13.600 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033678+0000", 2023-11-25T04:24:13.600 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0906e-05 2023-11-25T04:24:13.600 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.600 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.600 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:13.600 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034456+0000", 2023-11-25T04:24:13.601 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00077833899999999996 2023-11-25T04:24:13.601 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.601 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.601 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:13.601 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034575+0000", 2023-11-25T04:24:13.601 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011926000000000001 2023-11-25T04:24:13.601 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.601 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.601 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:13.601 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035001+0000", 2023-11-25T04:24:13.601 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042550999999999998 2023-11-25T04:24:13.601 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.602 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.602 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:13.602 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.037923+0000", 2023-11-25T04:24:13.602 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029219670000000001 2023-11-25T04:24:13.602 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.602 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.602 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:13.602 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044538+0000", 2023-11-25T04:24:13.602 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0066152349999999997 2023-11-25T04:24:13.602 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.602 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.603 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:13.603 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044594+0000", 2023-11-25T04:24:13.603 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5593000000000001e-05 2023-11-25T04:24:13.603 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.603 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.603 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:13.603 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044633+0000", 2023-11-25T04:24:13.603 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9511000000000002e-05 2023-11-25T04:24:13.603 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:13.603 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:13.603 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:13.603 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.604 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.604 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:34 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:13.604 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:24:13.604 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 86.528617170999993, 2023-11-25T04:24:13.604 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010920445000000001, 2023-11-25T04:24:13.604 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:13.604 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:13.604 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:13.604 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.604 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:13.604 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:24:13.604 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:13.605 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.605 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.605 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:13.605 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:24:13.605 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:13.605 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.605 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.605 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:13.605 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033904+0000", 2023-11-25T04:24:13.605 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6599999999999994e-07 2023-11-25T04:24:13.605 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.605 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.605 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:13.606 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033909+0000", 2023-11-25T04:24:13.606 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5639999999999996e-06 2023-11-25T04:24:13.606 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.606 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.606 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:13.606 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033923+0000", 2023-11-25T04:24:13.606 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3028e-05 2023-11-25T04:24:13.606 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.606 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.606 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:13.606 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034043+0000", 2023-11-25T04:24:13.606 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000120328 2023-11-25T04:24:13.606 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.607 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.607 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:13.607 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035576+0000", 2023-11-25T04:24:13.607 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015333720000000001 2023-11-25T04:24:13.607 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.607 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.607 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:13.607 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035699+0000", 2023-11-25T04:24:13.607 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000123001 2023-11-25T04:24:13.607 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.607 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.607 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:13.608 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044823+0000", 2023-11-25T04:24:13.608 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0091241859999999994 2023-11-25T04:24:13.608 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:13.608 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:13.608 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:13.608 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.608 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.608 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:35 3.1f 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:13.608 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:24:13.608 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 86.527312965999997, 2023-11-25T04:24:13.608 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096164779999999995, 2023-11-25T04:24:13.608 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:13.608 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:13.609 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:13.609 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.609 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:13.609 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:24:13.609 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:13.609 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.609 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.609 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:13.609 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:24:13.609 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:13.609 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.609 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.609 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:13.610 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035208+0000", 2023-11-25T04:24:13.610 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2699999999999998e-07 2023-11-25T04:24:13.610 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.610 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.610 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:13.610 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035213+0000", 2023-11-25T04:24:13.610 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.6070000000000001e-06 2023-11-25T04:24:13.610 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.610 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.610 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:13.610 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035350+0000", 2023-11-25T04:24:13.610 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013693599999999999 2023-11-25T04:24:13.611 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.611 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.611 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:13.611 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035358+0000", 2023-11-25T04:24:13.611 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0099999999999995e-06 2023-11-25T04:24:13.611 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.611 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.611 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:13.611 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036260+0000", 2023-11-25T04:24:13.611 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00090240100000000005 2023-11-25T04:24:13.611 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.611 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.612 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:13.612 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036517+0000", 2023-11-25T04:24:13.612 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00025736299999999998 2023-11-25T04:24:13.612 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.612 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.612 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:13.612 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044824+0000", 2023-11-25T04:24:13.612 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0083063340000000003 2023-11-25T04:24:13.612 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:13.612 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:13.612 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:13.612 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.612 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.613 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:37 3.1e 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:13.613 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:24:13.613 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 86.527146279999997, 2023-11-25T04:24:13.613 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0078322640000000002, 2023-11-25T04:24:13.613 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:13.613 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:13.613 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:13.613 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.613 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:13.613 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:24:13.613 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:13.613 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.614 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.614 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:13.614 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:24:13.614 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:13.614 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.614 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.614 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:13.614 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035375+0000", 2023-11-25T04:24:13.615 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9100000000000003e-07 2023-11-25T04:24:13.615 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.615 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.615 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:13.615 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035379+0000", 2023-11-25T04:24:13.615 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9430000000000002e-06 2023-11-25T04:24:13.615 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.615 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.615 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:13.615 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035388+0000", 2023-11-25T04:24:13.615 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2760000000000006e-06 2023-11-25T04:24:13.615 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.616 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.616 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:13.616 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035396+0000", 2023-11-25T04:24:13.616 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8480000000000001e-06 2023-11-25T04:24:13.616 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.616 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.616 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:13.616 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035452+0000", 2023-11-25T04:24:13.616 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5831000000000001e-05 2023-11-25T04:24:13.616 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.616 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.616 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:13.617 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035934+0000", 2023-11-25T04:24:13.617 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00048265599999999998 2023-11-25T04:24:13.617 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.617 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.617 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:13.617 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043206+0000", 2023-11-25T04:24:13.617 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.007271919 2023-11-25T04:24:13.617 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:13.617 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:13.617 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:13.617 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.618 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.618 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing(3.1a epoch 18/16 rep_tid 3 entries 18'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:36 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:24:13.618 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:24:13.618 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 86.525870694000005, 2023-11-25T04:24:13.618 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002073414, 2023-11-25T04:24:13.618 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:13.618 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:24:13.618 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:13.618 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.618 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:13.618 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:24:13.618 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:13.619 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.619 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.619 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:13.619 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:24:13.619 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:13.619 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.619 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.619 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:13.619 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036650+0000", 2023-11-25T04:24:13.619 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7300000000000002e-07 2023-11-25T04:24:13.619 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.619 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.620 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:13.620 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036659+0000", 2023-11-25T04:24:13.620 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.7499999999999992e-06 2023-11-25T04:24:13.620 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.620 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.620 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:13.620 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036673+0000", 2023-11-25T04:24:13.620 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4917e-05 2023-11-25T04:24:13.620 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.620 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.620 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:13.620 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036687+0000", 2023-11-25T04:24:13.621 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3883000000000001e-05 2023-11-25T04:24:13.621 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.621 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.621 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:13.621 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036781+0000", 2023-11-25T04:24:13.621 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3219000000000003e-05 2023-11-25T04:24:13.621 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.621 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.621 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:13.621 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038723+0000", 2023-11-25T04:24:13.621 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019422720000000001 2023-11-25T04:24:13.621 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:13.622 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:13.622 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:13.622 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.622 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.622 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:28 3.13 e18/16 3:cd5a64a3:::100.00000000:head v 18'1, mlcod=0'0)", 2023-11-25T04:24:13.622 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:24:13.622 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 86.522209536000005, 2023-11-25T04:24:13.622 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001850546, 2023-11-25T04:24:13.622 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:13.622 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:13.622 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:13.623 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.623 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:13.623 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:24:13.623 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:13.623 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.623 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.623 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:13.623 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:24:13.623 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:13.623 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.623 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.623 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:13.624 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:24:13.624 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7800000000000002e-07 2023-11-25T04:24:13.624 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.624 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.624 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:13.624 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040331+0000", 2023-11-25T04:24:13.624 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0449999999999999e-05 2023-11-25T04:24:13.624 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.624 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.624 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:13.624 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040339+0000", 2023-11-25T04:24:13.624 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0709999999999995e-06 2023-11-25T04:24:13.625 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.625 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.625 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:13.625 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040347+0000", 2023-11-25T04:24:13.625 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7079999999999997e-06 2023-11-25T04:24:13.625 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.625 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.625 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:13.625 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040410+0000", 2023-11-25T04:24:13.625 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2453000000000007e-05 2023-11-25T04:24:13.625 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.625 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.625 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:13.626 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040422+0000", 2023-11-25T04:24:13.626 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2429000000000001e-05 2023-11-25T04:24:13.626 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.626 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.626 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:13.626 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042129+0000", 2023-11-25T04:24:13.626 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001707266 2023-11-25T04:24:13.626 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.626 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.626 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:13.626 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042161+0000", 2023-11-25T04:24:13.627 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1791000000000001e-05 2023-11-25T04:24:13.627 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:13.627 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:13.627 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:13.627 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.627 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.627 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:27 3.13 e18/16 3:c91d4a1d:::mds0_inotable:head v 18'2, mlcod=0'0)", 2023-11-25T04:24:13.627 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:24:13.627 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 86.522006012999995, 2023-11-25T04:24:13.627 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0016886430000000001, 2023-11-25T04:24:13.627 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:13.627 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:13.628 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:13.628 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.628 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:13.628 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:24:13.628 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:13.628 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.628 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.628 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:13.628 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:24:13.628 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:13.628 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.628 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.628 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:13.629 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:24:13.629 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3999999999999997e-07 2023-11-25T04:24:13.629 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.629 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.629 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:13.629 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040520+0000", 2023-11-25T04:24:13.629 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2989999999999996e-06 2023-11-25T04:24:13.629 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.629 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.629 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:13.629 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040527+0000", 2023-11-25T04:24:13.629 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5730000000000002e-06 2023-11-25T04:24:13.630 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.630 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.630 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:13.630 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040535+0000", 2023-11-25T04:24:13.630 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8630000000000008e-06 2023-11-25T04:24:13.630 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.630 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.630 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:13.630 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040683+0000", 2023-11-25T04:24:13.630 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000148002 2023-11-25T04:24:13.630 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.630 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.631 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:13.631 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040700+0000", 2023-11-25T04:24:13.631 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7065e-05 2023-11-25T04:24:13.631 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.631 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.631 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:13.631 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042163+0000", 2023-11-25T04:24:13.631 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001462767 2023-11-25T04:24:13.631 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.631 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.631 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:13.631 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042203+0000", 2023-11-25T04:24:13.631 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9734000000000003e-05 2023-11-25T04:24:13.632 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:13.632 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:13.632 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:13.632 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.632 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.632 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:20 3.1c e18/16 3:3ed09add:::607.00000000:head v 18'1, mlcod=0'0)", 2023-11-25T04:24:13.632 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:24:13.632 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 86.521970972000005, 2023-11-25T04:24:13.632 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015271, 2023-11-25T04:24:13.632 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:13.632 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:13.632 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:13.633 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.633 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:13.633 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:24:13.633 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:13.633 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.633 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.633 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:13.633 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:24:13.633 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:13.633 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.633 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.633 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:13.633 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040549+0000", 2023-11-25T04:24:13.634 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4799999999999999e-07 2023-11-25T04:24:13.634 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.634 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.634 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:13.634 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040561+0000", 2023-11-25T04:24:13.634 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1800000000000001e-05 2023-11-25T04:24:13.634 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.634 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.634 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:13.634 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040568+0000", 2023-11-25T04:24:13.634 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7730000000000002e-06 2023-11-25T04:24:13.634 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.634 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.635 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:13.635 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040587+0000", 2023-11-25T04:24:13.635 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8691000000000001e-05 2023-11-25T04:24:13.635 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.635 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.635 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:13.635 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040638+0000", 2023-11-25T04:24:13.635 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1068e-05 2023-11-25T04:24:13.635 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.635 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.635 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:13.636 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040670+0000", 2023-11-25T04:24:13.636 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2125999999999997e-05 2023-11-25T04:24:13.636 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.636 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.636 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:13.636 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042032+0000", 2023-11-25T04:24:13.636 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001361821 2023-11-25T04:24:13.636 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.636 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.636 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:13.636 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042076+0000", 2023-11-25T04:24:13.636 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4472999999999999e-05 2023-11-25T04:24:13.636 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:13.637 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:13.637 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:13.637 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.637 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.637 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:21 3.1c e18/16 3:3ed09add:::607.00000000:head v 18'2, mlcod=0'0)", 2023-11-25T04:24:13.637 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:24:13.637 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 86.521736247000007, 2023-11-25T04:24:13.637 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0013228160000000001, 2023-11-25T04:24:13.637 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:13.637 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:13.637 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:13.637 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.637 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:13.637 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:24:13.638 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:13.638 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.638 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.638 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:13.638 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:24:13.638 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:13.638 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.638 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.638 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:13.638 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040784+0000", 2023-11-25T04:24:13.638 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9299999999999999e-07 2023-11-25T04:24:13.638 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.638 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.639 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:13.639 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040790+0000", 2023-11-25T04:24:13.639 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5770000000000001e-06 2023-11-25T04:24:13.639 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.639 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.639 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:13.639 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040799+0000", 2023-11-25T04:24:13.639 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6439999999999995e-06 2023-11-25T04:24:13.639 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.639 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.639 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:13.639 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040806+0000", 2023-11-25T04:24:13.639 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0180000000000004e-06 2023-11-25T04:24:13.640 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.640 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.640 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:13.640 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040907+0000", 2023-11-25T04:24:13.640 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000100096 2023-11-25T04:24:13.640 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.640 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.640 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:13.640 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040925+0000", 2023-11-25T04:24:13.640 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8536999999999999e-05 2023-11-25T04:24:13.640 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.640 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.640 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:13.640 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042079+0000", 2023-11-25T04:24:13.641 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00115358 2023-11-25T04:24:13.641 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.641 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.641 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:13.641 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042107+0000", 2023-11-25T04:24:13.641 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7971e-05 2023-11-25T04:24:13.641 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:13.641 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:13.641 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:13.642 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.642 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.642 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:39 3.6 3:654134d2:::mds0_openfiles.0:head [omap-clear,omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2023-11-25T04:24:13.642 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:24:13.642 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 81.458208741999997, 2023-11-25T04:24:13.642 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001788658, 2023-11-25T04:24:13.642 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:13.642 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:13.642 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:13.642 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.642 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:13.642 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:24:13.642 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:13.643 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.643 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.643 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:13.643 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:24:13.643 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:13.643 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.643 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.643 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:13.643 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104314+0000", 2023-11-25T04:24:13.643 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3099999999999999e-06 2023-11-25T04:24:13.643 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.643 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.643 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:13.644 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104325+0000", 2023-11-25T04:24:13.644 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1511999999999999e-05 2023-11-25T04:24:13.644 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.644 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.644 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:13.644 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104362+0000", 2023-11-25T04:24:13.644 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6903000000000002e-05 2023-11-25T04:24:13.644 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.644 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.644 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:13.644 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104387+0000", 2023-11-25T04:24:13.644 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4766e-05 2023-11-25T04:24:13.644 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.645 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.645 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:13.645 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104465+0000", 2023-11-25T04:24:13.645 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8119000000000002e-05 2023-11-25T04:24:13.645 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.645 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.645 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:13.645 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104557+0000", 2023-11-25T04:24:13.645 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2496000000000003e-05 2023-11-25T04:24:13.645 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.645 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.645 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:13.646 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104646+0000", 2023-11-25T04:24:13.646 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8818000000000002e-05 2023-11-25T04:24:13.646 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.646 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.646 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:13.646 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105201+0000", 2023-11-25T04:24:13.646 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00055506000000000002 2023-11-25T04:24:13.646 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.646 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.646 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:13.646 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105994+0000", 2023-11-25T04:24:13.646 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00079294700000000001 2023-11-25T04:24:13.647 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.647 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.647 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:13.647 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.106055+0000", 2023-11-25T04:24:13.647 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0566000000000003e-05 2023-11-25T04:24:13.647 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.647 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.647 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:13.647 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.106100+0000", 2023-11-25T04:24:13.647 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5160999999999997e-05 2023-11-25T04:24:13.647 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:13.647 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:13.648 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:13.648 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.648 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.648 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:43 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2023-11-25T04:24:13.648 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:24:13.648 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 66.457983745999996, 2023-11-25T04:24:13.648 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001751166, 2023-11-25T04:24:13.648 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:13.648 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:13.648 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:13.648 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.648 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:13.648 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:24:13.648 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:13.649 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.649 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.649 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:13.649 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:24:13.649 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:13.649 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.649 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.649 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:13.649 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104539+0000", 2023-11-25T04:24:13.649 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9529999999999999e-06 2023-11-25T04:24:13.649 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.649 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.649 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:13.650 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104557+0000", 2023-11-25T04:24:13.650 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7464000000000001e-05 2023-11-25T04:24:13.650 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.650 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.650 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:13.650 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104590+0000", 2023-11-25T04:24:13.650 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2966000000000003e-05 2023-11-25T04:24:13.650 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.650 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.650 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:13.650 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104608+0000", 2023-11-25T04:24:13.650 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8193e-05 2023-11-25T04:24:13.650 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.651 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.651 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:13.651 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104767+0000", 2023-11-25T04:24:13.651 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015882 2023-11-25T04:24:13.651 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.651 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.651 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:13.651 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104909+0000", 2023-11-25T04:24:13.651 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014244999999999999 2023-11-25T04:24:13.651 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.651 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.651 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:13.651 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.105059+0000", 2023-11-25T04:24:13.651 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014964399999999999 2023-11-25T04:24:13.652 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.652 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.652 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:13.652 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.105701+0000", 2023-11-25T04:24:13.652 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00064201099999999995 2023-11-25T04:24:13.652 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.652 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.652 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:13.652 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106233+0000", 2023-11-25T04:24:13.652 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00053178500000000001 2023-11-25T04:24:13.652 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.652 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.652 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:13.652 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106265+0000", 2023-11-25T04:24:13.653 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1986000000000002e-05 2023-11-25T04:24:13.653 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.653 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.653 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:13.653 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106288+0000", 2023-11-25T04:24:13.653 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2894000000000001e-05 2023-11-25T04:24:13.653 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:13.653 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:13.653 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:13.653 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.653 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.653 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:64 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e43)", 2023-11-25T04:24:13.653 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:12.105733+0000", 2023-11-25T04:24:13.653 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 1.4567868530000001, 2023-11-25T04:24:13.654 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0010789790000000001, 2023-11-25T04:24:13.654 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:13.654 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:13.654 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:13.654 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.654 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:13.654 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:12.105733+0000", 2023-11-25T04:24:13.654 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:13.654 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.654 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.654 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:13.654 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:12.105733+0000", 2023-11-25T04:24:13.654 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:13.655 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.655 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.655 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:13.655 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:12.105735+0000", 2023-11-25T04:24:13.655 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.632e-06 2023-11-25T04:24:13.655 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.655 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.655 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:13.655 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:12.105750+0000", 2023-11-25T04:24:13.655 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4721e-05 2023-11-25T04:24:13.655 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.655 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.655 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:13.655 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:12.105764+0000", 2023-11-25T04:24:13.656 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4558e-05 2023-11-25T04:24:13.656 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.656 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.656 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:13.656 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:12.105771+0000", 2023-11-25T04:24:13.656 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.041e-06 2023-11-25T04:24:13.656 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.656 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.656 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:13.656 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:12.105843+0000", 2023-11-25T04:24:13.656 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1818999999999998e-05 2023-11-25T04:24:13.656 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.656 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.656 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:13.657 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:12.105910+0000", 2023-11-25T04:24:13.657 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7340000000000002e-05 2023-11-25T04:24:13.657 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.657 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.657 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:13.657 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:12.106731+0000", 2023-11-25T04:24:13.657 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00082021600000000002 2023-11-25T04:24:13.657 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.657 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.657 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:13.657 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:12.106767+0000", 2023-11-25T04:24:13.657 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6028000000000001e-05 2023-11-25T04:24:13.657 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.658 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.658 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:13.658 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:12.106812+0000", 2023-11-25T04:24:13.658 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5624000000000002e-05 2023-11-25T04:24:13.658 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:13.658 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:13.658 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:13.658 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.658 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.658 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:66 3.1 3:86ee9bce:::500.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e43)", 2023-11-25T04:24:13.658 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:13.061832+0000", 2023-11-25T04:24:13.658 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 0.50068845200000001, 2023-11-25T04:24:13.658 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00106825, 2023-11-25T04:24:13.659 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:13.659 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:13.659 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:13.659 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.659 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:13.659 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:13.061832+0000", 2023-11-25T04:24:13.659 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:13.659 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.659 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.659 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:13.659 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:13.061832+0000", 2023-11-25T04:24:13.659 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:13.659 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.659 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.660 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:13.660 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:13.061833+0000", 2023-11-25T04:24:13.660 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6050000000000001e-06 2023-11-25T04:24:13.660 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.660 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.660 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:13.660 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:13.061839+0000", 2023-11-25T04:24:13.660 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4890000000000003e-06 2023-11-25T04:24:13.660 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.660 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.660 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:13.660 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:13.061853+0000", 2023-11-25T04:24:13.660 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4047e-05 2023-11-25T04:24:13.661 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.661 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.661 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:13.661 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:13.061877+0000", 2023-11-25T04:24:13.661 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4491e-05 2023-11-25T04:24:13.661 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.661 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.661 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:13.661 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:13.061940+0000", 2023-11-25T04:24:13.661 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2633999999999997e-05 2023-11-25T04:24:13.661 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.661 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.661 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:13.661 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:13.062073+0000", 2023-11-25T04:24:13.662 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000132879 2023-11-25T04:24:13.662 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.662 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.662 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:13.662 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:13.062900+0000", 2023-11-25T04:24:13.662 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00082710499999999996 2023-11-25T04:24:13.662 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:13.662 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:13.662 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:13.662 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.662 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.662 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:81 4.b 4:d2dbd4ac:::10000000217.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e43)", 2023-11-25T04:24:13.662 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:13.064495+0000", 2023-11-25T04:24:13.663 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 0.49802466099999998, 2023-11-25T04:24:13.663 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001134379, 2023-11-25T04:24:13.663 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:13.663 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:13.663 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:13.663 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.663 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:13.663 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:13.064495+0000", 2023-11-25T04:24:13.663 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:13.663 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.663 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.663 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:13.663 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:13.064495+0000", 2023-11-25T04:24:13.663 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:13.664 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.664 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.664 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:13.664 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:13.064496+0000", 2023-11-25T04:24:13.664 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7e-07 2023-11-25T04:24:13.664 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.664 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.664 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:13.664 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:13.064498+0000", 2023-11-25T04:24:13.664 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0059999999999999e-06 2023-11-25T04:24:13.664 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.664 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.664 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:13.665 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:13.064502+0000", 2023-11-25T04:24:13.665 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3499999999999999e-06 2023-11-25T04:24:13.665 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.665 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.665 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:13.665 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:13.064505+0000", 2023-11-25T04:24:13.665 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9459999999999998e-06 2023-11-25T04:24:13.665 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.665 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.665 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:13.665 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:13.064932+0000", 2023-11-25T04:24:13.665 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042655599999999997 2023-11-25T04:24:13.665 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.665 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.666 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:13.666 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:13.065043+0000", 2023-11-25T04:24:13.666 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000111482 2023-11-25T04:24:13.666 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.666 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.666 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:13.666 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:13.065552+0000", 2023-11-25T04:24:13.666 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00050906300000000003 2023-11-25T04:24:13.666 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.666 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.666 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:13.666 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:13.065593+0000", 2023-11-25T04:24:13.666 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.057e-05 2023-11-25T04:24:13.667 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.667 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.667 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:13.667 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:13.065630+0000", 2023-11-25T04:24:13.667 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7036000000000003e-05 2023-11-25T04:24:13.667 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:13.667 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:13.667 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:13.667 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.667 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.667 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:82 4.2d 4:b79a54f6:::10000000218.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e43)", 2023-11-25T04:24:13.667 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:13.064509+0000", 2023-11-25T04:24:13.667 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 0.49801064900000003, 2023-11-25T04:24:13.668 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001532781, 2023-11-25T04:24:13.668 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:13.668 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:13.668 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:13.668 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.668 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:13.668 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:13.064509+0000", 2023-11-25T04:24:13.668 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:13.668 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.668 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.668 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:13.668 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:13.064509+0000", 2023-11-25T04:24:13.668 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:13.668 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.669 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.669 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:13.669 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:13.064510+0000", 2023-11-25T04:24:13.669 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5699999999999999e-07 2023-11-25T04:24:13.669 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.669 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.669 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:13.669 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:13.064511+0000", 2023-11-25T04:24:13.669 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0410000000000001e-06 2023-11-25T04:24:13.669 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.669 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.669 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:13.669 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:13.064515+0000", 2023-11-25T04:24:13.669 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3290000000000004e-06 2023-11-25T04:24:13.670 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.670 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.670 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:13.670 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:13.064519+0000", 2023-11-25T04:24:13.670 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.031e-06 2023-11-25T04:24:13.670 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.670 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.670 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:13.670 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:13.064563+0000", 2023-11-25T04:24:13.670 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4243999999999999e-05 2023-11-25T04:24:13.670 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.670 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.670 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:13.671 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:13.064643+0000", 2023-11-25T04:24:13.671 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0036000000000004e-05 2023-11-25T04:24:13.671 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.671 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.671 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:13.671 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:13.064735+0000", 2023-11-25T04:24:13.671 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1924999999999998e-05 2023-11-25T04:24:13.671 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.672 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.672 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:13.672 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:13.065362+0000", 2023-11-25T04:24:13.673 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00062637000000000001 2023-11-25T04:24:13.673 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.673 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.673 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:13.673 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:13.065991+0000", 2023-11-25T04:24:13.673 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00062938899999999995 2023-11-25T04:24:13.673 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.673 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.673 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:13.673 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:13.066025+0000", 2023-11-25T04:24:13.673 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4034e-05 2023-11-25T04:24:13.673 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:13.673 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:13.674 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:13.674 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:13.066042+0000", 2023-11-25T04:24:13.674 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7125e-05 2023-11-25T04:24:13.674 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:13.674 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:13.674 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:13.674 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:13.674 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:13.674 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:13.676 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:13.674+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:13.676 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:24:13.777 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:13.775+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:13.783 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-11-25T04:24:13.785 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:13.785 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:24:13.786 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2023-11-25T04:24:13.786 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:13.794 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:24:13.878 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:13.877+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:13.901 INFO:teuthology.orchestra.run.smithi067.stderr:Error ENXIO: problem getting command descriptions from osd.0 2023-11-25T04:24:13.923 INFO:teuthology.orchestra.run.smithi067.stdout:osd.1: {} 2023-11-25T04:24:13.923 INFO:teuthology.orchestra.run.smithi067.stderr:osd.1: osd_enable_op_tracker = '' 2023-11-25T04:24:13.939 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:13.940 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:13.946 INFO:teuthology.orchestra.run.smithi067.stdout:osd.2: {} 2023-11-25T04:24:13.946 INFO:teuthology.orchestra.run.smithi067.stderr:osd.2: osd_enable_op_tracker = '' 2023-11-25T04:24:13.947 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-11-25T04:24:13.948 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-11-25T04:24:13.949 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:24:13.978 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:13.977+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:14.059 INFO:teuthology.orchestra.run.smithi067.stderr:nodeep-scrub is unset 2023-11-25T04:24:14.079 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:14.078+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:14.098 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:14.098 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:14.106 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:24:14.182 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:14.179+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:14.213 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:14.214 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:14.222 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:24:14.281 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:14.280+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:14.367 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:14.367 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:14.376 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:24:14.420 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:14.418+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:14.481 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:14.482 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:14.483 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:14.481+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:14.490 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:24:14.584 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:14.583+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:14.597 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:14.597 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:14.605 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:24:14.689 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:14.688+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:14.755 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:14.755 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:14.762 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:24:14.803 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:14.802+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:14.886 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:14.885+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:14.909 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:14.909 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:14.918 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:24:14.987 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:14.986+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:15.042 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:15.042 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:15.052 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:24:15.088 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:15.087+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:15.166 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:15.166 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:15.176 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:24:15.219 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:15.218+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:15.289 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:15.290 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:15.290 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:15.288+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:15.299 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:24:15.392 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:15.391+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:15.411 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:15.411 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:15.419 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:24:15.493 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:15.492+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:15.530 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:15.530 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:15.540 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:24:15.624 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:15.623+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:15.696 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:15.696 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:15.696 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:15.695+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:15.706 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:24:15.798 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:15.796+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:15.818 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:15.818 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:15.827 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:24:15.898 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:15.897+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:15.938 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:15.939 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:15.948 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:24:15.949 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-11-25T04:24:16.032 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:16.031+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:16.072 INFO:teuthology.orchestra.run.smithi067.stderr:Error ENXIO: problem getting command descriptions from osd.0 2023-11-25T04:24:16.075 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2023-11-25T04:24:16.095 INFO:teuthology.orchestra.run.smithi067.stdout:osd.1: {} 2023-11-25T04:24:16.095 INFO:teuthology.orchestra.run.smithi067.stderr:osd.1: osd_enable_op_tracker = '' 2023-11-25T04:24:16.099 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:16.099 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:16.101 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:16.099+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:16.108 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:24:16.119 INFO:teuthology.orchestra.run.smithi067.stdout:osd.2: {} 2023-11-25T04:24:16.119 INFO:teuthology.orchestra.run.smithi067.stderr:osd.2: osd_enable_op_tracker = '' 2023-11-25T04:24:16.122 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-11-25T04:24:16.123 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-11-25T04:24:16.203 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:16.201+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:16.224 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:16.225 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2023-11-25T04:24:16.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.225 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(client.4561.0:1485 4.6 4:60f44e0c:::100000007ec.00000000:head [write 0~88 in=88b] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2023-11-25T04:24:16.225 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:16.223513+0000", 2023-11-25T04:24:16.225 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.00054406699999999997, 2023-11-25T04:24:16.225 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00055955600000000001, 2023-11-25T04:24:16.225 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:16.225 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "waiting for sub ops", 2023-11-25T04:24:16.225 INFO:teuthology.orchestra.run.smithi092.stdout: "client_info": { 2023-11-25T04:24:16.225 INFO:teuthology.orchestra.run.smithi092.stdout: "client": "client.4561", 2023-11-25T04:24:16.226 INFO:teuthology.orchestra.run.smithi092.stdout: "client_addr": "192.168.0.1:0/1268434739", 2023-11-25T04:24:16.226 INFO:teuthology.orchestra.run.smithi092.stdout: "tid": 1485 2023-11-25T04:24:16.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.226 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:16.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.226 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:16.226 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:16.223513+0000", 2023-11-25T04:24:16.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:16.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.226 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:16.226 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:16.223513+0000", 2023-11-25T04:24:16.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:16.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:16.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:16.223514+0000", 2023-11-25T04:24:16.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1699999999999998e-07 2023-11-25T04:24:16.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.227 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:16.227 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:16.223517+0000", 2023-11-25T04:24:16.227 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8899999999999999e-06 2023-11-25T04:24:16.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.228 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:16.228 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:16.223524+0000", 2023-11-25T04:24:16.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8420000000000005e-06 2023-11-25T04:24:16.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.228 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:16.228 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:16.223531+0000", 2023-11-25T04:24:16.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6290000000000004e-06 2023-11-25T04:24:16.228 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.228 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:16.228 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:16.223585+0000", 2023-11-25T04:24:16.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4342000000000002e-05 2023-11-25T04:24:16.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.229 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:16.229 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:16.223708+0000", 2023-11-25T04:24:16.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012228299999999999 2023-11-25T04:24:16.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.230 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:16.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:16.223847+0000", 2023-11-25T04:24:16.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013923399999999999 2023-11-25T04:24:16.230 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:16.230 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:16.230 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:16.230 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:16.230 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2023-11-25T04:24:16.230 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 1 2023-11-25T04:24:16.230 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:16.235 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:24:16.352 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:16.350+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:16.452 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:16.451+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:16.461 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:16.462 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:24:16.462 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2023-11-25T04:24:16.462 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2023-11-25T04:24:16.462 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:16.471 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:24:16.555 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:16.554+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:16.627 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:16.627 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2023-11-25T04:24:16.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2023-11-25T04:24:16.627 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2023-11-25T04:24:16.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.627 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:155 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'109, mlcod=11'105)", 2023-11-25T04:24:16.628 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:24:16.628 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 98.839967908000006, 2023-11-25T04:24:16.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061719019999999999, 2023-11-25T04:24:16.628 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:16.628 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:16.628 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:16.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:16.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:24:16.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:16.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:16.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:24:16.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:16.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:16.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784884+0000", 2023-11-25T04:24:16.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3099999999999999e-07 2023-11-25T04:24:16.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:16.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784942+0000", 2023-11-25T04:24:16.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8570000000000003e-05 2023-11-25T04:24:16.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:16.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784949+0000", 2023-11-25T04:24:16.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0310000000000001e-06 2023-11-25T04:24:16.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:16.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784953+0000", 2023-11-25T04:24:16.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6320000000000001e-06 2023-11-25T04:24:16.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.631 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:16.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784993+0000", 2023-11-25T04:24:16.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9737000000000003e-05 2023-11-25T04:24:16.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.631 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:16.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785011+0000", 2023-11-25T04:24:16.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8349999999999999e-05 2023-11-25T04:24:16.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:16.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791024+0000", 2023-11-25T04:24:16.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006012997 2023-11-25T04:24:16.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:16.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791055+0000", 2023-11-25T04:24:16.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1254000000000003e-05 2023-11-25T04:24:16.632 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:16.632 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:16.633 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:16.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.633 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:156 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'110, mlcod=11'105)", 2023-11-25T04:24:16.633 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:24:16.633 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 98.839381859, 2023-11-25T04:24:16.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005616116, 2023-11-25T04:24:16.633 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:16.633 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:16.633 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:16.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:16.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:24:16.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:16.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:16.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:24:16.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:16.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:16.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785470+0000", 2023-11-25T04:24:16.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0699999999999998e-07 2023-11-25T04:24:16.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:16.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785474+0000", 2023-11-25T04:24:16.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.048e-06 2023-11-25T04:24:16.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:16.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785480+0000", 2023-11-25T04:24:16.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7729999999999996e-06 2023-11-25T04:24:16.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:16.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785483+0000", 2023-11-25T04:24:16.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1970000000000001e-06 2023-11-25T04:24:16.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:16.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787936+0000", 2023-11-25T04:24:16.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024529040000000001 2023-11-25T04:24:16.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:16.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787947+0000", 2023-11-25T04:24:16.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1669e-05 2023-11-25T04:24:16.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:16.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791057+0000", 2023-11-25T04:24:16.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031095200000000002 2023-11-25T04:24:16.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:16.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791086+0000", 2023-11-25T04:24:16.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8697999999999999e-05 2023-11-25T04:24:16.637 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:16.637 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:16.637 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:16.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.638 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:157 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'111, mlcod=11'106)", 2023-11-25T04:24:16.638 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:24:16.638 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 98.838789301000006, 2023-11-25T04:24:16.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050590280000000001, 2023-11-25T04:24:16.638 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:16.638 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:16.639 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:16.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:16.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:24:16.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:16.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:16.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:24:16.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:16.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.640 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:16.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786063+0000", 2023-11-25T04:24:16.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2600000000000002e-07 2023-11-25T04:24:16.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:16.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786069+0000", 2023-11-25T04:24:16.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8839999999999999e-06 2023-11-25T04:24:16.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:16.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786079+0000", 2023-11-25T04:24:16.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0124e-05 2023-11-25T04:24:16.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:16.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786085+0000", 2023-11-25T04:24:16.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6660000000000002e-06 2023-11-25T04:24:16.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:16.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788160+0000", 2023-11-25T04:24:16.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020747669999999999 2023-11-25T04:24:16.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:16.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788180+0000", 2023-11-25T04:24:16.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0208999999999999e-05 2023-11-25T04:24:16.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:16.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791087+0000", 2023-11-25T04:24:16.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029064630000000002 2023-11-25T04:24:16.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:16.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791121+0000", 2023-11-25T04:24:16.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4289000000000003e-05 2023-11-25T04:24:16.643 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:16.643 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:16.643 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:16.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.644 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:158 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'112, mlcod=11'106)", 2023-11-25T04:24:16.644 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:24:16.644 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 98.838372907999997, 2023-11-25T04:24:16.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046663030000000001, 2023-11-25T04:24:16.644 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:16.644 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:16.644 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:16.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.644 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:16.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:24:16.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:16.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:16.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:24:16.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:16.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:16.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786479+0000", 2023-11-25T04:24:16.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2799999999999996e-07 2023-11-25T04:24:16.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:16.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786498+0000", 2023-11-25T04:24:16.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9069e-05 2023-11-25T04:24:16.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:16.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786508+0000", 2023-11-25T04:24:16.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5810000000000003e-06 2023-11-25T04:24:16.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.647 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:16.647 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786512+0000", 2023-11-25T04:24:16.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4889999999999997e-06 2023-11-25T04:24:16.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.647 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:16.647 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788438+0000", 2023-11-25T04:24:16.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019257059999999999 2023-11-25T04:24:16.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.647 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:16.647 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788449+0000", 2023-11-25T04:24:16.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1419e-05 2023-11-25T04:24:16.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.648 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:16.648 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791122+0000", 2023-11-25T04:24:16.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002672822 2023-11-25T04:24:16.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.648 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:16.648 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791145+0000", 2023-11-25T04:24:16.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2589e-05 2023-11-25T04:24:16.648 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:16.648 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:16.648 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:16.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.649 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:206 1.0 e12/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 12'152, mlcod=12'148)", 2023-11-25T04:24:16.649 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:24:16.649 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 98.326604438000004, 2023-11-25T04:24:16.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046728189999999999, 2023-11-25T04:24:16.649 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:16.649 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:16.649 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:16.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.649 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:16.649 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:24:16.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:16.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:16.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:24:16.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:16.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:16.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298248+0000", 2023-11-25T04:24:16.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3600000000000003e-07 2023-11-25T04:24:16.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:16.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298278+0000", 2023-11-25T04:24:16.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0474e-05 2023-11-25T04:24:16.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:16.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298288+0000", 2023-11-25T04:24:16.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0156000000000001e-05 2023-11-25T04:24:16.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.652 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.652 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:16.652 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298293+0000", 2023-11-25T04:24:16.652 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7400000000000004e-06 2023-11-25T04:24:16.652 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.652 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.652 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:16.652 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298333+0000", 2023-11-25T04:24:16.652 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9666000000000003e-05 2023-11-25T04:24:16.652 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.652 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.652 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:16.653 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298351+0000", 2023-11-25T04:24:16.653 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8544999999999999e-05 2023-11-25T04:24:16.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.653 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:16.653 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302891+0000", 2023-11-25T04:24:16.653 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045398519999999996 2023-11-25T04:24:16.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:16.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302920+0000", 2023-11-25T04:24:16.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8750000000000001e-05 2023-11-25T04:24:16.654 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:16.654 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:16.654 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:16.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.654 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:14 3.15 3:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:16.655 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:24:16.655 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 89.591470080999997, 2023-11-25T04:24:16.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010990437, 2023-11-25T04:24:16.655 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:16.655 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:16.655 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:16.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:16.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:24:16.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:16.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.656 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:16.656 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:24:16.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:16.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.656 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:16.656 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033383+0000", 2023-11-25T04:24:16.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9870000000000002e-06 2023-11-25T04:24:16.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.657 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.657 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:16.657 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033398+0000", 2023-11-25T04:24:16.657 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4569999999999999e-05 2023-11-25T04:24:16.657 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.657 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.657 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:16.657 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033411+0000", 2023-11-25T04:24:16.657 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3543999999999999e-05 2023-11-25T04:24:16.657 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.657 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.657 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:16.658 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033423+0000", 2023-11-25T04:24:16.658 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1905999999999999e-05 2023-11-25T04:24:16.658 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.658 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.658 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:16.658 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034433+0000", 2023-11-25T04:24:16.658 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00101022 2023-11-25T04:24:16.658 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.658 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.658 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:16.658 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034539+0000", 2023-11-25T04:24:16.658 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010597199999999999 2023-11-25T04:24:16.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.659 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.659 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:16.659 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034667+0000", 2023-11-25T04:24:16.659 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012765500000000001 2023-11-25T04:24:16.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.659 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.659 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:16.659 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037803+0000", 2023-11-25T04:24:16.659 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031362220000000001 2023-11-25T04:24:16.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.659 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.660 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:16.660 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:24:16.660 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0064964669999999997 2023-11-25T04:24:16.660 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.660 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.660 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:16.660 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044331+0000", 2023-11-25T04:24:16.660 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1544999999999997e-05 2023-11-25T04:24:16.660 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.660 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.661 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:16.661 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044372+0000", 2023-11-25T04:24:16.661 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0349000000000003e-05 2023-11-25T04:24:16.661 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:16.661 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:16.661 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:16.661 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.661 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.661 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:15 3.15 3:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:16.661 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:24:16.661 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 89.590810430999994, 2023-11-25T04:24:16.662 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010464362, 2023-11-25T04:24:16.662 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:16.662 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:16.662 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:16.662 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.662 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:16.662 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:24:16.662 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:16.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:16.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:24:16.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:16.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:16.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034042+0000", 2023-11-25T04:24:16.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0999999999999997e-07 2023-11-25T04:24:16.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:16.664 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034048+0000", 2023-11-25T04:24:16.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0390000000000001e-06 2023-11-25T04:24:16.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:16.664 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034059+0000", 2023-11-25T04:24:16.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1182999999999999e-05 2023-11-25T04:24:16.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:16.665 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034067+0000", 2023-11-25T04:24:16.665 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1559999999999995e-06 2023-11-25T04:24:16.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.665 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.665 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:16.665 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035005+0000", 2023-11-25T04:24:16.665 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00093747399999999999 2023-11-25T04:24:16.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.665 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.665 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:16.665 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035077+0000", 2023-11-25T04:24:16.665 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2747e-05 2023-11-25T04:24:16.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.666 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.666 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:16.666 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035219+0000", 2023-11-25T04:24:16.666 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000141877 2023-11-25T04:24:16.666 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.666 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.666 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:16.666 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037810+0000", 2023-11-25T04:24:16.666 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002590752 2023-11-25T04:24:16.666 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.666 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.666 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:16.667 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044437+0000", 2023-11-25T04:24:16.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066270110000000004 2023-11-25T04:24:16.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.669 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:16.669 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044481+0000", 2023-11-25T04:24:16.669 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4106000000000001e-05 2023-11-25T04:24:16.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.669 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:16.669 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044505+0000", 2023-11-25T04:24:16.669 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4207e-05 2023-11-25T04:24:16.669 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:16.670 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:16.670 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:16.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.670 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:16 3.1d 3:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:16.670 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:24:16.670 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 89.590768582999999, 2023-11-25T04:24:16.670 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010413923, 2023-11-25T04:24:16.670 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:16.670 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:16.670 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:16.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:16.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:24:16.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:16.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:16.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:24:16.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:16.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:16.672 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:24:16.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4499999999999997e-07 2023-11-25T04:24:16.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.672 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:16.672 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034087+0000", 2023-11-25T04:24:16.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9770000000000002e-06 2023-11-25T04:24:16.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.673 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:16.673 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034094+0000", 2023-11-25T04:24:16.673 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8809999999999998e-06 2023-11-25T04:24:16.673 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.673 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.673 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:16.673 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034100+0000", 2023-11-25T04:24:16.673 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8440000000000003e-06 2023-11-25T04:24:16.673 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.673 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.673 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:16.673 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034993+0000", 2023-11-25T04:24:16.674 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00089336799999999996 2023-11-25T04:24:16.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.674 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.674 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:16.674 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035099+0000", 2023-11-25T04:24:16.674 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106037 2023-11-25T04:24:16.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.674 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.674 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:16.674 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035276+0000", 2023-11-25T04:24:16.674 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017631799999999999 2023-11-25T04:24:16.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.675 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.675 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:16.675 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037815+0000", 2023-11-25T04:24:16.675 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025395959999999999 2023-11-25T04:24:16.675 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.675 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.675 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:16.675 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044420+0000", 2023-11-25T04:24:16.675 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066045749999999997 2023-11-25T04:24:16.675 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.675 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.676 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:16.676 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044463+0000", 2023-11-25T04:24:16.676 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3205999999999999e-05 2023-11-25T04:24:16.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.676 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:16.676 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044497+0000", 2023-11-25T04:24:16.676 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3476000000000003e-05 2023-11-25T04:24:16.676 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:16.676 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:16.676 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:16.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.677 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:17 3.1d 3:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:16.677 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:24:16.677 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 89.590743373999999, 2023-11-25T04:24:16.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01127356, 2023-11-25T04:24:16.677 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:16.677 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:16.677 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:16.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.677 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:16.678 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:24:16.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:16.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.678 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.678 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:16.678 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:24:16.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:16.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.678 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.678 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:16.678 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:24:16.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1100000000000001e-07 2023-11-25T04:24:16.679 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.679 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.679 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:16.679 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034112+0000", 2023-11-25T04:24:16.679 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8600000000000003e-06 2023-11-25T04:24:16.679 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.679 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.679 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:16.679 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034120+0000", 2023-11-25T04:24:16.679 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7700000000000001e-06 2023-11-25T04:24:16.679 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.679 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.679 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:16.680 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034127+0000", 2023-11-25T04:24:16.680 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0999999999999998e-06 2023-11-25T04:24:16.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.680 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.680 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:16.680 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035601+0000", 2023-11-25T04:24:16.680 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014739410000000001 2023-11-25T04:24:16.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.680 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.680 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:16.680 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035685+0000", 2023-11-25T04:24:16.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3677999999999994e-05 2023-11-25T04:24:16.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.681 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:16.681 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035827+0000", 2023-11-25T04:24:16.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014232599999999999 2023-11-25T04:24:16.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.681 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:16.681 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038725+0000", 2023-11-25T04:24:16.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028982779999999998 2023-11-25T04:24:16.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:16.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045323+0000", 2023-11-25T04:24:16.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0065980780000000003 2023-11-25T04:24:16.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:16.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045354+0000", 2023-11-25T04:24:16.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0463e-05 2023-11-25T04:24:16.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:16.683 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045381+0000", 2023-11-25T04:24:16.683 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7654999999999999e-05 2023-11-25T04:24:16.683 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:16.683 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:16.683 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:16.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.683 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:22 3.e 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:16.683 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:24:16.683 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 89.590716960999998, 2023-11-25T04:24:16.684 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010133422, 2023-11-25T04:24:16.684 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:16.684 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:16.684 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:16.684 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.684 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:16.684 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:24:16.684 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:16.684 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.684 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.684 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:16.684 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:24:16.685 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:16.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.685 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.685 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:16.685 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034135+0000", 2023-11-25T04:24:16.685 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-07 2023-11-25T04:24:16.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.685 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.685 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:16.685 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034138+0000", 2023-11-25T04:24:16.685 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6009999999999999e-06 2023-11-25T04:24:16.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.685 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:16.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034145+0000", 2023-11-25T04:24:16.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1169999999999998e-06 2023-11-25T04:24:16.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:16.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034150+0000", 2023-11-25T04:24:16.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9080000000000003e-06 2023-11-25T04:24:16.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:16.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034209+0000", 2023-11-25T04:24:16.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8149000000000002e-05 2023-11-25T04:24:16.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.687 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:16.687 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034415+0000", 2023-11-25T04:24:16.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020637699999999999 2023-11-25T04:24:16.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.687 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:16.687 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034584+0000", 2023-11-25T04:24:16.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016870399999999999 2023-11-25T04:24:16.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.688 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:16.688 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.036979+0000", 2023-11-25T04:24:16.688 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023957509999999998 2023-11-25T04:24:16.688 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.688 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.688 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:16.688 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044161+0000", 2023-11-25T04:24:16.688 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0071811590000000003 2023-11-25T04:24:16.688 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.688 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.688 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:16.688 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044224+0000", 2023-11-25T04:24:16.688 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3454000000000006e-05 2023-11-25T04:24:16.689 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.689 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:16.689 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044268+0000", 2023-11-25T04:24:16.689 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3761999999999999e-05 2023-11-25T04:24:16.689 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:16.689 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:16.689 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:16.689 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.689 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:23 3.e 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:16.689 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:24:16.689 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 89.590685551999997, 2023-11-25T04:24:16.690 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010281676, 2023-11-25T04:24:16.690 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:16.690 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:16.690 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:16.690 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.690 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:16.690 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:24:16.690 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:16.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.690 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.690 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:16.690 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:24:16.690 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:16.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.691 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:16.691 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:24:16.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5799999999999999e-07 2023-11-25T04:24:16.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.691 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:16.691 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034169+0000", 2023-11-25T04:24:16.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1879999999999998e-06 2023-11-25T04:24:16.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.692 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:16.692 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034177+0000", 2023-11-25T04:24:16.692 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7020000000000001e-06 2023-11-25T04:24:16.692 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.692 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.692 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:16.692 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034184+0000", 2023-11-25T04:24:16.692 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3910000000000003e-06 2023-11-25T04:24:16.692 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.692 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.692 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:16.692 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034924+0000", 2023-11-25T04:24:16.692 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00074032999999999998 2023-11-25T04:24:16.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.693 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:16.693 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034991+0000", 2023-11-25T04:24:16.693 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7347000000000004e-05 2023-11-25T04:24:16.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.693 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:16.693 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035096+0000", 2023-11-25T04:24:16.693 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010488 2023-11-25T04:24:16.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.694 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:16.694 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037588+0000", 2023-11-25T04:24:16.694 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024918309999999999 2023-11-25T04:24:16.694 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.694 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.694 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:16.694 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044367+0000", 2023-11-25T04:24:16.694 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067787589999999997 2023-11-25T04:24:16.694 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.694 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.694 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:16.694 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044415+0000", 2023-11-25T04:24:16.695 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7803000000000003e-05 2023-11-25T04:24:16.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.695 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.695 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:16.695 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044447+0000", 2023-11-25T04:24:16.695 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2886999999999999e-05 2023-11-25T04:24:16.695 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:16.695 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:16.695 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:16.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.695 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.695 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:30 3.15 3:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:16.695 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:24:16.696 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 89.590627111999993, 2023-11-25T04:24:16.696 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010365070000000001, 2023-11-25T04:24:16.696 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:16.696 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:16.696 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:16.696 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.696 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:16.696 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:24:16.696 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:16.696 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.696 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.696 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:16.697 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:24:16.697 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:16.697 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.697 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:16.697 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034225+0000", 2023-11-25T04:24:16.697 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7400000000000002e-07 2023-11-25T04:24:16.697 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.697 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.697 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:16.697 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034229+0000", 2023-11-25T04:24:16.697 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3100000000000002e-06 2023-11-25T04:24:16.698 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.698 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.698 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:16.698 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034239+0000", 2023-11-25T04:24:16.698 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9140000000000003e-06 2023-11-25T04:24:16.698 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.698 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.698 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:16.698 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034246+0000", 2023-11-25T04:24:16.698 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0149999999999998e-06 2023-11-25T04:24:16.698 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.699 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.699 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:16.699 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035572+0000", 2023-11-25T04:24:16.699 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001325655 2023-11-25T04:24:16.699 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.699 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.699 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:16.699 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035730+0000", 2023-11-25T04:24:16.699 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015827899999999999 2023-11-25T04:24:16.699 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.699 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.699 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:16.700 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035857+0000", 2023-11-25T04:24:16.700 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012644699999999999 2023-11-25T04:24:16.700 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.700 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.700 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:16.700 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038737+0000", 2023-11-25T04:24:16.700 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002880077 2023-11-25T04:24:16.700 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.700 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.700 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:16.700 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044540+0000", 2023-11-25T04:24:16.700 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058034239999999997 2023-11-25T04:24:16.701 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.701 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.701 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:16.701 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044569+0000", 2023-11-25T04:24:16.701 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8388999999999999e-05 2023-11-25T04:24:16.701 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.701 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.701 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:16.701 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044589+0000", 2023-11-25T04:24:16.701 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0686000000000002e-05 2023-11-25T04:24:16.701 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:16.701 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:16.701 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:16.702 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.702 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:33 3.1b 3:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:16.702 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:24:16.702 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 89.590581614000001, 2023-11-25T04:24:16.702 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099754049999999997, 2023-11-25T04:24:16.702 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:16.702 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:16.702 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:16.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.702 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:16.702 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:24:16.702 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:16.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.703 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:16.703 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:24:16.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:16.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.703 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:16.703 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:24:16.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4799999999999998e-07 2023-11-25T04:24:16.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.704 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:16.704 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034273+0000", 2023-11-25T04:24:16.704 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7819999999999998e-06 2023-11-25T04:24:16.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.704 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.704 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:16.704 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034285+0000", 2023-11-25T04:24:16.704 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2058e-05 2023-11-25T04:24:16.704 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.704 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.704 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:16.704 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034291+0000", 2023-11-25T04:24:16.705 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0680000000000002e-06 2023-11-25T04:24:16.705 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.705 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.705 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:16.705 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034370+0000", 2023-11-25T04:24:16.705 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8452e-05 2023-11-25T04:24:16.705 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.705 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.705 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:16.705 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034477+0000", 2023-11-25T04:24:16.705 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107471 2023-11-25T04:24:16.705 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.706 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.706 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:16.706 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044245+0000", 2023-11-25T04:24:16.706 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0097679259999999997 2023-11-25T04:24:16.706 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:16.706 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:16.706 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:16.706 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.706 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.706 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:36 3.1a 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:16.706 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:24:16.706 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 89.590534941000001, 2023-11-25T04:24:16.707 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058179099999999999, 2023-11-25T04:24:16.707 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:16.707 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:16.707 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:16.707 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.707 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:16.707 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:24:16.707 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:16.707 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.707 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.708 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:16.708 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:24:16.708 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:16.708 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.708 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.708 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:16.708 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034317+0000", 2023-11-25T04:24:16.708 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1999999999999999e-07 2023-11-25T04:24:16.708 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.708 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.708 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:16.708 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034321+0000", 2023-11-25T04:24:16.708 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.233e-06 2023-11-25T04:24:16.709 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.709 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.709 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:16.709 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034337+0000", 2023-11-25T04:24:16.709 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5308999999999999e-05 2023-11-25T04:24:16.709 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.709 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.709 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:16.709 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034354+0000", 2023-11-25T04:24:16.709 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7411999999999999e-05 2023-11-25T04:24:16.709 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.709 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.710 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:16.710 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034465+0000", 2023-11-25T04:24:16.710 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011154699999999999 2023-11-25T04:24:16.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.710 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.710 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:16.710 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034560+0000", 2023-11-25T04:24:16.710 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4186999999999998e-05 2023-11-25T04:24:16.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.710 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.710 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:16.710 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.040134+0000", 2023-11-25T04:24:16.710 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055746019999999997 2023-11-25T04:24:16.711 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:16.711 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:16.711 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:16.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.711 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:9 3.10 e18/16 3:09401a81:::601.00000000:head v 18'4, mlcod=0'0)", 2023-11-25T04:24:16.711 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:24:16.711 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 89.581640633000006, 2023-11-25T04:24:16.711 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061641020000000003, 2023-11-25T04:24:16.711 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:16.711 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:16.712 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:16.712 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.712 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:16.712 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:24:16.712 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:16.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.712 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.712 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:16.712 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:24:16.712 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:16.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.712 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.712 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:16.713 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:24:16.713 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5300000000000001e-07 2023-11-25T04:24:16.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.713 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:16.713 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043222+0000", 2023-11-25T04:24:16.713 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.082e-05 2023-11-25T04:24:16.713 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.713 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:16.713 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043228+0000", 2023-11-25T04:24:16.713 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6699999999999997e-06 2023-11-25T04:24:16.714 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.714 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.714 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:16.714 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043236+0000", 2023-11-25T04:24:16.714 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8189999999999992e-06 2023-11-25T04:24:16.714 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.714 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.714 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:16.714 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043507+0000", 2023-11-25T04:24:16.714 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000271012 2023-11-25T04:24:16.714 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.714 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.715 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:16.715 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043530+0000", 2023-11-25T04:24:16.715 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2659000000000001e-05 2023-11-25T04:24:16.715 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.715 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.715 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:16.715 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049321+0000", 2023-11-25T04:24:16.715 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057915129999999999 2023-11-25T04:24:16.715 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.715 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.715 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:16.715 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049375+0000", 2023-11-25T04:24:16.716 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3455999999999998e-05 2023-11-25T04:24:16.716 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:16.716 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:16.716 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:16.716 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.716 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.716 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:1 3.14 e18/16 3:292cf221:::200.00000000:head v 18'3, mlcod=0'0)", 2023-11-25T04:24:16.716 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:24:16.716 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 89.581603920000006, 2023-11-25T04:24:16.716 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0060983599999999997, 2023-11-25T04:24:16.716 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:16.716 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:16.716 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:16.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.717 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:16.717 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:24:16.717 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:16.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.717 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:16.717 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:24:16.717 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:16.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.717 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:16.718 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043248+0000", 2023-11-25T04:24:16.718 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9399999999999995e-07 2023-11-25T04:24:16.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.718 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:16.718 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043252+0000", 2023-11-25T04:24:16.718 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1799999999999998e-06 2023-11-25T04:24:16.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.718 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:16.718 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043259+0000", 2023-11-25T04:24:16.718 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0049999999999998e-06 2023-11-25T04:24:16.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.719 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.719 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:16.719 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043265+0000", 2023-11-25T04:24:16.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0859999999999998e-06 2023-11-25T04:24:16.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.719 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.719 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:16.719 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043311+0000", 2023-11-25T04:24:16.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6082999999999999e-05 2023-11-25T04:24:16.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.719 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.719 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:16.720 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043330+0000", 2023-11-25T04:24:16.720 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8556e-05 2023-11-25T04:24:16.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.720 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.720 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:16.720 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049303+0000", 2023-11-25T04:24:16.720 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059733470000000004 2023-11-25T04:24:16.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.720 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.720 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:16.720 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049346+0000", 2023-11-25T04:24:16.720 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2608999999999998e-05 2023-11-25T04:24:16.721 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:16.721 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:16.721 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:16.721 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.721 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.721 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:3 3.14 e18/16 3:287034c4:::1.00000000.inode:head v 18'4, mlcod=0'0)", 2023-11-25T04:24:16.721 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:24:16.721 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 89.581087413999995, 2023-11-25T04:24:16.721 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057040090000000003, 2023-11-25T04:24:16.721 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:16.721 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:16.721 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:16.721 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.722 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:16.722 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:24:16.722 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:16.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.722 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.722 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:16.722 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:24:16.722 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:16.722 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.722 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.722 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:16.722 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:24:16.722 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9799999999999999e-07 2023-11-25T04:24:16.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.723 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:16.723 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043785+0000", 2023-11-25T04:24:16.723 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0721999999999999e-05 2023-11-25T04:24:16.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.723 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:16.723 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043793+0000", 2023-11-25T04:24:16.723 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5839999999999995e-06 2023-11-25T04:24:16.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.723 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:16.724 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043803+0000", 2023-11-25T04:24:16.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0729999999999994e-06 2023-11-25T04:24:16.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.724 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.724 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:16.724 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043843+0000", 2023-11-25T04:24:16.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0747999999999997e-05 2023-11-25T04:24:16.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.724 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.724 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:16.724 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043856+0000", 2023-11-25T04:24:16.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2697000000000001e-05 2023-11-25T04:24:16.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:16.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049426+0000", 2023-11-25T04:24:16.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055704409999999998 2023-11-25T04:24:16.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:16.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049468+0000", 2023-11-25T04:24:16.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1446000000000003e-05 2023-11-25T04:24:16.725 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:16.725 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:16.725 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:16.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.726 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "pg_update_log_missing(3.14 epoch 18/16 rep_tid 171 entries 18'5 (0'0) error 3:2e2fa760:::200.00000001:head by mds.0.3:32 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:24:16.726 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:24:16.726 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 89.580926450999996, 2023-11-25T04:24:16.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005556938, 2023-11-25T04:24:16.726 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:16.726 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2023-11-25T04:24:16.726 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:16.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.726 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:16.726 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:24:16.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:16.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.727 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.727 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:16.727 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:24:16.727 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:16.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.727 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.727 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:16.727 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:24:16.727 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.03e-07 2023-11-25T04:24:16.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.727 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.727 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:16.728 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043931+0000", 2023-11-25T04:24:16.728 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1079999999999998e-06 2023-11-25T04:24:16.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.728 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:16.728 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043944+0000", 2023-11-25T04:24:16.728 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2852e-05 2023-11-25T04:24:16.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.728 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:16.728 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043955+0000", 2023-11-25T04:24:16.728 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1037e-05 2023-11-25T04:24:16.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.729 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:16.729 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044088+0000", 2023-11-25T04:24:16.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013269599999999999 2023-11-25T04:24:16.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.729 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:16.729 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049482+0000", 2023-11-25T04:24:16.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053940419999999999 2023-11-25T04:24:16.729 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:16.729 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:16.729 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:16.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.730 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:10 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'5, mlcod=0'0)", 2023-11-25T04:24:16.730 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:24:16.730 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 89.580836422999994, 2023-11-25T04:24:16.730 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054493099999999997, 2023-11-25T04:24:16.730 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:16.730 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:16.730 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:16.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.730 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:16.730 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:24:16.731 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:16.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.731 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:16.731 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:24:16.731 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:16.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.731 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:16.731 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:24:16.731 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6100000000000002e-07 2023-11-25T04:24:16.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:16.732 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044019+0000", 2023-11-25T04:24:16.732 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.839e-06 2023-11-25T04:24:16.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:16.732 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044026+0000", 2023-11-25T04:24:16.732 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5869999999999998e-06 2023-11-25T04:24:16.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:16.732 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044031+0000", 2023-11-25T04:24:16.732 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4909999999999996e-06 2023-11-25T04:24:16.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.733 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:16.733 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044070+0000", 2023-11-25T04:24:16.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8829999999999999e-05 2023-11-25T04:24:16.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.733 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:16.733 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044081+0000", 2023-11-25T04:24:16.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1447999999999999e-05 2023-11-25T04:24:16.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.734 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:16.734 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049406+0000", 2023-11-25T04:24:16.734 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053246480000000004 2023-11-25T04:24:16.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.734 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:16.734 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049464+0000", 2023-11-25T04:24:16.734 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8205999999999998e-05 2023-11-25T04:24:16.734 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:16.734 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:16.734 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:16.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.735 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:11 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'6, mlcod=0'0)", 2023-11-25T04:24:16.735 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:24:16.735 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 89.580635158999996, 2023-11-25T04:24:16.735 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052673499999999996, 2023-11-25T04:24:16.735 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:16.735 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:16.735 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:16.735 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.735 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:16.735 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:24:16.735 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:16.735 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.735 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.736 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:16.736 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:24:16.736 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:16.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.736 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.736 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:16.736 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:24:16.736 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0199999999999998e-07 2023-11-25T04:24:16.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.736 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.736 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:16.736 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044220+0000", 2023-11-25T04:24:16.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7720000000000001e-06 2023-11-25T04:24:16.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:16.737 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044226+0000", 2023-11-25T04:24:16.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8579999999999996e-06 2023-11-25T04:24:16.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.737 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:16.737 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044232+0000", 2023-11-25T04:24:16.737 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1700000000000002e-06 2023-11-25T04:24:16.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:16.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044274+0000", 2023-11-25T04:24:16.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1254999999999997e-05 2023-11-25T04:24:16.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:16.738 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:24:16.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6880999999999999e-05 2023-11-25T04:24:16.738 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.738 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.738 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:16.739 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049466+0000", 2023-11-25T04:24:16.739 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051659219999999999 2023-11-25T04:24:16.739 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:16.739 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:16.739 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:16.739 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049483+0000", 2023-11-25T04:24:16.739 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.719e-05 2023-11-25T04:24:16.739 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:16.739 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:16.739 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:16.739 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:16.739 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:16.739 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:16.741 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:16.739+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:16.742 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:24:16.842 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:16.840+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:16.854 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:16.854 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2023-11-25T04:24:16.855 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:16.855 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4561.0:2191 4.2f 4:f64fcecc:::10000000aae.00000000:head [write 0~43 in=43b] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2023-11-25T04:24:16.855 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:16.853387+0000", 2023-11-25T04:24:16.855 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 0.000559039, 2023-11-25T04:24:16.855 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00057503299999999997, 2023-11-25T04:24:16.855 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:16.855 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "waiting for sub ops", 2023-11-25T04:24:16.855 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2023-11-25T04:24:16.855 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4561", 2023-11-25T04:24:16.855 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "192.168.0.1:0/1268434739", 2023-11-25T04:24:16.855 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 2191 2023-11-25T04:24:16.856 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:16.856 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:16.856 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:16.856 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:16.856 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.853387+0000", 2023-11-25T04:24:16.856 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:16.856 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:16.856 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:16.856 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:16.856 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.853387+0000", 2023-11-25T04:24:16.856 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:16.857 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:16.857 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:16.857 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:16.857 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.853387+0000", 2023-11-25T04:24:16.857 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7100000000000001e-07 2023-11-25T04:24:16.857 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:16.857 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:16.857 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:16.857 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.853390+0000", 2023-11-25T04:24:16.857 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9919999999999999e-06 2023-11-25T04:24:16.857 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:16.857 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:16.857 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:16.858 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.853399+0000", 2023-11-25T04:24:16.858 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2619999999999998e-06 2023-11-25T04:24:16.858 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:16.858 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:16.858 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:16.858 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.853403+0000", 2023-11-25T04:24:16.858 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1989999999999999e-06 2023-11-25T04:24:16.858 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:16.858 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:16.858 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:16.858 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.853443+0000", 2023-11-25T04:24:16.858 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9777000000000002e-05 2023-11-25T04:24:16.859 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:16.859 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:16.859 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:16.859 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.853552+0000", 2023-11-25T04:24:16.859 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010973599999999999 2023-11-25T04:24:16.859 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:16.859 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:16.859 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:16.859 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.853675+0000", 2023-11-25T04:24:16.859 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012229600000000001 2023-11-25T04:24:16.859 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:16.859 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:16.860 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:16.860 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:16.860 INFO:teuthology.orchestra.run.smithi195.stdout: ], 2023-11-25T04:24:16.860 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 1 2023-11-25T04:24:16.860 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:16.864 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:24:16.947 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:16.946+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:17.015 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:17.015 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:24:17.015 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2023-11-25T04:24:17.015 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2023-11-25T04:24:17.015 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:17.025 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:24:17.044 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:17.042+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:17.115 INFO:teuthology.orchestra.run.smithi067.stderr:noscrub is set 2023-11-25T04:24:17.144 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:17.143+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:17.186 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:17.186 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2023-11-25T04:24:17.186 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2023-11-25T04:24:17.186 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2023-11-25T04:24:17.186 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.186 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:17.186 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:24:17.186 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 90.151598763999999, 2023-11-25T04:24:17.187 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01072943, 2023-11-25T04:24:17.187 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:17.187 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:17.187 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:17.187 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.187 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:17.187 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:24:17.187 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:17.187 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.187 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.187 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:17.187 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:24:17.187 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:17.188 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.188 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.188 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:17.188 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031829+0000", 2023-11-25T04:24:17.188 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.666e-06 2023-11-25T04:24:17.188 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.188 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.188 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:17.188 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031839+0000", 2023-11-25T04:24:17.188 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9490000000000008e-06 2023-11-25T04:24:17.188 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.188 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.189 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:17.189 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031857+0000", 2023-11-25T04:24:17.189 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8093000000000001e-05 2023-11-25T04:24:17.189 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.189 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.189 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:17.189 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031882+0000", 2023-11-25T04:24:17.189 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4294000000000002e-05 2023-11-25T04:24:17.189 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.189 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.189 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:17.189 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033130+0000", 2023-11-25T04:24:17.189 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001248264 2023-11-25T04:24:17.190 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.190 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.190 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:17.190 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033426+0000", 2023-11-25T04:24:17.190 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029620999999999998 2023-11-25T04:24:17.190 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.190 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.190 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:17.190 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033814+0000", 2023-11-25T04:24:17.190 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000387742 2023-11-25T04:24:17.190 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.191 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.191 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:17.191 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036793+0000", 2023-11-25T04:24:17.191 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002979598 2023-11-25T04:24:17.191 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.191 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.191 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:17.191 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042455+0000", 2023-11-25T04:24:17.191 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0056619260000000003 2023-11-25T04:24:17.191 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.191 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.191 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:17.192 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042502+0000", 2023-11-25T04:24:17.192 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7139999999999999e-05 2023-11-25T04:24:17.192 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.192 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.192 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:17.192 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042557+0000", 2023-11-25T04:24:17.192 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4548e-05 2023-11-25T04:24:17.192 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:17.192 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:17.192 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:17.192 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.193 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.193 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:17.193 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:24:17.193 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 90.150400228999999, 2023-11-25T04:24:17.193 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011505202000000001, 2023-11-25T04:24:17.193 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:17.193 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:17.193 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:17.193 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.193 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:17.193 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:24:17.193 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:17.194 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.194 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.194 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:17.194 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:24:17.194 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:17.194 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.194 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.194 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:17.194 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033027+0000", 2023-11-25T04:24:17.194 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0699999999999999e-06 2023-11-25T04:24:17.194 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.195 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.195 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:17.195 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033035+0000", 2023-11-25T04:24:17.195 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0830000000000004e-06 2023-11-25T04:24:17.195 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.195 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.195 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:17.195 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033059+0000", 2023-11-25T04:24:17.196 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3975e-05 2023-11-25T04:24:17.196 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.196 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.196 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:17.196 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033482+0000", 2023-11-25T04:24:17.196 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042253700000000001 2023-11-25T04:24:17.196 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.196 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.196 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:17.196 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034520+0000", 2023-11-25T04:24:17.196 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001037991 2023-11-25T04:24:17.197 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.197 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.197 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:17.197 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034611+0000", 2023-11-25T04:24:17.197 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1113999999999995e-05 2023-11-25T04:24:17.197 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.197 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.197 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:17.197 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034769+0000", 2023-11-25T04:24:17.197 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000157641 2023-11-25T04:24:17.197 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.197 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.197 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:17.198 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038292+0000", 2023-11-25T04:24:17.198 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035231160000000002 2023-11-25T04:24:17.198 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.198 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.198 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:17.198 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044440+0000", 2023-11-25T04:24:17.198 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0061486980000000002 2023-11-25T04:24:17.198 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.198 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.198 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:17.198 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044495+0000", 2023-11-25T04:24:17.199 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4339000000000003e-05 2023-11-25T04:24:17.199 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.199 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.199 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:17.199 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044531+0000", 2023-11-25T04:24:17.199 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6637999999999997e-05 2023-11-25T04:24:17.199 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:17.199 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:17.199 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:17.199 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.199 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.200 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:17.200 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:24:17.200 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 90.149925874999994, 2023-11-25T04:24:17.200 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0057634039999999997, 2023-11-25T04:24:17.200 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:17.200 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:17.200 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:17.200 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.200 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:17.200 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:24:17.200 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:17.200 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.201 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.201 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:17.201 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:24:17.201 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:17.201 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.201 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.201 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:17.201 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:24:17.201 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7900000000000007e-07 2023-11-25T04:24:17.201 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.201 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.202 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:17.202 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033507+0000", 2023-11-25T04:24:17.202 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8900000000000004e-06 2023-11-25T04:24:17.202 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.202 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.202 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:17.202 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033519+0000", 2023-11-25T04:24:17.202 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1756000000000001e-05 2023-11-25T04:24:17.202 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.202 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.202 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:17.202 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033527+0000", 2023-11-25T04:24:17.203 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3610000000000001e-06 2023-11-25T04:24:17.203 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.203 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.203 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:17.203 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034451+0000", 2023-11-25T04:24:17.203 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00092372400000000001 2023-11-25T04:24:17.203 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.203 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.203 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:17.203 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034809+0000", 2023-11-25T04:24:17.203 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00035757699999999998 2023-11-25T04:24:17.204 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.204 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.204 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:17.204 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034965+0000", 2023-11-25T04:24:17.204 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015638699999999999 2023-11-25T04:24:17.204 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.204 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.205 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:17.205 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038299+0000", 2023-11-25T04:24:17.205 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0033336559999999999 2023-11-25T04:24:17.205 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.205 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.205 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:17.206 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039169+0000", 2023-11-25T04:24:17.206 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00087066400000000005 2023-11-25T04:24:17.206 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.206 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.206 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:17.206 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039226+0000", 2023-11-25T04:24:17.206 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6511999999999997e-05 2023-11-25T04:24:17.206 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.206 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.206 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:17.206 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039264+0000", 2023-11-25T04:24:17.206 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7898000000000002e-05 2023-11-25T04:24:17.207 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:17.207 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:17.207 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:17.207 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.207 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.207 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:25 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:17.207 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:24:17.207 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 90.149847291, 2023-11-25T04:24:17.207 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096656169999999996, 2023-11-25T04:24:17.207 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:17.207 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:17.207 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:17.207 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.208 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:17.208 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:24:17.208 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:17.208 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.208 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.208 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:17.208 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:24:17.208 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:17.208 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.208 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.209 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:17.209 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033580+0000", 2023-11-25T04:24:17.209 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8199999999999999e-07 2023-11-25T04:24:17.209 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.209 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.209 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:17.209 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033585+0000", 2023-11-25T04:24:17.209 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3680000000000001e-06 2023-11-25T04:24:17.209 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.209 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.209 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:17.209 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033600+0000", 2023-11-25T04:24:17.210 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4331000000000001e-05 2023-11-25T04:24:17.210 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.210 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.210 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:17.210 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033606+0000", 2023-11-25T04:24:17.210 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5780000000000004e-06 2023-11-25T04:24:17.210 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.210 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.210 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:17.210 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035747+0000", 2023-11-25T04:24:17.210 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0021406530000000002 2023-11-25T04:24:17.210 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.211 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.211 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:17.211 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035901+0000", 2023-11-25T04:24:17.211 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000153931 2023-11-25T04:24:17.211 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.211 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.211 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:17.211 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036193+0000", 2023-11-25T04:24:17.211 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029211299999999999 2023-11-25T04:24:17.211 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.211 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.211 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:17.211 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038655+0000", 2023-11-25T04:24:17.212 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0024621640000000002 2023-11-25T04:24:17.212 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.212 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.212 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:17.212 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043130+0000", 2023-11-25T04:24:17.212 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.004474698 2023-11-25T04:24:17.212 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.212 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.212 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:17.212 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043204+0000", 2023-11-25T04:24:17.212 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4091999999999995e-05 2023-11-25T04:24:17.212 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.212 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.213 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:17.213 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043245+0000", 2023-11-25T04:24:17.213 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0907e-05 2023-11-25T04:24:17.213 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:17.213 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:17.213 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:17.213 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.213 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.213 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:26 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:17.213 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:17.213 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 90.149811467000006, 2023-11-25T04:24:17.213 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011086945000000001, 2023-11-25T04:24:17.214 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:17.214 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:17.214 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:17.214 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.214 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:17.214 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:17.214 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:17.214 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.214 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.214 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:17.214 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:17.214 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:17.215 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.215 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.215 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:17.215 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:17.215 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1699999999999998e-07 2023-11-25T04:24:17.215 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.215 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.215 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:17.215 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033627+0000", 2023-11-25T04:24:17.215 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.163e-05 2023-11-25T04:24:17.215 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.215 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.216 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:17.216 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033636+0000", 2023-11-25T04:24:17.216 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6859999999999995e-06 2023-11-25T04:24:17.216 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.216 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.216 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:17.216 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033642+0000", 2023-11-25T04:24:17.216 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3149999999999997e-06 2023-11-25T04:24:17.216 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.216 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.216 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:17.217 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035414+0000", 2023-11-25T04:24:17.217 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0017716380000000001 2023-11-25T04:24:17.217 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.217 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.217 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:17.217 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035539+0000", 2023-11-25T04:24:17.217 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000125477 2023-11-25T04:24:17.217 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.217 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.217 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:17.217 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035672+0000", 2023-11-25T04:24:17.217 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000133316 2023-11-25T04:24:17.218 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.219 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.219 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:17.219 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038649+0000", 2023-11-25T04:24:17.219 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029764679999999999 2023-11-25T04:24:17.220 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.220 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.220 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:17.220 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044578+0000", 2023-11-25T04:24:17.220 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0059293510000000002 2023-11-25T04:24:17.220 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.220 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.220 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:17.220 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044651+0000", 2023-11-25T04:24:17.220 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2513000000000002e-05 2023-11-25T04:24:17.220 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.220 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.220 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:17.221 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044702+0000", 2023-11-25T04:24:17.221 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1034e-05 2023-11-25T04:24:17.221 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:17.221 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:17.221 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:17.221 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.221 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.221 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:29 3.b 3:d0630e4c:::mds0_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:17.221 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:17.221 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 90.149773546999995, 2023-11-25T04:24:17.222 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010980227, 2023-11-25T04:24:17.222 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:17.222 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:17.222 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:17.222 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.222 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:17.222 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:17.222 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:17.222 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.222 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.222 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:17.222 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:17.223 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:17.223 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.223 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.223 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:17.223 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:17.223 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3499999999999996e-07 2023-11-25T04:24:17.223 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.223 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.223 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:17.223 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033658+0000", 2023-11-25T04:24:17.223 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1280000000000001e-06 2023-11-25T04:24:17.223 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.224 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.224 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:17.224 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033667+0000", 2023-11-25T04:24:17.224 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1430000000000004e-06 2023-11-25T04:24:17.224 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.224 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.224 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:17.224 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033678+0000", 2023-11-25T04:24:17.224 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0906e-05 2023-11-25T04:24:17.224 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.224 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.225 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:17.225 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034456+0000", 2023-11-25T04:24:17.225 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00077833899999999996 2023-11-25T04:24:17.225 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.225 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.225 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:17.225 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034575+0000", 2023-11-25T04:24:17.225 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011926000000000001 2023-11-25T04:24:17.225 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.225 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.225 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:17.225 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035001+0000", 2023-11-25T04:24:17.225 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042550999999999998 2023-11-25T04:24:17.226 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.226 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.226 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:17.226 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.037923+0000", 2023-11-25T04:24:17.226 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029219670000000001 2023-11-25T04:24:17.226 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.226 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.226 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:17.226 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044538+0000", 2023-11-25T04:24:17.226 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0066152349999999997 2023-11-25T04:24:17.226 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.226 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.227 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:17.227 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044594+0000", 2023-11-25T04:24:17.227 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5593000000000001e-05 2023-11-25T04:24:17.227 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.227 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.227 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:17.227 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044633+0000", 2023-11-25T04:24:17.227 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9511000000000002e-05 2023-11-25T04:24:17.227 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:17.227 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:17.227 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:17.227 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.227 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.228 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:34 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:17.228 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:24:17.228 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 90.149523434000002, 2023-11-25T04:24:17.228 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010920445000000001, 2023-11-25T04:24:17.228 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:17.228 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:17.228 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:17.228 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.228 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:17.229 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:24:17.229 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:17.229 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.229 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.229 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:17.229 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:24:17.229 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:17.229 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.229 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.229 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:17.229 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033904+0000", 2023-11-25T04:24:17.229 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6599999999999994e-07 2023-11-25T04:24:17.230 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.230 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.230 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:17.230 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033909+0000", 2023-11-25T04:24:17.230 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5639999999999996e-06 2023-11-25T04:24:17.230 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.230 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.230 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:17.230 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033923+0000", 2023-11-25T04:24:17.230 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3028e-05 2023-11-25T04:24:17.230 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.230 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.231 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:17.231 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034043+0000", 2023-11-25T04:24:17.231 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000120328 2023-11-25T04:24:17.231 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.231 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.231 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:17.231 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035576+0000", 2023-11-25T04:24:17.231 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015333720000000001 2023-11-25T04:24:17.231 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.231 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.231 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:17.232 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035699+0000", 2023-11-25T04:24:17.232 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000123001 2023-11-25T04:24:17.232 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.232 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.232 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:17.232 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044823+0000", 2023-11-25T04:24:17.232 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0091241859999999994 2023-11-25T04:24:17.232 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:17.232 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:17.232 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:17.232 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.232 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.233 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:35 3.1f 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:17.233 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:24:17.233 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 90.148219229000006, 2023-11-25T04:24:17.233 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096164779999999995, 2023-11-25T04:24:17.233 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:17.233 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:17.233 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:17.233 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.233 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:17.233 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:24:17.234 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:17.234 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.234 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.234 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:17.234 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:24:17.234 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:17.234 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.234 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.234 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:17.234 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035208+0000", 2023-11-25T04:24:17.235 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2699999999999998e-07 2023-11-25T04:24:17.235 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.235 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.235 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:17.235 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035213+0000", 2023-11-25T04:24:17.235 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.6070000000000001e-06 2023-11-25T04:24:17.235 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.235 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.235 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:17.235 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035350+0000", 2023-11-25T04:24:17.235 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013693599999999999 2023-11-25T04:24:17.236 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.236 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.236 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:17.236 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035358+0000", 2023-11-25T04:24:17.236 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0099999999999995e-06 2023-11-25T04:24:17.236 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.236 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.236 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:17.236 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036260+0000", 2023-11-25T04:24:17.237 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00090240100000000005 2023-11-25T04:24:17.237 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.237 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.237 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:17.237 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036517+0000", 2023-11-25T04:24:17.237 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00025736299999999998 2023-11-25T04:24:17.237 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.237 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.237 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:17.237 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044824+0000", 2023-11-25T04:24:17.237 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0083063340000000003 2023-11-25T04:24:17.237 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:17.238 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:17.238 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:17.238 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.238 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.238 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:37 3.1e 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:17.238 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:24:17.239 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 90.148052543000006, 2023-11-25T04:24:17.239 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0078322640000000002, 2023-11-25T04:24:17.239 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:17.239 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:17.240 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:17.240 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.240 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:17.241 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:24:17.241 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:17.241 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.241 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.241 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:17.241 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:24:17.241 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:17.242 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.242 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.242 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:17.242 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035375+0000", 2023-11-25T04:24:17.242 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9100000000000003e-07 2023-11-25T04:24:17.242 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.242 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.242 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:17.242 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035379+0000", 2023-11-25T04:24:17.242 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9430000000000002e-06 2023-11-25T04:24:17.242 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.243 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.243 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:17.243 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035388+0000", 2023-11-25T04:24:17.243 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2760000000000006e-06 2023-11-25T04:24:17.243 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.243 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.243 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:17.243 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035396+0000", 2023-11-25T04:24:17.243 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8480000000000001e-06 2023-11-25T04:24:17.243 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.243 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.244 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:17.244 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035452+0000", 2023-11-25T04:24:17.244 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5831000000000001e-05 2023-11-25T04:24:17.244 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.244 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.244 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:17.244 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035934+0000", 2023-11-25T04:24:17.244 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00048265599999999998 2023-11-25T04:24:17.244 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.244 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.244 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:17.244 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043206+0000", 2023-11-25T04:24:17.245 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.007271919 2023-11-25T04:24:17.245 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:17.245 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:17.245 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:17.245 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.246 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.246 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing(3.1a epoch 18/16 rep_tid 3 entries 18'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:36 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:24:17.246 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:24:17.246 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 90.146776957, 2023-11-25T04:24:17.246 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002073414, 2023-11-25T04:24:17.246 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:17.246 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:24:17.247 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:17.247 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.247 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:17.247 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:24:17.247 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:17.247 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.247 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.247 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:17.247 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:24:17.247 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:17.248 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.248 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.248 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:17.248 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036650+0000", 2023-11-25T04:24:17.248 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7300000000000002e-07 2023-11-25T04:24:17.248 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.248 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.248 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:17.248 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036659+0000", 2023-11-25T04:24:17.248 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.7499999999999992e-06 2023-11-25T04:24:17.249 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.249 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.249 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:17.250 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036673+0000", 2023-11-25T04:24:17.250 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4917e-05 2023-11-25T04:24:17.250 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.250 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.250 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:17.250 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036687+0000", 2023-11-25T04:24:17.250 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3883000000000001e-05 2023-11-25T04:24:17.250 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.250 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.250 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:17.250 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036781+0000", 2023-11-25T04:24:17.251 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3219000000000003e-05 2023-11-25T04:24:17.251 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.251 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.251 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:17.251 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038723+0000", 2023-11-25T04:24:17.251 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019422720000000001 2023-11-25T04:24:17.251 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:17.251 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:17.251 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:17.251 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.251 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.251 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:28 3.13 e18/16 3:cd5a64a3:::100.00000000:head v 18'1, mlcod=0'0)", 2023-11-25T04:24:17.251 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:24:17.252 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 90.143115799, 2023-11-25T04:24:17.252 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001850546, 2023-11-25T04:24:17.252 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:17.252 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:17.252 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:17.252 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.252 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:17.252 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:24:17.252 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:17.252 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.252 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.252 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:17.253 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:24:17.253 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:17.253 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.253 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.253 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:17.253 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040311+0000", 2023-11-25T04:24:17.253 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7800000000000002e-07 2023-11-25T04:24:17.253 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.253 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.253 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:17.253 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040331+0000", 2023-11-25T04:24:17.254 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0449999999999999e-05 2023-11-25T04:24:17.254 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.254 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.254 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:17.254 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040339+0000", 2023-11-25T04:24:17.254 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0709999999999995e-06 2023-11-25T04:24:17.254 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.254 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.254 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:17.255 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040347+0000", 2023-11-25T04:24:17.255 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7079999999999997e-06 2023-11-25T04:24:17.255 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.255 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.255 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:17.255 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040410+0000", 2023-11-25T04:24:17.255 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2453000000000007e-05 2023-11-25T04:24:17.255 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.255 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.256 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:17.256 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040422+0000", 2023-11-25T04:24:17.256 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2429000000000001e-05 2023-11-25T04:24:17.256 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.256 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.256 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:17.256 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042129+0000", 2023-11-25T04:24:17.256 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001707266 2023-11-25T04:24:17.256 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.256 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.256 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:17.256 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042161+0000", 2023-11-25T04:24:17.256 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1791000000000001e-05 2023-11-25T04:24:17.257 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:17.257 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:17.257 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:17.257 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.257 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.257 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:27 3.13 e18/16 3:c91d4a1d:::mds0_inotable:head v 18'2, mlcod=0'0)", 2023-11-25T04:24:17.257 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:24:17.257 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 90.142912276000004, 2023-11-25T04:24:17.257 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0016886430000000001, 2023-11-25T04:24:17.257 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:17.257 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:17.257 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:17.258 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.258 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:17.258 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:24:17.258 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:17.258 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.258 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.258 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:17.258 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:24:17.258 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:17.258 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.258 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.258 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:17.259 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040514+0000", 2023-11-25T04:24:17.259 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3999999999999997e-07 2023-11-25T04:24:17.259 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.259 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.259 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:17.259 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040520+0000", 2023-11-25T04:24:17.259 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2989999999999996e-06 2023-11-25T04:24:17.259 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.259 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.259 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:17.259 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040527+0000", 2023-11-25T04:24:17.259 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5730000000000002e-06 2023-11-25T04:24:17.260 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.260 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.260 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:17.260 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040535+0000", 2023-11-25T04:24:17.260 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8630000000000008e-06 2023-11-25T04:24:17.260 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.260 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.260 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:17.260 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040683+0000", 2023-11-25T04:24:17.260 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000148002 2023-11-25T04:24:17.260 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.260 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.261 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:17.261 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.040700+0000", 2023-11-25T04:24:17.261 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7065e-05 2023-11-25T04:24:17.261 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.261 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.261 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:17.261 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042163+0000", 2023-11-25T04:24:17.261 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001462767 2023-11-25T04:24:17.261 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.261 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.261 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:17.262 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042203+0000", 2023-11-25T04:24:17.262 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9734000000000003e-05 2023-11-25T04:24:17.262 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:17.262 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:17.262 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:17.262 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.262 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.262 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:39 3.6 3:654134d2:::mds0_openfiles.0:head [omap-clear,omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2023-11-25T04:24:17.262 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:24:17.262 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 85.079115005000006, 2023-11-25T04:24:17.263 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001788658, 2023-11-25T04:24:17.263 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:17.263 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:17.263 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:17.264 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.264 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:17.264 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:24:17.264 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:17.264 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.264 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.264 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:17.264 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104311+0000", 2023-11-25T04:24:17.264 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:17.264 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.265 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.265 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:17.265 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104314+0000", 2023-11-25T04:24:17.265 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3099999999999999e-06 2023-11-25T04:24:17.265 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.265 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.265 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:17.265 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104325+0000", 2023-11-25T04:24:17.265 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1511999999999999e-05 2023-11-25T04:24:17.266 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.266 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.266 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:17.266 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104362+0000", 2023-11-25T04:24:17.266 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6903000000000002e-05 2023-11-25T04:24:17.266 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.266 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.266 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:17.266 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104387+0000", 2023-11-25T04:24:17.266 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4766e-05 2023-11-25T04:24:17.266 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.266 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.267 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:17.267 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104465+0000", 2023-11-25T04:24:17.267 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8119000000000002e-05 2023-11-25T04:24:17.267 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.267 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.267 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:17.267 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104557+0000", 2023-11-25T04:24:17.267 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2496000000000003e-05 2023-11-25T04:24:17.267 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.267 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.268 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:17.268 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.104646+0000", 2023-11-25T04:24:17.268 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8818000000000002e-05 2023-11-25T04:24:17.268 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.268 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.268 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:17.268 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105201+0000", 2023-11-25T04:24:17.268 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00055506000000000002 2023-11-25T04:24:17.269 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.269 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.269 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:17.269 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.105994+0000", 2023-11-25T04:24:17.269 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00079294700000000001 2023-11-25T04:24:17.269 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.269 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.269 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:17.269 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.106055+0000", 2023-11-25T04:24:17.269 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0566000000000003e-05 2023-11-25T04:24:17.270 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.270 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.270 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:17.270 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:52.106100+0000", 2023-11-25T04:24:17.270 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5160999999999997e-05 2023-11-25T04:24:17.270 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:17.270 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:17.270 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:17.270 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.270 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.270 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:43 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2023-11-25T04:24:17.271 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:24:17.271 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 70.078890009000006, 2023-11-25T04:24:17.271 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001751166, 2023-11-25T04:24:17.271 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:17.271 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:17.271 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:17.271 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.271 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:17.271 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:24:17.271 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:17.271 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.271 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.271 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:17.272 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104536+0000", 2023-11-25T04:24:17.272 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:17.272 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.272 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.272 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:17.272 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104539+0000", 2023-11-25T04:24:17.272 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9529999999999999e-06 2023-11-25T04:24:17.272 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.272 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.272 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:17.272 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104557+0000", 2023-11-25T04:24:17.272 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7464000000000001e-05 2023-11-25T04:24:17.272 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.273 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.273 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:17.273 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104590+0000", 2023-11-25T04:24:17.273 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2966000000000003e-05 2023-11-25T04:24:17.273 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.273 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.273 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:17.273 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104608+0000", 2023-11-25T04:24:17.273 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8193e-05 2023-11-25T04:24:17.273 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.273 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.273 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:17.274 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104767+0000", 2023-11-25T04:24:17.274 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015882 2023-11-25T04:24:17.274 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.274 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.274 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:17.274 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.104909+0000", 2023-11-25T04:24:17.274 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014244999999999999 2023-11-25T04:24:17.274 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.274 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.274 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:17.274 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.105059+0000", 2023-11-25T04:24:17.274 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014964399999999999 2023-11-25T04:24:17.274 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.275 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.275 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:17.275 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.105701+0000", 2023-11-25T04:24:17.275 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00064201099999999995 2023-11-25T04:24:17.275 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.275 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.275 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:17.275 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106233+0000", 2023-11-25T04:24:17.275 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00053178500000000001 2023-11-25T04:24:17.275 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.275 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.275 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:17.276 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106265+0000", 2023-11-25T04:24:17.276 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1986000000000002e-05 2023-11-25T04:24:17.276 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.276 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.276 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:17.276 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:23:07.106288+0000", 2023-11-25T04:24:17.276 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2894000000000001e-05 2023-11-25T04:24:17.276 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:17.276 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:17.276 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:17.276 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.276 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.277 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4561.0:1470 4.2f 4:f6bdc9b3:::100000007dd.00000000:head [write 0~1030] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2023-11-25T04:24:17.277 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:16.209116+0000", 2023-11-25T04:24:17.277 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 0.974310448, 2023-11-25T04:24:17.277 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001995031, 2023-11-25T04:24:17.277 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:17.277 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:17.277 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2023-11-25T04:24:17.277 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4561", 2023-11-25T04:24:17.277 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "192.168.0.1:0/1268434739", 2023-11-25T04:24:17.277 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1470 2023-11-25T04:24:17.277 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.277 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:17.278 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.278 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:17.278 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.209116+0000", 2023-11-25T04:24:17.278 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:17.278 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.278 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.278 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:17.278 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.209116+0000", 2023-11-25T04:24:17.278 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:17.278 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.278 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.278 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:17.279 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.209117+0000", 2023-11-25T04:24:17.279 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.61e-07 2023-11-25T04:24:17.279 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.279 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.279 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:17.279 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.209120+0000", 2023-11-25T04:24:17.279 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.76e-06 2023-11-25T04:24:17.279 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.279 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.279 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:17.280 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.209130+0000", 2023-11-25T04:24:17.280 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7990000000000005e-06 2023-11-25T04:24:17.280 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.280 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.280 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:17.280 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.209135+0000", 2023-11-25T04:24:17.280 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5970000000000002e-06 2023-11-25T04:24:17.280 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.280 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.280 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:17.280 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.209192+0000", 2023-11-25T04:24:17.280 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.7510000000000003e-05 2023-11-25T04:24:17.280 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.281 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.281 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:17.281 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.209264+0000", 2023-11-25T04:24:17.281 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1898999999999997e-05 2023-11-25T04:24:17.281 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.281 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.281 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:17.281 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.209335+0000", 2023-11-25T04:24:17.281 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0717999999999996e-05 2023-11-25T04:24:17.281 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.283 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.284 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:17.284 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.209799+0000", 2023-11-25T04:24:17.284 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00046449300000000001 2023-11-25T04:24:17.284 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.284 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.284 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:17.284 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.211061+0000", 2023-11-25T04:24:17.285 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001261535 2023-11-25T04:24:17.285 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.285 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.285 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:17.285 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.211092+0000", 2023-11-25T04:24:17.285 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1208999999999999e-05 2023-11-25T04:24:17.285 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.285 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.285 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:17.285 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.211111+0000", 2023-11-25T04:24:17.286 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8749999999999998e-05 2023-11-25T04:24:17.286 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:17.286 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:17.286 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:17.286 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.286 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.286 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4561.0:1495 4.4 4:21be38b0:::100000007f6.00000000:head [write 0~185] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2023-11-25T04:24:17.286 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:16.233053+0000", 2023-11-25T04:24:17.286 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 0.95037315700000002, 2023-11-25T04:24:17.286 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001657183, 2023-11-25T04:24:17.286 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:17.286 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:17.286 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2023-11-25T04:24:17.287 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4561", 2023-11-25T04:24:17.287 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "192.168.0.1:0/1268434739", 2023-11-25T04:24:17.287 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1495 2023-11-25T04:24:17.287 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.287 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:17.287 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.287 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:17.287 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.233053+0000", 2023-11-25T04:24:17.287 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:17.287 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.288 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.288 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:17.288 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.233053+0000", 2023-11-25T04:24:17.288 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:17.288 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.288 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.288 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:17.288 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.233055+0000", 2023-11-25T04:24:17.288 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.95e-06 2023-11-25T04:24:17.288 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.288 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.289 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:17.289 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.233067+0000", 2023-11-25T04:24:17.289 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2276000000000001e-05 2023-11-25T04:24:17.289 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.289 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.289 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:17.289 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.233095+0000", 2023-11-25T04:24:17.289 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7370000000000001e-05 2023-11-25T04:24:17.289 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.289 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.289 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:17.289 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.233108+0000", 2023-11-25T04:24:17.290 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3543000000000001e-05 2023-11-25T04:24:17.290 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.290 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.290 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:17.290 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.233205+0000", 2023-11-25T04:24:17.290 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6148000000000002e-05 2023-11-25T04:24:17.290 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.290 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.290 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:17.290 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.233311+0000", 2023-11-25T04:24:17.290 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010657 2023-11-25T04:24:17.290 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.291 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.291 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:17.291 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.233379+0000", 2023-11-25T04:24:17.291 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8349000000000006e-05 2023-11-25T04:24:17.291 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.291 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.291 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:17.291 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.233784+0000", 2023-11-25T04:24:17.291 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00040412800000000002 2023-11-25T04:24:17.291 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.291 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.291 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:17.292 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.234651+0000", 2023-11-25T04:24:17.292 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00086777499999999999 2023-11-25T04:24:17.292 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.292 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.292 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:17.292 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.234680+0000", 2023-11-25T04:24:17.292 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8798000000000001e-05 2023-11-25T04:24:17.292 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.292 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.292 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:17.292 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.234710+0000", 2023-11-25T04:24:17.292 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0275999999999999e-05 2023-11-25T04:24:17.292 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:17.293 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:17.293 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:17.293 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.293 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.293 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4561.0:2151 4.21 4:85442bc0:::10000000a86.00000000:head [write 0~200] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2023-11-25T04:24:17.293 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:16.819976+0000", 2023-11-25T04:24:17.293 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 0.36345034399999998, 2023-11-25T04:24:17.293 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0020804280000000001, 2023-11-25T04:24:17.293 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:17.293 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:17.293 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2023-11-25T04:24:17.293 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4561", 2023-11-25T04:24:17.294 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "192.168.0.1:0/1268434739", 2023-11-25T04:24:17.294 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 2151 2023-11-25T04:24:17.294 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.294 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:17.294 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.294 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:17.294 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.819976+0000", 2023-11-25T04:24:17.294 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:17.294 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.295 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.295 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:17.295 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.819976+0000", 2023-11-25T04:24:17.295 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:17.295 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.295 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.295 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:17.295 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.819977+0000", 2023-11-25T04:24:17.295 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.8899999999999998e-07 2023-11-25T04:24:17.295 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.295 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.295 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:17.295 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.819982+0000", 2023-11-25T04:24:17.296 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5279999999999997e-06 2023-11-25T04:24:17.296 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.296 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.296 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:17.296 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.819992+0000", 2023-11-25T04:24:17.296 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0478e-05 2023-11-25T04:24:17.296 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.296 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.296 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:17.296 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.820004+0000", 2023-11-25T04:24:17.296 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1922e-05 2023-11-25T04:24:17.297 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.297 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.297 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:17.297 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.820051+0000", 2023-11-25T04:24:17.297 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7136999999999999e-05 2023-11-25T04:24:17.297 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.297 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.297 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:17.297 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.820173+0000", 2023-11-25T04:24:17.297 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000121513 2023-11-25T04:24:17.297 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.297 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.298 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:17.298 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.820298+0000", 2023-11-25T04:24:17.298 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012530899999999999 2023-11-25T04:24:17.298 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.298 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.298 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:17.298 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.820811+0000", 2023-11-25T04:24:17.298 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00051313900000000002 2023-11-25T04:24:17.298 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.298 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.298 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:17.299 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.822004+0000", 2023-11-25T04:24:17.299 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00119253 2023-11-25T04:24:17.299 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.299 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.299 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:17.299 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.822034+0000", 2023-11-25T04:24:17.299 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0759999999999997e-05 2023-11-25T04:24:17.299 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.299 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.299 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:17.299 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.822056+0000", 2023-11-25T04:24:17.299 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2123000000000001e-05 2023-11-25T04:24:17.300 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:17.300 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:17.300 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:17.300 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.300 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.300 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:118 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals,omap-rm-keys] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e44)", 2023-11-25T04:24:17.300 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:17.112648+0000", 2023-11-25T04:24:17.300 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 0.070778093, 2023-11-25T04:24:17.300 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011250779000000001, 2023-11-25T04:24:17.300 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:17.300 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:17.300 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:17.300 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.301 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:17.301 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.112648+0000", 2023-11-25T04:24:17.301 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:17.301 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.301 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.301 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:17.301 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.112648+0000", 2023-11-25T04:24:17.301 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:17.301 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.301 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.301 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:17.302 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.112649+0000", 2023-11-25T04:24:17.302 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0070000000000001e-06 2023-11-25T04:24:17.302 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.302 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.302 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:17.302 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113182+0000", 2023-11-25T04:24:17.302 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00053268100000000002 2023-11-25T04:24:17.302 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.302 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.302 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:17.302 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113196+0000", 2023-11-25T04:24:17.303 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3742e-05 2023-11-25T04:24:17.303 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.303 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.303 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:17.303 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113398+0000", 2023-11-25T04:24:17.303 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000202726 2023-11-25T04:24:17.303 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.303 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.303 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:17.303 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113447+0000", 2023-11-25T04:24:17.303 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8655000000000001e-05 2023-11-25T04:24:17.303 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.304 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.304 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:17.304 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113529+0000", 2023-11-25T04:24:17.304 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1891999999999995e-05 2023-11-25T04:24:17.304 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.304 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.304 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:17.304 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123819+0000", 2023-11-25T04:24:17.304 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010290192 2023-11-25T04:24:17.304 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.304 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.304 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:17.305 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123852+0000", 2023-11-25T04:24:17.305 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2425999999999998e-05 2023-11-25T04:24:17.305 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.305 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.305 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:17.305 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123899+0000", 2023-11-25T04:24:17.305 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7457999999999999e-05 2023-11-25T04:24:17.305 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:17.305 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:17.305 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:17.305 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.306 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.306 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4561.0:2494 4.7 4:e1292348:::10000000bde.00000000:head [write 0~3041] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2023-11-25T04:24:17.306 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:17.120454+0000", 2023-11-25T04:24:17.306 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 0.062972696999999994, 2023-11-25T04:24:17.306 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0037327419999999998, 2023-11-25T04:24:17.306 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:17.306 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:17.306 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2023-11-25T04:24:17.306 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4561", 2023-11-25T04:24:17.306 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "192.168.0.1:0/1268434739", 2023-11-25T04:24:17.306 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 2494 2023-11-25T04:24:17.307 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.307 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:17.307 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.307 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:17.307 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.120454+0000", 2023-11-25T04:24:17.307 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:17.307 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.307 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.307 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:17.307 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.120454+0000", 2023-11-25T04:24:17.308 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:17.308 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.308 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.308 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:17.308 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.120456+0000", 2023-11-25T04:24:17.308 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0899999999999999e-06 2023-11-25T04:24:17.308 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.308 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.308 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:17.308 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.120466+0000", 2023-11-25T04:24:17.308 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0409e-05 2023-11-25T04:24:17.309 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.309 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.309 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:17.309 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.120484+0000", 2023-11-25T04:24:17.309 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7456e-05 2023-11-25T04:24:17.309 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.309 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.309 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:17.309 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.120495+0000", 2023-11-25T04:24:17.309 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1268e-05 2023-11-25T04:24:17.309 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.309 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.309 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:17.310 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.120568+0000", 2023-11-25T04:24:17.310 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3374999999999995e-05 2023-11-25T04:24:17.310 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.310 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.310 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:17.310 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.120707+0000", 2023-11-25T04:24:17.310 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013880699999999999 2023-11-25T04:24:17.310 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.310 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.310 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:17.310 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.120847+0000", 2023-11-25T04:24:17.310 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013949899999999999 2023-11-25T04:24:17.311 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.311 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.311 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:17.311 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.121627+0000", 2023-11-25T04:24:17.311 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00078049 2023-11-25T04:24:17.311 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.311 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.311 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:17.311 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.124139+0000", 2023-11-25T04:24:17.311 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0025116539999999999 2023-11-25T04:24:17.311 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.311 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.312 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:17.312 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.124168+0000", 2023-11-25T04:24:17.312 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9237000000000001e-05 2023-11-25T04:24:17.312 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.312 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.312 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:17.312 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.124186+0000", 2023-11-25T04:24:17.312 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8457e-05 2023-11-25T04:24:17.312 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:17.312 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:17.313 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:17.313 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.313 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.313 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4561.0:2497 4.37 4:ed6942ad:::10000000be1.00000000:head [write 0~1531] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2023-11-25T04:24:17.313 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:17.123256+0000", 2023-11-25T04:24:17.313 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 0.060170573999999998, 2023-11-25T04:24:17.313 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.003839617, 2023-11-25T04:24:17.313 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:17.313 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:17.313 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2023-11-25T04:24:17.313 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4561", 2023-11-25T04:24:17.313 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "192.168.0.1:0/1268434739", 2023-11-25T04:24:17.313 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 2497 2023-11-25T04:24:17.314 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.314 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:17.314 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.314 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:17.314 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123256+0000", 2023-11-25T04:24:17.314 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:17.314 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.314 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.314 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:17.314 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123256+0000", 2023-11-25T04:24:17.314 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:17.315 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.315 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.315 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:17.315 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123257+0000", 2023-11-25T04:24:17.315 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.062e-06 2023-11-25T04:24:17.315 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.315 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.315 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:17.315 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123262+0000", 2023-11-25T04:24:17.315 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3879999999999999e-06 2023-11-25T04:24:17.315 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.315 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.316 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:17.316 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123277+0000", 2023-11-25T04:24:17.316 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4416999999999999e-05 2023-11-25T04:24:17.316 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.316 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.316 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:17.316 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123285+0000", 2023-11-25T04:24:17.316 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1790000000000008e-06 2023-11-25T04:24:17.316 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.316 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.316 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:17.316 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123326+0000", 2023-11-25T04:24:17.317 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0707999999999997e-05 2023-11-25T04:24:17.317 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.317 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.317 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:17.317 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123396+0000", 2023-11-25T04:24:17.317 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0032999999999998e-05 2023-11-25T04:24:17.317 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.317 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.317 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:17.317 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123467+0000", 2023-11-25T04:24:17.317 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0937000000000002e-05 2023-11-25T04:24:17.317 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.317 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.318 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:17.318 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.124068+0000", 2023-11-25T04:24:17.318 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00060176299999999995 2023-11-25T04:24:17.318 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.318 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.318 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:17.318 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.127046+0000", 2023-11-25T04:24:17.318 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002977871 2023-11-25T04:24:17.318 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.318 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.318 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:17.318 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.127073+0000", 2023-11-25T04:24:17.319 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7118999999999999e-05 2023-11-25T04:24:17.319 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:17.319 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:17.319 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:17.319 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.127095+0000", 2023-11-25T04:24:17.319 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2140000000000001e-05 2023-11-25T04:24:17.319 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:17.319 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:17.319 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:17.319 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:17.319 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:17.319 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:17.321 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:17.319+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:17.322 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:24:17.421 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:17.420+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:17.446 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:17.446 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:24:17.446 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2023-11-25T04:24:17.446 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:17.454 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:24:17.522 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:17.520+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:17.610 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:17.610 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:24:17.610 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2023-11-25T04:24:17.610 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2023-11-25T04:24:17.611 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:17.619 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:24:17.666 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:17.663+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:17.724 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:17.722+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:17.743 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:17.743 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2023-11-25T04:24:17.744 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2023-11-25T04:24:17.744 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2023-11-25T04:24:17.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.744 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:155 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'109, mlcod=11'105)", 2023-11-25T04:24:17.744 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:24:17.744 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 99.956022321000006, 2023-11-25T04:24:17.744 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061719019999999999, 2023-11-25T04:24:17.744 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:17.744 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:17.744 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:17.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.744 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:17.744 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:24:17.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:17.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:17.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:24:17.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:17.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:17.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784884+0000", 2023-11-25T04:24:17.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3099999999999999e-07 2023-11-25T04:24:17.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:17.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784942+0000", 2023-11-25T04:24:17.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8570000000000003e-05 2023-11-25T04:24:17.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:17.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784949+0000", 2023-11-25T04:24:17.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0310000000000001e-06 2023-11-25T04:24:17.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.747 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:17.747 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784953+0000", 2023-11-25T04:24:17.747 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6320000000000001e-06 2023-11-25T04:24:17.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.747 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.747 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:17.747 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784993+0000", 2023-11-25T04:24:17.747 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9737000000000003e-05 2023-11-25T04:24:17.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.748 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.748 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:17.748 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785011+0000", 2023-11-25T04:24:17.748 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8349999999999999e-05 2023-11-25T04:24:17.748 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.748 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.748 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:17.748 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791024+0000", 2023-11-25T04:24:17.748 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006012997 2023-11-25T04:24:17.748 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.748 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.748 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:17.748 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791055+0000", 2023-11-25T04:24:17.749 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1254000000000003e-05 2023-11-25T04:24:17.749 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:17.749 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:17.749 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:17.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.749 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.749 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:156 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'110, mlcod=11'105)", 2023-11-25T04:24:17.750 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:24:17.750 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 99.955436272, 2023-11-25T04:24:17.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005616116, 2023-11-25T04:24:17.750 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:17.750 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:17.750 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:17.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:17.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:24:17.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:17.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:17.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:24:17.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:17.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:17.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785470+0000", 2023-11-25T04:24:17.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0699999999999998e-07 2023-11-25T04:24:17.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.752 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:17.752 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785474+0000", 2023-11-25T04:24:17.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.048e-06 2023-11-25T04:24:17.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.752 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:17.752 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785480+0000", 2023-11-25T04:24:17.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7729999999999996e-06 2023-11-25T04:24:17.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.753 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.753 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:17.753 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785483+0000", 2023-11-25T04:24:17.753 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1970000000000001e-06 2023-11-25T04:24:17.753 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.753 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.753 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:17.753 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787936+0000", 2023-11-25T04:24:17.753 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024529040000000001 2023-11-25T04:24:17.753 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.753 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.753 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:17.753 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787947+0000", 2023-11-25T04:24:17.754 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1669e-05 2023-11-25T04:24:17.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.754 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:17.754 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791057+0000", 2023-11-25T04:24:17.754 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031095200000000002 2023-11-25T04:24:17.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:17.756 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791086+0000", 2023-11-25T04:24:17.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8697999999999999e-05 2023-11-25T04:24:17.757 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:17.757 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:17.757 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:17.757 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.757 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.757 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:157 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'111, mlcod=11'106)", 2023-11-25T04:24:17.757 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:24:17.757 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 99.954843714000006, 2023-11-25T04:24:17.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050590280000000001, 2023-11-25T04:24:17.757 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:17.758 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:17.758 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:17.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.758 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:17.758 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:24:17.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:17.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.758 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:17.758 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:24:17.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:17.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:17.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786063+0000", 2023-11-25T04:24:17.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2600000000000002e-07 2023-11-25T04:24:17.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:17.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786069+0000", 2023-11-25T04:24:17.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8839999999999999e-06 2023-11-25T04:24:17.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:17.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786079+0000", 2023-11-25T04:24:17.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0124e-05 2023-11-25T04:24:17.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:17.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786085+0000", 2023-11-25T04:24:17.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6660000000000002e-06 2023-11-25T04:24:17.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:17.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788160+0000", 2023-11-25T04:24:17.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020747669999999999 2023-11-25T04:24:17.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:17.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788180+0000", 2023-11-25T04:24:17.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0208999999999999e-05 2023-11-25T04:24:17.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:17.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791087+0000", 2023-11-25T04:24:17.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029064630000000002 2023-11-25T04:24:17.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.762 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:17.762 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791121+0000", 2023-11-25T04:24:17.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4289000000000003e-05 2023-11-25T04:24:17.762 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:17.762 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:17.762 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:17.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.762 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:158 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'112, mlcod=11'106)", 2023-11-25T04:24:17.763 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:24:17.763 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 99.954427320999997, 2023-11-25T04:24:17.763 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046663030000000001, 2023-11-25T04:24:17.763 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:17.763 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:17.763 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:17.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.763 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:17.763 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:24:17.763 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:17.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:17.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786478+0000", 2023-11-25T04:24:17.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:17.764 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.764 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:17.764 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786479+0000", 2023-11-25T04:24:17.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2799999999999996e-07 2023-11-25T04:24:17.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:17.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786498+0000", 2023-11-25T04:24:17.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9069e-05 2023-11-25T04:24:17.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:17.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786508+0000", 2023-11-25T04:24:17.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.5810000000000003e-06 2023-11-25T04:24:17.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.766 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:17.766 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786512+0000", 2023-11-25T04:24:17.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4889999999999997e-06 2023-11-25T04:24:17.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.766 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:17.766 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788438+0000", 2023-11-25T04:24:17.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019257059999999999 2023-11-25T04:24:17.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.766 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:17.767 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788449+0000", 2023-11-25T04:24:17.767 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1419e-05 2023-11-25T04:24:17.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.767 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:17.767 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791122+0000", 2023-11-25T04:24:17.767 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002672822 2023-11-25T04:24:17.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.767 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:17.767 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791145+0000", 2023-11-25T04:24:17.767 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2589e-05 2023-11-25T04:24:17.768 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:17.768 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:17.768 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:17.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.768 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:206 1.0 e12/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 12'152, mlcod=12'148)", 2023-11-25T04:24:17.768 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:24:17.768 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 99.442658851000004, 2023-11-25T04:24:17.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0046728189999999999, 2023-11-25T04:24:17.768 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:17.768 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:17.768 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:17.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:17.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:24:17.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:17.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:17.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298247+0000", 2023-11-25T04:24:17.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:17.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:17.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298248+0000", 2023-11-25T04:24:17.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3600000000000003e-07 2023-11-25T04:24:17.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:17.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298278+0000", 2023-11-25T04:24:17.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0474e-05 2023-11-25T04:24:17.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:17.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298288+0000", 2023-11-25T04:24:17.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0156000000000001e-05 2023-11-25T04:24:17.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.771 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:17.771 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298293+0000", 2023-11-25T04:24:17.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7400000000000004e-06 2023-11-25T04:24:17.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.771 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:17.771 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298333+0000", 2023-11-25T04:24:17.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9666000000000003e-05 2023-11-25T04:24:17.771 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.771 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:17.772 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.298351+0000", 2023-11-25T04:24:17.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8544999999999999e-05 2023-11-25T04:24:17.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.772 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.772 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:17.772 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302891+0000", 2023-11-25T04:24:17.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0045398519999999996 2023-11-25T04:24:17.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.772 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.772 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:17.772 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:38.302920+0000", 2023-11-25T04:24:17.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8750000000000001e-05 2023-11-25T04:24:17.773 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:17.773 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:17.773 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:17.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.773 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:14 3.15 3:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:17.773 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:24:17.773 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 90.707524493999998, 2023-11-25T04:24:17.773 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010990437, 2023-11-25T04:24:17.773 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:17.773 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:17.773 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:17.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:17.774 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:24:17.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:17.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:17.774 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:24:17.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:17.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:17.774 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033383+0000", 2023-11-25T04:24:17.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9870000000000002e-06 2023-11-25T04:24:17.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:17.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033398+0000", 2023-11-25T04:24:17.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4569999999999999e-05 2023-11-25T04:24:17.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:17.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033411+0000", 2023-11-25T04:24:17.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3543999999999999e-05 2023-11-25T04:24:17.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:17.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033423+0000", 2023-11-25T04:24:17.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1905999999999999e-05 2023-11-25T04:24:17.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:17.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034433+0000", 2023-11-25T04:24:17.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00101022 2023-11-25T04:24:17.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:17.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034539+0000", 2023-11-25T04:24:17.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010597199999999999 2023-11-25T04:24:17.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.777 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:17.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034667+0000", 2023-11-25T04:24:17.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012765500000000001 2023-11-25T04:24:17.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.777 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:17.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037803+0000", 2023-11-25T04:24:17.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031362220000000001 2023-11-25T04:24:17.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:17.778 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:24:17.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0064964669999999997 2023-11-25T04:24:17.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:17.778 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044331+0000", 2023-11-25T04:24:17.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1544999999999997e-05 2023-11-25T04:24:17.778 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:17.778 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044372+0000", 2023-11-25T04:24:17.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0349000000000003e-05 2023-11-25T04:24:17.779 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:17.779 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:17.779 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:17.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.779 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:15 3.15 3:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:17.779 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:24:17.779 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 90.706864843999995, 2023-11-25T04:24:17.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010464362, 2023-11-25T04:24:17.779 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:17.779 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:17.779 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:17.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.780 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:17.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:24:17.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:17.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.780 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:17.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:24:17.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:17.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.780 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:17.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034042+0000", 2023-11-25T04:24:17.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0999999999999997e-07 2023-11-25T04:24:17.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:17.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034048+0000", 2023-11-25T04:24:17.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0390000000000001e-06 2023-11-25T04:24:17.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:17.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034059+0000", 2023-11-25T04:24:17.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1182999999999999e-05 2023-11-25T04:24:17.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:17.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034067+0000", 2023-11-25T04:24:17.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1559999999999995e-06 2023-11-25T04:24:17.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:17.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035005+0000", 2023-11-25T04:24:17.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00093747399999999999 2023-11-25T04:24:17.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:17.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035077+0000", 2023-11-25T04:24:17.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2747e-05 2023-11-25T04:24:17.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:17.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035219+0000", 2023-11-25T04:24:17.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000141877 2023-11-25T04:24:17.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:17.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037810+0000", 2023-11-25T04:24:17.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002590752 2023-11-25T04:24:17.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:17.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044437+0000", 2023-11-25T04:24:17.784 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066270110000000004 2023-11-25T04:24:17.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.784 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:17.784 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044481+0000", 2023-11-25T04:24:17.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4106000000000001e-05 2023-11-25T04:24:17.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.785 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:17.785 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044505+0000", 2023-11-25T04:24:17.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4207e-05 2023-11-25T04:24:17.785 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:17.785 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:17.785 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:17.785 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.785 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:16 3.1d 3:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:17.786 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:24:17.786 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 90.706822996, 2023-11-25T04:24:17.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010413923, 2023-11-25T04:24:17.786 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:17.786 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:17.786 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:17.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:17.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:24:17.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:17.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:17.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:24:17.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:17.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:17.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:24:17.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4499999999999997e-07 2023-11-25T04:24:17.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:17.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034087+0000", 2023-11-25T04:24:17.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9770000000000002e-06 2023-11-25T04:24:17.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:17.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034094+0000", 2023-11-25T04:24:17.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8809999999999998e-06 2023-11-25T04:24:17.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:17.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034100+0000", 2023-11-25T04:24:17.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8440000000000003e-06 2023-11-25T04:24:17.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:17.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034993+0000", 2023-11-25T04:24:17.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00089336799999999996 2023-11-25T04:24:17.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:17.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035099+0000", 2023-11-25T04:24:17.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106037 2023-11-25T04:24:17.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:17.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035276+0000", 2023-11-25T04:24:17.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017631799999999999 2023-11-25T04:24:17.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.790 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:17.790 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037815+0000", 2023-11-25T04:24:17.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025395959999999999 2023-11-25T04:24:17.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.790 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:17.790 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044420+0000", 2023-11-25T04:24:17.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066045749999999997 2023-11-25T04:24:17.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:17.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044463+0000", 2023-11-25T04:24:17.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3205999999999999e-05 2023-11-25T04:24:17.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.791 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:17.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044497+0000", 2023-11-25T04:24:17.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3476000000000003e-05 2023-11-25T04:24:17.791 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:17.791 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:17.791 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:17.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.792 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:17 3.1d 3:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:17.792 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:24:17.792 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 90.706797786999999, 2023-11-25T04:24:17.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01127356, 2023-11-25T04:24:17.792 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:17.792 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:17.792 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:17.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:17.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:24:17.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:17.792 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:17.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:24:17.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:17.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:17.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:24:17.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1100000000000001e-07 2023-11-25T04:24:17.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:17.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034112+0000", 2023-11-25T04:24:17.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8600000000000003e-06 2023-11-25T04:24:17.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.794 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:17.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034120+0000", 2023-11-25T04:24:17.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7700000000000001e-06 2023-11-25T04:24:17.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.794 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:17.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034127+0000", 2023-11-25T04:24:17.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0999999999999998e-06 2023-11-25T04:24:17.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:17.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035601+0000", 2023-11-25T04:24:17.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014739410000000001 2023-11-25T04:24:17.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:17.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035685+0000", 2023-11-25T04:24:17.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3677999999999994e-05 2023-11-25T04:24:17.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:17.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035827+0000", 2023-11-25T04:24:17.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014232599999999999 2023-11-25T04:24:17.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:17.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038725+0000", 2023-11-25T04:24:17.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028982779999999998 2023-11-25T04:24:17.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:17.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045323+0000", 2023-11-25T04:24:17.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0065980780000000003 2023-11-25T04:24:17.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:17.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045354+0000", 2023-11-25T04:24:17.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0463e-05 2023-11-25T04:24:17.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:17.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045381+0000", 2023-11-25T04:24:17.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7654999999999999e-05 2023-11-25T04:24:17.797 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:17.797 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:17.797 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:17.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.798 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:22 3.e 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:17.798 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:24:17.798 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 90.706771373999999, 2023-11-25T04:24:17.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010133422, 2023-11-25T04:24:17.798 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:17.798 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:17.798 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:17.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.798 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:17.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:24:17.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:17.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:17.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:24:17.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:17.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:17.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034135+0000", 2023-11-25T04:24:17.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-07 2023-11-25T04:24:17.799 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:17.799 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034138+0000", 2023-11-25T04:24:17.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6009999999999999e-06 2023-11-25T04:24:17.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:17.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034145+0000", 2023-11-25T04:24:17.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1169999999999998e-06 2023-11-25T04:24:17.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:17.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034150+0000", 2023-11-25T04:24:17.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9080000000000003e-06 2023-11-25T04:24:17.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:17.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034209+0000", 2023-11-25T04:24:17.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8149000000000002e-05 2023-11-25T04:24:17.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:17.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034415+0000", 2023-11-25T04:24:17.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020637699999999999 2023-11-25T04:24:17.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:17.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034584+0000", 2023-11-25T04:24:17.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016870399999999999 2023-11-25T04:24:17.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:17.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.036979+0000", 2023-11-25T04:24:17.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023957509999999998 2023-11-25T04:24:17.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:17.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044161+0000", 2023-11-25T04:24:17.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0071811590000000003 2023-11-25T04:24:17.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:17.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044224+0000", 2023-11-25T04:24:17.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3454000000000006e-05 2023-11-25T04:24:17.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:17.803 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044268+0000", 2023-11-25T04:24:17.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3761999999999999e-05 2023-11-25T04:24:17.803 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:17.803 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:17.803 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:17.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.803 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:23 3.e 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:17.803 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:24:17.803 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 90.706739964999997, 2023-11-25T04:24:17.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010281676, 2023-11-25T04:24:17.804 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:17.804 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:17.804 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:17.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.804 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:17.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:24:17.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:17.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.804 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:17.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:24:17.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:17.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:17.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:24:17.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5799999999999999e-07 2023-11-25T04:24:17.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:17.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034169+0000", 2023-11-25T04:24:17.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1879999999999998e-06 2023-11-25T04:24:17.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.805 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:17.805 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034177+0000", 2023-11-25T04:24:17.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7020000000000001e-06 2023-11-25T04:24:17.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:17.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034184+0000", 2023-11-25T04:24:17.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3910000000000003e-06 2023-11-25T04:24:17.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:17.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034924+0000", 2023-11-25T04:24:17.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00074032999999999998 2023-11-25T04:24:17.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:17.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034991+0000", 2023-11-25T04:24:17.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7347000000000004e-05 2023-11-25T04:24:17.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:17.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035096+0000", 2023-11-25T04:24:17.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010488 2023-11-25T04:24:17.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:17.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037588+0000", 2023-11-25T04:24:17.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024918309999999999 2023-11-25T04:24:17.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:17.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044367+0000", 2023-11-25T04:24:17.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067787589999999997 2023-11-25T04:24:17.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:17.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044415+0000", 2023-11-25T04:24:17.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7803000000000003e-05 2023-11-25T04:24:17.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:17.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044447+0000", 2023-11-25T04:24:17.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2886999999999999e-05 2023-11-25T04:24:17.809 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:17.809 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:17.809 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:17.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.809 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:30 3.15 3:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:17.809 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:24:17.809 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 90.706681524999993, 2023-11-25T04:24:17.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010365070000000001, 2023-11-25T04:24:17.810 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:17.810 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:17.810 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:17.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:17.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:24:17.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:17.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:17.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:24:17.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:17.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.811 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:17.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034225+0000", 2023-11-25T04:24:17.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7400000000000002e-07 2023-11-25T04:24:17.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.811 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:17.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034229+0000", 2023-11-25T04:24:17.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3100000000000002e-06 2023-11-25T04:24:17.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.811 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:17.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034239+0000", 2023-11-25T04:24:17.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9140000000000003e-06 2023-11-25T04:24:17.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.812 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:17.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034246+0000", 2023-11-25T04:24:17.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0149999999999998e-06 2023-11-25T04:24:17.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.812 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:17.812 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035572+0000", 2023-11-25T04:24:17.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001325655 2023-11-25T04:24:17.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:17.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035730+0000", 2023-11-25T04:24:17.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015827899999999999 2023-11-25T04:24:17.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:17.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035857+0000", 2023-11-25T04:24:17.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012644699999999999 2023-11-25T04:24:17.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:17.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038737+0000", 2023-11-25T04:24:17.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002880077 2023-11-25T04:24:17.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:17.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044540+0000", 2023-11-25T04:24:17.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058034239999999997 2023-11-25T04:24:17.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:17.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044569+0000", 2023-11-25T04:24:17.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8388999999999999e-05 2023-11-25T04:24:17.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:17.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044589+0000", 2023-11-25T04:24:17.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0686000000000002e-05 2023-11-25T04:24:17.815 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:17.815 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:17.815 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:17.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.815 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:33 3.1b 3:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:17.815 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:24:17.815 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 90.706636027000002, 2023-11-25T04:24:17.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099754049999999997, 2023-11-25T04:24:17.815 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:17.815 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:17.815 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:17.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:17.816 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:24:17.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:17.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:17.816 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:24:17.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:17.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:17.816 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:24:17.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4799999999999998e-07 2023-11-25T04:24:17.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:17.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034273+0000", 2023-11-25T04:24:17.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7819999999999998e-06 2023-11-25T04:24:17.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:17.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034285+0000", 2023-11-25T04:24:17.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2058e-05 2023-11-25T04:24:17.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:17.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034291+0000", 2023-11-25T04:24:17.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0680000000000002e-06 2023-11-25T04:24:17.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:17.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034370+0000", 2023-11-25T04:24:17.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8452e-05 2023-11-25T04:24:17.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:17.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034477+0000", 2023-11-25T04:24:17.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107471 2023-11-25T04:24:17.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.819 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:17.819 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044245+0000", 2023-11-25T04:24:17.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0097679259999999997 2023-11-25T04:24:17.819 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:17.819 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:17.819 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:17.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.819 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:36 3.1a 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:17.820 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:24:17.820 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 90.706589354000002, 2023-11-25T04:24:17.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058179099999999999, 2023-11-25T04:24:17.820 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:17.820 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:17.820 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:17.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:17.820 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:24:17.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:17.820 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.820 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.820 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:17.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:24:17.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:17.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:17.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034317+0000", 2023-11-25T04:24:17.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1999999999999999e-07 2023-11-25T04:24:17.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:17.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034321+0000", 2023-11-25T04:24:17.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.233e-06 2023-11-25T04:24:17.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:17.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034337+0000", 2023-11-25T04:24:17.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5308999999999999e-05 2023-11-25T04:24:17.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:17.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034354+0000", 2023-11-25T04:24:17.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7411999999999999e-05 2023-11-25T04:24:17.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:17.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034465+0000", 2023-11-25T04:24:17.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011154699999999999 2023-11-25T04:24:17.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.823 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:17.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034560+0000", 2023-11-25T04:24:17.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4186999999999998e-05 2023-11-25T04:24:17.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.823 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:17.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.040134+0000", 2023-11-25T04:24:17.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055746019999999997 2023-11-25T04:24:17.823 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:17.823 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:17.824 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:17.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.824 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:9 3.10 e18/16 3:09401a81:::601.00000000:head v 18'4, mlcod=0'0)", 2023-11-25T04:24:17.824 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:24:17.824 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 90.697695046000007, 2023-11-25T04:24:17.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061641020000000003, 2023-11-25T04:24:17.824 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:17.824 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:17.824 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:17.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:17.824 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:24:17.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:17.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:17.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:24:17.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:17.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:17.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:24:17.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5300000000000001e-07 2023-11-25T04:24:17.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:17.826 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043222+0000", 2023-11-25T04:24:17.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.082e-05 2023-11-25T04:24:17.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:17.826 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043228+0000", 2023-11-25T04:24:17.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6699999999999997e-06 2023-11-25T04:24:17.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:17.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043236+0000", 2023-11-25T04:24:17.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8189999999999992e-06 2023-11-25T04:24:17.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.827 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:17.827 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043507+0000", 2023-11-25T04:24:17.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000271012 2023-11-25T04:24:17.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:17.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043530+0000", 2023-11-25T04:24:17.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2659000000000001e-05 2023-11-25T04:24:17.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:17.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049321+0000", 2023-11-25T04:24:17.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057915129999999999 2023-11-25T04:24:17.828 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:17.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049375+0000", 2023-11-25T04:24:17.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3455999999999998e-05 2023-11-25T04:24:17.829 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:17.829 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:17.829 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:17.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.829 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:1 3.14 e18/16 3:292cf221:::200.00000000:head v 18'3, mlcod=0'0)", 2023-11-25T04:24:17.829 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:24:17.829 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 90.697658333000007, 2023-11-25T04:24:17.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0060983599999999997, 2023-11-25T04:24:17.829 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:17.830 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:17.830 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:17.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:17.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:24:17.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:17.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:17.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:24:17.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:17.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:17.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043248+0000", 2023-11-25T04:24:17.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9399999999999995e-07 2023-11-25T04:24:17.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:17.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043252+0000", 2023-11-25T04:24:17.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1799999999999998e-06 2023-11-25T04:24:17.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:17.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043259+0000", 2023-11-25T04:24:17.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0049999999999998e-06 2023-11-25T04:24:17.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.832 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:17.832 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043265+0000", 2023-11-25T04:24:17.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0859999999999998e-06 2023-11-25T04:24:17.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.832 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:17.832 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043311+0000", 2023-11-25T04:24:17.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6082999999999999e-05 2023-11-25T04:24:17.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:17.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043330+0000", 2023-11-25T04:24:17.835 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8556e-05 2023-11-25T04:24:17.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:17.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049303+0000", 2023-11-25T04:24:17.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059733470000000004 2023-11-25T04:24:17.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:17.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049346+0000", 2023-11-25T04:24:17.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2608999999999998e-05 2023-11-25T04:24:17.836 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:17.836 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:17.836 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:17.836 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.837 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:3 3.14 e18/16 3:287034c4:::1.00000000.inode:head v 18'4, mlcod=0'0)", 2023-11-25T04:24:17.837 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:24:17.837 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 90.697141826999996, 2023-11-25T04:24:17.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057040090000000003, 2023-11-25T04:24:17.837 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:17.838 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:17.838 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:17.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:17.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:24:17.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:17.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:17.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:24:17.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:17.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:17.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:24:17.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9799999999999999e-07 2023-11-25T04:24:17.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:17.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043785+0000", 2023-11-25T04:24:17.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0721999999999999e-05 2023-11-25T04:24:17.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:17.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043793+0000", 2023-11-25T04:24:17.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5839999999999995e-06 2023-11-25T04:24:17.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:17.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043803+0000", 2023-11-25T04:24:17.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0729999999999994e-06 2023-11-25T04:24:17.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:17.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043843+0000", 2023-11-25T04:24:17.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0747999999999997e-05 2023-11-25T04:24:17.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:17.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043856+0000", 2023-11-25T04:24:17.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2697000000000001e-05 2023-11-25T04:24:17.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:17.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049426+0000", 2023-11-25T04:24:17.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055704409999999998 2023-11-25T04:24:17.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:17.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049468+0000", 2023-11-25T04:24:17.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1446000000000003e-05 2023-11-25T04:24:17.842 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:17.842 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:17.842 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:17.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.842 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "pg_update_log_missing(3.14 epoch 18/16 rep_tid 171 entries 18'5 (0'0) error 3:2e2fa760:::200.00000001:head by mds.0.3:32 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:24:17.842 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:24:17.842 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 90.696980863999997, 2023-11-25T04:24:17.842 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005556938, 2023-11-25T04:24:17.842 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:17.842 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2023-11-25T04:24:17.842 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:17.842 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:17.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:24:17.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:17.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:17.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:24:17.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:17.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.843 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:17.843 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:24:17.843 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.03e-07 2023-11-25T04:24:17.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.844 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:17.844 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043931+0000", 2023-11-25T04:24:17.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1079999999999998e-06 2023-11-25T04:24:17.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.844 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:17.844 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043944+0000", 2023-11-25T04:24:17.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2852e-05 2023-11-25T04:24:17.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.844 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:17.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043955+0000", 2023-11-25T04:24:17.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1037e-05 2023-11-25T04:24:17.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.845 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:17.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044088+0000", 2023-11-25T04:24:17.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013269599999999999 2023-11-25T04:24:17.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:17.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049482+0000", 2023-11-25T04:24:17.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053940419999999999 2023-11-25T04:24:17.846 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:17.846 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:17.846 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:17.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.846 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:10 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'5, mlcod=0'0)", 2023-11-25T04:24:17.846 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:24:17.847 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 90.696890835999994, 2023-11-25T04:24:17.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054493099999999997, 2023-11-25T04:24:17.847 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:17.847 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:17.847 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:17.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:17.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:24:17.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:17.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:17.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:24:17.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:17.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.848 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:17.848 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:24:17.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6100000000000002e-07 2023-11-25T04:24:17.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.848 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:17.848 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044019+0000", 2023-11-25T04:24:17.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.839e-06 2023-11-25T04:24:17.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.848 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:17.849 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044026+0000", 2023-11-25T04:24:17.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5869999999999998e-06 2023-11-25T04:24:17.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:17.849 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044031+0000", 2023-11-25T04:24:17.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4909999999999996e-06 2023-11-25T04:24:17.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:17.849 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044070+0000", 2023-11-25T04:24:17.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8829999999999999e-05 2023-11-25T04:24:17.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.850 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:17.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044081+0000", 2023-11-25T04:24:17.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1447999999999999e-05 2023-11-25T04:24:17.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.850 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:17.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049406+0000", 2023-11-25T04:24:17.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053246480000000004 2023-11-25T04:24:17.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.850 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:17.850 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049464+0000", 2023-11-25T04:24:17.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8205999999999998e-05 2023-11-25T04:24:17.851 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:17.851 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:17.851 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:17.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.851 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:11 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'6, mlcod=0'0)", 2023-11-25T04:24:17.851 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:24:17.851 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 90.696689571999997, 2023-11-25T04:24:17.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052673499999999996, 2023-11-25T04:24:17.851 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:17.851 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:17.851 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:17.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.851 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:17.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:24:17.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:17.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:17.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:24:17.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:17.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:17.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:24:17.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0199999999999998e-07 2023-11-25T04:24:17.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:17.853 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044220+0000", 2023-11-25T04:24:17.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7720000000000001e-06 2023-11-25T04:24:17.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:17.853 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044226+0000", 2023-11-25T04:24:17.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8579999999999996e-06 2023-11-25T04:24:17.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:17.853 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044232+0000", 2023-11-25T04:24:17.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1700000000000002e-06 2023-11-25T04:24:17.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.854 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:17.854 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044274+0000", 2023-11-25T04:24:17.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1254999999999997e-05 2023-11-25T04:24:17.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.854 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:17.854 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:24:17.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6880999999999999e-05 2023-11-25T04:24:17.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.854 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:17.854 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049466+0000", 2023-11-25T04:24:17.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051659219999999999 2023-11-25T04:24:17.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:17.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:17.855 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:17.855 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049483+0000", 2023-11-25T04:24:17.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.719e-05 2023-11-25T04:24:17.855 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:17.855 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:17.855 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:17.855 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:17.855 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:17.855 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:17.857 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:24:17.858 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:17.855+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:17.957 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:17.955+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:17.969 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:17.969 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:24:17.969 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2023-11-25T04:24:17.969 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:17.979 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:24:18.058 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:18.057+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:18.123 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-11-25T04:24:18.131 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:18.131 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:24:18.132 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2023-11-25T04:24:18.132 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2023-11-25T04:24:18.132 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:18.141 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:24:18.160 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:18.158+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:18.250 INFO:teuthology.orchestra.run.smithi067.stderr:Error ENXIO: problem getting command descriptions from osd.0 2023-11-25T04:24:18.255 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:18.255 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2023-11-25T04:24:18.255 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2023-11-25T04:24:18.255 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2023-11-25T04:24:18.255 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.255 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:18.255 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:24:18.255 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 91.220679684999993, 2023-11-25T04:24:18.256 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01072943, 2023-11-25T04:24:18.256 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:18.256 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:18.256 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:18.256 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.256 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:18.256 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:24:18.256 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:18.256 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.256 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.256 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:18.256 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:24:18.256 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:18.256 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.257 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.257 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:18.257 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031829+0000", 2023-11-25T04:24:18.257 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.666e-06 2023-11-25T04:24:18.257 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.257 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.257 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:18.257 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031839+0000", 2023-11-25T04:24:18.257 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9490000000000008e-06 2023-11-25T04:24:18.257 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.257 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.257 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:18.257 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031857+0000", 2023-11-25T04:24:18.258 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8093000000000001e-05 2023-11-25T04:24:18.258 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.258 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.258 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:18.258 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031882+0000", 2023-11-25T04:24:18.258 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4294000000000002e-05 2023-11-25T04:24:18.258 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.259 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.259 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:18.259 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033130+0000", 2023-11-25T04:24:18.259 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001248264 2023-11-25T04:24:18.259 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.259 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.259 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:18.259 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033426+0000", 2023-11-25T04:24:18.259 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029620999999999998 2023-11-25T04:24:18.259 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.259 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.259 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:18.259 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033814+0000", 2023-11-25T04:24:18.260 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000387742 2023-11-25T04:24:18.260 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.260 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.260 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:18.260 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036793+0000", 2023-11-25T04:24:18.260 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002979598 2023-11-25T04:24:18.260 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.260 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.260 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:18.260 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042455+0000", 2023-11-25T04:24:18.260 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0056619260000000003 2023-11-25T04:24:18.260 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.260 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.261 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:18.261 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042502+0000", 2023-11-25T04:24:18.261 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7139999999999999e-05 2023-11-25T04:24:18.261 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.261 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.261 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:18.261 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042557+0000", 2023-11-25T04:24:18.261 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4548e-05 2023-11-25T04:24:18.261 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:18.261 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:18.261 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:18.261 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.261 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.261 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:18.262 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:24:18.262 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 91.219481150000007, 2023-11-25T04:24:18.262 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011505202000000001, 2023-11-25T04:24:18.262 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:18.262 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:18.262 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:18.262 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.262 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:18.262 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:24:18.262 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:18.262 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.263 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.263 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:18.263 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:24:18.263 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:18.263 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.263 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.263 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:18.263 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033027+0000", 2023-11-25T04:24:18.263 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0699999999999999e-06 2023-11-25T04:24:18.263 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.263 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.263 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:18.263 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033035+0000", 2023-11-25T04:24:18.264 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0830000000000004e-06 2023-11-25T04:24:18.264 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.264 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.264 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:18.264 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033059+0000", 2023-11-25T04:24:18.264 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3975e-05 2023-11-25T04:24:18.264 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.264 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.264 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:18.264 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033482+0000", 2023-11-25T04:24:18.264 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042253700000000001 2023-11-25T04:24:18.264 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.264 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.265 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:18.265 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034520+0000", 2023-11-25T04:24:18.265 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001037991 2023-11-25T04:24:18.265 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.265 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.265 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:18.265 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034611+0000", 2023-11-25T04:24:18.265 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1113999999999995e-05 2023-11-25T04:24:18.265 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.265 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.265 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:18.265 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034769+0000", 2023-11-25T04:24:18.265 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000157641 2023-11-25T04:24:18.266 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.266 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.266 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:18.266 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038292+0000", 2023-11-25T04:24:18.266 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035231160000000002 2023-11-25T04:24:18.266 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.266 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.266 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:18.266 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044440+0000", 2023-11-25T04:24:18.266 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0061486980000000002 2023-11-25T04:24:18.266 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.266 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.266 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:18.267 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044495+0000", 2023-11-25T04:24:18.267 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4339000000000003e-05 2023-11-25T04:24:18.267 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.267 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.267 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:18.267 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044531+0000", 2023-11-25T04:24:18.267 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6637999999999997e-05 2023-11-25T04:24:18.267 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:18.267 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:18.267 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:18.267 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.267 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.267 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:18.268 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:24:18.268 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 91.219006796000002, 2023-11-25T04:24:18.268 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0057634039999999997, 2023-11-25T04:24:18.268 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:18.268 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:18.268 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:18.268 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.268 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:18.269 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:24:18.269 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:18.269 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.269 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.269 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:18.269 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:24:18.269 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:18.269 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.269 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.269 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:18.269 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:24:18.269 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7900000000000007e-07 2023-11-25T04:24:18.269 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.270 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.270 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:18.270 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033507+0000", 2023-11-25T04:24:18.270 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8900000000000004e-06 2023-11-25T04:24:18.270 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.270 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.270 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:18.270 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033519+0000", 2023-11-25T04:24:18.270 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1756000000000001e-05 2023-11-25T04:24:18.270 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.271 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.271 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:18.271 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033527+0000", 2023-11-25T04:24:18.271 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3610000000000001e-06 2023-11-25T04:24:18.271 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.271 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.271 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:18.271 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034451+0000", 2023-11-25T04:24:18.271 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00092372400000000001 2023-11-25T04:24:18.271 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.271 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.271 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:18.272 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034809+0000", 2023-11-25T04:24:18.272 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00035757699999999998 2023-11-25T04:24:18.272 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.272 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.272 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:18.272 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034965+0000", 2023-11-25T04:24:18.272 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015638699999999999 2023-11-25T04:24:18.272 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.272 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.272 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:18.272 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038299+0000", 2023-11-25T04:24:18.272 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0033336559999999999 2023-11-25T04:24:18.272 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.272 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.273 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:18.273 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039169+0000", 2023-11-25T04:24:18.273 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00087066400000000005 2023-11-25T04:24:18.273 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.273 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.273 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:18.273 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039226+0000", 2023-11-25T04:24:18.273 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6511999999999997e-05 2023-11-25T04:24:18.273 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.273 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.273 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:18.273 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039264+0000", 2023-11-25T04:24:18.273 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7898000000000002e-05 2023-11-25T04:24:18.273 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:18.274 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:18.274 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:18.274 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.274 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.274 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:25 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:18.274 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:24:18.274 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 91.218928211999994, 2023-11-25T04:24:18.274 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096656169999999996, 2023-11-25T04:24:18.274 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:18.274 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:18.274 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:18.274 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.274 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:18.275 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:24:18.275 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:18.275 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.275 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.275 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:18.275 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:24:18.275 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:18.275 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.275 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.275 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:18.275 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033580+0000", 2023-11-25T04:24:18.275 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8199999999999999e-07 2023-11-25T04:24:18.275 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.275 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.276 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:18.276 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033585+0000", 2023-11-25T04:24:18.276 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3680000000000001e-06 2023-11-25T04:24:18.276 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.276 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.276 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:18.276 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033600+0000", 2023-11-25T04:24:18.276 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4331000000000001e-05 2023-11-25T04:24:18.276 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.276 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.276 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:18.276 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033606+0000", 2023-11-25T04:24:18.276 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5780000000000004e-06 2023-11-25T04:24:18.277 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.277 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.277 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:18.277 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035747+0000", 2023-11-25T04:24:18.277 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0021406530000000002 2023-11-25T04:24:18.277 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.277 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.277 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:18.277 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035901+0000", 2023-11-25T04:24:18.277 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000153931 2023-11-25T04:24:18.277 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.277 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.277 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:18.277 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036193+0000", 2023-11-25T04:24:18.278 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029211299999999999 2023-11-25T04:24:18.278 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.278 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.278 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:18.278 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038655+0000", 2023-11-25T04:24:18.278 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0024621640000000002 2023-11-25T04:24:18.278 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.278 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.278 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:18.279 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043130+0000", 2023-11-25T04:24:18.279 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.004474698 2023-11-25T04:24:18.279 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.279 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.279 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:18.279 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043204+0000", 2023-11-25T04:24:18.279 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4091999999999995e-05 2023-11-25T04:24:18.279 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.279 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.280 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:18.280 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043245+0000", 2023-11-25T04:24:18.280 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0907e-05 2023-11-25T04:24:18.280 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:18.280 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:18.280 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:18.280 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.280 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.280 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:26 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:18.280 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:18.280 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 91.218892388, 2023-11-25T04:24:18.280 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011086945000000001, 2023-11-25T04:24:18.280 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:18.281 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:18.281 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:18.281 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.281 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:18.281 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:18.281 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:18.281 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.281 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.281 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:18.281 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:18.281 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:18.281 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.281 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.282 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:18.282 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:18.282 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1699999999999998e-07 2023-11-25T04:24:18.282 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.282 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.282 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:18.282 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033627+0000", 2023-11-25T04:24:18.282 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.163e-05 2023-11-25T04:24:18.282 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.282 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.282 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:18.282 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033636+0000", 2023-11-25T04:24:18.282 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6859999999999995e-06 2023-11-25T04:24:18.283 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.283 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.283 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:18.283 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033642+0000", 2023-11-25T04:24:18.283 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3149999999999997e-06 2023-11-25T04:24:18.283 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.283 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.283 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:18.283 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035414+0000", 2023-11-25T04:24:18.283 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0017716380000000001 2023-11-25T04:24:18.283 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.283 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.284 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:18.284 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035539+0000", 2023-11-25T04:24:18.284 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000125477 2023-11-25T04:24:18.284 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.284 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.284 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:18.284 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035672+0000", 2023-11-25T04:24:18.284 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000133316 2023-11-25T04:24:18.284 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.284 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.284 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:18.284 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038649+0000", 2023-11-25T04:24:18.285 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029764679999999999 2023-11-25T04:24:18.285 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.285 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.285 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:18.285 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044578+0000", 2023-11-25T04:24:18.285 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0059293510000000002 2023-11-25T04:24:18.285 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.285 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.285 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:18.285 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044651+0000", 2023-11-25T04:24:18.285 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2513000000000002e-05 2023-11-25T04:24:18.285 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.285 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.286 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:18.286 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044702+0000", 2023-11-25T04:24:18.286 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1034e-05 2023-11-25T04:24:18.286 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:18.286 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:18.286 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:18.286 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.286 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.286 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:29 3.b 3:d0630e4c:::mds0_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:18.286 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:18.286 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 91.218854468000004, 2023-11-25T04:24:18.286 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010980227, 2023-11-25T04:24:18.286 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:18.287 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:18.287 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:18.287 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.287 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:18.287 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:18.287 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:18.287 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.287 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.287 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:18.287 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:18.287 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:18.287 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.287 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.288 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:18.288 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:18.288 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3499999999999996e-07 2023-11-25T04:24:18.288 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.288 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.288 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:18.288 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033658+0000", 2023-11-25T04:24:18.288 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1280000000000001e-06 2023-11-25T04:24:18.288 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.288 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.288 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:18.288 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033667+0000", 2023-11-25T04:24:18.288 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1430000000000004e-06 2023-11-25T04:24:18.288 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.289 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.289 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:18.289 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033678+0000", 2023-11-25T04:24:18.289 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0906e-05 2023-11-25T04:24:18.289 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.289 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.289 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:18.289 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034456+0000", 2023-11-25T04:24:18.290 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00077833899999999996 2023-11-25T04:24:18.290 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.290 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.290 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:18.290 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034575+0000", 2023-11-25T04:24:18.290 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011926000000000001 2023-11-25T04:24:18.290 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.290 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.290 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:18.290 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035001+0000", 2023-11-25T04:24:18.290 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042550999999999998 2023-11-25T04:24:18.290 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.290 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.291 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:18.291 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.037923+0000", 2023-11-25T04:24:18.291 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029219670000000001 2023-11-25T04:24:18.291 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.291 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.291 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:18.291 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044538+0000", 2023-11-25T04:24:18.291 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0066152349999999997 2023-11-25T04:24:18.291 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.291 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.291 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:18.291 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044594+0000", 2023-11-25T04:24:18.291 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5593000000000001e-05 2023-11-25T04:24:18.292 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.292 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.292 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:18.292 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044633+0000", 2023-11-25T04:24:18.292 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9511000000000002e-05 2023-11-25T04:24:18.292 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:18.292 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:18.292 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:18.292 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.292 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.292 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:34 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:18.292 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:24:18.292 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 91.218604354999997, 2023-11-25T04:24:18.292 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010920445000000001, 2023-11-25T04:24:18.293 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:18.293 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:18.293 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:18.293 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.293 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:18.293 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:24:18.293 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:18.293 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.293 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.293 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:18.293 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:24:18.293 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:18.293 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.294 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.294 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:18.294 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033904+0000", 2023-11-25T04:24:18.294 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6599999999999994e-07 2023-11-25T04:24:18.294 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.294 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.294 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:18.294 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033909+0000", 2023-11-25T04:24:18.294 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5639999999999996e-06 2023-11-25T04:24:18.294 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.294 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.294 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:18.294 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033923+0000", 2023-11-25T04:24:18.295 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3028e-05 2023-11-25T04:24:18.295 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.295 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.295 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:18.295 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034043+0000", 2023-11-25T04:24:18.295 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000120328 2023-11-25T04:24:18.295 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.295 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.295 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:18.295 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035576+0000", 2023-11-25T04:24:18.295 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015333720000000001 2023-11-25T04:24:18.295 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.295 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.296 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:18.296 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035699+0000", 2023-11-25T04:24:18.296 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000123001 2023-11-25T04:24:18.296 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.296 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.296 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:18.296 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044823+0000", 2023-11-25T04:24:18.296 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0091241859999999994 2023-11-25T04:24:18.296 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:18.296 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:18.296 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:18.296 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.296 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.296 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:35 3.1f 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:18.297 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:24:18.297 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 91.21730015, 2023-11-25T04:24:18.297 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096164779999999995, 2023-11-25T04:24:18.297 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:18.297 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:18.297 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:18.297 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.297 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:18.297 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:24:18.297 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:18.297 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.297 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.297 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:18.298 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:24:18.298 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:18.298 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.298 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.298 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:18.298 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035208+0000", 2023-11-25T04:24:18.298 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2699999999999998e-07 2023-11-25T04:24:18.298 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.298 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.298 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:18.298 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035213+0000", 2023-11-25T04:24:18.298 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.6070000000000001e-06 2023-11-25T04:24:18.298 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.298 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.299 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:18.299 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035350+0000", 2023-11-25T04:24:18.299 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013693599999999999 2023-11-25T04:24:18.299 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.299 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.299 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:18.299 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035358+0000", 2023-11-25T04:24:18.299 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0099999999999995e-06 2023-11-25T04:24:18.299 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.299 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.299 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:18.299 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036260+0000", 2023-11-25T04:24:18.299 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00090240100000000005 2023-11-25T04:24:18.300 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.300 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.300 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:18.300 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036517+0000", 2023-11-25T04:24:18.300 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00025736299999999998 2023-11-25T04:24:18.300 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.300 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.300 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:18.300 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044824+0000", 2023-11-25T04:24:18.300 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0083063340000000003 2023-11-25T04:24:18.300 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:18.300 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:18.300 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:18.300 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.301 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.301 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:37 3.1e 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:18.301 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:24:18.301 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 91.217133464, 2023-11-25T04:24:18.301 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0078322640000000002, 2023-11-25T04:24:18.301 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:18.301 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:18.301 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:18.301 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.301 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:18.301 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:24:18.301 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:18.301 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.302 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.302 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:18.302 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:24:18.302 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:18.302 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.302 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.302 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:18.302 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035375+0000", 2023-11-25T04:24:18.302 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9100000000000003e-07 2023-11-25T04:24:18.302 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.302 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.302 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:18.302 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035379+0000", 2023-11-25T04:24:18.303 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9430000000000002e-06 2023-11-25T04:24:18.303 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.303 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.303 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:18.303 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035388+0000", 2023-11-25T04:24:18.303 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2760000000000006e-06 2023-11-25T04:24:18.303 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.303 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.303 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:18.303 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035396+0000", 2023-11-25T04:24:18.303 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8480000000000001e-06 2023-11-25T04:24:18.303 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.303 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.303 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:18.304 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035452+0000", 2023-11-25T04:24:18.304 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5831000000000001e-05 2023-11-25T04:24:18.304 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.304 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.304 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:18.304 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035934+0000", 2023-11-25T04:24:18.304 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00048265599999999998 2023-11-25T04:24:18.304 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.304 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.304 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:18.304 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043206+0000", 2023-11-25T04:24:18.304 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.007271919 2023-11-25T04:24:18.304 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:18.305 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:18.305 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:18.305 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.305 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.305 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing(3.1a epoch 18/16 rep_tid 3 entries 18'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:36 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:24:18.305 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:24:18.305 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 91.215857877999994, 2023-11-25T04:24:18.305 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002073414, 2023-11-25T04:24:18.305 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:18.305 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:24:18.305 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:18.305 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.305 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:18.306 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:24:18.306 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:18.306 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.306 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.306 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:18.306 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:24:18.306 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:18.306 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.306 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.306 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:18.306 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036650+0000", 2023-11-25T04:24:18.306 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7300000000000002e-07 2023-11-25T04:24:18.306 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.306 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.307 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:18.307 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036659+0000", 2023-11-25T04:24:18.307 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.7499999999999992e-06 2023-11-25T04:24:18.307 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.307 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.307 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:18.307 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036673+0000", 2023-11-25T04:24:18.307 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4917e-05 2023-11-25T04:24:18.307 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.307 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.307 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:18.307 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036687+0000", 2023-11-25T04:24:18.307 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3883000000000001e-05 2023-11-25T04:24:18.308 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.308 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.308 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:18.308 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036781+0000", 2023-11-25T04:24:18.308 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3219000000000003e-05 2023-11-25T04:24:18.308 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.308 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.308 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:18.308 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038723+0000", 2023-11-25T04:24:18.308 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019422720000000001 2023-11-25T04:24:18.308 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:18.308 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:18.308 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:18.308 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.309 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.309 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4561.0:1470 4.2f 4:f6bdc9b3:::100000007dd.00000000:head [write 0~1030] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2023-11-25T04:24:18.309 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:16.209116+0000", 2023-11-25T04:24:18.309 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 2.0433913690000001, 2023-11-25T04:24:18.309 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001995031, 2023-11-25T04:24:18.309 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:18.309 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:18.309 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2023-11-25T04:24:18.309 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4561", 2023-11-25T04:24:18.309 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "192.168.0.1:0/1268434739", 2023-11-25T04:24:18.309 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1470 2023-11-25T04:24:18.309 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.309 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:18.309 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.310 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:18.310 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.209116+0000", 2023-11-25T04:24:18.310 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:18.310 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.310 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.310 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:18.310 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.209116+0000", 2023-11-25T04:24:18.310 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:18.310 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.310 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.310 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:18.310 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.209117+0000", 2023-11-25T04:24:18.311 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.61e-07 2023-11-25T04:24:18.311 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.311 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.311 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:18.311 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.209120+0000", 2023-11-25T04:24:18.311 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.76e-06 2023-11-25T04:24:18.311 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.311 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.311 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:18.311 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.209130+0000", 2023-11-25T04:24:18.311 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7990000000000005e-06 2023-11-25T04:24:18.311 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.312 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.312 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:18.312 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.209135+0000", 2023-11-25T04:24:18.312 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5970000000000002e-06 2023-11-25T04:24:18.312 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.312 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.312 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:18.312 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.209192+0000", 2023-11-25T04:24:18.312 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.7510000000000003e-05 2023-11-25T04:24:18.312 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.312 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.312 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:18.312 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.209264+0000", 2023-11-25T04:24:18.313 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1898999999999997e-05 2023-11-25T04:24:18.313 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.313 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.313 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:18.313 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.209335+0000", 2023-11-25T04:24:18.313 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0717999999999996e-05 2023-11-25T04:24:18.313 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.313 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.313 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:18.313 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.209799+0000", 2023-11-25T04:24:18.313 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00046449300000000001 2023-11-25T04:24:18.313 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.313 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.313 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:18.314 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.211061+0000", 2023-11-25T04:24:18.314 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001261535 2023-11-25T04:24:18.314 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.314 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.314 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:18.314 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.211092+0000", 2023-11-25T04:24:18.314 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1208999999999999e-05 2023-11-25T04:24:18.314 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.314 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.314 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:18.315 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.211111+0000", 2023-11-25T04:24:18.315 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8749999999999998e-05 2023-11-25T04:24:18.315 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:18.315 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:18.315 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:18.315 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.315 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.315 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4561.0:2151 4.21 4:85442bc0:::10000000a86.00000000:head [write 0~200] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2023-11-25T04:24:18.315 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:16.819976+0000", 2023-11-25T04:24:18.315 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 1.4325312649999999, 2023-11-25T04:24:18.315 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0020804280000000001, 2023-11-25T04:24:18.315 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:18.315 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:18.316 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2023-11-25T04:24:18.316 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4561", 2023-11-25T04:24:18.316 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "192.168.0.1:0/1268434739", 2023-11-25T04:24:18.316 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 2151 2023-11-25T04:24:18.316 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.316 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:18.316 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.316 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:18.316 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.819976+0000", 2023-11-25T04:24:18.316 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:18.316 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.316 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.316 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:18.316 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.819976+0000", 2023-11-25T04:24:18.317 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:18.317 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.317 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.317 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:18.317 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.819977+0000", 2023-11-25T04:24:18.317 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.8899999999999998e-07 2023-11-25T04:24:18.317 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.317 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.317 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:18.317 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.819982+0000", 2023-11-25T04:24:18.317 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5279999999999997e-06 2023-11-25T04:24:18.317 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.317 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.318 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:18.318 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.819992+0000", 2023-11-25T04:24:18.318 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0478e-05 2023-11-25T04:24:18.318 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.318 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.318 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:18.318 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.820004+0000", 2023-11-25T04:24:18.318 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1922e-05 2023-11-25T04:24:18.318 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.318 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.318 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:18.318 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.820051+0000", 2023-11-25T04:24:18.318 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7136999999999999e-05 2023-11-25T04:24:18.319 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.319 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.319 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:18.319 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.820173+0000", 2023-11-25T04:24:18.319 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000121513 2023-11-25T04:24:18.319 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.319 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.319 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:18.319 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.820298+0000", 2023-11-25T04:24:18.319 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012530899999999999 2023-11-25T04:24:18.319 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.319 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.319 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:18.320 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.820811+0000", 2023-11-25T04:24:18.320 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00051313900000000002 2023-11-25T04:24:18.320 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.320 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.320 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:18.320 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.822004+0000", 2023-11-25T04:24:18.320 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00119253 2023-11-25T04:24:18.320 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.320 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.320 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:18.320 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.822034+0000", 2023-11-25T04:24:18.320 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0759999999999997e-05 2023-11-25T04:24:18.320 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.320 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.321 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:18.321 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.822056+0000", 2023-11-25T04:24:18.321 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2123000000000001e-05 2023-11-25T04:24:18.321 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:18.321 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:18.321 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:18.321 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.321 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.321 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:118 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals,omap-rm-keys] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e44)", 2023-11-25T04:24:18.321 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:17.112648+0000", 2023-11-25T04:24:18.321 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 1.139859014, 2023-11-25T04:24:18.321 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011250779000000001, 2023-11-25T04:24:18.321 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:18.322 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:18.322 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:18.322 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.322 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:18.322 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.112648+0000", 2023-11-25T04:24:18.322 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:18.322 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.322 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.322 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:18.322 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.112648+0000", 2023-11-25T04:24:18.322 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:18.322 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.322 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.323 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:18.323 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.112649+0000", 2023-11-25T04:24:18.323 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0070000000000001e-06 2023-11-25T04:24:18.323 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.323 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.323 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:18.323 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113182+0000", 2023-11-25T04:24:18.323 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00053268100000000002 2023-11-25T04:24:18.323 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.323 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.323 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:18.323 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113196+0000", 2023-11-25T04:24:18.323 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3742e-05 2023-11-25T04:24:18.324 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.324 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.324 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:18.324 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113398+0000", 2023-11-25T04:24:18.324 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000202726 2023-11-25T04:24:18.324 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.324 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.324 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:18.324 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113447+0000", 2023-11-25T04:24:18.324 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8655000000000001e-05 2023-11-25T04:24:18.324 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.324 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.324 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:18.325 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113529+0000", 2023-11-25T04:24:18.325 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1891999999999995e-05 2023-11-25T04:24:18.325 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.325 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.325 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:18.325 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123819+0000", 2023-11-25T04:24:18.325 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010290192 2023-11-25T04:24:18.325 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.325 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.325 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:18.325 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123852+0000", 2023-11-25T04:24:18.325 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2425999999999998e-05 2023-11-25T04:24:18.325 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.325 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.326 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:18.326 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123899+0000", 2023-11-25T04:24:18.326 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7457999999999999e-05 2023-11-25T04:24:18.326 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:18.326 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:18.326 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:18.326 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.326 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.326 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4561.0:2494 4.7 4:e1292348:::10000000bde.00000000:head [write 0~3041] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2023-11-25T04:24:18.326 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:17.120454+0000", 2023-11-25T04:24:18.326 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 1.132053618, 2023-11-25T04:24:18.326 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0037327419999999998, 2023-11-25T04:24:18.326 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:18.327 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:18.327 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2023-11-25T04:24:18.327 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4561", 2023-11-25T04:24:18.327 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "192.168.0.1:0/1268434739", 2023-11-25T04:24:18.327 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 2494 2023-11-25T04:24:18.327 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.327 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:18.327 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.327 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:18.327 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.120454+0000", 2023-11-25T04:24:18.327 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:18.327 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.327 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.328 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:18.328 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.120454+0000", 2023-11-25T04:24:18.328 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:18.328 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.328 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.328 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:18.328 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.120456+0000", 2023-11-25T04:24:18.328 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0899999999999999e-06 2023-11-25T04:24:18.328 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.328 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.328 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:18.328 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.120466+0000", 2023-11-25T04:24:18.328 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0409e-05 2023-11-25T04:24:18.328 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.329 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.329 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:18.329 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.120484+0000", 2023-11-25T04:24:18.329 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7456e-05 2023-11-25T04:24:18.329 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.329 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.329 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:18.329 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.120495+0000", 2023-11-25T04:24:18.329 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1268e-05 2023-11-25T04:24:18.329 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.329 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.329 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:18.329 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.120568+0000", 2023-11-25T04:24:18.329 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3374999999999995e-05 2023-11-25T04:24:18.330 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.330 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.330 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:18.330 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.120707+0000", 2023-11-25T04:24:18.330 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013880699999999999 2023-11-25T04:24:18.330 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.330 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.330 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:18.330 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.120847+0000", 2023-11-25T04:24:18.330 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013949899999999999 2023-11-25T04:24:18.330 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.330 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.330 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:18.331 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.121627+0000", 2023-11-25T04:24:18.331 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00078049 2023-11-25T04:24:18.331 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.331 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.331 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:18.331 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.124139+0000", 2023-11-25T04:24:18.331 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0025116539999999999 2023-11-25T04:24:18.331 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.331 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.331 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:18.331 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.124168+0000", 2023-11-25T04:24:18.331 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9237000000000001e-05 2023-11-25T04:24:18.331 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.331 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.332 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:18.332 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.124186+0000", 2023-11-25T04:24:18.332 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8457e-05 2023-11-25T04:24:18.332 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:18.332 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:18.332 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:18.332 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.332 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.332 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4561.0:2497 4.37 4:ed6942ad:::10000000be1.00000000:head [write 0~1531] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2023-11-25T04:24:18.332 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:17.123256+0000", 2023-11-25T04:24:18.332 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 1.1292514950000001, 2023-11-25T04:24:18.332 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.003839617, 2023-11-25T04:24:18.332 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:18.333 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:18.333 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2023-11-25T04:24:18.333 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4561", 2023-11-25T04:24:18.333 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "192.168.0.1:0/1268434739", 2023-11-25T04:24:18.333 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 2497 2023-11-25T04:24:18.333 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.333 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:18.333 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.333 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:18.333 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123256+0000", 2023-11-25T04:24:18.333 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:18.333 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.333 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.333 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:18.334 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123256+0000", 2023-11-25T04:24:18.334 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:18.334 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.334 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.334 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:18.334 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123257+0000", 2023-11-25T04:24:18.334 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.062e-06 2023-11-25T04:24:18.334 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.334 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.334 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:18.334 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123262+0000", 2023-11-25T04:24:18.334 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3879999999999999e-06 2023-11-25T04:24:18.334 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.335 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.335 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:18.335 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123277+0000", 2023-11-25T04:24:18.335 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4416999999999999e-05 2023-11-25T04:24:18.335 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.335 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.335 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:18.335 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123285+0000", 2023-11-25T04:24:18.335 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1790000000000008e-06 2023-11-25T04:24:18.335 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.335 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.335 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:18.335 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123326+0000", 2023-11-25T04:24:18.335 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0707999999999997e-05 2023-11-25T04:24:18.336 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.336 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.336 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:18.336 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123396+0000", 2023-11-25T04:24:18.336 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0032999999999998e-05 2023-11-25T04:24:18.336 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.336 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.336 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:18.336 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123467+0000", 2023-11-25T04:24:18.336 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0937000000000002e-05 2023-11-25T04:24:18.336 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.336 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.336 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:18.336 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.124068+0000", 2023-11-25T04:24:18.337 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00060176299999999995 2023-11-25T04:24:18.337 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.337 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.337 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:18.337 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.127046+0000", 2023-11-25T04:24:18.337 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002977871 2023-11-25T04:24:18.337 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.337 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.337 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:18.337 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.127073+0000", 2023-11-25T04:24:18.337 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7118999999999999e-05 2023-11-25T04:24:18.337 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.337 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.338 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:18.338 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.127095+0000", 2023-11-25T04:24:18.338 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2140000000000001e-05 2023-11-25T04:24:18.338 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:18.338 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:18.338 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:18.338 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.338 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.338 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4561.0:2585 4.21 4:86018ef1:::10000000c39.00000000:head [write 0~267] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2023-11-25T04:24:18.338 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:17.199849+0000", 2023-11-25T04:24:18.338 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 1.0526584640000001, 2023-11-25T04:24:18.338 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0027665739999999999, 2023-11-25T04:24:18.338 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:18.338 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:18.339 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2023-11-25T04:24:18.339 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4561", 2023-11-25T04:24:18.339 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "192.168.0.1:0/1268434739", 2023-11-25T04:24:18.339 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 2585 2023-11-25T04:24:18.339 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.339 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:18.339 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.339 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:18.339 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.199849+0000", 2023-11-25T04:24:18.339 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:18.339 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.339 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.339 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:18.339 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.199849+0000", 2023-11-25T04:24:18.340 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:18.340 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.340 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.340 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:18.340 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.199851+0000", 2023-11-25T04:24:18.340 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8840000000000001e-06 2023-11-25T04:24:18.340 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.340 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.340 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:18.340 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.199860+0000", 2023-11-25T04:24:18.340 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2560000000000008e-06 2023-11-25T04:24:18.340 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.340 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.341 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:18.341 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.199887+0000", 2023-11-25T04:24:18.341 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6713e-05 2023-11-25T04:24:18.341 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.341 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.341 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:18.341 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.199900+0000", 2023-11-25T04:24:18.341 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3343000000000001e-05 2023-11-25T04:24:18.341 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.341 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.341 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:18.341 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.200030+0000", 2023-11-25T04:24:18.341 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012986700000000001 2023-11-25T04:24:18.341 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.342 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.342 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:18.342 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.200105+0000", 2023-11-25T04:24:18.342 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5438999999999994e-05 2023-11-25T04:24:18.342 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.342 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.342 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:18.342 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.200179+0000", 2023-11-25T04:24:18.342 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3800999999999994e-05 2023-11-25T04:24:18.342 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.342 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.342 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:18.342 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.200592+0000", 2023-11-25T04:24:18.342 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00041309700000000002 2023-11-25T04:24:18.343 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.343 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.343 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:18.343 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.202567+0000", 2023-11-25T04:24:18.343 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019752260000000001 2023-11-25T04:24:18.343 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.343 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.343 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:18.343 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.202597+0000", 2023-11-25T04:24:18.343 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9462e-05 2023-11-25T04:24:18.343 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.343 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.343 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:18.344 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.202615+0000", 2023-11-25T04:24:18.344 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8485999999999999e-05 2023-11-25T04:24:18.344 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:18.344 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:18.344 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:18.344 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.344 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.344 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4561.0:3113 4.21 4:86c8901d:::10000000e52.00000000:head [write 0~1674] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2023-11-25T04:24:18.344 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:17.652688+0000", 2023-11-25T04:24:18.344 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 0.59981917100000004, 2023-11-25T04:24:18.344 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001976701, 2023-11-25T04:24:18.344 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:18.344 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:18.344 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2023-11-25T04:24:18.345 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4561", 2023-11-25T04:24:18.345 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "192.168.0.1:0/1268434739", 2023-11-25T04:24:18.345 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 3113 2023-11-25T04:24:18.345 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.345 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:18.345 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.345 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:18.345 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.652688+0000", 2023-11-25T04:24:18.345 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:18.345 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.345 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.345 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:18.345 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.652688+0000", 2023-11-25T04:24:18.346 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:18.346 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.346 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.346 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:18.346 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.652689+0000", 2023-11-25T04:24:18.346 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2399999999999998e-07 2023-11-25T04:24:18.346 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.346 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.361 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:18.361 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.652692+0000", 2023-11-25T04:24:18.361 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7160000000000001e-06 2023-11-25T04:24:18.361 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.361 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.361 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:18.361 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.652702+0000", 2023-11-25T04:24:18.361 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.4280000000000002e-06 2023-11-25T04:24:18.361 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.361 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.361 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:18.362 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.652707+0000", 2023-11-25T04:24:18.362 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8779999999999997e-06 2023-11-25T04:24:18.362 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.362 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.362 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:18.362 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.652748+0000", 2023-11-25T04:24:18.362 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0808e-05 2023-11-25T04:24:18.362 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.362 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.362 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:18.362 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.652818+0000", 2023-11-25T04:24:18.362 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0764000000000002e-05 2023-11-25T04:24:18.362 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.362 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.363 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:18.363 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.652889+0000", 2023-11-25T04:24:18.363 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0537000000000006e-05 2023-11-25T04:24:18.363 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.363 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.363 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:18.363 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.653325+0000", 2023-11-25T04:24:18.363 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00043594100000000002 2023-11-25T04:24:18.363 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.363 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.363 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:18.363 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.654598+0000", 2023-11-25T04:24:18.363 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001273127 2023-11-25T04:24:18.364 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.364 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.364 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:18.364 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.654627+0000", 2023-11-25T04:24:18.364 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9088e-05 2023-11-25T04:24:18.364 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.364 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.364 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:18.364 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.654665+0000", 2023-11-25T04:24:18.364 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7790000000000002e-05 2023-11-25T04:24:18.364 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:18.364 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:18.364 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:18.364 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.365 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.365 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4561.0:3303 4.10 4:0a707b4d:::10000000f10.00000000:head [write 0~2465] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2023-11-25T04:24:18.365 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:17.817692+0000", 2023-11-25T04:24:18.365 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 0.43481504399999998, 2023-11-25T04:24:18.365 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0025840590000000001, 2023-11-25T04:24:18.365 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:18.365 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:18.365 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2023-11-25T04:24:18.365 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4561", 2023-11-25T04:24:18.365 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "192.168.0.1:0/1268434739", 2023-11-25T04:24:18.365 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 3303 2023-11-25T04:24:18.365 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.365 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:18.365 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.366 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:18.366 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.817692+0000", 2023-11-25T04:24:18.366 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:18.366 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.366 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.366 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:18.366 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.817692+0000", 2023-11-25T04:24:18.366 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:18.366 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.366 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.366 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:18.366 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.817693+0000", 2023-11-25T04:24:18.366 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2e-07 2023-11-25T04:24:18.366 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.367 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.367 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:18.367 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.817696+0000", 2023-11-25T04:24:18.367 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.207e-06 2023-11-25T04:24:18.367 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.367 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.367 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:18.367 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.817720+0000", 2023-11-25T04:24:18.367 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3575999999999999e-05 2023-11-25T04:24:18.367 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.367 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.367 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:18.367 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.817724+0000", 2023-11-25T04:24:18.368 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1219999999999997e-06 2023-11-25T04:24:18.368 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.368 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.368 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:18.368 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.817771+0000", 2023-11-25T04:24:18.368 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7193e-05 2023-11-25T04:24:18.368 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.368 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.368 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:18.368 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.817844+0000", 2023-11-25T04:24:18.369 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2735000000000001e-05 2023-11-25T04:24:18.369 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.369 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.369 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:18.369 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.817913+0000", 2023-11-25T04:24:18.369 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9249000000000001e-05 2023-11-25T04:24:18.369 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.369 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.369 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:18.369 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.818361+0000", 2023-11-25T04:24:18.369 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00044793200000000001 2023-11-25T04:24:18.370 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.370 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.370 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:18.370 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.820226+0000", 2023-11-25T04:24:18.370 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0018648440000000001 2023-11-25T04:24:18.370 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.370 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.370 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:18.370 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.820257+0000", 2023-11-25T04:24:18.370 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1188e-05 2023-11-25T04:24:18.370 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.370 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.370 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:18.371 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.820276+0000", 2023-11-25T04:24:18.371 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9493000000000001e-05 2023-11-25T04:24:18.372 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:18.372 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:18.372 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:18.372 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.372 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.372 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4561.0:3658 4.2d 4:b66d20b4:::10000001072.00000000:head [write 0~137] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2023-11-25T04:24:18.372 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:18.140586+0000", 2023-11-25T04:24:18.372 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 0.11192092300000001, 2023-11-25T04:24:18.372 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0022292969999999999, 2023-11-25T04:24:18.372 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:18.372 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:18.372 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2023-11-25T04:24:18.372 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4561", 2023-11-25T04:24:18.373 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "192.168.0.1:0/1268434739", 2023-11-25T04:24:18.373 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 3658 2023-11-25T04:24:18.373 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.373 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:18.373 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.373 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:18.373 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.140586+0000", 2023-11-25T04:24:18.373 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:18.373 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.373 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.373 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:18.373 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.140586+0000", 2023-11-25T04:24:18.374 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:18.374 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.374 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.374 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:18.374 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.140587+0000", 2023-11-25T04:24:18.374 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7599999999999997e-07 2023-11-25T04:24:18.374 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.374 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.374 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:18.374 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.140590+0000", 2023-11-25T04:24:18.375 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3189999999999999e-06 2023-11-25T04:24:18.375 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.375 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.375 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:18.375 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.140599+0000", 2023-11-25T04:24:18.375 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5609999999999999e-06 2023-11-25T04:24:18.375 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.375 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.375 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:18.375 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.140604+0000", 2023-11-25T04:24:18.375 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.775e-06 2023-11-25T04:24:18.375 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.376 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.376 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:18.376 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.140643+0000", 2023-11-25T04:24:18.376 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8943999999999998e-05 2023-11-25T04:24:18.376 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.376 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.376 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:18.376 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.140716+0000", 2023-11-25T04:24:18.376 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3638000000000002e-05 2023-11-25T04:24:18.376 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.377 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.377 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:18.377 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.140790+0000", 2023-11-25T04:24:18.377 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4164000000000003e-05 2023-11-25T04:24:18.377 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.378 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.378 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:18.378 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.141262+0000", 2023-11-25T04:24:18.378 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00047143700000000001 2023-11-25T04:24:18.378 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.378 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.378 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:18.378 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.142756+0000", 2023-11-25T04:24:18.378 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001493723 2023-11-25T04:24:18.378 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.378 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.378 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:18.378 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.142792+0000", 2023-11-25T04:24:18.379 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6455999999999998e-05 2023-11-25T04:24:18.379 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.379 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.379 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:18.379 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.142816+0000", 2023-11-25T04:24:18.379 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3604e-05 2023-11-25T04:24:18.379 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:18.379 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:18.379 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:18.379 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.380 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.380 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4561.0:3689 4.f 4:f0ecad07:::10000001091.00000000:head [write 0~886] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2023-11-25T04:24:18.380 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:18.169516+0000", 2023-11-25T04:24:18.380 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 0.082991616000000004, 2023-11-25T04:24:18.380 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0023233239999999999, 2023-11-25T04:24:18.380 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:18.380 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:18.380 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2023-11-25T04:24:18.380 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4561", 2023-11-25T04:24:18.380 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "192.168.0.1:0/1268434739", 2023-11-25T04:24:18.380 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 3689 2023-11-25T04:24:18.380 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.381 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:18.381 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.381 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:18.381 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.169516+0000", 2023-11-25T04:24:18.381 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:18.381 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.381 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.381 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:18.381 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.169516+0000", 2023-11-25T04:24:18.381 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:18.381 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.381 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.381 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:18.381 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.169516+0000", 2023-11-25T04:24:18.382 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6899999999999997e-07 2023-11-25T04:24:18.382 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.382 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.382 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:18.382 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.169520+0000", 2023-11-25T04:24:18.382 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6720000000000002e-06 2023-11-25T04:24:18.382 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.382 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.382 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:18.382 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.169530+0000", 2023-11-25T04:24:18.382 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.8039999999999996e-06 2023-11-25T04:24:18.382 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.382 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.382 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:18.383 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.169534+0000", 2023-11-25T04:24:18.383 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0940000000000001e-06 2023-11-25T04:24:18.383 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.383 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.383 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:18.383 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.169575+0000", 2023-11-25T04:24:18.383 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1090000000000001e-05 2023-11-25T04:24:18.383 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.383 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.383 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:18.383 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.169655+0000", 2023-11-25T04:24:18.384 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0006999999999995e-05 2023-11-25T04:24:18.384 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.384 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.384 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:18.384 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.169734+0000", 2023-11-25T04:24:18.384 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9419000000000007e-05 2023-11-25T04:24:18.384 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.384 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.384 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:18.384 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.170185+0000", 2023-11-25T04:24:18.384 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00045067499999999999 2023-11-25T04:24:18.384 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.384 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.384 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:18.385 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.171783+0000", 2023-11-25T04:24:18.385 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015982909999999999 2023-11-25T04:24:18.385 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.385 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.385 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:18.385 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.171820+0000", 2023-11-25T04:24:18.385 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6594999999999998e-05 2023-11-25T04:24:18.385 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:18.385 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:18.385 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:18.386 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.171839+0000", 2023-11-25T04:24:18.386 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9108000000000001e-05 2023-11-25T04:24:18.386 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:18.386 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:18.386 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:18.386 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:18.386 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:18.386 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:18.387 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:18.386+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:18.388 INFO:teuthology.orchestra.run.smithi067.stdout:osd.1: {} 2023-11-25T04:24:18.388 INFO:teuthology.orchestra.run.smithi067.stderr:osd.1: osd_enable_op_tracker = '' 2023-11-25T04:24:18.388 INFO:teuthology.orchestra.run.smithi067.stdout:osd.2: {} 2023-11-25T04:24:18.389 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:24:18.389 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-11-25T04:24:18.390 INFO:teuthology.orchestra.run.smithi067.stderr:osd.2: osd_enable_op_tracker = '' 2023-11-25T04:24:18.390 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-11-25T04:24:18.491 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:18.487+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:18.513 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:18.513 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:18.526 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:24:18.589 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:18.588+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:18.671 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 27% (1156/4217) Updating files: 28% (1181/4217) Updating files: 29% (1223/4217) Updating files: 30% (1266/4217) Updating files: 31% (1308/4217) Updating files: 32% (1350/4217) Updating files: 33% (1392/4217) Updating files: 34% (1434/4217) Updating files: 35% (1476/4217) Updating files: 36% (1519/4217) Updating files: 37% (1561/4217) Updating files: 38% (1603/4217) Updating files: 39% (1645/4217) Updating files: 40% (1687/4217) Updating files: 41% (1729/4217) Updating files: 42% (1772/4217) Updating files: 43% (1814/4217) Updating files: 44% (1856/4217) Updating files: 45% (1898/4217) Updating files: 46% (1940/4217) Updating files: 47% (1982/4217) Updating files: 48% (2025/4217) Updating files: 49% (2067/4217) Updating files: 50% (2109/4217) Updating files: 51% (2151/4217) Updating files: 52% (2193/4217) Updating files: 53% (2236/4217) Updating files: 53% (2252/4217) Updating files: 54% (2278/4217) Updating files: 55% (2320/4217) Updating files: 56% (2362/4217) Updating files: 57% (2404/4217) Updating files: 58% (2446/4217) Updating files: 59% (2489/4217) Updating files: 60% (2531/4217) Updating files: 61% (2573/4217) Updating files: 62% (2615/4217) Updating files: 63% (2657/4217) Updating files: 64% (2699/4217) Updating files: 65% (2742/4217) Updating files: 66% (2784/4217) Updating files: 67% (2826/4217) Updating files: 68% (2868/4217) Updating files: 69% (2910/4217) Updating files: 70% (2952/4217) Updating files: 71% (2995/4217) Updating files: 72% (3037/4217) Updating files: 73% (3079/4217) Updating files: 74% (3121/4217) Updating files: 75% (3163/4217) Updating files: 76% (3205/4217) Updating files: 77% (3248/4217) Updating files: 78% (3290/4217) Updating files: 79% (3332/4217) Updating files: 80% (3374/4217) Updating files: 80% (3402/4217) Updating files: 81% (3416/4217) Updating files: 82% (3458/4217) Updating files: 83% (3501/4217) Updating files: 84% (3543/4217) Updating files: 85% (3585/4217) Updating files: 86% (3627/4217) Updating files: 87% (3669/4217) Updating files: 88% (3711/4217) Updating files: 89% (3754/4217) Updating files: 90% (3796/4217) Updating files: 91% (3838/4217) Updating files: 92% (3880/4217) Updating files: 93% (3922/4217) Updating files: 94% (3964/4217) Updating files: 95% (4007/4217) Updating files: 96% (4049/4217) Updating files: 97% (4091/4217) Updating files: 98% (4133/4217) Updating files: 99% (4175/4217) Updating files: 100% (4217/4217) Updating files: 100% (4217/4217), done. 2023-11-25T04:24:18.677 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:18.677 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:18.679 INFO:tasks.workunit.client.0.smithi067.stderr:+ cd xfstests-dev 2023-11-25T04:24:18.680 INFO:tasks.workunit.client.0.smithi067.stderr:+ git checkout 12973fc04fd10d4af086901e10ffa8e48866b735 2023-11-25T04:24:18.686 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:24:18.729 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:18.728+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:18.790 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:18.790 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:18.791 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:18.790+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:18.798 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:24:18.892 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:18.891+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:18.905 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:18.905 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:18.913 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:24:18.993 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:18.991+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:19.020 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:19.020 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:19.028 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:24:19.112 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:19.111+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:19.131 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2023-11-25T04:24:19.174 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:19.174 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:19.183 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:24:19.227 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:19.226+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:19.290 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:19.290 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:19.295 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:19.294+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:19.298 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:24:19.331 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~70s 2023-11-25T04:24:19.396 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:19.395+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:19.443 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:19.443 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:19.454 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:24:19.499 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:19.498+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:19.559 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:19.559 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:19.567 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:24:19.610 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:19.609+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:19.673 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:19.673 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:19.682 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:24:19.723 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:19.721+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:19.800 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:19.799+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:19.827 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:19.827 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:19.835 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:24:19.901 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:19.900+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:19.942 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:19.943 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:19.951 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:24:20.002 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:20.000+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:20.057 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:20.057 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:20.068 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:24:20.112 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:20.111+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:20.124 INFO:teuthology.orchestra.run.smithi067.stderr:nodeep-scrub is set 2023-11-25T04:24:20.174 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:20.174 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:20.182 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:24:20.204 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:20.203+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:20.305 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:20.304+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:20.328 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:20.328 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:20.337 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:24:20.391 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-11-25T04:24:20.406 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:20.405+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:20.447 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:20.447 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:20.461 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:24:20.507 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:20.505+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:20.519 INFO:teuthology.orchestra.run.smithi067.stderr:Error ENXIO: problem getting command descriptions from osd.0 2023-11-25T04:24:20.540 INFO:teuthology.orchestra.run.smithi067.stdout:osd.1: {} 2023-11-25T04:24:20.540 INFO:teuthology.orchestra.run.smithi067.stderr:osd.1: osd_enable_op_tracker = '' 2023-11-25T04:24:20.561 INFO:teuthology.orchestra.run.smithi067.stdout:osd.2: {} 2023-11-25T04:24:20.561 INFO:teuthology.orchestra.run.smithi067.stderr:osd.2: osd_enable_op_tracker = '' 2023-11-25T04:24:20.564 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-11-25T04:24:20.565 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-11-25T04:24:20.574 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:20.574 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:24:20.574 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2023-11-25T04:24:20.574 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2023-11-25T04:24:20.574 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:20.584 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:24:20.624 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:20.623+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:20.709 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:20.708+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:20.736 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:20.736 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2023-11-25T04:24:20.737 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2023-11-25T04:24:20.737 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2023-11-25T04:24:20.737 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.737 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:20.737 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:24:20.737 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 93.701888409000006, 2023-11-25T04:24:20.737 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01072943, 2023-11-25T04:24:20.738 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:20.738 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:20.738 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:20.738 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.738 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:20.738 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:24:20.738 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:20.739 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.739 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.739 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:20.739 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:24:20.739 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:20.739 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.739 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.739 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:20.740 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031829+0000", 2023-11-25T04:24:20.740 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.666e-06 2023-11-25T04:24:20.740 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.740 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.740 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:20.740 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031839+0000", 2023-11-25T04:24:20.740 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9490000000000008e-06 2023-11-25T04:24:20.740 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.741 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.741 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:20.741 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031857+0000", 2023-11-25T04:24:20.741 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8093000000000001e-05 2023-11-25T04:24:20.741 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.741 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.741 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:20.741 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031882+0000", 2023-11-25T04:24:20.741 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4294000000000002e-05 2023-11-25T04:24:20.742 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.742 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.742 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:20.742 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033130+0000", 2023-11-25T04:24:20.742 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001248264 2023-11-25T04:24:20.742 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.742 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.742 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:20.742 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033426+0000", 2023-11-25T04:24:20.742 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029620999999999998 2023-11-25T04:24:20.742 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.743 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.743 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:20.743 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033814+0000", 2023-11-25T04:24:20.743 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000387742 2023-11-25T04:24:20.743 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.743 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.743 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:20.743 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036793+0000", 2023-11-25T04:24:20.743 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002979598 2023-11-25T04:24:20.743 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.744 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.744 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:20.744 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042455+0000", 2023-11-25T04:24:20.744 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0056619260000000003 2023-11-25T04:24:20.744 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.744 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.744 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:20.744 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042502+0000", 2023-11-25T04:24:20.744 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7139999999999999e-05 2023-11-25T04:24:20.744 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.744 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.744 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:20.745 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042557+0000", 2023-11-25T04:24:20.745 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4548e-05 2023-11-25T04:24:20.745 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:20.745 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:20.745 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:20.745 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.745 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.745 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:20.746 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:24:20.746 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 93.700689874000005, 2023-11-25T04:24:20.746 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011505202000000001, 2023-11-25T04:24:20.746 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:20.746 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:20.746 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:20.746 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.747 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:20.747 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:24:20.747 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:20.747 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.747 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.747 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:20.747 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:24:20.747 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:20.747 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.747 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.747 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:20.747 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033027+0000", 2023-11-25T04:24:20.748 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0699999999999999e-06 2023-11-25T04:24:20.748 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.748 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.748 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:20.748 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033035+0000", 2023-11-25T04:24:20.748 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0830000000000004e-06 2023-11-25T04:24:20.748 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.748 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.748 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:20.748 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033059+0000", 2023-11-25T04:24:20.748 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3975e-05 2023-11-25T04:24:20.748 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.748 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.749 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:20.749 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033482+0000", 2023-11-25T04:24:20.749 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042253700000000001 2023-11-25T04:24:20.749 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.749 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.749 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:20.749 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034520+0000", 2023-11-25T04:24:20.749 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001037991 2023-11-25T04:24:20.749 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.749 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.749 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:20.749 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034611+0000", 2023-11-25T04:24:20.750 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1113999999999995e-05 2023-11-25T04:24:20.750 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.750 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.750 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:20.750 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034769+0000", 2023-11-25T04:24:20.750 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000157641 2023-11-25T04:24:20.750 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.750 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.750 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:20.750 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038292+0000", 2023-11-25T04:24:20.750 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035231160000000002 2023-11-25T04:24:20.750 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.750 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.751 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:20.751 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044440+0000", 2023-11-25T04:24:20.751 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0061486980000000002 2023-11-25T04:24:20.751 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.751 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.751 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:20.751 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044495+0000", 2023-11-25T04:24:20.751 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4339000000000003e-05 2023-11-25T04:24:20.751 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.752 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.752 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:20.752 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044531+0000", 2023-11-25T04:24:20.752 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6637999999999997e-05 2023-11-25T04:24:20.752 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:20.752 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:20.752 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:20.752 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.752 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.752 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:20.752 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:24:20.752 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 93.70021552, 2023-11-25T04:24:20.752 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0057634039999999997, 2023-11-25T04:24:20.753 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:20.753 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:20.753 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:20.753 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.753 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:20.753 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:24:20.753 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:20.753 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.753 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.753 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:20.753 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:24:20.754 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:20.754 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.754 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.754 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:20.754 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:24:20.754 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7900000000000007e-07 2023-11-25T04:24:20.754 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.754 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.754 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:20.754 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033507+0000", 2023-11-25T04:24:20.754 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8900000000000004e-06 2023-11-25T04:24:20.754 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.754 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.755 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:20.755 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033519+0000", 2023-11-25T04:24:20.755 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1756000000000001e-05 2023-11-25T04:24:20.755 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.755 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.755 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:20.755 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033527+0000", 2023-11-25T04:24:20.755 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3610000000000001e-06 2023-11-25T04:24:20.755 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.755 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.755 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:20.756 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034451+0000", 2023-11-25T04:24:20.756 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00092372400000000001 2023-11-25T04:24:20.756 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.756 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.756 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:20.756 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034809+0000", 2023-11-25T04:24:20.756 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00035757699999999998 2023-11-25T04:24:20.756 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.756 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.756 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:20.756 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034965+0000", 2023-11-25T04:24:20.756 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015638699999999999 2023-11-25T04:24:20.756 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.757 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.757 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:20.757 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038299+0000", 2023-11-25T04:24:20.757 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0033336559999999999 2023-11-25T04:24:20.757 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.757 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.757 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:20.757 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039169+0000", 2023-11-25T04:24:20.757 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00087066400000000005 2023-11-25T04:24:20.757 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.757 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.758 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:20.758 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039226+0000", 2023-11-25T04:24:20.758 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6511999999999997e-05 2023-11-25T04:24:20.758 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.758 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.758 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:20.758 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039264+0000", 2023-11-25T04:24:20.758 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7898000000000002e-05 2023-11-25T04:24:20.758 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:20.758 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:20.758 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:20.758 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.759 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.759 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:25 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:20.759 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:24:20.759 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 93.700136936000007, 2023-11-25T04:24:20.759 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096656169999999996, 2023-11-25T04:24:20.759 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:20.759 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:20.759 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:20.759 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.759 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:20.759 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:24:20.759 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:20.759 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.760 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.760 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:20.760 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:24:20.760 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:20.760 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.760 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.760 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:20.760 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033580+0000", 2023-11-25T04:24:20.760 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8199999999999999e-07 2023-11-25T04:24:20.760 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.760 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.760 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:20.760 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033585+0000", 2023-11-25T04:24:20.761 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3680000000000001e-06 2023-11-25T04:24:20.761 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.761 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.761 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:20.761 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033600+0000", 2023-11-25T04:24:20.761 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4331000000000001e-05 2023-11-25T04:24:20.761 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.761 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.761 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:20.761 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033606+0000", 2023-11-25T04:24:20.761 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5780000000000004e-06 2023-11-25T04:24:20.761 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.762 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.762 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:20.762 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035747+0000", 2023-11-25T04:24:20.762 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0021406530000000002 2023-11-25T04:24:20.762 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.762 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.762 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:20.762 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035901+0000", 2023-11-25T04:24:20.762 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000153931 2023-11-25T04:24:20.762 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.762 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.762 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:20.763 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036193+0000", 2023-11-25T04:24:20.763 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029211299999999999 2023-11-25T04:24:20.763 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.763 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.763 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:20.763 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038655+0000", 2023-11-25T04:24:20.763 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0024621640000000002 2023-11-25T04:24:20.763 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.763 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.763 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:20.763 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043130+0000", 2023-11-25T04:24:20.763 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.004474698 2023-11-25T04:24:20.764 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.764 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.764 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:20.764 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043204+0000", 2023-11-25T04:24:20.764 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4091999999999995e-05 2023-11-25T04:24:20.764 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.764 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.764 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:20.764 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043245+0000", 2023-11-25T04:24:20.764 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0907e-05 2023-11-25T04:24:20.764 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:20.764 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:20.764 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:20.765 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.765 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.765 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:26 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:20.765 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:20.765 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 93.700101111999999, 2023-11-25T04:24:20.765 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011086945000000001, 2023-11-25T04:24:20.765 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:20.765 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:20.765 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:20.765 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.765 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:20.765 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:20.766 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:20.766 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.766 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.766 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:20.766 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:20.766 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:20.766 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.766 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.766 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:20.766 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:20.766 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1699999999999998e-07 2023-11-25T04:24:20.766 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.767 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.767 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:20.767 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033627+0000", 2023-11-25T04:24:20.767 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.163e-05 2023-11-25T04:24:20.767 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.767 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.767 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:20.767 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033636+0000", 2023-11-25T04:24:20.767 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6859999999999995e-06 2023-11-25T04:24:20.767 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.767 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.767 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:20.767 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033642+0000", 2023-11-25T04:24:20.768 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3149999999999997e-06 2023-11-25T04:24:20.768 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.768 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.768 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:20.768 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035414+0000", 2023-11-25T04:24:20.768 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0017716380000000001 2023-11-25T04:24:20.768 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.768 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.768 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:20.768 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035539+0000", 2023-11-25T04:24:20.768 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000125477 2023-11-25T04:24:20.768 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.769 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.769 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:20.769 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035672+0000", 2023-11-25T04:24:20.769 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000133316 2023-11-25T04:24:20.769 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.769 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.769 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:20.769 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038649+0000", 2023-11-25T04:24:20.769 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029764679999999999 2023-11-25T04:24:20.769 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.769 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.769 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:20.769 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044578+0000", 2023-11-25T04:24:20.770 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0059293510000000002 2023-11-25T04:24:20.770 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.770 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.770 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:20.770 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044651+0000", 2023-11-25T04:24:20.770 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2513000000000002e-05 2023-11-25T04:24:20.770 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.770 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.770 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:20.770 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044702+0000", 2023-11-25T04:24:20.770 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1034e-05 2023-11-25T04:24:20.770 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:20.771 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:20.771 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:20.771 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.771 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.771 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:29 3.b 3:d0630e4c:::mds0_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:20.771 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:20.771 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 93.700063192000002, 2023-11-25T04:24:20.771 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010980227, 2023-11-25T04:24:20.771 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:20.771 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:20.771 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:20.771 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.772 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:20.772 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:20.772 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:20.772 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.772 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.772 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:20.772 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:20.772 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:20.772 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.772 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.772 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:20.772 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:20.773 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3499999999999996e-07 2023-11-25T04:24:20.773 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.773 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.773 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:20.773 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033658+0000", 2023-11-25T04:24:20.773 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1280000000000001e-06 2023-11-25T04:24:20.773 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.773 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.773 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:20.773 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033667+0000", 2023-11-25T04:24:20.773 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1430000000000004e-06 2023-11-25T04:24:20.773 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.773 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.774 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:20.774 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033678+0000", 2023-11-25T04:24:20.774 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0906e-05 2023-11-25T04:24:20.774 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.774 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.774 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:20.774 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034456+0000", 2023-11-25T04:24:20.774 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00077833899999999996 2023-11-25T04:24:20.774 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.774 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.774 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:20.774 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034575+0000", 2023-11-25T04:24:20.775 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011926000000000001 2023-11-25T04:24:20.775 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.775 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.775 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:20.775 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035001+0000", 2023-11-25T04:24:20.775 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042550999999999998 2023-11-25T04:24:20.775 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.775 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.775 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:20.775 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.037923+0000", 2023-11-25T04:24:20.777 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029219670000000001 2023-11-25T04:24:20.777 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.777 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.777 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:20.778 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044538+0000", 2023-11-25T04:24:20.778 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0066152349999999997 2023-11-25T04:24:20.778 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.778 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.778 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:20.778 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044594+0000", 2023-11-25T04:24:20.779 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5593000000000001e-05 2023-11-25T04:24:20.779 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.779 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.779 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:20.779 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044633+0000", 2023-11-25T04:24:20.779 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9511000000000002e-05 2023-11-25T04:24:20.780 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:20.780 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:20.780 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:20.780 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.780 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.780 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:34 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:20.781 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:24:20.781 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 93.699813078999995, 2023-11-25T04:24:20.781 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010920445000000001, 2023-11-25T04:24:20.781 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:20.781 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:20.781 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:20.781 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.782 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:20.782 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:24:20.782 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:20.782 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.782 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.782 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:20.782 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:24:20.783 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:20.783 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.783 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.783 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:20.783 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033904+0000", 2023-11-25T04:24:20.783 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6599999999999994e-07 2023-11-25T04:24:20.783 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.783 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.783 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:20.783 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033909+0000", 2023-11-25T04:24:20.783 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5639999999999996e-06 2023-11-25T04:24:20.783 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.784 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.784 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:20.784 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033923+0000", 2023-11-25T04:24:20.784 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3028e-05 2023-11-25T04:24:20.784 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.784 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.784 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:20.784 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034043+0000", 2023-11-25T04:24:20.784 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000120328 2023-11-25T04:24:20.785 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.785 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.785 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:20.785 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035576+0000", 2023-11-25T04:24:20.785 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015333720000000001 2023-11-25T04:24:20.785 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.785 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.785 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:20.786 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035699+0000", 2023-11-25T04:24:20.786 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000123001 2023-11-25T04:24:20.786 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.786 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.786 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:20.786 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044823+0000", 2023-11-25T04:24:20.787 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0091241859999999994 2023-11-25T04:24:20.787 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:20.787 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:20.787 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:20.787 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.788 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.788 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:35 3.1f 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:20.788 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:24:20.788 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 93.698508873999998, 2023-11-25T04:24:20.788 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096164779999999995, 2023-11-25T04:24:20.788 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:20.789 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:20.789 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:20.789 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.789 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:20.790 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:24:20.790 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:20.790 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.790 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.790 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:20.791 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:24:20.791 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:20.791 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.791 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.791 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:20.791 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035208+0000", 2023-11-25T04:24:20.791 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2699999999999998e-07 2023-11-25T04:24:20.791 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.792 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.792 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:20.792 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035213+0000", 2023-11-25T04:24:20.792 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.6070000000000001e-06 2023-11-25T04:24:20.792 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.792 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.792 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:20.792 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035350+0000", 2023-11-25T04:24:20.793 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013693599999999999 2023-11-25T04:24:20.793 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.793 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.793 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:20.793 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035358+0000", 2023-11-25T04:24:20.793 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0099999999999995e-06 2023-11-25T04:24:20.793 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.793 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.794 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:20.794 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036260+0000", 2023-11-25T04:24:20.794 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00090240100000000005 2023-11-25T04:24:20.794 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.794 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.794 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:20.794 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036517+0000", 2023-11-25T04:24:20.795 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00025736299999999998 2023-11-25T04:24:20.795 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.795 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.795 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:20.795 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044824+0000", 2023-11-25T04:24:20.795 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0083063340000000003 2023-11-25T04:24:20.796 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:20.796 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:20.796 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:20.796 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.796 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.796 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:37 3.1e 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:20.797 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:24:20.797 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 93.698342187999998, 2023-11-25T04:24:20.797 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0078322640000000002, 2023-11-25T04:24:20.797 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:20.797 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:20.797 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:20.798 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.798 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:20.798 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:24:20.798 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:20.798 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.799 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.799 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:20.799 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:24:20.799 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:20.799 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.799 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.799 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:20.800 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035375+0000", 2023-11-25T04:24:20.800 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9100000000000003e-07 2023-11-25T04:24:20.800 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.800 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.800 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:20.800 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035379+0000", 2023-11-25T04:24:20.800 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9430000000000002e-06 2023-11-25T04:24:20.800 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.801 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.801 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:20.801 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035388+0000", 2023-11-25T04:24:20.801 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2760000000000006e-06 2023-11-25T04:24:20.801 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.801 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.801 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:20.802 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035396+0000", 2023-11-25T04:24:20.802 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8480000000000001e-06 2023-11-25T04:24:20.802 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.802 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.802 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:20.802 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035452+0000", 2023-11-25T04:24:20.803 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5831000000000001e-05 2023-11-25T04:24:20.803 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.803 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.803 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:20.803 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035934+0000", 2023-11-25T04:24:20.803 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00048265599999999998 2023-11-25T04:24:20.803 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.804 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.804 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:20.804 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043206+0000", 2023-11-25T04:24:20.804 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.007271919 2023-11-25T04:24:20.804 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:20.804 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:20.804 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:20.805 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.805 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.805 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing(3.1a epoch 18/16 rep_tid 3 entries 18'1 (0'0) error 3:5e41ab69:::200.00000005:head by mds.0.3:36 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:24:20.805 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:24:20.805 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 93.697066602000007, 2023-11-25T04:24:20.805 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002073414, 2023-11-25T04:24:20.806 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:20.806 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2023-11-25T04:24:20.806 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:20.806 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.806 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:20.806 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:24:20.806 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:20.807 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.807 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.807 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:20.807 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036649+0000", 2023-11-25T04:24:20.807 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:20.807 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.807 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.807 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:20.808 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036650+0000", 2023-11-25T04:24:20.808 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7300000000000002e-07 2023-11-25T04:24:20.808 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.808 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.808 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:20.808 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036659+0000", 2023-11-25T04:24:20.808 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.7499999999999992e-06 2023-11-25T04:24:20.808 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.809 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.809 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:20.809 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036673+0000", 2023-11-25T04:24:20.809 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4917e-05 2023-11-25T04:24:20.809 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.809 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.809 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:20.809 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036687+0000", 2023-11-25T04:24:20.810 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3883000000000001e-05 2023-11-25T04:24:20.810 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.810 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.810 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:20.810 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036781+0000", 2023-11-25T04:24:20.810 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3219000000000003e-05 2023-11-25T04:24:20.810 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.810 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.810 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:20.811 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038723+0000", 2023-11-25T04:24:20.811 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019422720000000001 2023-11-25T04:24:20.811 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:20.811 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:20.811 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:20.811 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.811 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.811 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4561.0:1470 4.2f 4:f6bdc9b3:::100000007dd.00000000:head [write 0~1030] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2023-11-25T04:24:20.812 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:16.209116+0000", 2023-11-25T04:24:20.812 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.5246000930000001, 2023-11-25T04:24:20.812 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001995031, 2023-11-25T04:24:20.812 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:20.812 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:20.812 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2023-11-25T04:24:20.812 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4561", 2023-11-25T04:24:20.812 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "192.168.0.1:0/1268434739", 2023-11-25T04:24:20.813 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1470 2023-11-25T04:24:20.813 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.813 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:20.813 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.813 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:20.813 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.209116+0000", 2023-11-25T04:24:20.813 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:20.813 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.814 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.814 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:20.814 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.209116+0000", 2023-11-25T04:24:20.814 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:20.814 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.814 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.814 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:20.814 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.209117+0000", 2023-11-25T04:24:20.815 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.61e-07 2023-11-25T04:24:20.815 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.815 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.815 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:20.815 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.209120+0000", 2023-11-25T04:24:20.815 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.76e-06 2023-11-25T04:24:20.815 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.816 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.816 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:20.816 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.209130+0000", 2023-11-25T04:24:20.816 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7990000000000005e-06 2023-11-25T04:24:20.816 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.816 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.816 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:20.816 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.209135+0000", 2023-11-25T04:24:20.816 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5970000000000002e-06 2023-11-25T04:24:20.816 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.816 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.816 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:20.817 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.209192+0000", 2023-11-25T04:24:20.817 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.7510000000000003e-05 2023-11-25T04:24:20.817 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.817 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.817 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:20.817 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.209264+0000", 2023-11-25T04:24:20.817 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1898999999999997e-05 2023-11-25T04:24:20.817 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.817 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.817 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:20.817 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.209335+0000", 2023-11-25T04:24:20.817 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0717999999999996e-05 2023-11-25T04:24:20.817 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.818 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.818 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:20.818 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.209799+0000", 2023-11-25T04:24:20.818 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00046449300000000001 2023-11-25T04:24:20.818 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.818 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.818 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:20.818 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.211061+0000", 2023-11-25T04:24:20.818 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001261535 2023-11-25T04:24:20.818 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.818 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.818 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:20.819 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.211092+0000", 2023-11-25T04:24:20.819 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1208999999999999e-05 2023-11-25T04:24:20.819 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.819 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.819 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:20.819 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.211111+0000", 2023-11-25T04:24:20.819 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8749999999999998e-05 2023-11-25T04:24:20.819 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:20.819 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:20.819 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:20.819 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.819 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.819 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4561.0:2151 4.21 4:85442bc0:::10000000a86.00000000:head [write 0~200] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2023-11-25T04:24:20.820 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:16.819976+0000", 2023-11-25T04:24:20.820 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.9137399889999998, 2023-11-25T04:24:20.820 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0020804280000000001, 2023-11-25T04:24:20.820 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:20.820 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:20.820 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2023-11-25T04:24:20.820 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4561", 2023-11-25T04:24:20.820 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "192.168.0.1:0/1268434739", 2023-11-25T04:24:20.820 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 2151 2023-11-25T04:24:20.820 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.820 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:20.820 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.821 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:20.821 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.819976+0000", 2023-11-25T04:24:20.821 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:20.821 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.821 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.821 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:20.821 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.819976+0000", 2023-11-25T04:24:20.821 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:20.821 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.821 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.821 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:20.821 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.819977+0000", 2023-11-25T04:24:20.822 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.8899999999999998e-07 2023-11-25T04:24:20.822 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.822 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.822 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:20.822 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.819982+0000", 2023-11-25T04:24:20.822 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5279999999999997e-06 2023-11-25T04:24:20.822 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.822 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.822 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:20.822 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.819992+0000", 2023-11-25T04:24:20.822 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0478e-05 2023-11-25T04:24:20.822 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.823 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.823 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:20.823 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.820004+0000", 2023-11-25T04:24:20.823 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1922e-05 2023-11-25T04:24:20.823 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.823 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.823 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:20.823 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.820051+0000", 2023-11-25T04:24:20.823 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7136999999999999e-05 2023-11-25T04:24:20.823 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.823 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.823 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:20.823 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.820173+0000", 2023-11-25T04:24:20.824 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000121513 2023-11-25T04:24:20.824 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.824 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.824 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:20.824 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.820298+0000", 2023-11-25T04:24:20.824 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012530899999999999 2023-11-25T04:24:20.824 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.824 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.824 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:20.824 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.820811+0000", 2023-11-25T04:24:20.824 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00051313900000000002 2023-11-25T04:24:20.824 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.825 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.825 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:20.825 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.822004+0000", 2023-11-25T04:24:20.825 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00119253 2023-11-25T04:24:20.825 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.825 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.825 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:20.825 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.822034+0000", 2023-11-25T04:24:20.825 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0759999999999997e-05 2023-11-25T04:24:20.825 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.825 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.825 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:20.826 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.822056+0000", 2023-11-25T04:24:20.826 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2123000000000001e-05 2023-11-25T04:24:20.826 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:20.826 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:20.826 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:20.826 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.826 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.826 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:118 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals,omap-rm-keys] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e44)", 2023-11-25T04:24:20.826 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:17.112648+0000", 2023-11-25T04:24:20.826 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.6210677379999998, 2023-11-25T04:24:20.826 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011250779000000001, 2023-11-25T04:24:20.826 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:20.826 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:20.827 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:20.827 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.827 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:20.827 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.112648+0000", 2023-11-25T04:24:20.827 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:20.827 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.827 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.827 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:20.827 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.112648+0000", 2023-11-25T04:24:20.827 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:20.827 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.827 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.828 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:20.828 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.112649+0000", 2023-11-25T04:24:20.828 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0070000000000001e-06 2023-11-25T04:24:20.828 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.828 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.828 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:20.828 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113182+0000", 2023-11-25T04:24:20.828 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00053268100000000002 2023-11-25T04:24:20.828 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.828 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.828 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:20.828 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113196+0000", 2023-11-25T04:24:20.828 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3742e-05 2023-11-25T04:24:20.829 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.829 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.829 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:20.829 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113398+0000", 2023-11-25T04:24:20.829 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000202726 2023-11-25T04:24:20.829 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.829 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.829 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:20.829 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113447+0000", 2023-11-25T04:24:20.829 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8655000000000001e-05 2023-11-25T04:24:20.829 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.829 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.829 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:20.830 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113529+0000", 2023-11-25T04:24:20.830 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1891999999999995e-05 2023-11-25T04:24:20.830 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.830 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.830 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:20.830 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123819+0000", 2023-11-25T04:24:20.830 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010290192 2023-11-25T04:24:20.830 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.830 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.830 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:20.830 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123852+0000", 2023-11-25T04:24:20.830 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2425999999999998e-05 2023-11-25T04:24:20.830 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.831 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.831 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:20.831 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123899+0000", 2023-11-25T04:24:20.831 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7457999999999999e-05 2023-11-25T04:24:20.831 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:20.831 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:20.831 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:20.831 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.831 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.831 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4561.0:2494 4.7 4:e1292348:::10000000bde.00000000:head [write 0~3041] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2023-11-25T04:24:20.831 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:17.120454+0000", 2023-11-25T04:24:20.831 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.6132623420000001, 2023-11-25T04:24:20.831 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0037327419999999998, 2023-11-25T04:24:20.832 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:20.832 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:20.832 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2023-11-25T04:24:20.832 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4561", 2023-11-25T04:24:20.832 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "192.168.0.1:0/1268434739", 2023-11-25T04:24:20.832 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 2494 2023-11-25T04:24:20.832 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.832 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:20.832 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.832 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:20.832 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.120454+0000", 2023-11-25T04:24:20.833 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:20.833 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.833 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.833 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:20.833 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.120454+0000", 2023-11-25T04:24:20.833 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:20.833 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.833 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.833 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:20.834 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.120456+0000", 2023-11-25T04:24:20.834 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0899999999999999e-06 2023-11-25T04:24:20.834 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.834 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.834 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:20.834 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.120466+0000", 2023-11-25T04:24:20.834 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0409e-05 2023-11-25T04:24:20.834 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.834 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.834 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:20.834 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.120484+0000", 2023-11-25T04:24:20.834 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7456e-05 2023-11-25T04:24:20.834 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.835 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.835 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:20.835 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.120495+0000", 2023-11-25T04:24:20.835 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1268e-05 2023-11-25T04:24:20.835 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.835 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.835 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:20.835 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.120568+0000", 2023-11-25T04:24:20.835 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3374999999999995e-05 2023-11-25T04:24:20.835 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.835 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.835 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:20.835 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.120707+0000", 2023-11-25T04:24:20.836 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013880699999999999 2023-11-25T04:24:20.836 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.836 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.836 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:20.836 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.120847+0000", 2023-11-25T04:24:20.836 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013949899999999999 2023-11-25T04:24:20.836 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.836 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.836 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:20.836 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.121627+0000", 2023-11-25T04:24:20.836 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00078049 2023-11-25T04:24:20.836 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.837 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.837 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:20.837 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.124139+0000", 2023-11-25T04:24:20.837 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0025116539999999999 2023-11-25T04:24:20.837 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.837 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.837 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:20.837 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.124168+0000", 2023-11-25T04:24:20.837 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9237000000000001e-05 2023-11-25T04:24:20.837 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.837 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.837 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:20.837 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.124186+0000", 2023-11-25T04:24:20.838 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8457e-05 2023-11-25T04:24:20.838 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:20.838 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:20.838 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:20.838 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.838 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.838 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4561.0:2497 4.37 4:ed6942ad:::10000000be1.00000000:head [write 0~1531] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2023-11-25T04:24:20.838 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:17.123256+0000", 2023-11-25T04:24:20.838 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.6104602190000001, 2023-11-25T04:24:20.838 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.003839617, 2023-11-25T04:24:20.838 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:20.838 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:20.838 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2023-11-25T04:24:20.839 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4561", 2023-11-25T04:24:20.839 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "192.168.0.1:0/1268434739", 2023-11-25T04:24:20.839 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 2497 2023-11-25T04:24:20.839 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.839 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:20.839 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.839 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:20.839 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123256+0000", 2023-11-25T04:24:20.839 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:20.839 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.839 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.839 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:20.839 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123256+0000", 2023-11-25T04:24:20.839 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:20.840 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.840 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.840 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:20.840 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123257+0000", 2023-11-25T04:24:20.840 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.062e-06 2023-11-25T04:24:20.840 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.840 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.840 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:20.840 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123262+0000", 2023-11-25T04:24:20.840 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3879999999999999e-06 2023-11-25T04:24:20.840 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.840 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.841 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:20.841 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123277+0000", 2023-11-25T04:24:20.841 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4416999999999999e-05 2023-11-25T04:24:20.841 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.841 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.841 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:20.841 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123285+0000", 2023-11-25T04:24:20.841 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1790000000000008e-06 2023-11-25T04:24:20.841 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.841 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.841 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:20.841 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123326+0000", 2023-11-25T04:24:20.841 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0707999999999997e-05 2023-11-25T04:24:20.842 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.842 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.842 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:20.842 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123396+0000", 2023-11-25T04:24:20.842 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0032999999999998e-05 2023-11-25T04:24:20.842 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.842 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.842 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:20.842 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123467+0000", 2023-11-25T04:24:20.842 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0937000000000002e-05 2023-11-25T04:24:20.842 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.842 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.842 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:20.843 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.124068+0000", 2023-11-25T04:24:20.843 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00060176299999999995 2023-11-25T04:24:20.843 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.843 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.843 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:20.843 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.127046+0000", 2023-11-25T04:24:20.843 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002977871 2023-11-25T04:24:20.843 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.843 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.843 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:20.843 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.127073+0000", 2023-11-25T04:24:20.843 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7118999999999999e-05 2023-11-25T04:24:20.843 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.844 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.844 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:20.844 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.127095+0000", 2023-11-25T04:24:20.844 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2140000000000001e-05 2023-11-25T04:24:20.844 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:20.844 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:20.844 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:20.844 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.844 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.844 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4561.0:2585 4.21 4:86018ef1:::10000000c39.00000000:head [write 0~267] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2023-11-25T04:24:20.844 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:17.199849+0000", 2023-11-25T04:24:20.844 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.5338671879999999, 2023-11-25T04:24:20.844 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0027665739999999999, 2023-11-25T04:24:20.844 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:20.845 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:20.845 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2023-11-25T04:24:20.845 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4561", 2023-11-25T04:24:20.845 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "192.168.0.1:0/1268434739", 2023-11-25T04:24:20.845 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 2585 2023-11-25T04:24:20.845 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.845 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:20.845 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.845 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:20.845 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.199849+0000", 2023-11-25T04:24:20.845 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:20.845 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.845 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.846 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:20.846 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.199849+0000", 2023-11-25T04:24:20.846 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:20.846 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.846 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.846 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:20.846 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.199851+0000", 2023-11-25T04:24:20.846 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8840000000000001e-06 2023-11-25T04:24:20.846 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.846 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.846 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:20.846 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.199860+0000", 2023-11-25T04:24:20.846 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2560000000000008e-06 2023-11-25T04:24:20.847 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.847 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.847 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:20.847 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.199887+0000", 2023-11-25T04:24:20.847 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6713e-05 2023-11-25T04:24:20.847 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.847 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.847 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:20.847 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.199900+0000", 2023-11-25T04:24:20.847 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3343000000000001e-05 2023-11-25T04:24:20.847 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.847 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.847 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:20.848 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.200030+0000", 2023-11-25T04:24:20.848 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012986700000000001 2023-11-25T04:24:20.848 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.848 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.848 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:20.848 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.200105+0000", 2023-11-25T04:24:20.848 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5438999999999994e-05 2023-11-25T04:24:20.848 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.848 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.848 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:20.848 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.200179+0000", 2023-11-25T04:24:20.848 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3800999999999994e-05 2023-11-25T04:24:20.848 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.849 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.849 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:20.849 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.200592+0000", 2023-11-25T04:24:20.849 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00041309700000000002 2023-11-25T04:24:20.849 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.849 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.849 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:20.849 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.202567+0000", 2023-11-25T04:24:20.849 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019752260000000001 2023-11-25T04:24:20.849 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.849 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.849 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:20.849 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.202597+0000", 2023-11-25T04:24:20.849 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9462e-05 2023-11-25T04:24:20.850 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.850 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.850 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:20.850 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.202615+0000", 2023-11-25T04:24:20.850 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8485999999999999e-05 2023-11-25T04:24:20.850 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:20.850 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:20.850 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:20.850 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.850 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.850 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4561.0:3113 4.21 4:86c8901d:::10000000e52.00000000:head [write 0~1674] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2023-11-25T04:24:20.850 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:17.652688+0000", 2023-11-25T04:24:20.850 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.0810278950000001, 2023-11-25T04:24:20.851 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001976701, 2023-11-25T04:24:20.851 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:20.851 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:20.851 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2023-11-25T04:24:20.851 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4561", 2023-11-25T04:24:20.851 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "192.168.0.1:0/1268434739", 2023-11-25T04:24:20.851 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 3113 2023-11-25T04:24:20.851 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.851 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:20.851 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.851 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:20.851 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.652688+0000", 2023-11-25T04:24:20.851 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:20.852 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.852 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.852 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:20.852 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.652688+0000", 2023-11-25T04:24:20.852 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:20.852 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.852 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.852 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:20.852 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.652689+0000", 2023-11-25T04:24:20.852 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2399999999999998e-07 2023-11-25T04:24:20.852 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.852 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.852 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:20.853 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.652692+0000", 2023-11-25T04:24:20.853 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7160000000000001e-06 2023-11-25T04:24:20.853 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.853 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.853 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:20.853 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.652702+0000", 2023-11-25T04:24:20.853 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.4280000000000002e-06 2023-11-25T04:24:20.853 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.853 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.853 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:20.853 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.652707+0000", 2023-11-25T04:24:20.853 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8779999999999997e-06 2023-11-25T04:24:20.853 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.853 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.854 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:20.854 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.652748+0000", 2023-11-25T04:24:20.854 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0808e-05 2023-11-25T04:24:20.854 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.854 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.854 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:20.854 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.652818+0000", 2023-11-25T04:24:20.854 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0764000000000002e-05 2023-11-25T04:24:20.854 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.854 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.854 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:20.854 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.652889+0000", 2023-11-25T04:24:20.854 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0537000000000006e-05 2023-11-25T04:24:20.854 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.855 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.855 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:20.855 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.653325+0000", 2023-11-25T04:24:20.855 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00043594100000000002 2023-11-25T04:24:20.855 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.855 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.855 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:20.855 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.654598+0000", 2023-11-25T04:24:20.855 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001273127 2023-11-25T04:24:20.855 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.855 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.855 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:20.855 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.654627+0000", 2023-11-25T04:24:20.856 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9088e-05 2023-11-25T04:24:20.856 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.856 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.856 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:20.856 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.654665+0000", 2023-11-25T04:24:20.856 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7790000000000002e-05 2023-11-25T04:24:20.856 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:20.856 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:20.856 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:20.856 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.856 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.856 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4561.0:3303 4.10 4:0a707b4d:::10000000f10.00000000:head [write 0~2465] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2023-11-25T04:24:20.856 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:17.817692+0000", 2023-11-25T04:24:20.857 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 2.9160237680000001, 2023-11-25T04:24:20.857 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0025840590000000001, 2023-11-25T04:24:20.857 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:20.857 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:20.857 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2023-11-25T04:24:20.857 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4561", 2023-11-25T04:24:20.857 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "192.168.0.1:0/1268434739", 2023-11-25T04:24:20.857 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 3303 2023-11-25T04:24:20.857 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.857 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:20.857 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.857 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:20.857 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.817692+0000", 2023-11-25T04:24:20.858 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:20.858 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.858 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.858 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:20.858 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.817692+0000", 2023-11-25T04:24:20.858 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:20.858 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.858 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.858 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:20.858 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.817693+0000", 2023-11-25T04:24:20.858 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2e-07 2023-11-25T04:24:20.858 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.858 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.859 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:20.859 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.817696+0000", 2023-11-25T04:24:20.859 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.207e-06 2023-11-25T04:24:20.859 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.859 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.859 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:20.859 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.817720+0000", 2023-11-25T04:24:20.859 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3575999999999999e-05 2023-11-25T04:24:20.859 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.859 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.859 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:20.859 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.817724+0000", 2023-11-25T04:24:20.859 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1219999999999997e-06 2023-11-25T04:24:20.859 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.860 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.860 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:20.860 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.817771+0000", 2023-11-25T04:24:20.860 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7193e-05 2023-11-25T04:24:20.860 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.860 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.860 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:20.860 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.817844+0000", 2023-11-25T04:24:20.860 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2735000000000001e-05 2023-11-25T04:24:20.860 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.860 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.860 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:20.860 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.817913+0000", 2023-11-25T04:24:20.860 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9249000000000001e-05 2023-11-25T04:24:20.861 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.861 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.861 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:20.861 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.818361+0000", 2023-11-25T04:24:20.861 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00044793200000000001 2023-11-25T04:24:20.861 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.861 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.861 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:20.861 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.820226+0000", 2023-11-25T04:24:20.861 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0018648440000000001 2023-11-25T04:24:20.861 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.861 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.861 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:20.862 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.820257+0000", 2023-11-25T04:24:20.862 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1188e-05 2023-11-25T04:24:20.862 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.862 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.862 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:20.862 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.820276+0000", 2023-11-25T04:24:20.862 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9493000000000001e-05 2023-11-25T04:24:20.862 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:20.862 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:20.862 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:20.862 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.862 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.862 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4561.0:3658 4.2d 4:b66d20b4:::10000001072.00000000:head [write 0~137] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2023-11-25T04:24:20.862 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:18.140586+0000", 2023-11-25T04:24:20.863 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 2.593129647, 2023-11-25T04:24:20.863 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0022292969999999999, 2023-11-25T04:24:20.863 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:20.863 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:20.863 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2023-11-25T04:24:20.863 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4561", 2023-11-25T04:24:20.863 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "192.168.0.1:0/1268434739", 2023-11-25T04:24:20.863 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 3658 2023-11-25T04:24:20.863 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.863 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:20.863 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.863 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:20.863 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.140586+0000", 2023-11-25T04:24:20.863 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:20.864 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.864 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.864 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:20.864 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.140586+0000", 2023-11-25T04:24:20.864 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:20.864 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.864 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.864 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:20.864 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.140587+0000", 2023-11-25T04:24:20.864 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7599999999999997e-07 2023-11-25T04:24:20.864 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.864 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.864 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:20.865 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.140590+0000", 2023-11-25T04:24:20.865 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3189999999999999e-06 2023-11-25T04:24:20.865 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.865 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.865 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:20.865 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.140599+0000", 2023-11-25T04:24:20.865 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5609999999999999e-06 2023-11-25T04:24:20.865 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.865 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.865 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:20.865 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.140604+0000", 2023-11-25T04:24:20.865 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.775e-06 2023-11-25T04:24:20.865 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.865 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.866 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:20.866 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.140643+0000", 2023-11-25T04:24:20.866 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8943999999999998e-05 2023-11-25T04:24:20.866 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.866 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.866 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:20.866 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.140716+0000", 2023-11-25T04:24:20.866 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3638000000000002e-05 2023-11-25T04:24:20.866 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.866 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.866 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:20.866 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.140790+0000", 2023-11-25T04:24:20.866 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4164000000000003e-05 2023-11-25T04:24:20.867 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.867 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.867 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:20.867 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.141262+0000", 2023-11-25T04:24:20.867 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00047143700000000001 2023-11-25T04:24:20.867 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.867 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.867 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:20.867 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.142756+0000", 2023-11-25T04:24:20.867 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001493723 2023-11-25T04:24:20.867 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.867 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.867 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:20.867 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.142792+0000", 2023-11-25T04:24:20.868 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6455999999999998e-05 2023-11-25T04:24:20.868 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.868 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.868 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:20.868 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.142816+0000", 2023-11-25T04:24:20.868 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3604e-05 2023-11-25T04:24:20.868 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:20.868 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:20.868 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:20.868 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.868 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.868 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4561.0:3689 4.f 4:f0ecad07:::10000001091.00000000:head [write 0~886] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2023-11-25T04:24:20.868 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:18.169516+0000", 2023-11-25T04:24:20.869 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 2.5642003400000002, 2023-11-25T04:24:20.869 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0023233239999999999, 2023-11-25T04:24:20.869 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:20.869 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:20.869 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2023-11-25T04:24:20.869 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4561", 2023-11-25T04:24:20.869 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "192.168.0.1:0/1268434739", 2023-11-25T04:24:20.869 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 3689 2023-11-25T04:24:20.869 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.869 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:20.869 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.869 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:20.869 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.169516+0000", 2023-11-25T04:24:20.869 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:20.870 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.870 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.870 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:20.870 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.169516+0000", 2023-11-25T04:24:20.870 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:20.870 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.870 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.870 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:20.870 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.169516+0000", 2023-11-25T04:24:20.870 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6899999999999997e-07 2023-11-25T04:24:20.870 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.870 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.870 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:20.870 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.169520+0000", 2023-11-25T04:24:20.871 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6720000000000002e-06 2023-11-25T04:24:20.871 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.871 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.871 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:20.871 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.169530+0000", 2023-11-25T04:24:20.871 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.8039999999999996e-06 2023-11-25T04:24:20.871 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.871 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.871 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:20.871 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.169534+0000", 2023-11-25T04:24:20.871 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0940000000000001e-06 2023-11-25T04:24:20.871 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.871 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.871 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:20.872 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.169575+0000", 2023-11-25T04:24:20.872 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1090000000000001e-05 2023-11-25T04:24:20.872 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.872 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.872 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:20.872 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.169655+0000", 2023-11-25T04:24:20.872 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0006999999999995e-05 2023-11-25T04:24:20.872 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.872 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.872 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:20.872 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.169734+0000", 2023-11-25T04:24:20.872 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9419000000000007e-05 2023-11-25T04:24:20.872 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.873 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.873 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:20.873 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.170185+0000", 2023-11-25T04:24:20.873 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00045067499999999999 2023-11-25T04:24:20.873 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.873 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.873 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:20.873 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.171783+0000", 2023-11-25T04:24:20.873 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015982909999999999 2023-11-25T04:24:20.873 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.873 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.873 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:20.873 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.171820+0000", 2023-11-25T04:24:20.874 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6594999999999998e-05 2023-11-25T04:24:20.874 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:20.874 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:20.874 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:20.874 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.171839+0000", 2023-11-25T04:24:20.874 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9108000000000001e-05 2023-11-25T04:24:20.874 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:20.874 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:20.874 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:20.874 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:20.874 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:20.874 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:20.876 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:20.875+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:20.876 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:24:20.976 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:20.975+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:20.985 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:20.986 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:24:20.986 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2023-11-25T04:24:20.986 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:20.995 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:24:21.078 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:21.076+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:21.143 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:21.143 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:24:21.143 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2023-11-25T04:24:21.144 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2023-11-25T04:24:21.144 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:21.153 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:24:21.179 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:21.177+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:21.266 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:21.267 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2023-11-25T04:24:21.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2023-11-25T04:24:21.267 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2023-11-25T04:24:21.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.267 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:155 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'109, mlcod=11'105)", 2023-11-25T04:24:21.267 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:24:21.267 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 103.47936817900001, 2023-11-25T04:24:21.267 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061719019999999999, 2023-11-25T04:24:21.267 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:21.267 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:21.267 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:21.267 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:21.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:24:21.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:21.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:21.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784883+0000", 2023-11-25T04:24:21.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:21.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.268 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.268 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:21.268 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784884+0000", 2023-11-25T04:24:21.268 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3099999999999999e-07 2023-11-25T04:24:21.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:21.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784942+0000", 2023-11-25T04:24:21.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8570000000000003e-05 2023-11-25T04:24:21.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:21.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784949+0000", 2023-11-25T04:24:21.269 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0310000000000001e-06 2023-11-25T04:24:21.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:21.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784953+0000", 2023-11-25T04:24:21.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6320000000000001e-06 2023-11-25T04:24:21.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:21.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.784993+0000", 2023-11-25T04:24:21.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9737000000000003e-05 2023-11-25T04:24:21.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:21.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785011+0000", 2023-11-25T04:24:21.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8349999999999999e-05 2023-11-25T04:24:21.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:21.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791024+0000", 2023-11-25T04:24:21.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.006012997 2023-11-25T04:24:21.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.271 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:21.271 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791055+0000", 2023-11-25T04:24:21.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1254000000000003e-05 2023-11-25T04:24:21.271 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:21.271 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:21.271 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:21.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.272 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:156 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'110, mlcod=11'105)", 2023-11-25T04:24:21.272 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:24:21.272 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 103.47878213, 2023-11-25T04:24:21.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005616116, 2023-11-25T04:24:21.272 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:21.272 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:21.272 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:21.272 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:21.272 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:24:21.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:21.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:21.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785469+0000", 2023-11-25T04:24:21.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:21.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:21.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785470+0000", 2023-11-25T04:24:21.273 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0699999999999998e-07 2023-11-25T04:24:21.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:21.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785474+0000", 2023-11-25T04:24:21.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.048e-06 2023-11-25T04:24:21.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:21.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785480+0000", 2023-11-25T04:24:21.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7729999999999996e-06 2023-11-25T04:24:21.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.274 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:21.274 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.785483+0000", 2023-11-25T04:24:21.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1970000000000001e-06 2023-11-25T04:24:21.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:21.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787936+0000", 2023-11-25T04:24:21.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024529040000000001 2023-11-25T04:24:21.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:21.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.787947+0000", 2023-11-25T04:24:21.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1669e-05 2023-11-25T04:24:21.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.275 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:21.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791057+0000", 2023-11-25T04:24:21.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031095200000000002 2023-11-25T04:24:21.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:21.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791086+0000", 2023-11-25T04:24:21.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8697999999999999e-05 2023-11-25T04:24:21.276 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:21.276 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:21.276 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:21.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.276 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(client.4147.0:157 1.0 e11/10 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 11'111, mlcod=11'106)", 2023-11-25T04:24:21.276 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:24:21.277 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 103.47818957200001, 2023-11-25T04:24:21.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0050590280000000001, 2023-11-25T04:24:21.277 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:21.277 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:21.277 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:21.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:21.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:24:21.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:21.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:21.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786062+0000", 2023-11-25T04:24:21.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:21.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:21.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786063+0000", 2023-11-25T04:24:21.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2600000000000002e-07 2023-11-25T04:24:21.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:21.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786069+0000", 2023-11-25T04:24:21.278 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8839999999999999e-06 2023-11-25T04:24:21.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:21.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786079+0000", 2023-11-25T04:24:21.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0124e-05 2023-11-25T04:24:21.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:21.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.786085+0000", 2023-11-25T04:24:21.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6660000000000002e-06 2023-11-25T04:24:21.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:21.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788160+0000", 2023-11-25T04:24:21.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020747669999999999 2023-11-25T04:24:21.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:21.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.788180+0000", 2023-11-25T04:24:21.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0208999999999999e-05 2023-11-25T04:24:21.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:21.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791087+0000", 2023-11-25T04:24:21.280 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0029064630000000002 2023-11-25T04:24:21.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.280 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:21.280 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:37.791121+0000", 2023-11-25T04:24:21.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4289000000000003e-05 2023-11-25T04:24:21.281 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:21.281 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:21.281 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:21.281 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.281 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:14 3.15 3:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:21.281 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:24:21.281 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 94.230870351999997, 2023-11-25T04:24:21.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010990437, 2023-11-25T04:24:21.281 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:21.281 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:21.282 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:21.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:21.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:24:21.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:21.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:21.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:24:21.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:21.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.282 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:21.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033383+0000", 2023-11-25T04:24:21.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9870000000000002e-06 2023-11-25T04:24:21.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:21.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033398+0000", 2023-11-25T04:24:21.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4569999999999999e-05 2023-11-25T04:24:21.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:21.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033411+0000", 2023-11-25T04:24:21.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3543999999999999e-05 2023-11-25T04:24:21.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:21.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033423+0000", 2023-11-25T04:24:21.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1905999999999999e-05 2023-11-25T04:24:21.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:21.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034433+0000", 2023-11-25T04:24:21.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00101022 2023-11-25T04:24:21.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:21.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034539+0000", 2023-11-25T04:24:21.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010597199999999999 2023-11-25T04:24:21.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:21.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034667+0000", 2023-11-25T04:24:21.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012765500000000001 2023-11-25T04:24:21.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.285 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.285 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:21.285 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037803+0000", 2023-11-25T04:24:21.285 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031362220000000001 2023-11-25T04:24:21.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:21.286 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:24:21.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0064964669999999997 2023-11-25T04:24:21.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:21.286 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044331+0000", 2023-11-25T04:24:21.286 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1544999999999997e-05 2023-11-25T04:24:21.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:21.287 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044372+0000", 2023-11-25T04:24:21.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0349000000000003e-05 2023-11-25T04:24:21.287 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:21.287 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:21.287 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:21.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.287 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:15 3.15 3:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:21.287 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:24:21.287 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 94.230210701999994, 2023-11-25T04:24:21.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010464362, 2023-11-25T04:24:21.287 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:21.287 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:21.288 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:21.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:21.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:24:21.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:21.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:21.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:24:21.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:21.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.288 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.288 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:21.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034042+0000", 2023-11-25T04:24:21.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0999999999999997e-07 2023-11-25T04:24:21.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:21.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034048+0000", 2023-11-25T04:24:21.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0390000000000001e-06 2023-11-25T04:24:21.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:21.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034059+0000", 2023-11-25T04:24:21.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1182999999999999e-05 2023-11-25T04:24:21.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.290 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:21.290 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034067+0000", 2023-11-25T04:24:21.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1559999999999995e-06 2023-11-25T04:24:21.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.290 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:21.290 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035005+0000", 2023-11-25T04:24:21.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00093747399999999999 2023-11-25T04:24:21.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.291 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:21.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035077+0000", 2023-11-25T04:24:21.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2747e-05 2023-11-25T04:24:21.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.291 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:21.291 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035219+0000", 2023-11-25T04:24:21.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000141877 2023-11-25T04:24:21.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:21.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037810+0000", 2023-11-25T04:24:21.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002590752 2023-11-25T04:24:21.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:21.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044437+0000", 2023-11-25T04:24:21.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066270110000000004 2023-11-25T04:24:21.292 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:21.293 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044481+0000", 2023-11-25T04:24:21.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4106000000000001e-05 2023-11-25T04:24:21.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.293 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:21.293 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044505+0000", 2023-11-25T04:24:21.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4207e-05 2023-11-25T04:24:21.293 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:21.293 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:21.293 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:21.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.294 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:16 3.1d 3:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:21.294 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:24:21.294 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 94.230168853999999, 2023-11-25T04:24:21.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010413923, 2023-11-25T04:24:21.294 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:21.294 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:21.294 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:21.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:21.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:24:21.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:21.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:21.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:24:21.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:21.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:21.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:24:21.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4499999999999997e-07 2023-11-25T04:24:21.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:21.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034087+0000", 2023-11-25T04:24:21.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9770000000000002e-06 2023-11-25T04:24:21.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.296 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:21.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034094+0000", 2023-11-25T04:24:21.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8809999999999998e-06 2023-11-25T04:24:21.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.296 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:21.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034100+0000", 2023-11-25T04:24:21.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8440000000000003e-06 2023-11-25T04:24:21.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:21.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034993+0000", 2023-11-25T04:24:21.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00089336799999999996 2023-11-25T04:24:21.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:21.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035099+0000", 2023-11-25T04:24:21.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106037 2023-11-25T04:24:21.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.298 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:21.298 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035276+0000", 2023-11-25T04:24:21.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017631799999999999 2023-11-25T04:24:21.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.298 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:21.298 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037815+0000", 2023-11-25T04:24:21.298 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025395959999999999 2023-11-25T04:24:21.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.298 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:21.298 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044420+0000", 2023-11-25T04:24:21.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066045749999999997 2023-11-25T04:24:21.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:21.299 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044463+0000", 2023-11-25T04:24:21.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3205999999999999e-05 2023-11-25T04:24:21.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:21.299 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044497+0000", 2023-11-25T04:24:21.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3476000000000003e-05 2023-11-25T04:24:21.299 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:21.300 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:21.300 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:21.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.300 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:17 3.1d 3:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:21.300 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:24:21.300 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 94.230143644999998, 2023-11-25T04:24:21.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01127356, 2023-11-25T04:24:21.300 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:21.300 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:21.300 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:21.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.301 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:21.301 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:24:21.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:21.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.301 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:21.301 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:24:21.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:21.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.301 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:21.301 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:24:21.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1100000000000001e-07 2023-11-25T04:24:21.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:21.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034112+0000", 2023-11-25T04:24:21.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8600000000000003e-06 2023-11-25T04:24:21.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:21.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034120+0000", 2023-11-25T04:24:21.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7700000000000001e-06 2023-11-25T04:24:21.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.303 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:21.303 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034127+0000", 2023-11-25T04:24:21.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0999999999999998e-06 2023-11-25T04:24:21.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.303 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:21.303 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035601+0000", 2023-11-25T04:24:21.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014739410000000001 2023-11-25T04:24:21.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.303 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:21.304 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035685+0000", 2023-11-25T04:24:21.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3677999999999994e-05 2023-11-25T04:24:21.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.304 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:21.304 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035827+0000", 2023-11-25T04:24:21.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014232599999999999 2023-11-25T04:24:21.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.304 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:21.304 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038725+0000", 2023-11-25T04:24:21.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028982779999999998 2023-11-25T04:24:21.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:21.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045323+0000", 2023-11-25T04:24:21.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0065980780000000003 2023-11-25T04:24:21.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.305 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:21.305 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045354+0000", 2023-11-25T04:24:21.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0463e-05 2023-11-25T04:24:21.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:21.306 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045381+0000", 2023-11-25T04:24:21.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7654999999999999e-05 2023-11-25T04:24:21.306 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:21.306 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:21.306 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:21.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.306 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:22 3.e 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:21.306 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:24:21.306 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 94.230117231999998, 2023-11-25T04:24:21.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010133422, 2023-11-25T04:24:21.306 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:21.307 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:21.307 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:21.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:21.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:24:21.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:21.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:21.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:24:21.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:21.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.308 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:21.308 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034135+0000", 2023-11-25T04:24:21.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-07 2023-11-25T04:24:21.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.308 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:21.308 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034138+0000", 2023-11-25T04:24:21.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6009999999999999e-06 2023-11-25T04:24:21.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.308 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:21.308 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034145+0000", 2023-11-25T04:24:21.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1169999999999998e-06 2023-11-25T04:24:21.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.309 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:21.309 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034150+0000", 2023-11-25T04:24:21.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9080000000000003e-06 2023-11-25T04:24:21.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.309 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:21.309 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034209+0000", 2023-11-25T04:24:21.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8149000000000002e-05 2023-11-25T04:24:21.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:21.310 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034415+0000", 2023-11-25T04:24:21.310 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020637699999999999 2023-11-25T04:24:21.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:21.310 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034584+0000", 2023-11-25T04:24:21.310 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016870399999999999 2023-11-25T04:24:21.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:21.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.036979+0000", 2023-11-25T04:24:21.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023957509999999998 2023-11-25T04:24:21.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:21.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044161+0000", 2023-11-25T04:24:21.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0071811590000000003 2023-11-25T04:24:21.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:21.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044224+0000", 2023-11-25T04:24:21.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3454000000000006e-05 2023-11-25T04:24:21.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.312 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:21.312 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044268+0000", 2023-11-25T04:24:21.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3761999999999999e-05 2023-11-25T04:24:21.312 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:21.312 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:21.312 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:21.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.312 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:23 3.e 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:21.312 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:24:21.312 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 94.230085822999996, 2023-11-25T04:24:21.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010281676, 2023-11-25T04:24:21.313 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:21.313 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:21.313 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:21.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.313 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:21.313 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:24:21.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:21.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.313 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:21.313 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:24:21.314 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:21.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.314 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:21.314 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:24:21.314 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5799999999999999e-07 2023-11-25T04:24:21.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.314 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:21.314 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034169+0000", 2023-11-25T04:24:21.314 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1879999999999998e-06 2023-11-25T04:24:21.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.315 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:21.315 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034177+0000", 2023-11-25T04:24:21.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7020000000000001e-06 2023-11-25T04:24:21.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.315 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:21.315 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034184+0000", 2023-11-25T04:24:21.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3910000000000003e-06 2023-11-25T04:24:21.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.315 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:21.315 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034924+0000", 2023-11-25T04:24:21.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00074032999999999998 2023-11-25T04:24:21.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.316 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:21.316 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034991+0000", 2023-11-25T04:24:21.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7347000000000004e-05 2023-11-25T04:24:21.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.316 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:21.316 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035096+0000", 2023-11-25T04:24:21.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010488 2023-11-25T04:24:21.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.317 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:21.317 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037588+0000", 2023-11-25T04:24:21.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024918309999999999 2023-11-25T04:24:21.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.317 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:21.317 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044367+0000", 2023-11-25T04:24:21.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067787589999999997 2023-11-25T04:24:21.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.317 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:21.318 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044415+0000", 2023-11-25T04:24:21.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7803000000000003e-05 2023-11-25T04:24:21.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.318 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:21.318 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044447+0000", 2023-11-25T04:24:21.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2886999999999999e-05 2023-11-25T04:24:21.318 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:21.318 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:21.318 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:21.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.318 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:30 3.15 3:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:21.319 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:24:21.319 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 94.230027383000007, 2023-11-25T04:24:21.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010365070000000001, 2023-11-25T04:24:21.319 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:21.319 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:21.319 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:21.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.319 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:21.319 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:24:21.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:21.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:21.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:24:21.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:21.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:21.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034225+0000", 2023-11-25T04:24:21.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7400000000000002e-07 2023-11-25T04:24:21.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:21.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034229+0000", 2023-11-25T04:24:21.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3100000000000002e-06 2023-11-25T04:24:21.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:21.321 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034239+0000", 2023-11-25T04:24:21.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9140000000000003e-06 2023-11-25T04:24:21.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:21.321 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034246+0000", 2023-11-25T04:24:21.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0149999999999998e-06 2023-11-25T04:24:21.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:21.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035572+0000", 2023-11-25T04:24:21.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001325655 2023-11-25T04:24:21.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:21.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035730+0000", 2023-11-25T04:24:21.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015827899999999999 2023-11-25T04:24:21.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:21.323 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035857+0000", 2023-11-25T04:24:21.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012644699999999999 2023-11-25T04:24:21.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.323 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:21.323 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038737+0000", 2023-11-25T04:24:21.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002880077 2023-11-25T04:24:21.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.323 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:21.323 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044540+0000", 2023-11-25T04:24:21.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058034239999999997 2023-11-25T04:24:21.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.324 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.324 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:21.324 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044569+0000", 2023-11-25T04:24:21.324 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8388999999999999e-05 2023-11-25T04:24:21.324 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.324 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.325 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:21.325 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044589+0000", 2023-11-25T04:24:21.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0686000000000002e-05 2023-11-25T04:24:21.325 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:21.325 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:21.325 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:21.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.325 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:33 3.1b 3:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:21.325 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:24:21.325 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 94.229981885000001, 2023-11-25T04:24:21.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099754049999999997, 2023-11-25T04:24:21.326 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:21.326 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:21.326 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:21.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:21.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:24:21.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:21.326 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.326 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:21.326 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:24:21.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:21.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.327 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:21.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:24:21.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4799999999999998e-07 2023-11-25T04:24:21.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.327 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:21.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034273+0000", 2023-11-25T04:24:21.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7819999999999998e-06 2023-11-25T04:24:21.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.328 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.328 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:21.328 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034285+0000", 2023-11-25T04:24:21.328 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2058e-05 2023-11-25T04:24:21.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.328 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.328 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:21.328 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034291+0000", 2023-11-25T04:24:21.328 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0680000000000002e-06 2023-11-25T04:24:21.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.328 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.328 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:21.328 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034370+0000", 2023-11-25T04:24:21.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8452e-05 2023-11-25T04:24:21.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.329 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.329 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:21.329 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034477+0000", 2023-11-25T04:24:21.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107471 2023-11-25T04:24:21.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.329 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.329 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:21.329 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044245+0000", 2023-11-25T04:24:21.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0097679259999999997 2023-11-25T04:24:21.330 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:21.330 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:21.330 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:21.330 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.330 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.330 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:36 3.1a 3:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:21.330 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:24:21.330 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 94.229935212000001, 2023-11-25T04:24:21.330 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058179099999999999, 2023-11-25T04:24:21.330 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:21.330 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:21.330 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:21.331 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.331 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:21.331 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:24:21.331 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:21.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.331 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.331 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:21.331 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034316+0000", 2023-11-25T04:24:21.331 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:21.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.331 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.331 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:21.331 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034317+0000", 2023-11-25T04:24:21.332 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1999999999999999e-07 2023-11-25T04:24:21.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.332 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.332 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:21.332 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034321+0000", 2023-11-25T04:24:21.332 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.233e-06 2023-11-25T04:24:21.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.332 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.332 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:21.332 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034337+0000", 2023-11-25T04:24:21.332 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5308999999999999e-05 2023-11-25T04:24:21.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.333 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.333 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:21.333 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034354+0000", 2023-11-25T04:24:21.333 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7411999999999999e-05 2023-11-25T04:24:21.333 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.333 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.333 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:21.333 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034465+0000", 2023-11-25T04:24:21.333 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011154699999999999 2023-11-25T04:24:21.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.334 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:21.334 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034560+0000", 2023-11-25T04:24:21.334 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4186999999999998e-05 2023-11-25T04:24:21.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.334 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:21.334 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.040134+0000", 2023-11-25T04:24:21.334 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055746019999999997 2023-11-25T04:24:21.334 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:21.334 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:21.335 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:21.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.335 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.335 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:9 3.10 e18/16 3:09401a81:::601.00000000:head v 18'4, mlcod=0'0)", 2023-11-25T04:24:21.335 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:24:21.335 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 94.221040904000006, 2023-11-25T04:24:21.335 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0061641020000000003, 2023-11-25T04:24:21.335 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:21.335 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:21.335 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:21.335 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.335 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:21.336 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:24:21.336 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:21.336 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.336 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.336 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:21.336 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:24:21.336 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:21.336 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.336 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.336 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:21.336 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043211+0000", 2023-11-25T04:24:21.336 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5300000000000001e-07 2023-11-25T04:24:21.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.337 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:21.337 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043222+0000", 2023-11-25T04:24:21.337 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.082e-05 2023-11-25T04:24:21.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.337 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:21.337 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043228+0000", 2023-11-25T04:24:21.337 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6699999999999997e-06 2023-11-25T04:24:21.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.337 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:21.338 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043236+0000", 2023-11-25T04:24:21.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8189999999999992e-06 2023-11-25T04:24:21.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.338 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:21.338 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043507+0000", 2023-11-25T04:24:21.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000271012 2023-11-25T04:24:21.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.338 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:21.338 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043530+0000", 2023-11-25T04:24:21.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2659000000000001e-05 2023-11-25T04:24:21.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.339 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:21.339 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049321+0000", 2023-11-25T04:24:21.339 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057915129999999999 2023-11-25T04:24:21.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.339 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:21.339 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049375+0000", 2023-11-25T04:24:21.339 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3455999999999998e-05 2023-11-25T04:24:21.339 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:21.339 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:21.339 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:21.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.340 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:1 3.14 e18/16 3:292cf221:::200.00000000:head v 18'3, mlcod=0'0)", 2023-11-25T04:24:21.340 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:24:21.340 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 94.221004191000006, 2023-11-25T04:24:21.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0060983599999999997, 2023-11-25T04:24:21.340 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:21.340 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:21.340 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:21.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.340 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:21.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:24:21.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:21.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.341 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:21.341 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043247+0000", 2023-11-25T04:24:21.341 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:21.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.341 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:21.341 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043248+0000", 2023-11-25T04:24:21.341 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9399999999999995e-07 2023-11-25T04:24:21.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.341 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:21.342 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043252+0000", 2023-11-25T04:24:21.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1799999999999998e-06 2023-11-25T04:24:21.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.342 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:21.342 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043259+0000", 2023-11-25T04:24:21.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0049999999999998e-06 2023-11-25T04:24:21.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.342 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:21.342 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043265+0000", 2023-11-25T04:24:21.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0859999999999998e-06 2023-11-25T04:24:21.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.343 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:21.343 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043311+0000", 2023-11-25T04:24:21.343 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6082999999999999e-05 2023-11-25T04:24:21.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.343 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:21.343 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043330+0000", 2023-11-25T04:24:21.343 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8556e-05 2023-11-25T04:24:21.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.343 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:21.344 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049303+0000", 2023-11-25T04:24:21.344 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0059733470000000004 2023-11-25T04:24:21.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.344 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:21.344 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049346+0000", 2023-11-25T04:24:21.344 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2608999999999998e-05 2023-11-25T04:24:21.344 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:21.344 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:21.344 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:21.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.345 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:3 3.14 e18/16 3:287034c4:::1.00000000.inode:head v 18'4, mlcod=0'0)", 2023-11-25T04:24:21.345 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:24:21.345 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 94.220487684999995, 2023-11-25T04:24:21.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057040090000000003, 2023-11-25T04:24:21.345 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:21.345 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:21.345 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:21.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.345 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:21.345 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:24:21.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:21.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:21.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:24:21.346 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:21.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:21.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043764+0000", 2023-11-25T04:24:21.346 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9799999999999999e-07 2023-11-25T04:24:21.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:21.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043785+0000", 2023-11-25T04:24:21.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0721999999999999e-05 2023-11-25T04:24:21.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.347 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:21.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043793+0000", 2023-11-25T04:24:21.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5839999999999995e-06 2023-11-25T04:24:21.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.347 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:21.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043803+0000", 2023-11-25T04:24:21.347 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0729999999999994e-06 2023-11-25T04:24:21.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.348 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:21.348 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043843+0000", 2023-11-25T04:24:21.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0747999999999997e-05 2023-11-25T04:24:21.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.348 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:21.348 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043856+0000", 2023-11-25T04:24:21.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2697000000000001e-05 2023-11-25T04:24:21.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.348 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:21.348 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049426+0000", 2023-11-25T04:24:21.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0055704409999999998 2023-11-25T04:24:21.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.349 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:21.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049468+0000", 2023-11-25T04:24:21.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1446000000000003e-05 2023-11-25T04:24:21.349 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:21.349 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:21.349 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:21.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.349 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "pg_update_log_missing(3.14 epoch 18/16 rep_tid 171 entries 18'5 (0'0) error 3:2e2fa760:::200.00000001:head by mds.0.3:32 0.000000 -2 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2023-11-25T04:24:21.349 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:24:21.350 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 94.220326721999996, 2023-11-25T04:24:21.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.005556938, 2023-11-25T04:24:21.350 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:21.350 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "reached pg", 2023-11-25T04:24:21.350 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:21.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.350 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:21.350 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:24:21.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:21.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.350 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:21.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:24:21.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:21.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.351 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:21.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043925+0000", 2023-11-25T04:24:21.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.03e-07 2023-11-25T04:24:21.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.351 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:21.351 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043931+0000", 2023-11-25T04:24:21.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1079999999999998e-06 2023-11-25T04:24:21.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.352 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:21.352 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043944+0000", 2023-11-25T04:24:21.352 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2852e-05 2023-11-25T04:24:21.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.352 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:21.352 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.043955+0000", 2023-11-25T04:24:21.352 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1037e-05 2023-11-25T04:24:21.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.352 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:21.353 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044088+0000", 2023-11-25T04:24:21.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013269599999999999 2023-11-25T04:24:21.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.353 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:21.353 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049482+0000", 2023-11-25T04:24:21.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053940419999999999 2023-11-25T04:24:21.353 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:21.353 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:21.353 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:21.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.354 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:10 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'5, mlcod=0'0)", 2023-11-25T04:24:21.354 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:24:21.354 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 94.220236693999993, 2023-11-25T04:24:21.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0054493099999999997, 2023-11-25T04:24:21.354 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:21.354 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:21.354 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:21.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.354 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:21.354 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:24:21.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:21.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.355 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.355 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:21.355 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:24:21.355 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:21.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.355 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.355 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:21.355 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044015+0000", 2023-11-25T04:24:21.355 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6100000000000002e-07 2023-11-25T04:24:21.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.356 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.356 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:21.356 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044019+0000", 2023-11-25T04:24:21.356 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.839e-06 2023-11-25T04:24:21.356 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.356 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.356 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:21.356 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044026+0000", 2023-11-25T04:24:21.356 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5869999999999998e-06 2023-11-25T04:24:21.356 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.356 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.356 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:21.357 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044031+0000", 2023-11-25T04:24:21.357 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4909999999999996e-06 2023-11-25T04:24:21.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.357 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:21.357 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044070+0000", 2023-11-25T04:24:21.357 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8829999999999999e-05 2023-11-25T04:24:21.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.357 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:21.357 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044081+0000", 2023-11-25T04:24:21.357 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1447999999999999e-05 2023-11-25T04:24:21.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.358 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:21.358 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049406+0000", 2023-11-25T04:24:21.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0053246480000000004 2023-11-25T04:24:21.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.358 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:21.358 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049464+0000", 2023-11-25T04:24:21.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8205999999999998e-05 2023-11-25T04:24:21.358 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:21.358 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:21.358 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:21.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.359 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:11 3.10 e18/16 3:0bd6d154:::602.00000000:head v 18'6, mlcod=0'0)", 2023-11-25T04:24:21.359 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:24:21.359 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 94.220035429999996, 2023-11-25T04:24:21.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0052673499999999996, 2023-11-25T04:24:21.359 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:21.359 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:21.359 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:21.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:21.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:24:21.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:21.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.360 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:21.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:24:21.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:21.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.360 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:21.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044216+0000", 2023-11-25T04:24:21.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0199999999999998e-07 2023-11-25T04:24:21.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.360 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:21.361 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044220+0000", 2023-11-25T04:24:21.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7720000000000001e-06 2023-11-25T04:24:21.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:21.361 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044226+0000", 2023-11-25T04:24:21.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8579999999999996e-06 2023-11-25T04:24:21.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:21.361 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044232+0000", 2023-11-25T04:24:21.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1700000000000002e-06 2023-11-25T04:24:21.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.362 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:21.362 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044274+0000", 2023-11-25T04:24:21.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1254999999999997e-05 2023-11-25T04:24:21.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.362 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:21.362 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:24:21.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6880999999999999e-05 2023-11-25T04:24:21.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:21.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049466+0000", 2023-11-25T04:24:21.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0051659219999999999 2023-11-25T04:24:21.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:21.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.049483+0000", 2023-11-25T04:24:21.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.719e-05 2023-11-25T04:24:21.363 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:21.363 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:21.363 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:21.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.364 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:123 3.1b 3:ddb4e4d7:::200.00000002:head [write 1044606~1917259 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e45)", 2023-11-25T04:24:21.364 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:17.954464+0000", 2023-11-25T04:24:21.364 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.3097872580000001, 2023-11-25T04:24:21.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0049729630000000004, 2023-11-25T04:24:21.364 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:21.364 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:21.364 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:21.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.364 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:21.364 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:17.954464+0000", 2023-11-25T04:24:21.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:21.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.365 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:21.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:17.954464+0000", 2023-11-25T04:24:21.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:21.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.365 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:21.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:17.954465+0000", 2023-11-25T04:24:21.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8899999999999998e-07 2023-11-25T04:24:21.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.365 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:21.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:17.956522+0000", 2023-11-25T04:24:21.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002057059 2023-11-25T04:24:21.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:21.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:17.956533+0000", 2023-11-25T04:24:21.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1224e-05 2023-11-25T04:24:21.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:21.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:17.956556+0000", 2023-11-25T04:24:21.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2527999999999998e-05 2023-11-25T04:24:21.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:21.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:17.956598+0000", 2023-11-25T04:24:21.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2443e-05 2023-11-25T04:24:21.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:21.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:17.956650+0000", 2023-11-25T04:24:21.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1687000000000002e-05 2023-11-25T04:24:21.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:21.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:17.959352+0000", 2023-11-25T04:24:21.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0027018670000000002 2023-11-25T04:24:21.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:21.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:17.959384+0000", 2023-11-25T04:24:21.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1735e-05 2023-11-25T04:24:21.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:21.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:17.959437+0000", 2023-11-25T04:24:21.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3631000000000002e-05 2023-11-25T04:24:21.368 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:21.369 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:21.369 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:21.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.369 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:152 3.1a 3:5e41ab69:::200.00000005:head [write 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:21.369 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:21.237401+0000", 2023-11-25T04:24:21.369 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.026850846000000001, 2023-11-25T04:24:21.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015253802, 2023-11-25T04:24:21.369 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:21.369 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:21.369 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:21.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:21.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.237401+0000", 2023-11-25T04:24:21.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:21.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:21.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.237401+0000", 2023-11-25T04:24:21.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:21.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:21.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.237402+0000", 2023-11-25T04:24:21.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3699999999999999e-07 2023-11-25T04:24:21.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:21.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243182+0000", 2023-11-25T04:24:21.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057801270000000004 2023-11-25T04:24:21.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:21.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243193+0000", 2023-11-25T04:24:21.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1221e-05 2023-11-25T04:24:21.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:21.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243198+0000", 2023-11-25T04:24:21.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3739999999999997e-06 2023-11-25T04:24:21.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:21.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243246+0000", 2023-11-25T04:24:21.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7667999999999998e-05 2023-11-25T04:24:21.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:21.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243292+0000", 2023-11-25T04:24:21.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5794000000000002e-05 2023-11-25T04:24:21.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:21.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243360+0000", 2023-11-25T04:24:21.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8479000000000006e-05 2023-11-25T04:24:21.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:21.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.250667+0000", 2023-11-25T04:24:21.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.007306618 2023-11-25T04:24:21.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:21.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.252598+0000", 2023-11-25T04:24:21.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019315129999999999 2023-11-25T04:24:21.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:21.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.252631+0000", 2023-11-25T04:24:21.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2925000000000001e-05 2023-11-25T04:24:21.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:21.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:21.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:21.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.252655+0000", 2023-11-25T04:24:21.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3246000000000001e-05 2023-11-25T04:24:21.375 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:21.375 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:21.375 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:21.375 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:21.375 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:21.375 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:21.377 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:24:21.377 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:21.375+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:21.477 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:21.476+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:21.487 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:21.487 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2023-11-25T04:24:21.487 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.487 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4561.0:5067 4.30 4:0e3dd0c4:::10000001604.00000000:head [write 0~2814 in=2814b] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2023-11-25T04:24:21.487 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:21.485483+0000", 2023-11-25T04:24:21.488 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 0.00053016200000000004, 2023-11-25T04:24:21.488 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00054577100000000002, 2023-11-25T04:24:21.488 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:21.488 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:21.488 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2023-11-25T04:24:21.488 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4561", 2023-11-25T04:24:21.488 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "192.168.0.1:0/1268434739", 2023-11-25T04:24:21.489 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 5067 2023-11-25T04:24:21.489 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.489 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:21.489 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.489 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:21.489 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.485483+0000", 2023-11-25T04:24:21.489 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:21.489 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.490 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.490 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:21.490 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.485483+0000", 2023-11-25T04:24:21.490 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:21.490 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.490 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.491 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:21.491 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.485484+0000", 2023-11-25T04:24:21.491 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2999999999999999e-07 2023-11-25T04:24:21.491 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.491 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.491 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:21.492 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.485489+0000", 2023-11-25T04:24:21.492 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.955e-06 2023-11-25T04:24:21.492 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.492 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.492 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:21.492 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.485502+0000", 2023-11-25T04:24:21.493 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3081999999999999e-05 2023-11-25T04:24:21.493 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.493 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.493 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:21.493 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.485511+0000", 2023-11-25T04:24:21.493 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.7280000000000001e-06 2023-11-25T04:24:21.493 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.494 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.494 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:21.494 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.485551+0000", 2023-11-25T04:24:21.494 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9866000000000001e-05 2023-11-25T04:24:21.494 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.494 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.495 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:21.495 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.485632+0000", 2023-11-25T04:24:21.495 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0896999999999995e-05 2023-11-25T04:24:21.495 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:21.495 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:21.495 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:21.495 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:21.496 INFO:teuthology.orchestra.run.smithi195.stdout: ], 2023-11-25T04:24:21.496 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 1 2023-11-25T04:24:21.496 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:21.497 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:24:21.578 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:21.576+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:21.613 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:21.613 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:24:21.614 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2023-11-25T04:24:21.614 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2023-11-25T04:24:21.614 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:21.623 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:24:21.679 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:21.678+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:21.773 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:21.774 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2023-11-25T04:24:21.774 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2023-11-25T04:24:21.774 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2023-11-25T04:24:21.774 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.774 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:21.774 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:24:21.774 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 94.739090751999996, 2023-11-25T04:24:21.774 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01072943, 2023-11-25T04:24:21.774 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:21.774 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:21.774 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:21.774 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.775 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:21.775 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:24:21.775 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:21.775 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.775 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.775 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:21.775 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:24:21.775 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:21.775 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.775 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.775 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:21.775 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031829+0000", 2023-11-25T04:24:21.775 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.666e-06 2023-11-25T04:24:21.775 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.776 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.776 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:21.776 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031839+0000", 2023-11-25T04:24:21.776 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9490000000000008e-06 2023-11-25T04:24:21.776 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.776 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.776 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:21.776 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031857+0000", 2023-11-25T04:24:21.776 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8093000000000001e-05 2023-11-25T04:24:21.777 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.777 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.777 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:21.777 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031882+0000", 2023-11-25T04:24:21.777 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4294000000000002e-05 2023-11-25T04:24:21.777 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.777 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.777 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:21.777 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033130+0000", 2023-11-25T04:24:21.777 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001248264 2023-11-25T04:24:21.777 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.777 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.778 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:21.778 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033426+0000", 2023-11-25T04:24:21.778 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029620999999999998 2023-11-25T04:24:21.778 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.778 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.778 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:21.778 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033814+0000", 2023-11-25T04:24:21.778 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000387742 2023-11-25T04:24:21.778 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.778 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.778 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:21.778 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036793+0000", 2023-11-25T04:24:21.778 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002979598 2023-11-25T04:24:21.779 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.779 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.779 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:21.779 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042455+0000", 2023-11-25T04:24:21.779 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0056619260000000003 2023-11-25T04:24:21.779 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.779 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.779 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:21.779 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042502+0000", 2023-11-25T04:24:21.779 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7139999999999999e-05 2023-11-25T04:24:21.779 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.779 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.779 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:21.780 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042557+0000", 2023-11-25T04:24:21.780 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4548e-05 2023-11-25T04:24:21.780 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:21.780 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:21.780 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:21.780 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.780 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.780 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:21.780 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:24:21.780 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 94.737892216999995, 2023-11-25T04:24:21.780 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011505202000000001, 2023-11-25T04:24:21.781 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:21.781 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:21.781 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:21.781 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.781 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:21.781 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:24:21.781 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:21.781 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.781 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.781 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:21.781 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:24:21.782 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:21.782 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.782 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.782 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:21.783 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033027+0000", 2023-11-25T04:24:21.783 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0699999999999999e-06 2023-11-25T04:24:21.783 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.783 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.783 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:21.783 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033035+0000", 2023-11-25T04:24:21.783 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0830000000000004e-06 2023-11-25T04:24:21.783 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.783 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.783 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:21.783 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033059+0000", 2023-11-25T04:24:21.784 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3975e-05 2023-11-25T04:24:21.784 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.784 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.784 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:21.784 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033482+0000", 2023-11-25T04:24:21.784 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042253700000000001 2023-11-25T04:24:21.784 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.784 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.784 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:21.784 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034520+0000", 2023-11-25T04:24:21.784 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001037991 2023-11-25T04:24:21.784 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.784 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.785 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:21.785 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034611+0000", 2023-11-25T04:24:21.785 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1113999999999995e-05 2023-11-25T04:24:21.785 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.785 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.786 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:21.786 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034769+0000", 2023-11-25T04:24:21.786 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000157641 2023-11-25T04:24:21.786 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.786 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.786 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:21.786 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038292+0000", 2023-11-25T04:24:21.786 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035231160000000002 2023-11-25T04:24:21.786 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.787 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.787 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:21.787 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044440+0000", 2023-11-25T04:24:21.787 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0061486980000000002 2023-11-25T04:24:21.787 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.787 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.787 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:21.787 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044495+0000", 2023-11-25T04:24:21.787 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4339000000000003e-05 2023-11-25T04:24:21.787 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.788 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.788 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:21.788 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044531+0000", 2023-11-25T04:24:21.788 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6637999999999997e-05 2023-11-25T04:24:21.788 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:21.788 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:21.788 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:21.788 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.788 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.788 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:24 3.7 3:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:21.788 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:24:21.788 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 94.737417863000005, 2023-11-25T04:24:21.788 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0057634039999999997, 2023-11-25T04:24:21.789 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:21.789 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:21.789 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:21.789 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.789 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:21.789 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:24:21.789 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:21.789 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.789 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.789 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:21.789 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:24:21.789 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:21.789 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.789 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.790 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:21.790 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033501+0000", 2023-11-25T04:24:21.790 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7900000000000007e-07 2023-11-25T04:24:21.790 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.790 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.790 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:21.790 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033507+0000", 2023-11-25T04:24:21.790 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8900000000000004e-06 2023-11-25T04:24:21.790 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.790 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.790 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:21.790 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033519+0000", 2023-11-25T04:24:21.790 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1756000000000001e-05 2023-11-25T04:24:21.791 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.791 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.791 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:21.791 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033527+0000", 2023-11-25T04:24:21.791 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3610000000000001e-06 2023-11-25T04:24:21.791 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.791 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.791 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:21.791 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034451+0000", 2023-11-25T04:24:21.791 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00092372400000000001 2023-11-25T04:24:21.791 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.791 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.791 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:21.792 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034809+0000", 2023-11-25T04:24:21.792 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00035757699999999998 2023-11-25T04:24:21.792 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.792 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.792 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:21.792 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034965+0000", 2023-11-25T04:24:21.792 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015638699999999999 2023-11-25T04:24:21.792 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.792 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.792 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:21.792 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038299+0000", 2023-11-25T04:24:21.792 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0033336559999999999 2023-11-25T04:24:21.792 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.792 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.793 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:21.793 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039169+0000", 2023-11-25T04:24:21.793 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00087066400000000005 2023-11-25T04:24:21.793 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.793 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.793 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:21.793 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039226+0000", 2023-11-25T04:24:21.794 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6511999999999997e-05 2023-11-25T04:24:21.794 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.794 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.794 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:21.794 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.039264+0000", 2023-11-25T04:24:21.794 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7898000000000002e-05 2023-11-25T04:24:21.794 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:21.794 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:21.794 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:21.794 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.794 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.794 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:25 3.7 3:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:21.794 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:24:21.795 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 94.737339278999997, 2023-11-25T04:24:21.795 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096656169999999996, 2023-11-25T04:24:21.795 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:21.795 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:21.795 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:21.795 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.795 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:21.795 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:24:21.795 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:21.795 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.795 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.795 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:21.795 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033579+0000", 2023-11-25T04:24:21.796 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:21.796 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.796 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.796 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:21.796 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033580+0000", 2023-11-25T04:24:21.796 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8199999999999999e-07 2023-11-25T04:24:21.796 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.796 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.796 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:21.796 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033585+0000", 2023-11-25T04:24:21.796 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3680000000000001e-06 2023-11-25T04:24:21.797 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.797 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.797 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:21.797 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033600+0000", 2023-11-25T04:24:21.797 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4331000000000001e-05 2023-11-25T04:24:21.797 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.797 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.797 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:21.797 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033606+0000", 2023-11-25T04:24:21.797 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5780000000000004e-06 2023-11-25T04:24:21.797 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.798 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.798 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:21.798 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035747+0000", 2023-11-25T04:24:21.798 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0021406530000000002 2023-11-25T04:24:21.798 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.798 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.798 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:21.798 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035901+0000", 2023-11-25T04:24:21.798 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000153931 2023-11-25T04:24:21.798 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.798 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.798 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:21.798 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036193+0000", 2023-11-25T04:24:21.798 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029211299999999999 2023-11-25T04:24:21.799 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.799 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.799 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:21.799 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038655+0000", 2023-11-25T04:24:21.799 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0024621640000000002 2023-11-25T04:24:21.799 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.799 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.799 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:21.799 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043130+0000", 2023-11-25T04:24:21.799 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.004474698 2023-11-25T04:24:21.799 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.799 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.799 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:21.799 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043204+0000", 2023-11-25T04:24:21.800 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4091999999999995e-05 2023-11-25T04:24:21.800 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.800 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.800 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:21.800 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043245+0000", 2023-11-25T04:24:21.800 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0907e-05 2023-11-25T04:24:21.800 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:21.800 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:21.800 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:21.800 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.800 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.800 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:26 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:21.800 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:21.801 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 94.737303455000003, 2023-11-25T04:24:21.801 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011086945000000001, 2023-11-25T04:24:21.801 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:21.801 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:21.801 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:21.801 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.801 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:21.801 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:21.801 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:21.801 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.801 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.801 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:21.801 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:21.801 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:21.802 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.802 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.802 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:21.802 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:21.802 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1699999999999998e-07 2023-11-25T04:24:21.803 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.803 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.803 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:21.803 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033627+0000", 2023-11-25T04:24:21.804 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.163e-05 2023-11-25T04:24:21.804 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.804 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.804 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:21.804 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033636+0000", 2023-11-25T04:24:21.804 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6859999999999995e-06 2023-11-25T04:24:21.804 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.804 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.804 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:21.804 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033642+0000", 2023-11-25T04:24:21.804 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3149999999999997e-06 2023-11-25T04:24:21.804 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.805 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.805 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:21.805 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035414+0000", 2023-11-25T04:24:21.805 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0017716380000000001 2023-11-25T04:24:21.805 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.805 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.805 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:21.805 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035539+0000", 2023-11-25T04:24:21.805 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000125477 2023-11-25T04:24:21.805 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.805 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.805 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:21.805 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035672+0000", 2023-11-25T04:24:21.805 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000133316 2023-11-25T04:24:21.806 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.806 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.806 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:21.806 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038649+0000", 2023-11-25T04:24:21.806 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029764679999999999 2023-11-25T04:24:21.806 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.806 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.806 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:21.806 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044578+0000", 2023-11-25T04:24:21.806 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0059293510000000002 2023-11-25T04:24:21.806 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.806 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.807 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:21.807 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044651+0000", 2023-11-25T04:24:21.807 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2513000000000002e-05 2023-11-25T04:24:21.807 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.807 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.807 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:21.807 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044702+0000", 2023-11-25T04:24:21.807 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1034e-05 2023-11-25T04:24:21.807 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:21.807 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:21.807 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:21.807 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.808 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.808 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:29 3.b 3:d0630e4c:::mds0_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:21.808 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:21.808 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 94.737265535000006, 2023-11-25T04:24:21.808 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010980227, 2023-11-25T04:24:21.808 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:21.808 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:21.808 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:21.808 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.808 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:21.809 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:21.809 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:21.809 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.809 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.809 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:21.809 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:21.809 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:21.809 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.809 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.809 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:21.809 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:21.809 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3499999999999996e-07 2023-11-25T04:24:21.809 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.810 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.810 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:21.810 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033658+0000", 2023-11-25T04:24:21.810 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1280000000000001e-06 2023-11-25T04:24:21.810 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.810 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.810 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:21.810 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033667+0000", 2023-11-25T04:24:21.810 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1430000000000004e-06 2023-11-25T04:24:21.810 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.810 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.810 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:21.811 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033678+0000", 2023-11-25T04:24:21.811 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0906e-05 2023-11-25T04:24:21.811 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.811 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.811 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:21.811 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034456+0000", 2023-11-25T04:24:21.811 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00077833899999999996 2023-11-25T04:24:21.811 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.811 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.811 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:21.811 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034575+0000", 2023-11-25T04:24:21.811 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011926000000000001 2023-11-25T04:24:21.811 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.812 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.812 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:21.812 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035001+0000", 2023-11-25T04:24:21.812 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042550999999999998 2023-11-25T04:24:21.812 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.812 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.812 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:21.812 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.037923+0000", 2023-11-25T04:24:21.812 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029219670000000001 2023-11-25T04:24:21.812 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.812 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.812 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:21.812 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044538+0000", 2023-11-25T04:24:21.813 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0066152349999999997 2023-11-25T04:24:21.813 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.813 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.813 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:21.813 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044594+0000", 2023-11-25T04:24:21.813 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5593000000000001e-05 2023-11-25T04:24:21.813 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.813 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.813 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:21.813 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044633+0000", 2023-11-25T04:24:21.813 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9511000000000002e-05 2023-11-25T04:24:21.813 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:21.813 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:21.813 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:21.814 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.814 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.814 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:34 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:21.814 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:24:21.814 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 94.737015421999999, 2023-11-25T04:24:21.814 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010920445000000001, 2023-11-25T04:24:21.814 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:21.814 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:21.814 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:21.814 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.814 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:21.814 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:24:21.815 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:21.815 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.815 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.815 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:21.815 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:24:21.815 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:21.815 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.815 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.815 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:21.815 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033904+0000", 2023-11-25T04:24:21.815 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6599999999999994e-07 2023-11-25T04:24:21.815 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.815 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.815 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:21.816 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033909+0000", 2023-11-25T04:24:21.816 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5639999999999996e-06 2023-11-25T04:24:21.816 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.816 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.816 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:21.816 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033923+0000", 2023-11-25T04:24:21.816 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3028e-05 2023-11-25T04:24:21.816 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.816 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.816 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:21.816 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034043+0000", 2023-11-25T04:24:21.817 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000120328 2023-11-25T04:24:21.817 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.817 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.817 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:21.817 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035576+0000", 2023-11-25T04:24:21.817 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015333720000000001 2023-11-25T04:24:21.817 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.817 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.817 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:21.817 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035699+0000", 2023-11-25T04:24:21.817 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000123001 2023-11-25T04:24:21.817 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.817 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.818 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:21.818 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044823+0000", 2023-11-25T04:24:21.818 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0091241859999999994 2023-11-25T04:24:21.818 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:21.818 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:21.818 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:21.818 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.818 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.818 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:35 3.1f 3:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:21.818 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:24:21.818 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 94.735711217000002, 2023-11-25T04:24:21.818 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096164779999999995, 2023-11-25T04:24:21.818 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:21.819 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:21.819 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:21.819 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.819 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:21.819 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:24:21.819 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:21.819 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.819 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.819 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:21.819 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035207+0000", 2023-11-25T04:24:21.819 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:21.819 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.819 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.820 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:21.820 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035208+0000", 2023-11-25T04:24:21.820 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2699999999999998e-07 2023-11-25T04:24:21.820 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.820 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.820 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:21.820 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035213+0000", 2023-11-25T04:24:21.820 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.6070000000000001e-06 2023-11-25T04:24:21.820 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.820 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.820 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:21.820 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035350+0000", 2023-11-25T04:24:21.821 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013693599999999999 2023-11-25T04:24:21.821 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.821 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.821 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:21.821 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035358+0000", 2023-11-25T04:24:21.821 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0099999999999995e-06 2023-11-25T04:24:21.821 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.821 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.821 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:21.821 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036260+0000", 2023-11-25T04:24:21.821 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00090240100000000005 2023-11-25T04:24:21.821 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.821 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.821 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:21.822 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036517+0000", 2023-11-25T04:24:21.822 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00025736299999999998 2023-11-25T04:24:21.822 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.822 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.822 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:21.822 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044824+0000", 2023-11-25T04:24:21.822 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0083063340000000003 2023-11-25T04:24:21.822 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:21.822 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:21.822 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:21.822 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.822 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.822 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:37 3.1e 3:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:21.823 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:24:21.823 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 94.735544531000002, 2023-11-25T04:24:21.823 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0078322640000000002, 2023-11-25T04:24:21.823 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:21.823 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:21.823 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:21.823 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.823 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:21.823 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:24:21.823 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:21.823 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.823 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.823 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:21.824 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035374+0000", 2023-11-25T04:24:21.824 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:21.824 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.824 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.824 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:21.824 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035375+0000", 2023-11-25T04:24:21.824 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9100000000000003e-07 2023-11-25T04:24:21.824 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.824 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.824 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:21.824 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035379+0000", 2023-11-25T04:24:21.824 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9430000000000002e-06 2023-11-25T04:24:21.824 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.824 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.825 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:21.825 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035388+0000", 2023-11-25T04:24:21.825 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2760000000000006e-06 2023-11-25T04:24:21.825 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.825 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.825 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:21.825 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035396+0000", 2023-11-25T04:24:21.825 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8480000000000001e-06 2023-11-25T04:24:21.825 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.825 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.825 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:21.825 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035452+0000", 2023-11-25T04:24:21.825 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5831000000000001e-05 2023-11-25T04:24:21.826 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.826 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.826 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:21.826 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035934+0000", 2023-11-25T04:24:21.827 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00048265599999999998 2023-11-25T04:24:21.827 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.827 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.827 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:21.827 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.043206+0000", 2023-11-25T04:24:21.827 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.007271919 2023-11-25T04:24:21.827 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:21.827 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:21.827 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:21.827 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.827 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.827 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4561.0:2151 4.21 4:85442bc0:::10000000a86.00000000:head [write 0~200] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2023-11-25T04:24:21.827 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:16.819976+0000", 2023-11-25T04:24:21.828 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.9509423320000003, 2023-11-25T04:24:21.828 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0020804280000000001, 2023-11-25T04:24:21.828 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:21.828 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:21.828 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2023-11-25T04:24:21.828 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4561", 2023-11-25T04:24:21.828 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "192.168.0.1:0/1268434739", 2023-11-25T04:24:21.828 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 2151 2023-11-25T04:24:21.828 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.828 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:21.828 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.828 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:21.828 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.819976+0000", 2023-11-25T04:24:21.828 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:21.829 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.829 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.829 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:21.829 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.819976+0000", 2023-11-25T04:24:21.829 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:21.829 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.829 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.829 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:21.829 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.819977+0000", 2023-11-25T04:24:21.829 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.8899999999999998e-07 2023-11-25T04:24:21.829 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.829 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.829 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:21.829 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.819982+0000", 2023-11-25T04:24:21.830 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5279999999999997e-06 2023-11-25T04:24:21.830 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.830 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.830 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:21.830 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.819992+0000", 2023-11-25T04:24:21.830 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0478e-05 2023-11-25T04:24:21.830 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.830 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.830 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:21.830 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.820004+0000", 2023-11-25T04:24:21.830 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1922e-05 2023-11-25T04:24:21.830 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.830 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.831 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:21.831 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.820051+0000", 2023-11-25T04:24:21.831 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7136999999999999e-05 2023-11-25T04:24:21.831 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.831 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.831 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:21.831 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.820173+0000", 2023-11-25T04:24:21.831 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000121513 2023-11-25T04:24:21.831 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.831 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.832 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:21.832 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.820298+0000", 2023-11-25T04:24:21.832 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012530899999999999 2023-11-25T04:24:21.832 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.832 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.832 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:21.832 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.820811+0000", 2023-11-25T04:24:21.832 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00051313900000000002 2023-11-25T04:24:21.832 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.832 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.832 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:21.832 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.822004+0000", 2023-11-25T04:24:21.832 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00119253 2023-11-25T04:24:21.832 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.833 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.833 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:21.833 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.822034+0000", 2023-11-25T04:24:21.833 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0759999999999997e-05 2023-11-25T04:24:21.833 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.833 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.833 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:21.833 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:16.822056+0000", 2023-11-25T04:24:21.833 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2123000000000001e-05 2023-11-25T04:24:21.833 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:21.833 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:21.833 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:21.833 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.834 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.834 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:118 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals,omap-rm-keys] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e44)", 2023-11-25T04:24:21.834 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:17.112648+0000", 2023-11-25T04:24:21.834 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.6582700809999995, 2023-11-25T04:24:21.834 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011250779000000001, 2023-11-25T04:24:21.834 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:21.834 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:21.834 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:21.834 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.834 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:21.835 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.112648+0000", 2023-11-25T04:24:21.835 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:21.835 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.835 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.835 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:21.835 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.112648+0000", 2023-11-25T04:24:21.835 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:21.835 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.835 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.835 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:21.835 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.112649+0000", 2023-11-25T04:24:21.835 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0070000000000001e-06 2023-11-25T04:24:21.835 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.835 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.836 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:21.836 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113182+0000", 2023-11-25T04:24:21.836 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00053268100000000002 2023-11-25T04:24:21.836 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.836 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.836 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:21.836 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113196+0000", 2023-11-25T04:24:21.836 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3742e-05 2023-11-25T04:24:21.836 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.836 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.836 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:21.836 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113398+0000", 2023-11-25T04:24:21.836 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000202726 2023-11-25T04:24:21.837 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.837 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.837 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:21.837 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113447+0000", 2023-11-25T04:24:21.837 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8655000000000001e-05 2023-11-25T04:24:21.837 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.837 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.837 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:21.837 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113529+0000", 2023-11-25T04:24:21.837 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1891999999999995e-05 2023-11-25T04:24:21.837 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.837 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.837 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:21.837 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123819+0000", 2023-11-25T04:24:21.838 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010290192 2023-11-25T04:24:21.838 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.838 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.838 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:21.838 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123852+0000", 2023-11-25T04:24:21.838 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2425999999999998e-05 2023-11-25T04:24:21.838 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.838 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.838 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:21.838 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123899+0000", 2023-11-25T04:24:21.838 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7457999999999999e-05 2023-11-25T04:24:21.838 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:21.838 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:21.839 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:21.839 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.839 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.839 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4561.0:2494 4.7 4:e1292348:::10000000bde.00000000:head [write 0~3041] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2023-11-25T04:24:21.839 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:17.120454+0000", 2023-11-25T04:24:21.839 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.6504646850000002, 2023-11-25T04:24:21.839 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0037327419999999998, 2023-11-25T04:24:21.839 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:21.839 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:21.839 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2023-11-25T04:24:21.839 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4561", 2023-11-25T04:24:21.839 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "192.168.0.1:0/1268434739", 2023-11-25T04:24:21.839 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 2494 2023-11-25T04:24:21.840 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.840 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:21.840 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.840 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:21.840 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.120454+0000", 2023-11-25T04:24:21.840 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:21.840 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.840 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.840 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:21.840 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.120454+0000", 2023-11-25T04:24:21.840 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:21.840 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.840 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.840 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:21.841 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.120456+0000", 2023-11-25T04:24:21.841 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0899999999999999e-06 2023-11-25T04:24:21.841 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.841 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.841 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:21.841 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.120466+0000", 2023-11-25T04:24:21.841 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0409e-05 2023-11-25T04:24:21.841 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.841 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.841 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:21.841 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.120484+0000", 2023-11-25T04:24:21.841 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7456e-05 2023-11-25T04:24:21.841 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.841 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.842 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:21.842 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.120495+0000", 2023-11-25T04:24:21.842 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1268e-05 2023-11-25T04:24:21.842 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.842 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.842 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:21.842 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.120568+0000", 2023-11-25T04:24:21.842 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3374999999999995e-05 2023-11-25T04:24:21.842 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.842 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.842 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:21.842 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.120707+0000", 2023-11-25T04:24:21.842 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013880699999999999 2023-11-25T04:24:21.842 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.843 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.843 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:21.843 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.120847+0000", 2023-11-25T04:24:21.843 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013949899999999999 2023-11-25T04:24:21.843 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.843 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.843 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:21.843 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.121627+0000", 2023-11-25T04:24:21.843 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00078049 2023-11-25T04:24:21.843 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.843 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.843 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:21.843 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.124139+0000", 2023-11-25T04:24:21.844 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0025116539999999999 2023-11-25T04:24:21.844 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.844 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.844 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:21.844 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.124168+0000", 2023-11-25T04:24:21.844 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9237000000000001e-05 2023-11-25T04:24:21.844 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.844 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.844 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:21.844 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.124186+0000", 2023-11-25T04:24:21.844 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8457e-05 2023-11-25T04:24:21.844 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:21.844 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:21.844 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:21.845 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.845 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.845 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4561.0:2497 4.37 4:ed6942ad:::10000000be1.00000000:head [write 0~1531] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2023-11-25T04:24:21.845 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:17.123256+0000", 2023-11-25T04:24:21.845 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.6476625619999998, 2023-11-25T04:24:21.845 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.003839617, 2023-11-25T04:24:21.845 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:21.845 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:21.845 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2023-11-25T04:24:21.845 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4561", 2023-11-25T04:24:21.845 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "192.168.0.1:0/1268434739", 2023-11-25T04:24:21.846 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 2497 2023-11-25T04:24:21.846 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.846 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:21.846 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.846 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:21.846 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123256+0000", 2023-11-25T04:24:21.846 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:21.846 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.846 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.846 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:21.846 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123256+0000", 2023-11-25T04:24:21.847 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:21.847 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.847 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.847 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:21.847 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123257+0000", 2023-11-25T04:24:21.847 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.062e-06 2023-11-25T04:24:21.847 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.847 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.847 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:21.847 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123262+0000", 2023-11-25T04:24:21.847 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3879999999999999e-06 2023-11-25T04:24:21.847 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.847 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.848 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:21.848 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123277+0000", 2023-11-25T04:24:21.848 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4416999999999999e-05 2023-11-25T04:24:21.848 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.848 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.848 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:21.848 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123285+0000", 2023-11-25T04:24:21.848 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1790000000000008e-06 2023-11-25T04:24:21.848 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.848 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.848 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:21.848 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123326+0000", 2023-11-25T04:24:21.848 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0707999999999997e-05 2023-11-25T04:24:21.848 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.849 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.849 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:21.849 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123396+0000", 2023-11-25T04:24:21.849 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0032999999999998e-05 2023-11-25T04:24:21.849 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.849 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.849 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:21.849 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123467+0000", 2023-11-25T04:24:21.849 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0937000000000002e-05 2023-11-25T04:24:21.849 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.849 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.849 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:21.849 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.124068+0000", 2023-11-25T04:24:21.849 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00060176299999999995 2023-11-25T04:24:21.850 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.850 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.850 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:21.850 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.127046+0000", 2023-11-25T04:24:21.850 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002977871 2023-11-25T04:24:21.850 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.850 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.850 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:21.850 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.127073+0000", 2023-11-25T04:24:21.850 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7118999999999999e-05 2023-11-25T04:24:21.850 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.850 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.850 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:21.851 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.127095+0000", 2023-11-25T04:24:21.851 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2140000000000001e-05 2023-11-25T04:24:21.851 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:21.851 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:21.851 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:21.851 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.851 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.851 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4561.0:2585 4.21 4:86018ef1:::10000000c39.00000000:head [write 0~267] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2023-11-25T04:24:21.851 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:17.199849+0000", 2023-11-25T04:24:21.851 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.571069531, 2023-11-25T04:24:21.851 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0027665739999999999, 2023-11-25T04:24:21.851 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:21.851 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:21.852 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2023-11-25T04:24:21.852 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4561", 2023-11-25T04:24:21.852 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "192.168.0.1:0/1268434739", 2023-11-25T04:24:21.852 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 2585 2023-11-25T04:24:21.852 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.852 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:21.852 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.852 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:21.852 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.199849+0000", 2023-11-25T04:24:21.852 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:21.852 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.852 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.852 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:21.853 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.199849+0000", 2023-11-25T04:24:21.853 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:21.853 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.853 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.853 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:21.853 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.199851+0000", 2023-11-25T04:24:21.853 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8840000000000001e-06 2023-11-25T04:24:21.853 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.853 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.853 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:21.853 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.199860+0000", 2023-11-25T04:24:21.853 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2560000000000008e-06 2023-11-25T04:24:21.853 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.853 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.854 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:21.854 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.199887+0000", 2023-11-25T04:24:21.854 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6713e-05 2023-11-25T04:24:21.854 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.854 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.854 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:21.854 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.199900+0000", 2023-11-25T04:24:21.854 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3343000000000001e-05 2023-11-25T04:24:21.854 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.854 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.854 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:21.854 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.200030+0000", 2023-11-25T04:24:21.854 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012986700000000001 2023-11-25T04:24:21.855 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.855 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.855 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:21.855 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.200105+0000", 2023-11-25T04:24:21.855 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5438999999999994e-05 2023-11-25T04:24:21.855 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.855 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.855 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:21.855 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.200179+0000", 2023-11-25T04:24:21.855 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3800999999999994e-05 2023-11-25T04:24:21.855 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.855 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.855 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:21.855 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.200592+0000", 2023-11-25T04:24:21.856 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00041309700000000002 2023-11-25T04:24:21.856 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.856 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.856 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:21.856 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.202567+0000", 2023-11-25T04:24:21.856 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019752260000000001 2023-11-25T04:24:21.856 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.856 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.856 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:21.856 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.202597+0000", 2023-11-25T04:24:21.856 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9462e-05 2023-11-25T04:24:21.856 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.856 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.857 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:21.857 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.202615+0000", 2023-11-25T04:24:21.857 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8485999999999999e-05 2023-11-25T04:24:21.857 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:21.857 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:21.857 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:21.857 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.857 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.857 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4561.0:3303 4.10 4:0a707b4d:::10000000f10.00000000:head [write 0~2465] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2023-11-25T04:24:21.857 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:17.817692+0000", 2023-11-25T04:24:21.857 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.9532261110000002, 2023-11-25T04:24:21.857 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0025840590000000001, 2023-11-25T04:24:21.857 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:21.858 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:21.858 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2023-11-25T04:24:21.858 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4561", 2023-11-25T04:24:21.858 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "192.168.0.1:0/1268434739", 2023-11-25T04:24:21.858 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 3303 2023-11-25T04:24:21.858 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.858 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:21.858 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.858 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:21.858 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.817692+0000", 2023-11-25T04:24:21.858 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:21.858 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.858 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.858 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:21.859 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.817692+0000", 2023-11-25T04:24:21.859 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:21.859 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.859 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.859 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:21.859 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.817693+0000", 2023-11-25T04:24:21.859 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2e-07 2023-11-25T04:24:21.859 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.859 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.859 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:21.859 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.817696+0000", 2023-11-25T04:24:21.859 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.207e-06 2023-11-25T04:24:21.860 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.860 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.860 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:21.860 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.817720+0000", 2023-11-25T04:24:21.860 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3575999999999999e-05 2023-11-25T04:24:21.860 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.860 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.860 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:21.860 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.817724+0000", 2023-11-25T04:24:21.860 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1219999999999997e-06 2023-11-25T04:24:21.860 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.860 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.860 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:21.860 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.817771+0000", 2023-11-25T04:24:21.861 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7193e-05 2023-11-25T04:24:21.861 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.861 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.861 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:21.861 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.817844+0000", 2023-11-25T04:24:21.861 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2735000000000001e-05 2023-11-25T04:24:21.861 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.861 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.861 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:21.861 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.817913+0000", 2023-11-25T04:24:21.861 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9249000000000001e-05 2023-11-25T04:24:21.861 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.861 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.862 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:21.862 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.818361+0000", 2023-11-25T04:24:21.862 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00044793200000000001 2023-11-25T04:24:21.862 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.862 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.862 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:21.862 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.820226+0000", 2023-11-25T04:24:21.862 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0018648440000000001 2023-11-25T04:24:21.862 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.862 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.862 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:21.862 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.820257+0000", 2023-11-25T04:24:21.862 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1188e-05 2023-11-25T04:24:21.862 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.863 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.863 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:21.863 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.820276+0000", 2023-11-25T04:24:21.863 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9493000000000001e-05 2023-11-25T04:24:21.863 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:21.863 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:21.863 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:21.863 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.863 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.863 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4561.0:3658 4.2d 4:b66d20b4:::10000001072.00000000:head [write 0~137] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2023-11-25T04:24:21.863 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:18.140586+0000", 2023-11-25T04:24:21.863 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.6303319900000002, 2023-11-25T04:24:21.863 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0022292969999999999, 2023-11-25T04:24:21.863 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:21.864 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:21.864 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2023-11-25T04:24:21.864 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4561", 2023-11-25T04:24:21.864 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "192.168.0.1:0/1268434739", 2023-11-25T04:24:21.864 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 3658 2023-11-25T04:24:21.864 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.864 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:21.864 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.864 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:21.864 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.140586+0000", 2023-11-25T04:24:21.864 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:21.864 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.864 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.865 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:21.865 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.140586+0000", 2023-11-25T04:24:21.865 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:21.865 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.865 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.865 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:21.865 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.140587+0000", 2023-11-25T04:24:21.865 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7599999999999997e-07 2023-11-25T04:24:21.865 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.865 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.865 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:21.865 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.140590+0000", 2023-11-25T04:24:21.865 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3189999999999999e-06 2023-11-25T04:24:21.865 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.866 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.866 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:21.866 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.140599+0000", 2023-11-25T04:24:21.866 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5609999999999999e-06 2023-11-25T04:24:21.866 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.866 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.866 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:21.866 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.140604+0000", 2023-11-25T04:24:21.866 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.775e-06 2023-11-25T04:24:21.866 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.866 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.866 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:21.866 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.140643+0000", 2023-11-25T04:24:21.866 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8943999999999998e-05 2023-11-25T04:24:21.867 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.867 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.867 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:21.867 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.140716+0000", 2023-11-25T04:24:21.867 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3638000000000002e-05 2023-11-25T04:24:21.867 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.867 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.867 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:21.867 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.140790+0000", 2023-11-25T04:24:21.867 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4164000000000003e-05 2023-11-25T04:24:21.867 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.867 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.867 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:21.868 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.141262+0000", 2023-11-25T04:24:21.868 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00047143700000000001 2023-11-25T04:24:21.868 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.868 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.868 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:21.868 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.142756+0000", 2023-11-25T04:24:21.868 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001493723 2023-11-25T04:24:21.868 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.868 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.868 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:21.868 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.142792+0000", 2023-11-25T04:24:21.869 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6455999999999998e-05 2023-11-25T04:24:21.869 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.869 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.869 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:21.869 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.142816+0000", 2023-11-25T04:24:21.869 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3604e-05 2023-11-25T04:24:21.869 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:21.869 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:21.869 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:21.869 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.869 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.869 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4561.0:3689 4.f 4:f0ecad07:::10000001091.00000000:head [write 0~886] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2023-11-25T04:24:21.869 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:18.169516+0000", 2023-11-25T04:24:21.870 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.6014026829999999, 2023-11-25T04:24:21.870 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0023233239999999999, 2023-11-25T04:24:21.870 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:21.870 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:21.870 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2023-11-25T04:24:21.870 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4561", 2023-11-25T04:24:21.870 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "192.168.0.1:0/1268434739", 2023-11-25T04:24:21.870 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 3689 2023-11-25T04:24:21.870 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.870 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:21.870 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.870 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:21.870 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.169516+0000", 2023-11-25T04:24:21.870 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:21.871 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.871 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.871 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:21.871 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.169516+0000", 2023-11-25T04:24:21.871 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:21.871 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.871 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.871 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:21.871 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.169516+0000", 2023-11-25T04:24:21.871 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6899999999999997e-07 2023-11-25T04:24:21.871 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.871 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.871 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:21.872 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.169520+0000", 2023-11-25T04:24:21.872 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6720000000000002e-06 2023-11-25T04:24:21.872 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.872 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.872 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:21.872 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.169530+0000", 2023-11-25T04:24:21.872 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.8039999999999996e-06 2023-11-25T04:24:21.872 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.872 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.872 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:21.872 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.169534+0000", 2023-11-25T04:24:21.872 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0940000000000001e-06 2023-11-25T04:24:21.872 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.872 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.873 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:21.873 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.169575+0000", 2023-11-25T04:24:21.873 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1090000000000001e-05 2023-11-25T04:24:21.873 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.873 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.873 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:21.873 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.169655+0000", 2023-11-25T04:24:21.873 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0006999999999995e-05 2023-11-25T04:24:21.873 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.873 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.873 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:21.873 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.169734+0000", 2023-11-25T04:24:21.873 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9419000000000007e-05 2023-11-25T04:24:21.873 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.874 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.874 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:21.874 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.170185+0000", 2023-11-25T04:24:21.874 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00045067499999999999 2023-11-25T04:24:21.874 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.874 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.874 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:21.874 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.171783+0000", 2023-11-25T04:24:21.874 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015982909999999999 2023-11-25T04:24:21.874 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.874 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.874 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:21.874 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.171820+0000", 2023-11-25T04:24:21.875 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6594999999999998e-05 2023-11-25T04:24:21.875 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.875 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.875 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:21.875 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:18.171839+0000", 2023-11-25T04:24:21.875 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9108000000000001e-05 2023-11-25T04:24:21.875 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:21.875 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:21.875 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:21.875 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.875 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.875 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4561.0:4900 4.4 4:206fea24:::1000000155d.00000000:head [write 0~776] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2023-11-25T04:24:21.875 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:21.242763+0000", 2023-11-25T04:24:21.875 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 0.52815501200000003, 2023-11-25T04:24:21.876 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0046403069999999998, 2023-11-25T04:24:21.876 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:21.876 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:21.876 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2023-11-25T04:24:21.876 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4561", 2023-11-25T04:24:21.876 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "192.168.0.1:0/1268434739", 2023-11-25T04:24:21.876 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4900 2023-11-25T04:24:21.876 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.876 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:21.876 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.876 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:21.876 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.242763+0000", 2023-11-25T04:24:21.876 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:21.876 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.877 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.877 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:21.877 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.242763+0000", 2023-11-25T04:24:21.877 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:21.877 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.877 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.877 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:21.877 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.242764+0000", 2023-11-25T04:24:21.877 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5000000000000002e-07 2023-11-25T04:24:21.877 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.877 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.877 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:21.877 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.242768+0000", 2023-11-25T04:24:21.877 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8210000000000003e-06 2023-11-25T04:24:21.878 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.878 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.878 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:21.878 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.242778+0000", 2023-11-25T04:24:21.878 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0530000000000001e-05 2023-11-25T04:24:21.878 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.878 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.878 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:21.878 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.242784+0000", 2023-11-25T04:24:21.878 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2349999999999999e-06 2023-11-25T04:24:21.878 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.878 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.879 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:21.879 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.242817+0000", 2023-11-25T04:24:21.879 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3803999999999997e-05 2023-11-25T04:24:21.879 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.879 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.879 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:21.879 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.242890+0000", 2023-11-25T04:24:21.879 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2875999999999998e-05 2023-11-25T04:24:21.879 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.879 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.879 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:21.879 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.242961+0000", 2023-11-25T04:24:21.879 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0395000000000006e-05 2023-11-25T04:24:21.879 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.880 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.880 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:21.880 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.243343+0000", 2023-11-25T04:24:21.880 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00038237699999999999 2023-11-25T04:24:21.880 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.880 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.880 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:21.880 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.247335+0000", 2023-11-25T04:24:21.880 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0039917989999999999 2023-11-25T04:24:21.880 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.880 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.880 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:21.880 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.247370+0000", 2023-11-25T04:24:21.881 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4671000000000001e-05 2023-11-25T04:24:21.881 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.881 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.881 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:21.881 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.247404+0000", 2023-11-25T04:24:21.881 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4048999999999999e-05 2023-11-25T04:24:21.881 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:21.881 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:21.881 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:21.881 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.881 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.881 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(mds.0.3:152 3.1a e46/16 3:5e41ab69:::200.00000005:head v 46'2, mlcod=0'0)", 2023-11-25T04:24:21.882 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:21.243441+0000", 2023-11-25T04:24:21.882 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 0.52747751600000004, 2023-11-25T04:24:21.882 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0090586009999999995, 2023-11-25T04:24:21.882 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:21.882 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:21.882 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:21.882 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.882 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:21.882 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.243441+0000", 2023-11-25T04:24:21.882 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:21.882 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.882 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.883 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:21.883 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.243441+0000", 2023-11-25T04:24:21.883 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:21.883 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.883 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.883 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:21.883 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.243441+0000", 2023-11-25T04:24:21.883 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4899999999999995e-07 2023-11-25T04:24:21.883 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.883 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.883 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:21.883 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.247245+0000", 2023-11-25T04:24:21.883 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0038034549999999999 2023-11-25T04:24:21.883 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.884 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.884 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:21.884 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.247257+0000", 2023-11-25T04:24:21.884 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2187999999999999e-05 2023-11-25T04:24:21.884 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.884 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.884 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:21.884 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.247262+0000", 2023-11-25T04:24:21.884 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0610000000000003e-06 2023-11-25T04:24:21.884 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.884 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.884 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:21.884 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.247302+0000", 2023-11-25T04:24:21.885 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9827999999999999e-05 2023-11-25T04:24:21.885 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.885 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.885 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:21.885 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.247317+0000", 2023-11-25T04:24:21.885 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4632e-05 2023-11-25T04:24:21.885 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.885 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.885 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:21.885 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.252471+0000", 2023-11-25T04:24:21.885 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0051540839999999997 2023-11-25T04:24:21.885 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.886 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.886 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:21.886 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.252499+0000", 2023-11-25T04:24:21.886 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8804000000000001e-05 2023-11-25T04:24:21.886 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:21.886 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:21.886 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:21.886 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.886 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.886 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4561.0:4904 4.21 4:855c761d:::10000001561.00000000:head [write 0~771] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2023-11-25T04:24:21.886 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:21.248232+0000", 2023-11-25T04:24:21.886 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 0.52268668900000004, 2023-11-25T04:24:21.886 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0028734820000000001, 2023-11-25T04:24:21.887 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:21.887 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:21.887 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2023-11-25T04:24:21.887 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4561", 2023-11-25T04:24:21.887 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "192.168.0.1:0/1268434739", 2023-11-25T04:24:21.887 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4904 2023-11-25T04:24:21.887 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.887 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:21.887 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.887 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:21.887 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.248232+0000", 2023-11-25T04:24:21.887 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:21.887 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.888 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.888 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:21.888 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.248232+0000", 2023-11-25T04:24:21.888 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:21.888 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.888 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.888 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:21.888 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.248232+0000", 2023-11-25T04:24:21.888 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7400000000000002e-07 2023-11-25T04:24:21.888 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.888 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.888 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:21.888 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.248237+0000", 2023-11-25T04:24:21.889 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5519999999999998e-06 2023-11-25T04:24:21.889 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.889 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.889 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:21.889 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.248247+0000", 2023-11-25T04:24:21.889 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6199999999999994e-06 2023-11-25T04:24:21.889 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.889 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.889 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:21.889 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.248252+0000", 2023-11-25T04:24:21.889 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.113e-06 2023-11-25T04:24:21.889 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.890 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.890 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:21.890 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.248294+0000", 2023-11-25T04:24:21.890 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2604000000000001e-05 2023-11-25T04:24:21.890 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.890 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.890 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:21.890 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.248390+0000", 2023-11-25T04:24:21.890 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.5622e-05 2023-11-25T04:24:21.890 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.890 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.890 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:21.890 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.248483+0000", 2023-11-25T04:24:21.891 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3201000000000004e-05 2023-11-25T04:24:21.891 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.891 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.891 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:21.891 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.248905+0000", 2023-11-25T04:24:21.891 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042145799999999999 2023-11-25T04:24:21.891 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.891 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.891 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:21.891 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.251041+0000", 2023-11-25T04:24:21.891 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0021359700000000001 2023-11-25T04:24:21.891 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.891 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.892 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:21.892 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.251071+0000", 2023-11-25T04:24:21.892 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0392e-05 2023-11-25T04:24:21.892 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:21.892 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:21.892 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:21.892 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.251105+0000", 2023-11-25T04:24:21.892 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4175999999999999e-05 2023-11-25T04:24:21.892 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:21.892 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:21.892 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:21.892 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:21.892 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:21.893 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:21.894 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:24:21.895 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:21.892+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:21.995 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:21.993+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:22.003 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:22.004 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:24:22.004 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2023-11-25T04:24:22.005 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:22.013 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:24:22.096 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:22.094+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:22.139 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2023-11-25T04:24:22.196 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:22.196 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:24:22.196 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2023-11-25T04:24:22.196 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2023-11-25T04:24:22.196 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:22.197 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:22.195+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:22.205 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:24:22.297 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:22.295+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:22.358 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:22.358 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2023-11-25T04:24:22.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2023-11-25T04:24:22.358 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2023-11-25T04:24:22.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.358 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:14 3.15 3:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:22.358 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:24:22.358 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 95.322083540999998, 2023-11-25T04:24:22.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010990437, 2023-11-25T04:24:22.358 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:22.359 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:22.359 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:22.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:22.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:24:22.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:22.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:22.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:24:22.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:22.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.360 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:22.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033383+0000", 2023-11-25T04:24:22.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9870000000000002e-06 2023-11-25T04:24:22.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.360 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:22.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033398+0000", 2023-11-25T04:24:22.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4569999999999999e-05 2023-11-25T04:24:22.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.360 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:22.361 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033411+0000", 2023-11-25T04:24:22.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3543999999999999e-05 2023-11-25T04:24:22.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:22.361 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033423+0000", 2023-11-25T04:24:22.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1905999999999999e-05 2023-11-25T04:24:22.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.362 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:22.362 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034433+0000", 2023-11-25T04:24:22.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00101022 2023-11-25T04:24:22.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.362 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:22.362 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034539+0000", 2023-11-25T04:24:22.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010597199999999999 2023-11-25T04:24:22.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:22.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034667+0000", 2023-11-25T04:24:22.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012765500000000001 2023-11-25T04:24:22.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:22.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037803+0000", 2023-11-25T04:24:22.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031362220000000001 2023-11-25T04:24:22.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:22.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:24:22.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0064964669999999997 2023-11-25T04:24:22.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.364 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:22.364 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044331+0000", 2023-11-25T04:24:22.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1544999999999997e-05 2023-11-25T04:24:22.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.364 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:22.364 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044372+0000", 2023-11-25T04:24:22.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0349000000000003e-05 2023-11-25T04:24:22.364 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:22.365 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:22.365 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:22.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.365 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:15 3.15 3:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:22.365 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:24:22.366 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 95.321423890999995, 2023-11-25T04:24:22.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010464362, 2023-11-25T04:24:22.366 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:22.366 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:22.366 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:22.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:22.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:24:22.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:22.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:22.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:24:22.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:22.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:22.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034042+0000", 2023-11-25T04:24:22.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0999999999999997e-07 2023-11-25T04:24:22.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:22.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034048+0000", 2023-11-25T04:24:22.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0390000000000001e-06 2023-11-25T04:24:22.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:22.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034059+0000", 2023-11-25T04:24:22.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1182999999999999e-05 2023-11-25T04:24:22.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:22.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034067+0000", 2023-11-25T04:24:22.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1559999999999995e-06 2023-11-25T04:24:22.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:22.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035005+0000", 2023-11-25T04:24:22.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00093747399999999999 2023-11-25T04:24:22.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:22.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035077+0000", 2023-11-25T04:24:22.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2747e-05 2023-11-25T04:24:22.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:22.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035219+0000", 2023-11-25T04:24:22.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000141877 2023-11-25T04:24:22.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:22.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037810+0000", 2023-11-25T04:24:22.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002590752 2023-11-25T04:24:22.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:22.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044437+0000", 2023-11-25T04:24:22.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066270110000000004 2023-11-25T04:24:22.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:22.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044481+0000", 2023-11-25T04:24:22.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4106000000000001e-05 2023-11-25T04:24:22.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:22.378 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044505+0000", 2023-11-25T04:24:22.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4207e-05 2023-11-25T04:24:22.378 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:22.378 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:22.378 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:22.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.378 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:16 3.1d 3:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:22.378 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:24:22.378 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 95.321382043, 2023-11-25T04:24:22.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010413923, 2023-11-25T04:24:22.379 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:22.379 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:22.379 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:22.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:22.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:24:22.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:22.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:22.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:24:22.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:22.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:22.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:24:22.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4499999999999997e-07 2023-11-25T04:24:22.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:22.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034087+0000", 2023-11-25T04:24:22.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9770000000000002e-06 2023-11-25T04:24:22.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:22.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034094+0000", 2023-11-25T04:24:22.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8809999999999998e-06 2023-11-25T04:24:22.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:22.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034100+0000", 2023-11-25T04:24:22.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8440000000000003e-06 2023-11-25T04:24:22.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:22.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034993+0000", 2023-11-25T04:24:22.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00089336799999999996 2023-11-25T04:24:22.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:22.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035099+0000", 2023-11-25T04:24:22.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106037 2023-11-25T04:24:22.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.385 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:22.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035276+0000", 2023-11-25T04:24:22.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017631799999999999 2023-11-25T04:24:22.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:22.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037815+0000", 2023-11-25T04:24:22.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025395959999999999 2023-11-25T04:24:22.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:22.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044420+0000", 2023-11-25T04:24:22.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066045749999999997 2023-11-25T04:24:22.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:22.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044463+0000", 2023-11-25T04:24:22.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3205999999999999e-05 2023-11-25T04:24:22.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:22.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044497+0000", 2023-11-25T04:24:22.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3476000000000003e-05 2023-11-25T04:24:22.388 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:22.388 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:22.388 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:22.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.388 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:17 3.1d 3:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:22.388 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:24:22.388 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 95.321356833999999, 2023-11-25T04:24:22.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01127356, 2023-11-25T04:24:22.388 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:22.388 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:22.389 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:22.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:22.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:24:22.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:22.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:22.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:24:22.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:22.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:22.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:24:22.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1100000000000001e-07 2023-11-25T04:24:22.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:22.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034112+0000", 2023-11-25T04:24:22.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8600000000000003e-06 2023-11-25T04:24:22.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:22.391 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034120+0000", 2023-11-25T04:24:22.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7700000000000001e-06 2023-11-25T04:24:22.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:22.391 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034127+0000", 2023-11-25T04:24:22.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0999999999999998e-06 2023-11-25T04:24:22.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:22.391 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035601+0000", 2023-11-25T04:24:22.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014739410000000001 2023-11-25T04:24:22.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:22.392 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035685+0000", 2023-11-25T04:24:22.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3677999999999994e-05 2023-11-25T04:24:22.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:22.392 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035827+0000", 2023-11-25T04:24:22.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014232599999999999 2023-11-25T04:24:22.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.392 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:22.392 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038725+0000", 2023-11-25T04:24:22.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028982779999999998 2023-11-25T04:24:22.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:22.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045323+0000", 2023-11-25T04:24:22.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0065980780000000003 2023-11-25T04:24:22.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:22.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045354+0000", 2023-11-25T04:24:22.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0463e-05 2023-11-25T04:24:22.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:22.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045381+0000", 2023-11-25T04:24:22.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7654999999999999e-05 2023-11-25T04:24:22.394 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:22.394 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:22.394 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:22.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.395 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:22 3.e 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:22.395 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:24:22.395 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 95.321330420999999, 2023-11-25T04:24:22.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010133422, 2023-11-25T04:24:22.395 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:22.395 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:22.395 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:22.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:22.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:24:22.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:22.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:22.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:24:22.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:22.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:22.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034135+0000", 2023-11-25T04:24:22.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-07 2023-11-25T04:24:22.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:22.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034138+0000", 2023-11-25T04:24:22.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6009999999999999e-06 2023-11-25T04:24:22.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:22.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034145+0000", 2023-11-25T04:24:22.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1169999999999998e-06 2023-11-25T04:24:22.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:22.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034150+0000", 2023-11-25T04:24:22.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9080000000000003e-06 2023-11-25T04:24:22.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:22.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034209+0000", 2023-11-25T04:24:22.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8149000000000002e-05 2023-11-25T04:24:22.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:22.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034415+0000", 2023-11-25T04:24:22.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020637699999999999 2023-11-25T04:24:22.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:22.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034584+0000", 2023-11-25T04:24:22.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016870399999999999 2023-11-25T04:24:22.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:22.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.036979+0000", 2023-11-25T04:24:22.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023957509999999998 2023-11-25T04:24:22.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:22.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044161+0000", 2023-11-25T04:24:22.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0071811590000000003 2023-11-25T04:24:22.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:22.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044224+0000", 2023-11-25T04:24:22.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3454000000000006e-05 2023-11-25T04:24:22.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:22.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044268+0000", 2023-11-25T04:24:22.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3761999999999999e-05 2023-11-25T04:24:22.400 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:22.400 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:22.400 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:22.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.401 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:23 3.e 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:22.401 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:24:22.401 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 95.321299011999997, 2023-11-25T04:24:22.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010281676, 2023-11-25T04:24:22.401 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:22.401 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:22.401 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:22.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:22.401 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:24:22.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:22.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:22.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:24:22.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:22.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:22.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:24:22.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5799999999999999e-07 2023-11-25T04:24:22.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:22.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034169+0000", 2023-11-25T04:24:22.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1879999999999998e-06 2023-11-25T04:24:22.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:22.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034177+0000", 2023-11-25T04:24:22.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7020000000000001e-06 2023-11-25T04:24:22.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:22.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034184+0000", 2023-11-25T04:24:22.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3910000000000003e-06 2023-11-25T04:24:22.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:22.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034924+0000", 2023-11-25T04:24:22.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00074032999999999998 2023-11-25T04:24:22.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:22.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034991+0000", 2023-11-25T04:24:22.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7347000000000004e-05 2023-11-25T04:24:22.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:22.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035096+0000", 2023-11-25T04:24:22.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010488 2023-11-25T04:24:22.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:22.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037588+0000", 2023-11-25T04:24:22.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024918309999999999 2023-11-25T04:24:22.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:22.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044367+0000", 2023-11-25T04:24:22.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067787589999999997 2023-11-25T04:24:22.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:22.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044415+0000", 2023-11-25T04:24:22.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7803000000000003e-05 2023-11-25T04:24:22.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:22.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044447+0000", 2023-11-25T04:24:22.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2886999999999999e-05 2023-11-25T04:24:22.407 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:22.408 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:22.408 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:22.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.408 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:30 3.15 3:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:22.408 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:24:22.408 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 95.321240571999994, 2023-11-25T04:24:22.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010365070000000001, 2023-11-25T04:24:22.408 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:22.408 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:22.408 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:22.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:22.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:24:22.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:22.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:22.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:24:22.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:22.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:22.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034225+0000", 2023-11-25T04:24:22.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7400000000000002e-07 2023-11-25T04:24:22.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:22.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034229+0000", 2023-11-25T04:24:22.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3100000000000002e-06 2023-11-25T04:24:22.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:22.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034239+0000", 2023-11-25T04:24:22.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9140000000000003e-06 2023-11-25T04:24:22.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:22.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034246+0000", 2023-11-25T04:24:22.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0149999999999998e-06 2023-11-25T04:24:22.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:22.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035572+0000", 2023-11-25T04:24:22.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001325655 2023-11-25T04:24:22.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.414 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:22.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035730+0000", 2023-11-25T04:24:22.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015827899999999999 2023-11-25T04:24:22.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:22.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035857+0000", 2023-11-25T04:24:22.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012644699999999999 2023-11-25T04:24:22.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:22.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038737+0000", 2023-11-25T04:24:22.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002880077 2023-11-25T04:24:22.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:22.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044540+0000", 2023-11-25T04:24:22.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058034239999999997 2023-11-25T04:24:22.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:22.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044569+0000", 2023-11-25T04:24:22.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8388999999999999e-05 2023-11-25T04:24:22.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:22.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044589+0000", 2023-11-25T04:24:22.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0686000000000002e-05 2023-11-25T04:24:22.421 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:22.421 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:22.421 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:22.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.421 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:33 3.1b 3:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:22.421 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:24:22.421 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 95.321195074000002, 2023-11-25T04:24:22.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099754049999999997, 2023-11-25T04:24:22.421 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:22.421 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:22.422 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:22.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:22.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:24:22.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:22.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:22.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:24:22.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:22.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:22.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:24:22.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4799999999999998e-07 2023-11-25T04:24:22.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:22.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034273+0000", 2023-11-25T04:24:22.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7819999999999998e-06 2023-11-25T04:24:22.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:22.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034285+0000", 2023-11-25T04:24:22.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2058e-05 2023-11-25T04:24:22.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:22.425 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034291+0000", 2023-11-25T04:24:22.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0680000000000002e-06 2023-11-25T04:24:22.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:22.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034370+0000", 2023-11-25T04:24:22.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8452e-05 2023-11-25T04:24:22.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:22.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034477+0000", 2023-11-25T04:24:22.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107471 2023-11-25T04:24:22.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:22.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044245+0000", 2023-11-25T04:24:22.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0097679259999999997 2023-11-25T04:24:22.427 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:22.427 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:22.427 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:22.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.428 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:152 3.1a 3:5e41ab69:::200.00000005:head [write 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:22.428 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:21.237401+0000", 2023-11-25T04:24:22.428 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.118064035, 2023-11-25T04:24:22.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015253802, 2023-11-25T04:24:22.428 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:22.428 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:22.428 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:22.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:22.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.237401+0000", 2023-11-25T04:24:22.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:22.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:22.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.237401+0000", 2023-11-25T04:24:22.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:22.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.430 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:22.430 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.237402+0000", 2023-11-25T04:24:22.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3699999999999999e-07 2023-11-25T04:24:22.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.430 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:22.430 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243182+0000", 2023-11-25T04:24:22.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057801270000000004 2023-11-25T04:24:22.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:22.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243193+0000", 2023-11-25T04:24:22.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1221e-05 2023-11-25T04:24:22.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:22.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243198+0000", 2023-11-25T04:24:22.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3739999999999997e-06 2023-11-25T04:24:22.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:22.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243246+0000", 2023-11-25T04:24:22.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7667999999999998e-05 2023-11-25T04:24:22.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:22.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243292+0000", 2023-11-25T04:24:22.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5794000000000002e-05 2023-11-25T04:24:22.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:22.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243360+0000", 2023-11-25T04:24:22.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8479000000000006e-05 2023-11-25T04:24:22.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:22.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.250667+0000", 2023-11-25T04:24:22.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.007306618 2023-11-25T04:24:22.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:22.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.252598+0000", 2023-11-25T04:24:22.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019315129999999999 2023-11-25T04:24:22.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:22.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.252631+0000", 2023-11-25T04:24:22.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2925000000000001e-05 2023-11-25T04:24:22.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:22.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.252655+0000", 2023-11-25T04:24:22.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3246000000000001e-05 2023-11-25T04:24:22.438 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:22.439 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:22.439 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:22.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.439 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:154 3.1e e46/16 3:7bf7e7b5:::200.00000006:head v 46'2, mlcod=0'0)", 2023-11-25T04:24:22.439 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:21.892635+0000", 2023-11-25T04:24:22.439 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.46283016300000002, 2023-11-25T04:24:22.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011376196999999999, 2023-11-25T04:24:22.439 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:22.439 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:22.439 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:22.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:22.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.892635+0000", 2023-11-25T04:24:22.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:22.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:22.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.892635+0000", 2023-11-25T04:24:22.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:22.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:22.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.892635+0000", 2023-11-25T04:24:22.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6300000000000005e-07 2023-11-25T04:24:22.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:22.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.896756+0000", 2023-11-25T04:24:22.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041204809999999996 2023-11-25T04:24:22.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:22.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.896771+0000", 2023-11-25T04:24:22.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5231999999999999e-05 2023-11-25T04:24:22.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:22.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.896778+0000", 2023-11-25T04:24:22.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3309999999999998e-06 2023-11-25T04:24:22.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:22.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.896842+0000", 2023-11-25T04:24:22.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3800999999999994e-05 2023-11-25T04:24:22.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:22.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.896865+0000", 2023-11-25T04:24:22.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2762e-05 2023-11-25T04:24:22.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:22.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.903974+0000", 2023-11-25T04:24:22.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0071089220000000002 2023-11-25T04:24:22.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:22.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.904011+0000", 2023-11-25T04:24:22.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7104999999999998e-05 2023-11-25T04:24:22.443 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:22.443 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:22.443 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:22.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.443 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:1074 4.19 4:9ae72bb2:::10000001201.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:22.444 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:22.155648+0000", 2023-11-25T04:24:22.444 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.19981657899999999, 2023-11-25T04:24:22.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0095681949999999998, 2023-11-25T04:24:22.444 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:22.444 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:22.444 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:22.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:22.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.155648+0000", 2023-11-25T04:24:22.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:22.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:22.445 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.155648+0000", 2023-11-25T04:24:22.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:22.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.445 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:22.445 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.155649+0000", 2023-11-25T04:24:22.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9500000000000001e-07 2023-11-25T04:24:22.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.445 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:22.445 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.155653+0000", 2023-11-25T04:24:22.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5949999999999999e-06 2023-11-25T04:24:22.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:22.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.155661+0000", 2023-11-25T04:24:22.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7809999999999994e-06 2023-11-25T04:24:22.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:22.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.155669+0000", 2023-11-25T04:24:22.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1009999999999992e-06 2023-11-25T04:24:22.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.449 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:22.449 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.155723+0000", 2023-11-25T04:24:22.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3066999999999999e-05 2023-11-25T04:24:22.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.449 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:22.449 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.155794+0000", 2023-11-25T04:24:22.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1227000000000001e-05 2023-11-25T04:24:22.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.450 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:22.450 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.155884+0000", 2023-11-25T04:24:22.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0614999999999999e-05 2023-11-25T04:24:22.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.451 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:22.451 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.157344+0000", 2023-11-25T04:24:22.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001459728 2023-11-25T04:24:22.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:22.452 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.165138+0000", 2023-11-25T04:24:22.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0077934839999999998 2023-11-25T04:24:22.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:22.452 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.165182+0000", 2023-11-25T04:24:22.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4036999999999999e-05 2023-11-25T04:24:22.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.453 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:22.453 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.165216+0000", 2023-11-25T04:24:22.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4764999999999997e-05 2023-11-25T04:24:22.453 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:22.454 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:22.454 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:22.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.454 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:1082 4.15 4:a87ae726:::10000001209.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:22.454 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:22.156227+0000", 2023-11-25T04:24:22.454 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.19923814300000001, 2023-11-25T04:24:22.454 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010001013, 2023-11-25T04:24:22.454 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:22.454 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:22.455 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:22.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:22.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156227+0000", 2023-11-25T04:24:22.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:22.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:22.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156227+0000", 2023-11-25T04:24:22.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:22.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:22.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156227+0000", 2023-11-25T04:24:22.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8500000000000002e-07 2023-11-25T04:24:22.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:22.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156230+0000", 2023-11-25T04:24:22.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1300000000000001e-06 2023-11-25T04:24:22.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:22.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156238+0000", 2023-11-25T04:24:22.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8949999999999998e-06 2023-11-25T04:24:22.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.458 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.458 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:22.458 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156244+0000", 2023-11-25T04:24:22.458 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5339999999999999e-06 2023-11-25T04:24:22.458 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.459 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:22.459 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156292+0000", 2023-11-25T04:24:22.459 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8359000000000002e-05 2023-11-25T04:24:22.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:22.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156363+0000", 2023-11-25T04:24:22.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1056999999999994e-05 2023-11-25T04:24:22.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:22.461 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156448+0000", 2023-11-25T04:24:22.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5100999999999997e-05 2023-11-25T04:24:22.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:22.461 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.158964+0000", 2023-11-25T04:24:22.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025160310000000002 2023-11-25T04:24:22.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.462 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:22.462 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.166062+0000", 2023-11-25T04:24:22.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0070979989999999998 2023-11-25T04:24:22.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.463 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:22.463 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.166165+0000", 2023-11-25T04:24:22.463 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010280099999999999 2023-11-25T04:24:22.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.464 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:22.464 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.166228+0000", 2023-11-25T04:24:22.464 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2620999999999996e-05 2023-11-25T04:24:22.464 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:22.464 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:22.465 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:22.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.465 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:1086 4.34 4:2d0cb294:::1000000120d.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:22.466 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:22.156608+0000", 2023-11-25T04:24:22.466 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.19885639799999999, 2023-11-25T04:24:22.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0096961140000000005, 2023-11-25T04:24:22.466 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:22.466 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:22.466 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:22.467 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.467 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:22.467 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156608+0000", 2023-11-25T04:24:22.467 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:22.467 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.467 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.467 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:22.467 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156608+0000", 2023-11-25T04:24:22.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:22.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.468 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:22.468 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156609+0000", 2023-11-25T04:24:22.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8899999999999999e-07 2023-11-25T04:24:22.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.468 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:22.468 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156612+0000", 2023-11-25T04:24:22.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6730000000000001e-06 2023-11-25T04:24:22.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.469 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.469 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:22.469 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156620+0000", 2023-11-25T04:24:22.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7159999999999992e-06 2023-11-25T04:24:22.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.469 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.469 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:22.469 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156626+0000", 2023-11-25T04:24:22.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.631e-06 2023-11-25T04:24:22.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.469 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.469 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:22.470 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156953+0000", 2023-11-25T04:24:22.470 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00032727 2023-11-25T04:24:22.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.470 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:22.470 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.157042+0000", 2023-11-25T04:24:22.470 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8349000000000004e-05 2023-11-25T04:24:22.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.470 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:22.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.166033+0000", 2023-11-25T04:24:22.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0089916370000000002 2023-11-25T04:24:22.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.471 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:22.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.166084+0000", 2023-11-25T04:24:22.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0538999999999998e-05 2023-11-25T04:24:22.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.471 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:22.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.166304+0000", 2023-11-25T04:24:22.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022070999999999999 2023-11-25T04:24:22.471 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:22.472 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:22.472 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:22.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.472 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:1088 4.3 4:c0636c6d:::1000000120f.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:22.472 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:22.156639+0000", 2023-11-25T04:24:22.472 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.19882556900000001, 2023-11-25T04:24:22.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010613015, 2023-11-25T04:24:22.473 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:22.473 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:22.473 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:22.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.473 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:22.473 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156639+0000", 2023-11-25T04:24:22.473 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:22.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.473 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:22.473 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156639+0000", 2023-11-25T04:24:22.473 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:22.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.474 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.474 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:22.474 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156640+0000", 2023-11-25T04:24:22.474 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7800000000000002e-07 2023-11-25T04:24:22.474 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.474 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.474 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:22.474 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156641+0000", 2023-11-25T04:24:22.474 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7719999999999999e-06 2023-11-25T04:24:22.474 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.474 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.474 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:22.474 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156647+0000", 2023-11-25T04:24:22.475 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0399999999999998e-06 2023-11-25T04:24:22.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.475 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.475 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:22.475 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156654+0000", 2023-11-25T04:24:22.475 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8900000000000001e-06 2023-11-25T04:24:22.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.475 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.475 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:22.475 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156942+0000", 2023-11-25T04:24:22.475 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028773999999999998 2023-11-25T04:24:22.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.475 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:22.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.157012+0000", 2023-11-25T04:24:22.476 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0290000000000006e-05 2023-11-25T04:24:22.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.476 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:22.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.167143+0000", 2023-11-25T04:24:22.476 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010130208999999999 2023-11-25T04:24:22.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.476 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:22.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.167183+0000", 2023-11-25T04:24:22.476 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0352000000000003e-05 2023-11-25T04:24:22.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.477 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:22.477 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.167252+0000", 2023-11-25T04:24:22.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9244000000000003e-05 2023-11-25T04:24:22.477 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:22.477 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:22.477 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:22.478 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.478 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.478 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:1090 4.32 4:4e235bd8:::10000001211.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:22.478 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:22.156665+0000", 2023-11-25T04:24:22.478 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.19879962700000001, 2023-11-25T04:24:22.478 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0098969169999999999, 2023-11-25T04:24:22.479 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:22.479 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:22.479 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:22.479 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.479 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:22.479 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156665+0000", 2023-11-25T04:24:22.479 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:22.479 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.479 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.479 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:22.480 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156665+0000", 2023-11-25T04:24:22.480 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:22.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.480 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.480 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:22.480 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156666+0000", 2023-11-25T04:24:22.480 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-07 2023-11-25T04:24:22.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.480 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.480 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:22.480 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156668+0000", 2023-11-25T04:24:22.480 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9379999999999999e-06 2023-11-25T04:24:22.481 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.481 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:22.481 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156675+0000", 2023-11-25T04:24:22.481 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4170000000000003e-06 2023-11-25T04:24:22.481 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.481 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:22.481 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156680+0000", 2023-11-25T04:24:22.481 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2000000000000002e-06 2023-11-25T04:24:22.481 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.482 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:22.482 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156949+0000", 2023-11-25T04:24:22.482 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026843800000000002 2023-11-25T04:24:22.482 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.482 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.482 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:22.482 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.157013+0000", 2023-11-25T04:24:22.482 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4424000000000005e-05 2023-11-25T04:24:22.482 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.482 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.482 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:22.483 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.166438+0000", 2023-11-25T04:24:22.483 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0094252150000000007 2023-11-25T04:24:22.483 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.483 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.483 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:22.483 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.166483+0000", 2023-11-25T04:24:22.483 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4916999999999997e-05 2023-11-25T04:24:22.483 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.483 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.483 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:22.483 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.166562+0000", 2023-11-25T04:24:22.484 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8928000000000001e-05 2023-11-25T04:24:22.484 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:22.484 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:22.484 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:22.484 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.484 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.484 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:1092 4.1 4:80d14d92:::10000001213.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:22.484 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:22.156698+0000", 2023-11-25T04:24:22.484 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.198767047, 2023-11-25T04:24:22.484 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0095673489999999993, 2023-11-25T04:24:22.484 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:22.484 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:22.485 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:22.485 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.485 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:22.485 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156698+0000", 2023-11-25T04:24:22.485 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:22.485 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.485 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.485 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:22.485 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156698+0000", 2023-11-25T04:24:22.485 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:22.485 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.485 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.485 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:22.486 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156698+0000", 2023-11-25T04:24:22.486 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7599999999999997e-07 2023-11-25T04:24:22.486 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.486 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.486 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:22.486 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156700+0000", 2023-11-25T04:24:22.486 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9199999999999998e-06 2023-11-25T04:24:22.486 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.486 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.487 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:22.487 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156718+0000", 2023-11-25T04:24:22.487 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7730000000000001e-05 2023-11-25T04:24:22.487 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.487 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.487 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:22.487 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156729+0000", 2023-11-25T04:24:22.487 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0900999999999999e-05 2023-11-25T04:24:22.487 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.487 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.488 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:22.488 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156945+0000", 2023-11-25T04:24:22.488 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021670500000000001 2023-11-25T04:24:22.488 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.488 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.488 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:22.488 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.157021+0000", 2023-11-25T04:24:22.488 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5986000000000001e-05 2023-11-25T04:24:22.489 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.489 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.489 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:22.489 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.166097+0000", 2023-11-25T04:24:22.489 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0090757649999999995 2023-11-25T04:24:22.489 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.489 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.489 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:22.489 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.166154+0000", 2023-11-25T04:24:22.490 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6805000000000002e-05 2023-11-25T04:24:22.490 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.490 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.490 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:22.490 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.166265+0000", 2023-11-25T04:24:22.490 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000111061 2023-11-25T04:24:22.490 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:22.490 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:22.490 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:22.491 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.491 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.491 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:1096 4.3 4:c1f14f5b:::10000001217.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:22.491 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:22.157851+0000", 2023-11-25T04:24:22.491 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.197613552, 2023-11-25T04:24:22.491 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010572455999999999, 2023-11-25T04:24:22.491 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:22.491 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:22.491 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:22.491 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.491 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:22.492 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.157851+0000", 2023-11-25T04:24:22.492 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:22.492 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.492 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.492 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:22.492 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.157851+0000", 2023-11-25T04:24:22.492 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:22.492 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.492 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.492 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:22.492 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.157852+0000", 2023-11-25T04:24:22.492 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6199999999999997e-07 2023-11-25T04:24:22.492 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.493 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.493 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:22.493 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.157879+0000", 2023-11-25T04:24:22.493 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6911e-05 2023-11-25T04:24:22.493 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.493 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.493 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:22.493 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.157887+0000", 2023-11-25T04:24:22.493 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5059999999999995e-06 2023-11-25T04:24:22.493 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.494 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.494 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:22.494 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.157895+0000", 2023-11-25T04:24:22.494 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2799999999999998e-06 2023-11-25T04:24:22.494 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.494 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.494 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:22.494 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.157948+0000", 2023-11-25T04:24:22.494 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2901000000000001e-05 2023-11-25T04:24:22.494 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.495 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.495 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:22.495 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.158027+0000", 2023-11-25T04:24:22.495 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9609000000000003e-05 2023-11-25T04:24:22.495 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.495 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.495 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:22.495 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.167260+0000", 2023-11-25T04:24:22.495 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0092331240000000005 2023-11-25T04:24:22.495 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.495 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.495 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:22.496 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.168357+0000", 2023-11-25T04:24:22.496 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0010965090000000001 2023-11-25T04:24:22.496 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.496 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.496 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:22.496 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.168424+0000", 2023-11-25T04:24:22.496 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6853999999999994e-05 2023-11-25T04:24:22.496 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:22.496 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:22.496 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:22.496 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.497 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.497 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:1104 4.27 4:e4eba14d:::1000000121f.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:22.497 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:22.158428+0000", 2023-11-25T04:24:22.497 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.19703709799999999, 2023-11-25T04:24:22.497 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0095411690000000004, 2023-11-25T04:24:22.497 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:22.497 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:22.497 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:22.497 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.498 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:22.498 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.158428+0000", 2023-11-25T04:24:22.498 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:22.498 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.498 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.498 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:22.498 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.158428+0000", 2023-11-25T04:24:22.498 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:22.498 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.498 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.498 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:22.499 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.158428+0000", 2023-11-25T04:24:22.499 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9200000000000001e-07 2023-11-25T04:24:22.499 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.499 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.499 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:22.499 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.158431+0000", 2023-11-25T04:24:22.500 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.531e-06 2023-11-25T04:24:22.500 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.500 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.500 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:22.500 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.158437+0000", 2023-11-25T04:24:22.500 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2380000000000002e-06 2023-11-25T04:24:22.500 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.500 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.500 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:22.500 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.158441+0000", 2023-11-25T04:24:22.500 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4469999999999999e-06 2023-11-25T04:24:22.501 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.501 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.501 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:22.501 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.158477+0000", 2023-11-25T04:24:22.501 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5543000000000002e-05 2023-11-25T04:24:22.501 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.501 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.501 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:22.501 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.158550+0000", 2023-11-25T04:24:22.501 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3289999999999998e-05 2023-11-25T04:24:22.501 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.502 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.502 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:22.502 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.167857+0000", 2023-11-25T04:24:22.502 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0093063420000000004 2023-11-25T04:24:22.502 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.502 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.502 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:22.502 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.167899+0000", 2023-11-25T04:24:22.502 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2400000000000001e-05 2023-11-25T04:24:22.502 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.502 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.503 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:22.503 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.167969+0000", 2023-11-25T04:24:22.503 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9886000000000001e-05 2023-11-25T04:24:22.503 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:22.503 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:22.503 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:22.503 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.503 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.503 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:1105 4.23 e46/17 4:c6ff07b4:::10000001220.00000000:head v 46'106, mlcod=46'104)", 2023-11-25T04:24:22.503 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:22.159179+0000", 2023-11-25T04:24:22.503 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.196286237, 2023-11-25T04:24:22.503 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0093609680000000008, 2023-11-25T04:24:22.503 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:22.504 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:22.504 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:22.504 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.504 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:22.504 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.159179+0000", 2023-11-25T04:24:22.504 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:22.504 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.504 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.504 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:22.504 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.159179+0000", 2023-11-25T04:24:22.504 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:22.504 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.505 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.505 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:22.505 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.159179+0000", 2023-11-25T04:24:22.505 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1399999999999997e-07 2023-11-25T04:24:22.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.505 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.505 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:22.505 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.159183+0000", 2023-11-25T04:24:22.505 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0579999999999999e-06 2023-11-25T04:24:22.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.506 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.506 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:22.506 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.159192+0000", 2023-11-25T04:24:22.506 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6300000000000004e-06 2023-11-25T04:24:22.506 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.506 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.506 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:22.507 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.159197+0000", 2023-11-25T04:24:22.507 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9180000000000002e-06 2023-11-25T04:24:22.507 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.507 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.507 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:22.508 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.159234+0000", 2023-11-25T04:24:22.508 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7490999999999997e-05 2023-11-25T04:24:22.508 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.508 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.508 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:22.509 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.159248+0000", 2023-11-25T04:24:22.509 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3991e-05 2023-11-25T04:24:22.509 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.509 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.509 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:22.509 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.168426+0000", 2023-11-25T04:24:22.510 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0091775869999999992 2023-11-25T04:24:22.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.510 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.510 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:22.510 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.168539+0000", 2023-11-25T04:24:22.510 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000113879 2023-11-25T04:24:22.511 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:22.511 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:22.511 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:22.511 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.511 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.511 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:1119 4.3f 4:fec262f2:::1000000122e.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:22.512 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:22.159421+0000", 2023-11-25T04:24:22.512 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.196044101, 2023-11-25T04:24:22.512 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0092470220000000006, 2023-11-25T04:24:22.512 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:22.512 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:22.512 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:22.513 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.513 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:22.513 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.159421+0000", 2023-11-25T04:24:22.513 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:22.513 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.513 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.513 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:22.514 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.159421+0000", 2023-11-25T04:24:22.514 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:22.514 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.514 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.514 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:22.514 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.159423+0000", 2023-11-25T04:24:22.515 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9860000000000001e-06 2023-11-25T04:24:22.515 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.515 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.515 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:22.515 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.159425+0000", 2023-11-25T04:24:22.515 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.661e-06 2023-11-25T04:24:22.515 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.516 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.516 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:22.516 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.159431+0000", 2023-11-25T04:24:22.516 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4940000000000003e-06 2023-11-25T04:24:22.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.516 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.517 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:22.517 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.159435+0000", 2023-11-25T04:24:22.517 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5680000000000001e-06 2023-11-25T04:24:22.517 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.517 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.517 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:22.518 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.165146+0000", 2023-11-25T04:24:22.518 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057110850000000003 2023-11-25T04:24:22.518 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.518 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.518 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:22.519 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.165216+0000", 2023-11-25T04:24:22.519 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9227000000000007e-05 2023-11-25T04:24:22.519 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.519 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.519 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:22.519 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.165304+0000", 2023-11-25T04:24:22.520 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8164000000000005e-05 2023-11-25T04:24:22.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.520 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.520 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:22.520 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.167359+0000", 2023-11-25T04:24:22.520 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020552349999999999 2023-11-25T04:24:22.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.521 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.521 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:22.521 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.168545+0000", 2023-11-25T04:24:22.521 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011858509999999999 2023-11-25T04:24:22.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.522 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.522 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:22.522 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.168634+0000", 2023-11-25T04:24:22.522 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8832999999999995e-05 2023-11-25T04:24:22.522 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:22.522 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:22.523 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:22.523 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.168668+0000", 2023-11-25T04:24:22.523 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3918000000000003e-05 2023-11-25T04:24:22.523 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:22.523 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:22.523 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:22.524 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:22.524 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:22.524 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:22.526 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:24:22.527 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:22.524+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:22.566 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-11-25T04:24:22.626 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:22.625+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:22.637 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:22.637 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:24:22.637 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2023-11-25T04:24:22.638 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:22.647 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:24:22.697 INFO:teuthology.orchestra.run.smithi067.stderr:Error ENXIO: problem getting command descriptions from osd.0 2023-11-25T04:24:22.717 INFO:teuthology.orchestra.run.smithi067.stdout:osd.1: {} 2023-11-25T04:24:22.717 INFO:teuthology.orchestra.run.smithi067.stderr:osd.1: osd_enable_op_tracker = '' 2023-11-25T04:24:22.727 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:22.726+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:22.737 INFO:teuthology.orchestra.run.smithi067.stdout:osd.2: {} 2023-11-25T04:24:22.737 INFO:teuthology.orchestra.run.smithi067.stderr:osd.2: osd_enable_op_tracker = '' 2023-11-25T04:24:22.740 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-11-25T04:24:22.740 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-11-25T04:24:22.799 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:22.800 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:22.809 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:24:22.829 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:22.828+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:22.924 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:22.924 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:22.929 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:22.928+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:22.933 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:24:22.942 INFO:tasks.workunit.client.0.smithi067.stderr:Note: switching to '12973fc04fd10d4af086901e10ffa8e48866b735'. 2023-11-25T04:24:22.942 INFO:tasks.workunit.client.0.smithi067.stderr: 2023-11-25T04:24:22.942 INFO:tasks.workunit.client.0.smithi067.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2023-11-25T04:24:22.943 INFO:tasks.workunit.client.0.smithi067.stderr:changes and commit them, and you can discard any commits you make in this 2023-11-25T04:24:22.943 INFO:tasks.workunit.client.0.smithi067.stderr:state without impacting any branches by switching back to a branch. 2023-11-25T04:24:22.943 INFO:tasks.workunit.client.0.smithi067.stderr: 2023-11-25T04:24:22.943 INFO:tasks.workunit.client.0.smithi067.stderr:If you want to create a new branch to retain commits you create, you may 2023-11-25T04:24:22.943 INFO:tasks.workunit.client.0.smithi067.stderr:do so (now or later) by using -c with the switch command. Example: 2023-11-25T04:24:22.943 INFO:tasks.workunit.client.0.smithi067.stderr: 2023-11-25T04:24:22.943 INFO:tasks.workunit.client.0.smithi067.stderr: git switch -c 2023-11-25T04:24:22.943 INFO:tasks.workunit.client.0.smithi067.stderr: 2023-11-25T04:24:22.943 INFO:tasks.workunit.client.0.smithi067.stderr:Or undo this operation with: 2023-11-25T04:24:22.943 INFO:tasks.workunit.client.0.smithi067.stderr: 2023-11-25T04:24:22.943 INFO:tasks.workunit.client.0.smithi067.stderr: git switch - 2023-11-25T04:24:22.943 INFO:tasks.workunit.client.0.smithi067.stderr: 2023-11-25T04:24:22.943 INFO:tasks.workunit.client.0.smithi067.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2023-11-25T04:24:22.944 INFO:tasks.workunit.client.0.smithi067.stderr: 2023-11-25T04:24:22.944 INFO:tasks.workunit.client.0.smithi067.stderr:HEAD is now at 12973fc0 btrfs/022: Correct seq number in golden output 2023-11-25T04:24:22.945 INFO:tasks.workunit.client.0.smithi067.stderr:+ make -j4 2023-11-25T04:24:22.959 INFO:tasks.workunit.client.0.smithi067.stdout:libtoolize -cfi 2023-11-25T04:24:23.030 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:23.029+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:23.037 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:23.037 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:23.046 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:24:23.126 INFO:teuthology.orchestra.run.smithi067.stderr:noscrub is unset 2023-11-25T04:24:23.131 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:23.129+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:23.167 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:23.167 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:23.175 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:24:23.234 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:23.233+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:23.320 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:23.320 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:23.327 INFO:tasks.workunit.client.0.smithi067.stdout:libtoolize: putting auxiliary files in '.'. 2023-11-25T04:24:23.327 INFO:tasks.workunit.client.0.smithi067.stdout:libtoolize: copying file './config.guess' 2023-11-25T04:24:23.331 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:24:23.336 INFO:tasks.workunit.client.0.smithi067.stdout:libtoolize: copying file './config.sub' 2023-11-25T04:24:23.337 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:23.334+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:23.342 INFO:tasks.workunit.client.0.smithi067.stdout:libtoolize: copying file './install-sh' 2023-11-25T04:24:23.349 INFO:tasks.workunit.client.0.smithi067.stdout:libtoolize: copying file './ltmain.sh' 2023-11-25T04:24:23.379 INFO:tasks.workunit.client.0.smithi067.stdout:libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'. 2023-11-25T04:24:23.380 INFO:tasks.workunit.client.0.smithi067.stdout:libtoolize: copying file 'm4/libtool.m4' 2023-11-25T04:24:23.386 INFO:tasks.workunit.client.0.smithi067.stdout:libtoolize: copying file 'm4/ltoptions.m4' 2023-11-25T04:24:23.393 INFO:tasks.workunit.client.0.smithi067.stdout:libtoolize: copying file 'm4/ltsugar.m4' 2023-11-25T04:24:23.402 INFO:tasks.workunit.client.0.smithi067.stdout:libtoolize: copying file 'm4/ltversion.m4' 2023-11-25T04:24:23.407 INFO:tasks.workunit.client.0.smithi067.stdout:libtoolize: copying file 'm4/lt~obsolete.m4' 2023-11-25T04:24:23.414 INFO:tasks.workunit.client.0.smithi067.stdout:libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am. 2023-11-25T04:24:23.416 INFO:tasks.workunit.client.0.smithi067.stdout:cp include/install-sh . 2023-11-25T04:24:23.421 INFO:tasks.workunit.client.0.smithi067.stdout:aclocal -I m4 2023-11-25T04:24:23.438 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:23.436+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:23.441 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:23.441 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:23.450 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:24:23.539 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:23.537+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:23.595 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:23.595 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:23.604 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:24:23.645 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:23.643+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:23.741 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:23.740+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:23.748 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:23.749 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:23.757 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:24:23.842 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:23.840+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:23.863 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:23.863 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:23.872 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:24:23.943 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:23.941+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:24.018 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:24.018 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:24.026 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:24:24.044 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:24.043+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:24.146 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:24.143+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:24.164 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:24.164 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:24.173 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:24:24.246 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:24.245+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:24.281 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:24.281 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:24.290 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:24:24.348 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:24.346+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:24.395 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:24.395 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:24.404 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:24:24.450 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:24.447+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:24.509 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:24.509 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:24.517 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:24:24.558 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:24.557+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:24.596 INFO:tasks.workunit.client.0.smithi067.stdout:autoheader 2023-11-25T04:24:24.625 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:24.625 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:24.633 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:24:24.678 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:24.677+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:24.738 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:24.738 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:24.741 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-11-25T04:24:24.746 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:24:24.750 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:24.749+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:24.851 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:24.850+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:24.854 INFO:teuthology.orchestra.run.smithi067.stderr:Error ENXIO: problem getting command descriptions from osd.0 2023-11-25T04:24:24.873 INFO:teuthology.orchestra.run.smithi067.stdout:osd.1: {} 2023-11-25T04:24:24.873 INFO:teuthology.orchestra.run.smithi067.stderr:osd.1: osd_enable_op_tracker = '' 2023-11-25T04:24:24.891 INFO:teuthology.orchestra.run.smithi067.stdout:osd.2: {} 2023-11-25T04:24:24.892 INFO:teuthology.orchestra.run.smithi067.stderr:osd.2: osd_enable_op_tracker = '' 2023-11-25T04:24:24.894 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:24.894 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2023-11-25T04:24:24.894 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2023-11-25T04:24:24.894 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2023-11-25T04:24:24.895 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.895 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:14 3.15 3:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:24.895 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:24:24.895 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 97.858646915999998, 2023-11-25T04:24:24.895 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010990437, 2023-11-25T04:24:24.895 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:24.895 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:24.896 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:24.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.896 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:24.896 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:24:24.896 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:24.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.896 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.897 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:24.897 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033381+0000", 2023-11-25T04:24:24.897 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:24.897 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.897 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.897 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:24.898 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033383+0000", 2023-11-25T04:24:24.898 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9870000000000002e-06 2023-11-25T04:24:24.898 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.898 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.898 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:24.898 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033398+0000", 2023-11-25T04:24:24.899 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4569999999999999e-05 2023-11-25T04:24:24.899 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.899 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.900 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:24.900 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033411+0000", 2023-11-25T04:24:24.900 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3543999999999999e-05 2023-11-25T04:24:24.900 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.900 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.901 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:24.901 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.033423+0000", 2023-11-25T04:24:24.901 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1905999999999999e-05 2023-11-25T04:24:24.901 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.901 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.902 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:24.902 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034433+0000", 2023-11-25T04:24:24.902 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00101022 2023-11-25T04:24:24.902 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.902 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.902 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:24.902 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034539+0000", 2023-11-25T04:24:24.903 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010597199999999999 2023-11-25T04:24:24.903 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.903 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.903 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:24.903 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034667+0000", 2023-11-25T04:24:24.903 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012765500000000001 2023-11-25T04:24:24.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.904 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:24.904 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037803+0000", 2023-11-25T04:24:24.904 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0031362220000000001 2023-11-25T04:24:24.904 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.904 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.905 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:24.905 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044300+0000", 2023-11-25T04:24:24.905 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0064964669999999997 2023-11-25T04:24:24.905 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.905 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.905 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:24.905 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044331+0000", 2023-11-25T04:24:24.906 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1544999999999997e-05 2023-11-25T04:24:24.906 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.906 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.906 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:24.906 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044372+0000", 2023-11-25T04:24:24.906 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0349000000000003e-05 2023-11-25T04:24:24.907 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:24.907 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:24.907 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:24.907 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.907 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.907 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:15 3.15 3:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:24.907 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:24:24.908 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 97.857987265999995, 2023-11-25T04:24:24.908 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010464362, 2023-11-25T04:24:24.908 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:24.908 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:24.908 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:24.908 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.908 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:24.909 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:24:24.909 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:24.909 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.909 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.909 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:24.909 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034041+0000", 2023-11-25T04:24:24.909 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:24.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.910 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.910 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:24.910 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034042+0000", 2023-11-25T04:24:24.910 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0999999999999997e-07 2023-11-25T04:24:24.910 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.910 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.911 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:24.911 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034048+0000", 2023-11-25T04:24:24.911 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0390000000000001e-06 2023-11-25T04:24:24.911 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.911 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.911 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:24.911 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034059+0000", 2023-11-25T04:24:24.912 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1182999999999999e-05 2023-11-25T04:24:24.912 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.912 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.912 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:24.912 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034067+0000", 2023-11-25T04:24:24.912 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1559999999999995e-06 2023-11-25T04:24:24.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.913 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.913 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:24.913 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035005+0000", 2023-11-25T04:24:24.913 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00093747399999999999 2023-11-25T04:24:24.913 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.913 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.913 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:24.914 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035077+0000", 2023-11-25T04:24:24.914 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2747e-05 2023-11-25T04:24:24.914 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.914 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.914 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:24.914 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035219+0000", 2023-11-25T04:24:24.914 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000141877 2023-11-25T04:24:24.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.915 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:24.915 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037810+0000", 2023-11-25T04:24:24.915 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002590752 2023-11-25T04:24:24.915 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.915 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:24.916 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044437+0000", 2023-11-25T04:24:24.916 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066270110000000004 2023-11-25T04:24:24.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.916 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:24.917 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044481+0000", 2023-11-25T04:24:24.917 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4106000000000001e-05 2023-11-25T04:24:24.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.917 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:24.917 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044505+0000", 2023-11-25T04:24:24.917 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4207e-05 2023-11-25T04:24:24.918 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:24.918 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:24.918 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:24.918 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.919 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.919 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:16 3.1d 3:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:24.919 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:24:24.919 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 97.857945418, 2023-11-25T04:24:24.919 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010413923, 2023-11-25T04:24:24.919 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:24.920 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:24.920 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:24.920 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.920 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:24.920 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:24:24.921 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:24.921 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.921 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.921 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:24.922 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:24:24.922 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:24.922 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.922 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.922 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:24.923 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034083+0000", 2023-11-25T04:24:24.923 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4499999999999997e-07 2023-11-25T04:24:24.923 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.923 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.923 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:24.923 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034087+0000", 2023-11-25T04:24:24.924 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9770000000000002e-06 2023-11-25T04:24:24.924 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.924 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.924 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:24.924 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034094+0000", 2023-11-25T04:24:24.924 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8809999999999998e-06 2023-11-25T04:24:24.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.925 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:24.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034100+0000", 2023-11-25T04:24:24.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8440000000000003e-06 2023-11-25T04:24:24.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.926 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.926 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:24.926 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034993+0000", 2023-11-25T04:24:24.926 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00089336799999999996 2023-11-25T04:24:24.927 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.927 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.927 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:24.927 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035099+0000", 2023-11-25T04:24:24.927 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000106037 2023-11-25T04:24:24.927 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.927 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.928 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:24.928 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035276+0000", 2023-11-25T04:24:24.928 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00017631799999999999 2023-11-25T04:24:24.928 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.928 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.929 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:24.929 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037815+0000", 2023-11-25T04:24:24.929 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025395959999999999 2023-11-25T04:24:24.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.929 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:24.929 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044420+0000", 2023-11-25T04:24:24.929 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0066045749999999997 2023-11-25T04:24:24.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.930 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.930 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:24.930 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044463+0000", 2023-11-25T04:24:24.930 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3205999999999999e-05 2023-11-25T04:24:24.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.930 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.930 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:24.930 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044497+0000", 2023-11-25T04:24:24.930 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3476000000000003e-05 2023-11-25T04:24:24.930 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:24.931 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:24.931 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:24.931 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.931 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.931 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:17 3.1d 3:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:24.931 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:24:24.931 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 97.857920209, 2023-11-25T04:24:24.931 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01127356, 2023-11-25T04:24:24.931 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:24.931 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:24.932 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:24.932 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.932 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:24.932 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:24:24.932 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:24.932 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.932 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.932 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:24.932 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:24:24.932 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:24.932 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.933 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.933 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:24.933 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034108+0000", 2023-11-25T04:24:24.933 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1100000000000001e-07 2023-11-25T04:24:24.933 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.933 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.933 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:24.933 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034112+0000", 2023-11-25T04:24:24.933 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8600000000000003e-06 2023-11-25T04:24:24.933 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.933 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.933 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:24.933 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034120+0000", 2023-11-25T04:24:24.934 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7700000000000001e-06 2023-11-25T04:24:24.934 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.934 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.934 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:24.934 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034127+0000", 2023-11-25T04:24:24.934 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0999999999999998e-06 2023-11-25T04:24:24.934 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.934 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.934 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:24.934 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035601+0000", 2023-11-25T04:24:24.934 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0014739410000000001 2023-11-25T04:24:24.934 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.935 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.935 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:24.935 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035685+0000", 2023-11-25T04:24:24.935 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3677999999999994e-05 2023-11-25T04:24:24.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.935 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.935 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:24.935 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035827+0000", 2023-11-25T04:24:24.935 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00014232599999999999 2023-11-25T04:24:24.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.935 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.935 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:24.935 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038725+0000", 2023-11-25T04:24:24.936 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0028982779999999998 2023-11-25T04:24:24.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.936 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:24.936 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045323+0000", 2023-11-25T04:24:24.936 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0065980780000000003 2023-11-25T04:24:24.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.936 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:24.936 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045354+0000", 2023-11-25T04:24:24.936 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0463e-05 2023-11-25T04:24:24.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.937 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:24.937 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.045381+0000", 2023-11-25T04:24:24.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7654999999999999e-05 2023-11-25T04:24:24.937 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:24.937 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:24.937 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:24.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.937 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.937 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:22 3.e 3:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:24.937 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:24:24.937 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 97.857893795999999, 2023-11-25T04:24:24.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010133422, 2023-11-25T04:24:24.937 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:24.938 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:24.938 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:24.938 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.938 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:24.938 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:24:24.938 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:24.938 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.938 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.938 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:24.938 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034134+0000", 2023-11-25T04:24:24.938 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:24.938 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.939 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:24.939 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034135+0000", 2023-11-25T04:24:24.939 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-07 2023-11-25T04:24:24.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.939 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:24.939 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034138+0000", 2023-11-25T04:24:24.939 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6009999999999999e-06 2023-11-25T04:24:24.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.939 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:24.940 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034145+0000", 2023-11-25T04:24:24.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1169999999999998e-06 2023-11-25T04:24:24.940 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.940 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.940 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:24.940 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034150+0000", 2023-11-25T04:24:24.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9080000000000003e-06 2023-11-25T04:24:24.940 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.940 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.940 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:24.940 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034209+0000", 2023-11-25T04:24:24.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8149000000000002e-05 2023-11-25T04:24:24.940 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.941 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:24.941 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034415+0000", 2023-11-25T04:24:24.941 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00020637699999999999 2023-11-25T04:24:24.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.941 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:24.941 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034584+0000", 2023-11-25T04:24:24.941 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00016870399999999999 2023-11-25T04:24:24.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.941 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:24.942 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.036979+0000", 2023-11-25T04:24:24.942 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0023957509999999998 2023-11-25T04:24:24.942 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.942 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.942 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:24.942 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044161+0000", 2023-11-25T04:24:24.942 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0071811590000000003 2023-11-25T04:24:24.942 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.942 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.942 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:24.942 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044224+0000", 2023-11-25T04:24:24.942 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3454000000000006e-05 2023-11-25T04:24:24.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.943 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:24.943 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044268+0000", 2023-11-25T04:24:24.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3761999999999999e-05 2023-11-25T04:24:24.943 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:24.943 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:24.943 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:24.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.943 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:23 3.e 3:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:24.943 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:24:24.944 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 97.857862386999997, 2023-11-25T04:24:24.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010281676, 2023-11-25T04:24:24.944 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:24.944 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:24.944 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:24.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:24.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:24:24.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:24.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:24.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:24:24.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:24.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:24.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034166+0000", 2023-11-25T04:24:24.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5799999999999999e-07 2023-11-25T04:24:24.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.946 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:24.946 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034169+0000", 2023-11-25T04:24:24.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1879999999999998e-06 2023-11-25T04:24:24.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.946 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:24.946 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034177+0000", 2023-11-25T04:24:24.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7020000000000001e-06 2023-11-25T04:24:24.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.946 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:24.946 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034184+0000", 2023-11-25T04:24:24.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3910000000000003e-06 2023-11-25T04:24:24.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.947 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:24.947 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034924+0000", 2023-11-25T04:24:24.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00074032999999999998 2023-11-25T04:24:24.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.947 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:24.947 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034991+0000", 2023-11-25T04:24:24.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7347000000000004e-05 2023-11-25T04:24:24.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.948 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:24.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035096+0000", 2023-11-25T04:24:24.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010488 2023-11-25T04:24:24.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.948 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:24.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.037588+0000", 2023-11-25T04:24:24.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0024918309999999999 2023-11-25T04:24:24.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:24.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044367+0000", 2023-11-25T04:24:24.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0067787589999999997 2023-11-25T04:24:24.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:24.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044415+0000", 2023-11-25T04:24:24.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7803000000000003e-05 2023-11-25T04:24:24.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:24.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044447+0000", 2023-11-25T04:24:24.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2886999999999999e-05 2023-11-25T04:24:24.950 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:24.950 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:24.950 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:24.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.950 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:30 3.15 3:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:24.950 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:24:24.950 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 97.857803946999994, 2023-11-25T04:24:24.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010365070000000001, 2023-11-25T04:24:24.950 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:24.950 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:24.950 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:24.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:24.951 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:24:24.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:24.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:24.951 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034224+0000", 2023-11-25T04:24:24.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:24.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:24.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034225+0000", 2023-11-25T04:24:24.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7400000000000002e-07 2023-11-25T04:24:24.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.952 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:24.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034229+0000", 2023-11-25T04:24:24.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3100000000000002e-06 2023-11-25T04:24:24.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.952 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:24.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034239+0000", 2023-11-25T04:24:24.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9140000000000003e-06 2023-11-25T04:24:24.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:24.953 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034246+0000", 2023-11-25T04:24:24.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0149999999999998e-06 2023-11-25T04:24:24.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:24.953 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035572+0000", 2023-11-25T04:24:24.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001325655 2023-11-25T04:24:24.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:24.953 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035730+0000", 2023-11-25T04:24:24.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00015827899999999999 2023-11-25T04:24:24.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.954 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:24.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.035857+0000", 2023-11-25T04:24:24.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012644699999999999 2023-11-25T04:24:24.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.954 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:24.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.038737+0000", 2023-11-25T04:24:24.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002880077 2023-11-25T04:24:24.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:24.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044540+0000", 2023-11-25T04:24:24.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0058034239999999997 2023-11-25T04:24:24.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:24.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044569+0000", 2023-11-25T04:24:24.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8388999999999999e-05 2023-11-25T04:24:24.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:24.956 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044589+0000", 2023-11-25T04:24:24.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0686000000000002e-05 2023-11-25T04:24:24.956 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:24.956 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:24.956 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:24.956 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.956 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.956 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:33 3.1b 3:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:24.956 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:24:24.956 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 97.857758449000002, 2023-11-25T04:24:24.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0099754049999999997, 2023-11-25T04:24:24.956 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:24.957 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:24.957 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:24.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.957 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:24.957 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:24:24.957 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:24.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.957 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:24.957 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:24:24.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:24.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:24.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034270+0000", 2023-11-25T04:24:24.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4799999999999998e-07 2023-11-25T04:24:24.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:24.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034273+0000", 2023-11-25T04:24:24.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7819999999999998e-06 2023-11-25T04:24:24.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.959 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:24.959 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034285+0000", 2023-11-25T04:24:24.959 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2058e-05 2023-11-25T04:24:24.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.959 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:24.959 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034291+0000", 2023-11-25T04:24:24.959 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0680000000000002e-06 2023-11-25T04:24:24.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.959 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:24.960 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034370+0000", 2023-11-25T04:24:24.960 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8452e-05 2023-11-25T04:24:24.960 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.960 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.960 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:24.960 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.034477+0000", 2023-11-25T04:24:24.960 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107471 2023-11-25T04:24:24.960 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.960 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.960 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:24.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:22:47.044245+0000", 2023-11-25T04:24:24.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0097679259999999997 2023-11-25T04:24:24.961 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:24.961 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:24.961 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:24.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.961 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:152 3.1a 3:5e41ab69:::200.00000005:head [write 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:24.961 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:21.237401+0000", 2023-11-25T04:24:24.961 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.6546274099999998, 2023-11-25T04:24:24.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015253802, 2023-11-25T04:24:24.961 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:24.961 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:24.962 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:24.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:24.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.237401+0000", 2023-11-25T04:24:24.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:24.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:24.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.237401+0000", 2023-11-25T04:24:24.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:24.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:24.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.237402+0000", 2023-11-25T04:24:24.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3699999999999999e-07 2023-11-25T04:24:24.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:24.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243182+0000", 2023-11-25T04:24:24.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057801270000000004 2023-11-25T04:24:24.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:24.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243193+0000", 2023-11-25T04:24:24.964 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1221e-05 2023-11-25T04:24:24.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.964 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.964 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:24.964 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243198+0000", 2023-11-25T04:24:24.964 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3739999999999997e-06 2023-11-25T04:24:24.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.964 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.964 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:24.964 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243246+0000", 2023-11-25T04:24:24.964 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7667999999999998e-05 2023-11-25T04:24:24.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.965 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.965 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:24.965 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243292+0000", 2023-11-25T04:24:24.965 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5794000000000002e-05 2023-11-25T04:24:24.965 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.965 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.965 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:24.965 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243360+0000", 2023-11-25T04:24:24.965 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8479000000000006e-05 2023-11-25T04:24:24.965 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.965 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.965 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:24.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.250667+0000", 2023-11-25T04:24:24.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.007306618 2023-11-25T04:24:24.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:24.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.252598+0000", 2023-11-25T04:24:24.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019315129999999999 2023-11-25T04:24:24.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:24.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.252631+0000", 2023-11-25T04:24:24.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2925000000000001e-05 2023-11-25T04:24:24.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.967 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:24.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.252655+0000", 2023-11-25T04:24:24.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3246000000000001e-05 2023-11-25T04:24:24.967 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:24.967 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:24.967 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:24.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.967 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:154 3.1e e46/16 3:7bf7e7b5:::200.00000006:head v 46'2, mlcod=0'0)", 2023-11-25T04:24:24.968 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:21.892635+0000", 2023-11-25T04:24:24.968 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.9993935380000001, 2023-11-25T04:24:24.968 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011376196999999999, 2023-11-25T04:24:24.968 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:24.968 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:24.968 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:24.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.968 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:24.968 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.892635+0000", 2023-11-25T04:24:24.968 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:24.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.969 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:24.969 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.892635+0000", 2023-11-25T04:24:24.969 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:24.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.969 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:24.969 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.892635+0000", 2023-11-25T04:24:24.969 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6300000000000005e-07 2023-11-25T04:24:24.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.969 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:24.969 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.896756+0000", 2023-11-25T04:24:24.969 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0041204809999999996 2023-11-25T04:24:24.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.970 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:24.970 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.896771+0000", 2023-11-25T04:24:24.970 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5231999999999999e-05 2023-11-25T04:24:24.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.970 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:24.970 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.896778+0000", 2023-11-25T04:24:24.970 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3309999999999998e-06 2023-11-25T04:24:24.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.970 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:24.971 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.896842+0000", 2023-11-25T04:24:24.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3800999999999994e-05 2023-11-25T04:24:24.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.971 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:24.971 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.896865+0000", 2023-11-25T04:24:24.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2762e-05 2023-11-25T04:24:24.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.971 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:24.971 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.903974+0000", 2023-11-25T04:24:24.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0071089220000000002 2023-11-25T04:24:24.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:24.972 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.904011+0000", 2023-11-25T04:24:24.972 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7104999999999998e-05 2023-11-25T04:24:24.972 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:24.972 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:24.972 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:24.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.972 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:1074 4.19 4:9ae72bb2:::10000001201.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:24.972 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:22.155648+0000", 2023-11-25T04:24:24.972 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.7363799540000002, 2023-11-25T04:24:24.973 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0095681949999999998, 2023-11-25T04:24:24.973 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:24.973 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:24.973 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:24.973 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.973 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:24.973 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.155648+0000", 2023-11-25T04:24:24.973 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:24.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.973 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.973 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:24.973 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.155648+0000", 2023-11-25T04:24:24.973 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:24.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.977 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:24.977 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.155649+0000", 2023-11-25T04:24:24.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9500000000000001e-07 2023-11-25T04:24:24.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.977 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:24.977 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.155653+0000", 2023-11-25T04:24:24.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5949999999999999e-06 2023-11-25T04:24:24.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.977 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:24.977 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.155661+0000", 2023-11-25T04:24:24.978 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7809999999999994e-06 2023-11-25T04:24:24.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.978 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.978 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:24.978 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.155669+0000", 2023-11-25T04:24:24.978 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1009999999999992e-06 2023-11-25T04:24:24.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.978 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.978 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:24.978 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.155723+0000", 2023-11-25T04:24:24.978 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3066999999999999e-05 2023-11-25T04:24:24.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:24.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.155794+0000", 2023-11-25T04:24:24.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1227000000000001e-05 2023-11-25T04:24:24.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:24.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.155884+0000", 2023-11-25T04:24:24.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.0614999999999999e-05 2023-11-25T04:24:24.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:24.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.157344+0000", 2023-11-25T04:24:24.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001459728 2023-11-25T04:24:24.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:24.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.165138+0000", 2023-11-25T04:24:24.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0077934839999999998 2023-11-25T04:24:24.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:24.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.165182+0000", 2023-11-25T04:24:24.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4036999999999999e-05 2023-11-25T04:24:24.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:24.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.165216+0000", 2023-11-25T04:24:24.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4764999999999997e-05 2023-11-25T04:24:24.981 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:24.981 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:24.981 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:24.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.981 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:1082 4.15 4:a87ae726:::10000001209.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:24.981 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:22.156227+0000", 2023-11-25T04:24:24.981 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.7358015180000002, 2023-11-25T04:24:24.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010001013, 2023-11-25T04:24:24.982 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:24.982 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:24.982 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:24.982 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.982 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:24.982 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156227+0000", 2023-11-25T04:24:24.982 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:24.982 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.982 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.982 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:24.982 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156227+0000", 2023-11-25T04:24:24.982 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:24.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.983 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:24.983 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156227+0000", 2023-11-25T04:24:24.983 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8500000000000002e-07 2023-11-25T04:24:24.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.983 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:24.983 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156230+0000", 2023-11-25T04:24:24.983 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1300000000000001e-06 2023-11-25T04:24:24.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.983 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:24.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156238+0000", 2023-11-25T04:24:24.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8949999999999998e-06 2023-11-25T04:24:24.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.984 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:24.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156244+0000", 2023-11-25T04:24:24.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5339999999999999e-06 2023-11-25T04:24:24.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.984 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:24.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156292+0000", 2023-11-25T04:24:24.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8359000000000002e-05 2023-11-25T04:24:24.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:24.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156363+0000", 2023-11-25T04:24:24.985 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1056999999999994e-05 2023-11-25T04:24:24.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:24.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156448+0000", 2023-11-25T04:24:24.985 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5100999999999997e-05 2023-11-25T04:24:24.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:24.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.158964+0000", 2023-11-25T04:24:24.986 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0025160310000000002 2023-11-25T04:24:24.986 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.986 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.986 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:24.986 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.166062+0000", 2023-11-25T04:24:24.986 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0070979989999999998 2023-11-25T04:24:24.986 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.986 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.986 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:24.986 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.166165+0000", 2023-11-25T04:24:24.987 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010280099999999999 2023-11-25T04:24:24.987 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.987 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.987 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:24.987 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.166228+0000", 2023-11-25T04:24:24.987 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2620999999999996e-05 2023-11-25T04:24:24.987 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:24.987 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:24.987 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:24.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.988 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:1086 4.34 4:2d0cb294:::1000000120d.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:24.988 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:22.156608+0000", 2023-11-25T04:24:24.988 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.7354197730000003, 2023-11-25T04:24:24.988 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0096961140000000005, 2023-11-25T04:24:24.988 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:24.988 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:24.988 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:24.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.988 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:24.988 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156608+0000", 2023-11-25T04:24:24.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:24.989 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.989 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.989 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:24.989 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156608+0000", 2023-11-25T04:24:24.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:24.989 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.989 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.989 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:24.989 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156609+0000", 2023-11-25T04:24:24.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8899999999999999e-07 2023-11-25T04:24:24.989 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.989 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.990 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:24.990 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156612+0000", 2023-11-25T04:24:24.990 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6730000000000001e-06 2023-11-25T04:24:24.990 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.990 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.990 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:24.990 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156620+0000", 2023-11-25T04:24:24.990 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.7159999999999992e-06 2023-11-25T04:24:24.990 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.990 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.990 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:24.990 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156626+0000", 2023-11-25T04:24:24.990 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.631e-06 2023-11-25T04:24:24.991 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.991 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.991 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:24.991 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156953+0000", 2023-11-25T04:24:24.991 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00032727 2023-11-25T04:24:24.991 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.991 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.991 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:24.991 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.157042+0000", 2023-11-25T04:24:24.991 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8349000000000004e-05 2023-11-25T04:24:24.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.992 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:24.992 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.166033+0000", 2023-11-25T04:24:24.992 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0089916370000000002 2023-11-25T04:24:24.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.992 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:24.992 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.166084+0000", 2023-11-25T04:24:24.992 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0538999999999998e-05 2023-11-25T04:24:24.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.993 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:24.993 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.166304+0000", 2023-11-25T04:24:24.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00022070999999999999 2023-11-25T04:24:24.993 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:24.993 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:24.993 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:24.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.993 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:1088 4.3 4:c0636c6d:::1000000120f.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:24.993 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:22.156639+0000", 2023-11-25T04:24:24.993 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.7353889439999999, 2023-11-25T04:24:24.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010613015, 2023-11-25T04:24:24.994 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:24.994 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:24.994 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:24.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:24.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156639+0000", 2023-11-25T04:24:24.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:24.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:24.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156639+0000", 2023-11-25T04:24:24.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:24.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.995 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:24.995 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156640+0000", 2023-11-25T04:24:24.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7800000000000002e-07 2023-11-25T04:24:24.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.995 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:24.995 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156641+0000", 2023-11-25T04:24:24.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7719999999999999e-06 2023-11-25T04:24:24.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.996 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:24.996 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156647+0000", 2023-11-25T04:24:24.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0399999999999998e-06 2023-11-25T04:24:24.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.996 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:24.996 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156654+0000", 2023-11-25T04:24:24.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8900000000000001e-06 2023-11-25T04:24:24.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.996 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:24.996 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156942+0000", 2023-11-25T04:24:24.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028773999999999998 2023-11-25T04:24:24.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.997 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:24.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.157012+0000", 2023-11-25T04:24:24.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0290000000000006e-05 2023-11-25T04:24:24.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.997 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:24.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.167143+0000", 2023-11-25T04:24:24.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010130208999999999 2023-11-25T04:24:24.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:24.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.167183+0000", 2023-11-25T04:24:24.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0352000000000003e-05 2023-11-25T04:24:24.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:24.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.167252+0000", 2023-11-25T04:24:24.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9244000000000003e-05 2023-11-25T04:24:24.998 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:24.998 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:24.998 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:24.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:24.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.999 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:1090 4.32 4:4e235bd8:::10000001211.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:24.999 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:22.156665+0000", 2023-11-25T04:24:24.999 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.7353630020000002, 2023-11-25T04:24:24.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0098969169999999999, 2023-11-25T04:24:24.999 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:24.999 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:24.999 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:24.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:24.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:25.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156665+0000", 2023-11-25T04:24:25.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:25.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:25.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156665+0000", 2023-11-25T04:24:25.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:25.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:25.001 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156666+0000", 2023-11-25T04:24:25.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4000000000000002e-07 2023-11-25T04:24:25.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.001 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:25.001 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156668+0000", 2023-11-25T04:24:25.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9379999999999999e-06 2023-11-25T04:24:25.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.001 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:25.001 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156675+0000", 2023-11-25T04:24:25.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4170000000000003e-06 2023-11-25T04:24:25.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:25.002 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156680+0000", 2023-11-25T04:24:25.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2000000000000002e-06 2023-11-25T04:24:25.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:25.002 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156949+0000", 2023-11-25T04:24:25.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026843800000000002 2023-11-25T04:24:25.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:25.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.157013+0000", 2023-11-25T04:24:25.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.4424000000000005e-05 2023-11-25T04:24:25.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:25.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.166438+0000", 2023-11-25T04:24:25.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0094252150000000007 2023-11-25T04:24:25.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:25.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.166483+0000", 2023-11-25T04:24:25.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4916999999999997e-05 2023-11-25T04:24:25.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:25.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.166562+0000", 2023-11-25T04:24:25.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.8928000000000001e-05 2023-11-25T04:24:25.004 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:25.004 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:25.004 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:25.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.005 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:1092 4.1 4:80d14d92:::10000001213.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:25.005 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:22.156698+0000", 2023-11-25T04:24:25.005 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.7353304220000001, 2023-11-25T04:24:25.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0095673489999999993, 2023-11-25T04:24:25.005 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:25.005 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:25.005 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:25.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:25.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156698+0000", 2023-11-25T04:24:25.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:25.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.006 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:25.006 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156698+0000", 2023-11-25T04:24:25.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:25.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.006 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:25.006 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156698+0000", 2023-11-25T04:24:25.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7599999999999997e-07 2023-11-25T04:24:25.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.006 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:25.006 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156700+0000", 2023-11-25T04:24:25.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9199999999999998e-06 2023-11-25T04:24:25.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:25.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156718+0000", 2023-11-25T04:24:25.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7730000000000001e-05 2023-11-25T04:24:25.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:25.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156729+0000", 2023-11-25T04:24:25.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0900999999999999e-05 2023-11-25T04:24:25.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:25.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.156945+0000", 2023-11-25T04:24:25.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00021670500000000001 2023-11-25T04:24:25.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:25.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.157021+0000", 2023-11-25T04:24:25.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5986000000000001e-05 2023-11-25T04:24:25.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:25.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.166097+0000", 2023-11-25T04:24:25.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0090757649999999995 2023-11-25T04:24:25.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:25.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.166154+0000", 2023-11-25T04:24:25.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6805000000000002e-05 2023-11-25T04:24:25.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:25.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.166265+0000", 2023-11-25T04:24:25.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000111061 2023-11-25T04:24:25.009 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:25.010 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:25.010 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:25.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.010 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:1096 4.3 4:c1f14f5b:::10000001217.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:25.010 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:22.157851+0000", 2023-11-25T04:24:25.010 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.734176927, 2023-11-25T04:24:25.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.010572455999999999, 2023-11-25T04:24:25.010 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:25.010 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:25.011 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:25.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:25.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.157851+0000", 2023-11-25T04:24:25.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:25.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:25.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.157851+0000", 2023-11-25T04:24:25.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:25.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:25.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.157852+0000", 2023-11-25T04:24:25.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6199999999999997e-07 2023-11-25T04:24:25.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:25.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.157879+0000", 2023-11-25T04:24:25.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6911e-05 2023-11-25T04:24:25.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:25.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.157887+0000", 2023-11-25T04:24:25.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5059999999999995e-06 2023-11-25T04:24:25.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:25.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.157895+0000", 2023-11-25T04:24:25.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2799999999999998e-06 2023-11-25T04:24:25.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:25.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.157948+0000", 2023-11-25T04:24:25.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2901000000000001e-05 2023-11-25T04:24:25.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:25.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.158027+0000", 2023-11-25T04:24:25.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9609000000000003e-05 2023-11-25T04:24:25.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:25.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.167260+0000", 2023-11-25T04:24:25.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0092331240000000005 2023-11-25T04:24:25.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:25.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.168357+0000", 2023-11-25T04:24:25.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0010965090000000001 2023-11-25T04:24:25.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:25.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.168424+0000", 2023-11-25T04:24:25.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6853999999999994e-05 2023-11-25T04:24:25.015 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:25.015 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:25.015 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:25.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.015 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:1104 4.27 4:e4eba14d:::1000000121f.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:25.016 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:22.158428+0000", 2023-11-25T04:24:25.016 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.7336004730000001, 2023-11-25T04:24:25.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0095411690000000004, 2023-11-25T04:24:25.016 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:25.016 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:25.016 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:25.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.016 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:25.016 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.158428+0000", 2023-11-25T04:24:25.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:25.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.016 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:25.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.158428+0000", 2023-11-25T04:24:25.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:25.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:25.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.158428+0000", 2023-11-25T04:24:25.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9200000000000001e-07 2023-11-25T04:24:25.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:25.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.158431+0000", 2023-11-25T04:24:25.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.531e-06 2023-11-25T04:24:25.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:25.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.158437+0000", 2023-11-25T04:24:25.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2380000000000002e-06 2023-11-25T04:24:25.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:25.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.158441+0000", 2023-11-25T04:24:25.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4469999999999999e-06 2023-11-25T04:24:25.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:25.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.158477+0000", 2023-11-25T04:24:25.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5543000000000002e-05 2023-11-25T04:24:25.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:25.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.158550+0000", 2023-11-25T04:24:25.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3289999999999998e-05 2023-11-25T04:24:25.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.020 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:25.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.167857+0000", 2023-11-25T04:24:25.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0093063420000000004 2023-11-25T04:24:25.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.021 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:25.021 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.167899+0000", 2023-11-25T04:24:25.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2400000000000001e-05 2023-11-25T04:24:25.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.021 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:25.021 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.167969+0000", 2023-11-25T04:24:25.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9886000000000001e-05 2023-11-25T04:24:25.021 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:25.021 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:25.021 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:25.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.022 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:1105 4.23 e46/17 4:c6ff07b4:::10000001220.00000000:head v 46'106, mlcod=46'104)", 2023-11-25T04:24:25.022 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:22.159179+0000", 2023-11-25T04:24:25.022 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.7328496119999999, 2023-11-25T04:24:25.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0093609680000000008, 2023-11-25T04:24:25.022 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:25.022 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:25.022 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:25.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:25.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.159179+0000", 2023-11-25T04:24:25.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:25.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:25.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.159179+0000", 2023-11-25T04:24:25.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:25.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:25.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.159179+0000", 2023-11-25T04:24:25.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1399999999999997e-07 2023-11-25T04:24:25.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:25.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.159183+0000", 2023-11-25T04:24:25.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0579999999999999e-06 2023-11-25T04:24:25.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:25.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.159192+0000", 2023-11-25T04:24:25.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6300000000000004e-06 2023-11-25T04:24:25.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:25.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.159197+0000", 2023-11-25T04:24:25.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9180000000000002e-06 2023-11-25T04:24:25.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:25.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.159234+0000", 2023-11-25T04:24:25.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7490999999999997e-05 2023-11-25T04:24:25.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:25.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.159248+0000", 2023-11-25T04:24:25.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.3991e-05 2023-11-25T04:24:25.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:25.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.168426+0000", 2023-11-25T04:24:25.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0091775869999999992 2023-11-25T04:24:25.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:25.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.168539+0000", 2023-11-25T04:24:25.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000113879 2023-11-25T04:24:25.026 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:25.026 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:25.026 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:25.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.027 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:1119 4.3f 4:fec262f2:::1000000122e.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:25.027 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:22.159421+0000", 2023-11-25T04:24:25.027 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.7326074760000001, 2023-11-25T04:24:25.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0092470220000000006, 2023-11-25T04:24:25.027 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:25.027 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:25.027 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:25.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:25.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.159421+0000", 2023-11-25T04:24:25.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:25.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:25.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.159421+0000", 2023-11-25T04:24:25.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:25.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:25.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.159423+0000", 2023-11-25T04:24:25.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9860000000000001e-06 2023-11-25T04:24:25.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:25.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.159425+0000", 2023-11-25T04:24:25.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.661e-06 2023-11-25T04:24:25.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:25.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.159431+0000", 2023-11-25T04:24:25.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4940000000000003e-06 2023-11-25T04:24:25.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:25.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.159435+0000", 2023-11-25T04:24:25.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5680000000000001e-06 2023-11-25T04:24:25.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:25.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.165146+0000", 2023-11-25T04:24:25.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057110850000000003 2023-11-25T04:24:25.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:25.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.165216+0000", 2023-11-25T04:24:25.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9227000000000007e-05 2023-11-25T04:24:25.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:25.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.165304+0000", 2023-11-25T04:24:25.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8164000000000005e-05 2023-11-25T04:24:25.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:25.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.167359+0000", 2023-11-25T04:24:25.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0020552349999999999 2023-11-25T04:24:25.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:25.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.168545+0000", 2023-11-25T04:24:25.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011858509999999999 2023-11-25T04:24:25.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:25.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.168634+0000", 2023-11-25T04:24:25.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8832999999999995e-05 2023-11-25T04:24:25.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:25.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:22.168668+0000", 2023-11-25T04:24:25.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3918000000000003e-05 2023-11-25T04:24:25.032 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:25.032 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:25.032 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:25.032 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:25.032 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:25.033 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:25.034 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:25.033+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:25.034 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-11-25T04:24:25.035 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-11-25T04:24:25.035 INFO:tasks.workunit.client.0.smithi067.stdout:autoconf 2023-11-25T04:24:25.036 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:24:25.135 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:25.134+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:25.141 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2023-11-25T04:24:25.144 INFO:tasks.workunit.client.0.smithi067.stdout:./configure \ 2023-11-25T04:24:25.144 INFO:tasks.workunit.client.0.smithi067.stdout: --libexecdir=/usr/lib \ 2023-11-25T04:24:25.145 INFO:tasks.workunit.client.0.smithi067.stdout: --exec_prefix=/var/lib 2023-11-25T04:24:25.149 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:25.150 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:24:25.150 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2023-11-25T04:24:25.150 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:25.159 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:24:25.235 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~76s 2023-11-25T04:24:25.236 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:25.235+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:25.294 INFO:tasks.workunit.client.0.smithi067.stdout:checking build system type... x86_64-pc-linux-gnu 2023-11-25T04:24:25.294 INFO:tasks.workunit.client.0.smithi067.stdout:checking host system type... x86_64-pc-linux-gnu 2023-11-25T04:24:25.296 INFO:tasks.workunit.client.0.smithi067.stdout:checking how to print strings... printf 2023-11-25T04:24:25.296 INFO:tasks.workunit.client.0.smithi067.stdout:checking for gcc... gcc 2023-11-25T04:24:25.305 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:25.305 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:24:25.305 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2023-11-25T04:24:25.305 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2023-11-25T04:24:25.305 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:25.314 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:24:25.337 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:25.336+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:25.374 INFO:tasks.workunit.client.0.smithi067.stdout:checking whether the C compiler works... yes 2023-11-25T04:24:25.374 INFO:tasks.workunit.client.0.smithi067.stdout:checking for C compiler default output file name... a.out 2023-11-25T04:24:25.418 INFO:tasks.workunit.client.0.smithi067.stdout:checking for suffix of executables... 2023-11-25T04:24:25.421 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:25.421 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2023-11-25T04:24:25.421 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2023-11-25T04:24:25.421 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2023-11-25T04:24:25.422 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.422 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:25.422 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:24:25.422 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 98.387319536000007, 2023-11-25T04:24:25.422 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01072943, 2023-11-25T04:24:25.422 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:25.422 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:25.422 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:25.422 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.422 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:25.422 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:24:25.423 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:25.423 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.423 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.423 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:25.423 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:24:25.423 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:25.423 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.423 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.423 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:25.423 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031829+0000", 2023-11-25T04:24:25.423 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.666e-06 2023-11-25T04:24:25.423 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.424 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.424 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:25.424 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031839+0000", 2023-11-25T04:24:25.424 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9490000000000008e-06 2023-11-25T04:24:25.424 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.424 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.424 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:25.424 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031857+0000", 2023-11-25T04:24:25.424 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8093000000000001e-05 2023-11-25T04:24:25.424 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.424 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.425 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:25.425 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031882+0000", 2023-11-25T04:24:25.425 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4294000000000002e-05 2023-11-25T04:24:25.425 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.425 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.425 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:25.425 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033130+0000", 2023-11-25T04:24:25.425 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001248264 2023-11-25T04:24:25.425 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.425 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.425 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:25.425 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033426+0000", 2023-11-25T04:24:25.426 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029620999999999998 2023-11-25T04:24:25.426 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.426 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.426 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:25.426 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033814+0000", 2023-11-25T04:24:25.426 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000387742 2023-11-25T04:24:25.426 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.426 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.426 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:25.426 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036793+0000", 2023-11-25T04:24:25.426 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002979598 2023-11-25T04:24:25.426 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.427 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.427 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:25.427 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042455+0000", 2023-11-25T04:24:25.427 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0056619260000000003 2023-11-25T04:24:25.427 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.427 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.427 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:25.427 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042502+0000", 2023-11-25T04:24:25.427 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7139999999999999e-05 2023-11-25T04:24:25.427 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.427 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.427 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:25.428 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042557+0000", 2023-11-25T04:24:25.428 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4548e-05 2023-11-25T04:24:25.428 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:25.428 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:25.428 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:25.428 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.428 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.428 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:25.428 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:24:25.428 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 98.386121001000006, 2023-11-25T04:24:25.428 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011505202000000001, 2023-11-25T04:24:25.428 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:25.429 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:25.429 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:25.429 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.429 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:25.429 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:24:25.429 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:25.429 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.429 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.429 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:25.429 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:24:25.429 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:25.429 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.430 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.430 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:25.430 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033027+0000", 2023-11-25T04:24:25.430 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0699999999999999e-06 2023-11-25T04:24:25.430 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.430 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.430 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:25.430 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033035+0000", 2023-11-25T04:24:25.430 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0830000000000004e-06 2023-11-25T04:24:25.430 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.431 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.431 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:25.431 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033059+0000", 2023-11-25T04:24:25.431 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3975e-05 2023-11-25T04:24:25.431 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.431 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.431 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:25.431 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033482+0000", 2023-11-25T04:24:25.431 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042253700000000001 2023-11-25T04:24:25.431 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.431 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.431 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:25.431 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034520+0000", 2023-11-25T04:24:25.431 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001037991 2023-11-25T04:24:25.432 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.432 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.432 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:25.432 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034611+0000", 2023-11-25T04:24:25.432 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1113999999999995e-05 2023-11-25T04:24:25.432 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.432 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.432 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:25.432 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034769+0000", 2023-11-25T04:24:25.432 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000157641 2023-11-25T04:24:25.432 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.432 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.432 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:25.433 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038292+0000", 2023-11-25T04:24:25.433 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035231160000000002 2023-11-25T04:24:25.433 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.433 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.433 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:25.433 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044440+0000", 2023-11-25T04:24:25.433 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0061486980000000002 2023-11-25T04:24:25.433 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.433 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.433 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:25.433 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044495+0000", 2023-11-25T04:24:25.433 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4339000000000003e-05 2023-11-25T04:24:25.433 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.433 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.434 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:25.434 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044531+0000", 2023-11-25T04:24:25.434 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6637999999999997e-05 2023-11-25T04:24:25.434 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:25.434 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:25.434 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:25.434 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.434 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.434 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:26 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:25.434 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:25.434 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 98.385532239, 2023-11-25T04:24:25.434 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011086945000000001, 2023-11-25T04:24:25.434 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:25.435 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:25.435 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:25.435 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.435 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:25.435 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:25.435 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:25.435 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.435 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.435 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:25.435 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:25.435 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:25.436 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.436 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.436 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:25.436 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:25.436 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1699999999999998e-07 2023-11-25T04:24:25.436 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.436 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.436 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:25.436 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033627+0000", 2023-11-25T04:24:25.436 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.163e-05 2023-11-25T04:24:25.436 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.437 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.437 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:25.437 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033636+0000", 2023-11-25T04:24:25.437 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6859999999999995e-06 2023-11-25T04:24:25.437 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.437 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.437 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:25.437 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033642+0000", 2023-11-25T04:24:25.437 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3149999999999997e-06 2023-11-25T04:24:25.438 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.438 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.438 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:25.438 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035414+0000", 2023-11-25T04:24:25.438 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0017716380000000001 2023-11-25T04:24:25.438 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.438 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.438 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:25.438 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035539+0000", 2023-11-25T04:24:25.438 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000125477 2023-11-25T04:24:25.438 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.438 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.439 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:25.439 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035672+0000", 2023-11-25T04:24:25.439 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000133316 2023-11-25T04:24:25.439 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.439 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.439 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:25.439 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038649+0000", 2023-11-25T04:24:25.439 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029764679999999999 2023-11-25T04:24:25.439 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.439 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.439 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:25.439 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044578+0000", 2023-11-25T04:24:25.440 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0059293510000000002 2023-11-25T04:24:25.440 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.440 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.440 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:25.440 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044651+0000", 2023-11-25T04:24:25.440 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2513000000000002e-05 2023-11-25T04:24:25.440 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.440 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.440 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:25.440 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044702+0000", 2023-11-25T04:24:25.440 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1034e-05 2023-11-25T04:24:25.440 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:25.441 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:25.441 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:25.441 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.441 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.441 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:29 3.b 3:d0630e4c:::mds0_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:25.441 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:25.441 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 98.385494319000003, 2023-11-25T04:24:25.441 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010980227, 2023-11-25T04:24:25.441 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:25.441 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:25.441 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:25.441 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.442 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:25.442 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:25.442 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:25.442 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.442 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.442 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:25.442 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:25.442 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:25.442 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.442 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.442 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:25.442 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:25.443 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3499999999999996e-07 2023-11-25T04:24:25.443 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.443 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.443 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:25.443 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033658+0000", 2023-11-25T04:24:25.443 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1280000000000001e-06 2023-11-25T04:24:25.443 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.443 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.443 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:25.443 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033667+0000", 2023-11-25T04:24:25.443 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1430000000000004e-06 2023-11-25T04:24:25.443 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.443 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.444 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:25.444 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033678+0000", 2023-11-25T04:24:25.444 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0906e-05 2023-11-25T04:24:25.444 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.444 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.444 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:25.444 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034456+0000", 2023-11-25T04:24:25.444 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00077833899999999996 2023-11-25T04:24:25.444 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.444 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.444 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:25.444 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034575+0000", 2023-11-25T04:24:25.445 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011926000000000001 2023-11-25T04:24:25.445 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.445 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.445 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:25.445 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035001+0000", 2023-11-25T04:24:25.445 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042550999999999998 2023-11-25T04:24:25.445 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.445 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.445 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:25.445 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.037923+0000", 2023-11-25T04:24:25.445 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029219670000000001 2023-11-25T04:24:25.446 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.446 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.446 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:25.446 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044538+0000", 2023-11-25T04:24:25.446 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0066152349999999997 2023-11-25T04:24:25.446 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.446 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.446 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:25.446 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044594+0000", 2023-11-25T04:24:25.446 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5593000000000001e-05 2023-11-25T04:24:25.446 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.447 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.447 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:25.447 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044633+0000", 2023-11-25T04:24:25.447 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9511000000000002e-05 2023-11-25T04:24:25.447 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:25.447 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:25.447 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:25.447 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.447 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.447 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:34 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:25.447 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:24:25.447 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 98.385244205999996, 2023-11-25T04:24:25.448 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010920445000000001, 2023-11-25T04:24:25.448 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:25.448 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:25.448 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:25.448 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.448 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:25.448 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:24:25.448 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:25.448 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.448 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.448 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:25.448 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:24:25.449 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:25.449 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.449 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.449 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:25.449 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033904+0000", 2023-11-25T04:24:25.449 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6599999999999994e-07 2023-11-25T04:24:25.449 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.449 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.449 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:25.449 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033909+0000", 2023-11-25T04:24:25.450 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5639999999999996e-06 2023-11-25T04:24:25.450 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.450 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.450 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:25.450 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033923+0000", 2023-11-25T04:24:25.450 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3028e-05 2023-11-25T04:24:25.450 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.450 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.450 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:25.450 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034043+0000", 2023-11-25T04:24:25.450 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000120328 2023-11-25T04:24:25.450 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.451 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.451 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:25.451 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035576+0000", 2023-11-25T04:24:25.451 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015333720000000001 2023-11-25T04:24:25.451 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.451 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.451 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:25.451 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035699+0000", 2023-11-25T04:24:25.451 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000123001 2023-11-25T04:24:25.451 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.451 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.451 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:25.451 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044823+0000", 2023-11-25T04:24:25.452 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0091241859999999994 2023-11-25T04:24:25.452 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:25.452 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:25.452 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:25.452 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.452 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.452 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:118 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals,omap-rm-keys] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e44)", 2023-11-25T04:24:25.452 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:17.112648+0000", 2023-11-25T04:24:25.452 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 8.306498865, 2023-11-25T04:24:25.452 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011250779000000001, 2023-11-25T04:24:25.452 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:25.452 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:25.452 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:25.453 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.453 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:25.453 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.112648+0000", 2023-11-25T04:24:25.453 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:25.453 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.453 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.453 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:25.453 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.112648+0000", 2023-11-25T04:24:25.453 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:25.453 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.453 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.453 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:25.454 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.112649+0000", 2023-11-25T04:24:25.454 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0070000000000001e-06 2023-11-25T04:24:25.454 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.454 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.454 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:25.454 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113182+0000", 2023-11-25T04:24:25.454 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00053268100000000002 2023-11-25T04:24:25.454 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.454 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.454 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:25.454 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113196+0000", 2023-11-25T04:24:25.454 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3742e-05 2023-11-25T04:24:25.455 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.455 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.455 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:25.455 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113398+0000", 2023-11-25T04:24:25.455 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000202726 2023-11-25T04:24:25.455 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.455 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.455 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:25.455 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113447+0000", 2023-11-25T04:24:25.455 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8655000000000001e-05 2023-11-25T04:24:25.455 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.455 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.456 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:25.456 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113529+0000", 2023-11-25T04:24:25.456 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1891999999999995e-05 2023-11-25T04:24:25.456 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.456 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.456 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:25.456 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123819+0000", 2023-11-25T04:24:25.456 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010290192 2023-11-25T04:24:25.456 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.456 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.456 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:25.456 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123852+0000", 2023-11-25T04:24:25.456 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2425999999999998e-05 2023-11-25T04:24:25.457 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.457 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.457 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:25.457 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123899+0000", 2023-11-25T04:24:25.457 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7457999999999999e-05 2023-11-25T04:24:25.457 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:25.457 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:25.457 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:25.457 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.457 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.457 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:154 3.1e 3:7bf7e7b5:::200.00000006:head [write 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:25.457 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:21.886430+0000", 2023-11-25T04:24:25.457 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.5327175990000002, 2023-11-25T04:24:25.458 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.017800107999999999, 2023-11-25T04:24:25.458 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:25.458 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:25.458 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:25.458 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.458 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:25.458 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.886430+0000", 2023-11-25T04:24:25.458 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:25.458 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.458 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.458 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:25.458 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.886430+0000", 2023-11-25T04:24:25.459 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:25.459 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.459 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.459 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:25.459 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.886431+0000", 2023-11-25T04:24:25.459 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.178e-06 2023-11-25T04:24:25.459 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.459 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.459 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:25.459 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.892277+0000", 2023-11-25T04:24:25.459 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0058467409999999999 2023-11-25T04:24:25.459 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.459 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.460 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:25.460 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.892288+0000", 2023-11-25T04:24:25.460 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0183000000000001e-05 2023-11-25T04:24:25.460 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.460 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.460 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:25.460 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.892292+0000", 2023-11-25T04:24:25.460 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7829999999999998e-06 2023-11-25T04:24:25.460 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.460 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.461 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:25.461 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.892362+0000", 2023-11-25T04:24:25.461 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0034000000000001e-05 2023-11-25T04:24:25.461 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.461 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.461 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:25.461 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.892432+0000", 2023-11-25T04:24:25.461 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9648999999999997e-05 2023-11-25T04:24:25.461 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.461 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.461 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:25.461 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.892550+0000", 2023-11-25T04:24:25.462 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011825 2023-11-25T04:24:25.462 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.462 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.462 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:25.462 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.899106+0000", 2023-11-25T04:24:25.462 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0065556759999999999 2023-11-25T04:24:25.462 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.462 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.462 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:25.462 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.904145+0000", 2023-11-25T04:24:25.462 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0050393490000000003 2023-11-25T04:24:25.462 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.463 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.463 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:25.463 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.904200+0000", 2023-11-25T04:24:25.463 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4234999999999998e-05 2023-11-25T04:24:25.463 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.463 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.463 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:25.463 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.904230+0000", 2023-11-25T04:24:25.463 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0029999999999999e-05 2023-11-25T04:24:25.463 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:25.463 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:25.463 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:25.463 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.464 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.464 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:576 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals,omap-rm-keys] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:25.464 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:22.111407+0000", 2023-11-25T04:24:25.464 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.3077399930000002, 2023-11-25T04:24:25.464 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.018987317, 2023-11-25T04:24:25.464 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:25.464 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:25.464 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:25.464 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.465 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:25.465 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111407+0000", 2023-11-25T04:24:25.465 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:25.465 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.465 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.465 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:25.465 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111407+0000", 2023-11-25T04:24:25.465 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:25.465 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.465 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.465 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:25.465 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111408+0000", 2023-11-25T04:24:25.466 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8000000000000004e-07 2023-11-25T04:24:25.466 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.466 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.466 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:25.466 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111659+0000", 2023-11-25T04:24:25.466 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000250537 2023-11-25T04:24:25.466 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.466 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.466 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:25.466 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111666+0000", 2023-11-25T04:24:25.466 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.973e-06 2023-11-25T04:24:25.466 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.466 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.467 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:25.467 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111670+0000", 2023-11-25T04:24:25.467 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.668e-06 2023-11-25T04:24:25.467 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.467 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.467 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:25.467 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111709+0000", 2023-11-25T04:24:25.467 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8800000000000001e-05 2023-11-25T04:24:25.467 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.467 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.467 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:25.467 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111761+0000", 2023-11-25T04:24:25.468 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1495000000000001e-05 2023-11-25T04:24:25.468 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.468 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.468 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:25.468 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.130255+0000", 2023-11-25T04:24:25.468 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.018494345999999998 2023-11-25T04:24:25.468 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.468 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.468 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:25.468 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.130311+0000", 2023-11-25T04:24:25.468 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5902000000000001e-05 2023-11-25T04:24:25.468 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.469 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.469 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:25.469 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.130394+0000", 2023-11-25T04:24:25.469 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3715999999999996e-05 2023-11-25T04:24:25.469 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:25.469 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:25.469 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:25.469 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.469 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.469 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:1067 4.21 4:869a7600:::100000011fa.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:25.469 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:22.155038+0000", 2023-11-25T04:24:25.469 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.2641086550000002, 2023-11-25T04:24:25.470 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011313148, 2023-11-25T04:24:25.470 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:25.470 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:25.470 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:25.470 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.470 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:25.470 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155038+0000", 2023-11-25T04:24:25.470 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:25.470 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.470 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.470 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:25.470 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155038+0000", 2023-11-25T04:24:25.470 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:25.471 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.471 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.471 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:25.471 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155039+0000", 2023-11-25T04:24:25.471 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0200000000000002e-07 2023-11-25T04:24:25.471 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.471 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.471 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:25.471 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155049+0000", 2023-11-25T04:24:25.471 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0295000000000001e-05 2023-11-25T04:24:25.471 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.471 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.471 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:25.472 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155058+0000", 2023-11-25T04:24:25.472 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.208e-06 2023-11-25T04:24:25.472 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.472 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.472 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:25.472 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155064+0000", 2023-11-25T04:24:25.472 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4489999999999996e-06 2023-11-25T04:24:25.472 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.472 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.472 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:25.472 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155104+0000", 2023-11-25T04:24:25.472 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0164999999999999e-05 2023-11-25T04:24:25.472 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.473 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.473 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:25.473 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155147+0000", 2023-11-25T04:24:25.473 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2576e-05 2023-11-25T04:24:25.473 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.473 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.473 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:25.473 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155212+0000", 2023-11-25T04:24:25.473 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5516000000000001e-05 2023-11-25T04:24:25.473 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.473 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.473 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:25.473 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.156025+0000", 2023-11-25T04:24:25.474 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00081248599999999996 2023-11-25T04:24:25.474 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.474 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.474 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:25.474 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.166250+0000", 2023-11-25T04:24:25.474 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010224736999999999 2023-11-25T04:24:25.477 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.477 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.477 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:25.477 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.166319+0000", 2023-11-25T04:24:25.478 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9145000000000003e-05 2023-11-25T04:24:25.478 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.478 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.478 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:25.478 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.166352+0000", 2023-11-25T04:24:25.478 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2969000000000002e-05 2023-11-25T04:24:25.478 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:25.478 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:25.478 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:25.478 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.478 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.478 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:1102 4.2f 4:f543b94c:::1000000121d.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:25.478 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:22.158246+0000", 2023-11-25T04:24:25.479 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.2609011319999999, 2023-11-25T04:24:25.479 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010094396, 2023-11-25T04:24:25.479 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:25.479 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:25.479 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:25.479 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.479 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:25.479 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158246+0000", 2023-11-25T04:24:25.479 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:25.479 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.479 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.479 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:25.480 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158246+0000", 2023-11-25T04:24:25.480 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:25.480 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.480 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.480 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:25.480 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158246+0000", 2023-11-25T04:24:25.480 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5299999999999999e-07 2023-11-25T04:24:25.480 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.480 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.480 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:25.480 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158249+0000", 2023-11-25T04:24:25.480 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2840000000000001e-06 2023-11-25T04:24:25.480 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.481 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.481 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:25.481 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158254+0000", 2023-11-25T04:24:25.481 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3449999999999996e-06 2023-11-25T04:24:25.481 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.481 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.481 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:25.481 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158258+0000", 2023-11-25T04:24:25.481 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5030000000000002e-06 2023-11-25T04:24:25.481 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.481 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.481 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:25.481 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158302+0000", 2023-11-25T04:24:25.482 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3908e-05 2023-11-25T04:24:25.482 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.482 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.482 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:25.482 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158398+0000", 2023-11-25T04:24:25.482 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6898000000000006e-05 2023-11-25T04:24:25.482 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.482 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.482 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:25.482 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158480+0000", 2023-11-25T04:24:25.482 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1578999999999999e-05 2023-11-25T04:24:25.482 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.483 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.483 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:25.483 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.159229+0000", 2023-11-25T04:24:25.483 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000749049 2023-11-25T04:24:25.483 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.483 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.483 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:25.483 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.168228+0000", 2023-11-25T04:24:25.483 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0089992930000000002 2023-11-25T04:24:25.483 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.483 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.483 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:25.484 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.168310+0000", 2023-11-25T04:24:25.484 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1348999999999997e-05 2023-11-25T04:24:25.484 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.484 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.484 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:25.484 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.168340+0000", 2023-11-25T04:24:25.484 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0735000000000003e-05 2023-11-25T04:24:25.484 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:25.484 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:25.484 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:25.484 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.484 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.484 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:1105 4.23 4:c6ff07b4:::10000001220.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:25.485 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:22.158714+0000", 2023-11-25T04:24:25.485 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.2604330180000001, 2023-11-25T04:24:25.485 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010412490999999999, 2023-11-25T04:24:25.485 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:25.485 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:25.485 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:25.485 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.485 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:25.485 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158714+0000", 2023-11-25T04:24:25.485 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:25.485 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.485 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.486 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:25.486 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158714+0000", 2023-11-25T04:24:25.486 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:25.486 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.486 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.486 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:25.486 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158715+0000", 2023-11-25T04:24:25.486 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4099999999999998e-07 2023-11-25T04:24:25.486 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.486 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.486 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:25.486 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158718+0000", 2023-11-25T04:24:25.487 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4319999999999999e-06 2023-11-25T04:24:25.487 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.487 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.487 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:25.487 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158729+0000", 2023-11-25T04:24:25.487 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.078e-05 2023-11-25T04:24:25.487 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.487 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.487 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:25.487 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158736+0000", 2023-11-25T04:24:25.487 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4649999999999997e-06 2023-11-25T04:24:25.487 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.487 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.488 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:25.488 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158779+0000", 2023-11-25T04:24:25.488 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3049000000000001e-05 2023-11-25T04:24:25.488 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.488 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.488 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:25.488 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158865+0000", 2023-11-25T04:24:25.488 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5138999999999999e-05 2023-11-25T04:24:25.488 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.488 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.488 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:25.488 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158983+0000", 2023-11-25T04:24:25.489 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000118658 2023-11-25T04:24:25.489 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.489 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.489 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:25.489 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.159645+0000", 2023-11-25T04:24:25.489 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00066193999999999997 2023-11-25T04:24:25.489 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.489 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.489 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:25.489 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.169046+0000", 2023-11-25T04:24:25.489 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0094005400000000006 2023-11-25T04:24:25.489 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.490 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.490 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:25.490 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.169092+0000", 2023-11-25T04:24:25.490 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5840000000000002e-05 2023-11-25T04:24:25.490 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.490 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.490 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:25.490 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.169127+0000", 2023-11-25T04:24:25.490 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5006999999999999e-05 2023-11-25T04:24:25.490 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:25.490 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:25.490 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:25.490 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.491 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.491 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2841 4.4 4:22111d42:::100000010df.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:25.491 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.092102+0000", 2023-11-25T04:24:25.491 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 0.32704554299999999, 2023-11-25T04:24:25.491 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.016449783999999999, 2023-11-25T04:24:25.491 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:25.491 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:25.491 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:25.491 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.491 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:25.491 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092102+0000", 2023-11-25T04:24:25.491 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:25.492 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.492 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.492 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:25.492 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092102+0000", 2023-11-25T04:24:25.492 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:25.492 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.492 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.492 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:25.492 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092102+0000", 2023-11-25T04:24:25.492 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3700000000000001e-07 2023-11-25T04:24:25.492 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.492 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.492 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:25.493 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092104+0000", 2023-11-25T04:24:25.493 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0679999999999999e-06 2023-11-25T04:24:25.493 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.493 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.493 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:25.493 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092111+0000", 2023-11-25T04:24:25.493 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0589999999999997e-06 2023-11-25T04:24:25.493 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.493 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.493 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:25.493 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092130+0000", 2023-11-25T04:24:25.493 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8686999999999999e-05 2023-11-25T04:24:25.494 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.494 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.494 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:25.494 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092443+0000", 2023-11-25T04:24:25.494 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000313519 2023-11-25T04:24:25.495 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.495 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.495 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:25.495 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092553+0000", 2023-11-25T04:24:25.495 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010932100000000001 2023-11-25T04:24:25.495 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.495 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.495 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:25.496 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092639+0000", 2023-11-25T04:24:25.496 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6094999999999995e-05 2023-11-25T04:24:25.496 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.496 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.496 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:25.496 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094679+0000", 2023-11-25T04:24:25.496 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0020402580000000001 2023-11-25T04:24:25.496 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.496 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.496 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:25.496 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108462+0000", 2023-11-25T04:24:25.496 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013783358000000001 2023-11-25T04:24:25.497 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.497 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.497 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:25.497 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108516+0000", 2023-11-25T04:24:25.497 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3483000000000003e-05 2023-11-25T04:24:25.497 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.497 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.497 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:25.497 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108551+0000", 2023-11-25T04:24:25.497 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5499e-05 2023-11-25T04:24:25.497 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:25.497 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:25.497 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:25.498 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.498 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.498 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2852 4.2d 4:b745ac76:::100000010f5.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:25.498 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.093477+0000", 2023-11-25T04:24:25.498 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 0.32567043099999998, 2023-11-25T04:24:25.498 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.015423084, 2023-11-25T04:24:25.498 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:25.498 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:25.498 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:25.498 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.498 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:25.498 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093477+0000", 2023-11-25T04:24:25.499 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:25.499 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.499 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.499 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:25.499 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093477+0000", 2023-11-25T04:24:25.499 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:25.499 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.499 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.499 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:25.499 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093477+0000", 2023-11-25T04:24:25.499 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2500000000000005e-07 2023-11-25T04:24:25.499 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.499 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.500 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:25.500 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093480+0000", 2023-11-25T04:24:25.500 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0189999999999998e-06 2023-11-25T04:24:25.500 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.500 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.500 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:25.500 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093642+0000", 2023-11-25T04:24:25.500 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000162072 2023-11-25T04:24:25.500 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.500 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.500 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:25.500 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093649+0000", 2023-11-25T04:24:25.501 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9659999999999999e-06 2023-11-25T04:24:25.501 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.501 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.501 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:25.501 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093693+0000", 2023-11-25T04:24:25.501 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3794000000000001e-05 2023-11-25T04:24:25.501 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.501 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.501 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:25.501 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093764+0000", 2023-11-25T04:24:25.501 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0981000000000004e-05 2023-11-25T04:24:25.501 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.501 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.502 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:25.502 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093850+0000", 2023-11-25T04:24:25.502 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5536000000000002e-05 2023-11-25T04:24:25.502 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.502 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.502 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:25.502 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094710+0000", 2023-11-25T04:24:25.502 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00086070299999999999 2023-11-25T04:24:25.502 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.502 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.502 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:25.502 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108776+0000", 2023-11-25T04:24:25.502 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01406542 2023-11-25T04:24:25.503 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.503 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.503 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:25.503 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108829+0000", 2023-11-25T04:24:25.503 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2936999999999998e-05 2023-11-25T04:24:25.503 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.503 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.503 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:25.503 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108900+0000", 2023-11-25T04:24:25.503 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1031000000000005e-05 2023-11-25T04:24:25.503 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:25.503 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:25.503 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:25.504 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.504 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.504 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2854 4.2d 4:b5c259cf:::100000010f9.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:25.504 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.093674+0000", 2023-11-25T04:24:25.504 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 0.32547277099999999, 2023-11-25T04:24:25.504 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.015970380999999999, 2023-11-25T04:24:25.504 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:25.504 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:25.504 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:25.504 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.504 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:25.504 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093674+0000", 2023-11-25T04:24:25.505 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:25.505 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.505 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.505 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:25.505 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093674+0000", 2023-11-25T04:24:25.505 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:25.505 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.505 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.505 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:25.505 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093675+0000", 2023-11-25T04:24:25.505 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8400000000000004e-07 2023-11-25T04:24:25.505 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.506 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.506 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:25.506 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093678+0000", 2023-11-25T04:24:25.506 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2229999999999999e-06 2023-11-25T04:24:25.506 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.506 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.506 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:25.506 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093686+0000", 2023-11-25T04:24:25.506 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9990000000000008e-06 2023-11-25T04:24:25.506 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.506 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.506 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:25.506 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093696+0000", 2023-11-25T04:24:25.507 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6479999999999998e-06 2023-11-25T04:24:25.507 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.507 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.507 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:25.507 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094104+0000", 2023-11-25T04:24:25.507 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00040823900000000002 2023-11-25T04:24:25.507 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.507 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.507 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:25.507 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094172+0000", 2023-11-25T04:24:25.507 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8023999999999998e-05 2023-11-25T04:24:25.507 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.508 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.508 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:25.508 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094256+0000", 2023-11-25T04:24:25.508 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3418999999999995e-05 2023-11-25T04:24:25.508 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.508 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.508 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:25.508 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095160+0000", 2023-11-25T04:24:25.508 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00090418599999999997 2023-11-25T04:24:25.508 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.508 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.508 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:25.508 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109551+0000", 2023-11-25T04:24:25.509 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014391571000000001 2023-11-25T04:24:25.509 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.509 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.509 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:25.509 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109601+0000", 2023-11-25T04:24:25.509 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9478000000000003e-05 2023-11-25T04:24:25.509 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.509 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.509 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:25.509 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109645+0000", 2023-11-25T04:24:25.509 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.401e-05 2023-11-25T04:24:25.509 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:25.509 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:25.510 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:25.510 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.510 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.510 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2861 4.4 4:214027a9:::10000001107.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:25.510 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.093829+0000", 2023-11-25T04:24:25.510 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 0.325318353, 2023-11-25T04:24:25.510 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014848917999999999, 2023-11-25T04:24:25.510 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:25.510 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:25.510 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:25.510 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.510 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:25.511 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093829+0000", 2023-11-25T04:24:25.511 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:25.511 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.511 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.511 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:25.511 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093829+0000", 2023-11-25T04:24:25.511 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:25.511 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.511 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.511 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:25.512 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093830+0000", 2023-11-25T04:24:25.512 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1999999999999998e-07 2023-11-25T04:24:25.512 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.512 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.512 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:25.512 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093834+0000", 2023-11-25T04:24:25.512 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.34e-06 2023-11-25T04:24:25.512 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.512 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.512 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:25.512 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093844+0000", 2023-11-25T04:24:25.512 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6690000000000001e-06 2023-11-25T04:24:25.512 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.513 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.513 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:25.513 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093857+0000", 2023-11-25T04:24:25.513 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3434999999999999e-05 2023-11-25T04:24:25.513 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.513 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.513 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:25.513 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093940+0000", 2023-11-25T04:24:25.513 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3047999999999995e-05 2023-11-25T04:24:25.513 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.513 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.513 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:25.514 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094052+0000", 2023-11-25T04:24:25.514 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000111538 2023-11-25T04:24:25.514 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.514 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.514 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:25.514 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094466+0000", 2023-11-25T04:24:25.514 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00041464600000000001 2023-11-25T04:24:25.514 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.514 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.514 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:25.514 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095570+0000", 2023-11-25T04:24:25.514 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001103981 2023-11-25T04:24:25.514 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.515 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.515 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:25.515 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108598+0000", 2023-11-25T04:24:25.515 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013027164000000001 2023-11-25T04:24:25.515 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.515 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.515 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:25.515 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108646+0000", 2023-11-25T04:24:25.515 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8090999999999997e-05 2023-11-25T04:24:25.515 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.515 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.515 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:25.515 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108678+0000", 2023-11-25T04:24:25.516 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2085999999999997e-05 2023-11-25T04:24:25.516 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:25.516 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:25.516 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:25.516 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.516 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.516 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2863 4.12 4:4b84c733:::1000000110b.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:25.516 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.093877+0000", 2023-11-25T04:24:25.516 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 0.32527002599999999, 2023-11-25T04:24:25.516 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014682755, 2023-11-25T04:24:25.516 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:25.516 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:25.517 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:25.517 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.517 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:25.517 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093877+0000", 2023-11-25T04:24:25.517 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:25.517 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.517 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.517 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:25.517 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093877+0000", 2023-11-25T04:24:25.517 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:25.517 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.517 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.517 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:25.518 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093878+0000", 2023-11-25T04:24:25.518 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4199999999999996e-07 2023-11-25T04:24:25.518 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.518 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.518 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:25.518 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093880+0000", 2023-11-25T04:24:25.518 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6189999999999998e-06 2023-11-25T04:24:25.518 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.518 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.518 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:25.518 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093888+0000", 2023-11-25T04:24:25.518 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5510000000000002e-06 2023-11-25T04:24:25.518 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.519 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.519 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:25.519 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093893+0000", 2023-11-25T04:24:25.519 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4179999999999996e-06 2023-11-25T04:24:25.519 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.519 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.519 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:25.519 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093934+0000", 2023-11-25T04:24:25.519 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0648000000000001e-05 2023-11-25T04:24:25.519 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.519 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.519 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:25.520 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094081+0000", 2023-11-25T04:24:25.520 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014753499999999999 2023-11-25T04:24:25.520 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.520 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.520 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:25.520 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094172+0000", 2023-11-25T04:24:25.520 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.0722000000000003e-05 2023-11-25T04:24:25.520 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.520 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.520 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:25.520 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094742+0000", 2023-11-25T04:24:25.520 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00056948600000000002 2023-11-25T04:24:25.520 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.521 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.521 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:25.521 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108491+0000", 2023-11-25T04:24:25.521 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013749568 2023-11-25T04:24:25.521 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.521 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.521 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:25.521 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108525+0000", 2023-11-25T04:24:25.521 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3683999999999998e-05 2023-11-25T04:24:25.521 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.521 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.521 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:25.521 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108560+0000", 2023-11-25T04:24:25.522 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4981999999999998e-05 2023-11-25T04:24:25.522 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:25.522 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:25.522 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:25.522 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.522 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.522 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2880 4.21 4:855834a4:::1000000112d.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:25.522 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.095098+0000", 2023-11-25T04:24:25.522 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 0.32404933200000002, 2023-11-25T04:24:25.522 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013847544, 2023-11-25T04:24:25.522 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:25.522 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:25.523 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:25.523 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.523 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:25.523 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095098+0000", 2023-11-25T04:24:25.523 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:25.523 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.523 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.523 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:25.523 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095098+0000", 2023-11-25T04:24:25.523 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:25.523 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.523 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.523 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:25.524 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095098+0000", 2023-11-25T04:24:25.524 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3000000000000002e-07 2023-11-25T04:24:25.524 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.524 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.524 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:25.524 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095099+0000", 2023-11-25T04:24:25.524 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.331e-06 2023-11-25T04:24:25.524 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.524 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.524 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:25.524 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095103+0000", 2023-11-25T04:24:25.524 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8319999999999999e-06 2023-11-25T04:24:25.524 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.525 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.525 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:25.525 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095108+0000", 2023-11-25T04:24:25.525 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4179999999999999e-06 2023-11-25T04:24:25.525 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.525 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.525 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:25.525 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095153+0000", 2023-11-25T04:24:25.525 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4799000000000003e-05 2023-11-25T04:24:25.525 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.525 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.525 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:25.525 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095268+0000", 2023-11-25T04:24:25.526 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000115437 2023-11-25T04:24:25.526 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.526 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.526 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:25.526 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095363+0000", 2023-11-25T04:24:25.526 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.5278000000000005e-05 2023-11-25T04:24:25.526 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.526 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.526 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:25.526 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095845+0000", 2023-11-25T04:24:25.526 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00048164600000000002 2023-11-25T04:24:25.526 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.526 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.527 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:25.527 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108854+0000", 2023-11-25T04:24:25.527 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013009289 2023-11-25T04:24:25.527 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.527 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.527 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:25.527 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108907+0000", 2023-11-25T04:24:25.527 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2528999999999998e-05 2023-11-25T04:24:25.527 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.527 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.527 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:25.527 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108945+0000", 2023-11-25T04:24:25.528 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8655000000000002e-05 2023-11-25T04:24:25.528 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:25.528 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:25.528 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:25.528 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.528 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.528 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2884 4.37 4:ef909cdc:::10000001135.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:25.528 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.095518+0000", 2023-11-25T04:24:25.528 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 0.32362940499999998, 2023-11-25T04:24:25.528 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014149988, 2023-11-25T04:24:25.528 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:25.528 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:25.528 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:25.529 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.529 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:25.529 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095518+0000", 2023-11-25T04:24:25.529 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:25.529 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.529 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.529 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:25.529 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095518+0000", 2023-11-25T04:24:25.529 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:25.529 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.529 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.529 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:25.529 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095518+0000", 2023-11-25T04:24:25.530 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2900000000000003e-07 2023-11-25T04:24:25.530 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.530 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.530 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:25.530 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095522+0000", 2023-11-25T04:24:25.530 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2119999999999999e-06 2023-11-25T04:24:25.530 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.530 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.530 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:25.530 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095532+0000", 2023-11-25T04:24:25.530 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0375e-05 2023-11-25T04:24:25.530 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.531 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.531 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:25.531 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095537+0000", 2023-11-25T04:24:25.531 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5909999999999998e-06 2023-11-25T04:24:25.531 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.531 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.531 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:25.531 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095584+0000", 2023-11-25T04:24:25.531 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7169999999999997e-05 2023-11-25T04:24:25.531 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.531 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.531 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:25.531 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095695+0000", 2023-11-25T04:24:25.532 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011078699999999999 2023-11-25T04:24:25.532 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.532 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.532 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:25.532 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095777+0000", 2023-11-25T04:24:25.532 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2089999999999995e-05 2023-11-25T04:24:25.532 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.532 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.532 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:25.532 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096286+0000", 2023-11-25T04:24:25.532 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000509247 2023-11-25T04:24:25.532 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.532 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.533 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:25.533 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109596+0000", 2023-11-25T04:24:25.533 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013309872 2023-11-25T04:24:25.533 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.533 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.533 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:25.533 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109638+0000", 2023-11-25T04:24:25.533 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2381e-05 2023-11-25T04:24:25.533 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.533 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.533 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:25.533 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109668+0000", 2023-11-25T04:24:25.533 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9634000000000001e-05 2023-11-25T04:24:25.534 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:25.534 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:25.534 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:25.534 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.534 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.534 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2890 4.12 4:4a2ad640:::10000001141.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:25.534 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.096274+0000", 2023-11-25T04:24:25.534 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 0.32287334200000001, 2023-11-25T04:24:25.534 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013647259, 2023-11-25T04:24:25.534 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:25.534 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:25.534 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:25.535 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.535 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:25.535 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096274+0000", 2023-11-25T04:24:25.535 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:25.535 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.535 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.535 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:25.535 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096274+0000", 2023-11-25T04:24:25.535 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:25.535 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.535 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.535 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:25.536 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096275+0000", 2023-11-25T04:24:25.536 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0299999999999998e-07 2023-11-25T04:24:25.536 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.536 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.536 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:25.536 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096279+0000", 2023-11-25T04:24:25.536 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9580000000000001e-06 2023-11-25T04:24:25.536 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.536 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.536 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:25.536 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096287+0000", 2023-11-25T04:24:25.536 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6929999999999996e-06 2023-11-25T04:24:25.536 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.537 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.537 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:25.537 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096294+0000", 2023-11-25T04:24:25.537 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.348e-06 2023-11-25T04:24:25.537 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.537 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.537 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:25.537 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096342+0000", 2023-11-25T04:24:25.537 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8068000000000001e-05 2023-11-25T04:24:25.537 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.537 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.537 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:25.537 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096462+0000", 2023-11-25T04:24:25.538 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000119918 2023-11-25T04:24:25.538 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.538 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.538 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:25.538 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096557+0000", 2023-11-25T04:24:25.538 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.5340999999999994e-05 2023-11-25T04:24:25.538 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.538 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.538 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:25.538 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096971+0000", 2023-11-25T04:24:25.538 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00041398100000000001 2023-11-25T04:24:25.538 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.538 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.539 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:25.539 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109849+0000", 2023-11-25T04:24:25.539 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.012877985999999999 2023-11-25T04:24:25.539 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.539 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.539 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:25.539 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109894+0000", 2023-11-25T04:24:25.539 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5595e-05 2023-11-25T04:24:25.539 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.539 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.539 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:25.539 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109921+0000", 2023-11-25T04:24:25.540 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6568e-05 2023-11-25T04:24:25.540 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:25.540 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:25.540 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:25.540 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.540 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.540 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2891 4.23 4:c4054dd3:::10000001143.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:25.540 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.096713+0000", 2023-11-25T04:24:25.540 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 0.32243464799999999, 2023-11-25T04:24:25.540 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013148692, 2023-11-25T04:24:25.540 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:25.540 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:25.540 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:25.541 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.541 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:25.541 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096713+0000", 2023-11-25T04:24:25.541 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:25.541 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.541 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.541 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:25.541 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096713+0000", 2023-11-25T04:24:25.541 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:25.541 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.541 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.541 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:25.541 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096713+0000", 2023-11-25T04:24:25.542 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9299999999999998e-07 2023-11-25T04:24:25.542 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.542 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.542 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:25.542 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096716+0000", 2023-11-25T04:24:25.542 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.988e-06 2023-11-25T04:24:25.542 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.542 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.542 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:25.542 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096722+0000", 2023-11-25T04:24:25.542 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0499999999999997e-06 2023-11-25T04:24:25.542 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.543 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.543 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:25.543 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096727+0000", 2023-11-25T04:24:25.543 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3380000000000004e-06 2023-11-25T04:24:25.543 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.543 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.543 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:25.543 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096770+0000", 2023-11-25T04:24:25.543 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2938000000000001e-05 2023-11-25T04:24:25.543 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.543 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.543 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:25.543 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096904+0000", 2023-11-25T04:24:25.544 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000133759 2023-11-25T04:24:25.544 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.544 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.544 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:25.544 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096983+0000", 2023-11-25T04:24:25.544 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8559999999999993e-05 2023-11-25T04:24:25.544 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.544 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.544 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:25.544 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.097437+0000", 2023-11-25T04:24:25.544 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00045439200000000002 2023-11-25T04:24:25.544 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.544 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.545 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:25.545 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109737+0000", 2023-11-25T04:24:25.545 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.012300190000000001 2023-11-25T04:24:25.545 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.545 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.545 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:25.545 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109782+0000", 2023-11-25T04:24:25.545 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4546999999999999e-05 2023-11-25T04:24:25.545 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:25.545 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:25.545 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:25.545 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109861+0000", 2023-11-25T04:24:25.545 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9337000000000003e-05 2023-11-25T04:24:25.546 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:25.546 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:25.546 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:25.546 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:25.546 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:25.546 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:25.548 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:25.546+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:25.548 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:24:25.549 INFO:tasks.workunit.client.0.smithi067.stdout:checking whether we are cross compiling... no 2023-11-25T04:24:25.549 INFO:tasks.workunit.client.0.smithi067.stdout:checking for suffix of object files... o 2023-11-25T04:24:25.549 INFO:tasks.workunit.client.0.smithi067.stdout:checking whether we are using the GNU C compiler... yes 2023-11-25T04:24:25.549 INFO:tasks.workunit.client.0.smithi067.stdout:checking whether gcc accepts -g... yes 2023-11-25T04:24:25.624 INFO:tasks.workunit.client.0.smithi067.stdout:checking for gcc option to accept ISO C89... none needed 2023-11-25T04:24:25.628 INFO:tasks.workunit.client.0.smithi067.stdout:checking for a sed that does not truncate output... /usr/bin/sed 2023-11-25T04:24:25.631 INFO:tasks.workunit.client.0.smithi067.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2023-11-25T04:24:25.632 INFO:tasks.workunit.client.0.smithi067.stdout:checking for egrep... /usr/bin/grep -E 2023-11-25T04:24:25.634 INFO:tasks.workunit.client.0.smithi067.stdout:checking for fgrep... /usr/bin/grep -F 2023-11-25T04:24:25.637 INFO:tasks.workunit.client.0.smithi067.stdout:checking for ld used by gcc... /usr/bin/ld 2023-11-25T04:24:25.639 INFO:tasks.workunit.client.0.smithi067.stdout:checking if the linker (/usr/bin/ld) is GNU ld... yes 2023-11-25T04:24:25.643 INFO:tasks.workunit.client.0.smithi067.stdout:checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B 2023-11-25T04:24:25.648 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:25.647+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:25.655 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:25.655 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:24:25.655 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2023-11-25T04:24:25.655 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:25.665 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:24:25.749 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:25.749+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:25.810 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:25.810 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:24:25.810 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2023-11-25T04:24:25.810 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2023-11-25T04:24:25.810 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:25.818 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:24:25.862 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:25.861+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:25.924 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:25.924 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2023-11-25T04:24:25.924 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2023-11-25T04:24:25.924 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2023-11-25T04:24:25.924 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.924 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:152 3.1a 3:5e41ab69:::200.00000005:head [write 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:25.924 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:21.237401+0000", 2023-11-25T04:24:25.924 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.6843110540000001, 2023-11-25T04:24:25.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015253802, 2023-11-25T04:24:25.925 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:25.925 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:25.925 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:25.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.925 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:25.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.237401+0000", 2023-11-25T04:24:25.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:25.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.925 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:25.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.237401+0000", 2023-11-25T04:24:25.926 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:25.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.926 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.926 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:25.926 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.237402+0000", 2023-11-25T04:24:25.926 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3699999999999999e-07 2023-11-25T04:24:25.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.926 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.926 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:25.926 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243182+0000", 2023-11-25T04:24:25.926 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057801270000000004 2023-11-25T04:24:25.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.926 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.927 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:25.927 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243193+0000", 2023-11-25T04:24:25.927 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1221e-05 2023-11-25T04:24:25.927 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.927 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.927 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:25.927 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243198+0000", 2023-11-25T04:24:25.927 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3739999999999997e-06 2023-11-25T04:24:25.927 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.928 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.928 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:25.928 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243246+0000", 2023-11-25T04:24:25.928 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7667999999999998e-05 2023-11-25T04:24:25.928 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.928 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.928 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:25.928 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243292+0000", 2023-11-25T04:24:25.928 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5794000000000002e-05 2023-11-25T04:24:25.928 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.929 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:25.929 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243360+0000", 2023-11-25T04:24:25.929 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8479000000000006e-05 2023-11-25T04:24:25.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.929 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:25.929 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.250667+0000", 2023-11-25T04:24:25.929 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.007306618 2023-11-25T04:24:25.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.929 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:25.930 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.252598+0000", 2023-11-25T04:24:25.930 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019315129999999999 2023-11-25T04:24:25.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.930 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.930 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:25.930 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.252631+0000", 2023-11-25T04:24:25.930 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2925000000000001e-05 2023-11-25T04:24:25.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.930 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.930 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:25.930 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.252655+0000", 2023-11-25T04:24:25.931 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3246000000000001e-05 2023-11-25T04:24:25.931 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:25.931 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:25.931 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:25.931 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.931 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.931 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:2841 4.4 e47/17 4:22111d42:::100000010df.00000000:head v 47'139, mlcod=47'137)", 2023-11-25T04:24:25.931 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.093610+0000", 2023-11-25T04:24:25.931 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.82810219100000004, 2023-11-25T04:24:25.931 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014758871999999999, 2023-11-25T04:24:25.931 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:25.931 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:25.931 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:25.932 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.932 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:25.932 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093610+0000", 2023-11-25T04:24:25.932 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:25.932 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.932 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.932 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:25.932 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093610+0000", 2023-11-25T04:24:25.932 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:25.932 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.932 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.932 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:25.933 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093610+0000", 2023-11-25T04:24:25.933 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1799999999999999e-07 2023-11-25T04:24:25.933 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.933 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.933 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:25.933 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093621+0000", 2023-11-25T04:24:25.933 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1107999999999999e-05 2023-11-25T04:24:25.933 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.933 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.933 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:25.933 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093627+0000", 2023-11-25T04:24:25.933 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0310000000000004e-06 2023-11-25T04:24:25.934 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.934 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.934 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:25.934 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093633+0000", 2023-11-25T04:24:25.934 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2700000000000001e-06 2023-11-25T04:24:25.934 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.934 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.934 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:25.934 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093722+0000", 2023-11-25T04:24:25.934 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8417000000000004e-05 2023-11-25T04:24:25.934 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.934 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.934 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:25.935 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093737+0000", 2023-11-25T04:24:25.935 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5781e-05 2023-11-25T04:24:25.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.935 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.935 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:25.935 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108203+0000", 2023-11-25T04:24:25.935 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014465844 2023-11-25T04:24:25.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.935 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.935 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:25.935 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108368+0000", 2023-11-25T04:24:25.935 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000165203 2023-11-25T04:24:25.936 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:25.936 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:25.936 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:25.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.936 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2860 4.3d 4:bf0d7877:::10000001105.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:25.936 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.093823+0000", 2023-11-25T04:24:25.936 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.827889023, 2023-11-25T04:24:25.936 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015582848, 2023-11-25T04:24:25.936 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:25.936 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:25.936 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:25.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.937 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:25.937 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093823+0000", 2023-11-25T04:24:25.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:25.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.937 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.937 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:25.937 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093823+0000", 2023-11-25T04:24:25.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:25.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.937 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.937 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:25.937 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093823+0000", 2023-11-25T04:24:25.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9100000000000004e-07 2023-11-25T04:24:25.938 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.938 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.938 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:25.938 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093826+0000", 2023-11-25T04:24:25.938 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4499999999999998e-06 2023-11-25T04:24:25.938 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.938 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.938 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:25.938 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093835+0000", 2023-11-25T04:24:25.939 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8179999999999993e-06 2023-11-25T04:24:25.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.939 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:25.939 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093840+0000", 2023-11-25T04:24:25.939 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0819999999999998e-06 2023-11-25T04:24:25.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.939 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:25.940 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093973+0000", 2023-11-25T04:24:25.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013308999999999999 2023-11-25T04:24:25.940 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.940 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.940 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:25.940 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094277+0000", 2023-11-25T04:24:25.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00030403299999999999 2023-11-25T04:24:25.940 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.940 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.940 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:25.940 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108608+0000", 2023-11-25T04:24:25.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014330888 2023-11-25T04:24:25.940 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.941 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:25.941 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108676+0000", 2023-11-25T04:24:25.941 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8577000000000004e-05 2023-11-25T04:24:25.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.941 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:25.941 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109406+0000", 2023-11-25T04:24:25.942 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00072931899999999997 2023-11-25T04:24:25.942 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:25.942 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:25.942 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:25.942 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.942 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.942 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2862 4.13 4:cb2e645e:::10000001109.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:25.942 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.093852+0000", 2023-11-25T04:24:25.942 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.82786006199999995, 2023-11-25T04:24:25.942 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015190636, 2023-11-25T04:24:25.942 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:25.942 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:25.942 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:25.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.943 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:25.943 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093852+0000", 2023-11-25T04:24:25.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:25.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.943 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:25.943 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093852+0000", 2023-11-25T04:24:25.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:25.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:25.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093852+0000", 2023-11-25T04:24:25.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0299999999999999e-07 2023-11-25T04:24:25.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:25.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093854+0000", 2023-11-25T04:24:25.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.846e-06 2023-11-25T04:24:25.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:25.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093861+0000", 2023-11-25T04:24:25.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5520000000000001e-06 2023-11-25T04:24:25.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:25.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093865+0000", 2023-11-25T04:24:25.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7870000000000001e-06 2023-11-25T04:24:25.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:25.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093940+0000", 2023-11-25T04:24:25.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4799e-05 2023-11-25T04:24:25.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.946 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:25.946 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094052+0000", 2023-11-25T04:24:25.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011156500000000001 2023-11-25T04:24:25.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.946 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:25.946 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108928+0000", 2023-11-25T04:24:25.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014876245 2023-11-25T04:24:25.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.946 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:25.947 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108972+0000", 2023-11-25T04:24:25.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4477999999999997e-05 2023-11-25T04:24:25.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.947 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:25.947 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109042+0000", 2023-11-25T04:24:25.947 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9861000000000001e-05 2023-11-25T04:24:25.947 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:25.947 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:25.947 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:25.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.948 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2865 4.36 4:6ebc5ac6:::1000000110f.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:25.948 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.093906+0000", 2023-11-25T04:24:25.948 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.82780578900000001, 2023-11-25T04:24:25.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014962628, 2023-11-25T04:24:25.948 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:25.948 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:25.948 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:25.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.948 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:25.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093906+0000", 2023-11-25T04:24:25.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:25.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:25.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093906+0000", 2023-11-25T04:24:25.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:25.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:25.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093906+0000", 2023-11-25T04:24:25.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4400000000000001e-07 2023-11-25T04:24:25.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:25.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093908+0000", 2023-11-25T04:24:25.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8640000000000001e-06 2023-11-25T04:24:25.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:25.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093914+0000", 2023-11-25T04:24:25.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0170000000000002e-06 2023-11-25T04:24:25.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:25.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093921+0000", 2023-11-25T04:24:25.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6529999999999997e-06 2023-11-25T04:24:25.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:25.951 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094258+0000", 2023-11-25T04:24:25.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00033703599999999998 2023-11-25T04:24:25.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:25.951 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094340+0000", 2023-11-25T04:24:25.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2279000000000003e-05 2023-11-25T04:24:25.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.952 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:25.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094416+0000", 2023-11-25T04:24:25.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6248999999999995e-05 2023-11-25T04:24:25.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.952 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:25.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.096134+0000", 2023-11-25T04:24:25.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001717611 2023-11-25T04:24:25.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.952 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:25.953 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108809+0000", 2023-11-25T04:24:25.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012675327 2023-11-25T04:24:25.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:25.953 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108836+0000", 2023-11-25T04:24:25.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6509999999999999e-05 2023-11-25T04:24:25.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:25.953 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108869+0000", 2023-11-25T04:24:25.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2737999999999997e-05 2023-11-25T04:24:25.954 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:25.954 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:25.954 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:25.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.954 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:2852 4.2d e47/17 4:b745ac76:::100000010f5.00000000:head v 47'126, mlcod=47'124)", 2023-11-25T04:24:25.954 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.093987+0000", 2023-11-25T04:24:25.954 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.82772522800000004, 2023-11-25T04:24:25.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014613256, 2023-11-25T04:24:25.954 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:25.954 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:25.954 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:25.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:25.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093987+0000", 2023-11-25T04:24:25.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:25.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:25.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093987+0000", 2023-11-25T04:24:25.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:25.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:25.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093987+0000", 2023-11-25T04:24:25.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7000000000000001e-07 2023-11-25T04:24:25.956 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.956 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.956 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:25.956 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093990+0000", 2023-11-25T04:24:25.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8729999999999999e-06 2023-11-25T04:24:25.956 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.956 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.956 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:25.956 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093997+0000", 2023-11-25T04:24:25.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.973e-06 2023-11-25T04:24:25.956 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.957 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:25.957 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094001+0000", 2023-11-25T04:24:25.957 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7360000000000001e-06 2023-11-25T04:24:25.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.957 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:25.957 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094263+0000", 2023-11-25T04:24:25.957 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026173400000000001 2023-11-25T04:24:25.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:25.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094278+0000", 2023-11-25T04:24:25.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4681999999999999e-05 2023-11-25T04:24:25.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:25.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108563+0000", 2023-11-25T04:24:25.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014284738 2023-11-25T04:24:25.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:25.959 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108600+0000", 2023-11-25T04:24:25.959 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7249999999999997e-05 2023-11-25T04:24:25.959 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:25.959 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:25.959 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:25.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.959 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2866 4.36 4:6f6907fb:::10000001111.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:25.959 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.094033+0000", 2023-11-25T04:24:25.959 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.82767906300000005, 2023-11-25T04:24:25.959 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014885591, 2023-11-25T04:24:25.959 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:25.959 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:25.960 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:25.960 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.960 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:25.960 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094033+0000", 2023-11-25T04:24:25.960 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:25.960 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.960 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.960 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:25.960 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094033+0000", 2023-11-25T04:24:25.960 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:25.960 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.960 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.960 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:25.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094033+0000", 2023-11-25T04:24:25.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.06e-07 2023-11-25T04:24:25.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.961 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:25.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094036+0000", 2023-11-25T04:24:25.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.508e-06 2023-11-25T04:24:25.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.961 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:25.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094044+0000", 2023-11-25T04:24:25.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5909999999999996e-06 2023-11-25T04:24:25.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:25.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094051+0000", 2023-11-25T04:24:25.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0319999999999998e-06 2023-11-25T04:24:25.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:25.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094727+0000", 2023-11-25T04:24:25.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00067523300000000002 2023-11-25T04:24:25.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:25.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094795+0000", 2023-11-25T04:24:25.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8156000000000003e-05 2023-11-25T04:24:25.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:25.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094898+0000", 2023-11-25T04:24:25.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103329 2023-11-25T04:24:25.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:25.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108803+0000", 2023-11-25T04:24:25.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013904385 2023-11-25T04:24:25.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.964 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.964 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:25.964 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108875+0000", 2023-11-25T04:24:25.964 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1958999999999993e-05 2023-11-25T04:24:25.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.964 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.964 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:25.964 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108899+0000", 2023-11-25T04:24:25.964 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4019000000000002e-05 2023-11-25T04:24:25.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.964 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.964 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:25.965 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108918+0000", 2023-11-25T04:24:25.965 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9772999999999999e-05 2023-11-25T04:24:25.965 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:25.965 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:25.965 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:25.965 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.965 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.965 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:2854 4.2d e47/17 4:b5c259cf:::100000010f9.00000000:head v 47'127, mlcod=47'124)", 2023-11-25T04:24:25.965 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.094793+0000", 2023-11-25T04:24:25.965 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.82691877400000002, 2023-11-25T04:24:25.965 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014650764, 2023-11-25T04:24:25.965 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:25.965 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:25.966 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:25.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:25.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094793+0000", 2023-11-25T04:24:25.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:25.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:25.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094793+0000", 2023-11-25T04:24:25.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:25.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.967 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:25.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094793+0000", 2023-11-25T04:24:25.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3700000000000001e-07 2023-11-25T04:24:25.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.967 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:25.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094798+0000", 2023-11-25T04:24:25.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1200000000000004e-06 2023-11-25T04:24:25.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.967 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:25.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094805+0000", 2023-11-25T04:24:25.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9759999999999998e-06 2023-11-25T04:24:25.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.968 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:25.968 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094811+0000", 2023-11-25T04:24:25.968 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1410000000000002e-06 2023-11-25T04:24:25.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.968 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:25.968 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094838+0000", 2023-11-25T04:24:25.968 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6995000000000002e-05 2023-11-25T04:24:25.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.968 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:25.969 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094848+0000", 2023-11-25T04:24:25.969 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0115999999999999e-05 2023-11-25T04:24:25.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.969 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:25.969 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109408+0000", 2023-11-25T04:24:25.969 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014559825 2023-11-25T04:24:25.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.969 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:25.969 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109444+0000", 2023-11-25T04:24:25.969 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6154e-05 2023-11-25T04:24:25.969 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:25.970 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:25.970 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:25.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.970 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2869 4.6 4:63719fb0:::10000001117.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:25.970 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.094910+0000", 2023-11-25T04:24:25.970 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.82680181699999999, 2023-11-25T04:24:25.970 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014132570000000001, 2023-11-25T04:24:25.970 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:25.970 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:25.970 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:25.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.971 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:25.971 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094910+0000", 2023-11-25T04:24:25.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:25.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.971 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:25.971 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094910+0000", 2023-11-25T04:24:25.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:25.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.971 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:25.971 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094910+0000", 2023-11-25T04:24:25.972 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.58e-07 2023-11-25T04:24:25.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:25.972 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094913+0000", 2023-11-25T04:24:25.972 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.198e-06 2023-11-25T04:24:25.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:25.972 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094921+0000", 2023-11-25T04:24:25.972 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1629999999999996e-06 2023-11-25T04:24:25.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.973 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:25.973 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094926+0000", 2023-11-25T04:24:25.973 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7559999999999999e-06 2023-11-25T04:24:25.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.973 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.973 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:25.973 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094966+0000", 2023-11-25T04:24:25.973 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0151999999999998e-05 2023-11-25T04:24:25.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.973 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.973 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:25.973 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095073+0000", 2023-11-25T04:24:25.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107657 2023-11-25T04:24:25.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.974 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:25.974 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095158+0000", 2023-11-25T04:24:25.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5136999999999995e-05 2023-11-25T04:24:25.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.974 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:25.974 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108924+0000", 2023-11-25T04:24:25.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013765727 2023-11-25T04:24:25.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:25.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108958+0000", 2023-11-25T04:24:25.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3417000000000002e-05 2023-11-25T04:24:25.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:25.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109012+0000", 2023-11-25T04:24:25.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3939e-05 2023-11-25T04:24:25.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.976 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:25.976 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109043+0000", 2023-11-25T04:24:25.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0966000000000001e-05 2023-11-25T04:24:25.976 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:25.976 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:25.976 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:25.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.976 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2871 4.3f 4:fff10e11:::1000000111b.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:25.977 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.094936+0000", 2023-11-25T04:24:25.977 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.82677597899999999, 2023-11-25T04:24:25.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014261726000000001, 2023-11-25T04:24:25.977 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:25.977 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:25.977 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:25.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.977 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:25.977 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094936+0000", 2023-11-25T04:24:25.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:25.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.978 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:25.978 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094936+0000", 2023-11-25T04:24:25.978 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:25.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.978 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.978 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:25.978 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094936+0000", 2023-11-25T04:24:25.978 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9900000000000001e-07 2023-11-25T04:24:25.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.978 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.978 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:25.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094938+0000", 2023-11-25T04:24:25.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9599999999999999e-06 2023-11-25T04:24:25.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:25.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094944+0000", 2023-11-25T04:24:25.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2219999999999999e-06 2023-11-25T04:24:25.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:25.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094950+0000", 2023-11-25T04:24:25.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1329999999999998e-06 2023-11-25T04:24:25.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:25.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094993+0000", 2023-11-25T04:24:25.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3305e-05 2023-11-25T04:24:25.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:25.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095107+0000", 2023-11-25T04:24:25.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011449400000000001 2023-11-25T04:24:25.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:25.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095176+0000", 2023-11-25T04:24:25.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8330000000000005e-05 2023-11-25T04:24:25.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:25.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107939+0000", 2023-11-25T04:24:25.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012763801 2023-11-25T04:24:25.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:25.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109128+0000", 2023-11-25T04:24:25.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011884770000000001 2023-11-25T04:24:25.982 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.982 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.982 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:25.982 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109166+0000", 2023-11-25T04:24:25.982 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7948999999999999e-05 2023-11-25T04:24:25.982 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.982 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.982 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:25.982 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109198+0000", 2023-11-25T04:24:25.982 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1656000000000003e-05 2023-11-25T04:24:25.982 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:25.982 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:25.983 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:25.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.983 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2872 4.32 4:4fb002bc:::1000000111d.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:25.983 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.094961+0000", 2023-11-25T04:24:25.983 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.82675127500000001, 2023-11-25T04:24:25.983 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014631564999999999, 2023-11-25T04:24:25.983 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:25.983 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:25.983 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:25.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.983 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:25.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094961+0000", 2023-11-25T04:24:25.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:25.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.984 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:25.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094961+0000", 2023-11-25T04:24:25.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:25.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.984 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:25.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094961+0000", 2023-11-25T04:24:25.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9299999999999999e-07 2023-11-25T04:24:25.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:25.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094963+0000", 2023-11-25T04:24:25.985 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9209999999999999e-06 2023-11-25T04:24:25.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:25.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094968+0000", 2023-11-25T04:24:25.985 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.502e-06 2023-11-25T04:24:25.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:25.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094973+0000", 2023-11-25T04:24:25.986 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8559999999999998e-06 2023-11-25T04:24:25.986 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.986 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.986 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:25.986 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107722+0000", 2023-11-25T04:24:25.986 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012748534000000001 2023-11-25T04:24:25.986 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.986 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.986 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:25.986 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107844+0000", 2023-11-25T04:24:25.986 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012233400000000001 2023-11-25T04:24:25.987 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.987 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.987 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:25.987 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109469+0000", 2023-11-25T04:24:25.987 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001625175 2023-11-25T04:24:25.987 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.987 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.987 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:25.987 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109518+0000", 2023-11-25T04:24:25.987 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8980000000000002e-05 2023-11-25T04:24:25.987 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.987 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.987 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:25.988 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109592+0000", 2023-11-25T04:24:25.988 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3869999999999996e-05 2023-11-25T04:24:25.988 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:25.988 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:25.988 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:25.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.988 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2873 4.13 4:cae4f303:::1000000111f.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:25.988 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.094991+0000", 2023-11-25T04:24:25.988 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.82672068300000001, 2023-11-25T04:24:25.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015101692, 2023-11-25T04:24:25.989 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:25.989 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:25.989 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:25.989 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.989 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:25.989 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094991+0000", 2023-11-25T04:24:25.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:25.989 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.989 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.989 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:25.990 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094991+0000", 2023-11-25T04:24:25.990 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:25.990 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.990 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.990 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:25.990 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094992+0000", 2023-11-25T04:24:25.990 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7e-07 2023-11-25T04:24:25.990 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.990 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.990 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:25.990 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094994+0000", 2023-11-25T04:24:25.990 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2110000000000001e-06 2023-11-25T04:24:25.990 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.991 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.991 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:25.991 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095000+0000", 2023-11-25T04:24:25.991 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.241e-06 2023-11-25T04:24:25.991 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.991 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.991 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:25.991 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095006+0000", 2023-11-25T04:24:25.991 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3659999999999997e-06 2023-11-25T04:24:25.991 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.991 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.991 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:25.992 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108369+0000", 2023-11-25T04:24:25.992 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01336247 2023-11-25T04:24:25.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.992 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:25.992 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108492+0000", 2023-11-25T04:24:25.992 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000123192 2023-11-25T04:24:25.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.992 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:25.992 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109975+0000", 2023-11-25T04:24:25.992 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001482894 2023-11-25T04:24:25.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.993 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:25.993 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110024+0000", 2023-11-25T04:24:25.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9097000000000001e-05 2023-11-25T04:24:25.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.993 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:25.993 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110093+0000", 2023-11-25T04:24:25.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8751000000000006e-05 2023-11-25T04:24:25.993 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:25.993 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:25.993 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:25.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.994 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2874 4.14 4:2b557d2e:::10000001121.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:25.994 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095019+0000", 2023-11-25T04:24:25.994 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.82669315700000001, 2023-11-25T04:24:25.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014998296, 2023-11-25T04:24:25.994 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:25.994 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:25.994 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:25.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:25.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095019+0000", 2023-11-25T04:24:25.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:25.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.995 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:25.995 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095019+0000", 2023-11-25T04:24:25.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:25.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.995 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:25.995 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095019+0000", 2023-11-25T04:24:25.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7300000000000001e-07 2023-11-25T04:24:25.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.996 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:25.996 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095021+0000", 2023-11-25T04:24:25.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0099999999999998e-06 2023-11-25T04:24:25.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.996 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:25.996 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095028+0000", 2023-11-25T04:24:25.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7870000000000004e-06 2023-11-25T04:24:25.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.996 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:25.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095033+0000", 2023-11-25T04:24:25.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2039999999999996e-06 2023-11-25T04:24:25.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.997 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:25.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107740+0000", 2023-11-25T04:24:25.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012706693 2023-11-25T04:24:25.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.997 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:25.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107817+0000", 2023-11-25T04:24:25.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7599e-05 2023-11-25T04:24:25.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:25.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107961+0000", 2023-11-25T04:24:25.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000143194 2023-11-25T04:24:25.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:25.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109364+0000", 2023-11-25T04:24:25.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001403226 2023-11-25T04:24:25.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:25.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109507+0000", 2023-11-25T04:24:25.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000142991 2023-11-25T04:24:25.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:25.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109986+0000", 2023-11-25T04:24:25.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00047906899999999999 2023-11-25T04:24:25.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:25.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:25.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:26.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110017+0000", 2023-11-25T04:24:26.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1050000000000003e-05 2023-11-25T04:24:26.000 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:26.000 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:26.000 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:26.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.000 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2875 4.d 4:b17d5dc6:::10000001123.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:26.000 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095049+0000", 2023-11-25T04:24:26.000 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.826663233, 2023-11-25T04:24:26.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014882886, 2023-11-25T04:24:26.001 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:26.001 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:26.001 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:26.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.001 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:26.001 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095049+0000", 2023-11-25T04:24:26.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:26.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.001 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:26.001 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095049+0000", 2023-11-25T04:24:26.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:26.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:26.002 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095049+0000", 2023-11-25T04:24:26.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6100000000000001e-07 2023-11-25T04:24:26.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:26.002 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095051+0000", 2023-11-25T04:24:26.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1449999999999998e-06 2023-11-25T04:24:26.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:26.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095058+0000", 2023-11-25T04:24:26.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.601e-06 2023-11-25T04:24:26.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:26.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095064+0000", 2023-11-25T04:24:26.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1079999999999998e-06 2023-11-25T04:24:26.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:26.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108015+0000", 2023-11-25T04:24:26.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012951582 2023-11-25T04:24:26.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:26.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108260+0000", 2023-11-25T04:24:26.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024445199999999998 2023-11-25T04:24:26.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:26.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109825+0000", 2023-11-25T04:24:26.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0015651759999999999 2023-11-25T04:24:26.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:26.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109866+0000", 2023-11-25T04:24:26.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0629e-05 2023-11-25T04:24:26.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:26.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109931+0000", 2023-11-25T04:24:26.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5732e-05 2023-11-25T04:24:26.005 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:26.005 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:26.005 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:26.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.006 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2876 4.3b 4:dc871776:::10000001125.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:26.006 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095077+0000", 2023-11-25T04:24:26.006 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.82663461199999999, 2023-11-25T04:24:26.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014500944, 2023-11-25T04:24:26.006 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:26.006 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:26.006 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:26.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.006 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:26.006 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095077+0000", 2023-11-25T04:24:26.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:26.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:26.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095077+0000", 2023-11-25T04:24:26.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:26.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:26.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095078+0000", 2023-11-25T04:24:26.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9799999999999999e-07 2023-11-25T04:24:26.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:26.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095079+0000", 2023-11-25T04:24:26.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.877e-06 2023-11-25T04:24:26.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:26.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095086+0000", 2023-11-25T04:24:26.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2439999999999998e-06 2023-11-25T04:24:26.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:26.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095094+0000", 2023-11-25T04:24:26.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9710000000000004e-06 2023-11-25T04:24:26.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:26.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107732+0000", 2023-11-25T04:24:26.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012638104000000001 2023-11-25T04:24:26.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:26.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107854+0000", 2023-11-25T04:24:26.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012261600000000001 2023-11-25T04:24:26.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:26.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107941+0000", 2023-11-25T04:24:26.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6935999999999996e-05 2023-11-25T04:24:26.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:26.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108871+0000", 2023-11-25T04:24:26.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00092955499999999999 2023-11-25T04:24:26.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:26.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109483+0000", 2023-11-25T04:24:26.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00061206100000000001 2023-11-25T04:24:26.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:26.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109527+0000", 2023-11-25T04:24:26.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4112e-05 2023-11-25T04:24:26.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:26.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109578+0000", 2023-11-25T04:24:26.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1069999999999997e-05 2023-11-25T04:24:26.012 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:26.012 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:26.012 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:26.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.013 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2878 4.3e 4:7ff98b9d:::10000001129.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:26.013 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095101+0000", 2023-11-25T04:24:26.013 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.82661082900000005, 2023-11-25T04:24:26.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015015137, 2023-11-25T04:24:26.013 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:26.013 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:26.013 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:26.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:26.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095101+0000", 2023-11-25T04:24:26.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:26.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:26.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095101+0000", 2023-11-25T04:24:26.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:26.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:26.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095101+0000", 2023-11-25T04:24:26.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7399999999999999e-07 2023-11-25T04:24:26.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:26.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095103+0000", 2023-11-25T04:24:26.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7600000000000001e-06 2023-11-25T04:24:26.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:26.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095109+0000", 2023-11-25T04:24:26.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8189999999999997e-06 2023-11-25T04:24:26.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:26.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095114+0000", 2023-11-25T04:24:26.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6770000000000003e-06 2023-11-25T04:24:26.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.016 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:26.016 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107724+0000", 2023-11-25T04:24:26.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012610909999999999 2023-11-25T04:24:26.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.016 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:26.016 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108483+0000", 2023-11-25T04:24:26.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000758407 2023-11-25T04:24:26.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:26.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108568+0000", 2023-11-25T04:24:26.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4707000000000001e-05 2023-11-25T04:24:26.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:26.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109687+0000", 2023-11-25T04:24:26.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011192750000000001 2023-11-25T04:24:26.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:26.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110049+0000", 2023-11-25T04:24:26.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00036210600000000001 2023-11-25T04:24:26.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:26.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110086+0000", 2023-11-25T04:24:26.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7500999999999999e-05 2023-11-25T04:24:26.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.021 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:26.021 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110116+0000", 2023-11-25T04:24:26.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9601e-05 2023-11-25T04:24:26.021 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:26.021 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:26.021 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:26.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.021 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2879 4.2a 4:5524ace2:::1000000112b.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:26.022 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095120+0000", 2023-11-25T04:24:26.022 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.82659204900000005, 2023-11-25T04:24:26.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014587937, 2023-11-25T04:24:26.022 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:26.022 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:26.022 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:26.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:26.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095120+0000", 2023-11-25T04:24:26.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:26.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:26.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095120+0000", 2023-11-25T04:24:26.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:26.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:26.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095120+0000", 2023-11-25T04:24:26.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5600000000000001e-07 2023-11-25T04:24:26.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:26.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095122+0000", 2023-11-25T04:24:26.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.694e-06 2023-11-25T04:24:26.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:26.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095127+0000", 2023-11-25T04:24:26.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5099999999999998e-06 2023-11-25T04:24:26.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:26.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095132+0000", 2023-11-25T04:24:26.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6210000000000003e-06 2023-11-25T04:24:26.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:26.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107729+0000", 2023-11-25T04:24:26.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012596931 2023-11-25T04:24:26.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:26.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107856+0000", 2023-11-25T04:24:26.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012707699999999999 2023-11-25T04:24:26.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:26.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109600+0000", 2023-11-25T04:24:26.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001744298 2023-11-25T04:24:26.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:26.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109641+0000", 2023-11-25T04:24:26.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0769000000000002e-05 2023-11-25T04:24:26.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:26.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109708+0000", 2023-11-25T04:24:26.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6680999999999994e-05 2023-11-25T04:24:26.027 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:26.027 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:26.027 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:26.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.027 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2882 4.1c 4:3a7c2e07:::10000001131.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:26.027 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095141+0000", 2023-11-25T04:24:26.027 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.82657067399999995, 2023-11-25T04:24:26.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014949469999999999, 2023-11-25T04:24:26.027 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:26.028 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:26.028 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:26.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:26.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095141+0000", 2023-11-25T04:24:26.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:26.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:26.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095141+0000", 2023-11-25T04:24:26.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:26.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:26.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095142+0000", 2023-11-25T04:24:26.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1899999999999998e-07 2023-11-25T04:24:26.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:26.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095143+0000", 2023-11-25T04:24:26.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7770000000000001e-06 2023-11-25T04:24:26.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:26.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095149+0000", 2023-11-25T04:24:26.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9909999999999999e-06 2023-11-25T04:24:26.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:26.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095154+0000", 2023-11-25T04:24:26.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3880000000000002e-06 2023-11-25T04:24:26.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:26.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107736+0000", 2023-11-25T04:24:26.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012582477999999999 2023-11-25T04:24:26.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:26.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107868+0000", 2023-11-25T04:24:26.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013134400000000001 2023-11-25T04:24:26.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:26.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107959+0000", 2023-11-25T04:24:26.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1253000000000002e-05 2023-11-25T04:24:26.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:26.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109325+0000", 2023-11-25T04:24:26.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0013662520000000001 2023-11-25T04:24:26.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:26.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110023+0000", 2023-11-25T04:24:26.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00069838799999999998 2023-11-25T04:24:26.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:26.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110061+0000", 2023-11-25T04:24:26.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7867999999999997e-05 2023-11-25T04:24:26.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.033 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:26.033 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110091+0000", 2023-11-25T04:24:26.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9312e-05 2023-11-25T04:24:26.033 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:26.033 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:26.033 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:26.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.033 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2883 4.3e 4:7d82be99:::10000001133.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:26.034 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095942+0000", 2023-11-25T04:24:26.034 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.82576938799999999, 2023-11-25T04:24:26.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014290573000000001, 2023-11-25T04:24:26.034 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:26.034 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:26.034 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:26.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:26.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095942+0000", 2023-11-25T04:24:26.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:26.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:26.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095942+0000", 2023-11-25T04:24:26.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:26.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:26.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095944+0000", 2023-11-25T04:24:26.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.632e-06 2023-11-25T04:24:26.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:26.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095947+0000", 2023-11-25T04:24:26.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7760000000000002e-06 2023-11-25T04:24:26.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:26.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095954+0000", 2023-11-25T04:24:26.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5379999999999997e-06 2023-11-25T04:24:26.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:26.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095960+0000", 2023-11-25T04:24:26.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9209999999999997e-06 2023-11-25T04:24:26.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.037 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:26.037 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108809+0000", 2023-11-25T04:24:26.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012848972 2023-11-25T04:24:26.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.037 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:26.037 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108921+0000", 2023-11-25T04:24:26.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000111767 2023-11-25T04:24:26.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:26.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109008+0000", 2023-11-25T04:24:26.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6576999999999995e-05 2023-11-25T04:24:26.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:26.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110124+0000", 2023-11-25T04:24:26.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011163709999999999 2023-11-25T04:24:26.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:26.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110155+0000", 2023-11-25T04:24:26.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1248999999999998e-05 2023-11-25T04:24:26.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:26.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110202+0000", 2023-11-25T04:24:26.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7097e-05 2023-11-25T04:24:26.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:26.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110233+0000", 2023-11-25T04:24:26.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0673000000000003e-05 2023-11-25T04:24:26.039 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:26.039 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:26.039 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:26.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.040 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2886 4.25 4:a579baeb:::10000001139.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:26.040 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095994+0000", 2023-11-25T04:24:26.040 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.82571806000000003, 2023-11-25T04:24:26.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013825356, 2023-11-25T04:24:26.040 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:26.040 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:26.040 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:26.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:26.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095994+0000", 2023-11-25T04:24:26.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:26.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:26.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095994+0000", 2023-11-25T04:24:26.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:26.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:26.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095996+0000", 2023-11-25T04:24:26.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8780000000000001e-06 2023-11-25T04:24:26.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:26.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095998+0000", 2023-11-25T04:24:26.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1560000000000002e-06 2023-11-25T04:24:26.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:26.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.096003+0000", 2023-11-25T04:24:26.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1080000000000001e-06 2023-11-25T04:24:26.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:26.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.096010+0000", 2023-11-25T04:24:26.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0650000000000001e-06 2023-11-25T04:24:26.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:26.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107742+0000", 2023-11-25T04:24:26.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011731691000000001 2023-11-25T04:24:26.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:26.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108063+0000", 2023-11-25T04:24:26.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00032168600000000001 2023-11-25T04:24:26.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:26.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109689+0000", 2023-11-25T04:24:26.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001625556 2023-11-25T04:24:26.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:26.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109740+0000", 2023-11-25T04:24:26.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1125000000000003e-05 2023-11-25T04:24:26.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:26.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109819+0000", 2023-11-25T04:24:26.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9091000000000006e-05 2023-11-25T04:24:26.045 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:26.045 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:26.045 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:26.045 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:26.045 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:26.045 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:26.047 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:24:26.047 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:26.045+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:26.141 INFO:teuthology.orchestra.run.smithi067.stderr:nodeep-scrub is unset 2023-11-25T04:24:26.147 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:26.146+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:26.156 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:26.156 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:24:26.156 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2023-11-25T04:24:26.156 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:26.165 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:24:26.247 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:26.246+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:26.270 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:26.270 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:24:26.270 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2023-11-25T04:24:26.271 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2023-11-25T04:24:26.271 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:26.279 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:24:26.349 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:26.347+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:26.385 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:26.386 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2023-11-25T04:24:26.386 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2023-11-25T04:24:26.386 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2023-11-25T04:24:26.386 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.386 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:26.386 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:24:26.386 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 99.351329258999996, 2023-11-25T04:24:26.386 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01072943, 2023-11-25T04:24:26.386 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:26.386 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:26.386 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:26.387 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.387 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:26.387 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:24:26.387 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:26.387 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.387 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.387 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:26.387 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:24:26.387 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:26.388 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.388 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.388 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:26.388 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031829+0000", 2023-11-25T04:24:26.388 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.666e-06 2023-11-25T04:24:26.388 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.388 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.388 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:26.388 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031839+0000", 2023-11-25T04:24:26.388 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9490000000000008e-06 2023-11-25T04:24:26.389 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.389 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.389 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:26.389 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031857+0000", 2023-11-25T04:24:26.389 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8093000000000001e-05 2023-11-25T04:24:26.389 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.389 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.389 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:26.389 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031882+0000", 2023-11-25T04:24:26.389 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4294000000000002e-05 2023-11-25T04:24:26.389 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.390 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.390 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:26.390 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033130+0000", 2023-11-25T04:24:26.390 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001248264 2023-11-25T04:24:26.390 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.390 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.390 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:26.390 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033426+0000", 2023-11-25T04:24:26.391 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029620999999999998 2023-11-25T04:24:26.391 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.391 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.391 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:26.391 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033814+0000", 2023-11-25T04:24:26.391 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000387742 2023-11-25T04:24:26.391 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.392 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.392 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:26.392 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036793+0000", 2023-11-25T04:24:26.392 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002979598 2023-11-25T04:24:26.392 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.392 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.392 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:26.392 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042455+0000", 2023-11-25T04:24:26.392 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0056619260000000003 2023-11-25T04:24:26.392 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.392 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.393 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:26.393 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042502+0000", 2023-11-25T04:24:26.393 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7139999999999999e-05 2023-11-25T04:24:26.393 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.393 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.393 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:26.393 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042557+0000", 2023-11-25T04:24:26.393 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4548e-05 2023-11-25T04:24:26.393 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:26.393 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:26.393 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:26.393 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.394 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.394 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:26.394 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:24:26.394 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 99.350130723999996, 2023-11-25T04:24:26.394 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011505202000000001, 2023-11-25T04:24:26.394 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:26.394 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:26.394 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:26.394 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.394 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:26.394 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:24:26.395 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:26.395 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.395 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.395 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:26.395 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:24:26.395 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:26.395 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.395 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.395 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:26.395 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033027+0000", 2023-11-25T04:24:26.395 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0699999999999999e-06 2023-11-25T04:24:26.395 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.396 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.396 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:26.396 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033035+0000", 2023-11-25T04:24:26.396 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0830000000000004e-06 2023-11-25T04:24:26.396 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.396 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.396 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:26.396 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033059+0000", 2023-11-25T04:24:26.396 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3975e-05 2023-11-25T04:24:26.396 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.396 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.396 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:26.397 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033482+0000", 2023-11-25T04:24:26.397 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042253700000000001 2023-11-25T04:24:26.397 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.397 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.397 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:26.397 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034520+0000", 2023-11-25T04:24:26.397 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001037991 2023-11-25T04:24:26.397 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.397 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.397 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:26.397 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034611+0000", 2023-11-25T04:24:26.398 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1113999999999995e-05 2023-11-25T04:24:26.398 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.398 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.398 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:26.398 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034769+0000", 2023-11-25T04:24:26.398 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000157641 2023-11-25T04:24:26.398 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.398 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.399 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:26.399 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038292+0000", 2023-11-25T04:24:26.399 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035231160000000002 2023-11-25T04:24:26.399 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.399 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.399 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:26.399 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044440+0000", 2023-11-25T04:24:26.399 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0061486980000000002 2023-11-25T04:24:26.399 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.399 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.399 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:26.399 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044495+0000", 2023-11-25T04:24:26.400 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4339000000000003e-05 2023-11-25T04:24:26.400 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.400 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.400 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:26.400 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044531+0000", 2023-11-25T04:24:26.400 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6637999999999997e-05 2023-11-25T04:24:26.400 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:26.400 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:26.400 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:26.400 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.400 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.400 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:26 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:26.401 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:26.401 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 99.349541962000004, 2023-11-25T04:24:26.401 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011086945000000001, 2023-11-25T04:24:26.401 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:26.401 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:26.401 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:26.401 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.401 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:26.401 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:26.401 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:26.402 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.402 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.402 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:26.402 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:26.402 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:26.402 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.402 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.402 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:26.402 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:26.402 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1699999999999998e-07 2023-11-25T04:24:26.402 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.403 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.403 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:26.403 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033627+0000", 2023-11-25T04:24:26.403 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.163e-05 2023-11-25T04:24:26.403 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.403 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.403 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:26.403 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033636+0000", 2023-11-25T04:24:26.403 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6859999999999995e-06 2023-11-25T04:24:26.403 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.403 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.403 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:26.404 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033642+0000", 2023-11-25T04:24:26.404 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3149999999999997e-06 2023-11-25T04:24:26.404 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.404 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.404 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:26.404 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035414+0000", 2023-11-25T04:24:26.404 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0017716380000000001 2023-11-25T04:24:26.404 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.404 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.404 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:26.404 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035539+0000", 2023-11-25T04:24:26.404 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000125477 2023-11-25T04:24:26.405 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.405 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.405 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:26.405 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035672+0000", 2023-11-25T04:24:26.405 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000133316 2023-11-25T04:24:26.405 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.405 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.405 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:26.405 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038649+0000", 2023-11-25T04:24:26.405 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029764679999999999 2023-11-25T04:24:26.405 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.405 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.406 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:26.406 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044578+0000", 2023-11-25T04:24:26.406 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0059293510000000002 2023-11-25T04:24:26.406 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.406 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.406 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:26.406 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044651+0000", 2023-11-25T04:24:26.406 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2513000000000002e-05 2023-11-25T04:24:26.406 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.406 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.406 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:26.406 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044702+0000", 2023-11-25T04:24:26.407 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1034e-05 2023-11-25T04:24:26.407 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:26.407 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:26.407 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:26.407 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.407 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.407 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:29 3.b 3:d0630e4c:::mds0_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:26.407 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:26.407 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 99.349504042000007, 2023-11-25T04:24:26.407 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010980227, 2023-11-25T04:24:26.407 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:26.407 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:26.408 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:26.408 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.408 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:26.408 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:26.408 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:26.408 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.408 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.408 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:26.408 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:26.408 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:26.408 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.408 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.409 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:26.409 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:26.409 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3499999999999996e-07 2023-11-25T04:24:26.409 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.409 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.409 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:26.409 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033658+0000", 2023-11-25T04:24:26.409 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1280000000000001e-06 2023-11-25T04:24:26.409 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.409 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.409 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:26.409 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033667+0000", 2023-11-25T04:24:26.409 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1430000000000004e-06 2023-11-25T04:24:26.410 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.410 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.410 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:26.410 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033678+0000", 2023-11-25T04:24:26.410 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0906e-05 2023-11-25T04:24:26.410 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.410 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.410 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:26.410 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034456+0000", 2023-11-25T04:24:26.410 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00077833899999999996 2023-11-25T04:24:26.410 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.410 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.411 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:26.411 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034575+0000", 2023-11-25T04:24:26.411 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011926000000000001 2023-11-25T04:24:26.411 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.411 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.411 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:26.411 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035001+0000", 2023-11-25T04:24:26.411 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042550999999999998 2023-11-25T04:24:26.411 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.411 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.411 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:26.411 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.037923+0000", 2023-11-25T04:24:26.412 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029219670000000001 2023-11-25T04:24:26.412 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.412 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.412 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:26.412 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044538+0000", 2023-11-25T04:24:26.412 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0066152349999999997 2023-11-25T04:24:26.412 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.412 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.412 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:26.412 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044594+0000", 2023-11-25T04:24:26.412 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5593000000000001e-05 2023-11-25T04:24:26.412 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.412 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.413 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:26.413 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044633+0000", 2023-11-25T04:24:26.413 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9511000000000002e-05 2023-11-25T04:24:26.413 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:26.413 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:26.413 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:26.413 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.413 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.413 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:34 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:26.413 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:24:26.413 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 99.349253929, 2023-11-25T04:24:26.413 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010920445000000001, 2023-11-25T04:24:26.414 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:26.414 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:26.414 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:26.414 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.414 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:26.414 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:24:26.414 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:26.414 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.414 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.414 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:26.414 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:24:26.414 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:26.415 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.415 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.415 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:26.415 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033904+0000", 2023-11-25T04:24:26.415 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6599999999999994e-07 2023-11-25T04:24:26.415 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.415 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.415 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:26.415 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033909+0000", 2023-11-25T04:24:26.415 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5639999999999996e-06 2023-11-25T04:24:26.415 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.415 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.416 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:26.416 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033923+0000", 2023-11-25T04:24:26.416 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3028e-05 2023-11-25T04:24:26.416 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.416 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.416 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:26.416 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034043+0000", 2023-11-25T04:24:26.416 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000120328 2023-11-25T04:24:26.416 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.416 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.416 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:26.416 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035576+0000", 2023-11-25T04:24:26.417 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015333720000000001 2023-11-25T04:24:26.417 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.417 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.417 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:26.417 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035699+0000", 2023-11-25T04:24:26.417 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000123001 2023-11-25T04:24:26.417 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.417 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.417 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:26.417 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044823+0000", 2023-11-25T04:24:26.417 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0091241859999999994 2023-11-25T04:24:26.417 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:26.418 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:26.418 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:26.418 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.418 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.418 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:118 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals,omap-rm-keys] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e44)", 2023-11-25T04:24:26.418 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:17.112648+0000", 2023-11-25T04:24:26.418 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 9.2705085880000002, 2023-11-25T04:24:26.418 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011250779000000001, 2023-11-25T04:24:26.418 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:26.418 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:26.418 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:26.418 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.419 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:26.419 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.112648+0000", 2023-11-25T04:24:26.419 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:26.419 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.419 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.419 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:26.419 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.112648+0000", 2023-11-25T04:24:26.419 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:26.419 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.419 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.419 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:26.419 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.112649+0000", 2023-11-25T04:24:26.420 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0070000000000001e-06 2023-11-25T04:24:26.420 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.420 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.420 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:26.420 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113182+0000", 2023-11-25T04:24:26.420 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00053268100000000002 2023-11-25T04:24:26.420 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.420 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.420 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:26.420 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113196+0000", 2023-11-25T04:24:26.420 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3742e-05 2023-11-25T04:24:26.420 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.421 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.421 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:26.421 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113398+0000", 2023-11-25T04:24:26.421 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000202726 2023-11-25T04:24:26.421 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.421 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.421 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:26.421 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113447+0000", 2023-11-25T04:24:26.421 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8655000000000001e-05 2023-11-25T04:24:26.421 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.421 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.421 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:26.422 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113529+0000", 2023-11-25T04:24:26.422 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1891999999999995e-05 2023-11-25T04:24:26.422 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.422 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.422 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:26.422 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123819+0000", 2023-11-25T04:24:26.422 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010290192 2023-11-25T04:24:26.422 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.422 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.422 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:26.422 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123852+0000", 2023-11-25T04:24:26.422 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2425999999999998e-05 2023-11-25T04:24:26.423 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.423 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.423 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:26.423 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123899+0000", 2023-11-25T04:24:26.423 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7457999999999999e-05 2023-11-25T04:24:26.423 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:26.423 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:26.423 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:26.423 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.423 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.423 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:154 3.1e 3:7bf7e7b5:::200.00000006:head [write 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:26.423 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:21.886430+0000", 2023-11-25T04:24:26.424 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.4967273219999999, 2023-11-25T04:24:26.424 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.017800107999999999, 2023-11-25T04:24:26.424 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:26.424 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:26.424 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:26.424 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.424 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:26.424 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.886430+0000", 2023-11-25T04:24:26.424 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:26.424 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.424 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.424 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:26.425 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.886430+0000", 2023-11-25T04:24:26.425 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:26.425 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.425 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.425 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:26.425 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.886431+0000", 2023-11-25T04:24:26.425 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.178e-06 2023-11-25T04:24:26.425 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.425 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.425 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:26.425 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.892277+0000", 2023-11-25T04:24:26.425 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0058467409999999999 2023-11-25T04:24:26.426 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.426 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.426 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:26.426 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.892288+0000", 2023-11-25T04:24:26.426 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0183000000000001e-05 2023-11-25T04:24:26.426 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.426 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.426 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:26.426 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.892292+0000", 2023-11-25T04:24:26.426 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7829999999999998e-06 2023-11-25T04:24:26.426 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.426 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.427 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:26.427 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.892362+0000", 2023-11-25T04:24:26.427 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0034000000000001e-05 2023-11-25T04:24:26.427 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.427 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.427 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:26.427 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.892432+0000", 2023-11-25T04:24:26.427 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9648999999999997e-05 2023-11-25T04:24:26.427 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.427 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.427 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:26.428 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.892550+0000", 2023-11-25T04:24:26.428 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011825 2023-11-25T04:24:26.428 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.428 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.428 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:26.428 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.899106+0000", 2023-11-25T04:24:26.428 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0065556759999999999 2023-11-25T04:24:26.428 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.428 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.428 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:26.428 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.904145+0000", 2023-11-25T04:24:26.428 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0050393490000000003 2023-11-25T04:24:26.428 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.428 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.429 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:26.429 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.904200+0000", 2023-11-25T04:24:26.429 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4234999999999998e-05 2023-11-25T04:24:26.429 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.429 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.429 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:26.429 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.904230+0000", 2023-11-25T04:24:26.429 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0029999999999999e-05 2023-11-25T04:24:26.429 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:26.429 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:26.429 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:26.429 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.429 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.429 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:576 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals,omap-rm-keys] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:26.430 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:22.111407+0000", 2023-11-25T04:24:26.430 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.2717497160000004, 2023-11-25T04:24:26.430 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.018987317, 2023-11-25T04:24:26.430 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:26.430 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:26.430 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:26.430 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.430 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:26.430 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111407+0000", 2023-11-25T04:24:26.430 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:26.430 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.430 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.430 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:26.431 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111407+0000", 2023-11-25T04:24:26.431 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:26.431 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.431 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.431 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:26.431 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111408+0000", 2023-11-25T04:24:26.431 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8000000000000004e-07 2023-11-25T04:24:26.431 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.431 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.431 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:26.431 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111659+0000", 2023-11-25T04:24:26.431 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000250537 2023-11-25T04:24:26.431 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.432 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.432 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:26.432 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111666+0000", 2023-11-25T04:24:26.432 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.973e-06 2023-11-25T04:24:26.432 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.432 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.432 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:26.432 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111670+0000", 2023-11-25T04:24:26.432 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.668e-06 2023-11-25T04:24:26.432 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.432 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.432 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:26.432 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111709+0000", 2023-11-25T04:24:26.432 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8800000000000001e-05 2023-11-25T04:24:26.433 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.433 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.433 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:26.433 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111761+0000", 2023-11-25T04:24:26.433 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1495000000000001e-05 2023-11-25T04:24:26.433 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.433 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.433 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:26.433 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.130255+0000", 2023-11-25T04:24:26.433 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.018494345999999998 2023-11-25T04:24:26.433 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.433 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.434 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:26.434 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.130311+0000", 2023-11-25T04:24:26.434 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5902000000000001e-05 2023-11-25T04:24:26.434 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.434 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.434 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:26.434 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.130394+0000", 2023-11-25T04:24:26.434 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3715999999999996e-05 2023-11-25T04:24:26.434 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:26.434 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:26.434 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:26.434 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.435 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.435 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:1067 4.21 4:869a7600:::100000011fa.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:26.435 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:22.155038+0000", 2023-11-25T04:24:26.435 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.2281183779999996, 2023-11-25T04:24:26.435 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011313148, 2023-11-25T04:24:26.435 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:26.435 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:26.435 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:26.435 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.435 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:26.435 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155038+0000", 2023-11-25T04:24:26.436 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:26.436 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.436 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.436 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:26.436 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155038+0000", 2023-11-25T04:24:26.436 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:26.436 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.436 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.436 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:26.436 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155039+0000", 2023-11-25T04:24:26.436 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0200000000000002e-07 2023-11-25T04:24:26.436 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.436 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.437 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:26.437 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155049+0000", 2023-11-25T04:24:26.437 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0295000000000001e-05 2023-11-25T04:24:26.437 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.437 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.437 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:26.437 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155058+0000", 2023-11-25T04:24:26.437 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.208e-06 2023-11-25T04:24:26.437 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.437 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.437 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:26.437 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155064+0000", 2023-11-25T04:24:26.437 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4489999999999996e-06 2023-11-25T04:24:26.438 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.438 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.438 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:26.438 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155104+0000", 2023-11-25T04:24:26.438 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0164999999999999e-05 2023-11-25T04:24:26.438 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.438 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.438 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:26.438 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155147+0000", 2023-11-25T04:24:26.438 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2576e-05 2023-11-25T04:24:26.438 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.438 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.439 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:26.439 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155212+0000", 2023-11-25T04:24:26.439 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5516000000000001e-05 2023-11-25T04:24:26.439 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.439 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.439 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:26.439 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.156025+0000", 2023-11-25T04:24:26.439 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00081248599999999996 2023-11-25T04:24:26.439 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.439 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.439 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:26.439 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.166250+0000", 2023-11-25T04:24:26.439 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010224736999999999 2023-11-25T04:24:26.440 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.440 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.440 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:26.440 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.166319+0000", 2023-11-25T04:24:26.440 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9145000000000003e-05 2023-11-25T04:24:26.440 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.440 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.440 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:26.440 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.166352+0000", 2023-11-25T04:24:26.440 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2969000000000002e-05 2023-11-25T04:24:26.440 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:26.440 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:26.441 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:26.441 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.441 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.441 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:1102 4.2f 4:f543b94c:::1000000121d.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:26.441 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:22.158246+0000", 2023-11-25T04:24:26.441 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.2249108550000001, 2023-11-25T04:24:26.441 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010094396, 2023-11-25T04:24:26.441 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:26.441 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:26.441 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:26.441 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.441 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:26.441 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158246+0000", 2023-11-25T04:24:26.442 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:26.442 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.442 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.442 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:26.442 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158246+0000", 2023-11-25T04:24:26.442 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:26.442 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.442 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.442 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:26.442 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158246+0000", 2023-11-25T04:24:26.442 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5299999999999999e-07 2023-11-25T04:24:26.442 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.443 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.443 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:26.443 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158249+0000", 2023-11-25T04:24:26.443 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2840000000000001e-06 2023-11-25T04:24:26.443 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.443 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.443 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:26.443 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158254+0000", 2023-11-25T04:24:26.443 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3449999999999996e-06 2023-11-25T04:24:26.443 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.443 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.443 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:26.444 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158258+0000", 2023-11-25T04:24:26.444 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5030000000000002e-06 2023-11-25T04:24:26.444 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.444 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.444 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:26.444 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158302+0000", 2023-11-25T04:24:26.444 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3908e-05 2023-11-25T04:24:26.444 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.444 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.444 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:26.444 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158398+0000", 2023-11-25T04:24:26.444 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6898000000000006e-05 2023-11-25T04:24:26.444 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.445 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.445 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:26.445 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158480+0000", 2023-11-25T04:24:26.445 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1578999999999999e-05 2023-11-25T04:24:26.445 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.445 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.445 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:26.445 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.159229+0000", 2023-11-25T04:24:26.445 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000749049 2023-11-25T04:24:26.445 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.445 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.445 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:26.445 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.168228+0000", 2023-11-25T04:24:26.446 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0089992930000000002 2023-11-25T04:24:26.446 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.446 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.446 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:26.446 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.168310+0000", 2023-11-25T04:24:26.446 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1348999999999997e-05 2023-11-25T04:24:26.446 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.446 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.446 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:26.446 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.168340+0000", 2023-11-25T04:24:26.446 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0735000000000003e-05 2023-11-25T04:24:26.446 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:26.447 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:26.447 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:26.447 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.447 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.447 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:1105 4.23 4:c6ff07b4:::10000001220.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:26.447 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:22.158714+0000", 2023-11-25T04:24:26.447 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.2244427409999998, 2023-11-25T04:24:26.447 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010412490999999999, 2023-11-25T04:24:26.447 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:26.447 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:26.447 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:26.447 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.448 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:26.448 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158714+0000", 2023-11-25T04:24:26.448 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:26.448 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.448 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.448 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:26.448 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158714+0000", 2023-11-25T04:24:26.448 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:26.448 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.448 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.448 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:26.448 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158715+0000", 2023-11-25T04:24:26.448 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4099999999999998e-07 2023-11-25T04:24:26.449 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.449 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.449 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:26.449 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158718+0000", 2023-11-25T04:24:26.449 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4319999999999999e-06 2023-11-25T04:24:26.449 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.449 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.449 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:26.449 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158729+0000", 2023-11-25T04:24:26.449 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.078e-05 2023-11-25T04:24:26.449 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.449 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.449 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:26.450 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158736+0000", 2023-11-25T04:24:26.450 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4649999999999997e-06 2023-11-25T04:24:26.450 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.450 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.450 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:26.450 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158779+0000", 2023-11-25T04:24:26.450 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3049000000000001e-05 2023-11-25T04:24:26.450 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.450 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.450 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:26.450 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158865+0000", 2023-11-25T04:24:26.450 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5138999999999999e-05 2023-11-25T04:24:26.451 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.451 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.451 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:26.451 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158983+0000", 2023-11-25T04:24:26.451 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000118658 2023-11-25T04:24:26.451 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.451 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.451 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:26.451 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.159645+0000", 2023-11-25T04:24:26.451 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00066193999999999997 2023-11-25T04:24:26.451 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.451 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.451 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:26.452 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.169046+0000", 2023-11-25T04:24:26.452 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0094005400000000006 2023-11-25T04:24:26.452 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.452 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.452 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:26.452 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.169092+0000", 2023-11-25T04:24:26.452 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5840000000000002e-05 2023-11-25T04:24:26.452 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.452 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.452 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:26.452 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.169127+0000", 2023-11-25T04:24:26.452 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5006999999999999e-05 2023-11-25T04:24:26.453 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:26.453 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:26.453 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:26.453 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.453 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.453 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2841 4.4 4:22111d42:::100000010df.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:26.453 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.092102+0000", 2023-11-25T04:24:26.453 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 1.2910552659999999, 2023-11-25T04:24:26.453 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.016449783999999999, 2023-11-25T04:24:26.453 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:26.453 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:26.453 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:26.453 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.454 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:26.454 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092102+0000", 2023-11-25T04:24:26.454 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:26.454 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.454 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.454 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:26.454 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092102+0000", 2023-11-25T04:24:26.454 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:26.454 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.454 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.454 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:26.454 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092102+0000", 2023-11-25T04:24:26.455 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3700000000000001e-07 2023-11-25T04:24:26.455 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.455 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.455 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:26.455 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092104+0000", 2023-11-25T04:24:26.455 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0679999999999999e-06 2023-11-25T04:24:26.455 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.455 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.455 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:26.455 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092111+0000", 2023-11-25T04:24:26.455 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0589999999999997e-06 2023-11-25T04:24:26.455 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.455 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.456 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:26.456 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092130+0000", 2023-11-25T04:24:26.456 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8686999999999999e-05 2023-11-25T04:24:26.456 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.456 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.456 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:26.456 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092443+0000", 2023-11-25T04:24:26.456 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000313519 2023-11-25T04:24:26.456 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.456 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.456 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:26.456 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092553+0000", 2023-11-25T04:24:26.456 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010932100000000001 2023-11-25T04:24:26.457 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.457 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.457 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:26.457 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092639+0000", 2023-11-25T04:24:26.457 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6094999999999995e-05 2023-11-25T04:24:26.457 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.457 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.457 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:26.457 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094679+0000", 2023-11-25T04:24:26.457 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0020402580000000001 2023-11-25T04:24:26.457 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.457 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.457 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:26.458 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108462+0000", 2023-11-25T04:24:26.458 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013783358000000001 2023-11-25T04:24:26.458 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.458 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.458 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:26.458 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108516+0000", 2023-11-25T04:24:26.458 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3483000000000003e-05 2023-11-25T04:24:26.458 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.458 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.458 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:26.458 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108551+0000", 2023-11-25T04:24:26.458 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5499e-05 2023-11-25T04:24:26.459 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:26.459 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:26.459 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:26.459 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.459 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.459 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2852 4.2d 4:b745ac76:::100000010f5.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:26.459 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.093477+0000", 2023-11-25T04:24:26.459 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 1.289680154, 2023-11-25T04:24:26.459 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.015423084, 2023-11-25T04:24:26.459 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:26.459 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:26.459 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:26.459 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.460 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:26.460 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093477+0000", 2023-11-25T04:24:26.460 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:26.460 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.460 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.460 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:26.460 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093477+0000", 2023-11-25T04:24:26.460 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:26.460 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.460 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.460 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:26.460 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093477+0000", 2023-11-25T04:24:26.461 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2500000000000005e-07 2023-11-25T04:24:26.461 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.461 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.461 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:26.461 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093480+0000", 2023-11-25T04:24:26.461 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0189999999999998e-06 2023-11-25T04:24:26.461 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.461 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.461 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:26.461 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093642+0000", 2023-11-25T04:24:26.461 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000162072 2023-11-25T04:24:26.461 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.461 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.462 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:26.462 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093649+0000", 2023-11-25T04:24:26.462 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9659999999999999e-06 2023-11-25T04:24:26.462 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.462 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.462 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:26.462 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093693+0000", 2023-11-25T04:24:26.462 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3794000000000001e-05 2023-11-25T04:24:26.462 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.462 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.462 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:26.462 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093764+0000", 2023-11-25T04:24:26.462 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0981000000000004e-05 2023-11-25T04:24:26.463 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.463 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.463 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:26.463 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093850+0000", 2023-11-25T04:24:26.463 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5536000000000002e-05 2023-11-25T04:24:26.463 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.463 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.463 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:26.463 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094710+0000", 2023-11-25T04:24:26.463 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00086070299999999999 2023-11-25T04:24:26.463 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.463 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.464 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:26.464 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108776+0000", 2023-11-25T04:24:26.464 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01406542 2023-11-25T04:24:26.464 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.464 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.464 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:26.464 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108829+0000", 2023-11-25T04:24:26.464 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2936999999999998e-05 2023-11-25T04:24:26.464 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.464 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.464 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:26.465 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108900+0000", 2023-11-25T04:24:26.465 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1031000000000005e-05 2023-11-25T04:24:26.465 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:26.465 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:26.465 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:26.465 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.465 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.465 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2854 4.2d 4:b5c259cf:::100000010f9.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:26.465 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.093674+0000", 2023-11-25T04:24:26.465 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 1.289482494, 2023-11-25T04:24:26.466 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.015970380999999999, 2023-11-25T04:24:26.466 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:26.466 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:26.466 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:26.466 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.466 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:26.466 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093674+0000", 2023-11-25T04:24:26.466 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:26.466 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.466 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.466 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:26.466 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093674+0000", 2023-11-25T04:24:26.467 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:26.467 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.467 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.467 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:26.467 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093675+0000", 2023-11-25T04:24:26.467 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8400000000000004e-07 2023-11-25T04:24:26.467 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.467 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.467 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:26.467 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093678+0000", 2023-11-25T04:24:26.467 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2229999999999999e-06 2023-11-25T04:24:26.467 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.467 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.468 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:26.468 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093686+0000", 2023-11-25T04:24:26.468 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9990000000000008e-06 2023-11-25T04:24:26.468 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.468 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.468 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:26.468 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093696+0000", 2023-11-25T04:24:26.468 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6479999999999998e-06 2023-11-25T04:24:26.468 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.468 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.468 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:26.468 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094104+0000", 2023-11-25T04:24:26.469 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00040823900000000002 2023-11-25T04:24:26.469 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.469 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.469 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:26.469 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094172+0000", 2023-11-25T04:24:26.469 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8023999999999998e-05 2023-11-25T04:24:26.469 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.469 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.469 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:26.469 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094256+0000", 2023-11-25T04:24:26.469 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3418999999999995e-05 2023-11-25T04:24:26.469 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.469 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.470 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:26.470 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095160+0000", 2023-11-25T04:24:26.470 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00090418599999999997 2023-11-25T04:24:26.470 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.470 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.470 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:26.470 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109551+0000", 2023-11-25T04:24:26.470 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014391571000000001 2023-11-25T04:24:26.470 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.470 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.470 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:26.470 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109601+0000", 2023-11-25T04:24:26.471 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9478000000000003e-05 2023-11-25T04:24:26.471 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.471 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.471 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:26.471 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109645+0000", 2023-11-25T04:24:26.471 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.401e-05 2023-11-25T04:24:26.471 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:26.471 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:26.471 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:26.471 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.471 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.471 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2861 4.4 4:214027a9:::10000001107.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:26.472 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.093829+0000", 2023-11-25T04:24:26.472 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 1.2893280759999999, 2023-11-25T04:24:26.472 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014848917999999999, 2023-11-25T04:24:26.472 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:26.472 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:26.472 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:26.472 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.472 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:26.472 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093829+0000", 2023-11-25T04:24:26.472 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:26.472 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.472 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.473 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:26.473 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093829+0000", 2023-11-25T04:24:26.473 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:26.473 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.473 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.473 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:26.473 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093830+0000", 2023-11-25T04:24:26.473 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1999999999999998e-07 2023-11-25T04:24:26.473 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.473 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.473 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:26.473 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093834+0000", 2023-11-25T04:24:26.473 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.34e-06 2023-11-25T04:24:26.474 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.474 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.474 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:26.474 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093844+0000", 2023-11-25T04:24:26.474 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6690000000000001e-06 2023-11-25T04:24:26.474 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.474 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.474 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:26.474 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093857+0000", 2023-11-25T04:24:26.474 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3434999999999999e-05 2023-11-25T04:24:26.474 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.474 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.475 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:26.475 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093940+0000", 2023-11-25T04:24:26.475 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3047999999999995e-05 2023-11-25T04:24:26.475 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.475 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.475 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:26.475 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094052+0000", 2023-11-25T04:24:26.475 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000111538 2023-11-25T04:24:26.475 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.475 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.475 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:26.475 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094466+0000", 2023-11-25T04:24:26.475 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00041464600000000001 2023-11-25T04:24:26.476 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.476 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.476 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:26.476 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095570+0000", 2023-11-25T04:24:26.476 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001103981 2023-11-25T04:24:26.476 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.476 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.476 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:26.476 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108598+0000", 2023-11-25T04:24:26.476 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013027164000000001 2023-11-25T04:24:26.476 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.476 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.476 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:26.477 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108646+0000", 2023-11-25T04:24:26.477 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8090999999999997e-05 2023-11-25T04:24:26.477 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.477 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.477 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:26.477 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108678+0000", 2023-11-25T04:24:26.477 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2085999999999997e-05 2023-11-25T04:24:26.477 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:26.477 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:26.477 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:26.477 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.477 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.477 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2863 4.12 4:4b84c733:::1000000110b.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:26.478 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.093877+0000", 2023-11-25T04:24:26.478 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 1.2892797489999999, 2023-11-25T04:24:26.478 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014682755, 2023-11-25T04:24:26.478 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:26.478 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:26.478 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:26.478 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.478 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:26.478 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093877+0000", 2023-11-25T04:24:26.478 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:26.478 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.478 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.478 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:26.479 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093877+0000", 2023-11-25T04:24:26.479 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:26.479 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.479 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.479 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:26.479 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093878+0000", 2023-11-25T04:24:26.479 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4199999999999996e-07 2023-11-25T04:24:26.479 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.479 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.479 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:26.479 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093880+0000", 2023-11-25T04:24:26.479 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6189999999999998e-06 2023-11-25T04:24:26.480 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.480 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.480 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:26.480 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093888+0000", 2023-11-25T04:24:26.480 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5510000000000002e-06 2023-11-25T04:24:26.480 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.480 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.480 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:26.480 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093893+0000", 2023-11-25T04:24:26.480 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4179999999999996e-06 2023-11-25T04:24:26.480 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.480 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.480 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:26.481 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093934+0000", 2023-11-25T04:24:26.481 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0648000000000001e-05 2023-11-25T04:24:26.481 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.481 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.481 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:26.481 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094081+0000", 2023-11-25T04:24:26.481 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014753499999999999 2023-11-25T04:24:26.481 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.481 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.481 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:26.481 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094172+0000", 2023-11-25T04:24:26.481 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.0722000000000003e-05 2023-11-25T04:24:26.481 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.482 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.482 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:26.482 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094742+0000", 2023-11-25T04:24:26.482 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00056948600000000002 2023-11-25T04:24:26.482 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.482 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.482 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:26.482 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108491+0000", 2023-11-25T04:24:26.482 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013749568 2023-11-25T04:24:26.482 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.482 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.482 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:26.482 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108525+0000", 2023-11-25T04:24:26.483 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3683999999999998e-05 2023-11-25T04:24:26.483 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.483 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.483 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:26.483 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108560+0000", 2023-11-25T04:24:26.483 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4981999999999998e-05 2023-11-25T04:24:26.483 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:26.483 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:26.483 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:26.483 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.483 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.483 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2880 4.21 4:855834a4:::1000000112d.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:26.484 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.095098+0000", 2023-11-25T04:24:26.484 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 1.288059055, 2023-11-25T04:24:26.484 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013847544, 2023-11-25T04:24:26.484 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:26.484 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:26.484 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:26.484 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.484 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:26.484 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095098+0000", 2023-11-25T04:24:26.484 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:26.484 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.484 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.484 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:26.484 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095098+0000", 2023-11-25T04:24:26.485 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:26.485 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.485 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.485 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:26.485 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095098+0000", 2023-11-25T04:24:26.485 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3000000000000002e-07 2023-11-25T04:24:26.485 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.485 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.485 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:26.485 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095099+0000", 2023-11-25T04:24:26.485 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.331e-06 2023-11-25T04:24:26.485 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.486 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.486 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:26.486 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095103+0000", 2023-11-25T04:24:26.486 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8319999999999999e-06 2023-11-25T04:24:26.486 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.486 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.486 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:26.486 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095108+0000", 2023-11-25T04:24:26.486 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4179999999999999e-06 2023-11-25T04:24:26.486 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.486 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.486 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:26.487 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095153+0000", 2023-11-25T04:24:26.487 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4799000000000003e-05 2023-11-25T04:24:26.487 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.487 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.487 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:26.487 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095268+0000", 2023-11-25T04:24:26.487 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000115437 2023-11-25T04:24:26.487 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.487 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.487 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:26.487 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095363+0000", 2023-11-25T04:24:26.487 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.5278000000000005e-05 2023-11-25T04:24:26.487 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.488 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.488 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:26.488 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095845+0000", 2023-11-25T04:24:26.488 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00048164600000000002 2023-11-25T04:24:26.488 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.488 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.488 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:26.488 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108854+0000", 2023-11-25T04:24:26.488 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013009289 2023-11-25T04:24:26.488 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.488 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.488 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:26.488 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108907+0000", 2023-11-25T04:24:26.489 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2528999999999998e-05 2023-11-25T04:24:26.489 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.489 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.489 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:26.489 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108945+0000", 2023-11-25T04:24:26.489 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8655000000000002e-05 2023-11-25T04:24:26.489 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:26.489 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:26.489 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:26.489 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.489 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.489 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2884 4.37 4:ef909cdc:::10000001135.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:26.490 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.095518+0000", 2023-11-25T04:24:26.490 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 1.2876391279999999, 2023-11-25T04:24:26.490 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014149988, 2023-11-25T04:24:26.490 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:26.490 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:26.490 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:26.490 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.490 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:26.490 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095518+0000", 2023-11-25T04:24:26.490 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:26.490 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.490 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.491 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:26.491 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095518+0000", 2023-11-25T04:24:26.491 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:26.491 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.491 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.491 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:26.491 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095518+0000", 2023-11-25T04:24:26.491 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2900000000000003e-07 2023-11-25T04:24:26.491 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.491 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.491 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:26.491 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095522+0000", 2023-11-25T04:24:26.491 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2119999999999999e-06 2023-11-25T04:24:26.492 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.492 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.492 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:26.492 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095532+0000", 2023-11-25T04:24:26.492 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0375e-05 2023-11-25T04:24:26.492 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.492 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.492 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:26.492 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095537+0000", 2023-11-25T04:24:26.492 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5909999999999998e-06 2023-11-25T04:24:26.492 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.492 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.493 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:26.493 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095584+0000", 2023-11-25T04:24:26.493 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7169999999999997e-05 2023-11-25T04:24:26.493 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.493 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.493 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:26.493 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095695+0000", 2023-11-25T04:24:26.493 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011078699999999999 2023-11-25T04:24:26.493 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.493 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.493 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:26.493 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095777+0000", 2023-11-25T04:24:26.493 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2089999999999995e-05 2023-11-25T04:24:26.494 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.494 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.494 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:26.494 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096286+0000", 2023-11-25T04:24:26.494 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000509247 2023-11-25T04:24:26.494 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.494 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.494 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:26.494 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109596+0000", 2023-11-25T04:24:26.494 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013309872 2023-11-25T04:24:26.494 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.494 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.494 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:26.494 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109638+0000", 2023-11-25T04:24:26.495 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2381e-05 2023-11-25T04:24:26.495 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.495 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.495 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:26.495 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109668+0000", 2023-11-25T04:24:26.495 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9634000000000001e-05 2023-11-25T04:24:26.495 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:26.495 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:26.495 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:26.495 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.495 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.495 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2890 4.12 4:4a2ad640:::10000001141.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:26.496 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.096274+0000", 2023-11-25T04:24:26.496 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 1.286883065, 2023-11-25T04:24:26.496 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013647259, 2023-11-25T04:24:26.496 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:26.496 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:26.496 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:26.496 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.496 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:26.496 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096274+0000", 2023-11-25T04:24:26.496 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:26.496 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.496 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.496 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:26.497 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096274+0000", 2023-11-25T04:24:26.497 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:26.497 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.497 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.497 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:26.497 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096275+0000", 2023-11-25T04:24:26.497 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0299999999999998e-07 2023-11-25T04:24:26.497 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.497 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.497 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:26.497 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096279+0000", 2023-11-25T04:24:26.497 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9580000000000001e-06 2023-11-25T04:24:26.497 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.497 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.498 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:26.498 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096287+0000", 2023-11-25T04:24:26.498 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6929999999999996e-06 2023-11-25T04:24:26.498 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.498 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.498 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:26.498 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096294+0000", 2023-11-25T04:24:26.498 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.348e-06 2023-11-25T04:24:26.498 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.498 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.498 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:26.498 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096342+0000", 2023-11-25T04:24:26.498 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8068000000000001e-05 2023-11-25T04:24:26.499 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.499 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.499 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:26.499 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096462+0000", 2023-11-25T04:24:26.499 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000119918 2023-11-25T04:24:26.499 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.499 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.499 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:26.499 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096557+0000", 2023-11-25T04:24:26.499 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.5340999999999994e-05 2023-11-25T04:24:26.499 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.499 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.499 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:26.500 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096971+0000", 2023-11-25T04:24:26.500 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00041398100000000001 2023-11-25T04:24:26.500 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.500 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.500 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:26.500 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109849+0000", 2023-11-25T04:24:26.500 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.012877985999999999 2023-11-25T04:24:26.500 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.500 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.500 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:26.500 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109894+0000", 2023-11-25T04:24:26.500 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5595e-05 2023-11-25T04:24:26.500 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.500 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.501 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:26.501 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109921+0000", 2023-11-25T04:24:26.501 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6568e-05 2023-11-25T04:24:26.501 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:26.501 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:26.501 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:26.501 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.501 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.501 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2891 4.23 4:c4054dd3:::10000001143.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:26.501 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.096713+0000", 2023-11-25T04:24:26.501 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 1.286444371, 2023-11-25T04:24:26.501 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013148692, 2023-11-25T04:24:26.501 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:26.502 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:26.502 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:26.502 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.502 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:26.502 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096713+0000", 2023-11-25T04:24:26.502 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:26.502 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.502 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.502 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:26.502 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096713+0000", 2023-11-25T04:24:26.502 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:26.503 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.503 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.503 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:26.503 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096713+0000", 2023-11-25T04:24:26.503 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9299999999999998e-07 2023-11-25T04:24:26.503 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.503 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.503 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:26.503 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096716+0000", 2023-11-25T04:24:26.503 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.988e-06 2023-11-25T04:24:26.503 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.503 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.503 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:26.504 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096722+0000", 2023-11-25T04:24:26.504 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0499999999999997e-06 2023-11-25T04:24:26.504 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.504 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.504 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:26.504 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096727+0000", 2023-11-25T04:24:26.504 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3380000000000004e-06 2023-11-25T04:24:26.504 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.504 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.504 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:26.504 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096770+0000", 2023-11-25T04:24:26.504 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2938000000000001e-05 2023-11-25T04:24:26.504 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.505 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.505 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:26.505 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096904+0000", 2023-11-25T04:24:26.505 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000133759 2023-11-25T04:24:26.505 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.505 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.505 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:26.505 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096983+0000", 2023-11-25T04:24:26.505 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8559999999999993e-05 2023-11-25T04:24:26.505 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.505 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.505 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:26.505 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.097437+0000", 2023-11-25T04:24:26.505 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00045439200000000002 2023-11-25T04:24:26.506 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.506 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.506 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:26.506 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109737+0000", 2023-11-25T04:24:26.506 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.012300190000000001 2023-11-25T04:24:26.506 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.506 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.506 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:26.506 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109782+0000", 2023-11-25T04:24:26.506 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4546999999999999e-05 2023-11-25T04:24:26.506 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:26.506 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:26.506 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:26.507 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109861+0000", 2023-11-25T04:24:26.507 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9337000000000003e-05 2023-11-25T04:24:26.507 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:26.507 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:26.507 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:26.507 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:26.507 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:26.507 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:26.508 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:26.507+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:26.509 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:24:26.540 INFO:tasks.workunit.client.0.smithi067.stdout:checking the name lister (/usr/bin/nm -B) interface... BSD nm 2023-11-25T04:24:26.540 INFO:tasks.workunit.client.0.smithi067.stdout:checking whether ln -s works... yes 2023-11-25T04:24:26.556 INFO:tasks.workunit.client.0.smithi067.stdout:checking the maximum length of command line arguments... 1572864 2023-11-25T04:24:26.559 INFO:tasks.workunit.client.0.smithi067.stdout:checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop 2023-11-25T04:24:26.560 INFO:tasks.workunit.client.0.smithi067.stdout:checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop 2023-11-25T04:24:26.560 INFO:tasks.workunit.client.0.smithi067.stdout:checking for /usr/bin/ld option to reload object files... -r 2023-11-25T04:24:26.561 INFO:tasks.workunit.client.0.smithi067.stdout:checking for objdump... objdump 2023-11-25T04:24:26.562 INFO:tasks.workunit.client.0.smithi067.stdout:checking how to recognize dependent libraries... pass_all 2023-11-25T04:24:26.562 INFO:tasks.workunit.client.0.smithi067.stdout:checking for dlltool... no 2023-11-25T04:24:26.563 INFO:tasks.workunit.client.0.smithi067.stdout:checking how to associate runtime and link libraries... printf %s\n 2023-11-25T04:24:26.564 INFO:tasks.workunit.client.0.smithi067.stdout:checking for ar... ar 2023-11-25T04:24:26.609 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:26.608+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:26.616 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:26.617 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:24:26.617 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2023-11-25T04:24:26.617 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:26.626 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:24:26.634 INFO:tasks.workunit.client.0.smithi067.stdout:checking for archiver @FILE support... @ 2023-11-25T04:24:26.634 INFO:tasks.workunit.client.0.smithi067.stdout:checking for strip... strip 2023-11-25T04:24:26.635 INFO:tasks.workunit.client.0.smithi067.stdout:checking for ranlib... ranlib 2023-11-25T04:24:26.636 INFO:tasks.workunit.client.0.smithi067.stdout:checking for gawk... gawk 2023-11-25T04:24:26.710 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:26.709+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:26.767 INFO:tasks.workunit.client.0.smithi067.stdout:checking command to parse /usr/bin/nm -B output from gcc object... ok 2023-11-25T04:24:26.768 INFO:tasks.workunit.client.0.smithi067.stdout:checking for sysroot... no 2023-11-25T04:24:26.775 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:26.775 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:24:26.775 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2023-11-25T04:24:26.775 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2023-11-25T04:24:26.775 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:26.777 INFO:tasks.workunit.client.0.smithi067.stdout:checking for a working dd... /usr/bin/dd 2023-11-25T04:24:26.784 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:24:26.784 INFO:tasks.workunit.client.0.smithi067.stdout:checking how to truncate binary pipes... /usr/bin/dd bs=4096 count=1 2023-11-25T04:24:26.806 INFO:tasks.workunit.client.0.smithi067.stdout:checking for mt... no 2023-11-25T04:24:26.813 INFO:tasks.workunit.client.0.smithi067.stdout:checking if : is a manifest tool... no 2023-11-25T04:24:26.825 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:26.824+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:26.889 INFO:tasks.workunit.client.0.smithi067.stdout:checking how to run the C preprocessor... gcc -E 2023-11-25T04:24:26.911 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:26.910+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:26.932 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:26.932 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2023-11-25T04:24:26.933 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2023-11-25T04:24:26.933 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2023-11-25T04:24:26.933 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.933 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:152 3.1a 3:5e41ab69:::200.00000005:head [write 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:26.933 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:21.237401+0000", 2023-11-25T04:24:26.933 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.6924819040000001, 2023-11-25T04:24:26.933 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015253802, 2023-11-25T04:24:26.933 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:26.933 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:26.933 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:26.934 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.934 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:26.934 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.237401+0000", 2023-11-25T04:24:26.934 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:26.934 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.934 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.934 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:26.934 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.237401+0000", 2023-11-25T04:24:26.934 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:26.934 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.935 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.935 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:26.935 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.237402+0000", 2023-11-25T04:24:26.935 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3699999999999999e-07 2023-11-25T04:24:26.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.935 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.935 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:26.935 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243182+0000", 2023-11-25T04:24:26.935 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057801270000000004 2023-11-25T04:24:26.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.935 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.936 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:26.936 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243193+0000", 2023-11-25T04:24:26.936 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1221e-05 2023-11-25T04:24:26.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.936 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:26.936 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243198+0000", 2023-11-25T04:24:26.936 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3739999999999997e-06 2023-11-25T04:24:26.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.937 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:26.937 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243246+0000", 2023-11-25T04:24:26.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7667999999999998e-05 2023-11-25T04:24:26.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.937 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.937 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:26.937 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243292+0000", 2023-11-25T04:24:26.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5794000000000002e-05 2023-11-25T04:24:26.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.938 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.938 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:26.938 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243360+0000", 2023-11-25T04:24:26.938 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8479000000000006e-05 2023-11-25T04:24:26.938 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.938 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.938 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:26.938 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.250667+0000", 2023-11-25T04:24:26.938 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.007306618 2023-11-25T04:24:26.938 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.939 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:26.939 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.252598+0000", 2023-11-25T04:24:26.939 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019315129999999999 2023-11-25T04:24:26.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.939 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:26.939 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.252631+0000", 2023-11-25T04:24:26.939 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2925000000000001e-05 2023-11-25T04:24:26.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.940 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.940 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:26.940 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.252655+0000", 2023-11-25T04:24:26.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3246000000000001e-05 2023-11-25T04:24:26.940 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:26.940 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:26.940 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:26.940 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.940 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.940 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:2841 4.4 e47/17 4:22111d42:::100000010df.00000000:head v 47'139, mlcod=47'137)", 2023-11-25T04:24:26.940 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.093610+0000", 2023-11-25T04:24:26.941 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.8362730410000001, 2023-11-25T04:24:26.941 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014758871999999999, 2023-11-25T04:24:26.941 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:26.941 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:26.941 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:26.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.941 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:26.941 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093610+0000", 2023-11-25T04:24:26.941 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:26.941 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.941 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.942 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:26.942 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093610+0000", 2023-11-25T04:24:26.942 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:26.942 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.942 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.942 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:26.942 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093610+0000", 2023-11-25T04:24:26.942 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1799999999999999e-07 2023-11-25T04:24:26.942 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.942 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.942 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:26.943 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093621+0000", 2023-11-25T04:24:26.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1107999999999999e-05 2023-11-25T04:24:26.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.943 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:26.943 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093627+0000", 2023-11-25T04:24:26.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0310000000000004e-06 2023-11-25T04:24:26.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:26.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093633+0000", 2023-11-25T04:24:26.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2700000000000001e-06 2023-11-25T04:24:26.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:26.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093722+0000", 2023-11-25T04:24:26.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8417000000000004e-05 2023-11-25T04:24:26.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:26.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093737+0000", 2023-11-25T04:24:26.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5781e-05 2023-11-25T04:24:26.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.945 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:26.945 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108203+0000", 2023-11-25T04:24:26.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014465844 2023-11-25T04:24:26.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.946 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:26.946 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108368+0000", 2023-11-25T04:24:26.946 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000165203 2023-11-25T04:24:26.946 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:26.946 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:26.947 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:26.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.948 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2860 4.3d 4:bf0d7877:::10000001105.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:26.948 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.093823+0000", 2023-11-25T04:24:26.948 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.836059873, 2023-11-25T04:24:26.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015582848, 2023-11-25T04:24:26.948 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:26.948 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:26.948 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:26.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.948 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:26.948 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093823+0000", 2023-11-25T04:24:26.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:26.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:26.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093823+0000", 2023-11-25T04:24:26.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:26.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:26.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093823+0000", 2023-11-25T04:24:26.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9100000000000004e-07 2023-11-25T04:24:26.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:26.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093826+0000", 2023-11-25T04:24:26.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4499999999999998e-06 2023-11-25T04:24:26.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:26.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093835+0000", 2023-11-25T04:24:26.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8179999999999993e-06 2023-11-25T04:24:26.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:26.951 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093840+0000", 2023-11-25T04:24:26.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0819999999999998e-06 2023-11-25T04:24:26.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.951 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:26.951 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093973+0000", 2023-11-25T04:24:26.951 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013308999999999999 2023-11-25T04:24:26.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.952 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:26.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094277+0000", 2023-11-25T04:24:26.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00030403299999999999 2023-11-25T04:24:26.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.952 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:26.953 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108608+0000", 2023-11-25T04:24:26.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014330888 2023-11-25T04:24:26.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:26.953 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108676+0000", 2023-11-25T04:24:26.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8577000000000004e-05 2023-11-25T04:24:26.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:26.954 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109406+0000", 2023-11-25T04:24:26.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00072931899999999997 2023-11-25T04:24:26.954 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:26.954 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:26.954 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:26.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.954 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2862 4.13 4:cb2e645e:::10000001109.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:26.954 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.093852+0000", 2023-11-25T04:24:26.954 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.836030912, 2023-11-25T04:24:26.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015190636, 2023-11-25T04:24:26.955 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:26.955 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:26.955 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:26.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:26.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093852+0000", 2023-11-25T04:24:26.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:26.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.956 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:26.956 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093852+0000", 2023-11-25T04:24:26.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:26.956 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.956 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.956 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:26.956 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093852+0000", 2023-11-25T04:24:26.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0299999999999999e-07 2023-11-25T04:24:26.956 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.956 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.957 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:26.957 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093854+0000", 2023-11-25T04:24:26.957 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.846e-06 2023-11-25T04:24:26.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.957 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:26.957 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093861+0000", 2023-11-25T04:24:26.957 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5520000000000001e-06 2023-11-25T04:24:26.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.957 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:26.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093865+0000", 2023-11-25T04:24:26.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7870000000000001e-06 2023-11-25T04:24:26.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:26.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093940+0000", 2023-11-25T04:24:26.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4799e-05 2023-11-25T04:24:26.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:26.959 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094052+0000", 2023-11-25T04:24:26.959 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011156500000000001 2023-11-25T04:24:26.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.959 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:26.959 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108928+0000", 2023-11-25T04:24:26.959 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014876245 2023-11-25T04:24:26.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.959 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:26.959 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108972+0000", 2023-11-25T04:24:26.960 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4477999999999997e-05 2023-11-25T04:24:26.960 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.960 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.960 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:26.960 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109042+0000", 2023-11-25T04:24:26.960 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9861000000000001e-05 2023-11-25T04:24:26.960 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:26.960 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:26.960 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:26.960 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.960 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.961 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2865 4.36 4:6ebc5ac6:::1000000110f.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:26.961 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.093906+0000", 2023-11-25T04:24:26.961 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.8359766390000001, 2023-11-25T04:24:26.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014962628, 2023-11-25T04:24:26.961 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:26.961 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:26.961 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:26.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.961 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:26.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093906+0000", 2023-11-25T04:24:26.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:26.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:26.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093906+0000", 2023-11-25T04:24:26.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:26.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:26.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093906+0000", 2023-11-25T04:24:26.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4400000000000001e-07 2023-11-25T04:24:26.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:26.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093908+0000", 2023-11-25T04:24:26.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8640000000000001e-06 2023-11-25T04:24:26.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:26.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093914+0000", 2023-11-25T04:24:26.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0170000000000002e-06 2023-11-25T04:24:26.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.964 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:26.964 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093921+0000", 2023-11-25T04:24:26.964 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6529999999999997e-06 2023-11-25T04:24:26.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.964 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.964 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:26.964 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094258+0000", 2023-11-25T04:24:26.964 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00033703599999999998 2023-11-25T04:24:26.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.964 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.965 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:26.965 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094340+0000", 2023-11-25T04:24:26.965 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2279000000000003e-05 2023-11-25T04:24:26.965 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.965 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.965 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:26.965 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094416+0000", 2023-11-25T04:24:26.965 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6248999999999995e-05 2023-11-25T04:24:26.965 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.965 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.965 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:26.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.096134+0000", 2023-11-25T04:24:26.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001717611 2023-11-25T04:24:26.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:26.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108809+0000", 2023-11-25T04:24:26.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012675327 2023-11-25T04:24:26.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:26.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108836+0000", 2023-11-25T04:24:26.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6509999999999999e-05 2023-11-25T04:24:26.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.967 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:26.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108869+0000", 2023-11-25T04:24:26.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2737999999999997e-05 2023-11-25T04:24:26.967 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:26.968 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:26.968 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:26.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.968 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:2852 4.2d e47/17 4:b745ac76:::100000010f5.00000000:head v 47'126, mlcod=47'124)", 2023-11-25T04:24:26.968 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.093987+0000", 2023-11-25T04:24:26.968 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.835896078, 2023-11-25T04:24:26.968 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014613256, 2023-11-25T04:24:26.968 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:26.968 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:26.968 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:26.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.969 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:26.969 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093987+0000", 2023-11-25T04:24:26.969 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:26.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.969 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:26.969 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093987+0000", 2023-11-25T04:24:26.969 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:26.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.970 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:26.970 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093987+0000", 2023-11-25T04:24:26.970 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7000000000000001e-07 2023-11-25T04:24:26.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.970 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:26.970 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093990+0000", 2023-11-25T04:24:26.970 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8729999999999999e-06 2023-11-25T04:24:26.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.971 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:26.971 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093997+0000", 2023-11-25T04:24:26.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.973e-06 2023-11-25T04:24:26.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.971 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:26.971 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094001+0000", 2023-11-25T04:24:26.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7360000000000001e-06 2023-11-25T04:24:26.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:26.972 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094263+0000", 2023-11-25T04:24:26.972 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026173400000000001 2023-11-25T04:24:26.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:26.972 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094278+0000", 2023-11-25T04:24:26.972 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4681999999999999e-05 2023-11-25T04:24:26.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.973 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.973 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:26.973 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108563+0000", 2023-11-25T04:24:26.973 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014284738 2023-11-25T04:24:26.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.973 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.973 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:26.973 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108600+0000", 2023-11-25T04:24:26.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7249999999999997e-05 2023-11-25T04:24:26.974 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:26.974 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:26.974 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:26.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.974 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2866 4.36 4:6f6907fb:::10000001111.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:26.974 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.094033+0000", 2023-11-25T04:24:26.974 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.8358499130000001, 2023-11-25T04:24:26.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014885591, 2023-11-25T04:24:26.975 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:26.975 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:26.975 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:26.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:26.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094033+0000", 2023-11-25T04:24:26.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:26.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.976 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:26.976 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094033+0000", 2023-11-25T04:24:26.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:26.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.976 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:26.976 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094033+0000", 2023-11-25T04:24:26.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.06e-07 2023-11-25T04:24:26.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.977 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:26.977 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094036+0000", 2023-11-25T04:24:26.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.508e-06 2023-11-25T04:24:26.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.977 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:26.977 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094044+0000", 2023-11-25T04:24:26.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5909999999999996e-06 2023-11-25T04:24:26.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.978 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:26.978 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094051+0000", 2023-11-25T04:24:26.978 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0319999999999998e-06 2023-11-25T04:24:26.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.978 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.978 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:26.978 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094727+0000", 2023-11-25T04:24:26.978 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00067523300000000002 2023-11-25T04:24:26.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:26.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094795+0000", 2023-11-25T04:24:26.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8156000000000003e-05 2023-11-25T04:24:26.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:26.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094898+0000", 2023-11-25T04:24:26.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103329 2023-11-25T04:24:26.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:26.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108803+0000", 2023-11-25T04:24:26.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013904385 2023-11-25T04:24:26.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:26.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108875+0000", 2023-11-25T04:24:26.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1958999999999993e-05 2023-11-25T04:24:26.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:26.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108899+0000", 2023-11-25T04:24:26.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4019000000000002e-05 2023-11-25T04:24:26.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:26.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108918+0000", 2023-11-25T04:24:26.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9772999999999999e-05 2023-11-25T04:24:26.981 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:26.982 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:26.982 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:26.982 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.982 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.982 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:2854 4.2d e47/17 4:b5c259cf:::100000010f9.00000000:head v 47'127, mlcod=47'124)", 2023-11-25T04:24:26.983 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.094793+0000", 2023-11-25T04:24:26.983 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.8350896240000001, 2023-11-25T04:24:26.983 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014650764, 2023-11-25T04:24:26.983 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:26.983 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:26.983 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:26.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.984 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:26.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094793+0000", 2023-11-25T04:24:26.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:26.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.984 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:26.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094793+0000", 2023-11-25T04:24:26.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:26.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:26.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094793+0000", 2023-11-25T04:24:26.985 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3700000000000001e-07 2023-11-25T04:24:26.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:26.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094798+0000", 2023-11-25T04:24:26.986 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1200000000000004e-06 2023-11-25T04:24:26.986 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.986 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.986 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:26.986 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094805+0000", 2023-11-25T04:24:26.986 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9759999999999998e-06 2023-11-25T04:24:26.986 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.986 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.986 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:26.986 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094811+0000", 2023-11-25T04:24:26.986 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1410000000000002e-06 2023-11-25T04:24:26.987 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.987 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.987 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:26.987 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094838+0000", 2023-11-25T04:24:26.987 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6995000000000002e-05 2023-11-25T04:24:26.987 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.987 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.987 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:26.987 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094848+0000", 2023-11-25T04:24:26.987 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0115999999999999e-05 2023-11-25T04:24:26.987 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.988 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:26.988 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109408+0000", 2023-11-25T04:24:26.988 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014559825 2023-11-25T04:24:26.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.988 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.988 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:26.988 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109444+0000", 2023-11-25T04:24:26.988 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6154e-05 2023-11-25T04:24:26.988 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:26.988 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:26.988 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:26.989 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.989 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.989 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2869 4.6 4:63719fb0:::10000001117.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:26.989 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.094910+0000", 2023-11-25T04:24:26.989 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.8349726669999999, 2023-11-25T04:24:26.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014132570000000001, 2023-11-25T04:24:26.989 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:26.989 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:26.989 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:26.989 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.990 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:26.990 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094910+0000", 2023-11-25T04:24:26.990 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:26.990 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.990 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.990 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:26.990 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094910+0000", 2023-11-25T04:24:26.990 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:26.990 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.991 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.991 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:26.991 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094910+0000", 2023-11-25T04:24:26.991 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.58e-07 2023-11-25T04:24:26.991 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.991 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.991 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:26.991 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094913+0000", 2023-11-25T04:24:26.991 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.198e-06 2023-11-25T04:24:26.991 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.991 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.992 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:26.992 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094921+0000", 2023-11-25T04:24:26.992 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1629999999999996e-06 2023-11-25T04:24:26.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.992 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:26.992 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094926+0000", 2023-11-25T04:24:26.992 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7559999999999999e-06 2023-11-25T04:24:26.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.993 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:26.993 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094966+0000", 2023-11-25T04:24:26.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0151999999999998e-05 2023-11-25T04:24:26.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.993 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:26.993 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095073+0000", 2023-11-25T04:24:26.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107657 2023-11-25T04:24:26.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.993 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:26.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095158+0000", 2023-11-25T04:24:26.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5136999999999995e-05 2023-11-25T04:24:26.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:26.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108924+0000", 2023-11-25T04:24:26.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013765727 2023-11-25T04:24:26.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:26.995 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108958+0000", 2023-11-25T04:24:26.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3417000000000002e-05 2023-11-25T04:24:26.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.995 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:26.996 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109012+0000", 2023-11-25T04:24:26.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3939e-05 2023-11-25T04:24:26.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.996 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:26.996 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109043+0000", 2023-11-25T04:24:26.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0966000000000001e-05 2023-11-25T04:24:26.996 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:26.997 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:26.997 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:26.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.997 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2871 4.3f 4:fff10e11:::1000000111b.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:26.997 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.094936+0000", 2023-11-25T04:24:26.997 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.8349468289999999, 2023-11-25T04:24:26.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014261726000000001, 2023-11-25T04:24:26.997 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:26.997 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:26.998 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:26.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:26.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094936+0000", 2023-11-25T04:24:26.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:26.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:26.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094936+0000", 2023-11-25T04:24:26.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:26.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:26.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094936+0000", 2023-11-25T04:24:26.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9900000000000001e-07 2023-11-25T04:24:26.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:26.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:26.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:26.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094938+0000", 2023-11-25T04:24:26.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9599999999999999e-06 2023-11-25T04:24:27.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:27.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094944+0000", 2023-11-25T04:24:27.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2219999999999999e-06 2023-11-25T04:24:27.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:27.001 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094950+0000", 2023-11-25T04:24:27.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1329999999999998e-06 2023-11-25T04:24:27.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.001 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:27.001 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094993+0000", 2023-11-25T04:24:27.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3305e-05 2023-11-25T04:24:27.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.001 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:27.002 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095107+0000", 2023-11-25T04:24:27.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011449400000000001 2023-11-25T04:24:27.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:27.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095176+0000", 2023-11-25T04:24:27.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8330000000000005e-05 2023-11-25T04:24:27.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:27.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107939+0000", 2023-11-25T04:24:27.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012763801 2023-11-25T04:24:27.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:27.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109128+0000", 2023-11-25T04:24:27.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011884770000000001 2023-11-25T04:24:27.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:27.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109166+0000", 2023-11-25T04:24:27.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7948999999999999e-05 2023-11-25T04:24:27.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:27.006 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109198+0000", 2023-11-25T04:24:27.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1656000000000003e-05 2023-11-25T04:24:27.006 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:27.006 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:27.006 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:27.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.007 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2872 4.32 4:4fb002bc:::1000000111d.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:27.007 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.094961+0000", 2023-11-25T04:24:27.007 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.8349221249999998, 2023-11-25T04:24:27.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014631564999999999, 2023-11-25T04:24:27.007 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:27.007 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:27.007 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:27.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:27.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094961+0000", 2023-11-25T04:24:27.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:27.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:27.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094961+0000", 2023-11-25T04:24:27.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:27.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:27.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094961+0000", 2023-11-25T04:24:27.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9299999999999999e-07 2023-11-25T04:24:27.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:27.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094963+0000", 2023-11-25T04:24:27.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9209999999999999e-06 2023-11-25T04:24:27.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:27.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094968+0000", 2023-11-25T04:24:27.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.502e-06 2023-11-25T04:24:27.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:27.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094973+0000", 2023-11-25T04:24:27.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8559999999999998e-06 2023-11-25T04:24:27.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:27.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107722+0000", 2023-11-25T04:24:27.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012748534000000001 2023-11-25T04:24:27.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:27.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107844+0000", 2023-11-25T04:24:27.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012233400000000001 2023-11-25T04:24:27.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:27.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109469+0000", 2023-11-25T04:24:27.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001625175 2023-11-25T04:24:27.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:27.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109518+0000", 2023-11-25T04:24:27.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8980000000000002e-05 2023-11-25T04:24:27.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:27.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109592+0000", 2023-11-25T04:24:27.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3869999999999996e-05 2023-11-25T04:24:27.013 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:27.013 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:27.013 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:27.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.013 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2873 4.13 4:cae4f303:::1000000111f.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:27.015 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.094991+0000", 2023-11-25T04:24:27.015 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.834891533, 2023-11-25T04:24:27.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015101692, 2023-11-25T04:24:27.015 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:27.015 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:27.015 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:27.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:27.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094991+0000", 2023-11-25T04:24:27.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:27.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.016 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:27.016 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094991+0000", 2023-11-25T04:24:27.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:27.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.016 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:27.016 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094992+0000", 2023-11-25T04:24:27.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7e-07 2023-11-25T04:24:27.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.016 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:27.016 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094994+0000", 2023-11-25T04:24:27.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2110000000000001e-06 2023-11-25T04:24:27.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:27.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095000+0000", 2023-11-25T04:24:27.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.241e-06 2023-11-25T04:24:27.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:27.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095006+0000", 2023-11-25T04:24:27.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3659999999999997e-06 2023-11-25T04:24:27.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:27.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108369+0000", 2023-11-25T04:24:27.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01336247 2023-11-25T04:24:27.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:27.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108492+0000", 2023-11-25T04:24:27.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000123192 2023-11-25T04:24:27.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:27.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109975+0000", 2023-11-25T04:24:27.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001482894 2023-11-25T04:24:27.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:27.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110024+0000", 2023-11-25T04:24:27.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9097000000000001e-05 2023-11-25T04:24:27.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.020 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:27.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110093+0000", 2023-11-25T04:24:27.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8751000000000006e-05 2023-11-25T04:24:27.020 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:27.020 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:27.020 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:27.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.020 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2874 4.14 4:2b557d2e:::10000001121.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:27.020 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095019+0000", 2023-11-25T04:24:27.020 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.834864007, 2023-11-25T04:24:27.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014998296, 2023-11-25T04:24:27.021 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:27.021 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:27.021 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:27.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.021 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:27.021 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095019+0000", 2023-11-25T04:24:27.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:27.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.021 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:27.021 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095019+0000", 2023-11-25T04:24:27.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:27.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:27.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095019+0000", 2023-11-25T04:24:27.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7300000000000001e-07 2023-11-25T04:24:27.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:27.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095021+0000", 2023-11-25T04:24:27.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0099999999999998e-06 2023-11-25T04:24:27.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:27.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095028+0000", 2023-11-25T04:24:27.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7870000000000004e-06 2023-11-25T04:24:27.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:27.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095033+0000", 2023-11-25T04:24:27.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2039999999999996e-06 2023-11-25T04:24:27.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:27.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107740+0000", 2023-11-25T04:24:27.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012706693 2023-11-25T04:24:27.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:27.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107817+0000", 2023-11-25T04:24:27.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7599e-05 2023-11-25T04:24:27.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:27.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107961+0000", 2023-11-25T04:24:27.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000143194 2023-11-25T04:24:27.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:27.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109364+0000", 2023-11-25T04:24:27.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001403226 2023-11-25T04:24:27.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:27.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109507+0000", 2023-11-25T04:24:27.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000142991 2023-11-25T04:24:27.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:27.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109986+0000", 2023-11-25T04:24:27.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00047906899999999999 2023-11-25T04:24:27.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:27.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110017+0000", 2023-11-25T04:24:27.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1050000000000003e-05 2023-11-25T04:24:27.026 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:27.026 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:27.026 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:27.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.026 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2875 4.d 4:b17d5dc6:::10000001123.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:27.026 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095049+0000", 2023-11-25T04:24:27.026 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.8348340830000001, 2023-11-25T04:24:27.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014882886, 2023-11-25T04:24:27.026 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:27.027 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:27.027 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:27.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:27.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095049+0000", 2023-11-25T04:24:27.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:27.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:27.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095049+0000", 2023-11-25T04:24:27.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:27.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:27.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095049+0000", 2023-11-25T04:24:27.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6100000000000001e-07 2023-11-25T04:24:27.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:27.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095051+0000", 2023-11-25T04:24:27.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1449999999999998e-06 2023-11-25T04:24:27.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:27.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095058+0000", 2023-11-25T04:24:27.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.601e-06 2023-11-25T04:24:27.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:27.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095064+0000", 2023-11-25T04:24:27.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1079999999999998e-06 2023-11-25T04:24:27.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:27.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108015+0000", 2023-11-25T04:24:27.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012951582 2023-11-25T04:24:27.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:27.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108260+0000", 2023-11-25T04:24:27.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024445199999999998 2023-11-25T04:24:27.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:27.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109825+0000", 2023-11-25T04:24:27.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0015651759999999999 2023-11-25T04:24:27.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:27.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109866+0000", 2023-11-25T04:24:27.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0629e-05 2023-11-25T04:24:27.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:27.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109931+0000", 2023-11-25T04:24:27.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5732e-05 2023-11-25T04:24:27.031 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:27.031 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:27.031 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:27.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.031 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2876 4.3b 4:dc871776:::10000001125.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:27.031 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095077+0000", 2023-11-25T04:24:27.031 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.8348054620000001, 2023-11-25T04:24:27.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014500944, 2023-11-25T04:24:27.032 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:27.032 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:27.032 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:27.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:27.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095077+0000", 2023-11-25T04:24:27.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:27.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:27.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095077+0000", 2023-11-25T04:24:27.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:27.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.033 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:27.033 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095078+0000", 2023-11-25T04:24:27.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9799999999999999e-07 2023-11-25T04:24:27.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.033 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:27.033 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095079+0000", 2023-11-25T04:24:27.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.877e-06 2023-11-25T04:24:27.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.033 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:27.033 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095086+0000", 2023-11-25T04:24:27.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2439999999999998e-06 2023-11-25T04:24:27.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:27.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095094+0000", 2023-11-25T04:24:27.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9710000000000004e-06 2023-11-25T04:24:27.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:27.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107732+0000", 2023-11-25T04:24:27.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012638104000000001 2023-11-25T04:24:27.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:27.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107854+0000", 2023-11-25T04:24:27.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012261600000000001 2023-11-25T04:24:27.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:27.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107941+0000", 2023-11-25T04:24:27.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6935999999999996e-05 2023-11-25T04:24:27.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:27.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108871+0000", 2023-11-25T04:24:27.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00092955499999999999 2023-11-25T04:24:27.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:27.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109483+0000", 2023-11-25T04:24:27.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00061206100000000001 2023-11-25T04:24:27.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:27.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109527+0000", 2023-11-25T04:24:27.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4112e-05 2023-11-25T04:24:27.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.037 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:27.037 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109578+0000", 2023-11-25T04:24:27.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1069999999999997e-05 2023-11-25T04:24:27.037 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:27.037 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:27.037 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:27.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.038 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2878 4.3e 4:7ff98b9d:::10000001129.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:27.038 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095101+0000", 2023-11-25T04:24:27.038 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.834781679, 2023-11-25T04:24:27.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015015137, 2023-11-25T04:24:27.038 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:27.038 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:27.038 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:27.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:27.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095101+0000", 2023-11-25T04:24:27.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:27.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:27.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095101+0000", 2023-11-25T04:24:27.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:27.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:27.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095101+0000", 2023-11-25T04:24:27.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7399999999999999e-07 2023-11-25T04:24:27.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:27.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095103+0000", 2023-11-25T04:24:27.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7600000000000001e-06 2023-11-25T04:24:27.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:27.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095109+0000", 2023-11-25T04:24:27.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8189999999999997e-06 2023-11-25T04:24:27.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:27.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095114+0000", 2023-11-25T04:24:27.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6770000000000003e-06 2023-11-25T04:24:27.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:27.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107724+0000", 2023-11-25T04:24:27.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012610909999999999 2023-11-25T04:24:27.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:27.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108483+0000", 2023-11-25T04:24:27.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000758407 2023-11-25T04:24:27.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:27.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108568+0000", 2023-11-25T04:24:27.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4707000000000001e-05 2023-11-25T04:24:27.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:27.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109687+0000", 2023-11-25T04:24:27.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011192750000000001 2023-11-25T04:24:27.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:27.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110049+0000", 2023-11-25T04:24:27.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00036210600000000001 2023-11-25T04:24:27.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:27.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110086+0000", 2023-11-25T04:24:27.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7500999999999999e-05 2023-11-25T04:24:27.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:27.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110116+0000", 2023-11-25T04:24:27.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9601e-05 2023-11-25T04:24:27.043 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:27.043 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:27.043 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:27.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.044 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2879 4.2a 4:5524ace2:::1000000112b.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:27.044 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095120+0000", 2023-11-25T04:24:27.044 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.834762899, 2023-11-25T04:24:27.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014587937, 2023-11-25T04:24:27.044 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:27.044 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:27.044 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:27.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:27.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095120+0000", 2023-11-25T04:24:27.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:27.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:27.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095120+0000", 2023-11-25T04:24:27.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:27.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:27.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095120+0000", 2023-11-25T04:24:27.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5600000000000001e-07 2023-11-25T04:24:27.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:27.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095122+0000", 2023-11-25T04:24:27.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.694e-06 2023-11-25T04:24:27.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:27.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095127+0000", 2023-11-25T04:24:27.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5099999999999998e-06 2023-11-25T04:24:27.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:27.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095132+0000", 2023-11-25T04:24:27.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6210000000000003e-06 2023-11-25T04:24:27.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:27.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107729+0000", 2023-11-25T04:24:27.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012596931 2023-11-25T04:24:27.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:27.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107856+0000", 2023-11-25T04:24:27.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012707699999999999 2023-11-25T04:24:27.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:27.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109600+0000", 2023-11-25T04:24:27.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001744298 2023-11-25T04:24:27.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:27.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109641+0000", 2023-11-25T04:24:27.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0769000000000002e-05 2023-11-25T04:24:27.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:27.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109708+0000", 2023-11-25T04:24:27.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6680999999999994e-05 2023-11-25T04:24:27.049 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:27.049 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:27.049 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:27.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.049 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2882 4.1c 4:3a7c2e07:::10000001131.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:27.049 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095141+0000", 2023-11-25T04:24:27.050 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.834741524, 2023-11-25T04:24:27.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014949469999999999, 2023-11-25T04:24:27.050 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:27.050 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:27.050 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:27.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:27.050 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095141+0000", 2023-11-25T04:24:27.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:27.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:27.050 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095141+0000", 2023-11-25T04:24:27.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:27.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:27.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095142+0000", 2023-11-25T04:24:27.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1899999999999998e-07 2023-11-25T04:24:27.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:27.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095143+0000", 2023-11-25T04:24:27.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7770000000000001e-06 2023-11-25T04:24:27.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:27.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095149+0000", 2023-11-25T04:24:27.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9909999999999999e-06 2023-11-25T04:24:27.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:27.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095154+0000", 2023-11-25T04:24:27.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3880000000000002e-06 2023-11-25T04:24:27.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:27.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107736+0000", 2023-11-25T04:24:27.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012582477999999999 2023-11-25T04:24:27.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:27.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107868+0000", 2023-11-25T04:24:27.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013134400000000001 2023-11-25T04:24:27.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:27.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107959+0000", 2023-11-25T04:24:27.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1253000000000002e-05 2023-11-25T04:24:27.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:27.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109325+0000", 2023-11-25T04:24:27.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0013662520000000001 2023-11-25T04:24:27.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.054 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:27.054 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110023+0000", 2023-11-25T04:24:27.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00069838799999999998 2023-11-25T04:24:27.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.054 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:27.054 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110061+0000", 2023-11-25T04:24:27.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7867999999999997e-05 2023-11-25T04:24:27.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:27.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110091+0000", 2023-11-25T04:24:27.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9312e-05 2023-11-25T04:24:27.055 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:27.055 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:27.055 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:27.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.055 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2883 4.3e 4:7d82be99:::10000001133.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:27.055 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095942+0000", 2023-11-25T04:24:27.056 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.8339402379999998, 2023-11-25T04:24:27.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014290573000000001, 2023-11-25T04:24:27.056 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:27.056 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:27.056 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:27.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:27.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095942+0000", 2023-11-25T04:24:27.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:27.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:27.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095942+0000", 2023-11-25T04:24:27.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:27.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:27.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095944+0000", 2023-11-25T04:24:27.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.632e-06 2023-11-25T04:24:27.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:27.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095947+0000", 2023-11-25T04:24:27.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7760000000000002e-06 2023-11-25T04:24:27.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:27.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095954+0000", 2023-11-25T04:24:27.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5379999999999997e-06 2023-11-25T04:24:27.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:27.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095960+0000", 2023-11-25T04:24:27.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9209999999999997e-06 2023-11-25T04:24:27.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:27.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108809+0000", 2023-11-25T04:24:27.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012848972 2023-11-25T04:24:27.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:27.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108921+0000", 2023-11-25T04:24:27.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000111767 2023-11-25T04:24:27.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:27.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109008+0000", 2023-11-25T04:24:27.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6576999999999995e-05 2023-11-25T04:24:27.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.060 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:27.060 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110124+0000", 2023-11-25T04:24:27.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011163709999999999 2023-11-25T04:24:27.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.060 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:27.060 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110155+0000", 2023-11-25T04:24:27.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1248999999999998e-05 2023-11-25T04:24:27.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.060 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:27.060 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110202+0000", 2023-11-25T04:24:27.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7097e-05 2023-11-25T04:24:27.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:27.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110233+0000", 2023-11-25T04:24:27.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0673000000000003e-05 2023-11-25T04:24:27.061 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:27.061 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:27.061 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:27.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.061 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2886 4.25 4:a579baeb:::10000001139.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:27.061 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095994+0000", 2023-11-25T04:24:27.062 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 1.83388891, 2023-11-25T04:24:27.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013825356, 2023-11-25T04:24:27.062 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:27.062 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:27.062 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:27.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:27.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095994+0000", 2023-11-25T04:24:27.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:27.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:27.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095994+0000", 2023-11-25T04:24:27.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:27.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:27.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095996+0000", 2023-11-25T04:24:27.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8780000000000001e-06 2023-11-25T04:24:27.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:27.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095998+0000", 2023-11-25T04:24:27.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1560000000000002e-06 2023-11-25T04:24:27.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:27.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.096003+0000", 2023-11-25T04:24:27.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1080000000000001e-06 2023-11-25T04:24:27.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:27.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.096010+0000", 2023-11-25T04:24:27.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0650000000000001e-06 2023-11-25T04:24:27.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:27.064 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107742+0000", 2023-11-25T04:24:27.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011731691000000001 2023-11-25T04:24:27.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:27.065 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108063+0000", 2023-11-25T04:24:27.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00032168600000000001 2023-11-25T04:24:27.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:27.065 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109689+0000", 2023-11-25T04:24:27.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001625556 2023-11-25T04:24:27.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:27.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109740+0000", 2023-11-25T04:24:27.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1125000000000003e-05 2023-11-25T04:24:27.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:27.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:27.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:27.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109819+0000", 2023-11-25T04:24:27.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9091000000000006e-05 2023-11-25T04:24:27.066 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:27.066 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:27.066 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:27.066 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:27.066 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:27.067 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:27.067 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-11-25T04:24:27.068 INFO:tasks.workunit.client.0.smithi067.stdout:checking for ANSI C header files... yes 2023-11-25T04:24:27.069 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:24:27.069 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:27.066+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:27.101 INFO:tasks.workunit.client.0.smithi067.stdout:checking for sys/types.h... yes 2023-11-25T04:24:27.148 INFO:tasks.workunit.client.0.smithi067.stdout:checking for sys/stat.h... yes 2023-11-25T04:24:27.168 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:27.167+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:27.180 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:27.181 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:24:27.181 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2023-11-25T04:24:27.181 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:27.188 INFO:teuthology.orchestra.run.smithi067.stderr:Error ENXIO: problem getting command descriptions from osd.0 2023-11-25T04:24:27.190 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:24:27.190 INFO:tasks.workunit.client.0.smithi067.stdout:checking for stdlib.h... yes 2023-11-25T04:24:27.209 INFO:teuthology.orchestra.run.smithi067.stdout:osd.1: {} 2023-11-25T04:24:27.209 INFO:teuthology.orchestra.run.smithi067.stderr:osd.1: osd_enable_op_tracker = '' 2023-11-25T04:24:27.237 INFO:teuthology.orchestra.run.smithi067.stdout:osd.2: {} 2023-11-25T04:24:27.238 INFO:teuthology.orchestra.run.smithi067.stderr:osd.2: osd_enable_op_tracker = '' 2023-11-25T04:24:27.238 INFO:tasks.workunit.client.0.smithi067.stdout:checking for string.h... yes 2023-11-25T04:24:27.241 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-11-25T04:24:27.241 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-11-25T04:24:27.270 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:27.268+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:27.284 INFO:tasks.workunit.client.0.smithi067.stdout:checking for memory.h... yes 2023-11-25T04:24:27.324 INFO:tasks.workunit.client.0.smithi067.stdout:checking for strings.h... yes 2023-11-25T04:24:27.336 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:27.336 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:27.344 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:24:27.363 INFO:tasks.workunit.client.0.smithi067.stdout:checking for inttypes.h... yes 2023-11-25T04:24:27.370 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:27.369+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:27.403 INFO:tasks.workunit.client.0.smithi067.stdout:checking for stdint.h... yes 2023-11-25T04:24:27.445 INFO:tasks.workunit.client.0.smithi067.stdout:checking for unistd.h... yes 2023-11-25T04:24:27.452 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:27.452 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:27.460 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:24:27.486 INFO:tasks.workunit.client.0.smithi067.stdout:checking for dlfcn.h... yes 2023-11-25T04:24:27.492 INFO:tasks.workunit.client.0.smithi067.stdout:checking for objdir... .libs 2023-11-25T04:24:27.508 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:27.506+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:27.570 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:27.571 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:27.573 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:27.572+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:27.579 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:24:27.596 INFO:tasks.workunit.client.0.smithi067.stdout:checking if gcc supports -fno-rtti -fno-exceptions... no 2023-11-25T04:24:27.596 INFO:tasks.workunit.client.0.smithi067.stdout:checking for gcc option to produce PIC... -fPIC -DPIC 2023-11-25T04:24:27.623 INFO:tasks.workunit.client.0.smithi067.stdout:checking if gcc PIC flag -fPIC -DPIC works... yes 2023-11-25T04:24:27.662 INFO:tasks.workunit.client.0.smithi067.stdout:checking if gcc static flag -static works... no 2023-11-25T04:24:27.674 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:27.673+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:27.725 INFO:tasks.workunit.client.0.smithi067.stdout:checking if gcc supports -c -o file.o... yes 2023-11-25T04:24:27.725 INFO:tasks.workunit.client.0.smithi067.stdout:checking if gcc supports -c -o file.o... (cached) yes 2023-11-25T04:24:27.727 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:27.727 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:27.732 INFO:tasks.workunit.client.0.smithi067.stdout:checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes 2023-11-25T04:24:27.736 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:24:27.776 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:27.775+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:27.800 INFO:tasks.workunit.client.0.smithi067.stdout:checking whether -lc should be explicitly linked in... no 2023-11-25T04:24:27.877 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:27.876+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:27.882 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:27.882 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:27.891 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:24:27.973 INFO:tasks.workunit.client.0.smithi067.stdout:checking dynamic linker characteristics... GNU/Linux ld.so 2023-11-25T04:24:27.973 INFO:tasks.workunit.client.0.smithi067.stdout:checking how to hardcode library paths into programs... immediate 2023-11-25T04:24:27.973 INFO:tasks.workunit.client.0.smithi067.stdout:checking whether stripping libraries is possible... yes 2023-11-25T04:24:27.973 INFO:tasks.workunit.client.0.smithi067.stdout:checking if libtool supports shared libraries... yes 2023-11-25T04:24:27.973 INFO:tasks.workunit.client.0.smithi067.stdout:checking whether to build shared libraries... yes 2023-11-25T04:24:27.974 INFO:tasks.workunit.client.0.smithi067.stdout:checking whether to build static libraries... yes 2023-11-25T04:24:27.978 INFO:tasks.workunit.client.0.smithi067.stdout:checking for gcc... (cached) gcc 2023-11-25T04:24:27.979 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:27.976+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:27.997 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:27.997 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:28.006 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:24:28.007 INFO:tasks.workunit.client.0.smithi067.stdout:checking whether we are using the GNU C compiler... (cached) yes 2023-11-25T04:24:28.008 INFO:tasks.workunit.client.0.smithi067.stdout:checking whether gcc accepts -g... (cached) yes 2023-11-25T04:24:28.008 INFO:tasks.workunit.client.0.smithi067.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2023-11-25T04:24:28.008 INFO:tasks.workunit.client.0.smithi067.stdout:checking for gmake... /usr/bin/gmake 2023-11-25T04:24:28.008 INFO:tasks.workunit.client.0.smithi067.stderr:./configure: line 11968: test: $(SHELL): binary operator expected 2023-11-25T04:24:28.009 INFO:tasks.workunit.client.0.smithi067.stdout:checking for tar... /bin/tar 2023-11-25T04:24:28.009 INFO:tasks.workunit.client.0.smithi067.stdout:checking for gzip... /bin/gzip 2023-11-25T04:24:28.079 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:28.078+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:28.108 INFO:tasks.workunit.client.0.smithi067.stdout:checking whether gcc -MM is supported... yes 2023-11-25T04:24:28.109 INFO:tasks.workunit.client.0.smithi067.stdout:checking for sort... /bin/sort 2023-11-25T04:24:28.109 INFO:tasks.workunit.client.0.smithi067.stdout:checking whether ln -s works... yes 2023-11-25T04:24:28.111 INFO:tasks.workunit.client.0.smithi067.stdout:checking for rpm... /bin/rpm 2023-11-25T04:24:28.114 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:28.114 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:28.115 INFO:tasks.workunit.client.0.smithi067.stdout:checking for rpmbuild... no 2023-11-25T04:24:28.115 INFO:tasks.workunit.client.0.smithi067.stdout:checking for ANSI C header files... (cached) yes 2023-11-25T04:24:28.125 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:24:28.158 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2023-11-25T04:24:28.182 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:28.179+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:28.230 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:28.230 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:28.238 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:24:28.276 INFO:tasks.workunit.client.0.smithi067.stdout:checking assert.h usability... yes 2023-11-25T04:24:28.282 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:28.281+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:28.289 INFO:tasks.workunit.client.0.smithi067.stdout:checking assert.h presence... yes 2023-11-25T04:24:28.294 INFO:tasks.workunit.client.0.smithi067.stdout:checking for assert.h... yes 2023-11-25T04:24:28.339 INFO:tasks.workunit.client.0.smithi067.stdout:checking bstring.h usability... no 2023-11-25T04:24:28.344 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:28.344 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:28.353 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:24:28.361 INFO:tasks.workunit.client.0.smithi067.stdout:checking bstring.h presence... no 2023-11-25T04:24:28.361 INFO:tasks.workunit.client.0.smithi067.stdout:checking for bstring.h... no 2023-11-25T04:24:28.382 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:28.381+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:28.420 INFO:tasks.workunit.client.0.smithi067.stdout:checking libgen.h usability... yes 2023-11-25T04:24:28.434 INFO:tasks.workunit.client.0.smithi067.stdout:checking libgen.h presence... yes 2023-11-25T04:24:28.434 INFO:tasks.workunit.client.0.smithi067.stdout:checking for libgen.h... yes 2023-11-25T04:24:28.483 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:28.482+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:28.499 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:28.499 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:28.508 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:24:28.530 INFO:tasks.workunit.client.0.smithi067.stdout:checking dirent.h usability... yes 2023-11-25T04:24:28.547 INFO:tasks.workunit.client.0.smithi067.stdout:checking dirent.h presence... yes 2023-11-25T04:24:28.547 INFO:tasks.workunit.client.0.smithi067.stdout:checking for dirent.h... yes 2023-11-25T04:24:28.587 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:28.584+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:28.605 INFO:tasks.workunit.client.0.smithi067.stdout:checking errno.h usability... yes 2023-11-25T04:24:28.618 INFO:tasks.workunit.client.0.smithi067.stdout:checking errno.h presence... yes 2023-11-25T04:24:28.619 INFO:tasks.workunit.client.0.smithi067.stdout:checking for errno.h... yes 2023-11-25T04:24:28.655 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:28.655 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:28.663 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:24:28.688 INFO:tasks.workunit.client.0.smithi067.stdout:checking malloc.h usability... yes 2023-11-25T04:24:28.688 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:28.687+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:28.700 INFO:tasks.workunit.client.0.smithi067.stdout:checking malloc.h presence... yes 2023-11-25T04:24:28.700 INFO:tasks.workunit.client.0.smithi067.stdout:checking for malloc.h... yes 2023-11-25T04:24:28.754 INFO:tasks.workunit.client.0.smithi067.stdout:checking uuid.h usability... no 2023-11-25T04:24:28.771 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:28.771 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:28.775 INFO:tasks.workunit.client.0.smithi067.stdout:checking uuid.h presence... no 2023-11-25T04:24:28.775 INFO:tasks.workunit.client.0.smithi067.stdout:checking for uuid.h... no 2023-11-25T04:24:28.779 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:24:28.823 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:28.822+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:28.883 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:28.883 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:28.893 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:24:28.894 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:28.890+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:28.897 INFO:tasks.workunit.client.0.smithi067.stdout:checking uuid/uuid.h usability... yes 2023-11-25T04:24:28.913 INFO:tasks.workunit.client.0.smithi067.stdout:checking uuid/uuid.h presence... yes 2023-11-25T04:24:28.913 INFO:tasks.workunit.client.0.smithi067.stdout:checking for uuid/uuid.h... yes 2023-11-25T04:24:28.958 INFO:tasks.workunit.client.0.smithi067.stdout:checking sys/uuid.h usability... no 2023-11-25T04:24:28.979 INFO:tasks.workunit.client.0.smithi067.stdout:checking sys/uuid.h presence... no 2023-11-25T04:24:28.979 INFO:tasks.workunit.client.0.smithi067.stdout:checking for sys/uuid.h... no 2023-11-25T04:24:28.992 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:28.991+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:29.017 INFO:tasks.workunit.client.0.smithi067.stdout:checking sys/file.h usability... yes 2023-11-25T04:24:29.031 INFO:tasks.workunit.client.0.smithi067.stdout:checking sys/file.h presence... yes 2023-11-25T04:24:29.031 INFO:tasks.workunit.client.0.smithi067.stdout:checking for sys/file.h... yes 2023-11-25T04:24:29.038 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:29.038 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:29.047 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:24:29.093 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:29.091+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:29.114 INFO:tasks.workunit.client.0.smithi067.stdout:checking sys/fcntl.h usability... yes 2023-11-25T04:24:29.128 INFO:tasks.workunit.client.0.smithi067.stdout:checking sys/fcntl.h presence... yes 2023-11-25T04:24:29.128 INFO:tasks.workunit.client.0.smithi067.stdout:checking for sys/fcntl.h... yes 2023-11-25T04:24:29.148 INFO:teuthology.orchestra.run.smithi067.stderr:noscrub is set 2023-11-25T04:24:29.163 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:29.163 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:29.172 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:24:29.177 INFO:tasks.workunit.client.0.smithi067.stdout:checking sys/syssgi.h usability... no 2023-11-25T04:24:29.199 INFO:tasks.workunit.client.0.smithi067.stdout:checking sys/syssgi.h presence... no 2023-11-25T04:24:29.199 INFO:tasks.workunit.client.0.smithi067.stdout:checking for sys/syssgi.h... no 2023-11-25T04:24:29.213 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:29.212+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:29.242 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-11-25T04:24:29.293 INFO:tasks.workunit.client.0.smithi067.stdout:checking sys/param.h usability... yes 2023-11-25T04:24:29.295 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:29.294+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:29.310 INFO:tasks.workunit.client.0.smithi067.stdout:checking sys/param.h presence... yes 2023-11-25T04:24:29.310 INFO:tasks.workunit.client.0.smithi067.stdout:checking for sys/param.h... yes 2023-11-25T04:24:29.320 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:29.320 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:29.321 INFO:tasks.workunit.client.0.smithi067.stdout:checking for sys/stat.h... (cached) yes 2023-11-25T04:24:29.329 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:24:29.357 INFO:teuthology.orchestra.run.smithi067.stderr:Error ENXIO: problem getting command descriptions from osd.0 2023-11-25T04:24:29.371 INFO:tasks.workunit.client.0.smithi067.stdout:checking sys/statvfs.h usability... yes 2023-11-25T04:24:29.377 INFO:teuthology.orchestra.run.smithi067.stdout:osd.1: {} 2023-11-25T04:24:29.377 INFO:teuthology.orchestra.run.smithi067.stderr:osd.1: osd_enable_op_tracker = '' 2023-11-25T04:24:29.387 INFO:tasks.workunit.client.0.smithi067.stdout:checking sys/statvfs.h presence... yes 2023-11-25T04:24:29.388 INFO:tasks.workunit.client.0.smithi067.stdout:checking for sys/statvfs.h... yes 2023-11-25T04:24:29.400 INFO:teuthology.orchestra.run.smithi067.stdout:osd.2: {} 2023-11-25T04:24:29.400 INFO:teuthology.orchestra.run.smithi067.stderr:osd.2: osd_enable_op_tracker = '' 2023-11-25T04:24:29.403 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-11-25T04:24:29.404 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-11-25T04:24:29.413 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:29.412+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:29.432 INFO:tasks.workunit.client.0.smithi067.stdout:checking sys/time.h usability... yes 2023-11-25T04:24:29.446 INFO:tasks.workunit.client.0.smithi067.stdout:checking sys/time.h presence... yes 2023-11-25T04:24:29.446 INFO:tasks.workunit.client.0.smithi067.stdout:checking for sys/time.h... yes 2023-11-25T04:24:29.477 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:29.477 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2023-11-25T04:24:29.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2023-11-25T04:24:29.477 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2023-11-25T04:24:29.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.477 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:152 3.1a 3:5e41ab69:::200.00000005:head [write 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:29.478 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:21.237401+0000", 2023-11-25T04:24:29.478 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.2375275220000006, 2023-11-25T04:24:29.478 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015253802, 2023-11-25T04:24:29.478 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:29.478 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:29.478 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:29.478 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.478 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:29.478 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.237401+0000", 2023-11-25T04:24:29.478 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:29.478 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.478 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.479 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:29.479 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.237401+0000", 2023-11-25T04:24:29.479 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:29.479 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.479 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.479 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:29.479 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.237402+0000", 2023-11-25T04:24:29.479 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3699999999999999e-07 2023-11-25T04:24:29.479 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.479 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.479 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:29.480 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243182+0000", 2023-11-25T04:24:29.480 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057801270000000004 2023-11-25T04:24:29.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.480 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.480 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:29.482 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243193+0000", 2023-11-25T04:24:29.482 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1221e-05 2023-11-25T04:24:29.482 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.482 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.483 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:29.483 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243198+0000", 2023-11-25T04:24:29.483 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3739999999999997e-06 2023-11-25T04:24:29.484 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.484 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.484 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:29.484 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243246+0000", 2023-11-25T04:24:29.484 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7667999999999998e-05 2023-11-25T04:24:29.484 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.484 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.484 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:29.484 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243292+0000", 2023-11-25T04:24:29.484 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5794000000000002e-05 2023-11-25T04:24:29.485 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.485 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.485 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:29.485 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243360+0000", 2023-11-25T04:24:29.485 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8479000000000006e-05 2023-11-25T04:24:29.485 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.485 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.485 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:29.485 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.250667+0000", 2023-11-25T04:24:29.485 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.007306618 2023-11-25T04:24:29.485 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.485 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.485 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:29.486 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.252598+0000", 2023-11-25T04:24:29.486 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019315129999999999 2023-11-25T04:24:29.486 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.486 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.486 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:29.486 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.252631+0000", 2023-11-25T04:24:29.486 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2925000000000001e-05 2023-11-25T04:24:29.486 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.486 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.486 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:29.486 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.252655+0000", 2023-11-25T04:24:29.486 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3246000000000001e-05 2023-11-25T04:24:29.487 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:29.487 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:29.487 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:29.487 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.487 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.487 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:2841 4.4 e47/17 4:22111d42:::100000010df.00000000:head v 47'139, mlcod=47'137)", 2023-11-25T04:24:29.487 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.093610+0000", 2023-11-25T04:24:29.487 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.3813186589999997, 2023-11-25T04:24:29.487 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014758871999999999, 2023-11-25T04:24:29.487 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:29.487 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:29.487 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:29.487 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.488 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:29.488 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093610+0000", 2023-11-25T04:24:29.488 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:29.488 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.488 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.488 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:29.488 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093610+0000", 2023-11-25T04:24:29.488 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:29.488 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.488 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.488 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:29.488 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093610+0000", 2023-11-25T04:24:29.488 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1799999999999999e-07 2023-11-25T04:24:29.488 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.489 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.489 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:29.489 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093621+0000", 2023-11-25T04:24:29.489 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1107999999999999e-05 2023-11-25T04:24:29.489 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.489 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.489 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:29.489 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093627+0000", 2023-11-25T04:24:29.489 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0310000000000004e-06 2023-11-25T04:24:29.489 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.489 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.489 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:29.489 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093633+0000", 2023-11-25T04:24:29.489 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2700000000000001e-06 2023-11-25T04:24:29.490 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.490 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.490 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:29.490 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093722+0000", 2023-11-25T04:24:29.490 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8417000000000004e-05 2023-11-25T04:24:29.490 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.490 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.490 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:29.490 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093737+0000", 2023-11-25T04:24:29.490 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5781e-05 2023-11-25T04:24:29.490 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.490 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.490 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:29.491 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108203+0000", 2023-11-25T04:24:29.491 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014465844 2023-11-25T04:24:29.491 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.491 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.491 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:29.491 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108368+0000", 2023-11-25T04:24:29.491 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000165203 2023-11-25T04:24:29.491 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:29.491 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:29.491 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:29.491 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.491 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.491 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2860 4.3d 4:bf0d7877:::10000001105.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:29.492 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.093823+0000", 2023-11-25T04:24:29.492 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.3811054909999996, 2023-11-25T04:24:29.492 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015582848, 2023-11-25T04:24:29.492 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:29.492 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:29.492 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:29.492 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.492 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:29.492 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093823+0000", 2023-11-25T04:24:29.492 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:29.492 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.492 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.492 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:29.492 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093823+0000", 2023-11-25T04:24:29.493 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:29.493 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.493 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.493 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:29.493 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093823+0000", 2023-11-25T04:24:29.493 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9100000000000004e-07 2023-11-25T04:24:29.493 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.493 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.493 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:29.493 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093826+0000", 2023-11-25T04:24:29.493 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4499999999999998e-06 2023-11-25T04:24:29.493 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.493 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.494 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:29.494 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093835+0000", 2023-11-25T04:24:29.494 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8179999999999993e-06 2023-11-25T04:24:29.494 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.494 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.494 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:29.494 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093840+0000", 2023-11-25T04:24:29.494 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0819999999999998e-06 2023-11-25T04:24:29.494 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.494 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.494 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:29.494 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093973+0000", 2023-11-25T04:24:29.495 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013308999999999999 2023-11-25T04:24:29.495 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.495 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.495 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:29.495 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094277+0000", 2023-11-25T04:24:29.495 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00030403299999999999 2023-11-25T04:24:29.495 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.495 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.495 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:29.495 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108608+0000", 2023-11-25T04:24:29.495 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014330888 2023-11-25T04:24:29.495 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.495 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.496 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:29.496 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108676+0000", 2023-11-25T04:24:29.496 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8577000000000004e-05 2023-11-25T04:24:29.496 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.496 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.496 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:29.496 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109406+0000", 2023-11-25T04:24:29.496 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00072931899999999997 2023-11-25T04:24:29.496 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:29.496 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:29.496 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:29.496 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.497 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.497 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2862 4.13 4:cb2e645e:::10000001109.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:29.497 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.093852+0000", 2023-11-25T04:24:29.497 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.3810765299999996, 2023-11-25T04:24:29.497 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015190636, 2023-11-25T04:24:29.497 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:29.497 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:29.497 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:29.497 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.497 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:29.497 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093852+0000", 2023-11-25T04:24:29.497 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:29.497 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.498 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.498 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:29.498 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093852+0000", 2023-11-25T04:24:29.498 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:29.498 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.498 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.498 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:29.498 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093852+0000", 2023-11-25T04:24:29.498 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0299999999999999e-07 2023-11-25T04:24:29.498 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.498 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.498 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:29.498 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093854+0000", 2023-11-25T04:24:29.499 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.846e-06 2023-11-25T04:24:29.499 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.499 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.499 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:29.499 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093861+0000", 2023-11-25T04:24:29.499 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5520000000000001e-06 2023-11-25T04:24:29.499 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.499 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.499 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:29.499 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093865+0000", 2023-11-25T04:24:29.499 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7870000000000001e-06 2023-11-25T04:24:29.499 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.499 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.500 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:29.500 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093940+0000", 2023-11-25T04:24:29.500 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4799e-05 2023-11-25T04:24:29.500 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.500 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.500 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:29.500 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094052+0000", 2023-11-25T04:24:29.500 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011156500000000001 2023-11-25T04:24:29.500 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.500 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.500 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:29.500 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108928+0000", 2023-11-25T04:24:29.500 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014876245 2023-11-25T04:24:29.500 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.501 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.501 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:29.501 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108972+0000", 2023-11-25T04:24:29.501 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4477999999999997e-05 2023-11-25T04:24:29.501 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.501 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.501 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:29.501 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109042+0000", 2023-11-25T04:24:29.501 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9861000000000001e-05 2023-11-25T04:24:29.501 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:29.501 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:29.501 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:29.501 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.501 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.502 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2865 4.36 4:6ebc5ac6:::1000000110f.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:29.502 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.093906+0000", 2023-11-25T04:24:29.502 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.3810222569999997, 2023-11-25T04:24:29.502 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014962628, 2023-11-25T04:24:29.502 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:29.502 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:29.502 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:29.502 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.502 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:29.502 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093906+0000", 2023-11-25T04:24:29.503 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:29.503 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.503 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.503 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:29.503 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093906+0000", 2023-11-25T04:24:29.503 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:29.503 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.503 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.503 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:29.503 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093906+0000", 2023-11-25T04:24:29.503 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4400000000000001e-07 2023-11-25T04:24:29.503 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.503 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.503 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:29.504 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093908+0000", 2023-11-25T04:24:29.504 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8640000000000001e-06 2023-11-25T04:24:29.504 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.504 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.504 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:29.504 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093914+0000", 2023-11-25T04:24:29.504 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0170000000000002e-06 2023-11-25T04:24:29.504 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.504 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.504 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:29.504 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093921+0000", 2023-11-25T04:24:29.504 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6529999999999997e-06 2023-11-25T04:24:29.504 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.504 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.505 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:29.505 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094258+0000", 2023-11-25T04:24:29.505 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00033703599999999998 2023-11-25T04:24:29.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.505 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.505 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:29.505 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094340+0000", 2023-11-25T04:24:29.505 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2279000000000003e-05 2023-11-25T04:24:29.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.505 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.505 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:29.505 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094416+0000", 2023-11-25T04:24:29.505 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6248999999999995e-05 2023-11-25T04:24:29.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.506 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.506 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:29.506 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.096134+0000", 2023-11-25T04:24:29.506 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001717611 2023-11-25T04:24:29.506 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.506 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.506 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:29.506 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108809+0000", 2023-11-25T04:24:29.506 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012675327 2023-11-25T04:24:29.506 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.506 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.506 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:29.506 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108836+0000", 2023-11-25T04:24:29.506 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6509999999999999e-05 2023-11-25T04:24:29.507 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.507 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.509 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:29.509 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108869+0000", 2023-11-25T04:24:29.509 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2737999999999997e-05 2023-11-25T04:24:29.509 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:29.509 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:29.509 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:29.509 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.510 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.510 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:2852 4.2d e47/17 4:b745ac76:::100000010f5.00000000:head v 47'126, mlcod=47'124)", 2023-11-25T04:24:29.510 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.093987+0000", 2023-11-25T04:24:29.510 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.3809416959999998, 2023-11-25T04:24:29.510 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014613256, 2023-11-25T04:24:29.510 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:29.510 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:29.510 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:29.510 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.510 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:29.510 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093987+0000", 2023-11-25T04:24:29.510 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:29.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.511 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.511 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:29.511 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093987+0000", 2023-11-25T04:24:29.511 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:29.511 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.511 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.511 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:29.511 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093987+0000", 2023-11-25T04:24:29.511 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7000000000000001e-07 2023-11-25T04:24:29.511 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.511 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.511 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:29.511 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093990+0000", 2023-11-25T04:24:29.512 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8729999999999999e-06 2023-11-25T04:24:29.512 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.512 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.512 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:29.512 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093997+0000", 2023-11-25T04:24:29.512 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.973e-06 2023-11-25T04:24:29.512 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.512 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.512 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:29.512 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094001+0000", 2023-11-25T04:24:29.513 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7360000000000001e-06 2023-11-25T04:24:29.513 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.513 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.513 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:29.513 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094263+0000", 2023-11-25T04:24:29.513 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026173400000000001 2023-11-25T04:24:29.513 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.513 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.513 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:29.514 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094278+0000", 2023-11-25T04:24:29.514 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4681999999999999e-05 2023-11-25T04:24:29.514 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.515 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.515 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:29.515 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108563+0000", 2023-11-25T04:24:29.515 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014284738 2023-11-25T04:24:29.515 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.515 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.515 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:29.515 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108600+0000", 2023-11-25T04:24:29.515 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7249999999999997e-05 2023-11-25T04:24:29.515 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:29.515 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:29.515 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:29.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.516 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.516 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2866 4.36 4:6f6907fb:::10000001111.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:29.516 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.094033+0000", 2023-11-25T04:24:29.516 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.3808955310000002, 2023-11-25T04:24:29.516 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014885591, 2023-11-25T04:24:29.516 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:29.516 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:29.516 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:29.516 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.516 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:29.516 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094033+0000", 2023-11-25T04:24:29.517 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:29.517 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.517 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.517 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:29.517 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094033+0000", 2023-11-25T04:24:29.517 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:29.517 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.517 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.517 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:29.517 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094033+0000", 2023-11-25T04:24:29.517 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.06e-07 2023-11-25T04:24:29.517 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.517 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.517 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:29.518 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094036+0000", 2023-11-25T04:24:29.518 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.508e-06 2023-11-25T04:24:29.518 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.518 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.518 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:29.518 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094044+0000", 2023-11-25T04:24:29.518 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5909999999999996e-06 2023-11-25T04:24:29.518 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.518 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.518 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:29.518 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094051+0000", 2023-11-25T04:24:29.518 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0319999999999998e-06 2023-11-25T04:24:29.518 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.519 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.519 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:29.519 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094727+0000", 2023-11-25T04:24:29.519 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00067523300000000002 2023-11-25T04:24:29.519 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.519 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.519 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:29.519 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094795+0000", 2023-11-25T04:24:29.519 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8156000000000003e-05 2023-11-25T04:24:29.519 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.519 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.519 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:29.519 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094898+0000", 2023-11-25T04:24:29.519 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103329 2023-11-25T04:24:29.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.520 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.520 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:29.520 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108803+0000", 2023-11-25T04:24:29.520 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013904385 2023-11-25T04:24:29.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.520 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.520 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:29.520 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108875+0000", 2023-11-25T04:24:29.520 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1958999999999993e-05 2023-11-25T04:24:29.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.520 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.520 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:29.520 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108899+0000", 2023-11-25T04:24:29.521 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4019000000000002e-05 2023-11-25T04:24:29.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.521 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.521 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:29.521 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108918+0000", 2023-11-25T04:24:29.521 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9772999999999999e-05 2023-11-25T04:24:29.521 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:29.521 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:29.521 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:29.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.521 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.521 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:2854 4.2d e47/17 4:b5c259cf:::100000010f9.00000000:head v 47'127, mlcod=47'124)", 2023-11-25T04:24:29.521 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.094793+0000", 2023-11-25T04:24:29.521 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.3801352419999997, 2023-11-25T04:24:29.522 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014650764, 2023-11-25T04:24:29.522 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:29.522 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:29.522 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:29.522 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.522 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:29.522 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094793+0000", 2023-11-25T04:24:29.522 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:29.522 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.522 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.522 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:29.522 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094793+0000", 2023-11-25T04:24:29.522 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:29.523 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.523 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.523 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:29.523 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094793+0000", 2023-11-25T04:24:29.523 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3700000000000001e-07 2023-11-25T04:24:29.523 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.523 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.523 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:29.523 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094798+0000", 2023-11-25T04:24:29.523 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1200000000000004e-06 2023-11-25T04:24:29.523 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.523 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.523 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:29.523 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094805+0000", 2023-11-25T04:24:29.524 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9759999999999998e-06 2023-11-25T04:24:29.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.524 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.524 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:29.524 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094811+0000", 2023-11-25T04:24:29.524 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1410000000000002e-06 2023-11-25T04:24:29.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.524 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.524 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:29.524 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094838+0000", 2023-11-25T04:24:29.524 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6995000000000002e-05 2023-11-25T04:24:29.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.524 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.524 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:29.525 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094848+0000", 2023-11-25T04:24:29.525 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0115999999999999e-05 2023-11-25T04:24:29.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.525 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.525 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:29.525 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109408+0000", 2023-11-25T04:24:29.525 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014559825 2023-11-25T04:24:29.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.525 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.525 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:29.525 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109444+0000", 2023-11-25T04:24:29.525 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6154e-05 2023-11-25T04:24:29.525 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:29.526 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:29.526 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:29.526 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.526 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.526 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2869 4.6 4:63719fb0:::10000001117.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:29.526 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.094910+0000", 2023-11-25T04:24:29.526 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.3800182850000002, 2023-11-25T04:24:29.526 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014132570000000001, 2023-11-25T04:24:29.526 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:29.526 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:29.526 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:29.527 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.527 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:29.527 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094910+0000", 2023-11-25T04:24:29.527 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:29.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.527 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.527 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:29.527 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094910+0000", 2023-11-25T04:24:29.527 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:29.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.527 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.527 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:29.527 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094910+0000", 2023-11-25T04:24:29.528 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.58e-07 2023-11-25T04:24:29.528 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.528 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.528 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:29.528 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094913+0000", 2023-11-25T04:24:29.528 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.198e-06 2023-11-25T04:24:29.528 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.528 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.528 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:29.528 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094921+0000", 2023-11-25T04:24:29.528 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1629999999999996e-06 2023-11-25T04:24:29.528 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.528 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.528 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:29.529 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094926+0000", 2023-11-25T04:24:29.529 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7559999999999999e-06 2023-11-25T04:24:29.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.529 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.529 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:29.529 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094966+0000", 2023-11-25T04:24:29.529 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0151999999999998e-05 2023-11-25T04:24:29.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.529 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.529 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:29.529 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095073+0000", 2023-11-25T04:24:29.529 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107657 2023-11-25T04:24:29.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.530 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.530 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:29.530 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095158+0000", 2023-11-25T04:24:29.530 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5136999999999995e-05 2023-11-25T04:24:29.530 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.530 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.530 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:29.530 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108924+0000", 2023-11-25T04:24:29.530 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013765727 2023-11-25T04:24:29.530 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.530 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.530 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:29.530 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108958+0000", 2023-11-25T04:24:29.530 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3417000000000002e-05 2023-11-25T04:24:29.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.531 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.531 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:29.531 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109012+0000", 2023-11-25T04:24:29.531 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3939e-05 2023-11-25T04:24:29.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.531 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.531 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:29.531 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109043+0000", 2023-11-25T04:24:29.531 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0966000000000001e-05 2023-11-25T04:24:29.531 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:29.531 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:29.531 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:29.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.532 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.532 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2871 4.3f 4:fff10e11:::1000000111b.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:29.532 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.094936+0000", 2023-11-25T04:24:29.532 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.3799924470000002, 2023-11-25T04:24:29.532 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014261726000000001, 2023-11-25T04:24:29.532 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:29.532 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:29.532 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:29.532 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.532 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:29.532 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094936+0000", 2023-11-25T04:24:29.532 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:29.532 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.533 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.533 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:29.533 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094936+0000", 2023-11-25T04:24:29.533 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:29.533 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.533 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.533 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:29.533 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094936+0000", 2023-11-25T04:24:29.533 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9900000000000001e-07 2023-11-25T04:24:29.533 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.533 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.533 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:29.533 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094938+0000", 2023-11-25T04:24:29.533 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9599999999999999e-06 2023-11-25T04:24:29.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.534 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.534 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:29.534 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094944+0000", 2023-11-25T04:24:29.534 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2219999999999999e-06 2023-11-25T04:24:29.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.534 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.534 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:29.534 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094950+0000", 2023-11-25T04:24:29.534 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1329999999999998e-06 2023-11-25T04:24:29.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.534 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.534 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:29.534 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094993+0000", 2023-11-25T04:24:29.535 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3305e-05 2023-11-25T04:24:29.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.535 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:29.535 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095107+0000", 2023-11-25T04:24:29.535 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011449400000000001 2023-11-25T04:24:29.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.535 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:29.535 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095176+0000", 2023-11-25T04:24:29.535 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8330000000000005e-05 2023-11-25T04:24:29.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.535 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:29.536 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107939+0000", 2023-11-25T04:24:29.536 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012763801 2023-11-25T04:24:29.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.536 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.536 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:29.536 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109128+0000", 2023-11-25T04:24:29.536 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011884770000000001 2023-11-25T04:24:29.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.536 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.536 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:29.536 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109166+0000", 2023-11-25T04:24:29.536 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7948999999999999e-05 2023-11-25T04:24:29.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.537 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.537 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:29.537 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109198+0000", 2023-11-25T04:24:29.537 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1656000000000003e-05 2023-11-25T04:24:29.537 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:29.537 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:29.537 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:29.537 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.537 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.537 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2872 4.32 4:4fb002bc:::1000000111d.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:29.537 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.094961+0000", 2023-11-25T04:24:29.537 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.3799677429999999, 2023-11-25T04:24:29.537 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014631564999999999, 2023-11-25T04:24:29.538 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:29.538 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:29.538 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:29.538 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.538 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:29.538 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094961+0000", 2023-11-25T04:24:29.538 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:29.538 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.538 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.538 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:29.538 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094961+0000", 2023-11-25T04:24:29.538 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:29.538 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.538 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.539 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:29.539 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094961+0000", 2023-11-25T04:24:29.539 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9299999999999999e-07 2023-11-25T04:24:29.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.539 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:29.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094963+0000", 2023-11-25T04:24:29.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9209999999999999e-06 2023-11-25T04:24:29.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.540 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.540 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:29.541 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094968+0000", 2023-11-25T04:24:29.541 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.502e-06 2023-11-25T04:24:29.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.541 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.541 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:29.541 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094973+0000", 2023-11-25T04:24:29.541 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8559999999999998e-06 2023-11-25T04:24:29.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.541 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.541 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:29.541 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107722+0000", 2023-11-25T04:24:29.541 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012748534000000001 2023-11-25T04:24:29.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.541 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.542 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:29.542 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107844+0000", 2023-11-25T04:24:29.542 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012233400000000001 2023-11-25T04:24:29.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.542 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.542 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:29.542 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109469+0000", 2023-11-25T04:24:29.542 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001625175 2023-11-25T04:24:29.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.542 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.542 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:29.542 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109518+0000", 2023-11-25T04:24:29.542 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8980000000000002e-05 2023-11-25T04:24:29.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.543 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.543 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:29.543 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109592+0000", 2023-11-25T04:24:29.543 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3869999999999996e-05 2023-11-25T04:24:29.543 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:29.543 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:29.543 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:29.543 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.543 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.543 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2873 4.13 4:cae4f303:::1000000111f.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:29.543 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.094991+0000", 2023-11-25T04:24:29.543 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.379937151, 2023-11-25T04:24:29.543 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015101692, 2023-11-25T04:24:29.544 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:29.544 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:29.544 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:29.544 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.544 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:29.544 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094991+0000", 2023-11-25T04:24:29.544 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:29.544 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.544 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.544 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:29.544 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094991+0000", 2023-11-25T04:24:29.544 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:29.544 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.544 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.545 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:29.545 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094992+0000", 2023-11-25T04:24:29.545 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7e-07 2023-11-25T04:24:29.545 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.545 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.545 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:29.545 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094994+0000", 2023-11-25T04:24:29.545 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2110000000000001e-06 2023-11-25T04:24:29.545 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.545 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.545 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:29.545 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095000+0000", 2023-11-25T04:24:29.545 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.241e-06 2023-11-25T04:24:29.546 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.546 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.546 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:29.546 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095006+0000", 2023-11-25T04:24:29.546 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3659999999999997e-06 2023-11-25T04:24:29.546 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.546 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.546 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:29.546 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108369+0000", 2023-11-25T04:24:29.546 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01336247 2023-11-25T04:24:29.546 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.546 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.546 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:29.547 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108492+0000", 2023-11-25T04:24:29.547 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000123192 2023-11-25T04:24:29.547 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.547 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.547 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:29.547 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109975+0000", 2023-11-25T04:24:29.547 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001482894 2023-11-25T04:24:29.547 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.547 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.547 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:29.547 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110024+0000", 2023-11-25T04:24:29.547 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9097000000000001e-05 2023-11-25T04:24:29.547 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.547 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.548 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:29.548 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110093+0000", 2023-11-25T04:24:29.548 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8751000000000006e-05 2023-11-25T04:24:29.548 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:29.548 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:29.548 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:29.548 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.548 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.548 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2874 4.14 4:2b557d2e:::10000001121.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:29.548 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095019+0000", 2023-11-25T04:24:29.548 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.3799096249999998, 2023-11-25T04:24:29.548 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014998296, 2023-11-25T04:24:29.548 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:29.548 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:29.549 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:29.549 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.549 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:29.549 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095019+0000", 2023-11-25T04:24:29.549 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:29.549 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.549 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.549 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:29.549 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095019+0000", 2023-11-25T04:24:29.549 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:29.549 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.549 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.549 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:29.549 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095019+0000", 2023-11-25T04:24:29.550 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7300000000000001e-07 2023-11-25T04:24:29.550 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.550 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.550 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:29.550 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095021+0000", 2023-11-25T04:24:29.550 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0099999999999998e-06 2023-11-25T04:24:29.550 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.550 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.550 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:29.550 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095028+0000", 2023-11-25T04:24:29.550 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7870000000000004e-06 2023-11-25T04:24:29.550 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.550 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.551 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:29.551 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095033+0000", 2023-11-25T04:24:29.551 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2039999999999996e-06 2023-11-25T04:24:29.551 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.551 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.551 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:29.551 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107740+0000", 2023-11-25T04:24:29.551 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012706693 2023-11-25T04:24:29.551 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.551 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.551 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:29.551 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107817+0000", 2023-11-25T04:24:29.551 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7599e-05 2023-11-25T04:24:29.551 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.551 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.552 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:29.552 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107961+0000", 2023-11-25T04:24:29.552 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000143194 2023-11-25T04:24:29.552 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.552 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.552 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:29.552 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109364+0000", 2023-11-25T04:24:29.552 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001403226 2023-11-25T04:24:29.552 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.552 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.552 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:29.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109507+0000", 2023-11-25T04:24:29.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000142991 2023-11-25T04:24:29.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:29.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109986+0000", 2023-11-25T04:24:29.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00047906899999999999 2023-11-25T04:24:29.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.572 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:29.572 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110017+0000", 2023-11-25T04:24:29.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1050000000000003e-05 2023-11-25T04:24:29.572 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:29.572 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:29.572 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:29.572 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.572 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.572 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2875 4.d 4:b17d5dc6:::10000001123.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:29.572 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095049+0000", 2023-11-25T04:24:29.572 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.3798797010000001, 2023-11-25T04:24:29.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014882886, 2023-11-25T04:24:29.573 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:29.573 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:29.573 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:29.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.573 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:29.573 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095049+0000", 2023-11-25T04:24:29.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:29.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.573 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:29.573 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095049+0000", 2023-11-25T04:24:29.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:29.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:29.574 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095049+0000", 2023-11-25T04:24:29.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6100000000000001e-07 2023-11-25T04:24:29.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:29.574 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095051+0000", 2023-11-25T04:24:29.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1449999999999998e-06 2023-11-25T04:24:29.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:29.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095058+0000", 2023-11-25T04:24:29.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.601e-06 2023-11-25T04:24:29.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:29.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095064+0000", 2023-11-25T04:24:29.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1079999999999998e-06 2023-11-25T04:24:29.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:29.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108015+0000", 2023-11-25T04:24:29.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012951582 2023-11-25T04:24:29.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.576 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:29.576 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108260+0000", 2023-11-25T04:24:29.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024445199999999998 2023-11-25T04:24:29.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.576 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:29.576 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109825+0000", 2023-11-25T04:24:29.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0015651759999999999 2023-11-25T04:24:29.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.576 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:29.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109866+0000", 2023-11-25T04:24:29.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0629e-05 2023-11-25T04:24:29.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.577 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:29.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109931+0000", 2023-11-25T04:24:29.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5732e-05 2023-11-25T04:24:29.577 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:29.577 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:29.577 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:29.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.578 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2876 4.3b 4:dc871776:::10000001125.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:29.578 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095077+0000", 2023-11-25T04:24:29.578 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.3798510799999999, 2023-11-25T04:24:29.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014500944, 2023-11-25T04:24:29.578 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:29.578 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:29.578 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:29.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.578 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:29.578 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095077+0000", 2023-11-25T04:24:29.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:29.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:29.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095077+0000", 2023-11-25T04:24:29.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:29.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:29.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095078+0000", 2023-11-25T04:24:29.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9799999999999999e-07 2023-11-25T04:24:29.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:29.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095079+0000", 2023-11-25T04:24:29.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.877e-06 2023-11-25T04:24:29.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:29.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095086+0000", 2023-11-25T04:24:29.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2439999999999998e-06 2023-11-25T04:24:29.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:29.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095094+0000", 2023-11-25T04:24:29.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9710000000000004e-06 2023-11-25T04:24:29.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.581 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:29.581 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107732+0000", 2023-11-25T04:24:29.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012638104000000001 2023-11-25T04:24:29.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.581 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:29.581 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107854+0000", 2023-11-25T04:24:29.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012261600000000001 2023-11-25T04:24:29.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.581 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:29.581 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107941+0000", 2023-11-25T04:24:29.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6935999999999996e-05 2023-11-25T04:24:29.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:29.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108871+0000", 2023-11-25T04:24:29.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00092955499999999999 2023-11-25T04:24:29.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:29.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109483+0000", 2023-11-25T04:24:29.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00061206100000000001 2023-11-25T04:24:29.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.583 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:29.583 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109527+0000", 2023-11-25T04:24:29.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4112e-05 2023-11-25T04:24:29.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.583 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:29.583 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109578+0000", 2023-11-25T04:24:29.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1069999999999997e-05 2023-11-25T04:24:29.583 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:29.583 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:29.583 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:29.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.583 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2878 4.3e 4:7ff98b9d:::10000001129.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:29.584 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095101+0000", 2023-11-25T04:24:29.584 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.3798272970000003, 2023-11-25T04:24:29.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015015137, 2023-11-25T04:24:29.584 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:29.584 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:29.584 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:29.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.584 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:29.584 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095101+0000", 2023-11-25T04:24:29.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:29.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.584 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:29.584 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095101+0000", 2023-11-25T04:24:29.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:29.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:29.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095101+0000", 2023-11-25T04:24:29.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7399999999999999e-07 2023-11-25T04:24:29.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:29.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095103+0000", 2023-11-25T04:24:29.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7600000000000001e-06 2023-11-25T04:24:29.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:29.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095109+0000", 2023-11-25T04:24:29.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8189999999999997e-06 2023-11-25T04:24:29.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:29.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095114+0000", 2023-11-25T04:24:29.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6770000000000003e-06 2023-11-25T04:24:29.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:29.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107724+0000", 2023-11-25T04:24:29.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012610909999999999 2023-11-25T04:24:29.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.587 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:29.587 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108483+0000", 2023-11-25T04:24:29.587 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000758407 2023-11-25T04:24:29.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.587 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:29.587 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108568+0000", 2023-11-25T04:24:29.587 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4707000000000001e-05 2023-11-25T04:24:29.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.587 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:29.587 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109687+0000", 2023-11-25T04:24:29.587 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011192750000000001 2023-11-25T04:24:29.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.588 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:29.588 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110049+0000", 2023-11-25T04:24:29.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00036210600000000001 2023-11-25T04:24:29.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.588 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:29.588 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110086+0000", 2023-11-25T04:24:29.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7500999999999999e-05 2023-11-25T04:24:29.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.588 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:29.588 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110116+0000", 2023-11-25T04:24:29.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9601e-05 2023-11-25T04:24:29.589 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:29.589 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:29.589 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:29.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.589 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2879 4.2a 4:5524ace2:::1000000112b.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:29.589 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095120+0000", 2023-11-25T04:24:29.589 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.3798085169999998, 2023-11-25T04:24:29.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014587937, 2023-11-25T04:24:29.589 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:29.589 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:29.589 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:29.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:29.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095120+0000", 2023-11-25T04:24:29.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:29.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:29.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095120+0000", 2023-11-25T04:24:29.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:29.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:29.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095120+0000", 2023-11-25T04:24:29.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5600000000000001e-07 2023-11-25T04:24:29.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.591 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:29.591 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095122+0000", 2023-11-25T04:24:29.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.694e-06 2023-11-25T04:24:29.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.591 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:29.591 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095127+0000", 2023-11-25T04:24:29.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5099999999999998e-06 2023-11-25T04:24:29.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.592 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:29.592 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095132+0000", 2023-11-25T04:24:29.592 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6210000000000003e-06 2023-11-25T04:24:29.592 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.592 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.592 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:29.592 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107729+0000", 2023-11-25T04:24:29.592 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012596931 2023-11-25T04:24:29.592 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.592 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.592 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:29.592 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107856+0000", 2023-11-25T04:24:29.592 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012707699999999999 2023-11-25T04:24:29.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:29.593 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109600+0000", 2023-11-25T04:24:29.593 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001744298 2023-11-25T04:24:29.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:29.593 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109641+0000", 2023-11-25T04:24:29.593 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0769000000000002e-05 2023-11-25T04:24:29.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.594 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:29.594 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109708+0000", 2023-11-25T04:24:29.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6680999999999994e-05 2023-11-25T04:24:29.594 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:29.594 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:29.594 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:29.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.594 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2882 4.1c 4:3a7c2e07:::10000001131.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:29.594 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095141+0000", 2023-11-25T04:24:29.594 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.3797871419999996, 2023-11-25T04:24:29.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014949469999999999, 2023-11-25T04:24:29.595 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:29.595 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:29.595 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:29.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:29.595 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095141+0000", 2023-11-25T04:24:29.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:29.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:29.595 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095141+0000", 2023-11-25T04:24:29.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:29.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.596 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.596 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:29.596 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095142+0000", 2023-11-25T04:24:29.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1899999999999998e-07 2023-11-25T04:24:29.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.596 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.596 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:29.596 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095143+0000", 2023-11-25T04:24:29.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7770000000000001e-06 2023-11-25T04:24:29.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.596 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.597 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:29.597 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095149+0000", 2023-11-25T04:24:29.597 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9909999999999999e-06 2023-11-25T04:24:29.597 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.597 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.597 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:29.597 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095154+0000", 2023-11-25T04:24:29.597 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3880000000000002e-06 2023-11-25T04:24:29.597 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.597 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.597 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:29.597 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107736+0000", 2023-11-25T04:24:29.597 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012582477999999999 2023-11-25T04:24:29.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:29.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107868+0000", 2023-11-25T04:24:29.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013134400000000001 2023-11-25T04:24:29.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:29.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107959+0000", 2023-11-25T04:24:29.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1253000000000002e-05 2023-11-25T04:24:29.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:29.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109325+0000", 2023-11-25T04:24:29.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0013662520000000001 2023-11-25T04:24:29.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:29.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110023+0000", 2023-11-25T04:24:29.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00069838799999999998 2023-11-25T04:24:29.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:29.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110061+0000", 2023-11-25T04:24:29.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7867999999999997e-05 2023-11-25T04:24:29.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.600 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:29.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110091+0000", 2023-11-25T04:24:29.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9312e-05 2023-11-25T04:24:29.600 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:29.600 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:29.600 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:29.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.600 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2883 4.3e 4:7d82be99:::10000001133.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:29.600 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095942+0000", 2023-11-25T04:24:29.600 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.3789858559999999, 2023-11-25T04:24:29.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014290573000000001, 2023-11-25T04:24:29.601 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:29.601 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:29.601 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:29.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:29.601 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095942+0000", 2023-11-25T04:24:29.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:29.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:29.601 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095942+0000", 2023-11-25T04:24:29.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:29.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:29.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095944+0000", 2023-11-25T04:24:29.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.632e-06 2023-11-25T04:24:29.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:29.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095947+0000", 2023-11-25T04:24:29.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7760000000000002e-06 2023-11-25T04:24:29.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:29.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095954+0000", 2023-11-25T04:24:29.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5379999999999997e-06 2023-11-25T04:24:29.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:29.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095960+0000", 2023-11-25T04:24:29.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9209999999999997e-06 2023-11-25T04:24:29.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:29.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108809+0000", 2023-11-25T04:24:29.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012848972 2023-11-25T04:24:29.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:29.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108921+0000", 2023-11-25T04:24:29.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000111767 2023-11-25T04:24:29.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:29.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109008+0000", 2023-11-25T04:24:29.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6576999999999995e-05 2023-11-25T04:24:29.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.605 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:29.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110124+0000", 2023-11-25T04:24:29.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011163709999999999 2023-11-25T04:24:29.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.605 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:29.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110155+0000", 2023-11-25T04:24:29.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1248999999999998e-05 2023-11-25T04:24:29.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:29.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110202+0000", 2023-11-25T04:24:29.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7097e-05 2023-11-25T04:24:29.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:29.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110233+0000", 2023-11-25T04:24:29.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0673000000000003e-05 2023-11-25T04:24:29.606 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:29.606 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:29.606 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:29.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.607 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2886 4.25 4:a579baeb:::10000001139.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:29.607 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095994+0000", 2023-11-25T04:24:29.607 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 4.3789345280000003, 2023-11-25T04:24:29.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013825356, 2023-11-25T04:24:29.607 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:29.607 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:29.607 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:29.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:29.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095994+0000", 2023-11-25T04:24:29.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:29.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:29.608 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095994+0000", 2023-11-25T04:24:29.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:29.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:29.608 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095996+0000", 2023-11-25T04:24:29.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8780000000000001e-06 2023-11-25T04:24:29.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:29.609 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095998+0000", 2023-11-25T04:24:29.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1560000000000002e-06 2023-11-25T04:24:29.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:29.609 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.096003+0000", 2023-11-25T04:24:29.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1080000000000001e-06 2023-11-25T04:24:29.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:29.609 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.096010+0000", 2023-11-25T04:24:29.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0650000000000001e-06 2023-11-25T04:24:29.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.610 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:29.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107742+0000", 2023-11-25T04:24:29.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011731691000000001 2023-11-25T04:24:29.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.610 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:29.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108063+0000", 2023-11-25T04:24:29.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00032168600000000001 2023-11-25T04:24:29.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.610 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:29.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109689+0000", 2023-11-25T04:24:29.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001625556 2023-11-25T04:24:29.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:29.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109740+0000", 2023-11-25T04:24:29.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1125000000000003e-05 2023-11-25T04:24:29.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:29.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:29.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:29.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109819+0000", 2023-11-25T04:24:29.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9091000000000006e-05 2023-11-25T04:24:29.611 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:29.612 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:29.612 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:29.612 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:29.612 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:29.612 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:29.614 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:24:29.614 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:29.612+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:29.616 INFO:tasks.workunit.client.0.smithi067.stdout:checking sys/ioctl.h usability... yes 2023-11-25T04:24:29.616 INFO:tasks.workunit.client.0.smithi067.stdout:checking sys/ioctl.h presence... yes 2023-11-25T04:24:29.616 INFO:tasks.workunit.client.0.smithi067.stdout:checking for sys/ioctl.h... yes 2023-11-25T04:24:29.616 INFO:tasks.workunit.client.0.smithi067.stdout:checking sys/wait.h usability... yes 2023-11-25T04:24:29.616 INFO:tasks.workunit.client.0.smithi067.stdout:checking sys/wait.h presence... yes 2023-11-25T04:24:29.616 INFO:tasks.workunit.client.0.smithi067.stdout:checking for sys/wait.h... yes 2023-11-25T04:24:29.616 INFO:tasks.workunit.client.0.smithi067.stdout:checking for sys/types.h... (cached) yes 2023-11-25T04:24:29.616 INFO:tasks.workunit.client.0.smithi067.stdout:checking for strings.h... (cached) yes 2023-11-25T04:24:29.646 INFO:tasks.workunit.client.0.smithi067.stdout:checking err.h usability... yes 2023-11-25T04:24:29.660 INFO:tasks.workunit.client.0.smithi067.stdout:checking err.h presence... yes 2023-11-25T04:24:29.660 INFO:tasks.workunit.client.0.smithi067.stdout:checking for err.h... yes 2023-11-25T04:24:29.702 INFO:tasks.workunit.client.0.smithi067.stdout:checking linux/falloc.h usability... yes 2023-11-25T04:24:29.714 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:29.712+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:29.715 INFO:tasks.workunit.client.0.smithi067.stdout:checking linux/falloc.h presence... yes 2023-11-25T04:24:29.715 INFO:tasks.workunit.client.0.smithi067.stdout:checking for linux/falloc.h... yes 2023-11-25T04:24:29.723 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:29.723 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:24:29.723 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2023-11-25T04:24:29.723 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:29.732 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:24:29.759 INFO:tasks.workunit.client.0.smithi067.stdout:checking sys/fs/xfs_fsops.h usability... no 2023-11-25T04:24:29.780 INFO:tasks.workunit.client.0.smithi067.stdout:checking sys/fs/xfs_fsops.h presence... no 2023-11-25T04:24:29.780 INFO:tasks.workunit.client.0.smithi067.stdout:checking for sys/fs/xfs_fsops.h... no 2023-11-25T04:24:29.814 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:29.813+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:29.818 INFO:tasks.workunit.client.0.smithi067.stdout:checking sys/fs/xfs_itable.h usability... no 2023-11-25T04:24:29.839 INFO:tasks.workunit.client.0.smithi067.stdout:checking sys/fs/xfs_itable.h presence... no 2023-11-25T04:24:29.839 INFO:tasks.workunit.client.0.smithi067.stdout:checking for sys/fs/xfs_itable.h... no 2023-11-25T04:24:29.843 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:29.843 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:24:29.844 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2023-11-25T04:24:29.844 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2023-11-25T04:24:29.844 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:29.852 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:24:29.878 INFO:tasks.workunit.client.0.smithi067.stdout:checking xfs/platform_defs.h usability... no 2023-11-25T04:24:29.898 INFO:tasks.workunit.client.0.smithi067.stdout:checking xfs/platform_defs.h presence... no 2023-11-25T04:24:29.899 INFO:tasks.workunit.client.0.smithi067.stdout:checking for xfs/platform_defs.h... no 2023-11-25T04:24:29.936 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:29.935+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:29.937 INFO:tasks.workunit.client.0.smithi067.stdout:checking btrfs/ioctl.h usability... no 2023-11-25T04:24:29.958 INFO:tasks.workunit.client.0.smithi067.stdout:checking btrfs/ioctl.h presence... no 2023-11-25T04:24:29.958 INFO:tasks.workunit.client.0.smithi067.stdout:checking for btrfs/ioctl.h... no 2023-11-25T04:24:29.996 INFO:tasks.workunit.client.0.smithi067.stdout:checking cifs/ioctl.h usability... no 2023-11-25T04:24:30.003 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:30.003 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2023-11-25T04:24:30.003 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2023-11-25T04:24:30.003 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2023-11-25T04:24:30.004 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.004 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:30.004 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:24:30.004 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 102.969137687, 2023-11-25T04:24:30.004 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01072943, 2023-11-25T04:24:30.004 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:30.004 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:30.004 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:30.004 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.004 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:30.004 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:24:30.004 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:30.005 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.005 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.005 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:30.005 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:24:30.005 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:30.005 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.005 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.005 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:30.005 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031829+0000", 2023-11-25T04:24:30.005 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.666e-06 2023-11-25T04:24:30.005 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.005 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.006 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:30.006 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031839+0000", 2023-11-25T04:24:30.006 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9490000000000008e-06 2023-11-25T04:24:30.006 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.006 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.006 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:30.006 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031857+0000", 2023-11-25T04:24:30.006 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8093000000000001e-05 2023-11-25T04:24:30.006 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.006 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.006 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:30.007 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031882+0000", 2023-11-25T04:24:30.007 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4294000000000002e-05 2023-11-25T04:24:30.007 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.007 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.007 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:30.007 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033130+0000", 2023-11-25T04:24:30.007 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001248264 2023-11-25T04:24:30.007 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.007 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.007 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:30.007 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033426+0000", 2023-11-25T04:24:30.007 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029620999999999998 2023-11-25T04:24:30.008 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.008 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.008 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:30.008 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033814+0000", 2023-11-25T04:24:30.008 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000387742 2023-11-25T04:24:30.008 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.008 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.008 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:30.008 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036793+0000", 2023-11-25T04:24:30.008 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002979598 2023-11-25T04:24:30.008 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.009 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.009 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:30.009 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042455+0000", 2023-11-25T04:24:30.009 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0056619260000000003 2023-11-25T04:24:30.009 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.009 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.010 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:30.010 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042502+0000", 2023-11-25T04:24:30.010 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7139999999999999e-05 2023-11-25T04:24:30.010 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.010 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.010 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:30.010 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042557+0000", 2023-11-25T04:24:30.010 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4548e-05 2023-11-25T04:24:30.011 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:30.011 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:30.011 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:30.011 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.011 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.011 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:30.011 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:24:30.011 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 102.967939152, 2023-11-25T04:24:30.011 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011505202000000001, 2023-11-25T04:24:30.011 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:30.011 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:30.011 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:30.012 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.012 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:30.012 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:24:30.012 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:30.012 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.012 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.012 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:30.012 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:24:30.012 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:30.012 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.012 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.013 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:30.013 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033027+0000", 2023-11-25T04:24:30.013 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0699999999999999e-06 2023-11-25T04:24:30.013 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.013 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.013 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:30.013 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033035+0000", 2023-11-25T04:24:30.013 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0830000000000004e-06 2023-11-25T04:24:30.013 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.013 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.013 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:30.013 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033059+0000", 2023-11-25T04:24:30.014 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3975e-05 2023-11-25T04:24:30.014 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.014 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.014 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:30.014 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033482+0000", 2023-11-25T04:24:30.014 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042253700000000001 2023-11-25T04:24:30.014 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.014 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.014 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:30.014 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034520+0000", 2023-11-25T04:24:30.015 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001037991 2023-11-25T04:24:30.015 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.015 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.015 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:30.015 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034611+0000", 2023-11-25T04:24:30.015 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1113999999999995e-05 2023-11-25T04:24:30.015 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.015 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.015 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:30.016 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034769+0000", 2023-11-25T04:24:30.016 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000157641 2023-11-25T04:24:30.016 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.016 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.016 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:30.016 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038292+0000", 2023-11-25T04:24:30.016 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035231160000000002 2023-11-25T04:24:30.016 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.016 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.016 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:30.016 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044440+0000", 2023-11-25T04:24:30.017 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0061486980000000002 2023-11-25T04:24:30.017 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.017 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.017 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:30.017 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044495+0000", 2023-11-25T04:24:30.017 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4339000000000003e-05 2023-11-25T04:24:30.017 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.017 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.017 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:30.017 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044531+0000", 2023-11-25T04:24:30.017 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6637999999999997e-05 2023-11-25T04:24:30.017 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:30.018 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:30.018 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:30.018 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.018 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.018 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:26 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:30.018 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:30.018 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 102.96735039000001, 2023-11-25T04:24:30.018 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011086945000000001, 2023-11-25T04:24:30.018 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:30.018 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:30.018 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:30.018 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.019 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:30.019 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:30.019 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:30.019 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.019 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.019 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:30.019 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:30.019 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:30.020 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.020 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.020 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:30.020 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:30.020 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1699999999999998e-07 2023-11-25T04:24:30.020 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.020 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.020 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:30.020 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033627+0000", 2023-11-25T04:24:30.020 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.163e-05 2023-11-25T04:24:30.020 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.021 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.021 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:30.021 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033636+0000", 2023-11-25T04:24:30.021 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6859999999999995e-06 2023-11-25T04:24:30.021 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.021 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.021 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:30.021 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033642+0000", 2023-11-25T04:24:30.021 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3149999999999997e-06 2023-11-25T04:24:30.021 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.021 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.022 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:30.022 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035414+0000", 2023-11-25T04:24:30.022 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0017716380000000001 2023-11-25T04:24:30.022 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.022 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.022 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:30.022 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035539+0000", 2023-11-25T04:24:30.022 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000125477 2023-11-25T04:24:30.022 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.022 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.022 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:30.022 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035672+0000", 2023-11-25T04:24:30.023 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000133316 2023-11-25T04:24:30.023 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.023 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.023 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:30.023 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038649+0000", 2023-11-25T04:24:30.023 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029764679999999999 2023-11-25T04:24:30.023 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.023 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.023 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:30.023 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044578+0000", 2023-11-25T04:24:30.023 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0059293510000000002 2023-11-25T04:24:30.023 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.024 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.024 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:30.024 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044651+0000", 2023-11-25T04:24:30.024 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2513000000000002e-05 2023-11-25T04:24:30.024 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.024 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.024 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:30.024 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044702+0000", 2023-11-25T04:24:30.024 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1034e-05 2023-11-25T04:24:30.024 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:30.024 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:30.024 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:30.025 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.025 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.025 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:29 3.b 3:d0630e4c:::mds0_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:30.025 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:30.025 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 102.96731247, 2023-11-25T04:24:30.025 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010980227, 2023-11-25T04:24:30.025 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:30.025 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:30.025 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:30.025 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.025 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:30.026 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:30.026 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:30.026 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.026 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.026 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:30.026 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:30.026 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:30.026 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.026 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.026 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:30.026 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:30.027 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3499999999999996e-07 2023-11-25T04:24:30.027 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.027 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.027 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:30.027 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033658+0000", 2023-11-25T04:24:30.027 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1280000000000001e-06 2023-11-25T04:24:30.027 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.027 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.027 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:30.027 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033667+0000", 2023-11-25T04:24:30.027 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1430000000000004e-06 2023-11-25T04:24:30.027 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.027 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.028 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:30.028 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033678+0000", 2023-11-25T04:24:30.028 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0906e-05 2023-11-25T04:24:30.028 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.028 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.028 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:30.028 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034456+0000", 2023-11-25T04:24:30.028 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00077833899999999996 2023-11-25T04:24:30.028 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.028 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.028 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:30.028 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034575+0000", 2023-11-25T04:24:30.029 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011926000000000001 2023-11-25T04:24:30.029 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.029 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.029 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:30.029 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035001+0000", 2023-11-25T04:24:30.029 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042550999999999998 2023-11-25T04:24:30.029 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.029 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.029 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:30.029 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.037923+0000", 2023-11-25T04:24:30.029 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029219670000000001 2023-11-25T04:24:30.029 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.029 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.030 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:30.031 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044538+0000", 2023-11-25T04:24:30.031 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0066152349999999997 2023-11-25T04:24:30.031 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.031 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.031 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:30.031 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044594+0000", 2023-11-25T04:24:30.031 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5593000000000001e-05 2023-11-25T04:24:30.031 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.031 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.032 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:30.032 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044633+0000", 2023-11-25T04:24:30.032 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9511000000000002e-05 2023-11-25T04:24:30.032 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:30.032 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:30.032 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:30.032 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.032 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.032 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:34 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:30.032 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:24:30.032 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 102.967062357, 2023-11-25T04:24:30.032 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010920445000000001, 2023-11-25T04:24:30.032 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:30.033 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:30.033 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:30.033 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.033 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:30.033 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:24:30.033 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:30.033 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.033 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.033 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:30.033 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:24:30.033 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:30.033 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.034 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.034 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:30.034 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033904+0000", 2023-11-25T04:24:30.034 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6599999999999994e-07 2023-11-25T04:24:30.034 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.034 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.034 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:30.034 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033909+0000", 2023-11-25T04:24:30.034 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5639999999999996e-06 2023-11-25T04:24:30.034 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.034 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.034 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:30.035 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033923+0000", 2023-11-25T04:24:30.035 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3028e-05 2023-11-25T04:24:30.035 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.035 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.035 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:30.035 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034043+0000", 2023-11-25T04:24:30.035 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000120328 2023-11-25T04:24:30.035 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.035 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.035 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:30.035 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035576+0000", 2023-11-25T04:24:30.035 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015333720000000001 2023-11-25T04:24:30.036 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.036 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.036 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:30.036 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035699+0000", 2023-11-25T04:24:30.036 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000123001 2023-11-25T04:24:30.036 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.036 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.036 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:30.036 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044823+0000", 2023-11-25T04:24:30.036 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0091241859999999994 2023-11-25T04:24:30.037 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:30.037 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:30.037 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:30.037 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.037 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.037 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:118 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals,omap-rm-keys] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e44)", 2023-11-25T04:24:30.037 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:17.112648+0000", 2023-11-25T04:24:30.037 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 12.888317016, 2023-11-25T04:24:30.037 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011250779000000001, 2023-11-25T04:24:30.037 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:30.037 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:30.038 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:30.038 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.038 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:30.038 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.112648+0000", 2023-11-25T04:24:30.038 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:30.038 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.038 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.038 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:30.038 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.112648+0000", 2023-11-25T04:24:30.038 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:30.038 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.038 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.039 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:30.039 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.112649+0000", 2023-11-25T04:24:30.039 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0070000000000001e-06 2023-11-25T04:24:30.039 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.039 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.039 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:30.039 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113182+0000", 2023-11-25T04:24:30.039 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00053268100000000002 2023-11-25T04:24:30.039 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.039 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.039 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:30.039 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113196+0000", 2023-11-25T04:24:30.040 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3742e-05 2023-11-25T04:24:30.040 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.040 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.040 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:30.040 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113398+0000", 2023-11-25T04:24:30.040 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000202726 2023-11-25T04:24:30.040 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.041 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.041 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:30.041 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113447+0000", 2023-11-25T04:24:30.041 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8655000000000001e-05 2023-11-25T04:24:30.041 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.041 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.041 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:30.041 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113529+0000", 2023-11-25T04:24:30.041 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1891999999999995e-05 2023-11-25T04:24:30.041 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.041 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.042 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:30.042 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123819+0000", 2023-11-25T04:24:30.042 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010290192 2023-11-25T04:24:30.042 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.042 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.042 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:30.042 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123852+0000", 2023-11-25T04:24:30.042 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2425999999999998e-05 2023-11-25T04:24:30.042 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.042 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.042 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:30.042 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123899+0000", 2023-11-25T04:24:30.043 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7457999999999999e-05 2023-11-25T04:24:30.043 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:30.043 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:30.043 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:30.043 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.043 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.043 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:154 3.1e 3:7bf7e7b5:::200.00000006:head [write 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:30.043 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:21.886430+0000", 2023-11-25T04:24:30.043 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 8.1145357499999999, 2023-11-25T04:24:30.043 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.017800107999999999, 2023-11-25T04:24:30.043 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:30.043 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:30.043 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:30.044 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.044 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:30.044 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.886430+0000", 2023-11-25T04:24:30.044 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:30.044 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.044 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.044 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:30.044 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.886430+0000", 2023-11-25T04:24:30.044 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:30.044 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.044 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.045 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:30.045 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.886431+0000", 2023-11-25T04:24:30.045 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.178e-06 2023-11-25T04:24:30.045 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.045 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.045 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:30.045 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.892277+0000", 2023-11-25T04:24:30.045 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0058467409999999999 2023-11-25T04:24:30.045 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.045 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.045 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:30.046 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.892288+0000", 2023-11-25T04:24:30.046 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0183000000000001e-05 2023-11-25T04:24:30.046 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.046 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.046 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:30.046 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.892292+0000", 2023-11-25T04:24:30.046 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7829999999999998e-06 2023-11-25T04:24:30.046 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.046 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.046 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:30.046 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.892362+0000", 2023-11-25T04:24:30.046 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0034000000000001e-05 2023-11-25T04:24:30.047 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.047 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.047 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:30.047 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.892432+0000", 2023-11-25T04:24:30.047 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9648999999999997e-05 2023-11-25T04:24:30.047 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.047 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.047 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:30.047 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.892550+0000", 2023-11-25T04:24:30.047 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011825 2023-11-25T04:24:30.047 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.047 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.047 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:30.048 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.899106+0000", 2023-11-25T04:24:30.048 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0065556759999999999 2023-11-25T04:24:30.048 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.048 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.048 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:30.048 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.904145+0000", 2023-11-25T04:24:30.048 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0050393490000000003 2023-11-25T04:24:30.048 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.048 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.048 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:30.048 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.904200+0000", 2023-11-25T04:24:30.048 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4234999999999998e-05 2023-11-25T04:24:30.048 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.049 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.049 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:30.049 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.904230+0000", 2023-11-25T04:24:30.049 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0029999999999999e-05 2023-11-25T04:24:30.049 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:30.049 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:30.049 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:30.049 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.049 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.049 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:576 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals,omap-rm-keys] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:30.049 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:22.111407+0000", 2023-11-25T04:24:30.049 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 7.8895581440000004, 2023-11-25T04:24:30.050 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.018987317, 2023-11-25T04:24:30.050 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:30.050 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:30.050 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:30.050 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.050 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:30.050 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111407+0000", 2023-11-25T04:24:30.050 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:30.050 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.050 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.050 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:30.050 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111407+0000", 2023-11-25T04:24:30.050 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:30.051 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.051 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.051 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:30.051 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111408+0000", 2023-11-25T04:24:30.051 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8000000000000004e-07 2023-11-25T04:24:30.051 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.051 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.051 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:30.051 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111659+0000", 2023-11-25T04:24:30.051 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000250537 2023-11-25T04:24:30.051 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.051 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.051 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:30.052 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111666+0000", 2023-11-25T04:24:30.052 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.973e-06 2023-11-25T04:24:30.052 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.052 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.052 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:30.052 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111670+0000", 2023-11-25T04:24:30.052 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.668e-06 2023-11-25T04:24:30.052 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.052 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.052 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:30.052 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111709+0000", 2023-11-25T04:24:30.052 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8800000000000001e-05 2023-11-25T04:24:30.052 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.053 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.053 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:30.053 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111761+0000", 2023-11-25T04:24:30.053 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1495000000000001e-05 2023-11-25T04:24:30.053 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.053 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.053 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:30.053 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.130255+0000", 2023-11-25T04:24:30.053 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.018494345999999998 2023-11-25T04:24:30.053 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.053 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.053 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:30.054 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.130311+0000", 2023-11-25T04:24:30.054 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5902000000000001e-05 2023-11-25T04:24:30.054 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.054 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.054 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:30.054 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.130394+0000", 2023-11-25T04:24:30.054 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3715999999999996e-05 2023-11-25T04:24:30.054 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:30.054 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:30.054 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:30.054 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.054 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.055 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:1067 4.21 4:869a7600:::100000011fa.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:30.055 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:22.155038+0000", 2023-11-25T04:24:30.055 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 7.8459268059999996, 2023-11-25T04:24:30.055 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011313148, 2023-11-25T04:24:30.055 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:30.055 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:30.055 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:30.055 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.055 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:30.055 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155038+0000", 2023-11-25T04:24:30.055 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:30.055 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.055 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.056 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:30.056 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155038+0000", 2023-11-25T04:24:30.056 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:30.056 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.056 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.056 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:30.056 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155039+0000", 2023-11-25T04:24:30.056 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0200000000000002e-07 2023-11-25T04:24:30.056 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.056 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.056 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:30.056 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155049+0000", 2023-11-25T04:24:30.056 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0295000000000001e-05 2023-11-25T04:24:30.056 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.057 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.057 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:30.057 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155058+0000", 2023-11-25T04:24:30.057 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.208e-06 2023-11-25T04:24:30.057 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.057 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.057 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:30.057 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155064+0000", 2023-11-25T04:24:30.057 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4489999999999996e-06 2023-11-25T04:24:30.057 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.057 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.057 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:30.058 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155104+0000", 2023-11-25T04:24:30.058 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0164999999999999e-05 2023-11-25T04:24:30.058 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.058 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.058 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:30.058 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155147+0000", 2023-11-25T04:24:30.058 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2576e-05 2023-11-25T04:24:30.058 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.058 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.058 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:30.058 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155212+0000", 2023-11-25T04:24:30.058 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5516000000000001e-05 2023-11-25T04:24:30.058 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.059 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.059 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:30.059 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.156025+0000", 2023-11-25T04:24:30.059 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00081248599999999996 2023-11-25T04:24:30.059 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.059 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.059 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:30.059 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.166250+0000", 2023-11-25T04:24:30.059 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010224736999999999 2023-11-25T04:24:30.059 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.059 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.059 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:30.060 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.166319+0000", 2023-11-25T04:24:30.060 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9145000000000003e-05 2023-11-25T04:24:30.060 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.060 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.060 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:30.060 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.166352+0000", 2023-11-25T04:24:30.060 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2969000000000002e-05 2023-11-25T04:24:30.060 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:30.060 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:30.060 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:30.060 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.060 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.061 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:1102 4.2f 4:f543b94c:::1000000121d.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:30.061 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:22.158246+0000", 2023-11-25T04:24:30.061 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 7.8427192830000001, 2023-11-25T04:24:30.061 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010094396, 2023-11-25T04:24:30.061 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:30.061 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:30.061 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:30.061 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.061 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:30.061 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158246+0000", 2023-11-25T04:24:30.061 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:30.061 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.061 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.062 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:30.062 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158246+0000", 2023-11-25T04:24:30.062 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:30.062 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.062 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.062 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:30.062 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158246+0000", 2023-11-25T04:24:30.062 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5299999999999999e-07 2023-11-25T04:24:30.062 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.062 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.062 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:30.062 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158249+0000", 2023-11-25T04:24:30.063 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2840000000000001e-06 2023-11-25T04:24:30.063 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.063 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.063 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:30.063 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158254+0000", 2023-11-25T04:24:30.063 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3449999999999996e-06 2023-11-25T04:24:30.063 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.063 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.063 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:30.063 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158258+0000", 2023-11-25T04:24:30.063 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5030000000000002e-06 2023-11-25T04:24:30.063 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.063 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.064 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:30.064 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158302+0000", 2023-11-25T04:24:30.064 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3908e-05 2023-11-25T04:24:30.064 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.064 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.064 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:30.064 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158398+0000", 2023-11-25T04:24:30.064 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6898000000000006e-05 2023-11-25T04:24:30.064 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.064 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.064 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:30.064 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158480+0000", 2023-11-25T04:24:30.064 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1578999999999999e-05 2023-11-25T04:24:30.065 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.065 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.065 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:30.065 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.159229+0000", 2023-11-25T04:24:30.065 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000749049 2023-11-25T04:24:30.065 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.065 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.065 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:30.065 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.168228+0000", 2023-11-25T04:24:30.065 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0089992930000000002 2023-11-25T04:24:30.065 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.065 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.065 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:30.066 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.168310+0000", 2023-11-25T04:24:30.066 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1348999999999997e-05 2023-11-25T04:24:30.066 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.066 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.066 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:30.066 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.168340+0000", 2023-11-25T04:24:30.066 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0735000000000003e-05 2023-11-25T04:24:30.066 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:30.066 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:30.066 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:30.066 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.066 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.067 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:1105 4.23 4:c6ff07b4:::10000001220.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:30.067 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:22.158714+0000", 2023-11-25T04:24:30.067 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 7.8422511689999999, 2023-11-25T04:24:30.067 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010412490999999999, 2023-11-25T04:24:30.067 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:30.067 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:30.067 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:30.067 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.067 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:30.067 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158714+0000", 2023-11-25T04:24:30.068 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:30.068 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.068 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.068 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:30.068 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158714+0000", 2023-11-25T04:24:30.068 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:30.068 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.068 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.068 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:30.068 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158715+0000", 2023-11-25T04:24:30.068 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4099999999999998e-07 2023-11-25T04:24:30.068 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.068 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.069 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:30.069 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158718+0000", 2023-11-25T04:24:30.069 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4319999999999999e-06 2023-11-25T04:24:30.069 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.069 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.069 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:30.069 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158729+0000", 2023-11-25T04:24:30.069 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.078e-05 2023-11-25T04:24:30.069 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.069 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.069 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:30.069 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158736+0000", 2023-11-25T04:24:30.069 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4649999999999997e-06 2023-11-25T04:24:30.070 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.070 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.070 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:30.070 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158779+0000", 2023-11-25T04:24:30.070 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3049000000000001e-05 2023-11-25T04:24:30.070 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.070 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.070 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:30.070 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158865+0000", 2023-11-25T04:24:30.070 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5138999999999999e-05 2023-11-25T04:24:30.070 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.070 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.071 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:30.071 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158983+0000", 2023-11-25T04:24:30.071 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000118658 2023-11-25T04:24:30.071 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.071 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.071 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:30.071 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.159645+0000", 2023-11-25T04:24:30.071 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00066193999999999997 2023-11-25T04:24:30.071 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.071 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.071 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:30.071 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.169046+0000", 2023-11-25T04:24:30.071 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0094005400000000006 2023-11-25T04:24:30.072 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.072 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.072 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:30.072 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.169092+0000", 2023-11-25T04:24:30.072 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5840000000000002e-05 2023-11-25T04:24:30.072 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.072 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.072 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:30.072 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.169127+0000", 2023-11-25T04:24:30.072 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5006999999999999e-05 2023-11-25T04:24:30.072 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:30.072 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:30.072 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:30.073 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.073 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.073 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2841 4.4 4:22111d42:::100000010df.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:30.073 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.092102+0000", 2023-11-25T04:24:30.073 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.9088636939999999, 2023-11-25T04:24:30.073 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.016449783999999999, 2023-11-25T04:24:30.073 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:30.073 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:30.073 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:30.073 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.074 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:30.074 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092102+0000", 2023-11-25T04:24:30.074 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:30.074 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.074 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.074 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:30.074 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092102+0000", 2023-11-25T04:24:30.074 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:30.074 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.074 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.074 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:30.074 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092102+0000", 2023-11-25T04:24:30.075 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3700000000000001e-07 2023-11-25T04:24:30.075 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.075 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.075 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:30.075 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092104+0000", 2023-11-25T04:24:30.075 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0679999999999999e-06 2023-11-25T04:24:30.075 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.075 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.075 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:30.075 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092111+0000", 2023-11-25T04:24:30.075 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0589999999999997e-06 2023-11-25T04:24:30.075 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.075 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.076 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:30.076 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092130+0000", 2023-11-25T04:24:30.076 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8686999999999999e-05 2023-11-25T04:24:30.076 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.076 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.076 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:30.076 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092443+0000", 2023-11-25T04:24:30.076 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000313519 2023-11-25T04:24:30.076 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.076 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.076 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:30.076 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092553+0000", 2023-11-25T04:24:30.076 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010932100000000001 2023-11-25T04:24:30.077 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.077 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.077 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:30.077 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092639+0000", 2023-11-25T04:24:30.077 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6094999999999995e-05 2023-11-25T04:24:30.077 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.077 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.077 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:30.077 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094679+0000", 2023-11-25T04:24:30.077 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0020402580000000001 2023-11-25T04:24:30.077 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.077 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.077 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:30.078 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108462+0000", 2023-11-25T04:24:30.078 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013783358000000001 2023-11-25T04:24:30.078 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.078 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.078 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:30.078 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108516+0000", 2023-11-25T04:24:30.078 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3483000000000003e-05 2023-11-25T04:24:30.078 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.078 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.078 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:30.078 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108551+0000", 2023-11-25T04:24:30.078 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5499e-05 2023-11-25T04:24:30.078 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:30.079 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:30.079 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:30.079 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.079 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.079 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2852 4.2d 4:b745ac76:::100000010f5.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:30.079 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.093477+0000", 2023-11-25T04:24:30.079 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.907488582, 2023-11-25T04:24:30.079 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.015423084, 2023-11-25T04:24:30.079 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:30.079 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:30.079 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:30.079 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.080 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:30.080 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093477+0000", 2023-11-25T04:24:30.080 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:30.080 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.080 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.080 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:30.080 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093477+0000", 2023-11-25T04:24:30.080 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:30.080 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.080 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.080 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:30.080 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093477+0000", 2023-11-25T04:24:30.080 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2500000000000005e-07 2023-11-25T04:24:30.081 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.081 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.081 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:30.081 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093480+0000", 2023-11-25T04:24:30.081 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0189999999999998e-06 2023-11-25T04:24:30.081 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.081 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.081 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:30.081 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093642+0000", 2023-11-25T04:24:30.081 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000162072 2023-11-25T04:24:30.081 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.081 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.082 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:30.082 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093649+0000", 2023-11-25T04:24:30.082 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9659999999999999e-06 2023-11-25T04:24:30.082 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.082 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.082 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:30.082 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093693+0000", 2023-11-25T04:24:30.082 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3794000000000001e-05 2023-11-25T04:24:30.082 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.082 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.082 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:30.082 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093764+0000", 2023-11-25T04:24:30.082 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0981000000000004e-05 2023-11-25T04:24:30.083 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.083 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.083 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:30.083 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093850+0000", 2023-11-25T04:24:30.083 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5536000000000002e-05 2023-11-25T04:24:30.083 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.083 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.083 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:30.083 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094710+0000", 2023-11-25T04:24:30.083 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00086070299999999999 2023-11-25T04:24:30.083 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.083 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.084 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:30.084 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108776+0000", 2023-11-25T04:24:30.084 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01406542 2023-11-25T04:24:30.084 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.084 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.084 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:30.084 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108829+0000", 2023-11-25T04:24:30.084 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2936999999999998e-05 2023-11-25T04:24:30.084 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.084 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.084 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:30.084 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108900+0000", 2023-11-25T04:24:30.084 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1031000000000005e-05 2023-11-25T04:24:30.085 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:30.085 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:30.085 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:30.085 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.085 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.085 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2854 4.2d 4:b5c259cf:::100000010f9.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:30.085 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.093674+0000", 2023-11-25T04:24:30.085 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.9072909219999996, 2023-11-25T04:24:30.085 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.015970380999999999, 2023-11-25T04:24:30.085 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:30.085 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:30.085 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:30.086 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.086 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:30.086 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093674+0000", 2023-11-25T04:24:30.086 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:30.086 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.086 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.086 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:30.086 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093674+0000", 2023-11-25T04:24:30.086 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:30.086 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.086 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.086 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:30.087 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093675+0000", 2023-11-25T04:24:30.087 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8400000000000004e-07 2023-11-25T04:24:30.087 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.087 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.087 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:30.087 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093678+0000", 2023-11-25T04:24:30.087 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2229999999999999e-06 2023-11-25T04:24:30.087 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.087 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.087 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:30.087 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093686+0000", 2023-11-25T04:24:30.087 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9990000000000008e-06 2023-11-25T04:24:30.088 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.088 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.088 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:30.088 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093696+0000", 2023-11-25T04:24:30.088 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6479999999999998e-06 2023-11-25T04:24:30.088 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.088 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.088 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:30.088 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094104+0000", 2023-11-25T04:24:30.088 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00040823900000000002 2023-11-25T04:24:30.088 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.088 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.089 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:30.089 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094172+0000", 2023-11-25T04:24:30.089 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8023999999999998e-05 2023-11-25T04:24:30.089 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.089 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.089 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:30.089 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094256+0000", 2023-11-25T04:24:30.089 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3418999999999995e-05 2023-11-25T04:24:30.089 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.089 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.089 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:30.089 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095160+0000", 2023-11-25T04:24:30.089 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00090418599999999997 2023-11-25T04:24:30.090 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.090 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.090 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:30.090 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109551+0000", 2023-11-25T04:24:30.090 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014391571000000001 2023-11-25T04:24:30.090 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.090 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.090 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:30.090 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109601+0000", 2023-11-25T04:24:30.090 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9478000000000003e-05 2023-11-25T04:24:30.090 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.090 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.090 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:30.091 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109645+0000", 2023-11-25T04:24:30.091 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.401e-05 2023-11-25T04:24:30.091 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:30.091 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:30.091 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:30.091 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.091 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.091 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2861 4.4 4:214027a9:::10000001107.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:30.091 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.093829+0000", 2023-11-25T04:24:30.091 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.9071365040000003, 2023-11-25T04:24:30.091 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014848917999999999, 2023-11-25T04:24:30.091 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:30.091 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:30.092 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:30.092 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.092 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:30.092 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093829+0000", 2023-11-25T04:24:30.092 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:30.092 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.092 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.092 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:30.092 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093829+0000", 2023-11-25T04:24:30.092 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:30.092 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.092 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.093 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:30.093 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093830+0000", 2023-11-25T04:24:30.093 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1999999999999998e-07 2023-11-25T04:24:30.093 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.093 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.093 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:30.093 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093834+0000", 2023-11-25T04:24:30.093 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.34e-06 2023-11-25T04:24:30.093 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.093 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.093 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:30.093 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093844+0000", 2023-11-25T04:24:30.094 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6690000000000001e-06 2023-11-25T04:24:30.094 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.094 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.094 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:30.094 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093857+0000", 2023-11-25T04:24:30.094 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3434999999999999e-05 2023-11-25T04:24:30.094 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.094 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.094 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:30.094 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093940+0000", 2023-11-25T04:24:30.094 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3047999999999995e-05 2023-11-25T04:24:30.094 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.095 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.095 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:30.095 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094052+0000", 2023-11-25T04:24:30.095 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000111538 2023-11-25T04:24:30.095 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.095 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.095 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:30.095 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094466+0000", 2023-11-25T04:24:30.095 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00041464600000000001 2023-11-25T04:24:30.095 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.095 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.095 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:30.096 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095570+0000", 2023-11-25T04:24:30.096 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001103981 2023-11-25T04:24:30.096 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.096 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.096 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:30.096 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108598+0000", 2023-11-25T04:24:30.096 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013027164000000001 2023-11-25T04:24:30.096 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.096 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.096 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:30.096 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108646+0000", 2023-11-25T04:24:30.096 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8090999999999997e-05 2023-11-25T04:24:30.096 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.097 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.097 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:30.097 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108678+0000", 2023-11-25T04:24:30.097 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2085999999999997e-05 2023-11-25T04:24:30.097 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:30.097 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:30.097 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:30.097 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.097 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.097 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2863 4.12 4:4b84c733:::1000000110b.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:30.097 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.093877+0000", 2023-11-25T04:24:30.097 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.9070881770000003, 2023-11-25T04:24:30.097 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014682755, 2023-11-25T04:24:30.097 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:30.098 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:30.098 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:30.098 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.098 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:30.098 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093877+0000", 2023-11-25T04:24:30.098 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:30.098 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.098 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.098 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:30.098 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093877+0000", 2023-11-25T04:24:30.098 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:30.098 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.099 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.099 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:30.099 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093878+0000", 2023-11-25T04:24:30.099 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4199999999999996e-07 2023-11-25T04:24:30.099 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.099 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.099 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:30.099 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093880+0000", 2023-11-25T04:24:30.099 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6189999999999998e-06 2023-11-25T04:24:30.099 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.099 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.099 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:30.099 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093888+0000", 2023-11-25T04:24:30.100 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5510000000000002e-06 2023-11-25T04:24:30.100 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.100 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.100 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:30.100 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093893+0000", 2023-11-25T04:24:30.100 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4179999999999996e-06 2023-11-25T04:24:30.100 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.100 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.100 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:30.100 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093934+0000", 2023-11-25T04:24:30.100 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0648000000000001e-05 2023-11-25T04:24:30.100 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.101 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.101 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:30.101 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094081+0000", 2023-11-25T04:24:30.101 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014753499999999999 2023-11-25T04:24:30.101 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.101 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.101 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:30.101 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094172+0000", 2023-11-25T04:24:30.101 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.0722000000000003e-05 2023-11-25T04:24:30.101 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.101 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.101 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:30.101 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094742+0000", 2023-11-25T04:24:30.102 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00056948600000000002 2023-11-25T04:24:30.102 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.102 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.102 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:30.102 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108491+0000", 2023-11-25T04:24:30.102 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013749568 2023-11-25T04:24:30.102 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.102 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.102 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:30.102 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108525+0000", 2023-11-25T04:24:30.102 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3683999999999998e-05 2023-11-25T04:24:30.102 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.103 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.103 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:30.103 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108560+0000", 2023-11-25T04:24:30.103 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4981999999999998e-05 2023-11-25T04:24:30.103 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:30.103 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:30.103 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:30.103 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.103 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.103 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2880 4.21 4:855834a4:::1000000112d.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:30.103 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.095098+0000", 2023-11-25T04:24:30.103 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.9058674829999998, 2023-11-25T04:24:30.103 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013847544, 2023-11-25T04:24:30.104 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:30.104 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:30.104 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:30.104 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.104 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:30.104 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095098+0000", 2023-11-25T04:24:30.104 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:30.104 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.104 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.104 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:30.104 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095098+0000", 2023-11-25T04:24:30.104 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:30.105 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.105 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.105 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:30.105 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095098+0000", 2023-11-25T04:24:30.105 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3000000000000002e-07 2023-11-25T04:24:30.105 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.105 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.105 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:30.105 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095099+0000", 2023-11-25T04:24:30.105 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.331e-06 2023-11-25T04:24:30.105 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.105 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.105 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:30.106 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095103+0000", 2023-11-25T04:24:30.106 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8319999999999999e-06 2023-11-25T04:24:30.106 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.106 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.106 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:30.106 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095108+0000", 2023-11-25T04:24:30.106 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4179999999999999e-06 2023-11-25T04:24:30.106 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.106 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.106 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:30.106 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095153+0000", 2023-11-25T04:24:30.106 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4799000000000003e-05 2023-11-25T04:24:30.106 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.107 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.107 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:30.107 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095268+0000", 2023-11-25T04:24:30.107 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000115437 2023-11-25T04:24:30.107 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.107 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.107 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:30.107 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095363+0000", 2023-11-25T04:24:30.107 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.5278000000000005e-05 2023-11-25T04:24:30.107 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.107 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.107 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:30.108 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095845+0000", 2023-11-25T04:24:30.108 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00048164600000000002 2023-11-25T04:24:30.108 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.108 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.108 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:30.108 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108854+0000", 2023-11-25T04:24:30.108 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013009289 2023-11-25T04:24:30.108 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.108 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.108 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:30.108 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108907+0000", 2023-11-25T04:24:30.108 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2528999999999998e-05 2023-11-25T04:24:30.109 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.109 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.109 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:30.109 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108945+0000", 2023-11-25T04:24:30.109 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8655000000000002e-05 2023-11-25T04:24:30.109 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:30.109 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:30.109 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:30.109 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.109 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.109 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2884 4.37 4:ef909cdc:::10000001135.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:30.109 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.095518+0000", 2023-11-25T04:24:30.110 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.9054475560000004, 2023-11-25T04:24:30.110 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014149988, 2023-11-25T04:24:30.110 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:30.110 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:30.110 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:30.110 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.110 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:30.110 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095518+0000", 2023-11-25T04:24:30.110 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:30.110 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.110 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.110 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:30.110 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095518+0000", 2023-11-25T04:24:30.111 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:30.111 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.111 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.111 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:30.111 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095518+0000", 2023-11-25T04:24:30.111 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2900000000000003e-07 2023-11-25T04:24:30.111 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.111 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.111 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:30.111 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095522+0000", 2023-11-25T04:24:30.111 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2119999999999999e-06 2023-11-25T04:24:30.111 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.112 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.112 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:30.112 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095532+0000", 2023-11-25T04:24:30.112 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0375e-05 2023-11-25T04:24:30.112 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.112 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.112 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:30.112 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095537+0000", 2023-11-25T04:24:30.112 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5909999999999998e-06 2023-11-25T04:24:30.112 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.113 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.113 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:30.113 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095584+0000", 2023-11-25T04:24:30.113 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7169999999999997e-05 2023-11-25T04:24:30.113 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.113 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.113 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:30.113 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095695+0000", 2023-11-25T04:24:30.113 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011078699999999999 2023-11-25T04:24:30.113 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.113 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.113 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:30.114 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095777+0000", 2023-11-25T04:24:30.114 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2089999999999995e-05 2023-11-25T04:24:30.114 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.114 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.114 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:30.114 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096286+0000", 2023-11-25T04:24:30.114 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000509247 2023-11-25T04:24:30.114 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.114 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.114 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:30.114 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109596+0000", 2023-11-25T04:24:30.114 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013309872 2023-11-25T04:24:30.115 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.115 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.115 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:30.115 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109638+0000", 2023-11-25T04:24:30.115 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2381e-05 2023-11-25T04:24:30.115 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.115 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.115 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:30.115 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109668+0000", 2023-11-25T04:24:30.115 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9634000000000001e-05 2023-11-25T04:24:30.115 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:30.115 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:30.115 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:30.116 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.116 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.116 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2890 4.12 4:4a2ad640:::10000001141.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:30.116 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.096274+0000", 2023-11-25T04:24:30.116 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.9046914929999996, 2023-11-25T04:24:30.116 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013647259, 2023-11-25T04:24:30.116 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:30.116 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:30.116 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:30.117 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.117 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:30.117 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096274+0000", 2023-11-25T04:24:30.117 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:30.117 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.117 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.117 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:30.117 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096274+0000", 2023-11-25T04:24:30.117 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:30.117 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.117 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.117 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:30.117 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096275+0000", 2023-11-25T04:24:30.118 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0299999999999998e-07 2023-11-25T04:24:30.118 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.118 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.118 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:30.118 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096279+0000", 2023-11-25T04:24:30.118 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9580000000000001e-06 2023-11-25T04:24:30.118 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.118 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.118 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:30.118 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096287+0000", 2023-11-25T04:24:30.118 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6929999999999996e-06 2023-11-25T04:24:30.118 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.119 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.119 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:30.119 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096294+0000", 2023-11-25T04:24:30.119 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.348e-06 2023-11-25T04:24:30.119 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.119 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.119 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:30.119 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096342+0000", 2023-11-25T04:24:30.119 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8068000000000001e-05 2023-11-25T04:24:30.119 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.119 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.119 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:30.119 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096462+0000", 2023-11-25T04:24:30.120 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000119918 2023-11-25T04:24:30.120 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.120 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.120 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:30.120 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096557+0000", 2023-11-25T04:24:30.120 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.5340999999999994e-05 2023-11-25T04:24:30.120 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.120 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.120 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:30.120 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096971+0000", 2023-11-25T04:24:30.120 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00041398100000000001 2023-11-25T04:24:30.120 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.120 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.121 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:30.121 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109849+0000", 2023-11-25T04:24:30.121 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.012877985999999999 2023-11-25T04:24:30.121 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.121 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.121 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:30.121 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109894+0000", 2023-11-25T04:24:30.121 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5595e-05 2023-11-25T04:24:30.121 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.121 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.121 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:30.121 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109921+0000", 2023-11-25T04:24:30.122 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6568e-05 2023-11-25T04:24:30.122 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:30.122 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:30.122 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:30.122 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.122 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.122 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2891 4.23 4:c4054dd3:::10000001143.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:30.122 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.096713+0000", 2023-11-25T04:24:30.122 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.904252799, 2023-11-25T04:24:30.122 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013148692, 2023-11-25T04:24:30.122 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:30.122 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:30.122 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:30.123 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.123 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:30.123 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096713+0000", 2023-11-25T04:24:30.123 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:30.123 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.123 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.123 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:30.123 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096713+0000", 2023-11-25T04:24:30.123 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:30.123 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.123 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.123 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:30.123 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096713+0000", 2023-11-25T04:24:30.124 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9299999999999998e-07 2023-11-25T04:24:30.124 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.124 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.124 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:30.124 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096716+0000", 2023-11-25T04:24:30.124 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.988e-06 2023-11-25T04:24:30.124 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.124 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.124 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:30.124 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096722+0000", 2023-11-25T04:24:30.124 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0499999999999997e-06 2023-11-25T04:24:30.124 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.124 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.125 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:30.126 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096727+0000", 2023-11-25T04:24:30.126 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3380000000000004e-06 2023-11-25T04:24:30.126 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.126 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.126 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:30.126 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096770+0000", 2023-11-25T04:24:30.126 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2938000000000001e-05 2023-11-25T04:24:30.126 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.126 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.126 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:30.126 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096904+0000", 2023-11-25T04:24:30.127 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000133759 2023-11-25T04:24:30.127 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.127 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.127 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:30.127 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096983+0000", 2023-11-25T04:24:30.127 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8559999999999993e-05 2023-11-25T04:24:30.127 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.127 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.127 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:30.127 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.097437+0000", 2023-11-25T04:24:30.127 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00045439200000000002 2023-11-25T04:24:30.128 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.129 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.129 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:30.129 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109737+0000", 2023-11-25T04:24:30.129 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.012300190000000001 2023-11-25T04:24:30.129 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.129 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.129 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:30.129 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109782+0000", 2023-11-25T04:24:30.129 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4546999999999999e-05 2023-11-25T04:24:30.129 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:30.130 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:30.130 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:30.130 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109861+0000", 2023-11-25T04:24:30.130 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9337000000000003e-05 2023-11-25T04:24:30.130 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:30.130 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:30.130 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:30.130 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:30.130 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:30.130 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:30.133 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:24:30.133 INFO:tasks.workunit.client.0.smithi067.stdout:checking cifs/ioctl.h presence... no 2023-11-25T04:24:30.133 INFO:tasks.workunit.client.0.smithi067.stdout:checking for cifs/ioctl.h... no 2023-11-25T04:24:30.134 INFO:tasks.workunit.client.0.smithi067.stdout:checking sys/mman.h usability... yes 2023-11-25T04:24:30.134 INFO:tasks.workunit.client.0.smithi067.stdout:checking sys/mman.h presence... yes 2023-11-25T04:24:30.134 INFO:tasks.workunit.client.0.smithi067.stdout:checking for sys/mman.h... yes 2023-11-25T04:24:30.134 INFO:tasks.workunit.client.0.smithi067.stdout:checking for xfs/xfs_log_format.h... no 2023-11-25T04:24:30.135 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:30.130+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:30.156 INFO:tasks.workunit.client.0.smithi067.stdout:checking for uuid_compare... no 2023-11-25T04:24:30.206 INFO:tasks.workunit.client.0.smithi067.stdout:checking for uuid_compare in -luuid... yes 2023-11-25T04:24:30.233 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:30.231+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:30.240 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:30.240 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:24:30.240 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2023-11-25T04:24:30.240 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:30.249 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:24:30.288 INFO:tasks.workunit.client.0.smithi067.stdout:checking for xfs/xfs.h... yes 2023-11-25T04:24:30.312 INFO:tasks.workunit.client.0.smithi067.stdout:checking for xfs/libxfs.h... no 2023-11-25T04:24:30.334 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:30.333+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:30.338 INFO:tasks.workunit.client.0.smithi067.stdout:checking whether xlog_assign_lsn is declared... no 2023-11-25T04:24:30.377 INFO:tasks.workunit.client.0.smithi067.stdout:checking for xfs/xqm.h... yes 2023-11-25T04:24:30.395 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:30.395 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:24:30.396 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2023-11-25T04:24:30.396 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2023-11-25T04:24:30.396 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:30.404 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:24:30.441 INFO:tasks.workunit.client.0.smithi067.stdout:checking xfsctl from xfs/xfs.h... ok 2023-11-25T04:24:30.445 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:30.444+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:30.475 INFO:tasks.workunit.client.0.smithi067.stdout:checking xfs/handle.h usability... yes 2023-11-25T04:24:30.488 INFO:tasks.workunit.client.0.smithi067.stdout:checking xfs/handle.h presence... yes 2023-11-25T04:24:30.488 INFO:tasks.workunit.client.0.smithi067.stdout:checking for xfs/handle.h... yes 2023-11-25T04:24:30.535 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:30.534+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:30.548 INFO:tasks.workunit.client.0.smithi067.stdout:checking for attr_list_by_handle in -lhandle... yes 2023-11-25T04:24:30.552 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:30.552 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2023-11-25T04:24:30.552 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2023-11-25T04:24:30.552 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2023-11-25T04:24:30.552 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.552 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:152 3.1a 3:5e41ab69:::200.00000005:head [write 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:30.553 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:21.237401+0000", 2023-11-25T04:24:30.553 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.3125372659999996, 2023-11-25T04:24:30.553 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015253802, 2023-11-25T04:24:30.553 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:30.553 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:30.553 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:30.553 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.553 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:30.553 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.237401+0000", 2023-11-25T04:24:30.553 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:30.553 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.554 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.554 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:30.554 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.237401+0000", 2023-11-25T04:24:30.554 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:30.554 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.554 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.554 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:30.554 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.237402+0000", 2023-11-25T04:24:30.554 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3699999999999999e-07 2023-11-25T04:24:30.554 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.555 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.555 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:30.555 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243182+0000", 2023-11-25T04:24:30.555 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057801270000000004 2023-11-25T04:24:30.555 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.555 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.555 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:30.555 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243193+0000", 2023-11-25T04:24:30.555 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1221e-05 2023-11-25T04:24:30.555 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.555 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.555 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:30.556 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243198+0000", 2023-11-25T04:24:30.556 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3739999999999997e-06 2023-11-25T04:24:30.556 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.556 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.556 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:30.556 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243246+0000", 2023-11-25T04:24:30.556 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7667999999999998e-05 2023-11-25T04:24:30.556 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.556 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.556 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:30.556 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243292+0000", 2023-11-25T04:24:30.557 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5794000000000002e-05 2023-11-25T04:24:30.557 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.557 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.557 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:30.557 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243360+0000", 2023-11-25T04:24:30.557 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8479000000000006e-05 2023-11-25T04:24:30.557 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.557 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.557 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:30.557 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.250667+0000", 2023-11-25T04:24:30.558 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.007306618 2023-11-25T04:24:30.558 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.558 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.558 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:30.558 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.252598+0000", 2023-11-25T04:24:30.558 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019315129999999999 2023-11-25T04:24:30.558 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.558 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.558 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:30.558 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.252631+0000", 2023-11-25T04:24:30.558 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2925000000000001e-05 2023-11-25T04:24:30.559 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.559 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.559 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:30.559 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.252655+0000", 2023-11-25T04:24:30.559 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3246000000000001e-05 2023-11-25T04:24:30.559 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:30.559 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:30.559 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:30.559 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.559 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.560 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:2841 4.4 e47/17 4:22111d42:::100000010df.00000000:head v 47'139, mlcod=47'137)", 2023-11-25T04:24:30.560 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.093610+0000", 2023-11-25T04:24:30.560 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.4563284029999997, 2023-11-25T04:24:30.560 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014758871999999999, 2023-11-25T04:24:30.560 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:30.560 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:30.560 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:30.560 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.560 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:30.560 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093610+0000", 2023-11-25T04:24:30.560 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:30.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.561 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:30.561 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093610+0000", 2023-11-25T04:24:30.561 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:30.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.561 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:30.561 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093610+0000", 2023-11-25T04:24:30.561 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1799999999999999e-07 2023-11-25T04:24:30.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.562 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:30.562 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093621+0000", 2023-11-25T04:24:30.562 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1107999999999999e-05 2023-11-25T04:24:30.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.562 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.562 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:30.562 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093627+0000", 2023-11-25T04:24:30.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0310000000000004e-06 2023-11-25T04:24:30.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.563 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:30.563 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093633+0000", 2023-11-25T04:24:30.563 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2700000000000001e-06 2023-11-25T04:24:30.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.563 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.563 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:30.563 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093722+0000", 2023-11-25T04:24:30.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8417000000000004e-05 2023-11-25T04:24:30.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:30.564 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093737+0000", 2023-11-25T04:24:30.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5781e-05 2023-11-25T04:24:30.564 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:30.564 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108203+0000", 2023-11-25T04:24:30.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014465844 2023-11-25T04:24:30.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.565 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:30.565 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108368+0000", 2023-11-25T04:24:30.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000165203 2023-11-25T04:24:30.565 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:30.565 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:30.565 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:30.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.565 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2860 4.3d 4:bf0d7877:::10000001105.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:30.566 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.093823+0000", 2023-11-25T04:24:30.566 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.4561152350000004, 2023-11-25T04:24:30.566 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015582848, 2023-11-25T04:24:30.566 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:30.566 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:30.566 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:30.566 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.566 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:30.566 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093823+0000", 2023-11-25T04:24:30.566 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:30.566 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.566 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:30.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093823+0000", 2023-11-25T04:24:30.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:30.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:30.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093823+0000", 2023-11-25T04:24:30.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9100000000000004e-07 2023-11-25T04:24:30.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:30.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093826+0000", 2023-11-25T04:24:30.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4499999999999998e-06 2023-11-25T04:24:30.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:30.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093835+0000", 2023-11-25T04:24:30.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8179999999999993e-06 2023-11-25T04:24:30.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:30.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093840+0000", 2023-11-25T04:24:30.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0819999999999998e-06 2023-11-25T04:24:30.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:30.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093973+0000", 2023-11-25T04:24:30.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013308999999999999 2023-11-25T04:24:30.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:30.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094277+0000", 2023-11-25T04:24:30.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00030403299999999999 2023-11-25T04:24:30.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.570 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:30.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108608+0000", 2023-11-25T04:24:30.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014330888 2023-11-25T04:24:30.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.570 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:30.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108676+0000", 2023-11-25T04:24:30.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8577000000000004e-05 2023-11-25T04:24:30.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:30.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109406+0000", 2023-11-25T04:24:30.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00072931899999999997 2023-11-25T04:24:30.571 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:30.571 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:30.571 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:30.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.571 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2862 4.13 4:cb2e645e:::10000001109.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:30.572 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.093852+0000", 2023-11-25T04:24:30.572 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.4560862740000005, 2023-11-25T04:24:30.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015190636, 2023-11-25T04:24:30.572 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:30.572 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:30.572 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:30.572 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.572 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:30.572 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093852+0000", 2023-11-25T04:24:30.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:30.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.573 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:30.573 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093852+0000", 2023-11-25T04:24:30.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:30.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.573 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:30.573 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093852+0000", 2023-11-25T04:24:30.573 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0299999999999999e-07 2023-11-25T04:24:30.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:30.574 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093854+0000", 2023-11-25T04:24:30.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.846e-06 2023-11-25T04:24:30.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:30.574 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093861+0000", 2023-11-25T04:24:30.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5520000000000001e-06 2023-11-25T04:24:30.574 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:30.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093865+0000", 2023-11-25T04:24:30.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7870000000000001e-06 2023-11-25T04:24:30.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:30.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093940+0000", 2023-11-25T04:24:30.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4799e-05 2023-11-25T04:24:30.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:30.576 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094052+0000", 2023-11-25T04:24:30.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011156500000000001 2023-11-25T04:24:30.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.576 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:30.576 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108928+0000", 2023-11-25T04:24:30.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014876245 2023-11-25T04:24:30.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.576 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:30.576 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108972+0000", 2023-11-25T04:24:30.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4477999999999997e-05 2023-11-25T04:24:30.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.577 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:30.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109042+0000", 2023-11-25T04:24:30.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9861000000000001e-05 2023-11-25T04:24:30.577 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:30.577 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:30.577 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:30.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.577 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2865 4.36 4:6ebc5ac6:::1000000110f.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:30.577 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.093906+0000", 2023-11-25T04:24:30.578 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.4560320009999996, 2023-11-25T04:24:30.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014962628, 2023-11-25T04:24:30.578 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:30.578 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:30.578 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:30.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.578 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:30.578 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093906+0000", 2023-11-25T04:24:30.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:30.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:30.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093906+0000", 2023-11-25T04:24:30.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:30.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:30.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093906+0000", 2023-11-25T04:24:30.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4400000000000001e-07 2023-11-25T04:24:30.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:30.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093908+0000", 2023-11-25T04:24:30.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8640000000000001e-06 2023-11-25T04:24:30.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:30.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093914+0000", 2023-11-25T04:24:30.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0170000000000002e-06 2023-11-25T04:24:30.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:30.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093921+0000", 2023-11-25T04:24:30.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6529999999999997e-06 2023-11-25T04:24:30.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.581 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:30.581 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094258+0000", 2023-11-25T04:24:30.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00033703599999999998 2023-11-25T04:24:30.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:30.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094340+0000", 2023-11-25T04:24:30.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2279000000000003e-05 2023-11-25T04:24:30.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:30.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094416+0000", 2023-11-25T04:24:30.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6248999999999995e-05 2023-11-25T04:24:30.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.583 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:30.583 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.096134+0000", 2023-11-25T04:24:30.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001717611 2023-11-25T04:24:30.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.583 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:30.583 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108809+0000", 2023-11-25T04:24:30.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012675327 2023-11-25T04:24:30.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.584 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:30.584 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108836+0000", 2023-11-25T04:24:30.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6509999999999999e-05 2023-11-25T04:24:30.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.584 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:30.584 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108869+0000", 2023-11-25T04:24:30.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2737999999999997e-05 2023-11-25T04:24:30.584 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:30.584 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:30.584 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:30.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.585 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:2852 4.2d e47/17 4:b745ac76:::100000010f5.00000000:head v 47'126, mlcod=47'124)", 2023-11-25T04:24:30.585 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.093987+0000", 2023-11-25T04:24:30.585 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.4559514399999998, 2023-11-25T04:24:30.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014613256, 2023-11-25T04:24:30.585 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:30.585 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:30.585 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:30.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:30.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093987+0000", 2023-11-25T04:24:30.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:30.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:30.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093987+0000", 2023-11-25T04:24:30.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:30.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:30.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093987+0000", 2023-11-25T04:24:30.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7000000000000001e-07 2023-11-25T04:24:30.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:30.587 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093990+0000", 2023-11-25T04:24:30.587 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8729999999999999e-06 2023-11-25T04:24:30.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.587 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:30.587 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093997+0000", 2023-11-25T04:24:30.587 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.973e-06 2023-11-25T04:24:30.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.587 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:30.588 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094001+0000", 2023-11-25T04:24:30.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7360000000000001e-06 2023-11-25T04:24:30.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.588 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:30.588 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094263+0000", 2023-11-25T04:24:30.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026173400000000001 2023-11-25T04:24:30.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.588 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:30.588 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094278+0000", 2023-11-25T04:24:30.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4681999999999999e-05 2023-11-25T04:24:30.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:30.589 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108563+0000", 2023-11-25T04:24:30.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014284738 2023-11-25T04:24:30.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:30.589 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108600+0000", 2023-11-25T04:24:30.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7249999999999997e-05 2023-11-25T04:24:30.589 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:30.590 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:30.590 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:30.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.590 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2866 4.36 4:6f6907fb:::10000001111.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:30.590 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.094033+0000", 2023-11-25T04:24:30.590 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.4559052750000001, 2023-11-25T04:24:30.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014885591, 2023-11-25T04:24:30.590 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:30.590 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:30.590 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:30.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:30.591 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094033+0000", 2023-11-25T04:24:30.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:30.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.591 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:30.591 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094033+0000", 2023-11-25T04:24:30.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:30.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.592 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:30.592 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094033+0000", 2023-11-25T04:24:30.592 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.06e-07 2023-11-25T04:24:30.592 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.592 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:30.593 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094036+0000", 2023-11-25T04:24:30.593 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.508e-06 2023-11-25T04:24:30.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:30.593 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094044+0000", 2023-11-25T04:24:30.593 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5909999999999996e-06 2023-11-25T04:24:30.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:30.593 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094051+0000", 2023-11-25T04:24:30.593 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0319999999999998e-06 2023-11-25T04:24:30.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.594 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:30.594 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094727+0000", 2023-11-25T04:24:30.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00067523300000000002 2023-11-25T04:24:30.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.594 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:30.594 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094795+0000", 2023-11-25T04:24:30.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8156000000000003e-05 2023-11-25T04:24:30.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.594 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:30.595 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094898+0000", 2023-11-25T04:24:30.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103329 2023-11-25T04:24:30.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:30.595 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108803+0000", 2023-11-25T04:24:30.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013904385 2023-11-25T04:24:30.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:30.595 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108875+0000", 2023-11-25T04:24:30.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1958999999999993e-05 2023-11-25T04:24:30.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.596 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.596 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:30.596 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108899+0000", 2023-11-25T04:24:30.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4019000000000002e-05 2023-11-25T04:24:30.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.596 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.596 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:30.596 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108918+0000", 2023-11-25T04:24:30.597 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9772999999999999e-05 2023-11-25T04:24:30.597 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:30.597 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:30.597 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:30.597 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.597 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.597 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:2854 4.2d e47/17 4:b5c259cf:::100000010f9.00000000:head v 47'127, mlcod=47'124)", 2023-11-25T04:24:30.597 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.094793+0000", 2023-11-25T04:24:30.597 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.4551449859999996, 2023-11-25T04:24:30.597 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014650764, 2023-11-25T04:24:30.598 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:30.598 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:30.598 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:30.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:30.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094793+0000", 2023-11-25T04:24:30.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:30.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:30.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094793+0000", 2023-11-25T04:24:30.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:30.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:30.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094793+0000", 2023-11-25T04:24:30.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3700000000000001e-07 2023-11-25T04:24:30.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:30.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094798+0000", 2023-11-25T04:24:30.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1200000000000004e-06 2023-11-25T04:24:30.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.600 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:30.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094805+0000", 2023-11-25T04:24:30.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9759999999999998e-06 2023-11-25T04:24:30.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.600 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:30.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094811+0000", 2023-11-25T04:24:30.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1410000000000002e-06 2023-11-25T04:24:30.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:30.601 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094838+0000", 2023-11-25T04:24:30.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6995000000000002e-05 2023-11-25T04:24:30.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:30.601 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094848+0000", 2023-11-25T04:24:30.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0115999999999999e-05 2023-11-25T04:24:30.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.601 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:30.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109408+0000", 2023-11-25T04:24:30.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014559825 2023-11-25T04:24:30.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:30.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109444+0000", 2023-11-25T04:24:30.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6154e-05 2023-11-25T04:24:30.602 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:30.602 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:30.602 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:30.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.602 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2869 4.6 4:63719fb0:::10000001117.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:30.603 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.094910+0000", 2023-11-25T04:24:30.603 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.4550280290000002, 2023-11-25T04:24:30.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014132570000000001, 2023-11-25T04:24:30.603 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:30.603 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:30.603 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:30.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:30.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094910+0000", 2023-11-25T04:24:30.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:30.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:30.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094910+0000", 2023-11-25T04:24:30.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:30.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:30.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094910+0000", 2023-11-25T04:24:30.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.58e-07 2023-11-25T04:24:30.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:30.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094913+0000", 2023-11-25T04:24:30.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.198e-06 2023-11-25T04:24:30.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.605 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:30.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094921+0000", 2023-11-25T04:24:30.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1629999999999996e-06 2023-11-25T04:24:30.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.605 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.605 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:30.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094926+0000", 2023-11-25T04:24:30.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7559999999999999e-06 2023-11-25T04:24:30.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:30.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094966+0000", 2023-11-25T04:24:30.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0151999999999998e-05 2023-11-25T04:24:30.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:30.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095073+0000", 2023-11-25T04:24:30.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107657 2023-11-25T04:24:30.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:30.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095158+0000", 2023-11-25T04:24:30.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5136999999999995e-05 2023-11-25T04:24:30.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:30.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108924+0000", 2023-11-25T04:24:30.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013765727 2023-11-25T04:24:30.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:30.608 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108958+0000", 2023-11-25T04:24:30.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3417000000000002e-05 2023-11-25T04:24:30.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:30.608 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109012+0000", 2023-11-25T04:24:30.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3939e-05 2023-11-25T04:24:30.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:30.609 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109043+0000", 2023-11-25T04:24:30.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0966000000000001e-05 2023-11-25T04:24:30.609 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:30.609 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:30.609 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:30.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.609 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2871 4.3f 4:fff10e11:::1000000111b.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:30.609 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.094936+0000", 2023-11-25T04:24:30.609 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.4550021910000002, 2023-11-25T04:24:30.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014261726000000001, 2023-11-25T04:24:30.610 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:30.610 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:30.610 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:30.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.610 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:30.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094936+0000", 2023-11-25T04:24:30.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:30.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.610 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:30.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094936+0000", 2023-11-25T04:24:30.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:30.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:30.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094936+0000", 2023-11-25T04:24:30.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9900000000000001e-07 2023-11-25T04:24:30.611 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.611 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:30.611 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094938+0000", 2023-11-25T04:24:30.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9599999999999999e-06 2023-11-25T04:24:30.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:30.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094944+0000", 2023-11-25T04:24:30.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2219999999999999e-06 2023-11-25T04:24:30.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:30.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094950+0000", 2023-11-25T04:24:30.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1329999999999998e-06 2023-11-25T04:24:30.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.613 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:30.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094993+0000", 2023-11-25T04:24:30.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3305e-05 2023-11-25T04:24:30.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.613 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:30.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095107+0000", 2023-11-25T04:24:30.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011449400000000001 2023-11-25T04:24:30.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.613 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:30.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095176+0000", 2023-11-25T04:24:30.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8330000000000005e-05 2023-11-25T04:24:30.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:30.614 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107939+0000", 2023-11-25T04:24:30.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012763801 2023-11-25T04:24:30.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:30.614 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109128+0000", 2023-11-25T04:24:30.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011884770000000001 2023-11-25T04:24:30.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:30.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109166+0000", 2023-11-25T04:24:30.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7948999999999999e-05 2023-11-25T04:24:30.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:30.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109198+0000", 2023-11-25T04:24:30.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1656000000000003e-05 2023-11-25T04:24:30.615 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:30.615 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:30.615 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:30.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.616 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2872 4.32 4:4fb002bc:::1000000111d.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:30.616 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.094961+0000", 2023-11-25T04:24:30.616 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.4549774869999998, 2023-11-25T04:24:30.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014631564999999999, 2023-11-25T04:24:30.616 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:30.616 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:30.617 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:30.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.617 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:30.617 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094961+0000", 2023-11-25T04:24:30.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:30.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.617 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:30.617 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094961+0000", 2023-11-25T04:24:30.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:30.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.617 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:30.618 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094961+0000", 2023-11-25T04:24:30.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9299999999999999e-07 2023-11-25T04:24:30.618 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.618 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:30.618 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094963+0000", 2023-11-25T04:24:30.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9209999999999999e-06 2023-11-25T04:24:30.618 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.618 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:30.618 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094968+0000", 2023-11-25T04:24:30.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.502e-06 2023-11-25T04:24:30.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:30.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094973+0000", 2023-11-25T04:24:30.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8559999999999998e-06 2023-11-25T04:24:30.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:30.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107722+0000", 2023-11-25T04:24:30.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012748534000000001 2023-11-25T04:24:30.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:30.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107844+0000", 2023-11-25T04:24:30.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012233400000000001 2023-11-25T04:24:30.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.620 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:30.620 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109469+0000", 2023-11-25T04:24:30.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001625175 2023-11-25T04:24:30.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:30.621 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109518+0000", 2023-11-25T04:24:30.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8980000000000002e-05 2023-11-25T04:24:30.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:30.621 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109592+0000", 2023-11-25T04:24:30.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3869999999999996e-05 2023-11-25T04:24:30.621 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:30.621 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:30.621 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:30.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.622 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2873 4.13 4:cae4f303:::1000000111f.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:30.622 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.094991+0000", 2023-11-25T04:24:30.622 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.454946895, 2023-11-25T04:24:30.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015101692, 2023-11-25T04:24:30.622 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:30.622 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:30.622 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:30.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.623 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:30.623 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094991+0000", 2023-11-25T04:24:30.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:30.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.623 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:30.623 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094991+0000", 2023-11-25T04:24:30.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:30.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.623 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:30.623 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094992+0000", 2023-11-25T04:24:30.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7e-07 2023-11-25T04:24:30.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:30.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094994+0000", 2023-11-25T04:24:30.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2110000000000001e-06 2023-11-25T04:24:30.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:30.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095000+0000", 2023-11-25T04:24:30.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.241e-06 2023-11-25T04:24:30.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:30.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095006+0000", 2023-11-25T04:24:30.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3659999999999997e-06 2023-11-25T04:24:30.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:30.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108369+0000", 2023-11-25T04:24:30.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01336247 2023-11-25T04:24:30.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:30.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108492+0000", 2023-11-25T04:24:30.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000123192 2023-11-25T04:24:30.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:30.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109975+0000", 2023-11-25T04:24:30.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001482894 2023-11-25T04:24:30.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:30.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110024+0000", 2023-11-25T04:24:30.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9097000000000001e-05 2023-11-25T04:24:30.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.627 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:30.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110093+0000", 2023-11-25T04:24:30.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8751000000000006e-05 2023-11-25T04:24:30.627 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:30.627 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:30.627 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:30.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.627 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2874 4.14 4:2b557d2e:::10000001121.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:30.627 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095019+0000", 2023-11-25T04:24:30.627 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.4549193689999997, 2023-11-25T04:24:30.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014998296, 2023-11-25T04:24:30.628 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:30.628 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:30.628 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:30.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:30.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095019+0000", 2023-11-25T04:24:30.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:30.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:30.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095019+0000", 2023-11-25T04:24:30.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:30.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:30.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095019+0000", 2023-11-25T04:24:30.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7300000000000001e-07 2023-11-25T04:24:30.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.629 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:30.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095021+0000", 2023-11-25T04:24:30.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0099999999999998e-06 2023-11-25T04:24:30.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:30.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095028+0000", 2023-11-25T04:24:30.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7870000000000004e-06 2023-11-25T04:24:30.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:30.630 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095033+0000", 2023-11-25T04:24:30.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2039999999999996e-06 2023-11-25T04:24:30.630 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.631 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:30.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107740+0000", 2023-11-25T04:24:30.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012706693 2023-11-25T04:24:30.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:30.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107817+0000", 2023-11-25T04:24:30.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7599e-05 2023-11-25T04:24:30.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:30.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107961+0000", 2023-11-25T04:24:30.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000143194 2023-11-25T04:24:30.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:30.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109364+0000", 2023-11-25T04:24:30.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001403226 2023-11-25T04:24:30.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:30.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109507+0000", 2023-11-25T04:24:30.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000142991 2023-11-25T04:24:30.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:30.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109986+0000", 2023-11-25T04:24:30.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00047906899999999999 2023-11-25T04:24:30.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:30.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110017+0000", 2023-11-25T04:24:30.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1050000000000003e-05 2023-11-25T04:24:30.634 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:30.634 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:30.634 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:30.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.634 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.635 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2875 4.d 4:b17d5dc6:::10000001123.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:30.635 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095049+0000", 2023-11-25T04:24:30.635 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.4548894450000001, 2023-11-25T04:24:30.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014882886, 2023-11-25T04:24:30.635 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:30.635 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:30.635 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:30.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:30.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095049+0000", 2023-11-25T04:24:30.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:30.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:30.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095049+0000", 2023-11-25T04:24:30.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:30.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:30.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095049+0000", 2023-11-25T04:24:30.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6100000000000001e-07 2023-11-25T04:24:30.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:30.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095051+0000", 2023-11-25T04:24:30.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1449999999999998e-06 2023-11-25T04:24:30.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:30.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095058+0000", 2023-11-25T04:24:30.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.601e-06 2023-11-25T04:24:30.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:30.637 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095064+0000", 2023-11-25T04:24:30.637 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1079999999999998e-06 2023-11-25T04:24:30.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:30.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108015+0000", 2023-11-25T04:24:30.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012951582 2023-11-25T04:24:30.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.638 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:30.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108260+0000", 2023-11-25T04:24:30.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024445199999999998 2023-11-25T04:24:30.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:30.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109825+0000", 2023-11-25T04:24:30.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0015651759999999999 2023-11-25T04:24:30.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:30.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109866+0000", 2023-11-25T04:24:30.639 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0629e-05 2023-11-25T04:24:30.639 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:30.640 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109931+0000", 2023-11-25T04:24:30.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5732e-05 2023-11-25T04:24:30.640 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:30.640 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:30.640 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:30.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.640 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2876 4.3b 4:dc871776:::10000001125.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:30.640 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095077+0000", 2023-11-25T04:24:30.640 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.4548608239999998, 2023-11-25T04:24:30.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014500944, 2023-11-25T04:24:30.640 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:30.641 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:30.641 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:30.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:30.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095077+0000", 2023-11-25T04:24:30.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:30.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.641 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:30.641 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095077+0000", 2023-11-25T04:24:30.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:30.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:30.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095078+0000", 2023-11-25T04:24:30.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9799999999999999e-07 2023-11-25T04:24:30.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:30.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095079+0000", 2023-11-25T04:24:30.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.877e-06 2023-11-25T04:24:30.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.644 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:30.644 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095086+0000", 2023-11-25T04:24:30.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2439999999999998e-06 2023-11-25T04:24:30.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.644 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:30.644 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095094+0000", 2023-11-25T04:24:30.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9710000000000004e-06 2023-11-25T04:24:30.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.644 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:30.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107732+0000", 2023-11-25T04:24:30.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012638104000000001 2023-11-25T04:24:30.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:30.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107854+0000", 2023-11-25T04:24:30.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012261600000000001 2023-11-25T04:24:30.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:30.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107941+0000", 2023-11-25T04:24:30.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6935999999999996e-05 2023-11-25T04:24:30.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:30.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108871+0000", 2023-11-25T04:24:30.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00092955499999999999 2023-11-25T04:24:30.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:30.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109483+0000", 2023-11-25T04:24:30.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00061206100000000001 2023-11-25T04:24:30.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.647 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:30.647 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109527+0000", 2023-11-25T04:24:30.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4112e-05 2023-11-25T04:24:30.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.647 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:30.647 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109578+0000", 2023-11-25T04:24:30.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1069999999999997e-05 2023-11-25T04:24:30.647 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:30.647 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:30.647 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:30.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.648 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2878 4.3e 4:7ff98b9d:::10000001129.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:30.648 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095101+0000", 2023-11-25T04:24:30.649 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.4548370410000002, 2023-11-25T04:24:30.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015015137, 2023-11-25T04:24:30.649 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:30.650 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:30.650 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:30.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.650 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:30.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095101+0000", 2023-11-25T04:24:30.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:30.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:30.651 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095101+0000", 2023-11-25T04:24:30.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:30.651 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.652 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.652 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:30.652 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095101+0000", 2023-11-25T04:24:30.652 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7399999999999999e-07 2023-11-25T04:24:30.652 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.652 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.652 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:30.652 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095103+0000", 2023-11-25T04:24:30.653 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7600000000000001e-06 2023-11-25T04:24:30.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.653 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:30.653 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095109+0000", 2023-11-25T04:24:30.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8189999999999997e-06 2023-11-25T04:24:30.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:30.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095114+0000", 2023-11-25T04:24:30.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6770000000000003e-06 2023-11-25T04:24:30.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:30.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107724+0000", 2023-11-25T04:24:30.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012610909999999999 2023-11-25T04:24:30.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:30.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108483+0000", 2023-11-25T04:24:30.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000758407 2023-11-25T04:24:30.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:30.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108568+0000", 2023-11-25T04:24:30.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4707000000000001e-05 2023-11-25T04:24:30.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.656 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:30.656 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109687+0000", 2023-11-25T04:24:30.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011192750000000001 2023-11-25T04:24:30.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.656 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:30.656 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110049+0000", 2023-11-25T04:24:30.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00036210600000000001 2023-11-25T04:24:30.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.657 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:30.657 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110086+0000", 2023-11-25T04:24:30.657 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7500999999999999e-05 2023-11-25T04:24:30.657 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.657 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.657 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:30.657 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110116+0000", 2023-11-25T04:24:30.657 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9601e-05 2023-11-25T04:24:30.657 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:30.657 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:30.657 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:30.657 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.657 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.658 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2879 4.2a 4:5524ace2:::1000000112b.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:30.658 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095120+0000", 2023-11-25T04:24:30.658 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.4548182609999998, 2023-11-25T04:24:30.658 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014587937, 2023-11-25T04:24:30.658 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:30.658 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:30.658 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:30.658 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.658 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:30.658 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095120+0000", 2023-11-25T04:24:30.658 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:30.658 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.659 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.659 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:30.659 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095120+0000", 2023-11-25T04:24:30.659 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:30.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.659 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.659 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:30.659 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095120+0000", 2023-11-25T04:24:30.659 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5600000000000001e-07 2023-11-25T04:24:30.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.659 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.659 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:30.659 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095122+0000", 2023-11-25T04:24:30.660 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.694e-06 2023-11-25T04:24:30.660 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.660 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.660 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:30.660 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095127+0000", 2023-11-25T04:24:30.660 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5099999999999998e-06 2023-11-25T04:24:30.660 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.660 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.660 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:30.660 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095132+0000", 2023-11-25T04:24:30.660 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6210000000000003e-06 2023-11-25T04:24:30.660 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.661 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.661 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:30.661 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107729+0000", 2023-11-25T04:24:30.661 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012596931 2023-11-25T04:24:30.661 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.661 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.661 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:30.661 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107856+0000", 2023-11-25T04:24:30.661 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012707699999999999 2023-11-25T04:24:30.661 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.661 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.661 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:30.661 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109600+0000", 2023-11-25T04:24:30.662 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001744298 2023-11-25T04:24:30.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.662 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.662 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:30.662 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109641+0000", 2023-11-25T04:24:30.662 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0769000000000002e-05 2023-11-25T04:24:30.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.662 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.662 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:30.662 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109708+0000", 2023-11-25T04:24:30.662 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6680999999999994e-05 2023-11-25T04:24:30.662 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:30.663 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:30.663 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:30.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.663 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2882 4.1c 4:3a7c2e07:::10000001131.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:30.663 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095141+0000", 2023-11-25T04:24:30.663 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.4547968860000005, 2023-11-25T04:24:30.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014949469999999999, 2023-11-25T04:24:30.663 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:30.663 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:30.663 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:30.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:30.664 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095141+0000", 2023-11-25T04:24:30.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:30.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:30.664 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095141+0000", 2023-11-25T04:24:30.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:30.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:30.664 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095142+0000", 2023-11-25T04:24:30.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1899999999999998e-07 2023-11-25T04:24:30.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.665 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.665 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:30.665 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095143+0000", 2023-11-25T04:24:30.665 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7770000000000001e-06 2023-11-25T04:24:30.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.665 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.665 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:30.665 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095149+0000", 2023-11-25T04:24:30.665 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9909999999999999e-06 2023-11-25T04:24:30.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.665 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.665 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:30.666 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095154+0000", 2023-11-25T04:24:30.666 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3880000000000002e-06 2023-11-25T04:24:30.666 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.666 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.666 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:30.666 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107736+0000", 2023-11-25T04:24:30.666 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012582477999999999 2023-11-25T04:24:30.666 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.666 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.666 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:30.666 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107868+0000", 2023-11-25T04:24:30.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013134400000000001 2023-11-25T04:24:30.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.667 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:30.667 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107959+0000", 2023-11-25T04:24:30.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1253000000000002e-05 2023-11-25T04:24:30.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.668 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:30.668 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109325+0000", 2023-11-25T04:24:30.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0013662520000000001 2023-11-25T04:24:30.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.668 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:30.668 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110023+0000", 2023-11-25T04:24:30.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00069838799999999998 2023-11-25T04:24:30.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.669 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:30.669 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110061+0000", 2023-11-25T04:24:30.669 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7867999999999997e-05 2023-11-25T04:24:30.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.669 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:30.669 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110091+0000", 2023-11-25T04:24:30.669 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9312e-05 2023-11-25T04:24:30.669 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:30.669 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:30.669 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:30.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.670 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2883 4.3e 4:7d82be99:::10000001133.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:30.670 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095942+0000", 2023-11-25T04:24:30.670 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.4539955999999998, 2023-11-25T04:24:30.670 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014290573000000001, 2023-11-25T04:24:30.670 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:30.670 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:30.670 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:30.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.670 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:30.670 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095942+0000", 2023-11-25T04:24:30.670 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:30.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:30.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095942+0000", 2023-11-25T04:24:30.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:30.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:30.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095944+0000", 2023-11-25T04:24:30.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.632e-06 2023-11-25T04:24:30.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:30.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095947+0000", 2023-11-25T04:24:30.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7760000000000002e-06 2023-11-25T04:24:30.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.672 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:30.672 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095954+0000", 2023-11-25T04:24:30.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5379999999999997e-06 2023-11-25T04:24:30.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.672 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:30.672 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095960+0000", 2023-11-25T04:24:30.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9209999999999997e-06 2023-11-25T04:24:30.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.673 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:30.673 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108809+0000", 2023-11-25T04:24:30.673 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012848972 2023-11-25T04:24:30.673 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.673 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.673 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:30.673 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108921+0000", 2023-11-25T04:24:30.673 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000111767 2023-11-25T04:24:30.673 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.673 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.673 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:30.673 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109008+0000", 2023-11-25T04:24:30.673 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6576999999999995e-05 2023-11-25T04:24:30.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.674 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.674 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:30.674 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110124+0000", 2023-11-25T04:24:30.674 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011163709999999999 2023-11-25T04:24:30.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.674 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.674 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:30.674 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110155+0000", 2023-11-25T04:24:30.674 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1248999999999998e-05 2023-11-25T04:24:30.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.674 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.674 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:30.675 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110202+0000", 2023-11-25T04:24:30.675 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7097e-05 2023-11-25T04:24:30.675 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.675 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.675 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:30.675 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110233+0000", 2023-11-25T04:24:30.675 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0673000000000003e-05 2023-11-25T04:24:30.675 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:30.675 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:30.675 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:30.675 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.675 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.675 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2886 4.25 4:a579baeb:::10000001139.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:30.676 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095994+0000", 2023-11-25T04:24:30.676 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 5.4539442720000002, 2023-11-25T04:24:30.676 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013825356, 2023-11-25T04:24:30.676 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:30.676 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:30.676 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:30.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.676 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:30.676 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095994+0000", 2023-11-25T04:24:30.676 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:30.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.677 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:30.677 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095994+0000", 2023-11-25T04:24:30.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:30.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.677 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:30.677 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095996+0000", 2023-11-25T04:24:30.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8780000000000001e-06 2023-11-25T04:24:30.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.677 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:30.677 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095998+0000", 2023-11-25T04:24:30.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1560000000000002e-06 2023-11-25T04:24:30.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.678 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.678 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:30.678 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.096003+0000", 2023-11-25T04:24:30.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1080000000000001e-06 2023-11-25T04:24:30.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.678 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.678 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:30.678 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.096010+0000", 2023-11-25T04:24:30.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0650000000000001e-06 2023-11-25T04:24:30.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.678 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.679 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:30.679 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107742+0000", 2023-11-25T04:24:30.679 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011731691000000001 2023-11-25T04:24:30.679 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.679 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.679 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:30.679 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108063+0000", 2023-11-25T04:24:30.679 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00032168600000000001 2023-11-25T04:24:30.679 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.679 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.679 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:30.679 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109689+0000", 2023-11-25T04:24:30.679 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001625556 2023-11-25T04:24:30.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.680 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.680 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:30.680 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109740+0000", 2023-11-25T04:24:30.680 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1125000000000003e-05 2023-11-25T04:24:30.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:30.680 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:30.680 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:30.680 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109819+0000", 2023-11-25T04:24:30.680 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9091000000000006e-05 2023-11-25T04:24:30.680 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:30.680 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:30.680 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:30.681 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:30.681 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:30.681 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:30.683 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:24:30.684 INFO:tasks.workunit.client.0.smithi067.stdout:checking attr/attributes.h usability... yes 2023-11-25T04:24:30.684 INFO:tasks.workunit.client.0.smithi067.stdout:checking attr/attributes.h presence... yes 2023-11-25T04:24:30.684 INFO:tasks.workunit.client.0.smithi067.stdout:checking for attr/attributes.h... yes 2023-11-25T04:24:30.684 INFO:tasks.workunit.client.0.smithi067.stdout:checking sys/attributes.h usability... no 2023-11-25T04:24:30.684 INFO:tasks.workunit.client.0.smithi067.stdout:checking sys/attributes.h presence... no 2023-11-25T04:24:30.684 INFO:tasks.workunit.client.0.smithi067.stdout:checking for sys/attributes.h... no 2023-11-25T04:24:30.685 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:30.680+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:30.727 INFO:tasks.workunit.client.0.smithi067.stdout:checking for attr_list in -lattr... yes 2023-11-25T04:24:30.775 INFO:tasks.workunit.client.0.smithi067.stdout:checking for attr_set in -lattr... yes 2023-11-25T04:24:30.782 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:30.780+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:30.793 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:30.793 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:24:30.794 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2023-11-25T04:24:30.794 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:30.802 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:24:30.831 INFO:tasks.workunit.client.0.smithi067.stdout:checking sys/acl.h usability... yes 2023-11-25T04:24:30.846 INFO:tasks.workunit.client.0.smithi067.stdout:checking sys/acl.h presence... yes 2023-11-25T04:24:30.846 INFO:tasks.workunit.client.0.smithi067.stdout:checking for sys/acl.h... yes 2023-11-25T04:24:30.886 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:30.885+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:30.889 INFO:tasks.workunit.client.0.smithi067.stdout:checking acl/libacl.h usability... yes 2023-11-25T04:24:30.904 INFO:tasks.workunit.client.0.smithi067.stdout:checking acl/libacl.h presence... yes 2023-11-25T04:24:30.904 INFO:tasks.workunit.client.0.smithi067.stdout:checking for acl/libacl.h... yes 2023-11-25T04:24:30.950 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:30.950 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:24:30.950 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2023-11-25T04:24:30.950 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2023-11-25T04:24:30.950 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:30.953 INFO:tasks.workunit.client.0.smithi067.stdout:checking for acl_init in -lacl... yes 2023-11-25T04:24:30.959 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:24:30.989 INFO:tasks.workunit.client.0.smithi067.stdout:checking gdbm-ndbm.h usability... no 2023-11-25T04:24:31.001 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:30.999+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:31.010 INFO:tasks.workunit.client.0.smithi067.stdout:checking gdbm-ndbm.h presence... no 2023-11-25T04:24:31.010 INFO:tasks.workunit.client.0.smithi067.stdout:checking for gdbm-ndbm.h... no 2023-11-25T04:24:31.043 INFO:tasks.workunit.client.0.smithi067.stdout:checking gdbm/ndbm.h usability... no 2023-11-25T04:24:31.064 INFO:tasks.workunit.client.0.smithi067.stdout:checking gdbm/ndbm.h presence... no 2023-11-25T04:24:31.064 INFO:tasks.workunit.client.0.smithi067.stdout:checking for gdbm/ndbm.h... no 2023-11-25T04:24:31.085 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~82s 2023-11-25T04:24:31.085 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:31.084+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:31.098 INFO:tasks.workunit.client.0.smithi067.stdout:checking ndbm.h usability... no 2023-11-25T04:24:31.113 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:31.113 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2023-11-25T04:24:31.113 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2023-11-25T04:24:31.113 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2023-11-25T04:24:31.113 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.113 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:31.113 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:24:31.113 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 104.078622106, 2023-11-25T04:24:31.114 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01072943, 2023-11-25T04:24:31.114 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:31.114 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:31.114 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:31.114 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.114 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:31.114 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:24:31.114 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:31.114 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.114 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.114 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:31.114 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:24:31.115 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:31.115 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.115 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.115 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:31.115 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031829+0000", 2023-11-25T04:24:31.115 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.666e-06 2023-11-25T04:24:31.115 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.115 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.115 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:31.115 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031839+0000", 2023-11-25T04:24:31.115 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9490000000000008e-06 2023-11-25T04:24:31.116 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.116 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.116 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:31.116 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031857+0000", 2023-11-25T04:24:31.116 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8093000000000001e-05 2023-11-25T04:24:31.116 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.116 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.116 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:31.116 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031882+0000", 2023-11-25T04:24:31.116 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4294000000000002e-05 2023-11-25T04:24:31.117 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.117 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.117 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:31.117 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033130+0000", 2023-11-25T04:24:31.117 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001248264 2023-11-25T04:24:31.117 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.117 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.118 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:31.118 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033426+0000", 2023-11-25T04:24:31.118 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029620999999999998 2023-11-25T04:24:31.118 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.118 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.118 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:31.118 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033814+0000", 2023-11-25T04:24:31.118 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000387742 2023-11-25T04:24:31.118 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.118 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.118 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:31.118 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036793+0000", 2023-11-25T04:24:31.119 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002979598 2023-11-25T04:24:31.119 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.119 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.119 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:31.119 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042455+0000", 2023-11-25T04:24:31.119 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0056619260000000003 2023-11-25T04:24:31.119 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.119 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.119 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:31.119 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042502+0000", 2023-11-25T04:24:31.119 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7139999999999999e-05 2023-11-25T04:24:31.119 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.120 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.120 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:31.120 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042557+0000", 2023-11-25T04:24:31.120 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4548e-05 2023-11-25T04:24:31.120 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:31.120 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:31.120 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:31.120 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.120 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.120 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:31.120 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:24:31.120 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 104.077423571, 2023-11-25T04:24:31.121 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011505202000000001, 2023-11-25T04:24:31.121 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:31.121 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:31.121 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:31.121 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.121 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:31.121 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:24:31.121 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:31.121 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.121 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.121 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:31.122 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:24:31.122 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:31.122 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.122 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.122 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:31.122 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033027+0000", 2023-11-25T04:24:31.122 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0699999999999999e-06 2023-11-25T04:24:31.122 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.122 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.122 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:31.122 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033035+0000", 2023-11-25T04:24:31.122 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0830000000000004e-06 2023-11-25T04:24:31.123 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.123 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.123 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:31.123 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033059+0000", 2023-11-25T04:24:31.123 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3975e-05 2023-11-25T04:24:31.123 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.123 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.123 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:31.123 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033482+0000", 2023-11-25T04:24:31.123 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042253700000000001 2023-11-25T04:24:31.123 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.123 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.123 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:31.124 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034520+0000", 2023-11-25T04:24:31.124 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001037991 2023-11-25T04:24:31.124 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.124 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.124 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:31.124 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034611+0000", 2023-11-25T04:24:31.124 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1113999999999995e-05 2023-11-25T04:24:31.124 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.124 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.124 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:31.124 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034769+0000", 2023-11-25T04:24:31.124 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000157641 2023-11-25T04:24:31.125 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.125 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.125 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:31.125 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038292+0000", 2023-11-25T04:24:31.125 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035231160000000002 2023-11-25T04:24:31.125 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.125 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.125 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:31.125 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044440+0000", 2023-11-25T04:24:31.125 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0061486980000000002 2023-11-25T04:24:31.125 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.125 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.125 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:31.126 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044495+0000", 2023-11-25T04:24:31.126 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4339000000000003e-05 2023-11-25T04:24:31.126 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.127 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.127 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:31.127 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044531+0000", 2023-11-25T04:24:31.127 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6637999999999997e-05 2023-11-25T04:24:31.127 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:31.127 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:31.127 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:31.127 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.127 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.127 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:26 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:31.127 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:31.128 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 104.076834809, 2023-11-25T04:24:31.128 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011086945000000001, 2023-11-25T04:24:31.128 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:31.128 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:31.128 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:31.128 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.128 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:31.128 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:31.128 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:31.128 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.128 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.128 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:31.128 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:31.129 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:31.129 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.129 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.129 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:31.129 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:31.129 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1699999999999998e-07 2023-11-25T04:24:31.129 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.129 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.129 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:31.129 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033627+0000", 2023-11-25T04:24:31.129 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.163e-05 2023-11-25T04:24:31.129 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.129 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.130 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:31.130 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033636+0000", 2023-11-25T04:24:31.130 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6859999999999995e-06 2023-11-25T04:24:31.130 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.130 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.130 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:31.130 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033642+0000", 2023-11-25T04:24:31.130 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3149999999999997e-06 2023-11-25T04:24:31.130 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.130 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.130 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:31.131 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035414+0000", 2023-11-25T04:24:31.131 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0017716380000000001 2023-11-25T04:24:31.131 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.131 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.131 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:31.131 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035539+0000", 2023-11-25T04:24:31.131 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000125477 2023-11-25T04:24:31.131 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.131 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.131 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:31.131 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035672+0000", 2023-11-25T04:24:31.131 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000133316 2023-11-25T04:24:31.132 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.132 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.132 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:31.132 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038649+0000", 2023-11-25T04:24:31.132 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029764679999999999 2023-11-25T04:24:31.132 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.132 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.132 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:31.132 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044578+0000", 2023-11-25T04:24:31.132 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0059293510000000002 2023-11-25T04:24:31.132 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.132 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.133 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:31.133 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044651+0000", 2023-11-25T04:24:31.133 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2513000000000002e-05 2023-11-25T04:24:31.133 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.133 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.133 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:31.133 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044702+0000", 2023-11-25T04:24:31.133 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1034e-05 2023-11-25T04:24:31.133 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:31.133 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:31.133 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:31.133 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.134 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.134 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:29 3.b 3:d0630e4c:::mds0_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:31.134 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:31.134 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 104.07679688899999, 2023-11-25T04:24:31.134 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010980227, 2023-11-25T04:24:31.134 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:31.134 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:31.134 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:31.134 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.134 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:31.134 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:31.134 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:31.135 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.135 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.135 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:31.135 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:31.135 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:31.135 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.135 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.135 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:31.135 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:31.135 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3499999999999996e-07 2023-11-25T04:24:31.135 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.135 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.135 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:31.136 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033658+0000", 2023-11-25T04:24:31.136 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1280000000000001e-06 2023-11-25T04:24:31.136 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.136 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.136 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:31.136 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033667+0000", 2023-11-25T04:24:31.136 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1430000000000004e-06 2023-11-25T04:24:31.136 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.136 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.136 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:31.136 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033678+0000", 2023-11-25T04:24:31.137 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0906e-05 2023-11-25T04:24:31.137 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.137 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.137 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:31.137 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034456+0000", 2023-11-25T04:24:31.137 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00077833899999999996 2023-11-25T04:24:31.137 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.137 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.137 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:31.137 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034575+0000", 2023-11-25T04:24:31.137 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011926000000000001 2023-11-25T04:24:31.137 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.138 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.138 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:31.138 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035001+0000", 2023-11-25T04:24:31.138 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042550999999999998 2023-11-25T04:24:31.138 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.138 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.138 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:31.138 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.037923+0000", 2023-11-25T04:24:31.138 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029219670000000001 2023-11-25T04:24:31.138 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.138 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.139 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:31.139 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044538+0000", 2023-11-25T04:24:31.139 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0066152349999999997 2023-11-25T04:24:31.139 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.139 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.139 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:31.139 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044594+0000", 2023-11-25T04:24:31.139 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5593000000000001e-05 2023-11-25T04:24:31.139 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.139 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.139 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:31.139 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044633+0000", 2023-11-25T04:24:31.139 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9511000000000002e-05 2023-11-25T04:24:31.140 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:31.140 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:31.140 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:31.140 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.140 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.140 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:34 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:31.140 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:24:31.140 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 104.076546776, 2023-11-25T04:24:31.140 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010920445000000001, 2023-11-25T04:24:31.140 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:31.140 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:31.140 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:31.141 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.141 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:31.141 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:24:31.141 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:31.141 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.141 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.141 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:31.141 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:24:31.141 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:31.141 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.141 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.142 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:31.142 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033904+0000", 2023-11-25T04:24:31.142 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6599999999999994e-07 2023-11-25T04:24:31.142 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.142 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.142 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:31.142 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033909+0000", 2023-11-25T04:24:31.142 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5639999999999996e-06 2023-11-25T04:24:31.142 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.142 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.143 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:31.143 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033923+0000", 2023-11-25T04:24:31.143 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3028e-05 2023-11-25T04:24:31.143 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.143 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.143 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:31.143 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034043+0000", 2023-11-25T04:24:31.143 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000120328 2023-11-25T04:24:31.143 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.143 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.143 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:31.143 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035576+0000", 2023-11-25T04:24:31.144 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015333720000000001 2023-11-25T04:24:31.144 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.144 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.144 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:31.144 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035699+0000", 2023-11-25T04:24:31.144 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000123001 2023-11-25T04:24:31.144 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.144 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.144 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:31.144 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044823+0000", 2023-11-25T04:24:31.144 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0091241859999999994 2023-11-25T04:24:31.144 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:31.145 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:31.145 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:31.145 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.145 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.145 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:118 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals,omap-rm-keys] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e44)", 2023-11-25T04:24:31.145 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:17.112648+0000", 2023-11-25T04:24:31.145 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 13.997801435, 2023-11-25T04:24:31.145 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011250779000000001, 2023-11-25T04:24:31.145 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:31.145 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:31.145 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:31.146 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.146 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:31.146 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.112648+0000", 2023-11-25T04:24:31.146 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:31.146 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.146 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.146 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:31.146 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.112648+0000", 2023-11-25T04:24:31.146 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:31.146 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.146 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.147 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:31.147 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.112649+0000", 2023-11-25T04:24:31.147 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0070000000000001e-06 2023-11-25T04:24:31.147 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.147 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.147 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:31.147 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113182+0000", 2023-11-25T04:24:31.147 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00053268100000000002 2023-11-25T04:24:31.147 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.147 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.147 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:31.148 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113196+0000", 2023-11-25T04:24:31.148 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3742e-05 2023-11-25T04:24:31.148 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.148 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.148 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:31.148 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113398+0000", 2023-11-25T04:24:31.148 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000202726 2023-11-25T04:24:31.148 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.148 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.148 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:31.148 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113447+0000", 2023-11-25T04:24:31.149 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8655000000000001e-05 2023-11-25T04:24:31.149 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.149 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.149 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:31.149 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113529+0000", 2023-11-25T04:24:31.149 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1891999999999995e-05 2023-11-25T04:24:31.149 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.149 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.149 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:31.149 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123819+0000", 2023-11-25T04:24:31.149 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010290192 2023-11-25T04:24:31.150 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.150 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.150 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:31.150 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123852+0000", 2023-11-25T04:24:31.150 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2425999999999998e-05 2023-11-25T04:24:31.150 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.150 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.150 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:31.150 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123899+0000", 2023-11-25T04:24:31.150 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7457999999999999e-05 2023-11-25T04:24:31.151 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:31.151 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:31.151 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:31.151 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.151 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.151 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:154 3.1e 3:7bf7e7b5:::200.00000006:head [write 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:31.151 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:21.886430+0000", 2023-11-25T04:24:31.151 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 9.2240201689999992, 2023-11-25T04:24:31.151 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.017800107999999999, 2023-11-25T04:24:31.151 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:31.151 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:31.152 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:31.152 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.152 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:31.152 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.886430+0000", 2023-11-25T04:24:31.152 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:31.152 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.152 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.152 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:31.152 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.886430+0000", 2023-11-25T04:24:31.152 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:31.152 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.152 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.153 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:31.153 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.886431+0000", 2023-11-25T04:24:31.153 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.178e-06 2023-11-25T04:24:31.153 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.153 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.153 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:31.153 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.892277+0000", 2023-11-25T04:24:31.153 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0058467409999999999 2023-11-25T04:24:31.153 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.153 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.153 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:31.153 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.892288+0000", 2023-11-25T04:24:31.154 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0183000000000001e-05 2023-11-25T04:24:31.154 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.154 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.154 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:31.154 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.892292+0000", 2023-11-25T04:24:31.154 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7829999999999998e-06 2023-11-25T04:24:31.154 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.154 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.154 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:31.154 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.892362+0000", 2023-11-25T04:24:31.154 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0034000000000001e-05 2023-11-25T04:24:31.155 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.155 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.155 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:31.155 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.892432+0000", 2023-11-25T04:24:31.155 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9648999999999997e-05 2023-11-25T04:24:31.155 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.155 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.155 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:31.155 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.892550+0000", 2023-11-25T04:24:31.155 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011825 2023-11-25T04:24:31.155 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.156 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.156 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:31.156 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.899106+0000", 2023-11-25T04:24:31.156 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0065556759999999999 2023-11-25T04:24:31.156 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.156 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.156 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:31.156 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.904145+0000", 2023-11-25T04:24:31.156 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0050393490000000003 2023-11-25T04:24:31.156 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.157 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.157 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:31.157 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.904200+0000", 2023-11-25T04:24:31.157 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4234999999999998e-05 2023-11-25T04:24:31.157 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.157 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.157 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:31.157 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.904230+0000", 2023-11-25T04:24:31.157 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0029999999999999e-05 2023-11-25T04:24:31.157 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:31.157 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:31.157 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:31.158 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.158 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.158 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:576 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals,omap-rm-keys] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:31.158 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:22.111407+0000", 2023-11-25T04:24:31.158 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 8.9990425629999997, 2023-11-25T04:24:31.158 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.018987317, 2023-11-25T04:24:31.158 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:31.158 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:31.158 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:31.158 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.158 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:31.159 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111407+0000", 2023-11-25T04:24:31.159 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:31.159 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.159 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.159 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:31.159 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111407+0000", 2023-11-25T04:24:31.159 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:31.159 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.159 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.159 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:31.159 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111408+0000", 2023-11-25T04:24:31.159 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8000000000000004e-07 2023-11-25T04:24:31.160 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.160 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.160 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:31.160 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111659+0000", 2023-11-25T04:24:31.160 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000250537 2023-11-25T04:24:31.160 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.160 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.160 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:31.160 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111666+0000", 2023-11-25T04:24:31.160 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.973e-06 2023-11-25T04:24:31.160 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.160 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.160 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:31.161 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111670+0000", 2023-11-25T04:24:31.161 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.668e-06 2023-11-25T04:24:31.161 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.161 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.161 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:31.161 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111709+0000", 2023-11-25T04:24:31.161 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8800000000000001e-05 2023-11-25T04:24:31.161 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.161 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.161 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:31.161 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111761+0000", 2023-11-25T04:24:31.161 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1495000000000001e-05 2023-11-25T04:24:31.162 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.162 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.162 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:31.162 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.130255+0000", 2023-11-25T04:24:31.162 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.018494345999999998 2023-11-25T04:24:31.162 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.162 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.162 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:31.162 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.130311+0000", 2023-11-25T04:24:31.162 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5902000000000001e-05 2023-11-25T04:24:31.162 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.163 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.163 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:31.163 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.130394+0000", 2023-11-25T04:24:31.163 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3715999999999996e-05 2023-11-25T04:24:31.163 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:31.163 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:31.163 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:31.163 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.163 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.163 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:1067 4.21 4:869a7600:::100000011fa.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:31.163 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:22.155038+0000", 2023-11-25T04:24:31.164 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 8.9554112250000006, 2023-11-25T04:24:31.164 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011313148, 2023-11-25T04:24:31.164 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:31.164 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:31.164 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:31.164 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.164 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:31.164 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155038+0000", 2023-11-25T04:24:31.164 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:31.164 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.164 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.164 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:31.165 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155038+0000", 2023-11-25T04:24:31.165 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:31.165 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.165 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.165 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:31.165 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155039+0000", 2023-11-25T04:24:31.165 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0200000000000002e-07 2023-11-25T04:24:31.165 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.165 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.165 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:31.165 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155049+0000", 2023-11-25T04:24:31.165 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0295000000000001e-05 2023-11-25T04:24:31.165 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.166 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.166 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:31.166 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155058+0000", 2023-11-25T04:24:31.166 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.208e-06 2023-11-25T04:24:31.166 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.166 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.166 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:31.166 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155064+0000", 2023-11-25T04:24:31.166 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4489999999999996e-06 2023-11-25T04:24:31.166 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.167 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.167 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:31.167 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155104+0000", 2023-11-25T04:24:31.167 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0164999999999999e-05 2023-11-25T04:24:31.167 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.167 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.167 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:31.167 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155147+0000", 2023-11-25T04:24:31.167 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2576e-05 2023-11-25T04:24:31.167 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.167 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.167 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:31.168 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155212+0000", 2023-11-25T04:24:31.168 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5516000000000001e-05 2023-11-25T04:24:31.168 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.168 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.168 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:31.168 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.156025+0000", 2023-11-25T04:24:31.168 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00081248599999999996 2023-11-25T04:24:31.168 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.168 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.168 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:31.168 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.166250+0000", 2023-11-25T04:24:31.168 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010224736999999999 2023-11-25T04:24:31.169 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.169 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.169 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:31.169 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.166319+0000", 2023-11-25T04:24:31.169 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9145000000000003e-05 2023-11-25T04:24:31.169 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.169 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.169 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:31.169 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.166352+0000", 2023-11-25T04:24:31.169 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2969000000000002e-05 2023-11-25T04:24:31.169 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:31.169 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:31.170 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:31.170 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.170 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.170 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:1102 4.2f 4:f543b94c:::1000000121d.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:31.170 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:22.158246+0000", 2023-11-25T04:24:31.170 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 8.9522037020000003, 2023-11-25T04:24:31.170 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010094396, 2023-11-25T04:24:31.170 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:31.171 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:31.171 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:31.171 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.171 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:31.171 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158246+0000", 2023-11-25T04:24:31.171 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:31.171 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.171 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.171 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:31.171 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158246+0000", 2023-11-25T04:24:31.171 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:31.171 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.171 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.172 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:31.172 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158246+0000", 2023-11-25T04:24:31.172 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5299999999999999e-07 2023-11-25T04:24:31.172 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.172 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.172 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:31.172 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158249+0000", 2023-11-25T04:24:31.172 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2840000000000001e-06 2023-11-25T04:24:31.172 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.172 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.172 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:31.172 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158254+0000", 2023-11-25T04:24:31.173 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3449999999999996e-06 2023-11-25T04:24:31.173 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.173 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.173 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:31.173 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158258+0000", 2023-11-25T04:24:31.173 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5030000000000002e-06 2023-11-25T04:24:31.173 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.173 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.173 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:31.173 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158302+0000", 2023-11-25T04:24:31.173 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3908e-05 2023-11-25T04:24:31.173 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.174 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.174 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:31.174 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158398+0000", 2023-11-25T04:24:31.174 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6898000000000006e-05 2023-11-25T04:24:31.174 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.174 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.174 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:31.174 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158480+0000", 2023-11-25T04:24:31.174 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1578999999999999e-05 2023-11-25T04:24:31.174 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.174 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.174 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:31.175 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.159229+0000", 2023-11-25T04:24:31.175 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000749049 2023-11-25T04:24:31.175 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.175 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.175 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:31.175 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.168228+0000", 2023-11-25T04:24:31.175 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0089992930000000002 2023-11-25T04:24:31.175 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.175 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.175 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:31.175 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.168310+0000", 2023-11-25T04:24:31.175 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1348999999999997e-05 2023-11-25T04:24:31.176 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.176 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.176 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:31.176 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.168340+0000", 2023-11-25T04:24:31.176 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0735000000000003e-05 2023-11-25T04:24:31.176 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:31.176 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:31.176 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:31.176 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.176 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.176 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:1105 4.23 4:c6ff07b4:::10000001220.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:31.176 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:22.158714+0000", 2023-11-25T04:24:31.177 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 8.951735588, 2023-11-25T04:24:31.177 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010412490999999999, 2023-11-25T04:24:31.177 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:31.177 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:31.177 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:31.177 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.177 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:31.177 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158714+0000", 2023-11-25T04:24:31.177 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:31.177 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.177 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.178 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:31.178 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158714+0000", 2023-11-25T04:24:31.178 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:31.178 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.178 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.178 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:31.178 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158715+0000", 2023-11-25T04:24:31.178 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4099999999999998e-07 2023-11-25T04:24:31.178 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.178 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.178 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:31.179 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158718+0000", 2023-11-25T04:24:31.179 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4319999999999999e-06 2023-11-25T04:24:31.179 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.179 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.179 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:31.179 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158729+0000", 2023-11-25T04:24:31.179 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.078e-05 2023-11-25T04:24:31.179 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.179 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.179 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:31.179 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158736+0000", 2023-11-25T04:24:31.180 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4649999999999997e-06 2023-11-25T04:24:31.180 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.180 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.180 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:31.180 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158779+0000", 2023-11-25T04:24:31.180 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3049000000000001e-05 2023-11-25T04:24:31.180 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.180 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.180 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:31.180 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158865+0000", 2023-11-25T04:24:31.180 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5138999999999999e-05 2023-11-25T04:24:31.181 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.181 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.181 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:31.181 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158983+0000", 2023-11-25T04:24:31.181 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000118658 2023-11-25T04:24:31.181 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.181 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.181 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:31.181 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.159645+0000", 2023-11-25T04:24:31.181 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00066193999999999997 2023-11-25T04:24:31.181 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.182 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.182 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:31.182 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.169046+0000", 2023-11-25T04:24:31.182 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0094005400000000006 2023-11-25T04:24:31.182 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.182 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.182 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:31.182 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.169092+0000", 2023-11-25T04:24:31.182 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5840000000000002e-05 2023-11-25T04:24:31.182 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.182 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.183 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:31.183 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.169127+0000", 2023-11-25T04:24:31.183 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5006999999999999e-05 2023-11-25T04:24:31.183 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:31.183 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:31.183 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:31.183 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.183 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.183 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2841 4.4 4:22111d42:::100000010df.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:31.183 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.092102+0000", 2023-11-25T04:24:31.183 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 6.0183481130000001, 2023-11-25T04:24:31.184 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.016449783999999999, 2023-11-25T04:24:31.184 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:31.184 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:31.184 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:31.184 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.184 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:31.184 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092102+0000", 2023-11-25T04:24:31.184 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:31.184 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.184 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.184 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:31.184 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092102+0000", 2023-11-25T04:24:31.185 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:31.185 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.185 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.185 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:31.185 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092102+0000", 2023-11-25T04:24:31.185 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3700000000000001e-07 2023-11-25T04:24:31.185 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.185 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.185 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:31.186 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092104+0000", 2023-11-25T04:24:31.186 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0679999999999999e-06 2023-11-25T04:24:31.186 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.186 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.186 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:31.186 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092111+0000", 2023-11-25T04:24:31.186 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0589999999999997e-06 2023-11-25T04:24:31.186 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.186 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.186 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:31.186 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092130+0000", 2023-11-25T04:24:31.187 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8686999999999999e-05 2023-11-25T04:24:31.187 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.187 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.187 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:31.187 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092443+0000", 2023-11-25T04:24:31.187 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000313519 2023-11-25T04:24:31.187 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.187 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.187 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:31.187 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092553+0000", 2023-11-25T04:24:31.187 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010932100000000001 2023-11-25T04:24:31.187 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.188 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.188 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:31.188 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092639+0000", 2023-11-25T04:24:31.188 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6094999999999995e-05 2023-11-25T04:24:31.188 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.188 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.188 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:31.188 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094679+0000", 2023-11-25T04:24:31.188 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0020402580000000001 2023-11-25T04:24:31.188 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.188 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.189 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:31.189 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108462+0000", 2023-11-25T04:24:31.189 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013783358000000001 2023-11-25T04:24:31.189 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.189 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.189 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:31.189 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108516+0000", 2023-11-25T04:24:31.189 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3483000000000003e-05 2023-11-25T04:24:31.189 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.189 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.189 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:31.190 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108551+0000", 2023-11-25T04:24:31.190 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5499e-05 2023-11-25T04:24:31.190 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:31.190 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:31.190 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:31.190 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.190 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.190 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2852 4.2d 4:b745ac76:::100000010f5.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:31.190 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.093477+0000", 2023-11-25T04:24:31.191 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 6.0169730010000002, 2023-11-25T04:24:31.191 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.015423084, 2023-11-25T04:24:31.191 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:31.191 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:31.191 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:31.191 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.191 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:31.191 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093477+0000", 2023-11-25T04:24:31.191 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:31.191 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.191 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.191 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:31.191 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093477+0000", 2023-11-25T04:24:31.192 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:31.192 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.192 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.192 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:31.192 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093477+0000", 2023-11-25T04:24:31.192 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2500000000000005e-07 2023-11-25T04:24:31.192 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.192 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.192 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:31.192 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093480+0000", 2023-11-25T04:24:31.192 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0189999999999998e-06 2023-11-25T04:24:31.192 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.193 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.193 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:31.193 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093642+0000", 2023-11-25T04:24:31.193 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000162072 2023-11-25T04:24:31.193 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.193 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.193 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:31.193 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093649+0000", 2023-11-25T04:24:31.193 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9659999999999999e-06 2023-11-25T04:24:31.193 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.193 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.193 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:31.193 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093693+0000", 2023-11-25T04:24:31.194 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3794000000000001e-05 2023-11-25T04:24:31.194 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.194 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.194 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:31.194 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093764+0000", 2023-11-25T04:24:31.194 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0981000000000004e-05 2023-11-25T04:24:31.194 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.194 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.194 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:31.194 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093850+0000", 2023-11-25T04:24:31.194 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5536000000000002e-05 2023-11-25T04:24:31.194 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.195 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.195 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:31.195 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094710+0000", 2023-11-25T04:24:31.195 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00086070299999999999 2023-11-25T04:24:31.195 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.195 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.195 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:31.195 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108776+0000", 2023-11-25T04:24:31.195 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01406542 2023-11-25T04:24:31.195 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.195 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.196 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:31.196 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108829+0000", 2023-11-25T04:24:31.196 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2936999999999998e-05 2023-11-25T04:24:31.196 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.196 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.196 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:31.196 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108900+0000", 2023-11-25T04:24:31.196 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1031000000000005e-05 2023-11-25T04:24:31.196 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:31.196 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:31.196 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:31.197 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.197 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.197 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2854 4.2d 4:b5c259cf:::100000010f9.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:31.197 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.093674+0000", 2023-11-25T04:24:31.197 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 6.0167753409999998, 2023-11-25T04:24:31.197 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.015970380999999999, 2023-11-25T04:24:31.197 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:31.197 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:31.197 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:31.197 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.197 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:31.198 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093674+0000", 2023-11-25T04:24:31.198 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:31.198 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.198 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.198 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:31.198 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093674+0000", 2023-11-25T04:24:31.198 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:31.198 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.198 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.198 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:31.199 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093675+0000", 2023-11-25T04:24:31.199 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8400000000000004e-07 2023-11-25T04:24:31.199 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.199 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.199 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:31.199 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093678+0000", 2023-11-25T04:24:31.199 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2229999999999999e-06 2023-11-25T04:24:31.199 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.199 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.200 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:31.200 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093686+0000", 2023-11-25T04:24:31.200 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9990000000000008e-06 2023-11-25T04:24:31.200 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.200 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.200 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:31.200 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093696+0000", 2023-11-25T04:24:31.200 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6479999999999998e-06 2023-11-25T04:24:31.200 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.200 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.200 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:31.200 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094104+0000", 2023-11-25T04:24:31.201 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00040823900000000002 2023-11-25T04:24:31.201 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.201 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.201 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:31.201 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094172+0000", 2023-11-25T04:24:31.201 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8023999999999998e-05 2023-11-25T04:24:31.201 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.201 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.201 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:31.201 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094256+0000", 2023-11-25T04:24:31.201 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3418999999999995e-05 2023-11-25T04:24:31.201 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.202 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.202 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:31.202 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095160+0000", 2023-11-25T04:24:31.202 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00090418599999999997 2023-11-25T04:24:31.202 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.202 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.202 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:31.202 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109551+0000", 2023-11-25T04:24:31.202 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014391571000000001 2023-11-25T04:24:31.202 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.202 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.202 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:31.202 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109601+0000", 2023-11-25T04:24:31.203 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9478000000000003e-05 2023-11-25T04:24:31.203 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.203 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.203 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:31.203 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109645+0000", 2023-11-25T04:24:31.203 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.401e-05 2023-11-25T04:24:31.203 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:31.203 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:31.203 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:31.203 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.204 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.204 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2861 4.4 4:214027a9:::10000001107.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:31.204 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.093829+0000", 2023-11-25T04:24:31.204 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 6.0166209229999996, 2023-11-25T04:24:31.204 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014848917999999999, 2023-11-25T04:24:31.204 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:31.204 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:31.204 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:31.204 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.204 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:31.205 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093829+0000", 2023-11-25T04:24:31.205 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:31.205 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.205 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.205 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:31.205 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093829+0000", 2023-11-25T04:24:31.205 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:31.205 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.205 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.205 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:31.205 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093830+0000", 2023-11-25T04:24:31.205 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1999999999999998e-07 2023-11-25T04:24:31.206 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.206 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.206 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:31.206 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093834+0000", 2023-11-25T04:24:31.206 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.34e-06 2023-11-25T04:24:31.206 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.206 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.206 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:31.206 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093844+0000", 2023-11-25T04:24:31.206 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6690000000000001e-06 2023-11-25T04:24:31.206 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.207 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.207 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:31.207 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093857+0000", 2023-11-25T04:24:31.207 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3434999999999999e-05 2023-11-25T04:24:31.207 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.207 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.207 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:31.207 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093940+0000", 2023-11-25T04:24:31.207 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3047999999999995e-05 2023-11-25T04:24:31.207 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.207 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.207 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:31.207 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094052+0000", 2023-11-25T04:24:31.208 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000111538 2023-11-25T04:24:31.208 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.208 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.208 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:31.209 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094466+0000", 2023-11-25T04:24:31.209 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00041464600000000001 2023-11-25T04:24:31.209 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.209 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.209 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:31.209 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095570+0000", 2023-11-25T04:24:31.209 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001103981 2023-11-25T04:24:31.209 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.209 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.209 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:31.209 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108598+0000", 2023-11-25T04:24:31.210 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013027164000000001 2023-11-25T04:24:31.210 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.210 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.210 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:31.210 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108646+0000", 2023-11-25T04:24:31.210 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8090999999999997e-05 2023-11-25T04:24:31.210 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.210 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.210 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:31.210 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108678+0000", 2023-11-25T04:24:31.210 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2085999999999997e-05 2023-11-25T04:24:31.210 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:31.211 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:31.211 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:31.211 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.211 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.211 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2863 4.12 4:4b84c733:::1000000110b.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:31.211 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.093877+0000", 2023-11-25T04:24:31.211 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 6.0165725959999996, 2023-11-25T04:24:31.211 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014682755, 2023-11-25T04:24:31.211 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:31.211 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:31.211 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:31.212 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.212 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:31.212 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093877+0000", 2023-11-25T04:24:31.212 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:31.212 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.212 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.212 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:31.212 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093877+0000", 2023-11-25T04:24:31.212 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:31.212 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.212 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.212 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:31.212 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093878+0000", 2023-11-25T04:24:31.213 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4199999999999996e-07 2023-11-25T04:24:31.213 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.213 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.213 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:31.213 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093880+0000", 2023-11-25T04:24:31.213 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6189999999999998e-06 2023-11-25T04:24:31.213 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.213 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.213 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:31.213 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093888+0000", 2023-11-25T04:24:31.213 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5510000000000002e-06 2023-11-25T04:24:31.213 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.214 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.214 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:31.214 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093893+0000", 2023-11-25T04:24:31.214 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4179999999999996e-06 2023-11-25T04:24:31.214 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.214 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.214 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:31.214 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093934+0000", 2023-11-25T04:24:31.214 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0648000000000001e-05 2023-11-25T04:24:31.214 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.214 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.214 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:31.215 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094081+0000", 2023-11-25T04:24:31.215 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014753499999999999 2023-11-25T04:24:31.215 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.215 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.215 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:31.215 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094172+0000", 2023-11-25T04:24:31.215 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.0722000000000003e-05 2023-11-25T04:24:31.215 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.215 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.215 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:31.215 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094742+0000", 2023-11-25T04:24:31.216 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00056948600000000002 2023-11-25T04:24:31.216 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.216 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.216 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:31.216 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108491+0000", 2023-11-25T04:24:31.216 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013749568 2023-11-25T04:24:31.216 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.216 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.216 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:31.216 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108525+0000", 2023-11-25T04:24:31.216 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3683999999999998e-05 2023-11-25T04:24:31.216 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.216 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.217 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:31.217 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108560+0000", 2023-11-25T04:24:31.217 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4981999999999998e-05 2023-11-25T04:24:31.217 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:31.217 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:31.217 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:31.217 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.217 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.217 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2880 4.21 4:855834a4:::1000000112d.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:31.217 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.095098+0000", 2023-11-25T04:24:31.217 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 6.0153519019999999, 2023-11-25T04:24:31.217 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013847544, 2023-11-25T04:24:31.218 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:31.218 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:31.218 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:31.218 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.218 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:31.218 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095098+0000", 2023-11-25T04:24:31.218 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:31.218 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.218 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.218 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:31.218 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095098+0000", 2023-11-25T04:24:31.218 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:31.219 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.219 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.219 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:31.219 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095098+0000", 2023-11-25T04:24:31.219 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3000000000000002e-07 2023-11-25T04:24:31.219 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.219 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.219 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:31.219 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095099+0000", 2023-11-25T04:24:31.220 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.331e-06 2023-11-25T04:24:31.220 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.220 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.220 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:31.220 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095103+0000", 2023-11-25T04:24:31.220 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8319999999999999e-06 2023-11-25T04:24:31.220 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.220 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.220 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:31.220 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095108+0000", 2023-11-25T04:24:31.220 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4179999999999999e-06 2023-11-25T04:24:31.221 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.221 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.221 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:31.221 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095153+0000", 2023-11-25T04:24:31.221 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4799000000000003e-05 2023-11-25T04:24:31.221 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.221 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.221 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:31.221 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095268+0000", 2023-11-25T04:24:31.221 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000115437 2023-11-25T04:24:31.221 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.222 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.222 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:31.222 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095363+0000", 2023-11-25T04:24:31.222 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.5278000000000005e-05 2023-11-25T04:24:31.222 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.222 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.222 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:31.222 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095845+0000", 2023-11-25T04:24:31.222 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00048164600000000002 2023-11-25T04:24:31.222 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.223 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.223 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:31.223 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108854+0000", 2023-11-25T04:24:31.223 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013009289 2023-11-25T04:24:31.223 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.223 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.223 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:31.223 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108907+0000", 2023-11-25T04:24:31.223 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2528999999999998e-05 2023-11-25T04:24:31.223 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.223 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.224 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:31.224 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108945+0000", 2023-11-25T04:24:31.224 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8655000000000002e-05 2023-11-25T04:24:31.224 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:31.224 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:31.224 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:31.224 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.224 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.224 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2884 4.37 4:ef909cdc:::10000001135.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:31.225 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.095518+0000", 2023-11-25T04:24:31.225 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 6.0149319749999997, 2023-11-25T04:24:31.225 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014149988, 2023-11-25T04:24:31.225 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:31.225 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:31.225 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:31.225 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.225 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:31.225 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095518+0000", 2023-11-25T04:24:31.225 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:31.225 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.226 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.226 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:31.226 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095518+0000", 2023-11-25T04:24:31.226 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:31.226 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.226 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.226 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:31.226 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095518+0000", 2023-11-25T04:24:31.226 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2900000000000003e-07 2023-11-25T04:24:31.226 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.226 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.227 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:31.227 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095522+0000", 2023-11-25T04:24:31.227 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2119999999999999e-06 2023-11-25T04:24:31.227 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.227 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.227 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:31.227 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095532+0000", 2023-11-25T04:24:31.227 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0375e-05 2023-11-25T04:24:31.227 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.227 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.228 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:31.228 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095537+0000", 2023-11-25T04:24:31.228 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5909999999999998e-06 2023-11-25T04:24:31.228 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.228 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.228 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:31.228 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095584+0000", 2023-11-25T04:24:31.228 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7169999999999997e-05 2023-11-25T04:24:31.228 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.228 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.228 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:31.229 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095695+0000", 2023-11-25T04:24:31.229 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011078699999999999 2023-11-25T04:24:31.229 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.229 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.229 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:31.229 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095777+0000", 2023-11-25T04:24:31.229 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2089999999999995e-05 2023-11-25T04:24:31.229 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.229 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.229 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:31.230 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096286+0000", 2023-11-25T04:24:31.230 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000509247 2023-11-25T04:24:31.230 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.230 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.230 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:31.230 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109596+0000", 2023-11-25T04:24:31.230 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013309872 2023-11-25T04:24:31.230 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.230 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.231 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:31.231 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109638+0000", 2023-11-25T04:24:31.231 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2381e-05 2023-11-25T04:24:31.231 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.231 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.231 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:31.232 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109668+0000", 2023-11-25T04:24:31.232 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9634000000000001e-05 2023-11-25T04:24:31.232 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:31.232 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:31.232 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:31.232 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.232 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.232 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2890 4.12 4:4a2ad640:::10000001141.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:31.232 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.096274+0000", 2023-11-25T04:24:31.232 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 6.0141759119999998, 2023-11-25T04:24:31.232 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013647259, 2023-11-25T04:24:31.232 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:31.233 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:31.233 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:31.233 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.233 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:31.233 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096274+0000", 2023-11-25T04:24:31.233 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:31.233 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.233 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.233 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:31.233 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096274+0000", 2023-11-25T04:24:31.233 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:31.233 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.234 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.234 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:31.234 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096275+0000", 2023-11-25T04:24:31.234 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0299999999999998e-07 2023-11-25T04:24:31.234 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.234 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.234 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:31.234 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096279+0000", 2023-11-25T04:24:31.234 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9580000000000001e-06 2023-11-25T04:24:31.234 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.234 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.234 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:31.235 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096287+0000", 2023-11-25T04:24:31.235 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6929999999999996e-06 2023-11-25T04:24:31.235 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.235 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.235 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:31.235 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096294+0000", 2023-11-25T04:24:31.235 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.348e-06 2023-11-25T04:24:31.235 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.235 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.235 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:31.235 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096342+0000", 2023-11-25T04:24:31.235 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8068000000000001e-05 2023-11-25T04:24:31.235 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.236 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.236 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:31.236 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096462+0000", 2023-11-25T04:24:31.236 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000119918 2023-11-25T04:24:31.236 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.236 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.236 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:31.236 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096557+0000", 2023-11-25T04:24:31.236 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.5340999999999994e-05 2023-11-25T04:24:31.236 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.236 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.236 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:31.237 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096971+0000", 2023-11-25T04:24:31.237 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00041398100000000001 2023-11-25T04:24:31.237 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.237 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.237 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:31.237 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109849+0000", 2023-11-25T04:24:31.237 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.012877985999999999 2023-11-25T04:24:31.237 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.237 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.237 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:31.237 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109894+0000", 2023-11-25T04:24:31.237 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5595e-05 2023-11-25T04:24:31.238 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.238 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.238 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:31.238 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109921+0000", 2023-11-25T04:24:31.238 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6568e-05 2023-11-25T04:24:31.238 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:31.238 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:31.238 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:31.238 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.238 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.238 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2891 4.23 4:c4054dd3:::10000001143.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:31.238 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.096713+0000", 2023-11-25T04:24:31.238 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 6.0137372180000002, 2023-11-25T04:24:31.239 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013148692, 2023-11-25T04:24:31.239 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:31.239 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:31.239 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:31.239 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.239 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:31.239 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096713+0000", 2023-11-25T04:24:31.239 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:31.239 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.239 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.239 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:31.240 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096713+0000", 2023-11-25T04:24:31.240 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:31.240 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.240 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.240 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:31.240 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096713+0000", 2023-11-25T04:24:31.240 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9299999999999998e-07 2023-11-25T04:24:31.240 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.240 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.240 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:31.240 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096716+0000", 2023-11-25T04:24:31.240 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.988e-06 2023-11-25T04:24:31.240 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.241 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.241 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:31.241 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096722+0000", 2023-11-25T04:24:31.241 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0499999999999997e-06 2023-11-25T04:24:31.241 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.241 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.241 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:31.241 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096727+0000", 2023-11-25T04:24:31.241 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3380000000000004e-06 2023-11-25T04:24:31.241 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.241 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.241 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:31.242 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096770+0000", 2023-11-25T04:24:31.242 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2938000000000001e-05 2023-11-25T04:24:31.242 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.242 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.242 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:31.242 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096904+0000", 2023-11-25T04:24:31.242 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000133759 2023-11-25T04:24:31.242 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.242 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.242 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:31.242 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096983+0000", 2023-11-25T04:24:31.242 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8559999999999993e-05 2023-11-25T04:24:31.243 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.243 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.243 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:31.243 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.097437+0000", 2023-11-25T04:24:31.243 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00045439200000000002 2023-11-25T04:24:31.243 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.243 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.243 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:31.243 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109737+0000", 2023-11-25T04:24:31.243 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.012300190000000001 2023-11-25T04:24:31.243 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.243 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.243 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:31.244 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109782+0000", 2023-11-25T04:24:31.244 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4546999999999999e-05 2023-11-25T04:24:31.244 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:31.244 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:31.244 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:31.244 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109861+0000", 2023-11-25T04:24:31.244 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9337000000000003e-05 2023-11-25T04:24:31.244 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:31.244 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:31.244 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:31.244 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:31.244 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:31.244 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:31.245 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2023-11-25T04:24:31.247 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:24:31.248 INFO:tasks.workunit.client.0.smithi067.stdout:checking ndbm.h presence... no 2023-11-25T04:24:31.248 INFO:tasks.workunit.client.0.smithi067.stdout:checking for ndbm.h... no 2023-11-25T04:24:31.249 INFO:tasks.workunit.client.0.smithi067.stdout:checking libaio.h usability... yes 2023-11-25T04:24:31.249 INFO:tasks.workunit.client.0.smithi067.stdout:checking libaio.h presence... yes 2023-11-25T04:24:31.249 INFO:tasks.workunit.client.0.smithi067.stdout:checking for libaio.h... yes 2023-11-25T04:24:31.249 INFO:tasks.workunit.client.0.smithi067.stdout:checking for dm_make_handle in -ldm... no 2023-11-25T04:24:31.249 INFO:tasks.workunit.client.0.smithi067.stdout: 2023-11-25T04:24:31.249 INFO:tasks.workunit.client.0.smithi067.stdout:WARNING: could not find a valid DMAPI base library. 2023-11-25T04:24:31.249 INFO:tasks.workunit.client.0.smithi067.stdout:If you want DMAPI support please install the data migration 2023-11-25T04:24:31.249 INFO:tasks.workunit.client.0.smithi067.stdout:API (dmapi) library package. Alternatively, run "make install" 2023-11-25T04:24:31.249 INFO:tasks.workunit.client.0.smithi067.stdout:from the dmapi source. 2023-11-25T04:24:31.249 INFO:tasks.workunit.client.0.smithi067.stdout: 2023-11-25T04:24:31.249 INFO:tasks.workunit.client.0.smithi067.stdout:checking linux/fiemap.h usability... yes 2023-11-25T04:24:31.250 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:31.245+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:31.254 INFO:tasks.workunit.client.0.smithi067.stdout:checking linux/fiemap.h presence... yes 2023-11-25T04:24:31.254 INFO:tasks.workunit.client.0.smithi067.stdout:checking for linux/fiemap.h... yes 2023-11-25T04:24:31.307 INFO:tasks.workunit.client.0.smithi067.stdout:checking for fallocate... yes 2023-11-25T04:24:31.347 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:31.346+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:31.354 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:31.355 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:24:31.355 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2023-11-25T04:24:31.355 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:31.369 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:24:31.370 INFO:tasks.workunit.client.0.smithi067.stdout:checking for open_by_handle_at... yes 2023-11-25T04:24:31.404 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-11-25T04:24:31.419 INFO:tasks.workunit.client.0.smithi067.stdout:checking sys/prctl.h usability... yes 2023-11-25T04:24:31.436 INFO:tasks.workunit.client.0.smithi067.stdout:checking sys/prctl.h presence... yes 2023-11-25T04:24:31.436 INFO:tasks.workunit.client.0.smithi067.stdout:checking for sys/prctl.h... yes 2023-11-25T04:24:31.448 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:31.447+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:31.472 INFO:tasks.workunit.client.0.smithi067.stdout:checking linux/fs.h usability... yes 2023-11-25T04:24:31.490 INFO:tasks.workunit.client.0.smithi067.stdout:checking linux/fs.h presence... yes 2023-11-25T04:24:31.491 INFO:tasks.workunit.client.0.smithi067.stdout:checking for linux/fs.h... yes 2023-11-25T04:24:31.515 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:31.515 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:24:31.515 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2023-11-25T04:24:31.515 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2023-11-25T04:24:31.515 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:31.523 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:24:31.534 INFO:teuthology.orchestra.run.smithi067.stderr:Error ENXIO: problem getting command descriptions from osd.0 2023-11-25T04:24:31.535 INFO:tasks.workunit.client.0.smithi067.stdout:checking btrfsutil.h usability... no 2023-11-25T04:24:31.555 INFO:teuthology.orchestra.run.smithi067.stdout:osd.1: {} 2023-11-25T04:24:31.556 INFO:teuthology.orchestra.run.smithi067.stderr:osd.1: osd_enable_op_tracker = '' 2023-11-25T04:24:31.558 INFO:tasks.workunit.client.0.smithi067.stdout:checking btrfsutil.h presence... no 2023-11-25T04:24:31.558 INFO:tasks.workunit.client.0.smithi067.stdout:checking for btrfsutil.h... no 2023-11-25T04:24:31.567 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:31.566+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:31.575 INFO:teuthology.orchestra.run.smithi067.stdout:osd.2: {} 2023-11-25T04:24:31.575 INFO:teuthology.orchestra.run.smithi067.stderr:osd.2: osd_enable_op_tracker = '' 2023-11-25T04:24:31.578 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-11-25T04:24:31.578 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-11-25T04:24:31.616 INFO:tasks.workunit.client.0.smithi067.stdout:checking for copy_file_range... yes 2023-11-25T04:24:31.629 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:31.629 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:31.640 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:24:31.669 INFO:tasks.workunit.client.0.smithi067.stdout:checking for renameat2... yes 2023-11-25T04:24:31.684 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:31.683+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:31.688 INFO:tasks.workunit.client.0.smithi067.stdout:configure: creating ./config.status 2023-11-25T04:24:31.748 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:31.748 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:31.751 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:31.750+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:31.757 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:24:31.852 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:31.851+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:31.905 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:31.905 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:31.914 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:24:31.953 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:31.951+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:32.019 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:32.019 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:32.027 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:24:32.039 INFO:tasks.workunit.client.0.smithi067.stdout:config.status: creating include/builddefs 2023-11-25T04:24:32.054 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:32.052+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:32.054 INFO:tasks.workunit.client.0.smithi067.stdout:config.status: creating include/config.h 2023-11-25T04:24:32.067 INFO:tasks.workunit.client.0.smithi067.stdout:config.status: executing libtool commands 2023-11-25T04:24:32.139 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:32.140 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:32.150 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:24:32.158 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:32.153+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:32.162 INFO:tasks.workunit.client.0.smithi067.stdout:Building include 2023-11-25T04:24:32.162 INFO:tasks.workunit.client.0.smithi067.stdout:Building lib 2023-11-25T04:24:32.162 INFO:tasks.workunit.client.0.smithi067.stdout:Building tests 2023-11-25T04:24:32.169 INFO:teuthology.orchestra.run.smithi067.stderr:nodeep-scrub is set 2023-11-25T04:24:32.178 INFO:tasks.workunit.client.0.smithi067.stdout: [LTDEP] 2023-11-25T04:24:32.256 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:32.254+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:32.258 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:32.259 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:32.267 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:24:32.359 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:32.355+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:32.372 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:32.372 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:32.384 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:24:32.458 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:32.457+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:32.475 INFO:tasks.workunit.client.0.smithi067.stdout: [CC] dataascii.lo 2023-11-25T04:24:32.476 INFO:tasks.workunit.client.0.smithi067.stdout: [CC] databin.lo 2023-11-25T04:24:32.476 INFO:tasks.workunit.client.0.smithi067.stdout: [CC] datapid.lo 2023-11-25T04:24:32.476 INFO:tasks.workunit.client.0.smithi067.stdout: [CC] file_lock.lo 2023-11-25T04:24:32.492 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:32.492 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:32.501 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:24:32.559 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:32.558+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:32.606 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:32.607 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:32.615 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:24:32.650 INFO:tasks.workunit.client.0.smithi067.stdout: [CC] forker.lo 2023-11-25T04:24:32.651 INFO:tasks.workunit.client.0.smithi067.stdout: [CC] pattern.lo 2023-11-25T04:24:32.660 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:32.659+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:32.709 INFO:tasks.workunit.client.0.smithi067.stdout: [CC] open_flags.lo 2023-11-25T04:24:32.720 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:32.720 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:32.727 INFO:tasks.workunit.client.0.smithi067.stdout: [CC] random_range.lo 2023-11-25T04:24:32.729 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:24:32.760 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:32.759+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:32.820 INFO:tasks.workunit.client.0.smithi067.stdout: [CC] string_to_tokens.lo 2023-11-25T04:24:32.836 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:32.836 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:32.844 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:24:32.854 INFO:tasks.workunit.client.0.smithi067.stdout: [CC] str_to_bytes.lo 2023-11-25T04:24:32.888 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:32.887+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:32.934 INFO:tasks.workunit.client.0.smithi067.stdout: [CC] tlibio.lo 2023-11-25T04:24:32.950 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:32.950 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:32.959 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:24:32.962 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:32.961+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:32.964 INFO:tasks.workunit.client.0.smithi067.stdout: [CC] write_log.lo 2023-11-25T04:24:33.056 INFO:tasks.workunit.client.0.smithi067.stdout: [CC] random.lo 2023-11-25T04:24:33.063 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:33.062+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:33.110 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:33.110 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:33.119 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:24:33.165 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:33.163+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:33.225 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:33.225 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:33.234 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:24:33.275 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:33.273+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:33.300 INFO:tasks.workunit.client.0.smithi067.stdout: [LD] 2023-11-25T04:24:33.368 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:33.366+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:33.380 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:33.381 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:33.393 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:24:33.455 INFO:tasks.workunit.client.0.smithi067.stdout:Building ltp 2023-11-25T04:24:33.455 INFO:tasks.workunit.client.0.smithi067.stdout:Building src 2023-11-25T04:24:33.456 INFO:tasks.workunit.client.0.smithi067.stdout:Building m4 2023-11-25T04:24:33.456 INFO:tasks.workunit.client.0.smithi067.stdout:Building common 2023-11-25T04:24:33.469 INFO:tasks.workunit.client.0.smithi067.stdout: [DEP] 2023-11-25T04:24:33.470 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:33.469+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:33.487 INFO:tasks.workunit.client.0.smithi067.stdout: [DEP] 2023-11-25T04:24:33.537 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:33.537 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:33.546 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:24:33.579 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-11-25T04:24:33.589 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:33.588+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:33.654 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:33.654 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:33.661 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:24:33.671 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:33.669+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:33.705 INFO:teuthology.orchestra.run.smithi067.stderr:Error ENXIO: problem getting command descriptions from osd.0 2023-11-25T04:24:33.726 INFO:teuthology.orchestra.run.smithi067.stdout:osd.1: {} 2023-11-25T04:24:33.726 INFO:teuthology.orchestra.run.smithi067.stderr:osd.1: osd_enable_op_tracker = '' 2023-11-25T04:24:33.748 INFO:teuthology.orchestra.run.smithi067.stdout:osd.2: {} 2023-11-25T04:24:33.748 INFO:teuthology.orchestra.run.smithi067.stderr:osd.2: osd_enable_op_tracker = '' 2023-11-25T04:24:33.751 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-11-25T04:24:33.752 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-11-25T04:24:33.772 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:33.771+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:33.806 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:33.807 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:24:33.807 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2023-11-25T04:24:33.807 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2023-11-25T04:24:33.807 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:33.815 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:24:33.873 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:33.871+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:33.922 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:33.922 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2023-11-25T04:24:33.922 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2023-11-25T04:24:33.922 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2023-11-25T04:24:33.922 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.922 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:152 3.1a 3:5e41ab69:::200.00000005:head [write 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:33.923 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:21.237401+0000", 2023-11-25T04:24:33.923 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 12.682385904, 2023-11-25T04:24:33.923 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015253802, 2023-11-25T04:24:33.923 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:33.923 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:33.923 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:33.923 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.923 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:33.923 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.237401+0000", 2023-11-25T04:24:33.923 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:33.923 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.923 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.924 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:33.924 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.237401+0000", 2023-11-25T04:24:33.924 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:33.924 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.924 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.924 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:33.924 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.237402+0000", 2023-11-25T04:24:33.924 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3699999999999999e-07 2023-11-25T04:24:33.924 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.924 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.924 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:33.924 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243182+0000", 2023-11-25T04:24:33.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057801270000000004 2023-11-25T04:24:33.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.925 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:33.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243193+0000", 2023-11-25T04:24:33.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1221e-05 2023-11-25T04:24:33.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.925 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:33.925 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243198+0000", 2023-11-25T04:24:33.925 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3739999999999997e-06 2023-11-25T04:24:33.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.926 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.926 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:33.926 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243246+0000", 2023-11-25T04:24:33.926 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7667999999999998e-05 2023-11-25T04:24:33.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.926 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.926 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:33.926 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243292+0000", 2023-11-25T04:24:33.926 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5794000000000002e-05 2023-11-25T04:24:33.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.926 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.926 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:33.927 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243360+0000", 2023-11-25T04:24:33.927 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8479000000000006e-05 2023-11-25T04:24:33.927 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.927 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.927 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:33.927 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.250667+0000", 2023-11-25T04:24:33.927 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.007306618 2023-11-25T04:24:33.927 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.927 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.928 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:33.928 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.252598+0000", 2023-11-25T04:24:33.928 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019315129999999999 2023-11-25T04:24:33.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.929 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:33.929 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.252631+0000", 2023-11-25T04:24:33.929 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2925000000000001e-05 2023-11-25T04:24:33.929 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.929 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.929 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:33.929 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.252655+0000", 2023-11-25T04:24:33.929 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3246000000000001e-05 2023-11-25T04:24:33.929 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:33.929 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:33.929 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:33.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.930 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.930 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:2841 4.4 e47/17 4:22111d42:::100000010df.00000000:head v 47'139, mlcod=47'137)", 2023-11-25T04:24:33.930 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.093610+0000", 2023-11-25T04:24:33.931 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.8261770409999993, 2023-11-25T04:24:33.931 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014758871999999999, 2023-11-25T04:24:33.931 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:33.931 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:33.931 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:33.931 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.931 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:33.931 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093610+0000", 2023-11-25T04:24:33.931 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:33.931 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.931 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.931 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:33.931 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093610+0000", 2023-11-25T04:24:33.932 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:33.932 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.932 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.932 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:33.932 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093610+0000", 2023-11-25T04:24:33.932 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1799999999999999e-07 2023-11-25T04:24:33.932 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.932 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.932 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:33.932 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093621+0000", 2023-11-25T04:24:33.932 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1107999999999999e-05 2023-11-25T04:24:33.933 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.933 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.933 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:33.933 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093627+0000", 2023-11-25T04:24:33.933 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0310000000000004e-06 2023-11-25T04:24:33.933 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.933 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.933 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:33.933 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093633+0000", 2023-11-25T04:24:33.933 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2700000000000001e-06 2023-11-25T04:24:33.933 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.933 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.933 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:33.934 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093722+0000", 2023-11-25T04:24:33.934 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8417000000000004e-05 2023-11-25T04:24:33.934 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.934 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.934 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:33.934 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093737+0000", 2023-11-25T04:24:33.934 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5781e-05 2023-11-25T04:24:33.934 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.934 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.934 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:33.934 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108203+0000", 2023-11-25T04:24:33.934 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014465844 2023-11-25T04:24:33.934 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.935 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.935 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:33.935 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108368+0000", 2023-11-25T04:24:33.935 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000165203 2023-11-25T04:24:33.935 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:33.935 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:33.935 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:33.935 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.935 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.935 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2860 4.3d 4:bf0d7877:::10000001105.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:33.935 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.093823+0000", 2023-11-25T04:24:33.935 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.8259638729999992, 2023-11-25T04:24:33.936 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015582848, 2023-11-25T04:24:33.936 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:33.936 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:33.936 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:33.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.936 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:33.936 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093823+0000", 2023-11-25T04:24:33.936 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:33.936 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.936 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.936 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:33.936 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093823+0000", 2023-11-25T04:24:33.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:33.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.937 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.937 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:33.937 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093823+0000", 2023-11-25T04:24:33.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9100000000000004e-07 2023-11-25T04:24:33.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.937 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.937 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:33.937 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093826+0000", 2023-11-25T04:24:33.937 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4499999999999998e-06 2023-11-25T04:24:33.937 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.937 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.938 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:33.938 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093835+0000", 2023-11-25T04:24:33.938 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8179999999999993e-06 2023-11-25T04:24:33.938 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.938 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.938 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:33.938 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093840+0000", 2023-11-25T04:24:33.938 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0819999999999998e-06 2023-11-25T04:24:33.938 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.938 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.938 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:33.938 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093973+0000", 2023-11-25T04:24:33.938 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013308999999999999 2023-11-25T04:24:33.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.939 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:33.939 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094277+0000", 2023-11-25T04:24:33.939 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00030403299999999999 2023-11-25T04:24:33.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.939 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:33.939 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108608+0000", 2023-11-25T04:24:33.939 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014330888 2023-11-25T04:24:33.939 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.939 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.939 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:33.940 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108676+0000", 2023-11-25T04:24:33.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8577000000000004e-05 2023-11-25T04:24:33.940 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.940 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.940 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:33.940 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109406+0000", 2023-11-25T04:24:33.940 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00072931899999999997 2023-11-25T04:24:33.940 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:33.942 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:33.942 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:33.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.943 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2862 4.13 4:cb2e645e:::10000001109.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:33.943 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.093852+0000", 2023-11-25T04:24:33.943 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.8259349119999992, 2023-11-25T04:24:33.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015190636, 2023-11-25T04:24:33.943 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:33.943 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:33.943 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:33.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.943 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:33.943 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093852+0000", 2023-11-25T04:24:33.943 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:33.943 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:33.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093852+0000", 2023-11-25T04:24:33.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:33.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:33.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093852+0000", 2023-11-25T04:24:33.944 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0299999999999999e-07 2023-11-25T04:24:33.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.944 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:33.944 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093854+0000", 2023-11-25T04:24:33.945 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.846e-06 2023-11-25T04:24:33.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.946 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:33.946 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093861+0000", 2023-11-25T04:24:33.948 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5520000000000001e-06 2023-11-25T04:24:33.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.949 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:33.949 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093865+0000", 2023-11-25T04:24:33.949 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7870000000000001e-06 2023-11-25T04:24:33.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.950 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:33.950 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093940+0000", 2023-11-25T04:24:33.950 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4799e-05 2023-11-25T04:24:33.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.952 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:33.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094052+0000", 2023-11-25T04:24:33.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011156500000000001 2023-11-25T04:24:33.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.952 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:33.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108928+0000", 2023-11-25T04:24:33.952 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014876245 2023-11-25T04:24:33.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.952 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:33.952 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108972+0000", 2023-11-25T04:24:33.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4477999999999997e-05 2023-11-25T04:24:33.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.953 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:33.953 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109042+0000", 2023-11-25T04:24:33.953 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9861000000000001e-05 2023-11-25T04:24:33.953 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:33.953 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:33.953 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:33.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.954 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2865 4.36 4:6ebc5ac6:::1000000110f.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:33.954 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.093906+0000", 2023-11-25T04:24:33.954 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.8258806389999993, 2023-11-25T04:24:33.954 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014962628, 2023-11-25T04:24:33.954 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:33.954 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:33.954 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:33.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:33.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093906+0000", 2023-11-25T04:24:33.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:33.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.955 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:33.955 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093906+0000", 2023-11-25T04:24:33.955 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:33.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.956 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:33.956 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093906+0000", 2023-11-25T04:24:33.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4400000000000001e-07 2023-11-25T04:24:33.956 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.956 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.956 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:33.956 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093908+0000", 2023-11-25T04:24:33.956 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8640000000000001e-06 2023-11-25T04:24:33.956 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.956 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.956 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:33.956 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093914+0000", 2023-11-25T04:24:33.957 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0170000000000002e-06 2023-11-25T04:24:33.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:33.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093921+0000", 2023-11-25T04:24:33.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6529999999999997e-06 2023-11-25T04:24:33.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:33.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094258+0000", 2023-11-25T04:24:33.958 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00033703599999999998 2023-11-25T04:24:33.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.958 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:33.958 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094340+0000", 2023-11-25T04:24:33.959 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2279000000000003e-05 2023-11-25T04:24:33.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.959 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:33.959 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094416+0000", 2023-11-25T04:24:33.960 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6248999999999995e-05 2023-11-25T04:24:33.960 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.960 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.960 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:33.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.096134+0000", 2023-11-25T04:24:33.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001717611 2023-11-25T04:24:33.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.961 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:33.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108809+0000", 2023-11-25T04:24:33.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012675327 2023-11-25T04:24:33.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.961 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:33.961 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108836+0000", 2023-11-25T04:24:33.961 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6509999999999999e-05 2023-11-25T04:24:33.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.962 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:33.962 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108869+0000", 2023-11-25T04:24:33.962 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2737999999999997e-05 2023-11-25T04:24:33.962 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:33.962 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:33.962 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:33.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.962 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:2852 4.2d e47/17 4:b745ac76:::100000010f5.00000000:head v 47'126, mlcod=47'124)", 2023-11-25T04:24:33.963 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.093987+0000", 2023-11-25T04:24:33.963 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.8258000780000003, 2023-11-25T04:24:33.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014613256, 2023-11-25T04:24:33.963 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:33.963 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:33.963 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:33.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:33.963 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093987+0000", 2023-11-25T04:24:33.963 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:33.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.963 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:33.964 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093987+0000", 2023-11-25T04:24:33.964 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:33.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.965 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.965 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:33.965 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093987+0000", 2023-11-25T04:24:33.965 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7000000000000001e-07 2023-11-25T04:24:33.965 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:33.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093990+0000", 2023-11-25T04:24:33.966 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8729999999999999e-06 2023-11-25T04:24:33.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.966 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:33.966 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093997+0000", 2023-11-25T04:24:33.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.973e-06 2023-11-25T04:24:33.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.967 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:33.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094001+0000", 2023-11-25T04:24:33.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7360000000000001e-06 2023-11-25T04:24:33.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.967 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:33.967 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094263+0000", 2023-11-25T04:24:33.967 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026173400000000001 2023-11-25T04:24:33.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.968 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:33.968 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094278+0000", 2023-11-25T04:24:33.968 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4681999999999999e-05 2023-11-25T04:24:33.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.968 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:33.968 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108563+0000", 2023-11-25T04:24:33.968 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014284738 2023-11-25T04:24:33.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.969 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:33.969 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108600+0000", 2023-11-25T04:24:33.969 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7249999999999997e-05 2023-11-25T04:24:33.969 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:33.969 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:33.969 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:33.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.969 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2866 4.36 4:6f6907fb:::10000001111.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:33.969 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.094033+0000", 2023-11-25T04:24:33.969 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.8257539129999998, 2023-11-25T04:24:33.969 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014885591, 2023-11-25T04:24:33.969 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:33.970 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:33.970 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:33.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.970 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:33.970 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094033+0000", 2023-11-25T04:24:33.970 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:33.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.970 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:33.970 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094033+0000", 2023-11-25T04:24:33.970 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:33.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.971 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:33.971 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094033+0000", 2023-11-25T04:24:33.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.06e-07 2023-11-25T04:24:33.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.971 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:33.971 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094036+0000", 2023-11-25T04:24:33.971 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.508e-06 2023-11-25T04:24:33.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.971 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:33.972 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094044+0000", 2023-11-25T04:24:33.972 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5909999999999996e-06 2023-11-25T04:24:33.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:33.972 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094051+0000", 2023-11-25T04:24:33.972 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0319999999999998e-06 2023-11-25T04:24:33.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.972 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:33.972 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094727+0000", 2023-11-25T04:24:33.973 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00067523300000000002 2023-11-25T04:24:33.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.973 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.973 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:33.973 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094795+0000", 2023-11-25T04:24:33.973 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8156000000000003e-05 2023-11-25T04:24:33.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.973 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.973 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:33.973 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094898+0000", 2023-11-25T04:24:33.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103329 2023-11-25T04:24:33.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.974 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:33.974 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108803+0000", 2023-11-25T04:24:33.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013904385 2023-11-25T04:24:33.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.974 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:33.974 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108875+0000", 2023-11-25T04:24:33.974 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1958999999999993e-05 2023-11-25T04:24:33.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:33.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108899+0000", 2023-11-25T04:24:33.975 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4019000000000002e-05 2023-11-25T04:24:33.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.975 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:33.975 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108918+0000", 2023-11-25T04:24:33.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9772999999999999e-05 2023-11-25T04:24:33.976 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:33.976 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:33.976 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:33.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.976 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:2854 4.2d e47/17 4:b5c259cf:::100000010f9.00000000:head v 47'127, mlcod=47'124)", 2023-11-25T04:24:33.976 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.094793+0000", 2023-11-25T04:24:33.976 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.8249936239999993, 2023-11-25T04:24:33.976 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014650764, 2023-11-25T04:24:33.977 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:33.977 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:33.977 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:33.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.977 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:33.977 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094793+0000", 2023-11-25T04:24:33.977 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:33.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.978 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.978 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:33.978 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094793+0000", 2023-11-25T04:24:33.978 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:33.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.978 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.978 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:33.978 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094793+0000", 2023-11-25T04:24:33.978 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3700000000000001e-07 2023-11-25T04:24:33.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:33.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094798+0000", 2023-11-25T04:24:33.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1200000000000004e-06 2023-11-25T04:24:33.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.979 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:33.979 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094805+0000", 2023-11-25T04:24:33.979 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9759999999999998e-06 2023-11-25T04:24:33.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:33.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094811+0000", 2023-11-25T04:24:33.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1410000000000002e-06 2023-11-25T04:24:33.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.980 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:33.980 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094838+0000", 2023-11-25T04:24:33.980 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6995000000000002e-05 2023-11-25T04:24:33.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:33.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094848+0000", 2023-11-25T04:24:33.981 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0115999999999999e-05 2023-11-25T04:24:33.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.981 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:33.981 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109408+0000", 2023-11-25T04:24:33.982 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014559825 2023-11-25T04:24:33.982 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.982 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.982 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:33.982 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109444+0000", 2023-11-25T04:24:33.982 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6154e-05 2023-11-25T04:24:33.982 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:33.982 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:33.983 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:33.983 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.983 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.983 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2869 4.6 4:63719fb0:::10000001117.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:33.983 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.094910+0000", 2023-11-25T04:24:33.983 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.8248766669999998, 2023-11-25T04:24:33.983 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014132570000000001, 2023-11-25T04:24:33.983 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:33.984 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:33.984 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:33.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.984 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:33.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094910+0000", 2023-11-25T04:24:33.984 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:33.984 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.984 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.984 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:33.984 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094910+0000", 2023-11-25T04:24:33.985 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:33.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:33.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094910+0000", 2023-11-25T04:24:33.985 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.58e-07 2023-11-25T04:24:33.985 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.985 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:33.985 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094913+0000", 2023-11-25T04:24:33.986 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.198e-06 2023-11-25T04:24:33.986 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.986 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.986 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:33.986 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094921+0000", 2023-11-25T04:24:33.986 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1629999999999996e-06 2023-11-25T04:24:33.986 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.987 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.987 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:33.987 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094926+0000", 2023-11-25T04:24:33.988 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7559999999999999e-06 2023-11-25T04:24:33.989 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.989 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.989 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:33.989 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094966+0000", 2023-11-25T04:24:33.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0151999999999998e-05 2023-11-25T04:24:33.989 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.989 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.989 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:33.989 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095073+0000", 2023-11-25T04:24:33.989 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107657 2023-11-25T04:24:33.989 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.989 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.990 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:33.990 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095158+0000", 2023-11-25T04:24:33.990 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5136999999999995e-05 2023-11-25T04:24:33.990 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.990 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.990 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:33.990 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108924+0000", 2023-11-25T04:24:33.990 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013765727 2023-11-25T04:24:33.990 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.990 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.991 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:33.991 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108958+0000", 2023-11-25T04:24:33.991 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3417000000000002e-05 2023-11-25T04:24:33.991 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.991 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.991 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:33.991 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109012+0000", 2023-11-25T04:24:33.991 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3939e-05 2023-11-25T04:24:33.991 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.991 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.991 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:33.991 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109043+0000", 2023-11-25T04:24:33.991 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0966000000000001e-05 2023-11-25T04:24:33.991 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:33.992 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:33.992 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:33.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.992 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2871 4.3f 4:fff10e11:::1000000111b.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:33.992 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.094936+0000", 2023-11-25T04:24:33.992 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.8248508290000007, 2023-11-25T04:24:33.992 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014261726000000001, 2023-11-25T04:24:33.992 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:33.992 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:33.992 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:33.992 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.992 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:33.993 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094936+0000", 2023-11-25T04:24:33.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:33.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.993 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:33.993 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094936+0000", 2023-11-25T04:24:33.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:33.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.993 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:33.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094936+0000", 2023-11-25T04:24:33.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9900000000000001e-07 2023-11-25T04:24:33.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:33.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094938+0000", 2023-11-25T04:24:33.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9599999999999999e-06 2023-11-25T04:24:33.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.995 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:33.995 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094944+0000", 2023-11-25T04:24:33.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2219999999999999e-06 2023-11-25T04:24:33.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.995 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:33.995 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094950+0000", 2023-11-25T04:24:33.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1329999999999998e-06 2023-11-25T04:24:33.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.995 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:33.995 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094993+0000", 2023-11-25T04:24:33.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3305e-05 2023-11-25T04:24:33.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.996 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:33.996 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095107+0000", 2023-11-25T04:24:33.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011449400000000001 2023-11-25T04:24:33.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.996 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:33.996 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095176+0000", 2023-11-25T04:24:33.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8330000000000005e-05 2023-11-25T04:24:33.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.997 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:33.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107939+0000", 2023-11-25T04:24:33.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012763801 2023-11-25T04:24:33.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.997 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:33.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109128+0000", 2023-11-25T04:24:33.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011884770000000001 2023-11-25T04:24:33.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.997 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:33.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109166+0000", 2023-11-25T04:24:33.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7948999999999999e-05 2023-11-25T04:24:33.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:33.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109198+0000", 2023-11-25T04:24:33.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1656000000000003e-05 2023-11-25T04:24:33.998 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:33.998 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:33.998 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:33.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:33.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.999 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2872 4.32 4:4fb002bc:::1000000111d.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:33.999 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.094961+0000", 2023-11-25T04:24:33.999 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.8248261249999995, 2023-11-25T04:24:33.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014631564999999999, 2023-11-25T04:24:33.999 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:33.999 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:33.999 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:33.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:33.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:33.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094961+0000", 2023-11-25T04:24:34.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:34.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:34.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094961+0000", 2023-11-25T04:24:34.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:34.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:34.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094961+0000", 2023-11-25T04:24:34.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9299999999999999e-07 2023-11-25T04:24:34.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.001 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:34.001 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094963+0000", 2023-11-25T04:24:34.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9209999999999999e-06 2023-11-25T04:24:34.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.001 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:34.001 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094968+0000", 2023-11-25T04:24:34.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.502e-06 2023-11-25T04:24:34.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:34.002 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094973+0000", 2023-11-25T04:24:34.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8559999999999998e-06 2023-11-25T04:24:34.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:34.002 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107722+0000", 2023-11-25T04:24:34.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012748534000000001 2023-11-25T04:24:34.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:34.002 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107844+0000", 2023-11-25T04:24:34.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012233400000000001 2023-11-25T04:24:34.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:34.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109469+0000", 2023-11-25T04:24:34.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001625175 2023-11-25T04:24:34.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.003 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.003 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:34.003 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109518+0000", 2023-11-25T04:24:34.003 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8980000000000002e-05 2023-11-25T04:24:34.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:34.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109592+0000", 2023-11-25T04:24:34.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3869999999999996e-05 2023-11-25T04:24:34.004 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:34.004 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:34.004 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:34.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.004 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2873 4.13 4:cae4f303:::1000000111f.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:34.004 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.094991+0000", 2023-11-25T04:24:34.004 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.8247955329999996, 2023-11-25T04:24:34.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015101692, 2023-11-25T04:24:34.005 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:34.005 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:34.005 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:34.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:34.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094991+0000", 2023-11-25T04:24:34.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:34.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.006 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:34.006 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094991+0000", 2023-11-25T04:24:34.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:34.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.006 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:34.006 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094992+0000", 2023-11-25T04:24:34.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7e-07 2023-11-25T04:24:34.006 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.006 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:34.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094994+0000", 2023-11-25T04:24:34.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2110000000000001e-06 2023-11-25T04:24:34.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:34.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095000+0000", 2023-11-25T04:24:34.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.241e-06 2023-11-25T04:24:34.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:34.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095006+0000", 2023-11-25T04:24:34.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3659999999999997e-06 2023-11-25T04:24:34.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:34.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108369+0000", 2023-11-25T04:24:34.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01336247 2023-11-25T04:24:34.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:34.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108492+0000", 2023-11-25T04:24:34.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000123192 2023-11-25T04:24:34.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:34.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109975+0000", 2023-11-25T04:24:34.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001482894 2023-11-25T04:24:34.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:34.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110024+0000", 2023-11-25T04:24:34.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9097000000000001e-05 2023-11-25T04:24:34.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:34.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110093+0000", 2023-11-25T04:24:34.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8751000000000006e-05 2023-11-25T04:24:34.009 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:34.009 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:34.010 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:34.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.010 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2874 4.14 4:2b557d2e:::10000001121.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:34.010 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095019+0000", 2023-11-25T04:24:34.010 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.8247680069999994, 2023-11-25T04:24:34.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014998296, 2023-11-25T04:24:34.010 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:34.011 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:34.011 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:34.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:34.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095019+0000", 2023-11-25T04:24:34.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:34.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:34.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095019+0000", 2023-11-25T04:24:34.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:34.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:34.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095019+0000", 2023-11-25T04:24:34.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7300000000000001e-07 2023-11-25T04:24:34.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:34.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095021+0000", 2023-11-25T04:24:34.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0099999999999998e-06 2023-11-25T04:24:34.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:34.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095028+0000", 2023-11-25T04:24:34.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7870000000000004e-06 2023-11-25T04:24:34.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:34.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095033+0000", 2023-11-25T04:24:34.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2039999999999996e-06 2023-11-25T04:24:34.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:34.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107740+0000", 2023-11-25T04:24:34.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012706693 2023-11-25T04:24:34.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:34.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107817+0000", 2023-11-25T04:24:34.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7599e-05 2023-11-25T04:24:34.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:34.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107961+0000", 2023-11-25T04:24:34.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000143194 2023-11-25T04:24:34.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:34.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109364+0000", 2023-11-25T04:24:34.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001403226 2023-11-25T04:24:34.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:34.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109507+0000", 2023-11-25T04:24:34.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000142991 2023-11-25T04:24:34.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:34.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109986+0000", 2023-11-25T04:24:34.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00047906899999999999 2023-11-25T04:24:34.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.016 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:34.016 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110017+0000", 2023-11-25T04:24:34.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1050000000000003e-05 2023-11-25T04:24:34.016 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:34.016 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:34.016 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:34.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.016 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2875 4.d 4:b17d5dc6:::10000001123.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:34.016 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095049+0000", 2023-11-25T04:24:34.016 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.8247380829999997, 2023-11-25T04:24:34.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014882886, 2023-11-25T04:24:34.017 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:34.017 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:34.017 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:34.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:34.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095049+0000", 2023-11-25T04:24:34.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:34.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:34.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095049+0000", 2023-11-25T04:24:34.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:34.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:34.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095049+0000", 2023-11-25T04:24:34.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6100000000000001e-07 2023-11-25T04:24:34.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:34.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095051+0000", 2023-11-25T04:24:34.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1449999999999998e-06 2023-11-25T04:24:34.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:34.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095058+0000", 2023-11-25T04:24:34.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.601e-06 2023-11-25T04:24:34.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:34.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095064+0000", 2023-11-25T04:24:34.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1079999999999998e-06 2023-11-25T04:24:34.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:34.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108015+0000", 2023-11-25T04:24:34.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012951582 2023-11-25T04:24:34.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.020 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:34.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108260+0000", 2023-11-25T04:24:34.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024445199999999998 2023-11-25T04:24:34.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.020 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:34.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109825+0000", 2023-11-25T04:24:34.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0015651759999999999 2023-11-25T04:24:34.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.020 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:34.020 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109866+0000", 2023-11-25T04:24:34.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0629e-05 2023-11-25T04:24:34.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.021 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:34.021 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109931+0000", 2023-11-25T04:24:34.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5732e-05 2023-11-25T04:24:34.021 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:34.021 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:34.021 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:34.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.021 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2876 4.3b 4:dc871776:::10000001125.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:34.022 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095077+0000", 2023-11-25T04:24:34.022 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.8247094619999995, 2023-11-25T04:24:34.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014500944, 2023-11-25T04:24:34.022 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:34.022 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:34.022 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:34.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:34.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095077+0000", 2023-11-25T04:24:34.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:34.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:34.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095077+0000", 2023-11-25T04:24:34.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:34.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:34.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095078+0000", 2023-11-25T04:24:34.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9799999999999999e-07 2023-11-25T04:24:34.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:34.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095079+0000", 2023-11-25T04:24:34.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.877e-06 2023-11-25T04:24:34.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:34.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095086+0000", 2023-11-25T04:24:34.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2439999999999998e-06 2023-11-25T04:24:34.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:34.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095094+0000", 2023-11-25T04:24:34.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9710000000000004e-06 2023-11-25T04:24:34.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:34.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107732+0000", 2023-11-25T04:24:34.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012638104000000001 2023-11-25T04:24:34.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:34.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107854+0000", 2023-11-25T04:24:34.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012261600000000001 2023-11-25T04:24:34.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:34.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107941+0000", 2023-11-25T04:24:34.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6935999999999996e-05 2023-11-25T04:24:34.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:34.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108871+0000", 2023-11-25T04:24:34.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00092955499999999999 2023-11-25T04:24:34.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:34.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109483+0000", 2023-11-25T04:24:34.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00061206100000000001 2023-11-25T04:24:34.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:34.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109527+0000", 2023-11-25T04:24:34.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4112e-05 2023-11-25T04:24:34.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:34.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109578+0000", 2023-11-25T04:24:34.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1069999999999997e-05 2023-11-25T04:24:34.027 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:34.027 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:34.027 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:34.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.027 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2878 4.3e 4:7ff98b9d:::10000001129.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:34.027 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095101+0000", 2023-11-25T04:24:34.027 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.8246856789999999, 2023-11-25T04:24:34.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015015137, 2023-11-25T04:24:34.028 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:34.028 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:34.028 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:34.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:34.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095101+0000", 2023-11-25T04:24:34.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:34.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:34.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095101+0000", 2023-11-25T04:24:34.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:34.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:34.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095101+0000", 2023-11-25T04:24:34.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7399999999999999e-07 2023-11-25T04:24:34.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:34.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095103+0000", 2023-11-25T04:24:34.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7600000000000001e-06 2023-11-25T04:24:34.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:34.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095109+0000", 2023-11-25T04:24:34.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8189999999999997e-06 2023-11-25T04:24:34.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:34.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095114+0000", 2023-11-25T04:24:34.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6770000000000003e-06 2023-11-25T04:24:34.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:34.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107724+0000", 2023-11-25T04:24:34.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012610909999999999 2023-11-25T04:24:34.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:34.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108483+0000", 2023-11-25T04:24:34.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000758407 2023-11-25T04:24:34.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:34.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108568+0000", 2023-11-25T04:24:34.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4707000000000001e-05 2023-11-25T04:24:34.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:34.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109687+0000", 2023-11-25T04:24:34.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011192750000000001 2023-11-25T04:24:34.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:34.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110049+0000", 2023-11-25T04:24:34.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00036210600000000001 2023-11-25T04:24:34.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:34.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110086+0000", 2023-11-25T04:24:34.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7500999999999999e-05 2023-11-25T04:24:34.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.033 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:34.033 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110116+0000", 2023-11-25T04:24:34.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9601e-05 2023-11-25T04:24:34.033 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:34.033 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:34.033 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:34.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.033 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2879 4.2a 4:5524ace2:::1000000112b.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:34.033 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095120+0000", 2023-11-25T04:24:34.033 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.8246668990000003, 2023-11-25T04:24:34.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014587937, 2023-11-25T04:24:34.033 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:34.034 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:34.034 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:34.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:34.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095120+0000", 2023-11-25T04:24:34.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:34.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:34.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095120+0000", 2023-11-25T04:24:34.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:34.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:34.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095120+0000", 2023-11-25T04:24:34.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5600000000000001e-07 2023-11-25T04:24:34.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:34.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095122+0000", 2023-11-25T04:24:34.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.694e-06 2023-11-25T04:24:34.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:34.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095127+0000", 2023-11-25T04:24:34.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5099999999999998e-06 2023-11-25T04:24:34.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:34.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095132+0000", 2023-11-25T04:24:34.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6210000000000003e-06 2023-11-25T04:24:34.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:34.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107729+0000", 2023-11-25T04:24:34.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012596931 2023-11-25T04:24:34.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.037 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:34.037 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107856+0000", 2023-11-25T04:24:34.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012707699999999999 2023-11-25T04:24:34.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.037 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:34.037 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109600+0000", 2023-11-25T04:24:34.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001744298 2023-11-25T04:24:34.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.037 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:34.037 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109641+0000", 2023-11-25T04:24:34.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0769000000000002e-05 2023-11-25T04:24:34.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.038 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:34.038 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109708+0000", 2023-11-25T04:24:34.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6680999999999994e-05 2023-11-25T04:24:34.038 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:34.038 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:34.038 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:34.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.038 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2882 4.1c 4:3a7c2e07:::10000001131.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:34.039 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095141+0000", 2023-11-25T04:24:34.039 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.8246455239999992, 2023-11-25T04:24:34.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014949469999999999, 2023-11-25T04:24:34.039 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:34.039 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:34.039 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:34.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:34.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095141+0000", 2023-11-25T04:24:34.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:34.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:34.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095141+0000", 2023-11-25T04:24:34.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:34.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:34.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095142+0000", 2023-11-25T04:24:34.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1899999999999998e-07 2023-11-25T04:24:34.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:34.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095143+0000", 2023-11-25T04:24:34.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7770000000000001e-06 2023-11-25T04:24:34.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:34.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095149+0000", 2023-11-25T04:24:34.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9909999999999999e-06 2023-11-25T04:24:34.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:34.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095154+0000", 2023-11-25T04:24:34.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3880000000000002e-06 2023-11-25T04:24:34.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:34.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107736+0000", 2023-11-25T04:24:34.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012582477999999999 2023-11-25T04:24:34.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:34.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107868+0000", 2023-11-25T04:24:34.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013134400000000001 2023-11-25T04:24:34.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:34.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107959+0000", 2023-11-25T04:24:34.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1253000000000002e-05 2023-11-25T04:24:34.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:34.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109325+0000", 2023-11-25T04:24:34.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0013662520000000001 2023-11-25T04:24:34.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:34.043 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110023+0000", 2023-11-25T04:24:34.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00069838799999999998 2023-11-25T04:24:34.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.043 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:34.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110061+0000", 2023-11-25T04:24:34.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7867999999999997e-05 2023-11-25T04:24:34.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:34.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110091+0000", 2023-11-25T04:24:34.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9312e-05 2023-11-25T04:24:34.044 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:34.044 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:34.044 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:34.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.044 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2883 4.3e 4:7d82be99:::10000001133.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:34.045 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095942+0000", 2023-11-25T04:24:34.045 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.8238442379999995, 2023-11-25T04:24:34.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014290573000000001, 2023-11-25T04:24:34.045 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:34.045 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:34.045 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:34.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:34.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095942+0000", 2023-11-25T04:24:34.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:34.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:34.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095942+0000", 2023-11-25T04:24:34.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:34.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:34.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095944+0000", 2023-11-25T04:24:34.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.632e-06 2023-11-25T04:24:34.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:34.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095947+0000", 2023-11-25T04:24:34.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7760000000000002e-06 2023-11-25T04:24:34.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:34.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095954+0000", 2023-11-25T04:24:34.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5379999999999997e-06 2023-11-25T04:24:34.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:34.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095960+0000", 2023-11-25T04:24:34.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9209999999999997e-06 2023-11-25T04:24:34.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:34.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108809+0000", 2023-11-25T04:24:34.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012848972 2023-11-25T04:24:34.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:34.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108921+0000", 2023-11-25T04:24:34.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000111767 2023-11-25T04:24:34.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:34.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109008+0000", 2023-11-25T04:24:34.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6576999999999995e-05 2023-11-25T04:24:34.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:34.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110124+0000", 2023-11-25T04:24:34.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011163709999999999 2023-11-25T04:24:34.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:34.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110155+0000", 2023-11-25T04:24:34.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1248999999999998e-05 2023-11-25T04:24:34.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.049 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:34.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110202+0000", 2023-11-25T04:24:34.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7097e-05 2023-11-25T04:24:34.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:34.050 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110233+0000", 2023-11-25T04:24:34.050 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0673000000000003e-05 2023-11-25T04:24:34.050 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:34.050 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:34.050 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:34.050 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.050 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2886 4.25 4:a579baeb:::10000001139.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:34.050 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095994+0000", 2023-11-25T04:24:34.051 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 8.8237929099999999, 2023-11-25T04:24:34.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013825356, 2023-11-25T04:24:34.051 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:34.051 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:34.051 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:34.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:34.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095994+0000", 2023-11-25T04:24:34.051 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:34.051 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.051 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.051 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:34.051 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095994+0000", 2023-11-25T04:24:34.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:34.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:34.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095996+0000", 2023-11-25T04:24:34.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8780000000000001e-06 2023-11-25T04:24:34.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:34.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095998+0000", 2023-11-25T04:24:34.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1560000000000002e-06 2023-11-25T04:24:34.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:34.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.096003+0000", 2023-11-25T04:24:34.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1080000000000001e-06 2023-11-25T04:24:34.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:34.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.096010+0000", 2023-11-25T04:24:34.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0650000000000001e-06 2023-11-25T04:24:34.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.053 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.053 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:34.053 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107742+0000", 2023-11-25T04:24:34.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011731691000000001 2023-11-25T04:24:34.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.054 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:34.054 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108063+0000", 2023-11-25T04:24:34.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00032168600000000001 2023-11-25T04:24:34.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.054 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:34.054 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109689+0000", 2023-11-25T04:24:34.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001625556 2023-11-25T04:24:34.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.054 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:34.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109740+0000", 2023-11-25T04:24:34.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1125000000000003e-05 2023-11-25T04:24:34.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:34.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109819+0000", 2023-11-25T04:24:34.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9091000000000006e-05 2023-11-25T04:24:34.055 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:34.055 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:34.055 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:34.055 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:34.055 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:34.055 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:34.057 INFO:tasks.workunit.client.0.smithi067.stdout: [CC] doio 2023-11-25T04:24:34.057 INFO:tasks.workunit.client.0.smithi067.stdout: [CC] fsstress 2023-11-25T04:24:34.057 INFO:tasks.workunit.client.0.smithi067.stdout: [CC] fsx 2023-11-25T04:24:34.057 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:24:34.058 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:34.055+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:34.158 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:34.156+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:34.169 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:34.170 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:24:34.170 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2023-11-25T04:24:34.170 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:34.178 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:24:34.184 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2023-11-25T04:24:34.264 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:34.263+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:34.292 INFO:tasks.workunit.client.0.smithi067.stderr:fsstress.c: In function 'attr_list_path': 2023-11-25T04:24:34.292 INFO:tasks.workunit.client.0.smithi067.stderr:fsstress.c:859:9: warning: 'attr_list' is deprecated: Use listxattr or llistxattr instead [-Wdeprecated-declarations] 2023-11-25T04:24:34.292 INFO:tasks.workunit.client.0.smithi067.stderr: 859 | rval = attr_list(name->path, buffer, buffersize, flags, cursor); 2023-11-25T04:24:34.292 INFO:tasks.workunit.client.0.smithi067.stderr: | ^~~~ 2023-11-25T04:24:34.292 INFO:tasks.workunit.client.0.smithi067.stderr:In file included from ../src/global.h:25, 2023-11-25T04:24:34.292 INFO:tasks.workunit.client.0.smithi067.stderr: from fsstress.c:12: 2023-11-25T04:24:34.292 INFO:tasks.workunit.client.0.smithi067.stderr:/usr/include/attr/attributes.h:165:12: note: declared here 2023-11-25T04:24:34.292 INFO:tasks.workunit.client.0.smithi067.stderr: 165 | extern int attr_list(const char *__path, char *__buffer, const int __buffersize, 2023-11-25T04:24:34.292 INFO:tasks.workunit.client.0.smithi067.stderr: | ^~~~~~~~~ 2023-11-25T04:24:34.292 INFO:tasks.workunit.client.0.smithi067.stderr:fsstress.c: In function 'attr_remove_path': 2023-11-25T04:24:34.292 INFO:tasks.workunit.client.0.smithi067.stderr:fsstress.c:878:9: warning: 'attr_remove' is deprecated: Use removexattr or lremovexattr instead [-Wdeprecated-declarations] 2023-11-25T04:24:34.293 INFO:tasks.workunit.client.0.smithi067.stderr: 878 | rval = attr_remove(name->path, attrname, flags); 2023-11-25T04:24:34.293 INFO:tasks.workunit.client.0.smithi067.stderr: | ^~~~ 2023-11-25T04:24:34.293 INFO:tasks.workunit.client.0.smithi067.stderr:In file included from ../src/global.h:25, 2023-11-25T04:24:34.293 INFO:tasks.workunit.client.0.smithi067.stderr: from fsstress.c:12: 2023-11-25T04:24:34.293 INFO:tasks.workunit.client.0.smithi067.stderr:/usr/include/attr/attributes.h:152:12: note: declared here 2023-11-25T04:24:34.293 INFO:tasks.workunit.client.0.smithi067.stderr: 152 | extern int attr_remove (const char *__path, const char *__attrname, 2023-11-25T04:24:34.293 INFO:tasks.workunit.client.0.smithi067.stderr: | ^~~~~~~~~~~ 2023-11-25T04:24:34.293 INFO:tasks.workunit.client.0.smithi067.stderr:fsstress.c: In function 'attr_set_path': 2023-11-25T04:24:34.293 INFO:tasks.workunit.client.0.smithi067.stderr:fsstress.c:898:9: warning: 'attr_set' is deprecated: Use setxattr or lsetxattr instead [-Wdeprecated-declarations] 2023-11-25T04:24:34.293 INFO:tasks.workunit.client.0.smithi067.stderr: 898 | rval = attr_set(name->path, attrname, attrvalue, valuelength, flags); 2023-11-25T04:24:34.293 INFO:tasks.workunit.client.0.smithi067.stderr: | ^~~~ 2023-11-25T04:24:34.293 INFO:tasks.workunit.client.0.smithi067.stderr:In file included from ../src/global.h:25, 2023-11-25T04:24:34.294 INFO:tasks.workunit.client.0.smithi067.stderr: from fsstress.c:12: 2023-11-25T04:24:34.294 INFO:tasks.workunit.client.0.smithi067.stderr:/usr/include/attr/attributes.h:139:12: note: declared here 2023-11-25T04:24:34.294 INFO:tasks.workunit.client.0.smithi067.stderr: 139 | extern int attr_set (const char *__path, const char *__attrname, 2023-11-25T04:24:34.294 INFO:tasks.workunit.client.0.smithi067.stderr: | ^~~~~~~~ 2023-11-25T04:24:34.310 INFO:tasks.workunit.client.0.smithi067.stderr:fsstress.c: In function 'allocsp_f': 2023-11-25T04:24:34.310 INFO:tasks.workunit.client.0.smithi067.stderr:fsstress.c:2047:32: error: 'XFS_IOC_ALLOCSP64' undeclared (first use in this function); did you mean 'XFS_IOC_RESVSP64'? 2023-11-25T04:24:34.310 INFO:tasks.workunit.client.0.smithi067.stderr: 2047 | e = xfsctl(f.path, fd, XFS_IOC_ALLOCSP64, &fl) < 0 ? errno : 0; 2023-11-25T04:24:34.310 INFO:tasks.workunit.client.0.smithi067.stderr: | ^~~~~~~~~~~~~~~~~ 2023-11-25T04:24:34.310 INFO:tasks.workunit.client.0.smithi067.stderr: | XFS_IOC_RESVSP64 2023-11-25T04:24:34.310 INFO:tasks.workunit.client.0.smithi067.stderr:fsstress.c:2047:32: note: each undeclared identifier is reported only once for each function it appears in 2023-11-25T04:24:34.325 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:34.326 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:24:34.326 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2023-11-25T04:24:34.326 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2023-11-25T04:24:34.326 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:34.334 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:24:34.340 INFO:tasks.workunit.client.0.smithi067.stderr:fsstress.c: In function 'freesp_f': 2023-11-25T04:24:34.340 INFO:tasks.workunit.client.0.smithi067.stderr:fsstress.c:3640:32: error: 'XFS_IOC_FREESP64' undeclared (first use in this function); did you mean 'XFS_IOC_RESVSP64'? 2023-11-25T04:24:34.340 INFO:tasks.workunit.client.0.smithi067.stderr: 3640 | e = xfsctl(f.path, fd, XFS_IOC_FREESP64, &fl) < 0 ? errno : 0; 2023-11-25T04:24:34.340 INFO:tasks.workunit.client.0.smithi067.stderr: | ^~~~~~~~~~~~~~~~ 2023-11-25T04:24:34.340 INFO:tasks.workunit.client.0.smithi067.stderr: | XFS_IOC_RESVSP64 2023-11-25T04:24:34.375 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:34.373+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:34.395 INFO:tasks.workunit.client.0.smithi067.stderr:gmake[3]: *** [Makefile:46: fsstress] Error 1 2023-11-25T04:24:34.395 INFO:tasks.workunit.client.0.smithi067.stderr:gmake[3]: *** Waiting for unfinished jobs.... 2023-11-25T04:24:34.460 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:34.459+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:34.481 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:34.482 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2023-11-25T04:24:34.482 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2023-11-25T04:24:34.482 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2023-11-25T04:24:34.482 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.482 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:34.482 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:24:34.482 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 107.447401791, 2023-11-25T04:24:34.482 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01072943, 2023-11-25T04:24:34.482 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:34.482 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:34.482 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:34.483 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.483 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:34.483 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:24:34.483 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:34.483 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.483 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.483 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:34.483 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:24:34.483 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:34.484 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.484 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.484 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:34.484 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031829+0000", 2023-11-25T04:24:34.484 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.666e-06 2023-11-25T04:24:34.484 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.484 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.484 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:34.484 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031839+0000", 2023-11-25T04:24:34.484 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9490000000000008e-06 2023-11-25T04:24:34.484 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.484 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.485 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:34.485 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031857+0000", 2023-11-25T04:24:34.485 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8093000000000001e-05 2023-11-25T04:24:34.485 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.485 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.485 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:34.485 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031882+0000", 2023-11-25T04:24:34.485 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4294000000000002e-05 2023-11-25T04:24:34.485 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.485 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.485 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:34.485 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033130+0000", 2023-11-25T04:24:34.485 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001248264 2023-11-25T04:24:34.486 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.486 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.486 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:34.486 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033426+0000", 2023-11-25T04:24:34.486 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029620999999999998 2023-11-25T04:24:34.486 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.486 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.486 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:34.486 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033814+0000", 2023-11-25T04:24:34.486 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000387742 2023-11-25T04:24:34.486 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.486 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.487 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:34.487 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036793+0000", 2023-11-25T04:24:34.487 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002979598 2023-11-25T04:24:34.487 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.487 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.487 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:34.487 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042455+0000", 2023-11-25T04:24:34.487 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0056619260000000003 2023-11-25T04:24:34.487 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.487 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.487 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:34.487 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042502+0000", 2023-11-25T04:24:34.488 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7139999999999999e-05 2023-11-25T04:24:34.488 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.488 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.488 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:34.488 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042557+0000", 2023-11-25T04:24:34.488 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4548e-05 2023-11-25T04:24:34.488 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:34.488 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:34.488 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:34.488 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.488 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.488 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:34.488 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:24:34.489 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 107.446203256, 2023-11-25T04:24:34.489 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011505202000000001, 2023-11-25T04:24:34.489 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:34.489 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:34.489 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:34.489 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.489 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:34.489 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:24:34.489 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:34.489 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.489 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.489 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:34.490 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:24:34.490 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:34.490 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.490 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.490 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:34.490 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033027+0000", 2023-11-25T04:24:34.490 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0699999999999999e-06 2023-11-25T04:24:34.490 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.490 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.490 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:34.490 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033035+0000", 2023-11-25T04:24:34.490 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0830000000000004e-06 2023-11-25T04:24:34.491 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.491 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.491 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:34.491 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033059+0000", 2023-11-25T04:24:34.491 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3975e-05 2023-11-25T04:24:34.491 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.491 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.491 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:34.491 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033482+0000", 2023-11-25T04:24:34.491 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042253700000000001 2023-11-25T04:24:34.491 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.491 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.492 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:34.492 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034520+0000", 2023-11-25T04:24:34.492 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001037991 2023-11-25T04:24:34.492 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.492 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.492 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:34.492 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034611+0000", 2023-11-25T04:24:34.492 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1113999999999995e-05 2023-11-25T04:24:34.492 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.492 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.492 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:34.492 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034769+0000", 2023-11-25T04:24:34.493 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000157641 2023-11-25T04:24:34.493 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.493 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.493 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:34.493 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038292+0000", 2023-11-25T04:24:34.493 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035231160000000002 2023-11-25T04:24:34.493 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.493 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.493 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:34.493 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044440+0000", 2023-11-25T04:24:34.493 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0061486980000000002 2023-11-25T04:24:34.493 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.494 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.494 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:34.494 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044495+0000", 2023-11-25T04:24:34.494 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4339000000000003e-05 2023-11-25T04:24:34.494 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.494 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.494 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:34.494 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044531+0000", 2023-11-25T04:24:34.494 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6637999999999997e-05 2023-11-25T04:24:34.494 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:34.494 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:34.494 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:34.495 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.495 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.495 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:26 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:34.495 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:34.495 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 107.445614494, 2023-11-25T04:24:34.495 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011086945000000001, 2023-11-25T04:24:34.495 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:34.495 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:34.495 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:34.495 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.495 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:34.495 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:34.496 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:34.496 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.496 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.496 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:34.496 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:34.496 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:34.496 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.496 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.496 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:34.496 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:34.496 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1699999999999998e-07 2023-11-25T04:24:34.496 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.497 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.497 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:34.497 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033627+0000", 2023-11-25T04:24:34.497 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.163e-05 2023-11-25T04:24:34.497 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.497 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.497 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:34.497 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033636+0000", 2023-11-25T04:24:34.497 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6859999999999995e-06 2023-11-25T04:24:34.497 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.498 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.498 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:34.498 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033642+0000", 2023-11-25T04:24:34.498 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3149999999999997e-06 2023-11-25T04:24:34.498 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.498 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.498 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:34.498 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035414+0000", 2023-11-25T04:24:34.498 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0017716380000000001 2023-11-25T04:24:34.498 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.498 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.499 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:34.499 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035539+0000", 2023-11-25T04:24:34.499 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000125477 2023-11-25T04:24:34.499 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.499 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.499 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:34.499 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035672+0000", 2023-11-25T04:24:34.499 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000133316 2023-11-25T04:24:34.499 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.499 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.499 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:34.499 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038649+0000", 2023-11-25T04:24:34.499 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029764679999999999 2023-11-25T04:24:34.500 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.500 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.500 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:34.500 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044578+0000", 2023-11-25T04:24:34.500 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0059293510000000002 2023-11-25T04:24:34.500 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.500 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.500 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:34.500 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044651+0000", 2023-11-25T04:24:34.500 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2513000000000002e-05 2023-11-25T04:24:34.500 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.500 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.501 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:34.501 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044702+0000", 2023-11-25T04:24:34.501 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1034e-05 2023-11-25T04:24:34.501 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:34.501 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:34.501 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:34.501 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.501 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.501 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:29 3.b 3:d0630e4c:::mds0_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:34.501 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:34.501 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 107.445576574, 2023-11-25T04:24:34.501 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010980227, 2023-11-25T04:24:34.501 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:34.502 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:34.502 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:34.502 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.502 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:34.502 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:34.502 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:34.502 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.502 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.502 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:34.502 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:34.502 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:34.502 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.502 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.502 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:34.503 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:34.503 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3499999999999996e-07 2023-11-25T04:24:34.503 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.503 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.503 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:34.503 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033658+0000", 2023-11-25T04:24:34.503 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1280000000000001e-06 2023-11-25T04:24:34.503 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.503 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.505 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:34.505 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033667+0000", 2023-11-25T04:24:34.505 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1430000000000004e-06 2023-11-25T04:24:34.505 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.505 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.505 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:34.505 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033678+0000", 2023-11-25T04:24:34.505 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0906e-05 2023-11-25T04:24:34.505 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.506 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.506 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:34.506 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034456+0000", 2023-11-25T04:24:34.506 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00077833899999999996 2023-11-25T04:24:34.506 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.506 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.506 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:34.506 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034575+0000", 2023-11-25T04:24:34.506 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011926000000000001 2023-11-25T04:24:34.506 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.506 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.506 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:34.507 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035001+0000", 2023-11-25T04:24:34.507 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042550999999999998 2023-11-25T04:24:34.507 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.507 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.507 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:34.507 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.037923+0000", 2023-11-25T04:24:34.507 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029219670000000001 2023-11-25T04:24:34.507 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.507 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.507 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:34.507 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044538+0000", 2023-11-25T04:24:34.507 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0066152349999999997 2023-11-25T04:24:34.507 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.508 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.508 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:34.508 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044594+0000", 2023-11-25T04:24:34.508 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5593000000000001e-05 2023-11-25T04:24:34.508 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.508 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.508 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:34.508 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044633+0000", 2023-11-25T04:24:34.508 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9511000000000002e-05 2023-11-25T04:24:34.508 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:34.508 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:34.508 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:34.508 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.509 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.509 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:34 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:34.509 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:24:34.509 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 107.44532646099999, 2023-11-25T04:24:34.509 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010920445000000001, 2023-11-25T04:24:34.509 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:34.509 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:34.509 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:34.509 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.509 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:34.509 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:24:34.509 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:34.510 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.510 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.510 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:34.510 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:24:34.510 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:34.510 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.510 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.510 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:34.510 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033904+0000", 2023-11-25T04:24:34.510 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6599999999999994e-07 2023-11-25T04:24:34.510 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.510 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.511 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:34.511 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033909+0000", 2023-11-25T04:24:34.511 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5639999999999996e-06 2023-11-25T04:24:34.511 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.511 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.511 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:34.511 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033923+0000", 2023-11-25T04:24:34.511 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3028e-05 2023-11-25T04:24:34.511 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.511 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.511 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:34.511 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034043+0000", 2023-11-25T04:24:34.512 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000120328 2023-11-25T04:24:34.512 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.512 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.512 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:34.512 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035576+0000", 2023-11-25T04:24:34.512 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015333720000000001 2023-11-25T04:24:34.512 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.512 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.512 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:34.512 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035699+0000", 2023-11-25T04:24:34.512 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000123001 2023-11-25T04:24:34.512 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.513 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.513 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:34.513 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044823+0000", 2023-11-25T04:24:34.513 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0091241859999999994 2023-11-25T04:24:34.513 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:34.513 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:34.513 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:34.513 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.513 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.513 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:118 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals,omap-rm-keys] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e44)", 2023-11-25T04:24:34.513 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:17.112648+0000", 2023-11-25T04:24:34.513 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 17.366581119999999, 2023-11-25T04:24:34.513 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011250779000000001, 2023-11-25T04:24:34.514 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:34.514 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:34.514 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:34.514 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.514 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:34.514 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.112648+0000", 2023-11-25T04:24:34.514 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:34.514 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.514 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.514 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:34.514 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.112648+0000", 2023-11-25T04:24:34.514 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:34.515 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.515 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.515 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:34.515 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.112649+0000", 2023-11-25T04:24:34.515 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0070000000000001e-06 2023-11-25T04:24:34.515 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.515 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.515 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:34.515 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113182+0000", 2023-11-25T04:24:34.515 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00053268100000000002 2023-11-25T04:24:34.515 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.516 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.516 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:34.516 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113196+0000", 2023-11-25T04:24:34.516 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3742e-05 2023-11-25T04:24:34.516 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.516 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.516 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:34.516 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113398+0000", 2023-11-25T04:24:34.516 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000202726 2023-11-25T04:24:34.516 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.516 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.516 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:34.516 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113447+0000", 2023-11-25T04:24:34.517 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8655000000000001e-05 2023-11-25T04:24:34.517 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.517 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.517 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:34.517 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113529+0000", 2023-11-25T04:24:34.517 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1891999999999995e-05 2023-11-25T04:24:34.517 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.517 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.517 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:34.517 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123819+0000", 2023-11-25T04:24:34.517 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010290192 2023-11-25T04:24:34.517 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.517 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.518 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:34.518 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123852+0000", 2023-11-25T04:24:34.518 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2425999999999998e-05 2023-11-25T04:24:34.518 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.518 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.518 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:34.518 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123899+0000", 2023-11-25T04:24:34.518 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7457999999999999e-05 2023-11-25T04:24:34.518 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:34.518 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:34.519 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:34.519 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.519 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.519 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:154 3.1e 3:7bf7e7b5:::200.00000006:head [write 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:34.519 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:21.886430+0000", 2023-11-25T04:24:34.519 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 12.592799854000001, 2023-11-25T04:24:34.519 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.017800107999999999, 2023-11-25T04:24:34.519 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:34.519 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:34.519 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:34.519 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.520 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:34.520 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.886430+0000", 2023-11-25T04:24:34.520 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:34.520 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.520 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.520 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:34.520 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.886430+0000", 2023-11-25T04:24:34.520 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:34.520 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.520 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.521 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:34.521 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.886431+0000", 2023-11-25T04:24:34.521 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.178e-06 2023-11-25T04:24:34.521 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.521 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.521 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:34.521 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.892277+0000", 2023-11-25T04:24:34.521 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0058467409999999999 2023-11-25T04:24:34.521 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.521 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.521 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:34.521 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.892288+0000", 2023-11-25T04:24:34.522 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0183000000000001e-05 2023-11-25T04:24:34.522 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.522 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.522 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:34.522 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.892292+0000", 2023-11-25T04:24:34.522 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7829999999999998e-06 2023-11-25T04:24:34.522 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.522 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.522 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:34.522 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.892362+0000", 2023-11-25T04:24:34.522 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0034000000000001e-05 2023-11-25T04:24:34.522 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.523 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.523 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:34.523 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.892432+0000", 2023-11-25T04:24:34.523 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9648999999999997e-05 2023-11-25T04:24:34.523 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.523 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.523 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:34.523 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.892550+0000", 2023-11-25T04:24:34.523 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011825 2023-11-25T04:24:34.523 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.523 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.523 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:34.523 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.899106+0000", 2023-11-25T04:24:34.524 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0065556759999999999 2023-11-25T04:24:34.524 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.524 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.524 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:34.524 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.904145+0000", 2023-11-25T04:24:34.524 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0050393490000000003 2023-11-25T04:24:34.524 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.524 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.524 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:34.524 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.904200+0000", 2023-11-25T04:24:34.524 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4234999999999998e-05 2023-11-25T04:24:34.524 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.525 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.525 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:34.525 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.904230+0000", 2023-11-25T04:24:34.525 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0029999999999999e-05 2023-11-25T04:24:34.525 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:34.525 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:34.525 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:34.525 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.525 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.525 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:576 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals,omap-rm-keys] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:34.525 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:22.111407+0000", 2023-11-25T04:24:34.525 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 12.367822248, 2023-11-25T04:24:34.526 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.018987317, 2023-11-25T04:24:34.526 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:34.526 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:34.526 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:34.526 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.526 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:34.526 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111407+0000", 2023-11-25T04:24:34.526 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:34.526 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.526 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.526 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:34.527 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111407+0000", 2023-11-25T04:24:34.527 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:34.527 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.527 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.527 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:34.527 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111408+0000", 2023-11-25T04:24:34.527 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8000000000000004e-07 2023-11-25T04:24:34.527 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.527 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.527 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:34.527 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111659+0000", 2023-11-25T04:24:34.527 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000250537 2023-11-25T04:24:34.528 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.528 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.528 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:34.528 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111666+0000", 2023-11-25T04:24:34.528 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.973e-06 2023-11-25T04:24:34.528 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.528 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.528 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:34.528 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111670+0000", 2023-11-25T04:24:34.528 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.668e-06 2023-11-25T04:24:34.528 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.528 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.528 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:34.529 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111709+0000", 2023-11-25T04:24:34.529 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8800000000000001e-05 2023-11-25T04:24:34.529 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.529 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.529 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:34.529 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111761+0000", 2023-11-25T04:24:34.529 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1495000000000001e-05 2023-11-25T04:24:34.529 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.529 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.529 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:34.529 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.130255+0000", 2023-11-25T04:24:34.529 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.018494345999999998 2023-11-25T04:24:34.529 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.530 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.530 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:34.530 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.130311+0000", 2023-11-25T04:24:34.530 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5902000000000001e-05 2023-11-25T04:24:34.530 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.530 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.530 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:34.530 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.130394+0000", 2023-11-25T04:24:34.530 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3715999999999996e-05 2023-11-25T04:24:34.530 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:34.530 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:34.531 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:34.531 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.531 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.531 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:1067 4.21 4:869a7600:::100000011fa.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:34.531 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:22.155038+0000", 2023-11-25T04:24:34.531 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 12.32419091, 2023-11-25T04:24:34.531 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011313148, 2023-11-25T04:24:34.531 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:34.531 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:34.531 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:34.531 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.531 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:34.532 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155038+0000", 2023-11-25T04:24:34.532 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:34.532 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.532 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.532 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:34.532 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155038+0000", 2023-11-25T04:24:34.532 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:34.532 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.532 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.532 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:34.532 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155039+0000", 2023-11-25T04:24:34.532 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0200000000000002e-07 2023-11-25T04:24:34.533 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.533 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.533 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:34.533 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155049+0000", 2023-11-25T04:24:34.533 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0295000000000001e-05 2023-11-25T04:24:34.533 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.533 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.533 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:34.533 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155058+0000", 2023-11-25T04:24:34.533 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.208e-06 2023-11-25T04:24:34.533 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.534 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.534 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:34.534 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155064+0000", 2023-11-25T04:24:34.534 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4489999999999996e-06 2023-11-25T04:24:34.534 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.534 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.534 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:34.534 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155104+0000", 2023-11-25T04:24:34.534 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0164999999999999e-05 2023-11-25T04:24:34.534 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.534 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.535 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:34.535 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155147+0000", 2023-11-25T04:24:34.535 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2576e-05 2023-11-25T04:24:34.535 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.535 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.535 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:34.535 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155212+0000", 2023-11-25T04:24:34.535 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5516000000000001e-05 2023-11-25T04:24:34.535 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.535 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.535 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:34.535 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.156025+0000", 2023-11-25T04:24:34.536 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00081248599999999996 2023-11-25T04:24:34.536 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.536 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.536 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:34.536 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.166250+0000", 2023-11-25T04:24:34.536 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010224736999999999 2023-11-25T04:24:34.536 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.536 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.536 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:34.536 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.166319+0000", 2023-11-25T04:24:34.536 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9145000000000003e-05 2023-11-25T04:24:34.536 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.537 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.537 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:34.537 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.166352+0000", 2023-11-25T04:24:34.537 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2969000000000002e-05 2023-11-25T04:24:34.537 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:34.537 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:34.537 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:34.537 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.537 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.537 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:1102 4.2f 4:f543b94c:::1000000121d.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:34.537 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:22.158246+0000", 2023-11-25T04:24:34.537 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 12.320983387, 2023-11-25T04:24:34.538 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010094396, 2023-11-25T04:24:34.538 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:34.538 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:34.538 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:34.538 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.538 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:34.538 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158246+0000", 2023-11-25T04:24:34.538 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:34.538 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.538 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.538 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:34.538 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158246+0000", 2023-11-25T04:24:34.539 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:34.539 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.539 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.539 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:34.539 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158246+0000", 2023-11-25T04:24:34.539 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5299999999999999e-07 2023-11-25T04:24:34.539 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.539 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.539 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:34.539 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158249+0000", 2023-11-25T04:24:34.539 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2840000000000001e-06 2023-11-25T04:24:34.539 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.540 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.540 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:34.540 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158254+0000", 2023-11-25T04:24:34.540 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3449999999999996e-06 2023-11-25T04:24:34.540 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.540 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.540 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:34.540 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158258+0000", 2023-11-25T04:24:34.540 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5030000000000002e-06 2023-11-25T04:24:34.540 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.540 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.540 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:34.541 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158302+0000", 2023-11-25T04:24:34.541 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3908e-05 2023-11-25T04:24:34.541 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.541 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.541 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:34.541 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158398+0000", 2023-11-25T04:24:34.541 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6898000000000006e-05 2023-11-25T04:24:34.541 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.541 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.541 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:34.541 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158480+0000", 2023-11-25T04:24:34.542 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1578999999999999e-05 2023-11-25T04:24:34.542 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.542 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.542 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:34.542 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.159229+0000", 2023-11-25T04:24:34.542 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000749049 2023-11-25T04:24:34.542 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.542 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.542 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:34.542 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.168228+0000", 2023-11-25T04:24:34.543 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0089992930000000002 2023-11-25T04:24:34.543 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.543 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.543 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:34.543 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.168310+0000", 2023-11-25T04:24:34.543 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1348999999999997e-05 2023-11-25T04:24:34.543 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.543 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.543 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:34.543 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.168340+0000", 2023-11-25T04:24:34.543 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0735000000000003e-05 2023-11-25T04:24:34.544 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:34.544 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:34.544 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:34.544 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.544 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.544 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:1105 4.23 4:c6ff07b4:::10000001220.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:34.544 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:22.158714+0000", 2023-11-25T04:24:34.544 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 12.320515273, 2023-11-25T04:24:34.544 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010412490999999999, 2023-11-25T04:24:34.544 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:34.544 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:34.545 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:34.545 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.545 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:34.545 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158714+0000", 2023-11-25T04:24:34.545 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:34.545 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.545 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.545 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:34.545 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158714+0000", 2023-11-25T04:24:34.545 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:34.545 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.546 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.546 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:34.546 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158715+0000", 2023-11-25T04:24:34.546 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4099999999999998e-07 2023-11-25T04:24:34.546 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.546 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.546 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:34.546 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158718+0000", 2023-11-25T04:24:34.546 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4319999999999999e-06 2023-11-25T04:24:34.546 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.546 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.546 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:34.547 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158729+0000", 2023-11-25T04:24:34.547 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.078e-05 2023-11-25T04:24:34.547 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.547 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.547 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:34.547 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158736+0000", 2023-11-25T04:24:34.547 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4649999999999997e-06 2023-11-25T04:24:34.547 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.547 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.547 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:34.547 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158779+0000", 2023-11-25T04:24:34.548 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3049000000000001e-05 2023-11-25T04:24:34.548 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.548 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.548 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:34.548 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158865+0000", 2023-11-25T04:24:34.548 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5138999999999999e-05 2023-11-25T04:24:34.548 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.548 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.548 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:34.548 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158983+0000", 2023-11-25T04:24:34.548 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000118658 2023-11-25T04:24:34.548 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.549 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.549 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:34.549 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.159645+0000", 2023-11-25T04:24:34.549 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00066193999999999997 2023-11-25T04:24:34.549 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.549 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.549 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:34.549 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.169046+0000", 2023-11-25T04:24:34.549 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0094005400000000006 2023-11-25T04:24:34.549 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.549 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.549 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:34.550 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.169092+0000", 2023-11-25T04:24:34.550 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5840000000000002e-05 2023-11-25T04:24:34.550 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.550 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.550 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:34.550 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.169127+0000", 2023-11-25T04:24:34.550 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5006999999999999e-05 2023-11-25T04:24:34.550 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:34.550 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:34.550 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:34.550 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.551 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.551 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2841 4.4 4:22111d42:::100000010df.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:34.551 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.092102+0000", 2023-11-25T04:24:34.551 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 9.3871277979999999, 2023-11-25T04:24:34.551 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.016449783999999999, 2023-11-25T04:24:34.551 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:34.551 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:34.551 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:34.551 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.551 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:34.551 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092102+0000", 2023-11-25T04:24:34.551 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:34.552 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.552 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.552 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:34.552 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092102+0000", 2023-11-25T04:24:34.552 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:34.552 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.552 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.552 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:34.552 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092102+0000", 2023-11-25T04:24:34.552 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3700000000000001e-07 2023-11-25T04:24:34.552 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.552 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.553 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:34.553 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092104+0000", 2023-11-25T04:24:34.553 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0679999999999999e-06 2023-11-25T04:24:34.553 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.553 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.553 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:34.553 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092111+0000", 2023-11-25T04:24:34.553 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0589999999999997e-06 2023-11-25T04:24:34.553 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.553 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.553 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:34.553 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092130+0000", 2023-11-25T04:24:34.553 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8686999999999999e-05 2023-11-25T04:24:34.554 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.554 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.554 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:34.554 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092443+0000", 2023-11-25T04:24:34.554 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000313519 2023-11-25T04:24:34.554 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.554 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.554 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:34.554 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092553+0000", 2023-11-25T04:24:34.554 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010932100000000001 2023-11-25T04:24:34.554 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.554 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.555 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:34.555 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092639+0000", 2023-11-25T04:24:34.555 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6094999999999995e-05 2023-11-25T04:24:34.555 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.555 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.555 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:34.555 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094679+0000", 2023-11-25T04:24:34.555 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0020402580000000001 2023-11-25T04:24:34.555 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.555 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.555 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:34.555 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108462+0000", 2023-11-25T04:24:34.555 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013783358000000001 2023-11-25T04:24:34.556 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.556 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.556 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:34.556 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108516+0000", 2023-11-25T04:24:34.556 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3483000000000003e-05 2023-11-25T04:24:34.556 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.556 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.556 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:34.556 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108551+0000", 2023-11-25T04:24:34.556 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5499e-05 2023-11-25T04:24:34.556 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:34.556 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:34.557 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:34.557 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.557 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.557 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2852 4.2d 4:b745ac76:::100000010f5.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:34.557 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.093477+0000", 2023-11-25T04:24:34.557 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 9.385752686, 2023-11-25T04:24:34.557 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.015423084, 2023-11-25T04:24:34.557 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:34.557 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:34.557 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:34.557 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.557 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:34.557 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093477+0000", 2023-11-25T04:24:34.558 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:34.558 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.558 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.558 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:34.558 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093477+0000", 2023-11-25T04:24:34.558 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:34.558 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.558 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.558 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:34.558 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093477+0000", 2023-11-25T04:24:34.558 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2500000000000005e-07 2023-11-25T04:24:34.558 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.559 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.559 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:34.559 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093480+0000", 2023-11-25T04:24:34.559 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0189999999999998e-06 2023-11-25T04:24:34.559 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.559 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.559 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:34.559 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093642+0000", 2023-11-25T04:24:34.559 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000162072 2023-11-25T04:24:34.559 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.559 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.559 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:34.559 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093649+0000", 2023-11-25T04:24:34.560 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9659999999999999e-06 2023-11-25T04:24:34.560 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.560 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.560 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:34.560 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093693+0000", 2023-11-25T04:24:34.560 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3794000000000001e-05 2023-11-25T04:24:34.560 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.560 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.560 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:34.560 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093764+0000", 2023-11-25T04:24:34.560 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0981000000000004e-05 2023-11-25T04:24:34.560 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.560 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.561 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:34.561 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093850+0000", 2023-11-25T04:24:34.561 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5536000000000002e-05 2023-11-25T04:24:34.561 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.561 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.561 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:34.561 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094710+0000", 2023-11-25T04:24:34.561 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00086070299999999999 2023-11-25T04:24:34.561 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.561 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.561 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:34.561 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108776+0000", 2023-11-25T04:24:34.562 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01406542 2023-11-25T04:24:34.562 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.562 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.562 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:34.562 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108829+0000", 2023-11-25T04:24:34.562 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2936999999999998e-05 2023-11-25T04:24:34.562 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.562 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.562 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:34.562 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108900+0000", 2023-11-25T04:24:34.562 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1031000000000005e-05 2023-11-25T04:24:34.562 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:34.563 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:34.563 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:34.563 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.563 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.563 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2854 4.2d 4:b5c259cf:::100000010f9.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:34.563 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.093674+0000", 2023-11-25T04:24:34.563 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 9.3855550260000005, 2023-11-25T04:24:34.563 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.015970380999999999, 2023-11-25T04:24:34.563 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:34.563 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:34.563 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:34.563 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.563 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:34.564 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093674+0000", 2023-11-25T04:24:34.564 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:34.564 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.564 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.564 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:34.564 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093674+0000", 2023-11-25T04:24:34.564 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:34.564 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.564 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.564 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:34.564 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093675+0000", 2023-11-25T04:24:34.564 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8400000000000004e-07 2023-11-25T04:24:34.565 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.565 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.565 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:34.565 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093678+0000", 2023-11-25T04:24:34.565 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2229999999999999e-06 2023-11-25T04:24:34.565 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.565 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.565 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:34.565 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093686+0000", 2023-11-25T04:24:34.565 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9990000000000008e-06 2023-11-25T04:24:34.565 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.565 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.566 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:34.566 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093696+0000", 2023-11-25T04:24:34.566 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6479999999999998e-06 2023-11-25T04:24:34.566 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.566 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.566 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:34.566 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094104+0000", 2023-11-25T04:24:34.566 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00040823900000000002 2023-11-25T04:24:34.566 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.566 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.566 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:34.566 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094172+0000", 2023-11-25T04:24:34.567 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8023999999999998e-05 2023-11-25T04:24:34.567 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.567 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.567 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:34.567 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094256+0000", 2023-11-25T04:24:34.567 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3418999999999995e-05 2023-11-25T04:24:34.567 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.567 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.567 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:34.567 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095160+0000", 2023-11-25T04:24:34.567 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00090418599999999997 2023-11-25T04:24:34.567 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.567 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.568 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:34.568 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109551+0000", 2023-11-25T04:24:34.568 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014391571000000001 2023-11-25T04:24:34.568 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.568 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.568 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:34.568 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109601+0000", 2023-11-25T04:24:34.568 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9478000000000003e-05 2023-11-25T04:24:34.568 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.568 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.568 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:34.568 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109645+0000", 2023-11-25T04:24:34.568 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.401e-05 2023-11-25T04:24:34.569 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:34.569 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:34.569 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:34.569 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.569 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.569 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2861 4.4 4:214027a9:::10000001107.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:34.569 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.093829+0000", 2023-11-25T04:24:34.569 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 9.3854006079999994, 2023-11-25T04:24:34.569 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014848917999999999, 2023-11-25T04:24:34.569 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:34.569 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:34.569 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:34.570 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.570 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:34.570 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093829+0000", 2023-11-25T04:24:34.570 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:34.570 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.570 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.570 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:34.570 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093829+0000", 2023-11-25T04:24:34.570 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:34.570 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.570 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.570 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:34.570 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093830+0000", 2023-11-25T04:24:34.571 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1999999999999998e-07 2023-11-25T04:24:34.571 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.571 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.571 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:34.571 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093834+0000", 2023-11-25T04:24:34.571 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.34e-06 2023-11-25T04:24:34.571 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.571 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.571 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:34.571 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093844+0000", 2023-11-25T04:24:34.571 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6690000000000001e-06 2023-11-25T04:24:34.571 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.572 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.572 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:34.572 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093857+0000", 2023-11-25T04:24:34.572 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3434999999999999e-05 2023-11-25T04:24:34.572 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.572 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.572 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:34.572 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093940+0000", 2023-11-25T04:24:34.572 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3047999999999995e-05 2023-11-25T04:24:34.572 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.572 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.572 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:34.572 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094052+0000", 2023-11-25T04:24:34.572 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000111538 2023-11-25T04:24:34.573 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.573 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.573 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:34.573 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094466+0000", 2023-11-25T04:24:34.573 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00041464600000000001 2023-11-25T04:24:34.573 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.573 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.573 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:34.573 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095570+0000", 2023-11-25T04:24:34.573 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001103981 2023-11-25T04:24:34.573 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.573 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.573 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:34.574 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108598+0000", 2023-11-25T04:24:34.574 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013027164000000001 2023-11-25T04:24:34.574 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.574 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.574 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:34.574 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108646+0000", 2023-11-25T04:24:34.574 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8090999999999997e-05 2023-11-25T04:24:34.574 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.574 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.574 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:34.574 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108678+0000", 2023-11-25T04:24:34.574 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2085999999999997e-05 2023-11-25T04:24:34.575 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:34.575 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:34.575 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:34.575 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.575 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.575 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2863 4.12 4:4b84c733:::1000000110b.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:34.575 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.093877+0000", 2023-11-25T04:24:34.575 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 9.3853522809999994, 2023-11-25T04:24:34.575 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014682755, 2023-11-25T04:24:34.575 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:34.575 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:34.575 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:34.575 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.576 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:34.576 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093877+0000", 2023-11-25T04:24:34.576 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:34.576 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.576 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.576 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:34.576 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093877+0000", 2023-11-25T04:24:34.576 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:34.576 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.576 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.576 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:34.576 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093878+0000", 2023-11-25T04:24:34.576 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4199999999999996e-07 2023-11-25T04:24:34.577 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.577 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.577 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:34.577 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093880+0000", 2023-11-25T04:24:34.577 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6189999999999998e-06 2023-11-25T04:24:34.577 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.577 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.577 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:34.577 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093888+0000", 2023-11-25T04:24:34.577 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5510000000000002e-06 2023-11-25T04:24:34.577 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.577 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.577 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:34.578 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093893+0000", 2023-11-25T04:24:34.578 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4179999999999996e-06 2023-11-25T04:24:34.578 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.578 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.578 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:34.578 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093934+0000", 2023-11-25T04:24:34.578 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0648000000000001e-05 2023-11-25T04:24:34.578 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.578 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.578 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:34.578 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094081+0000", 2023-11-25T04:24:34.578 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014753499999999999 2023-11-25T04:24:34.578 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.579 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.579 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:34.579 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094172+0000", 2023-11-25T04:24:34.579 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.0722000000000003e-05 2023-11-25T04:24:34.579 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.579 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.579 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:34.579 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094742+0000", 2023-11-25T04:24:34.579 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00056948600000000002 2023-11-25T04:24:34.579 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.579 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.579 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:34.579 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108491+0000", 2023-11-25T04:24:34.580 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013749568 2023-11-25T04:24:34.580 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.580 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.580 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:34.580 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108525+0000", 2023-11-25T04:24:34.580 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3683999999999998e-05 2023-11-25T04:24:34.580 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.580 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.580 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:34.580 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108560+0000", 2023-11-25T04:24:34.580 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4981999999999998e-05 2023-11-25T04:24:34.580 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:34.581 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:34.581 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:34.581 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.581 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.581 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2880 4.21 4:855834a4:::1000000112d.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:34.581 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.095098+0000", 2023-11-25T04:24:34.581 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 9.3841315870000006, 2023-11-25T04:24:34.581 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013847544, 2023-11-25T04:24:34.581 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:34.581 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:34.582 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:34.582 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.582 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:34.582 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095098+0000", 2023-11-25T04:24:34.582 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:34.582 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.582 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.582 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:34.582 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095098+0000", 2023-11-25T04:24:34.582 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:34.582 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.582 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.582 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:34.583 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095098+0000", 2023-11-25T04:24:34.583 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3000000000000002e-07 2023-11-25T04:24:34.583 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.583 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.583 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:34.583 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095099+0000", 2023-11-25T04:24:34.583 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.331e-06 2023-11-25T04:24:34.583 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.583 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.583 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:34.583 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095103+0000", 2023-11-25T04:24:34.584 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8319999999999999e-06 2023-11-25T04:24:34.584 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.584 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.584 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:34.584 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095108+0000", 2023-11-25T04:24:34.584 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4179999999999999e-06 2023-11-25T04:24:34.584 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.584 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.584 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:34.584 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095153+0000", 2023-11-25T04:24:34.585 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4799000000000003e-05 2023-11-25T04:24:34.585 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.585 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.585 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:34.585 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095268+0000", 2023-11-25T04:24:34.585 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000115437 2023-11-25T04:24:34.585 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.585 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.585 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:34.585 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095363+0000", 2023-11-25T04:24:34.585 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.5278000000000005e-05 2023-11-25T04:24:34.585 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.586 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.586 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:34.586 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095845+0000", 2023-11-25T04:24:34.586 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00048164600000000002 2023-11-25T04:24:34.586 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.586 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.586 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:34.586 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108854+0000", 2023-11-25T04:24:34.586 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013009289 2023-11-25T04:24:34.586 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.586 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.587 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:34.587 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108907+0000", 2023-11-25T04:24:34.587 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2528999999999998e-05 2023-11-25T04:24:34.587 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.587 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.587 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:34.587 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108945+0000", 2023-11-25T04:24:34.587 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8655000000000002e-05 2023-11-25T04:24:34.587 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:34.587 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:34.587 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:34.587 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.587 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.588 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2884 4.37 4:ef909cdc:::10000001135.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:34.588 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.095518+0000", 2023-11-25T04:24:34.588 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 9.3837116599999995, 2023-11-25T04:24:34.588 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014149988, 2023-11-25T04:24:34.588 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:34.588 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:34.588 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:34.588 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.588 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:34.588 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095518+0000", 2023-11-25T04:24:34.588 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:34.588 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.588 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.589 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:34.589 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095518+0000", 2023-11-25T04:24:34.589 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:34.589 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.589 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.589 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:34.589 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095518+0000", 2023-11-25T04:24:34.589 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2900000000000003e-07 2023-11-25T04:24:34.589 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.589 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.589 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:34.589 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095522+0000", 2023-11-25T04:24:34.589 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2119999999999999e-06 2023-11-25T04:24:34.590 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.590 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.590 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:34.590 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095532+0000", 2023-11-25T04:24:34.590 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0375e-05 2023-11-25T04:24:34.590 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.590 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.590 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:34.590 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095537+0000", 2023-11-25T04:24:34.590 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5909999999999998e-06 2023-11-25T04:24:34.590 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.590 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.591 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:34.591 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095584+0000", 2023-11-25T04:24:34.591 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7169999999999997e-05 2023-11-25T04:24:34.591 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.591 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.591 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:34.591 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095695+0000", 2023-11-25T04:24:34.591 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011078699999999999 2023-11-25T04:24:34.591 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.591 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.591 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:34.591 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095777+0000", 2023-11-25T04:24:34.591 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2089999999999995e-05 2023-11-25T04:24:34.592 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.592 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.592 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:34.592 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096286+0000", 2023-11-25T04:24:34.592 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000509247 2023-11-25T04:24:34.592 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.592 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.592 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:34.592 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109596+0000", 2023-11-25T04:24:34.592 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013309872 2023-11-25T04:24:34.592 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.592 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.592 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:34.593 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109638+0000", 2023-11-25T04:24:34.593 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2381e-05 2023-11-25T04:24:34.593 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.593 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.593 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:34.593 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109668+0000", 2023-11-25T04:24:34.593 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9634000000000001e-05 2023-11-25T04:24:34.593 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:34.593 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:34.593 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:34.593 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.593 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.593 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2890 4.12 4:4a2ad640:::10000001141.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:34.594 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.096274+0000", 2023-11-25T04:24:34.594 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 9.3829555970000005, 2023-11-25T04:24:34.594 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013647259, 2023-11-25T04:24:34.594 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:34.594 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:34.594 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:34.594 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.594 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:34.594 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096274+0000", 2023-11-25T04:24:34.594 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:34.594 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.594 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.595 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:34.595 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096274+0000", 2023-11-25T04:24:34.595 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:34.595 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.595 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.595 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:34.595 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096275+0000", 2023-11-25T04:24:34.595 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0299999999999998e-07 2023-11-25T04:24:34.595 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.595 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.595 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:34.595 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096279+0000", 2023-11-25T04:24:34.596 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9580000000000001e-06 2023-11-25T04:24:34.596 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.596 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.596 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:34.596 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096287+0000", 2023-11-25T04:24:34.596 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6929999999999996e-06 2023-11-25T04:24:34.596 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.596 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.596 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:34.596 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096294+0000", 2023-11-25T04:24:34.596 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.348e-06 2023-11-25T04:24:34.596 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.596 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.596 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:34.597 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096342+0000", 2023-11-25T04:24:34.597 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8068000000000001e-05 2023-11-25T04:24:34.597 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.597 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.597 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:34.597 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096462+0000", 2023-11-25T04:24:34.597 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000119918 2023-11-25T04:24:34.597 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.597 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.597 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:34.597 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096557+0000", 2023-11-25T04:24:34.597 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.5340999999999994e-05 2023-11-25T04:24:34.598 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.598 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.598 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:34.598 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096971+0000", 2023-11-25T04:24:34.598 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00041398100000000001 2023-11-25T04:24:34.598 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.598 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.598 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:34.598 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109849+0000", 2023-11-25T04:24:34.598 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.012877985999999999 2023-11-25T04:24:34.598 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.598 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.598 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:34.599 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109894+0000", 2023-11-25T04:24:34.599 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5595e-05 2023-11-25T04:24:34.599 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.599 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.599 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:34.599 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109921+0000", 2023-11-25T04:24:34.599 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6568e-05 2023-11-25T04:24:34.599 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:34.599 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:34.599 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:34.599 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.599 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.600 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2891 4.23 4:c4054dd3:::10000001143.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:34.600 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.096713+0000", 2023-11-25T04:24:34.600 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 9.3825169030000009, 2023-11-25T04:24:34.600 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013148692, 2023-11-25T04:24:34.600 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:34.600 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:34.600 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:34.600 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.600 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:34.600 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096713+0000", 2023-11-25T04:24:34.600 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:34.600 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.600 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.601 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:34.601 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096713+0000", 2023-11-25T04:24:34.601 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:34.601 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.601 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.601 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:34.601 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096713+0000", 2023-11-25T04:24:34.601 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9299999999999998e-07 2023-11-25T04:24:34.601 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.601 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.601 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:34.601 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096716+0000", 2023-11-25T04:24:34.601 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.988e-06 2023-11-25T04:24:34.602 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.602 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.602 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:34.602 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096722+0000", 2023-11-25T04:24:34.602 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0499999999999997e-06 2023-11-25T04:24:34.602 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.602 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.602 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:34.602 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096727+0000", 2023-11-25T04:24:34.602 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3380000000000004e-06 2023-11-25T04:24:34.602 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.602 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.603 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:34.603 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096770+0000", 2023-11-25T04:24:34.603 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2938000000000001e-05 2023-11-25T04:24:34.603 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.603 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.603 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:34.603 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096904+0000", 2023-11-25T04:24:34.603 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000133759 2023-11-25T04:24:34.603 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.603 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.603 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:34.603 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096983+0000", 2023-11-25T04:24:34.603 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8559999999999993e-05 2023-11-25T04:24:34.604 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.604 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.604 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:34.604 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.097437+0000", 2023-11-25T04:24:34.604 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00045439200000000002 2023-11-25T04:24:34.604 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.604 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.604 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:34.604 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109737+0000", 2023-11-25T04:24:34.604 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.012300190000000001 2023-11-25T04:24:34.604 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.604 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.604 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:34.605 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109782+0000", 2023-11-25T04:24:34.605 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4546999999999999e-05 2023-11-25T04:24:34.605 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:34.605 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:34.605 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:34.605 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109861+0000", 2023-11-25T04:24:34.605 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9337000000000003e-05 2023-11-25T04:24:34.605 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:34.605 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:34.605 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:34.605 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:34.605 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:34.605 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:34.607 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:34.605+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:34.607 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:24:34.710 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:34.708+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:34.721 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:34.722 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:24:34.722 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2023-11-25T04:24:34.722 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:34.731 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:24:34.758 INFO:tasks.workunit.client.0.smithi067.stderr:doio.c: In function 'parse_cmdline': 2023-11-25T04:24:34.758 INFO:tasks.workunit.client.0.smithi067.stderr:doio.c:3113:33: warning: '%s' directive argument is null [-Wformat-overflow=] 2023-11-25T04:24:34.759 INFO:tasks.workunit.client.0.smithi067.stderr: 3113 | fprintf(stderr, 2023-11-25T04:24:34.759 INFO:tasks.workunit.client.0.smithi067.stderr: | ^~~~~~~~~~~~~~~ 2023-11-25T04:24:34.759 INFO:tasks.workunit.client.0.smithi067.stderr: 3114 | "%s%s: Illegal -C arg (%s). Must be one of: ", 2023-11-25T04:24:34.759 INFO:tasks.workunit.client.0.smithi067.stderr: | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-11-25T04:24:34.759 INFO:tasks.workunit.client.0.smithi067.stderr: 3115 | Prog, TagName, tok); 2023-11-25T04:24:34.759 INFO:tasks.workunit.client.0.smithi067.stderr: | ~~~~~~~~~~~~~~~~~~~ 2023-11-25T04:24:34.792 INFO:tasks.workunit.client.0.smithi067.stderr:fsx.c: In function 'main': 2023-11-25T04:24:34.792 INFO:tasks.workunit.client.0.smithi067.stderr:fsx.c:2662:33: warning: 'strncpy' specified bound 4096 equals destination size [-Wstringop-truncation] 2023-11-25T04:24:34.792 INFO:tasks.workunit.client.0.smithi067.stderr: 2662 | strncpy(opsfile, optarg, sizeof(opsfile)); 2023-11-25T04:24:34.792 INFO:tasks.workunit.client.0.smithi067.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-11-25T04:24:34.793 INFO:tasks.workunit.client.0.smithi067.stderr:fsx.c:2632:25: warning: 'strncpy' specified bound 1024 equals destination size [-Wstringop-truncation] 2023-11-25T04:24:34.793 INFO:tasks.workunit.client.0.smithi067.stderr: 2632 | strncpy(dname, optarg, sizeof(dname)); 2023-11-25T04:24:34.793 INFO:tasks.workunit.client.0.smithi067.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-11-25T04:24:34.811 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:34.810+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:34.837 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:34.838 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:24:34.838 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2023-11-25T04:24:34.838 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2023-11-25T04:24:34.838 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:34.846 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:24:34.914 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:34.912+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:34.923 INFO:tasks.workunit.client.0.smithi067.stderr:gmake[2]: *** [include/buildrules:30: ltp] Error 2 2023-11-25T04:24:34.923 INFO:tasks.workunit.client.0.smithi067.stderr:gmake[2]: *** Waiting for unfinished jobs.... 2023-11-25T04:24:34.992 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:34.992 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2023-11-25T04:24:34.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2023-11-25T04:24:34.993 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2023-11-25T04:24:34.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.993 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:152 3.1a 3:5e41ab69:::200.00000005:head [write 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:34.993 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:21.237401+0000", 2023-11-25T04:24:34.993 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.752886200000001, 2023-11-25T04:24:34.993 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015253802, 2023-11-25T04:24:34.993 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:34.993 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:34.993 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:34.993 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:34.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.237401+0000", 2023-11-25T04:24:34.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:34.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:34.994 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.237401+0000", 2023-11-25T04:24:34.994 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:34.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.994 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.994 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:34.995 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.237402+0000", 2023-11-25T04:24:34.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3699999999999999e-07 2023-11-25T04:24:34.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.995 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:34.995 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243182+0000", 2023-11-25T04:24:34.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057801270000000004 2023-11-25T04:24:34.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.995 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.995 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:34.995 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243193+0000", 2023-11-25T04:24:34.995 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1221e-05 2023-11-25T04:24:34.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.996 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:34.996 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243198+0000", 2023-11-25T04:24:34.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3739999999999997e-06 2023-11-25T04:24:34.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.996 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:34.996 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243246+0000", 2023-11-25T04:24:34.996 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7667999999999998e-05 2023-11-25T04:24:34.996 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.996 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.996 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:34.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243292+0000", 2023-11-25T04:24:34.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5794000000000002e-05 2023-11-25T04:24:34.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.997 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:34.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243360+0000", 2023-11-25T04:24:34.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8479000000000006e-05 2023-11-25T04:24:34.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.997 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.997 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:34.997 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.250667+0000", 2023-11-25T04:24:34.997 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.007306618 2023-11-25T04:24:34.997 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:34.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.252598+0000", 2023-11-25T04:24:34.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019315129999999999 2023-11-25T04:24:34.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.998 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:34.998 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.252631+0000", 2023-11-25T04:24:34.998 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2925000000000001e-05 2023-11-25T04:24:34.998 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.998 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.999 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:34.999 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.252655+0000", 2023-11-25T04:24:34.999 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3246000000000001e-05 2023-11-25T04:24:34.999 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:34.999 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:34.999 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:34.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:34.999 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:34.999 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:2841 4.4 e47/17 4:22111d42:::100000010df.00000000:head v 47'139, mlcod=47'137)", 2023-11-25T04:24:34.999 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.093610+0000", 2023-11-25T04:24:34.999 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.8966773369999999, 2023-11-25T04:24:35.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014758871999999999, 2023-11-25T04:24:35.000 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:35.000 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:35.000 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:35.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:35.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093610+0000", 2023-11-25T04:24:35.000 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:35.000 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.000 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.000 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:35.000 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093610+0000", 2023-11-25T04:24:35.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:35.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.001 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:35.001 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093610+0000", 2023-11-25T04:24:35.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1799999999999999e-07 2023-11-25T04:24:35.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.001 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.001 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:35.001 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093621+0000", 2023-11-25T04:24:35.001 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1107999999999999e-05 2023-11-25T04:24:35.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:35.002 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093627+0000", 2023-11-25T04:24:35.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0310000000000004e-06 2023-11-25T04:24:35.002 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.002 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.002 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:35.002 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093633+0000", 2023-11-25T04:24:35.002 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2700000000000001e-06 2023-11-25T04:24:35.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:35.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093722+0000", 2023-11-25T04:24:35.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8417000000000004e-05 2023-11-25T04:24:35.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:35.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093737+0000", 2023-11-25T04:24:35.004 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5781e-05 2023-11-25T04:24:35.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.004 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.004 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:35.004 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108203+0000", 2023-11-25T04:24:35.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014465844 2023-11-25T04:24:35.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.005 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:35.005 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108368+0000", 2023-11-25T04:24:35.005 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000165203 2023-11-25T04:24:35.005 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:35.005 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:35.005 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:35.005 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.005 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.006 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2860 4.3d 4:bf0d7877:::10000001105.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:35.006 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.093823+0000", 2023-11-25T04:24:35.006 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.8964641689999997, 2023-11-25T04:24:35.006 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015582848, 2023-11-25T04:24:35.006 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:35.006 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:35.006 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:35.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.006 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:35.006 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093823+0000", 2023-11-25T04:24:35.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:35.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:35.007 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093823+0000", 2023-11-25T04:24:35.007 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:35.007 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.007 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.007 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:35.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093823+0000", 2023-11-25T04:24:35.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9100000000000004e-07 2023-11-25T04:24:35.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:35.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093826+0000", 2023-11-25T04:24:35.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4499999999999998e-06 2023-11-25T04:24:35.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.008 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.008 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:35.008 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093835+0000", 2023-11-25T04:24:35.008 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8179999999999993e-06 2023-11-25T04:24:35.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:35.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093840+0000", 2023-11-25T04:24:35.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0819999999999998e-06 2023-11-25T04:24:35.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:35.009 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093973+0000", 2023-11-25T04:24:35.009 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013308999999999999 2023-11-25T04:24:35.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.009 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.009 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:35.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094277+0000", 2023-11-25T04:24:35.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00030403299999999999 2023-11-25T04:24:35.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:35.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108608+0000", 2023-11-25T04:24:35.010 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014330888 2023-11-25T04:24:35.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.010 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.010 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:35.010 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108676+0000", 2023-11-25T04:24:35.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8577000000000004e-05 2023-11-25T04:24:35.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.011 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:35.011 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109406+0000", 2023-11-25T04:24:35.011 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00072931899999999997 2023-11-25T04:24:35.011 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:35.011 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:35.011 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:35.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.011 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.011 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2862 4.13 4:cb2e645e:::10000001109.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:35.012 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.093852+0000", 2023-11-25T04:24:35.012 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.8964352079999998, 2023-11-25T04:24:35.012 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015190636, 2023-11-25T04:24:35.012 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:35.012 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:35.012 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:35.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.012 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:35.012 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093852+0000", 2023-11-25T04:24:35.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:35.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:35.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093852+0000", 2023-11-25T04:24:35.013 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:35.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.013 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.013 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:35.013 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093852+0000", 2023-11-25T04:24:35.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0299999999999999e-07 2023-11-25T04:24:35.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.014 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.014 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:35.014 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093854+0000", 2023-11-25T04:24:35.014 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.846e-06 2023-11-25T04:24:35.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:35.015 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093861+0000", 2023-11-25T04:24:35.015 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5520000000000001e-06 2023-11-25T04:24:35.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.015 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:35.016 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093865+0000", 2023-11-25T04:24:35.016 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7870000000000001e-06 2023-11-25T04:24:35.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:35.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093940+0000", 2023-11-25T04:24:35.017 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4799e-05 2023-11-25T04:24:35.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.017 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.017 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:35.017 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094052+0000", 2023-11-25T04:24:35.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011156500000000001 2023-11-25T04:24:35.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.018 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.018 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:35.018 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108928+0000", 2023-11-25T04:24:35.018 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014876245 2023-11-25T04:24:35.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:35.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108972+0000", 2023-11-25T04:24:35.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4477999999999997e-05 2023-11-25T04:24:35.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.019 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.019 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:35.019 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109042+0000", 2023-11-25T04:24:35.019 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9861000000000001e-05 2023-11-25T04:24:35.019 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:35.020 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:35.020 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:35.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.020 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2865 4.36 4:6ebc5ac6:::1000000110f.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:35.020 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.093906+0000", 2023-11-25T04:24:35.020 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.8963809349999998, 2023-11-25T04:24:35.020 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014962628, 2023-11-25T04:24:35.020 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:35.020 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:35.020 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:35.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.021 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:35.021 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093906+0000", 2023-11-25T04:24:35.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:35.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.021 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:35.021 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093906+0000", 2023-11-25T04:24:35.021 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:35.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.021 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.021 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:35.021 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093906+0000", 2023-11-25T04:24:35.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4400000000000001e-07 2023-11-25T04:24:35.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:35.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093908+0000", 2023-11-25T04:24:35.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8640000000000001e-06 2023-11-25T04:24:35.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.022 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.022 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:35.022 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093914+0000", 2023-11-25T04:24:35.022 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0170000000000002e-06 2023-11-25T04:24:35.022 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:35.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093921+0000", 2023-11-25T04:24:35.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6529999999999997e-06 2023-11-25T04:24:35.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:35.023 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094258+0000", 2023-11-25T04:24:35.023 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00033703599999999998 2023-11-25T04:24:35.023 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.023 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.023 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:35.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094340+0000", 2023-11-25T04:24:35.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2279000000000003e-05 2023-11-25T04:24:35.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:35.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094416+0000", 2023-11-25T04:24:35.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6248999999999995e-05 2023-11-25T04:24:35.024 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.024 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:35.024 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.096134+0000", 2023-11-25T04:24:35.024 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001717611 2023-11-25T04:24:35.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:35.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108809+0000", 2023-11-25T04:24:35.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012675327 2023-11-25T04:24:35.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.025 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:35.025 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108836+0000", 2023-11-25T04:24:35.025 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6509999999999999e-05 2023-11-25T04:24:35.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.026 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:35.026 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108869+0000", 2023-11-25T04:24:35.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2737999999999997e-05 2023-11-25T04:24:35.026 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:35.026 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:35.026 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:35.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.026 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:2852 4.2d e47/17 4:b745ac76:::100000010f5.00000000:head v 47'126, mlcod=47'124)", 2023-11-25T04:24:35.026 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.093987+0000", 2023-11-25T04:24:35.026 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.8963003739999991, 2023-11-25T04:24:35.026 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014613256, 2023-11-25T04:24:35.027 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:35.027 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:35.027 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:35.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:35.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093987+0000", 2023-11-25T04:24:35.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:35.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.027 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.027 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:35.027 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093987+0000", 2023-11-25T04:24:35.027 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:35.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:35.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093987+0000", 2023-11-25T04:24:35.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7000000000000001e-07 2023-11-25T04:24:35.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.028 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:35.028 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093990+0000", 2023-11-25T04:24:35.028 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8729999999999999e-06 2023-11-25T04:24:35.028 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.028 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:35.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093997+0000", 2023-11-25T04:24:35.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.973e-06 2023-11-25T04:24:35.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:35.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094001+0000", 2023-11-25T04:24:35.029 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7360000000000001e-06 2023-11-25T04:24:35.029 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.029 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.029 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:35.029 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094263+0000", 2023-11-25T04:24:35.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026173400000000001 2023-11-25T04:24:35.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:35.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094278+0000", 2023-11-25T04:24:35.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4681999999999999e-05 2023-11-25T04:24:35.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.030 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:35.030 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108563+0000", 2023-11-25T04:24:35.030 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014284738 2023-11-25T04:24:35.030 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.031 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:35.031 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108600+0000", 2023-11-25T04:24:35.031 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7249999999999997e-05 2023-11-25T04:24:35.031 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:35.031 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:35.031 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:35.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.031 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.031 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2866 4.36 4:6f6907fb:::10000001111.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:35.031 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.094033+0000", 2023-11-25T04:24:35.031 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.8962542090000003, 2023-11-25T04:24:35.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014885591, 2023-11-25T04:24:35.032 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:35.032 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:35.032 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:35.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:35.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094033+0000", 2023-11-25T04:24:35.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:35.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.032 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.032 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:35.032 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094033+0000", 2023-11-25T04:24:35.032 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:35.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.033 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:35.033 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094033+0000", 2023-11-25T04:24:35.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.06e-07 2023-11-25T04:24:35.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.033 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.033 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:35.033 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094036+0000", 2023-11-25T04:24:35.033 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.508e-06 2023-11-25T04:24:35.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:35.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094044+0000", 2023-11-25T04:24:35.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5909999999999996e-06 2023-11-25T04:24:35.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:35.034 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094051+0000", 2023-11-25T04:24:35.034 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0319999999999998e-06 2023-11-25T04:24:35.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.034 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.034 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:35.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094727+0000", 2023-11-25T04:24:35.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00067523300000000002 2023-11-25T04:24:35.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:35.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094795+0000", 2023-11-25T04:24:35.035 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8156000000000003e-05 2023-11-25T04:24:35.035 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.035 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.035 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:35.035 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094898+0000", 2023-11-25T04:24:35.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103329 2023-11-25T04:24:35.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:35.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108803+0000", 2023-11-25T04:24:35.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013904385 2023-11-25T04:24:35.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.036 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.036 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:35.036 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108875+0000", 2023-11-25T04:24:35.036 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1958999999999993e-05 2023-11-25T04:24:35.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.037 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:35.037 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108899+0000", 2023-11-25T04:24:35.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4019000000000002e-05 2023-11-25T04:24:35.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.037 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.037 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:35.037 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108918+0000", 2023-11-25T04:24:35.037 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9772999999999999e-05 2023-11-25T04:24:35.037 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:35.037 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:35.038 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:35.038 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.038 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:2854 4.2d e47/17 4:b5c259cf:::100000010f9.00000000:head v 47'127, mlcod=47'124)", 2023-11-25T04:24:35.038 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.094793+0000", 2023-11-25T04:24:35.038 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.8954939199999998, 2023-11-25T04:24:35.038 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014650764, 2023-11-25T04:24:35.038 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:35.038 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:35.038 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:35.038 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:35.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094793+0000", 2023-11-25T04:24:35.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:35.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:35.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094793+0000", 2023-11-25T04:24:35.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:35.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.039 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.039 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:35.039 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094793+0000", 2023-11-25T04:24:35.039 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3700000000000001e-07 2023-11-25T04:24:35.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:35.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094798+0000", 2023-11-25T04:24:35.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1200000000000004e-06 2023-11-25T04:24:35.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.040 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:35.040 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094805+0000", 2023-11-25T04:24:35.040 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9759999999999998e-06 2023-11-25T04:24:35.040 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.040 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:35.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094811+0000", 2023-11-25T04:24:35.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1410000000000002e-06 2023-11-25T04:24:35.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:35.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094838+0000", 2023-11-25T04:24:35.041 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6995000000000002e-05 2023-11-25T04:24:35.041 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.041 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.041 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:35.041 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094848+0000", 2023-11-25T04:24:35.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0115999999999999e-05 2023-11-25T04:24:35.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:35.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109408+0000", 2023-11-25T04:24:35.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014559825 2023-11-25T04:24:35.042 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.042 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.042 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:35.042 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109444+0000", 2023-11-25T04:24:35.042 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6154e-05 2023-11-25T04:24:35.043 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:35.043 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:35.043 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:35.043 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.043 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2869 4.6 4:63719fb0:::10000001117.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:35.043 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.094910+0000", 2023-11-25T04:24:35.043 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.8953769630000004, 2023-11-25T04:24:35.043 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014132570000000001, 2023-11-25T04:24:35.043 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:35.043 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:35.043 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:35.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:35.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094910+0000", 2023-11-25T04:24:35.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:35.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:35.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094910+0000", 2023-11-25T04:24:35.044 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:35.044 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.044 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.044 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:35.044 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094910+0000", 2023-11-25T04:24:35.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.58e-07 2023-11-25T04:24:35.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:35.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094913+0000", 2023-11-25T04:24:35.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.198e-06 2023-11-25T04:24:35.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.045 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.045 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:35.045 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094921+0000", 2023-11-25T04:24:35.045 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1629999999999996e-06 2023-11-25T04:24:35.045 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:35.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094926+0000", 2023-11-25T04:24:35.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7559999999999999e-06 2023-11-25T04:24:35.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:35.046 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094966+0000", 2023-11-25T04:24:35.046 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0151999999999998e-05 2023-11-25T04:24:35.046 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.046 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.046 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:35.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095073+0000", 2023-11-25T04:24:35.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107657 2023-11-25T04:24:35.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:35.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095158+0000", 2023-11-25T04:24:35.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5136999999999995e-05 2023-11-25T04:24:35.047 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.047 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.047 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:35.047 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108924+0000", 2023-11-25T04:24:35.047 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013765727 2023-11-25T04:24:35.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:35.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108958+0000", 2023-11-25T04:24:35.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3417000000000002e-05 2023-11-25T04:24:35.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:35.048 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109012+0000", 2023-11-25T04:24:35.048 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3939e-05 2023-11-25T04:24:35.048 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.048 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.048 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:35.049 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109043+0000", 2023-11-25T04:24:35.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0966000000000001e-05 2023-11-25T04:24:35.049 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:35.049 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:35.049 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:35.049 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.049 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.049 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2871 4.3f 4:fff10e11:::1000000111b.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:35.049 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.094936+0000", 2023-11-25T04:24:35.049 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.8953511249999995, 2023-11-25T04:24:35.049 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014261726000000001, 2023-11-25T04:24:35.049 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:35.050 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:35.050 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:35.050 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.050 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:35.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094936+0000", 2023-11-25T04:24:35.052 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:35.052 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.052 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.052 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:35.052 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094936+0000", 2023-11-25T04:24:35.053 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:35.053 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.054 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:35.054 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094936+0000", 2023-11-25T04:24:35.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9900000000000001e-07 2023-11-25T04:24:35.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.054 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:35.054 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094938+0000", 2023-11-25T04:24:35.054 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9599999999999999e-06 2023-11-25T04:24:35.054 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.054 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:35.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094944+0000", 2023-11-25T04:24:35.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2219999999999999e-06 2023-11-25T04:24:35.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:35.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094950+0000", 2023-11-25T04:24:35.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1329999999999998e-06 2023-11-25T04:24:35.055 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.055 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.055 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:35.055 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094993+0000", 2023-11-25T04:24:35.055 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3305e-05 2023-11-25T04:24:35.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:35.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095107+0000", 2023-11-25T04:24:35.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011449400000000001 2023-11-25T04:24:35.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.056 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:35.056 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095176+0000", 2023-11-25T04:24:35.056 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8330000000000005e-05 2023-11-25T04:24:35.056 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.056 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:35.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107939+0000", 2023-11-25T04:24:35.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012763801 2023-11-25T04:24:35.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:35.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109128+0000", 2023-11-25T04:24:35.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011884770000000001 2023-11-25T04:24:35.057 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.057 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.057 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:35.057 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109166+0000", 2023-11-25T04:24:35.057 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7948999999999999e-05 2023-11-25T04:24:35.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.058 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:35.058 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109198+0000", 2023-11-25T04:24:35.058 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1656000000000003e-05 2023-11-25T04:24:35.058 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:35.058 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:35.058 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:35.058 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.058 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.058 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2872 4.32 4:4fb002bc:::1000000111d.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:35.058 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.094961+0000", 2023-11-25T04:24:35.059 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.895326421, 2023-11-25T04:24:35.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014631564999999999, 2023-11-25T04:24:35.059 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:35.059 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:35.059 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:35.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:35.059 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094961+0000", 2023-11-25T04:24:35.059 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:35.059 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.059 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.059 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:35.060 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094961+0000", 2023-11-25T04:24:35.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:35.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.060 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:35.060 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094961+0000", 2023-11-25T04:24:35.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9299999999999999e-07 2023-11-25T04:24:35.060 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.060 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.060 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:35.060 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094963+0000", 2023-11-25T04:24:35.060 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9209999999999999e-06 2023-11-25T04:24:35.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:35.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094968+0000", 2023-11-25T04:24:35.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.502e-06 2023-11-25T04:24:35.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.061 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:35.061 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094973+0000", 2023-11-25T04:24:35.061 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8559999999999998e-06 2023-11-25T04:24:35.061 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.061 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:35.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107722+0000", 2023-11-25T04:24:35.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012748534000000001 2023-11-25T04:24:35.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:35.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107844+0000", 2023-11-25T04:24:35.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012233400000000001 2023-11-25T04:24:35.062 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.062 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.062 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:35.062 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109469+0000", 2023-11-25T04:24:35.062 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001625175 2023-11-25T04:24:35.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:35.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109518+0000", 2023-11-25T04:24:35.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8980000000000002e-05 2023-11-25T04:24:35.063 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.063 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.063 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:35.063 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109592+0000", 2023-11-25T04:24:35.063 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3869999999999996e-05 2023-11-25T04:24:35.063 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:35.063 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:35.064 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:35.064 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.064 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2873 4.13 4:cae4f303:::1000000111f.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:35.064 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.094991+0000", 2023-11-25T04:24:35.064 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.8952958290000002, 2023-11-25T04:24:35.064 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015101692, 2023-11-25T04:24:35.064 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:35.064 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:35.064 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:35.064 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.064 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:35.065 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094991+0000", 2023-11-25T04:24:35.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:35.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:35.065 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094991+0000", 2023-11-25T04:24:35.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:35.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.065 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.065 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:35.065 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094992+0000", 2023-11-25T04:24:35.065 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7e-07 2023-11-25T04:24:35.065 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:35.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094994+0000", 2023-11-25T04:24:35.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2110000000000001e-06 2023-11-25T04:24:35.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:35.066 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095000+0000", 2023-11-25T04:24:35.066 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.241e-06 2023-11-25T04:24:35.066 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.066 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.066 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:35.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095006+0000", 2023-11-25T04:24:35.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3659999999999997e-06 2023-11-25T04:24:35.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:35.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108369+0000", 2023-11-25T04:24:35.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01336247 2023-11-25T04:24:35.067 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.067 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.067 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:35.067 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108492+0000", 2023-11-25T04:24:35.067 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000123192 2023-11-25T04:24:35.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:35.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109975+0000", 2023-11-25T04:24:35.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001482894 2023-11-25T04:24:35.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:35.068 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110024+0000", 2023-11-25T04:24:35.068 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9097000000000001e-05 2023-11-25T04:24:35.068 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.068 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.068 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:35.069 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110093+0000", 2023-11-25T04:24:35.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8751000000000006e-05 2023-11-25T04:24:35.069 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:35.069 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:35.069 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:35.069 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.069 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.069 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2874 4.14 4:2b557d2e:::10000001121.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:35.069 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095019+0000", 2023-11-25T04:24:35.069 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.8952683029999999, 2023-11-25T04:24:35.069 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014998296, 2023-11-25T04:24:35.069 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:35.070 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:35.070 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:35.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:35.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095019+0000", 2023-11-25T04:24:35.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:35.070 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.070 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.070 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:35.070 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095019+0000", 2023-11-25T04:24:35.070 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:35.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:35.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095019+0000", 2023-11-25T04:24:35.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7300000000000001e-07 2023-11-25T04:24:35.071 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.071 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.071 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:35.071 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095021+0000", 2023-11-25T04:24:35.071 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0099999999999998e-06 2023-11-25T04:24:35.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:35.072 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095028+0000", 2023-11-25T04:24:35.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7870000000000004e-06 2023-11-25T04:24:35.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.072 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.072 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:35.072 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095033+0000", 2023-11-25T04:24:35.072 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2039999999999996e-06 2023-11-25T04:24:35.072 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.073 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:35.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107740+0000", 2023-11-25T04:24:35.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012706693 2023-11-25T04:24:35.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.073 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.073 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:35.073 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107817+0000", 2023-11-25T04:24:35.073 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7599e-05 2023-11-25T04:24:35.073 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:35.074 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107961+0000", 2023-11-25T04:24:35.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000143194 2023-11-25T04:24:35.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:35.074 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109364+0000", 2023-11-25T04:24:35.074 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001403226 2023-11-25T04:24:35.074 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.074 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.074 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:35.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109507+0000", 2023-11-25T04:24:35.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000142991 2023-11-25T04:24:35.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:35.075 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109986+0000", 2023-11-25T04:24:35.075 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00047906899999999999 2023-11-25T04:24:35.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.075 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:35.076 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110017+0000", 2023-11-25T04:24:35.076 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1050000000000003e-05 2023-11-25T04:24:35.076 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:35.076 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:35.076 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:35.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.076 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2875 4.d 4:b17d5dc6:::10000001123.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:35.076 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095049+0000", 2023-11-25T04:24:35.077 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.8952383790000003, 2023-11-25T04:24:35.077 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014882886, 2023-11-25T04:24:35.077 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:35.077 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:35.078 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:35.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.078 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:35.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095049+0000", 2023-11-25T04:24:35.078 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:35.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.078 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:35.078 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095049+0000", 2023-11-25T04:24:35.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:35.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:35.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095049+0000", 2023-11-25T04:24:35.079 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6100000000000001e-07 2023-11-25T04:24:35.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.079 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:35.079 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095051+0000", 2023-11-25T04:24:35.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1449999999999998e-06 2023-11-25T04:24:35.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.080 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:35.080 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095058+0000", 2023-11-25T04:24:35.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.601e-06 2023-11-25T04:24:35.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.080 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:35.080 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095064+0000", 2023-11-25T04:24:35.080 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1079999999999998e-06 2023-11-25T04:24:35.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:35.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108015+0000", 2023-11-25T04:24:35.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012951582 2023-11-25T04:24:35.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.081 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:35.081 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108260+0000", 2023-11-25T04:24:35.081 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024445199999999998 2023-11-25T04:24:35.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:35.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109825+0000", 2023-11-25T04:24:35.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0015651759999999999 2023-11-25T04:24:35.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:35.082 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109866+0000", 2023-11-25T04:24:35.082 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0629e-05 2023-11-25T04:24:35.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.082 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:35.083 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109931+0000", 2023-11-25T04:24:35.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5732e-05 2023-11-25T04:24:35.083 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:35.083 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:35.083 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:35.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.083 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2876 4.3b 4:dc871776:::10000001125.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:35.083 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095077+0000", 2023-11-25T04:24:35.083 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.895209758, 2023-11-25T04:24:35.083 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014500944, 2023-11-25T04:24:35.084 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:35.084 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:35.084 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:35.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.084 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:35.084 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095077+0000", 2023-11-25T04:24:35.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:35.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.085 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:35.085 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095077+0000", 2023-11-25T04:24:35.085 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:35.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.086 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:35.086 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095078+0000", 2023-11-25T04:24:35.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9799999999999999e-07 2023-11-25T04:24:35.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.086 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:35.086 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095079+0000", 2023-11-25T04:24:35.086 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.877e-06 2023-11-25T04:24:35.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.086 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:35.086 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095086+0000", 2023-11-25T04:24:35.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2439999999999998e-06 2023-11-25T04:24:35.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.087 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:35.087 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095094+0000", 2023-11-25T04:24:35.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9710000000000004e-06 2023-11-25T04:24:35.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.087 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:35.087 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107732+0000", 2023-11-25T04:24:35.087 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012638104000000001 2023-11-25T04:24:35.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:35.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107854+0000", 2023-11-25T04:24:35.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012261600000000001 2023-11-25T04:24:35.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:35.088 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107941+0000", 2023-11-25T04:24:35.088 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6935999999999996e-05 2023-11-25T04:24:35.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.088 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:35.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108871+0000", 2023-11-25T04:24:35.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00092955499999999999 2023-11-25T04:24:35.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:35.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109483+0000", 2023-11-25T04:24:35.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00061206100000000001 2023-11-25T04:24:35.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.089 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:35.089 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109527+0000", 2023-11-25T04:24:35.089 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4112e-05 2023-11-25T04:24:35.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.090 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:35.090 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109578+0000", 2023-11-25T04:24:35.090 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1069999999999997e-05 2023-11-25T04:24:35.090 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:35.090 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:35.090 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:35.090 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.090 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.090 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2878 4.3e 4:7ff98b9d:::10000001129.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:35.090 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095101+0000", 2023-11-25T04:24:35.091 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.8951859750000004, 2023-11-25T04:24:35.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015015137, 2023-11-25T04:24:35.091 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:35.091 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:35.091 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:35.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:35.091 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095101+0000", 2023-11-25T04:24:35.091 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:35.091 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.091 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.091 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:35.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095101+0000", 2023-11-25T04:24:35.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:35.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.092 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:35.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095101+0000", 2023-11-25T04:24:35.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7399999999999999e-07 2023-11-25T04:24:35.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.092 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.092 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:35.092 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095103+0000", 2023-11-25T04:24:35.092 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7600000000000001e-06 2023-11-25T04:24:35.092 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:35.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095109+0000", 2023-11-25T04:24:35.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8189999999999997e-06 2023-11-25T04:24:35.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:35.093 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095114+0000", 2023-11-25T04:24:35.093 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6770000000000003e-06 2023-11-25T04:24:35.093 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.093 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.093 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:35.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107724+0000", 2023-11-25T04:24:35.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012610909999999999 2023-11-25T04:24:35.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:35.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108483+0000", 2023-11-25T04:24:35.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000758407 2023-11-25T04:24:35.094 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.094 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.094 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:35.094 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108568+0000", 2023-11-25T04:24:35.094 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4707000000000001e-05 2023-11-25T04:24:35.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:35.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109687+0000", 2023-11-25T04:24:35.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011192750000000001 2023-11-25T04:24:35.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:35.095 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110049+0000", 2023-11-25T04:24:35.095 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00036210600000000001 2023-11-25T04:24:35.095 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.095 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.095 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:35.096 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110086+0000", 2023-11-25T04:24:35.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7500999999999999e-05 2023-11-25T04:24:35.096 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.096 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.096 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:35.096 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110116+0000", 2023-11-25T04:24:35.096 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9601e-05 2023-11-25T04:24:35.096 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:35.096 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:35.096 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:35.097 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.097 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2879 4.2a 4:5524ace2:::1000000112b.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:35.097 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095120+0000", 2023-11-25T04:24:35.097 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.8951671949999991, 2023-11-25T04:24:35.097 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014587937, 2023-11-25T04:24:35.097 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:35.097 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:35.097 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:35.097 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.097 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:35.098 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095120+0000", 2023-11-25T04:24:35.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:35.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.098 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:35.098 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095120+0000", 2023-11-25T04:24:35.098 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:35.098 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.098 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.098 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:35.098 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095120+0000", 2023-11-25T04:24:35.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5600000000000001e-07 2023-11-25T04:24:35.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.099 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:35.099 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095122+0000", 2023-11-25T04:24:35.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.694e-06 2023-11-25T04:24:35.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.099 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.099 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:35.099 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095127+0000", 2023-11-25T04:24:35.099 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5099999999999998e-06 2023-11-25T04:24:35.099 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:35.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095132+0000", 2023-11-25T04:24:35.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6210000000000003e-06 2023-11-25T04:24:35.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:35.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107729+0000", 2023-11-25T04:24:35.100 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012596931 2023-11-25T04:24:35.100 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.100 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.100 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:35.100 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107856+0000", 2023-11-25T04:24:35.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012707699999999999 2023-11-25T04:24:35.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:35.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109600+0000", 2023-11-25T04:24:35.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001744298 2023-11-25T04:24:35.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.101 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.101 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:35.101 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109641+0000", 2023-11-25T04:24:35.101 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0769000000000002e-05 2023-11-25T04:24:35.101 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.102 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:35.102 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109708+0000", 2023-11-25T04:24:35.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6680999999999994e-05 2023-11-25T04:24:35.102 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:35.102 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:35.102 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:35.102 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.102 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.102 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2882 4.1c 4:3a7c2e07:::10000001131.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:35.102 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095141+0000", 2023-11-25T04:24:35.102 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.8951458199999998, 2023-11-25T04:24:35.102 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014949469999999999, 2023-11-25T04:24:35.103 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:35.103 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:35.103 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:35.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.103 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:35.103 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095141+0000", 2023-11-25T04:24:35.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:35.103 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.103 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.103 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:35.103 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095141+0000", 2023-11-25T04:24:35.103 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:35.104 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.104 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:35.104 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095142+0000", 2023-11-25T04:24:35.104 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1899999999999998e-07 2023-11-25T04:24:35.104 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.104 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:35.104 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095143+0000", 2023-11-25T04:24:35.104 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7770000000000001e-06 2023-11-25T04:24:35.104 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.104 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.104 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:35.105 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095149+0000", 2023-11-25T04:24:35.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9909999999999999e-06 2023-11-25T04:24:35.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.105 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:35.105 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095154+0000", 2023-11-25T04:24:35.105 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3880000000000002e-06 2023-11-25T04:24:35.105 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.105 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.105 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:35.105 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107736+0000", 2023-11-25T04:24:35.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012582477999999999 2023-11-25T04:24:35.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.106 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:35.106 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107868+0000", 2023-11-25T04:24:35.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013134400000000001 2023-11-25T04:24:35.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.106 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.106 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:35.106 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107959+0000", 2023-11-25T04:24:35.106 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1253000000000002e-05 2023-11-25T04:24:35.106 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.107 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:35.107 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109325+0000", 2023-11-25T04:24:35.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0013662520000000001 2023-11-25T04:24:35.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.107 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:35.107 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110023+0000", 2023-11-25T04:24:35.107 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00069838799999999998 2023-11-25T04:24:35.107 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.107 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.107 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:35.108 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110061+0000", 2023-11-25T04:24:35.108 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7867999999999997e-05 2023-11-25T04:24:35.108 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.108 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.108 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:35.108 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110091+0000", 2023-11-25T04:24:35.108 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9312e-05 2023-11-25T04:24:35.108 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:35.109 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:35.109 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:35.109 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.109 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2883 4.3e 4:7d82be99:::10000001133.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:35.109 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095942+0000", 2023-11-25T04:24:35.109 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.894344534, 2023-11-25T04:24:35.109 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014290573000000001, 2023-11-25T04:24:35.109 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:35.109 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:35.109 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:35.109 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.110 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:35.110 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095942+0000", 2023-11-25T04:24:35.110 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:35.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.110 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.110 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:35.110 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095942+0000", 2023-11-25T04:24:35.110 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:35.110 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.111 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.111 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:35.111 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095944+0000", 2023-11-25T04:24:35.111 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.632e-06 2023-11-25T04:24:35.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.111 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.111 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:35.111 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095947+0000", 2023-11-25T04:24:35.111 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7760000000000002e-06 2023-11-25T04:24:35.111 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.111 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.111 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:35.111 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095954+0000", 2023-11-25T04:24:35.112 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5379999999999997e-06 2023-11-25T04:24:35.112 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.112 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.112 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:35.112 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095960+0000", 2023-11-25T04:24:35.112 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9209999999999997e-06 2023-11-25T04:24:35.112 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.112 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.112 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:35.113 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108809+0000", 2023-11-25T04:24:35.113 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012848972 2023-11-25T04:24:35.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.113 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:35.113 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108921+0000", 2023-11-25T04:24:35.113 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000111767 2023-11-25T04:24:35.113 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.113 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.113 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:35.113 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109008+0000", 2023-11-25T04:24:35.114 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6576999999999995e-05 2023-11-25T04:24:35.114 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.114 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.114 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:35.114 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110124+0000", 2023-11-25T04:24:35.114 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011163709999999999 2023-11-25T04:24:35.114 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.114 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.114 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:35.114 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110155+0000", 2023-11-25T04:24:35.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1248999999999998e-05 2023-11-25T04:24:35.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.115 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:35.115 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110202+0000", 2023-11-25T04:24:35.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7097e-05 2023-11-25T04:24:35.115 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.115 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.115 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:35.115 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110233+0000", 2023-11-25T04:24:35.115 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0673000000000003e-05 2023-11-25T04:24:35.116 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:35.116 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:35.116 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:35.116 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.116 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.116 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2886 4.25 4:a579baeb:::10000001139.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:35.116 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095994+0000", 2023-11-25T04:24:35.116 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 9.8942932060000004, 2023-11-25T04:24:35.116 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013825356, 2023-11-25T04:24:35.117 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:35.117 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:35.117 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:35.117 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.117 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:35.117 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095994+0000", 2023-11-25T04:24:35.117 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:35.117 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.118 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:35.118 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095994+0000", 2023-11-25T04:24:35.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:35.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.118 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:35.118 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095996+0000", 2023-11-25T04:24:35.118 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8780000000000001e-06 2023-11-25T04:24:35.118 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.118 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:35.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095998+0000", 2023-11-25T04:24:35.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1560000000000002e-06 2023-11-25T04:24:35.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:35.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.096003+0000", 2023-11-25T04:24:35.119 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1080000000000001e-06 2023-11-25T04:24:35.119 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.119 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.119 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:35.119 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.096010+0000", 2023-11-25T04:24:35.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0650000000000001e-06 2023-11-25T04:24:35.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.120 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:35.120 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107742+0000", 2023-11-25T04:24:35.120 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011731691000000001 2023-11-25T04:24:35.120 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.120 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.120 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:35.120 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108063+0000", 2023-11-25T04:24:35.121 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00032168600000000001 2023-11-25T04:24:35.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.121 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:35.121 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109689+0000", 2023-11-25T04:24:35.121 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001625556 2023-11-25T04:24:35.121 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.121 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.121 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:35.121 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109740+0000", 2023-11-25T04:24:35.121 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1125000000000003e-05 2023-11-25T04:24:35.122 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:35.122 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:35.122 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:35.122 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109819+0000", 2023-11-25T04:24:35.122 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9091000000000006e-05 2023-11-25T04:24:35.122 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:35.122 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:35.122 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:35.122 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:35.122 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:35.122 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:35.124 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:24:35.124 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:35.122+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:35.184 INFO:teuthology.orchestra.run.smithi067.stderr:noscrub is unset 2023-11-25T04:24:35.224 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:35.223+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:35.232 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:35.232 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:24:35.232 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2023-11-25T04:24:35.232 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:35.241 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:24:35.325 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:35.324+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:35.346 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:35.347 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:24:35.347 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2023-11-25T04:24:35.347 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2023-11-25T04:24:35.347 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:35.355 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:24:35.426 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:35.424+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:35.463 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:35.463 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2023-11-25T04:24:35.464 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2023-11-25T04:24:35.464 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2023-11-25T04:24:35.464 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.464 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:35.464 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:24:35.464 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 108.428411915, 2023-11-25T04:24:35.464 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01072943, 2023-11-25T04:24:35.464 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:35.464 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:35.464 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:35.464 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.465 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:35.465 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:24:35.465 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:35.465 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.465 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.465 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:35.465 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:24:35.465 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:35.465 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.465 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.465 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:35.465 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031829+0000", 2023-11-25T04:24:35.465 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.666e-06 2023-11-25T04:24:35.466 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.466 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.466 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:35.466 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031839+0000", 2023-11-25T04:24:35.466 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9490000000000008e-06 2023-11-25T04:24:35.466 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.466 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.466 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:35.466 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031857+0000", 2023-11-25T04:24:35.466 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8093000000000001e-05 2023-11-25T04:24:35.467 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.467 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.467 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:35.467 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031882+0000", 2023-11-25T04:24:35.467 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4294000000000002e-05 2023-11-25T04:24:35.467 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.467 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.467 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:35.467 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033130+0000", 2023-11-25T04:24:35.467 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001248264 2023-11-25T04:24:35.467 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.467 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.467 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:35.468 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033426+0000", 2023-11-25T04:24:35.468 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029620999999999998 2023-11-25T04:24:35.468 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.468 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.468 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:35.468 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033814+0000", 2023-11-25T04:24:35.468 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000387742 2023-11-25T04:24:35.468 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.468 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.468 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:35.468 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036793+0000", 2023-11-25T04:24:35.468 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002979598 2023-11-25T04:24:35.469 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.469 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.469 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:35.469 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042455+0000", 2023-11-25T04:24:35.469 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0056619260000000003 2023-11-25T04:24:35.469 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.469 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.469 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:35.469 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042502+0000", 2023-11-25T04:24:35.469 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7139999999999999e-05 2023-11-25T04:24:35.469 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.469 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.469 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:35.470 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042557+0000", 2023-11-25T04:24:35.470 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4548e-05 2023-11-25T04:24:35.470 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:35.470 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:35.470 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:35.470 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.470 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.470 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:35.470 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:24:35.470 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 108.42721338, 2023-11-25T04:24:35.470 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011505202000000001, 2023-11-25T04:24:35.470 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:35.471 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:35.471 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:35.471 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.471 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:35.471 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:24:35.471 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:35.471 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.471 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.471 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:35.471 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:24:35.471 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:35.471 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.471 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.472 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:35.472 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033027+0000", 2023-11-25T04:24:35.472 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0699999999999999e-06 2023-11-25T04:24:35.472 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.472 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.472 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:35.472 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033035+0000", 2023-11-25T04:24:35.472 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0830000000000004e-06 2023-11-25T04:24:35.472 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.472 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.472 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:35.472 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033059+0000", 2023-11-25T04:24:35.473 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3975e-05 2023-11-25T04:24:35.473 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.473 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.473 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:35.473 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033482+0000", 2023-11-25T04:24:35.473 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042253700000000001 2023-11-25T04:24:35.473 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.473 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.474 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:35.474 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034520+0000", 2023-11-25T04:24:35.474 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001037991 2023-11-25T04:24:35.474 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.474 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.474 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:35.474 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034611+0000", 2023-11-25T04:24:35.474 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1113999999999995e-05 2023-11-25T04:24:35.474 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.474 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.474 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:35.474 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034769+0000", 2023-11-25T04:24:35.475 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000157641 2023-11-25T04:24:35.475 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.475 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.475 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:35.475 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038292+0000", 2023-11-25T04:24:35.476 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035231160000000002 2023-11-25T04:24:35.476 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.476 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.476 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:35.476 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044440+0000", 2023-11-25T04:24:35.476 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0061486980000000002 2023-11-25T04:24:35.476 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.476 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.476 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:35.476 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044495+0000", 2023-11-25T04:24:35.476 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4339000000000003e-05 2023-11-25T04:24:35.476 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.477 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.477 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:35.477 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044531+0000", 2023-11-25T04:24:35.477 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6637999999999997e-05 2023-11-25T04:24:35.477 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:35.477 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:35.477 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:35.477 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.477 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.477 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:26 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:35.477 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:35.477 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 108.42662461800001, 2023-11-25T04:24:35.477 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011086945000000001, 2023-11-25T04:24:35.478 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:35.478 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:35.478 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:35.478 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.478 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:35.478 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:35.478 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:35.478 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.478 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.479 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:35.479 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:35.479 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:35.479 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.479 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.479 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:35.479 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:35.479 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1699999999999998e-07 2023-11-25T04:24:35.479 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.479 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.479 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:35.479 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033627+0000", 2023-11-25T04:24:35.480 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.163e-05 2023-11-25T04:24:35.480 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.480 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.480 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:35.480 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033636+0000", 2023-11-25T04:24:35.480 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6859999999999995e-06 2023-11-25T04:24:35.480 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.480 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.480 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:35.480 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033642+0000", 2023-11-25T04:24:35.480 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3149999999999997e-06 2023-11-25T04:24:35.480 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.481 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.481 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:35.481 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035414+0000", 2023-11-25T04:24:35.481 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0017716380000000001 2023-11-25T04:24:35.481 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.481 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.481 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:35.481 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035539+0000", 2023-11-25T04:24:35.481 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000125477 2023-11-25T04:24:35.481 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.481 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.481 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:35.482 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035672+0000", 2023-11-25T04:24:35.482 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000133316 2023-11-25T04:24:35.482 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.482 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.482 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:35.482 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038649+0000", 2023-11-25T04:24:35.482 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029764679999999999 2023-11-25T04:24:35.482 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.482 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.482 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:35.482 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044578+0000", 2023-11-25T04:24:35.482 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0059293510000000002 2023-11-25T04:24:35.482 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.483 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.483 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:35.483 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044651+0000", 2023-11-25T04:24:35.483 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2513000000000002e-05 2023-11-25T04:24:35.483 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.483 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.483 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:35.483 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044702+0000", 2023-11-25T04:24:35.483 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1034e-05 2023-11-25T04:24:35.483 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:35.483 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:35.483 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:35.484 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.484 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.484 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:29 3.b 3:d0630e4c:::mds0_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:35.484 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:35.484 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 108.42658669799999, 2023-11-25T04:24:35.484 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010980227, 2023-11-25T04:24:35.484 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:35.484 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:35.484 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:35.484 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.484 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:35.484 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:35.484 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:35.485 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.485 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.485 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:35.485 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:35.485 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:35.485 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.485 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.485 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:35.485 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:35.485 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3499999999999996e-07 2023-11-25T04:24:35.486 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.486 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.486 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:35.486 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033658+0000", 2023-11-25T04:24:35.486 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1280000000000001e-06 2023-11-25T04:24:35.486 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.486 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.486 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:35.486 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033667+0000", 2023-11-25T04:24:35.486 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1430000000000004e-06 2023-11-25T04:24:35.486 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.486 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.487 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:35.487 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033678+0000", 2023-11-25T04:24:35.487 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0906e-05 2023-11-25T04:24:35.487 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.487 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.487 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:35.487 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034456+0000", 2023-11-25T04:24:35.487 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00077833899999999996 2023-11-25T04:24:35.487 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.487 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.487 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:35.487 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034575+0000", 2023-11-25T04:24:35.487 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011926000000000001 2023-11-25T04:24:35.488 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.488 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.488 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:35.488 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035001+0000", 2023-11-25T04:24:35.488 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042550999999999998 2023-11-25T04:24:35.488 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.488 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.488 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:35.488 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.037923+0000", 2023-11-25T04:24:35.488 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029219670000000001 2023-11-25T04:24:35.488 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.488 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.489 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:35.489 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044538+0000", 2023-11-25T04:24:35.489 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0066152349999999997 2023-11-25T04:24:35.489 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.489 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.489 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:35.489 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044594+0000", 2023-11-25T04:24:35.489 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5593000000000001e-05 2023-11-25T04:24:35.489 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.489 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.489 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:35.489 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044633+0000", 2023-11-25T04:24:35.489 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9511000000000002e-05 2023-11-25T04:24:35.490 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:35.490 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:35.490 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:35.490 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.490 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.490 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:34 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:35.490 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:24:35.490 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 108.426336585, 2023-11-25T04:24:35.490 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010920445000000001, 2023-11-25T04:24:35.490 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:35.490 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:35.490 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:35.491 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.491 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:35.491 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:24:35.491 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:35.491 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.491 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.491 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:35.491 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:24:35.491 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:35.491 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.491 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.491 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:35.491 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033904+0000", 2023-11-25T04:24:35.492 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6599999999999994e-07 2023-11-25T04:24:35.492 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.492 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.492 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:35.492 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033909+0000", 2023-11-25T04:24:35.492 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5639999999999996e-06 2023-11-25T04:24:35.492 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.492 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.492 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:35.492 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033923+0000", 2023-11-25T04:24:35.492 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3028e-05 2023-11-25T04:24:35.492 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.493 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.493 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:35.493 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034043+0000", 2023-11-25T04:24:35.493 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000120328 2023-11-25T04:24:35.493 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.493 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.493 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:35.493 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035576+0000", 2023-11-25T04:24:35.493 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015333720000000001 2023-11-25T04:24:35.493 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.493 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.493 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:35.493 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035699+0000", 2023-11-25T04:24:35.494 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000123001 2023-11-25T04:24:35.494 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.494 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.494 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:35.494 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044823+0000", 2023-11-25T04:24:35.494 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0091241859999999994 2023-11-25T04:24:35.494 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:35.494 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:35.494 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:35.494 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.494 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.494 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:118 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals,omap-rm-keys] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e44)", 2023-11-25T04:24:35.495 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:17.112648+0000", 2023-11-25T04:24:35.495 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 18.347591244, 2023-11-25T04:24:35.495 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011250779000000001, 2023-11-25T04:24:35.495 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:35.495 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:35.495 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:35.495 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.495 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:35.495 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.112648+0000", 2023-11-25T04:24:35.495 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:35.495 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.495 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.495 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:35.495 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.112648+0000", 2023-11-25T04:24:35.496 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:35.496 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.496 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.496 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:35.496 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.112649+0000", 2023-11-25T04:24:35.496 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0070000000000001e-06 2023-11-25T04:24:35.496 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.496 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.496 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:35.496 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113182+0000", 2023-11-25T04:24:35.496 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00053268100000000002 2023-11-25T04:24:35.497 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.497 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.497 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:35.497 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113196+0000", 2023-11-25T04:24:35.497 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3742e-05 2023-11-25T04:24:35.497 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.497 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.497 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:35.497 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113398+0000", 2023-11-25T04:24:35.497 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000202726 2023-11-25T04:24:35.497 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.497 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.497 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:35.498 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113447+0000", 2023-11-25T04:24:35.498 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8655000000000001e-05 2023-11-25T04:24:35.498 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.498 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.498 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:35.498 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113529+0000", 2023-11-25T04:24:35.498 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1891999999999995e-05 2023-11-25T04:24:35.498 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.498 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.498 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:35.498 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123819+0000", 2023-11-25T04:24:35.499 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010290192 2023-11-25T04:24:35.499 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.499 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.499 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:35.499 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123852+0000", 2023-11-25T04:24:35.499 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2425999999999998e-05 2023-11-25T04:24:35.499 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.499 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.499 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:35.499 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123899+0000", 2023-11-25T04:24:35.499 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7457999999999999e-05 2023-11-25T04:24:35.499 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:35.499 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:35.500 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:35.500 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.500 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.500 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:154 3.1e 3:7bf7e7b5:::200.00000006:head [write 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:35.500 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:21.886430+0000", 2023-11-25T04:24:35.500 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 13.573809978, 2023-11-25T04:24:35.500 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.017800107999999999, 2023-11-25T04:24:35.500 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:35.500 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:35.501 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:35.501 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.501 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:35.501 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.886430+0000", 2023-11-25T04:24:35.501 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:35.501 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.501 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.501 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:35.501 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.886430+0000", 2023-11-25T04:24:35.501 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:35.501 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.501 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.501 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:35.502 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.886431+0000", 2023-11-25T04:24:35.502 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.178e-06 2023-11-25T04:24:35.502 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.502 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.502 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:35.502 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.892277+0000", 2023-11-25T04:24:35.502 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0058467409999999999 2023-11-25T04:24:35.502 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.502 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.502 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:35.502 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.892288+0000", 2023-11-25T04:24:35.502 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0183000000000001e-05 2023-11-25T04:24:35.503 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.503 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.503 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:35.503 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.892292+0000", 2023-11-25T04:24:35.503 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7829999999999998e-06 2023-11-25T04:24:35.503 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.503 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.503 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:35.503 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.892362+0000", 2023-11-25T04:24:35.503 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0034000000000001e-05 2023-11-25T04:24:35.503 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.503 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.503 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:35.504 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.892432+0000", 2023-11-25T04:24:35.504 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9648999999999997e-05 2023-11-25T04:24:35.504 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.504 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.504 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:35.504 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.892550+0000", 2023-11-25T04:24:35.504 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011825 2023-11-25T04:24:35.504 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.504 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.505 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:35.505 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.899106+0000", 2023-11-25T04:24:35.505 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0065556759999999999 2023-11-25T04:24:35.505 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.505 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.505 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:35.505 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.904145+0000", 2023-11-25T04:24:35.505 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0050393490000000003 2023-11-25T04:24:35.505 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.505 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.505 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:35.505 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.904200+0000", 2023-11-25T04:24:35.505 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4234999999999998e-05 2023-11-25T04:24:35.506 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.506 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.506 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:35.506 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.904230+0000", 2023-11-25T04:24:35.506 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0029999999999999e-05 2023-11-25T04:24:35.506 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:35.506 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:35.506 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:35.507 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.507 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.507 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:576 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals,omap-rm-keys] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:35.507 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:22.111407+0000", 2023-11-25T04:24:35.507 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 13.348832372, 2023-11-25T04:24:35.507 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.018987317, 2023-11-25T04:24:35.507 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:35.507 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:35.507 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:35.507 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.507 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:35.507 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111407+0000", 2023-11-25T04:24:35.508 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:35.508 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.508 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.508 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:35.508 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111407+0000", 2023-11-25T04:24:35.508 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:35.508 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.509 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.509 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:35.509 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111408+0000", 2023-11-25T04:24:35.509 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8000000000000004e-07 2023-11-25T04:24:35.509 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.509 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.509 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:35.510 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111659+0000", 2023-11-25T04:24:35.510 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000250537 2023-11-25T04:24:35.510 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.510 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.510 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:35.510 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111666+0000", 2023-11-25T04:24:35.510 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.973e-06 2023-11-25T04:24:35.510 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.510 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.511 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:35.511 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111670+0000", 2023-11-25T04:24:35.511 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.668e-06 2023-11-25T04:24:35.511 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.511 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.511 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:35.511 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111709+0000", 2023-11-25T04:24:35.511 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8800000000000001e-05 2023-11-25T04:24:35.511 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.512 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.512 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:35.512 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111761+0000", 2023-11-25T04:24:35.512 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1495000000000001e-05 2023-11-25T04:24:35.512 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.512 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.512 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:35.512 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.130255+0000", 2023-11-25T04:24:35.512 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.018494345999999998 2023-11-25T04:24:35.513 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.513 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.513 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:35.513 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.130311+0000", 2023-11-25T04:24:35.513 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5902000000000001e-05 2023-11-25T04:24:35.513 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.513 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.513 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:35.513 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.130394+0000", 2023-11-25T04:24:35.514 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3715999999999996e-05 2023-11-25T04:24:35.515 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:35.515 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:35.515 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:35.515 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.515 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.515 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:1067 4.21 4:869a7600:::100000011fa.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:35.515 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:22.155038+0000", 2023-11-25T04:24:35.516 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 13.305201034, 2023-11-25T04:24:35.516 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011313148, 2023-11-25T04:24:35.516 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:35.516 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:35.516 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:35.516 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.516 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:35.516 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155038+0000", 2023-11-25T04:24:35.516 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:35.517 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.517 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.517 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:35.517 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155038+0000", 2023-11-25T04:24:35.517 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:35.517 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.517 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.517 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:35.517 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155039+0000", 2023-11-25T04:24:35.517 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0200000000000002e-07 2023-11-25T04:24:35.517 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.518 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.518 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:35.518 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155049+0000", 2023-11-25T04:24:35.518 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0295000000000001e-05 2023-11-25T04:24:35.518 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.518 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.518 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:35.518 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155058+0000", 2023-11-25T04:24:35.518 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.208e-06 2023-11-25T04:24:35.518 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.519 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.519 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:35.519 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155064+0000", 2023-11-25T04:24:35.519 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4489999999999996e-06 2023-11-25T04:24:35.519 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.519 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.519 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:35.519 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155104+0000", 2023-11-25T04:24:35.519 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0164999999999999e-05 2023-11-25T04:24:35.519 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.520 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.520 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:35.520 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155147+0000", 2023-11-25T04:24:35.520 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2576e-05 2023-11-25T04:24:35.520 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.520 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.520 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:35.520 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155212+0000", 2023-11-25T04:24:35.520 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5516000000000001e-05 2023-11-25T04:24:35.520 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.520 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.521 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:35.521 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.156025+0000", 2023-11-25T04:24:35.521 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00081248599999999996 2023-11-25T04:24:35.521 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.521 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.521 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:35.521 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.166250+0000", 2023-11-25T04:24:35.521 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010224736999999999 2023-11-25T04:24:35.521 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.521 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.522 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:35.522 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.166319+0000", 2023-11-25T04:24:35.522 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9145000000000003e-05 2023-11-25T04:24:35.522 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.522 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.522 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:35.522 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.166352+0000", 2023-11-25T04:24:35.522 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2969000000000002e-05 2023-11-25T04:24:35.522 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:35.522 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:35.523 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:35.523 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.523 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.523 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:1102 4.2f 4:f543b94c:::1000000121d.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:35.523 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:22.158246+0000", 2023-11-25T04:24:35.523 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 13.301993510999999, 2023-11-25T04:24:35.523 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010094396, 2023-11-25T04:24:35.523 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:35.523 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:35.523 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:35.523 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.524 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:35.524 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158246+0000", 2023-11-25T04:24:35.524 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:35.524 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.524 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.524 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:35.525 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158246+0000", 2023-11-25T04:24:35.525 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:35.525 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.525 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.525 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:35.525 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158246+0000", 2023-11-25T04:24:35.525 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5299999999999999e-07 2023-11-25T04:24:35.526 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.526 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.526 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:35.526 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158249+0000", 2023-11-25T04:24:35.526 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2840000000000001e-06 2023-11-25T04:24:35.526 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.526 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.526 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:35.526 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158254+0000", 2023-11-25T04:24:35.526 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3449999999999996e-06 2023-11-25T04:24:35.527 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.527 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.527 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:35.527 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158258+0000", 2023-11-25T04:24:35.527 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5030000000000002e-06 2023-11-25T04:24:35.527 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.527 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.527 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:35.529 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158302+0000", 2023-11-25T04:24:35.529 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3908e-05 2023-11-25T04:24:35.529 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.529 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.529 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:35.529 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158398+0000", 2023-11-25T04:24:35.530 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6898000000000006e-05 2023-11-25T04:24:35.530 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.530 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.530 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:35.530 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158480+0000", 2023-11-25T04:24:35.530 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1578999999999999e-05 2023-11-25T04:24:35.530 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.530 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.530 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:35.530 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.159229+0000", 2023-11-25T04:24:35.531 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000749049 2023-11-25T04:24:35.531 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.531 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.531 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:35.531 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.168228+0000", 2023-11-25T04:24:35.531 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0089992930000000002 2023-11-25T04:24:35.531 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.532 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.532 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:35.532 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.168310+0000", 2023-11-25T04:24:35.532 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1348999999999997e-05 2023-11-25T04:24:35.532 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.532 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.532 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:35.532 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.168340+0000", 2023-11-25T04:24:35.532 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0735000000000003e-05 2023-11-25T04:24:35.533 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:35.533 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:35.533 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:35.533 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.533 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.533 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:1105 4.23 4:c6ff07b4:::10000001220.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:35.533 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:22.158714+0000", 2023-11-25T04:24:35.533 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 13.301525397000001, 2023-11-25T04:24:35.533 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010412490999999999, 2023-11-25T04:24:35.533 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:35.534 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:35.534 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:35.534 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.534 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:35.534 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158714+0000", 2023-11-25T04:24:35.534 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:35.534 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.534 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.534 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:35.534 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158714+0000", 2023-11-25T04:24:35.535 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:35.535 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.535 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.535 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:35.536 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158715+0000", 2023-11-25T04:24:35.536 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4099999999999998e-07 2023-11-25T04:24:35.537 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.537 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.537 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:35.537 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158718+0000", 2023-11-25T04:24:35.537 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4319999999999999e-06 2023-11-25T04:24:35.537 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.538 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.538 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:35.538 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158729+0000", 2023-11-25T04:24:35.538 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.078e-05 2023-11-25T04:24:35.538 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.538 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.538 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:35.538 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158736+0000", 2023-11-25T04:24:35.538 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4649999999999997e-06 2023-11-25T04:24:35.538 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.539 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.539 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:35.539 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158779+0000", 2023-11-25T04:24:35.539 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3049000000000001e-05 2023-11-25T04:24:35.539 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.539 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.539 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:35.539 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158865+0000", 2023-11-25T04:24:35.540 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5138999999999999e-05 2023-11-25T04:24:35.540 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.540 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.540 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:35.540 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158983+0000", 2023-11-25T04:24:35.540 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000118658 2023-11-25T04:24:35.540 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.540 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.541 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:35.541 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.159645+0000", 2023-11-25T04:24:35.541 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00066193999999999997 2023-11-25T04:24:35.541 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.541 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.541 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:35.541 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.169046+0000", 2023-11-25T04:24:35.541 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0094005400000000006 2023-11-25T04:24:35.542 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.542 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.542 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:35.542 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.169092+0000", 2023-11-25T04:24:35.542 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5840000000000002e-05 2023-11-25T04:24:35.542 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.542 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.542 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:35.542 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.169127+0000", 2023-11-25T04:24:35.543 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5006999999999999e-05 2023-11-25T04:24:35.543 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:35.543 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:35.543 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:35.543 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.543 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.543 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2841 4.4 4:22111d42:::100000010df.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:35.543 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.092102+0000", 2023-11-25T04:24:35.543 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 10.368137922000001, 2023-11-25T04:24:35.543 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.016449783999999999, 2023-11-25T04:24:35.544 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:35.544 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:35.544 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:35.544 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.544 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:35.544 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092102+0000", 2023-11-25T04:24:35.544 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:35.544 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.544 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.545 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:35.545 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092102+0000", 2023-11-25T04:24:35.545 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:35.545 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.545 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.545 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:35.546 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092102+0000", 2023-11-25T04:24:35.546 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3700000000000001e-07 2023-11-25T04:24:35.546 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.546 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.546 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:35.546 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092104+0000", 2023-11-25T04:24:35.546 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0679999999999999e-06 2023-11-25T04:24:35.547 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.547 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.547 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:35.547 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092111+0000", 2023-11-25T04:24:35.547 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0589999999999997e-06 2023-11-25T04:24:35.548 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.548 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.548 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:35.548 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092130+0000", 2023-11-25T04:24:35.548 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8686999999999999e-05 2023-11-25T04:24:35.548 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.548 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.548 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:35.548 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092443+0000", 2023-11-25T04:24:35.548 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000313519 2023-11-25T04:24:35.549 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.549 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.549 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:35.549 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092553+0000", 2023-11-25T04:24:35.549 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010932100000000001 2023-11-25T04:24:35.549 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.549 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.549 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:35.549 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092639+0000", 2023-11-25T04:24:35.549 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6094999999999995e-05 2023-11-25T04:24:35.549 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.550 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.550 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:35.550 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094679+0000", 2023-11-25T04:24:35.550 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0020402580000000001 2023-11-25T04:24:35.550 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.550 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.550 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:35.550 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108462+0000", 2023-11-25T04:24:35.550 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013783358000000001 2023-11-25T04:24:35.550 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.551 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.551 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:35.551 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108516+0000", 2023-11-25T04:24:35.551 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3483000000000003e-05 2023-11-25T04:24:35.551 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.551 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.551 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:35.551 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108551+0000", 2023-11-25T04:24:35.551 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5499e-05 2023-11-25T04:24:35.551 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:35.551 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:35.552 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:35.552 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.552 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.552 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2852 4.2d 4:b745ac76:::100000010f5.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:35.552 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.093477+0000", 2023-11-25T04:24:35.552 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 10.366762810000001, 2023-11-25T04:24:35.552 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.015423084, 2023-11-25T04:24:35.552 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:35.552 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:35.553 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:35.553 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.553 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:35.553 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093477+0000", 2023-11-25T04:24:35.553 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:35.553 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.553 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.553 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:35.553 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093477+0000", 2023-11-25T04:24:35.554 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:35.554 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.554 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.554 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:35.554 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093477+0000", 2023-11-25T04:24:35.554 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2500000000000005e-07 2023-11-25T04:24:35.554 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.554 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.554 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:35.555 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093480+0000", 2023-11-25T04:24:35.555 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0189999999999998e-06 2023-11-25T04:24:35.555 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.555 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.555 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:35.555 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093642+0000", 2023-11-25T04:24:35.555 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000162072 2023-11-25T04:24:35.555 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.555 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.555 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:35.556 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093649+0000", 2023-11-25T04:24:35.556 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9659999999999999e-06 2023-11-25T04:24:35.556 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.556 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.556 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:35.556 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093693+0000", 2023-11-25T04:24:35.556 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3794000000000001e-05 2023-11-25T04:24:35.556 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.556 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.557 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:35.557 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093764+0000", 2023-11-25T04:24:35.557 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0981000000000004e-05 2023-11-25T04:24:35.557 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.557 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.557 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:35.557 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093850+0000", 2023-11-25T04:24:35.557 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5536000000000002e-05 2023-11-25T04:24:35.557 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.557 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.558 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:35.558 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094710+0000", 2023-11-25T04:24:35.558 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00086070299999999999 2023-11-25T04:24:35.558 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.558 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.558 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:35.558 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108776+0000", 2023-11-25T04:24:35.558 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01406542 2023-11-25T04:24:35.558 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.559 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.559 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:35.559 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108829+0000", 2023-11-25T04:24:35.559 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2936999999999998e-05 2023-11-25T04:24:35.559 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.559 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.559 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:35.559 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108900+0000", 2023-11-25T04:24:35.559 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1031000000000005e-05 2023-11-25T04:24:35.559 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:35.560 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:35.560 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:35.560 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.560 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.560 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2854 4.2d 4:b5c259cf:::100000010f9.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:35.560 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.093674+0000", 2023-11-25T04:24:35.560 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 10.36656515, 2023-11-25T04:24:35.560 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.015970380999999999, 2023-11-25T04:24:35.560 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:35.560 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:35.561 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:35.561 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.561 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:35.561 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093674+0000", 2023-11-25T04:24:35.561 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:35.561 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.561 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.561 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:35.561 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093674+0000", 2023-11-25T04:24:35.562 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:35.562 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.562 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.562 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:35.562 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093675+0000", 2023-11-25T04:24:35.562 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8400000000000004e-07 2023-11-25T04:24:35.562 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.562 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.562 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:35.563 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093678+0000", 2023-11-25T04:24:35.563 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2229999999999999e-06 2023-11-25T04:24:35.563 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.563 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.563 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:35.563 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093686+0000", 2023-11-25T04:24:35.563 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9990000000000008e-06 2023-11-25T04:24:35.563 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.563 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.563 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:35.564 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093696+0000", 2023-11-25T04:24:35.564 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6479999999999998e-06 2023-11-25T04:24:35.564 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.564 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.564 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:35.564 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094104+0000", 2023-11-25T04:24:35.564 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00040823900000000002 2023-11-25T04:24:35.564 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.564 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.564 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:35.565 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094172+0000", 2023-11-25T04:24:35.565 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8023999999999998e-05 2023-11-25T04:24:35.565 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.565 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.565 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:35.565 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094256+0000", 2023-11-25T04:24:35.565 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3418999999999995e-05 2023-11-25T04:24:35.565 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.565 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.565 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:35.566 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095160+0000", 2023-11-25T04:24:35.566 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00090418599999999997 2023-11-25T04:24:35.566 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.566 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.566 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:35.566 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109551+0000", 2023-11-25T04:24:35.566 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014391571000000001 2023-11-25T04:24:35.566 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.566 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.567 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:35.567 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109601+0000", 2023-11-25T04:24:35.567 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9478000000000003e-05 2023-11-25T04:24:35.567 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.567 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.567 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:35.567 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109645+0000", 2023-11-25T04:24:35.567 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.401e-05 2023-11-25T04:24:35.567 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:35.568 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:35.568 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:35.568 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.568 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.568 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2861 4.4 4:214027a9:::10000001107.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:35.568 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.093829+0000", 2023-11-25T04:24:35.568 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 10.366410732, 2023-11-25T04:24:35.568 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014848917999999999, 2023-11-25T04:24:35.568 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:35.568 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:35.569 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:35.569 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.569 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:35.569 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093829+0000", 2023-11-25T04:24:35.569 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:35.569 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.569 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.569 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:35.569 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093829+0000", 2023-11-25T04:24:35.569 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:35.570 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.570 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.570 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:35.570 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093830+0000", 2023-11-25T04:24:35.570 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1999999999999998e-07 2023-11-25T04:24:35.570 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.570 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.570 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:35.570 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093834+0000", 2023-11-25T04:24:35.570 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.34e-06 2023-11-25T04:24:35.571 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.571 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.571 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:35.571 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093844+0000", 2023-11-25T04:24:35.571 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6690000000000001e-06 2023-11-25T04:24:35.571 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.571 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.571 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:35.571 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093857+0000", 2023-11-25T04:24:35.572 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3434999999999999e-05 2023-11-25T04:24:35.572 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.572 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.572 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:35.572 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093940+0000", 2023-11-25T04:24:35.572 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3047999999999995e-05 2023-11-25T04:24:35.572 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.572 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.572 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:35.572 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094052+0000", 2023-11-25T04:24:35.573 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000111538 2023-11-25T04:24:35.573 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.573 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.573 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:35.573 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094466+0000", 2023-11-25T04:24:35.573 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00041464600000000001 2023-11-25T04:24:35.573 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.573 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.573 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:35.573 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095570+0000", 2023-11-25T04:24:35.574 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001103981 2023-11-25T04:24:35.574 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.574 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.574 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:35.574 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108598+0000", 2023-11-25T04:24:35.574 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013027164000000001 2023-11-25T04:24:35.574 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.574 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.574 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:35.574 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108646+0000", 2023-11-25T04:24:35.575 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8090999999999997e-05 2023-11-25T04:24:35.575 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.575 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.575 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:35.575 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108678+0000", 2023-11-25T04:24:35.575 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2085999999999997e-05 2023-11-25T04:24:35.575 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:35.575 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:35.575 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:35.575 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.576 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.576 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2863 4.12 4:4b84c733:::1000000110b.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:35.576 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.093877+0000", 2023-11-25T04:24:35.576 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 10.366362405, 2023-11-25T04:24:35.576 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014682755, 2023-11-25T04:24:35.576 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:35.576 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:35.576 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:35.576 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.576 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:35.577 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093877+0000", 2023-11-25T04:24:35.577 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:35.577 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.577 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.577 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:35.577 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093877+0000", 2023-11-25T04:24:35.577 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:35.577 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.577 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.577 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:35.578 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093878+0000", 2023-11-25T04:24:35.578 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4199999999999996e-07 2023-11-25T04:24:35.578 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.578 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.578 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:35.578 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093880+0000", 2023-11-25T04:24:35.578 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6189999999999998e-06 2023-11-25T04:24:35.578 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.578 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.579 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:35.579 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093888+0000", 2023-11-25T04:24:35.579 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5510000000000002e-06 2023-11-25T04:24:35.579 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.579 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.579 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:35.579 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093893+0000", 2023-11-25T04:24:35.579 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4179999999999996e-06 2023-11-25T04:24:35.579 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.579 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.580 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:35.580 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093934+0000", 2023-11-25T04:24:35.580 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0648000000000001e-05 2023-11-25T04:24:35.580 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.580 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.580 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:35.580 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094081+0000", 2023-11-25T04:24:35.580 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014753499999999999 2023-11-25T04:24:35.581 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.581 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.581 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:35.581 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094172+0000", 2023-11-25T04:24:35.581 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.0722000000000003e-05 2023-11-25T04:24:35.581 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.581 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.581 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:35.581 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094742+0000", 2023-11-25T04:24:35.582 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00056948600000000002 2023-11-25T04:24:35.582 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.582 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.582 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:35.582 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108491+0000", 2023-11-25T04:24:35.582 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013749568 2023-11-25T04:24:35.582 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.582 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.582 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:35.583 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108525+0000", 2023-11-25T04:24:35.583 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3683999999999998e-05 2023-11-25T04:24:35.583 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.583 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.583 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:35.583 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108560+0000", 2023-11-25T04:24:35.583 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4981999999999998e-05 2023-11-25T04:24:35.583 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:35.583 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:35.584 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:35.584 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.584 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.584 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2880 4.21 4:855834a4:::1000000112d.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:35.584 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.095098+0000", 2023-11-25T04:24:35.584 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 10.365141711, 2023-11-25T04:24:35.584 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013847544, 2023-11-25T04:24:35.584 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:35.584 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:35.584 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:35.585 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.585 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:35.585 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095098+0000", 2023-11-25T04:24:35.585 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:35.585 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.585 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.585 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:35.585 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095098+0000", 2023-11-25T04:24:35.585 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:35.586 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.586 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.586 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:35.586 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095098+0000", 2023-11-25T04:24:35.586 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3000000000000002e-07 2023-11-25T04:24:35.586 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.586 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.586 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:35.586 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095099+0000", 2023-11-25T04:24:35.586 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.331e-06 2023-11-25T04:24:35.587 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.587 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.587 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:35.587 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095103+0000", 2023-11-25T04:24:35.587 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8319999999999999e-06 2023-11-25T04:24:35.587 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.587 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.587 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:35.587 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095108+0000", 2023-11-25T04:24:35.588 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4179999999999999e-06 2023-11-25T04:24:35.588 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.588 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.588 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:35.588 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095153+0000", 2023-11-25T04:24:35.588 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4799000000000003e-05 2023-11-25T04:24:35.588 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.588 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.588 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:35.588 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095268+0000", 2023-11-25T04:24:35.589 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000115437 2023-11-25T04:24:35.589 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.589 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.589 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:35.589 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095363+0000", 2023-11-25T04:24:35.589 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.5278000000000005e-05 2023-11-25T04:24:35.589 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.589 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.589 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:35.589 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095845+0000", 2023-11-25T04:24:35.590 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00048164600000000002 2023-11-25T04:24:35.590 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.590 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.590 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:35.590 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108854+0000", 2023-11-25T04:24:35.590 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013009289 2023-11-25T04:24:35.590 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.590 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.590 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:35.591 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108907+0000", 2023-11-25T04:24:35.591 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2528999999999998e-05 2023-11-25T04:24:35.591 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.591 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.591 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:35.591 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108945+0000", 2023-11-25T04:24:35.591 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8655000000000002e-05 2023-11-25T04:24:35.591 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:35.591 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:35.591 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:35.592 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.592 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.592 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2884 4.37 4:ef909cdc:::10000001135.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:35.592 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.095518+0000", 2023-11-25T04:24:35.592 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 10.364721784, 2023-11-25T04:24:35.592 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014149988, 2023-11-25T04:24:35.592 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:35.592 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:35.592 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:35.592 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.593 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:35.593 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095518+0000", 2023-11-25T04:24:35.593 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:35.593 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.593 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.593 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:35.593 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095518+0000", 2023-11-25T04:24:35.593 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:35.593 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.593 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.594 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:35.594 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095518+0000", 2023-11-25T04:24:35.594 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2900000000000003e-07 2023-11-25T04:24:35.594 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.594 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.594 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:35.594 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095522+0000", 2023-11-25T04:24:35.594 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2119999999999999e-06 2023-11-25T04:24:35.594 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.594 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.595 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:35.595 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095532+0000", 2023-11-25T04:24:35.595 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0375e-05 2023-11-25T04:24:35.595 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.595 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.595 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:35.595 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095537+0000", 2023-11-25T04:24:35.595 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5909999999999998e-06 2023-11-25T04:24:35.595 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.595 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.596 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:35.596 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095584+0000", 2023-11-25T04:24:35.596 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7169999999999997e-05 2023-11-25T04:24:35.596 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.596 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.596 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:35.596 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095695+0000", 2023-11-25T04:24:35.596 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011078699999999999 2023-11-25T04:24:35.596 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.597 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.597 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:35.597 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095777+0000", 2023-11-25T04:24:35.597 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2089999999999995e-05 2023-11-25T04:24:35.597 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.597 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.597 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:35.597 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096286+0000", 2023-11-25T04:24:35.597 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000509247 2023-11-25T04:24:35.598 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.598 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.598 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:35.598 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109596+0000", 2023-11-25T04:24:35.598 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013309872 2023-11-25T04:24:35.598 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.598 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.598 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:35.598 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109638+0000", 2023-11-25T04:24:35.598 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2381e-05 2023-11-25T04:24:35.599 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.599 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.599 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:35.599 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109668+0000", 2023-11-25T04:24:35.599 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9634000000000001e-05 2023-11-25T04:24:35.599 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:35.599 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:35.599 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:35.599 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.599 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.600 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2890 4.12 4:4a2ad640:::10000001141.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:35.600 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.096274+0000", 2023-11-25T04:24:35.600 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 10.363965721, 2023-11-25T04:24:35.600 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013647259, 2023-11-25T04:24:35.600 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:35.600 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:35.600 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:35.600 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.600 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:35.601 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096274+0000", 2023-11-25T04:24:35.601 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:35.601 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.601 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.601 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:35.601 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096274+0000", 2023-11-25T04:24:35.601 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:35.601 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.601 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.602 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:35.602 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096275+0000", 2023-11-25T04:24:35.602 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0299999999999998e-07 2023-11-25T04:24:35.602 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.602 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.602 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:35.602 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096279+0000", 2023-11-25T04:24:35.602 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9580000000000001e-06 2023-11-25T04:24:35.602 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.603 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.603 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:35.603 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096287+0000", 2023-11-25T04:24:35.603 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6929999999999996e-06 2023-11-25T04:24:35.603 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.603 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.603 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:35.603 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096294+0000", 2023-11-25T04:24:35.603 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.348e-06 2023-11-25T04:24:35.604 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.604 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.604 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:35.604 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096342+0000", 2023-11-25T04:24:35.604 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8068000000000001e-05 2023-11-25T04:24:35.604 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.604 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.604 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:35.604 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096462+0000", 2023-11-25T04:24:35.604 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000119918 2023-11-25T04:24:35.605 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.605 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.605 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:35.605 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096557+0000", 2023-11-25T04:24:35.605 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.5340999999999994e-05 2023-11-25T04:24:35.605 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.605 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.605 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:35.605 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096971+0000", 2023-11-25T04:24:35.605 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00041398100000000001 2023-11-25T04:24:35.606 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.606 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.606 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:35.606 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109849+0000", 2023-11-25T04:24:35.606 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.012877985999999999 2023-11-25T04:24:35.606 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.606 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.606 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:35.606 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109894+0000", 2023-11-25T04:24:35.607 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5595e-05 2023-11-25T04:24:35.607 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.607 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.607 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:35.607 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109921+0000", 2023-11-25T04:24:35.607 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6568e-05 2023-11-25T04:24:35.607 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:35.607 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:35.607 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:35.607 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.608 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.608 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2891 4.23 4:c4054dd3:::10000001143.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:35.608 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.096713+0000", 2023-11-25T04:24:35.608 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 10.363527027, 2023-11-25T04:24:35.608 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013148692, 2023-11-25T04:24:35.608 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:35.608 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:35.608 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:35.608 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.608 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:35.608 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096713+0000", 2023-11-25T04:24:35.609 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:35.609 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.609 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.609 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:35.609 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096713+0000", 2023-11-25T04:24:35.609 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:35.609 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.609 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.609 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:35.610 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096713+0000", 2023-11-25T04:24:35.610 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9299999999999998e-07 2023-11-25T04:24:35.610 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.610 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.610 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:35.610 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096716+0000", 2023-11-25T04:24:35.610 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.988e-06 2023-11-25T04:24:35.610 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.610 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.610 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:35.611 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096722+0000", 2023-11-25T04:24:35.611 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0499999999999997e-06 2023-11-25T04:24:35.611 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.611 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.611 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:35.611 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096727+0000", 2023-11-25T04:24:35.611 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3380000000000004e-06 2023-11-25T04:24:35.611 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.611 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.611 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:35.612 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096770+0000", 2023-11-25T04:24:35.612 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2938000000000001e-05 2023-11-25T04:24:35.612 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.612 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.612 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:35.612 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096904+0000", 2023-11-25T04:24:35.612 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000133759 2023-11-25T04:24:35.612 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.612 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.613 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:35.613 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096983+0000", 2023-11-25T04:24:35.613 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8559999999999993e-05 2023-11-25T04:24:35.613 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.613 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.613 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:35.613 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.097437+0000", 2023-11-25T04:24:35.613 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00045439200000000002 2023-11-25T04:24:35.613 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.614 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.614 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:35.614 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109737+0000", 2023-11-25T04:24:35.614 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.012300190000000001 2023-11-25T04:24:35.614 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.614 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.614 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:35.614 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109782+0000", 2023-11-25T04:24:35.614 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4546999999999999e-05 2023-11-25T04:24:35.614 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:35.615 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:35.615 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:35.615 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109861+0000", 2023-11-25T04:24:35.615 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9337000000000003e-05 2023-11-25T04:24:35.615 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:35.615 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:35.615 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:35.615 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:35.615 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:35.615 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:35.617 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:24:35.618 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:35.615+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:35.718 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:35.716+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:35.725 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:35.725 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:24:35.725 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2023-11-25T04:24:35.725 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:35.735 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:24:35.753 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-11-25T04:24:35.819 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:35.817+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:35.871 INFO:teuthology.orchestra.run.smithi067.stderr:Error ENXIO: problem getting command descriptions from osd.0 2023-11-25T04:24:35.880 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:35.880 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:24:35.880 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2023-11-25T04:24:35.880 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2023-11-25T04:24:35.880 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:35.888 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:24:35.891 INFO:teuthology.orchestra.run.smithi067.stdout:osd.1: {} 2023-11-25T04:24:35.891 INFO:teuthology.orchestra.run.smithi067.stderr:osd.1: osd_enable_op_tracker = '' 2023-11-25T04:24:35.911 INFO:teuthology.orchestra.run.smithi067.stdout:osd.2: {} 2023-11-25T04:24:35.911 INFO:teuthology.orchestra.run.smithi067.stderr:osd.2: osd_enable_op_tracker = '' 2023-11-25T04:24:35.914 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-11-25T04:24:35.914 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-11-25T04:24:35.920 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:35.919+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:36.021 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:36.019+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:36.033 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:36.033 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:36.043 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:24:36.122 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:36.120+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:36.155 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:36.156 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:36.164 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:24:36.222 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:36.221+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:36.269 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:36.269 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:36.277 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:24:36.361 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:36.360+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:36.424 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:36.424 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:36.424 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:36.423+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:36.432 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:24:36.525 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:36.523+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:36.539 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:36.539 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:36.548 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:24:36.626 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:36.625+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:36.691 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:36.692 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:36.700 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:24:36.730 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:36.728+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:36.833 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:36.834 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:36.834 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:24:36.835 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:36.832+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:36.934 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~88s 2023-11-25T04:24:36.934 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:36.933+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:36.941 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:36.941 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:36.949 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:24:37.035 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:37.033+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:37.088 INFO:tasks.workunit.client.0.smithi067.stdout: [DEP] 2023-11-25T04:24:37.088 INFO:tasks.workunit.client.0.smithi067.stdout: [DEP] 2023-11-25T04:24:37.092 INFO:tasks.workunit.client.0.smithi067.stdout: [DEP] 2023-11-25T04:24:37.096 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:37.096 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:37.105 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:24:37.136 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:37.135+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:37.200 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2023-11-25T04:24:37.237 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:37.236+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:37.251 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:37.252 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:37.259 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:24:37.338 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:37.337+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:37.366 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:37.366 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:37.375 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:24:37.439 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:37.438+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:37.480 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:37.480 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:37.489 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:24:37.540 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:37.539+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:37.594 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:37.594 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:37.602 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:24:37.616 INFO:tasks.workunit.client.0.smithi067.stdout: [CC] dirstress 2023-11-25T04:24:37.625 INFO:tasks.workunit.client.0.smithi067.stdout: [CC] fill 2023-11-25T04:24:37.625 INFO:tasks.workunit.client.0.smithi067.stdout: [CC] fill2 2023-11-25T04:24:37.628 INFO:tasks.workunit.client.0.smithi067.stdout: [CC] getpagesize 2023-11-25T04:24:37.641 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:37.639+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:37.709 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:37.709 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:37.717 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:24:37.762 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:37.760+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:37.822 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:37.822 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:37.825 INFO:tasks.workunit.client.0.smithi067.stdout: [CC] holes 2023-11-25T04:24:37.831 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:24:37.862 INFO:tasks.workunit.client.0.smithi067.stdout: [CC] lstat64 2023-11-25T04:24:37.873 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:37.871+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:37.915 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-11-25T04:24:37.943 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:37.942+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:37.976 INFO:tasks.workunit.client.0.smithi067.stdout: [CC] nametest 2023-11-25T04:24:37.978 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:37.978 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:37.982 INFO:tasks.workunit.client.0.smithi067.stdout: [CC] permname 2023-11-25T04:24:37.988 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:24:38.044 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:38.043+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:38.063 INFO:teuthology.orchestra.run.smithi067.stderr:Error ENXIO: problem getting command descriptions from osd.0 2023-11-25T04:24:38.084 INFO:teuthology.orchestra.run.smithi067.stdout:osd.1: {} 2023-11-25T04:24:38.084 INFO:teuthology.orchestra.run.smithi067.stderr:osd.1: osd_enable_op_tracker = '' 2023-11-25T04:24:38.094 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:38.094 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:24:38.094 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2023-11-25T04:24:38.094 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:38.103 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:24:38.110 INFO:teuthology.orchestra.run.smithi067.stdout:osd.2: {} 2023-11-25T04:24:38.110 INFO:teuthology.orchestra.run.smithi067.stderr:osd.2: osd_enable_op_tracker = '' 2023-11-25T04:24:38.112 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-11-25T04:24:38.112 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-11-25T04:24:38.146 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:38.144+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:38.167 INFO:tasks.workunit.client.0.smithi067.stdout: [CC] randholes 2023-11-25T04:24:38.170 INFO:tasks.workunit.client.0.smithi067.stdout: [CC] runas 2023-11-25T04:24:38.205 INFO:teuthology.orchestra.run.smithi067.stderr:nodeep-scrub is unset 2023-11-25T04:24:38.208 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:38.208 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:24:38.208 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2023-11-25T04:24:38.208 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2023-11-25T04:24:38.208 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:38.217 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:24:38.246 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:38.245+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:38.302 INFO:tasks.workunit.client.0.smithi067.stdout: [CC] truncfile 2023-11-25T04:24:38.347 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:38.345+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:38.350 INFO:tasks.workunit.client.0.smithi067.stdout: [CC] usemem 2023-11-25T04:24:38.365 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:38.366 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2023-11-25T04:24:38.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2023-11-25T04:24:38.366 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2023-11-25T04:24:38.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.366 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:152 3.1a 3:5e41ab69:::200.00000005:head [write 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:38.366 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:21.237401+0000", 2023-11-25T04:24:38.366 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 17.125792984, 2023-11-25T04:24:38.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015253802, 2023-11-25T04:24:38.366 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:38.366 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:38.366 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:38.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:38.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.237401+0000", 2023-11-25T04:24:38.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:38.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:38.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.237401+0000", 2023-11-25T04:24:38.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:38.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:38.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.237402+0000", 2023-11-25T04:24:38.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3699999999999999e-07 2023-11-25T04:24:38.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:38.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243182+0000", 2023-11-25T04:24:38.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057801270000000004 2023-11-25T04:24:38.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:38.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243193+0000", 2023-11-25T04:24:38.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1221e-05 2023-11-25T04:24:38.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:38.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243198+0000", 2023-11-25T04:24:38.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3739999999999997e-06 2023-11-25T04:24:38.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:38.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243246+0000", 2023-11-25T04:24:38.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7667999999999998e-05 2023-11-25T04:24:38.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:38.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243292+0000", 2023-11-25T04:24:38.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5794000000000002e-05 2023-11-25T04:24:38.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:38.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243360+0000", 2023-11-25T04:24:38.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8479000000000006e-05 2023-11-25T04:24:38.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:38.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.250667+0000", 2023-11-25T04:24:38.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.007306618 2023-11-25T04:24:38.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:38.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.252598+0000", 2023-11-25T04:24:38.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019315129999999999 2023-11-25T04:24:38.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:38.371 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.252631+0000", 2023-11-25T04:24:38.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2925000000000001e-05 2023-11-25T04:24:38.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:38.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.252655+0000", 2023-11-25T04:24:38.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3246000000000001e-05 2023-11-25T04:24:38.372 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:38.372 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:38.372 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:38.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.372 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:2841 4.4 e47/17 4:22111d42:::100000010df.00000000:head v 47'139, mlcod=47'137)", 2023-11-25T04:24:38.372 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.093610+0000", 2023-11-25T04:24:38.372 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.269584120999999, 2023-11-25T04:24:38.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014758871999999999, 2023-11-25T04:24:38.373 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:38.373 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:38.373 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:38.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:38.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093610+0000", 2023-11-25T04:24:38.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:38.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:38.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093610+0000", 2023-11-25T04:24:38.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:38.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:38.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093610+0000", 2023-11-25T04:24:38.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1799999999999999e-07 2023-11-25T04:24:38.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:38.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093621+0000", 2023-11-25T04:24:38.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1107999999999999e-05 2023-11-25T04:24:38.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:38.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093627+0000", 2023-11-25T04:24:38.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0310000000000004e-06 2023-11-25T04:24:38.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:38.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093633+0000", 2023-11-25T04:24:38.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2700000000000001e-06 2023-11-25T04:24:38.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:38.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093722+0000", 2023-11-25T04:24:38.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8417000000000004e-05 2023-11-25T04:24:38.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:38.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093737+0000", 2023-11-25T04:24:38.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5781e-05 2023-11-25T04:24:38.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.377 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:38.377 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108203+0000", 2023-11-25T04:24:38.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014465844 2023-11-25T04:24:38.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.377 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:38.377 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108368+0000", 2023-11-25T04:24:38.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000165203 2023-11-25T04:24:38.377 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:38.377 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:38.377 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:38.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.378 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2860 4.3d 4:bf0d7877:::10000001105.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:38.378 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.093823+0000", 2023-11-25T04:24:38.378 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.269370952999999, 2023-11-25T04:24:38.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015582848, 2023-11-25T04:24:38.378 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:38.378 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:38.378 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:38.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.378 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:38.378 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093823+0000", 2023-11-25T04:24:38.378 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:38.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:38.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093823+0000", 2023-11-25T04:24:38.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:38.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:38.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093823+0000", 2023-11-25T04:24:38.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9100000000000004e-07 2023-11-25T04:24:38.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:38.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093826+0000", 2023-11-25T04:24:38.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4499999999999998e-06 2023-11-25T04:24:38.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:38.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093835+0000", 2023-11-25T04:24:38.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8179999999999993e-06 2023-11-25T04:24:38.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:38.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093840+0000", 2023-11-25T04:24:38.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0819999999999998e-06 2023-11-25T04:24:38.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:38.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093973+0000", 2023-11-25T04:24:38.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013308999999999999 2023-11-25T04:24:38.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:38.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094277+0000", 2023-11-25T04:24:38.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00030403299999999999 2023-11-25T04:24:38.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:38.382 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108608+0000", 2023-11-25T04:24:38.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014330888 2023-11-25T04:24:38.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.382 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:38.382 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108676+0000", 2023-11-25T04:24:38.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8577000000000004e-05 2023-11-25T04:24:38.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.382 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:38.382 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109406+0000", 2023-11-25T04:24:38.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00072931899999999997 2023-11-25T04:24:38.383 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:38.383 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:38.383 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:38.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.383 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2862 4.13 4:cb2e645e:::10000001109.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:38.383 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.093852+0000", 2023-11-25T04:24:38.384 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.269341991999999, 2023-11-25T04:24:38.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015190636, 2023-11-25T04:24:38.384 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:38.384 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:38.384 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:38.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:38.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093852+0000", 2023-11-25T04:24:38.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:38.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.384 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:38.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093852+0000", 2023-11-25T04:24:38.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:38.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.385 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:38.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093852+0000", 2023-11-25T04:24:38.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0299999999999999e-07 2023-11-25T04:24:38.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.385 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:38.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093854+0000", 2023-11-25T04:24:38.385 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.846e-06 2023-11-25T04:24:38.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:38.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093861+0000", 2023-11-25T04:24:38.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5520000000000001e-06 2023-11-25T04:24:38.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:38.386 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093865+0000", 2023-11-25T04:24:38.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7870000000000001e-06 2023-11-25T04:24:38.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:38.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093940+0000", 2023-11-25T04:24:38.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4799e-05 2023-11-25T04:24:38.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:38.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094052+0000", 2023-11-25T04:24:38.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011156500000000001 2023-11-25T04:24:38.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:38.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108928+0000", 2023-11-25T04:24:38.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014876245 2023-11-25T04:24:38.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.388 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:38.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108972+0000", 2023-11-25T04:24:38.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4477999999999997e-05 2023-11-25T04:24:38.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.388 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:38.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109042+0000", 2023-11-25T04:24:38.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9861000000000001e-05 2023-11-25T04:24:38.388 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:38.388 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:38.388 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:38.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.389 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2865 4.36 4:6ebc5ac6:::1000000110f.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:38.389 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.093906+0000", 2023-11-25T04:24:38.389 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.269287718999999, 2023-11-25T04:24:38.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014962628, 2023-11-25T04:24:38.389 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:38.389 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:38.389 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:38.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:38.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093906+0000", 2023-11-25T04:24:38.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:38.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:38.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093906+0000", 2023-11-25T04:24:38.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:38.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:38.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093906+0000", 2023-11-25T04:24:38.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4400000000000001e-07 2023-11-25T04:24:38.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:38.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093908+0000", 2023-11-25T04:24:38.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8640000000000001e-06 2023-11-25T04:24:38.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:38.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093914+0000", 2023-11-25T04:24:38.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0170000000000002e-06 2023-11-25T04:24:38.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:38.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093921+0000", 2023-11-25T04:24:38.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6529999999999997e-06 2023-11-25T04:24:38.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:38.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094258+0000", 2023-11-25T04:24:38.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00033703599999999998 2023-11-25T04:24:38.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:38.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094340+0000", 2023-11-25T04:24:38.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2279000000000003e-05 2023-11-25T04:24:38.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:38.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094416+0000", 2023-11-25T04:24:38.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6248999999999995e-05 2023-11-25T04:24:38.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:38.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.096134+0000", 2023-11-25T04:24:38.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001717611 2023-11-25T04:24:38.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:38.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108809+0000", 2023-11-25T04:24:38.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012675327 2023-11-25T04:24:38.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:38.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108836+0000", 2023-11-25T04:24:38.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6509999999999999e-05 2023-11-25T04:24:38.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:38.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108869+0000", 2023-11-25T04:24:38.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2737999999999997e-05 2023-11-25T04:24:38.398 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:38.398 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:38.398 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:38.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.398 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:2852 4.2d e47/17 4:b745ac76:::100000010f5.00000000:head v 47'126, mlcod=47'124)", 2023-11-25T04:24:38.398 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.093987+0000", 2023-11-25T04:24:38.398 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.269207158, 2023-11-25T04:24:38.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014613256, 2023-11-25T04:24:38.399 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:38.399 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:38.399 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:38.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:38.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093987+0000", 2023-11-25T04:24:38.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:38.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.399 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:38.399 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093987+0000", 2023-11-25T04:24:38.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:38.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:38.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093987+0000", 2023-11-25T04:24:38.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7000000000000001e-07 2023-11-25T04:24:38.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:38.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093990+0000", 2023-11-25T04:24:38.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8729999999999999e-06 2023-11-25T04:24:38.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:38.401 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093997+0000", 2023-11-25T04:24:38.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.973e-06 2023-11-25T04:24:38.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:38.401 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094001+0000", 2023-11-25T04:24:38.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7360000000000001e-06 2023-11-25T04:24:38.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:38.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094263+0000", 2023-11-25T04:24:38.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026173400000000001 2023-11-25T04:24:38.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:38.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094278+0000", 2023-11-25T04:24:38.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4681999999999999e-05 2023-11-25T04:24:38.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:38.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108563+0000", 2023-11-25T04:24:38.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014284738 2023-11-25T04:24:38.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:38.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108600+0000", 2023-11-25T04:24:38.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7249999999999997e-05 2023-11-25T04:24:38.405 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:38.405 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:38.405 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:38.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.405 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2866 4.36 4:6f6907fb:::10000001111.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:38.405 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.094033+0000", 2023-11-25T04:24:38.405 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.269160993, 2023-11-25T04:24:38.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014885591, 2023-11-25T04:24:38.405 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:38.405 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:38.406 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:38.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:38.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094033+0000", 2023-11-25T04:24:38.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:38.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.406 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:38.406 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094033+0000", 2023-11-25T04:24:38.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:38.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:38.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094033+0000", 2023-11-25T04:24:38.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.06e-07 2023-11-25T04:24:38.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:38.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094036+0000", 2023-11-25T04:24:38.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.508e-06 2023-11-25T04:24:38.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:38.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094044+0000", 2023-11-25T04:24:38.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5909999999999996e-06 2023-11-25T04:24:38.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:38.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094051+0000", 2023-11-25T04:24:38.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0319999999999998e-06 2023-11-25T04:24:38.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:38.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094727+0000", 2023-11-25T04:24:38.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00067523300000000002 2023-11-25T04:24:38.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:38.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094795+0000", 2023-11-25T04:24:38.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8156000000000003e-05 2023-11-25T04:24:38.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:38.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094898+0000", 2023-11-25T04:24:38.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103329 2023-11-25T04:24:38.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:38.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108803+0000", 2023-11-25T04:24:38.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013904385 2023-11-25T04:24:38.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:38.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108875+0000", 2023-11-25T04:24:38.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1958999999999993e-05 2023-11-25T04:24:38.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:38.411 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108899+0000", 2023-11-25T04:24:38.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4019000000000002e-05 2023-11-25T04:24:38.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.411 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:38.411 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108918+0000", 2023-11-25T04:24:38.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9772999999999999e-05 2023-11-25T04:24:38.411 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:38.411 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:38.411 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:38.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.412 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:2854 4.2d e47/17 4:b5c259cf:::100000010f9.00000000:head v 47'127, mlcod=47'124)", 2023-11-25T04:24:38.412 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.094793+0000", 2023-11-25T04:24:38.412 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.268400703999999, 2023-11-25T04:24:38.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014650764, 2023-11-25T04:24:38.412 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:38.412 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:38.412 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:38.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.412 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:38.412 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094793+0000", 2023-11-25T04:24:38.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:38.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:38.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094793+0000", 2023-11-25T04:24:38.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:38.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.413 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:38.413 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094793+0000", 2023-11-25T04:24:38.413 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3700000000000001e-07 2023-11-25T04:24:38.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.414 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:38.414 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094798+0000", 2023-11-25T04:24:38.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1200000000000004e-06 2023-11-25T04:24:38.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.414 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:38.414 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094805+0000", 2023-11-25T04:24:38.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9759999999999998e-06 2023-11-25T04:24:38.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:38.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094811+0000", 2023-11-25T04:24:38.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1410000000000002e-06 2023-11-25T04:24:38.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:38.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094838+0000", 2023-11-25T04:24:38.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6995000000000002e-05 2023-11-25T04:24:38.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:38.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094848+0000", 2023-11-25T04:24:38.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0115999999999999e-05 2023-11-25T04:24:38.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:38.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109408+0000", 2023-11-25T04:24:38.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014559825 2023-11-25T04:24:38.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:38.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109444+0000", 2023-11-25T04:24:38.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6154e-05 2023-11-25T04:24:38.416 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:38.417 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:38.417 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:38.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.417 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2869 4.6 4:63719fb0:::10000001117.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:38.417 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.094910+0000", 2023-11-25T04:24:38.417 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.268283747, 2023-11-25T04:24:38.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014132570000000001, 2023-11-25T04:24:38.417 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:38.417 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:38.417 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:38.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:38.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094910+0000", 2023-11-25T04:24:38.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:38.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:38.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094910+0000", 2023-11-25T04:24:38.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:38.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:38.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094910+0000", 2023-11-25T04:24:38.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.58e-07 2023-11-25T04:24:38.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:38.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094913+0000", 2023-11-25T04:24:38.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.198e-06 2023-11-25T04:24:38.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:38.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094921+0000", 2023-11-25T04:24:38.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1629999999999996e-06 2023-11-25T04:24:38.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:38.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094926+0000", 2023-11-25T04:24:38.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7559999999999999e-06 2023-11-25T04:24:38.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.420 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:38.420 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094966+0000", 2023-11-25T04:24:38.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0151999999999998e-05 2023-11-25T04:24:38.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:38.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095073+0000", 2023-11-25T04:24:38.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107657 2023-11-25T04:24:38.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:38.421 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095158+0000", 2023-11-25T04:24:38.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5136999999999995e-05 2023-11-25T04:24:38.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:38.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108924+0000", 2023-11-25T04:24:38.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013765727 2023-11-25T04:24:38.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:38.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108958+0000", 2023-11-25T04:24:38.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3417000000000002e-05 2023-11-25T04:24:38.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:38.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109012+0000", 2023-11-25T04:24:38.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3939e-05 2023-11-25T04:24:38.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:38.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109043+0000", 2023-11-25T04:24:38.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0966000000000001e-05 2023-11-25T04:24:38.424 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:38.425 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:38.425 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:38.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.425 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2871 4.3f 4:fff10e11:::1000000111b.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:38.425 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.094936+0000", 2023-11-25T04:24:38.426 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.268257909000001, 2023-11-25T04:24:38.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014261726000000001, 2023-11-25T04:24:38.426 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:38.426 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:38.426 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:38.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:38.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094936+0000", 2023-11-25T04:24:38.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:38.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.427 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:38.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094936+0000", 2023-11-25T04:24:38.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:38.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:38.428 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094936+0000", 2023-11-25T04:24:38.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9900000000000001e-07 2023-11-25T04:24:38.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:38.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094938+0000", 2023-11-25T04:24:38.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9599999999999999e-06 2023-11-25T04:24:38.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.430 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:38.430 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094944+0000", 2023-11-25T04:24:38.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2219999999999999e-06 2023-11-25T04:24:38.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:38.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094950+0000", 2023-11-25T04:24:38.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1329999999999998e-06 2023-11-25T04:24:38.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:38.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094993+0000", 2023-11-25T04:24:38.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3305e-05 2023-11-25T04:24:38.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:38.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095107+0000", 2023-11-25T04:24:38.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011449400000000001 2023-11-25T04:24:38.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:38.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095176+0000", 2023-11-25T04:24:38.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8330000000000005e-05 2023-11-25T04:24:38.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:38.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107939+0000", 2023-11-25T04:24:38.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012763801 2023-11-25T04:24:38.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:38.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109128+0000", 2023-11-25T04:24:38.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011884770000000001 2023-11-25T04:24:38.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:38.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109166+0000", 2023-11-25T04:24:38.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7948999999999999e-05 2023-11-25T04:24:38.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.434 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:38.434 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109198+0000", 2023-11-25T04:24:38.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1656000000000003e-05 2023-11-25T04:24:38.434 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:38.434 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:38.434 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:38.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.435 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2872 4.32 4:4fb002bc:::1000000111d.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:38.435 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.094961+0000", 2023-11-25T04:24:38.435 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.268233205, 2023-11-25T04:24:38.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014631564999999999, 2023-11-25T04:24:38.435 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:38.435 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:38.435 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:38.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:38.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094961+0000", 2023-11-25T04:24:38.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:38.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:38.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094961+0000", 2023-11-25T04:24:38.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:38.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:38.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094961+0000", 2023-11-25T04:24:38.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9299999999999999e-07 2023-11-25T04:24:38.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:38.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094963+0000", 2023-11-25T04:24:38.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9209999999999999e-06 2023-11-25T04:24:38.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:38.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094968+0000", 2023-11-25T04:24:38.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.502e-06 2023-11-25T04:24:38.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:38.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094973+0000", 2023-11-25T04:24:38.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8559999999999998e-06 2023-11-25T04:24:38.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.438 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:38.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107722+0000", 2023-11-25T04:24:38.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012748534000000001 2023-11-25T04:24:38.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.438 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:38.439 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107844+0000", 2023-11-25T04:24:38.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012233400000000001 2023-11-25T04:24:38.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.439 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:38.439 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109469+0000", 2023-11-25T04:24:38.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001625175 2023-11-25T04:24:38.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:38.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109518+0000", 2023-11-25T04:24:38.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8980000000000002e-05 2023-11-25T04:24:38.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:38.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109592+0000", 2023-11-25T04:24:38.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3869999999999996e-05 2023-11-25T04:24:38.442 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:38.442 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:38.442 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:38.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.442 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2873 4.13 4:cae4f303:::1000000111f.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:38.442 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.094991+0000", 2023-11-25T04:24:38.442 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.268202613, 2023-11-25T04:24:38.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015101692, 2023-11-25T04:24:38.442 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:38.442 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:38.443 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:38.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:38.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094991+0000", 2023-11-25T04:24:38.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:38.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:38.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094991+0000", 2023-11-25T04:24:38.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:38.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.445 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:38.445 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094992+0000", 2023-11-25T04:24:38.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7e-07 2023-11-25T04:24:38.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.445 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:38.445 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094994+0000", 2023-11-25T04:24:38.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2110000000000001e-06 2023-11-25T04:24:38.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:38.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095000+0000", 2023-11-25T04:24:38.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.241e-06 2023-11-25T04:24:38.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:38.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095006+0000", 2023-11-25T04:24:38.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3659999999999997e-06 2023-11-25T04:24:38.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.447 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:38.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108369+0000", 2023-11-25T04:24:38.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01336247 2023-11-25T04:24:38.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:38.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108492+0000", 2023-11-25T04:24:38.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000123192 2023-11-25T04:24:38.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.449 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:38.449 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109975+0000", 2023-11-25T04:24:38.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001482894 2023-11-25T04:24:38.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.450 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:38.450 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110024+0000", 2023-11-25T04:24:38.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9097000000000001e-05 2023-11-25T04:24:38.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.451 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:38.451 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110093+0000", 2023-11-25T04:24:38.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8751000000000006e-05 2023-11-25T04:24:38.451 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:38.452 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:38.452 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:38.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.452 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2874 4.14 4:2b557d2e:::10000001121.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:38.452 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095019+0000", 2023-11-25T04:24:38.453 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.268175086999999, 2023-11-25T04:24:38.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014998296, 2023-11-25T04:24:38.453 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:38.453 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:38.453 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:38.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.454 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:38.454 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095019+0000", 2023-11-25T04:24:38.454 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:38.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:38.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095019+0000", 2023-11-25T04:24:38.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:38.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:38.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095019+0000", 2023-11-25T04:24:38.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7300000000000001e-07 2023-11-25T04:24:38.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:38.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095021+0000", 2023-11-25T04:24:38.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0099999999999998e-06 2023-11-25T04:24:38.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:38.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095028+0000", 2023-11-25T04:24:38.458 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7870000000000004e-06 2023-11-25T04:24:38.458 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.458 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.458 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:38.458 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095033+0000", 2023-11-25T04:24:38.459 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2039999999999996e-06 2023-11-25T04:24:38.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.459 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:38.459 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107740+0000", 2023-11-25T04:24:38.459 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012706693 2023-11-25T04:24:38.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:38.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107817+0000", 2023-11-25T04:24:38.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7599e-05 2023-11-25T04:24:38.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:38.461 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107961+0000", 2023-11-25T04:24:38.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000143194 2023-11-25T04:24:38.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.462 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:38.462 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109364+0000", 2023-11-25T04:24:38.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001403226 2023-11-25T04:24:38.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.463 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.463 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:38.463 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109507+0000", 2023-11-25T04:24:38.463 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000142991 2023-11-25T04:24:38.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.464 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:38.464 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109986+0000", 2023-11-25T04:24:38.464 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00047906899999999999 2023-11-25T04:24:38.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.465 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:38.465 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110017+0000", 2023-11-25T04:24:38.465 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1050000000000003e-05 2023-11-25T04:24:38.465 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:38.465 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:38.466 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:38.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.466 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2875 4.d 4:b17d5dc6:::10000001123.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:38.466 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095049+0000", 2023-11-25T04:24:38.466 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.268145163, 2023-11-25T04:24:38.467 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014882886, 2023-11-25T04:24:38.467 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:38.467 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:38.467 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:38.467 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.468 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:38.468 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095049+0000", 2023-11-25T04:24:38.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:38.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.468 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:38.469 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095049+0000", 2023-11-25T04:24:38.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:38.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.469 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.469 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:38.470 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095049+0000", 2023-11-25T04:24:38.470 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6100000000000001e-07 2023-11-25T04:24:38.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.470 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:38.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095051+0000", 2023-11-25T04:24:38.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1449999999999998e-06 2023-11-25T04:24:38.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.471 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:38.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095058+0000", 2023-11-25T04:24:38.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.601e-06 2023-11-25T04:24:38.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.472 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:38.472 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095064+0000", 2023-11-25T04:24:38.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1079999999999998e-06 2023-11-25T04:24:38.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.473 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:38.473 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108015+0000", 2023-11-25T04:24:38.473 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012951582 2023-11-25T04:24:38.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.474 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.474 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:38.474 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108260+0000", 2023-11-25T04:24:38.474 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024445199999999998 2023-11-25T04:24:38.474 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.474 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.475 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:38.475 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109825+0000", 2023-11-25T04:24:38.475 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0015651759999999999 2023-11-25T04:24:38.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.475 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.475 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:38.475 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109866+0000", 2023-11-25T04:24:38.476 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0629e-05 2023-11-25T04:24:38.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.476 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:38.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109931+0000", 2023-11-25T04:24:38.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5732e-05 2023-11-25T04:24:38.477 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:38.477 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:38.477 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:38.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.478 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2876 4.3b 4:dc871776:::10000001125.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:38.478 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095077+0000", 2023-11-25T04:24:38.478 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.268116542, 2023-11-25T04:24:38.478 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014500944, 2023-11-25T04:24:38.478 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:38.478 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:38.479 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:38.479 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.479 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:38.479 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095077+0000", 2023-11-25T04:24:38.479 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:38.479 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.480 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.480 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:38.480 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095077+0000", 2023-11-25T04:24:38.480 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:38.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.481 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:38.481 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095078+0000", 2023-11-25T04:24:38.481 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9799999999999999e-07 2023-11-25T04:24:38.481 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.482 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:38.482 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095079+0000", 2023-11-25T04:24:38.482 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.877e-06 2023-11-25T04:24:38.482 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.482 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.482 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:38.483 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095086+0000", 2023-11-25T04:24:38.483 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2439999999999998e-06 2023-11-25T04:24:38.483 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.483 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.484 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:38.484 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095094+0000", 2023-11-25T04:24:38.484 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9710000000000004e-06 2023-11-25T04:24:38.484 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.484 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.485 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:38.485 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107732+0000", 2023-11-25T04:24:38.485 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012638104000000001 2023-11-25T04:24:38.485 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.485 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.485 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:38.486 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107854+0000", 2023-11-25T04:24:38.486 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012261600000000001 2023-11-25T04:24:38.486 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.486 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.486 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:38.487 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107941+0000", 2023-11-25T04:24:38.487 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6935999999999996e-05 2023-11-25T04:24:38.487 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.487 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.487 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:38.487 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108871+0000", 2023-11-25T04:24:38.488 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00092955499999999999 2023-11-25T04:24:38.488 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.488 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.488 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:38.488 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109483+0000", 2023-11-25T04:24:38.488 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00061206100000000001 2023-11-25T04:24:38.489 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.489 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.489 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:38.489 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109527+0000", 2023-11-25T04:24:38.489 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4112e-05 2023-11-25T04:24:38.490 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.490 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.490 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:38.490 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109578+0000", 2023-11-25T04:24:38.490 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1069999999999997e-05 2023-11-25T04:24:38.490 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:38.491 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:38.491 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:38.491 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.491 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.491 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2878 4.3e 4:7ff98b9d:::10000001129.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:38.492 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095101+0000", 2023-11-25T04:24:38.492 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.268092759, 2023-11-25T04:24:38.492 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015015137, 2023-11-25T04:24:38.492 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:38.492 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:38.492 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:38.493 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.493 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:38.493 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095101+0000", 2023-11-25T04:24:38.493 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:38.493 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.494 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.494 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:38.494 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095101+0000", 2023-11-25T04:24:38.494 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:38.494 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.494 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.495 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:38.495 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095101+0000", 2023-11-25T04:24:38.495 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7399999999999999e-07 2023-11-25T04:24:38.495 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.495 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.495 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:38.496 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095103+0000", 2023-11-25T04:24:38.496 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7600000000000001e-06 2023-11-25T04:24:38.496 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.496 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.496 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:38.497 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095109+0000", 2023-11-25T04:24:38.497 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8189999999999997e-06 2023-11-25T04:24:38.497 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.497 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.497 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:38.497 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095114+0000", 2023-11-25T04:24:38.498 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6770000000000003e-06 2023-11-25T04:24:38.498 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.498 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.498 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:38.498 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107724+0000", 2023-11-25T04:24:38.498 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012610909999999999 2023-11-25T04:24:38.499 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.499 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.499 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:38.499 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108483+0000", 2023-11-25T04:24:38.499 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000758407 2023-11-25T04:24:38.499 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.499 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.500 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:38.500 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108568+0000", 2023-11-25T04:24:38.500 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4707000000000001e-05 2023-11-25T04:24:38.500 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.500 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.500 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:38.501 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109687+0000", 2023-11-25T04:24:38.501 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011192750000000001 2023-11-25T04:24:38.501 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.501 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.501 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:38.501 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110049+0000", 2023-11-25T04:24:38.502 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00036210600000000001 2023-11-25T04:24:38.502 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.502 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.502 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:38.502 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110086+0000", 2023-11-25T04:24:38.503 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7500999999999999e-05 2023-11-25T04:24:38.503 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.503 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.503 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:38.503 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110116+0000", 2023-11-25T04:24:38.503 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9601e-05 2023-11-25T04:24:38.503 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:38.503 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:38.504 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:38.504 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.504 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.504 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2879 4.2a 4:5524ace2:::1000000112b.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:38.504 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095120+0000", 2023-11-25T04:24:38.504 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.268073979, 2023-11-25T04:24:38.504 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014587937, 2023-11-25T04:24:38.505 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:38.505 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:38.505 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:38.505 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.505 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:38.505 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095120+0000", 2023-11-25T04:24:38.505 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:38.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.506 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.506 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:38.506 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095120+0000", 2023-11-25T04:24:38.506 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:38.506 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.506 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.506 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:38.507 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095120+0000", 2023-11-25T04:24:38.507 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5600000000000001e-07 2023-11-25T04:24:38.507 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.507 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.507 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:38.507 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095122+0000", 2023-11-25T04:24:38.508 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.694e-06 2023-11-25T04:24:38.508 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.508 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.508 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:38.508 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095127+0000", 2023-11-25T04:24:38.508 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5099999999999998e-06 2023-11-25T04:24:38.508 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.509 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.509 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:38.509 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095132+0000", 2023-11-25T04:24:38.509 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6210000000000003e-06 2023-11-25T04:24:38.509 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.509 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.509 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:38.509 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107729+0000", 2023-11-25T04:24:38.510 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012596931 2023-11-25T04:24:38.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.510 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.510 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:38.510 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107856+0000", 2023-11-25T04:24:38.510 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012707699999999999 2023-11-25T04:24:38.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.510 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.510 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:38.511 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109600+0000", 2023-11-25T04:24:38.511 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001744298 2023-11-25T04:24:38.511 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.511 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.511 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:38.511 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109641+0000", 2023-11-25T04:24:38.511 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0769000000000002e-05 2023-11-25T04:24:38.512 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.512 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.512 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:38.512 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109708+0000", 2023-11-25T04:24:38.512 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6680999999999994e-05 2023-11-25T04:24:38.512 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:38.513 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:38.513 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:38.513 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.513 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.513 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2882 4.1c 4:3a7c2e07:::10000001131.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:38.513 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095141+0000", 2023-11-25T04:24:38.514 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.268052603999999, 2023-11-25T04:24:38.514 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014949469999999999, 2023-11-25T04:24:38.514 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:38.514 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:38.514 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:38.515 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.515 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:38.515 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095141+0000", 2023-11-25T04:24:38.515 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:38.515 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.515 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.516 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:38.516 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095141+0000", 2023-11-25T04:24:38.516 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:38.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.516 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.516 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:38.517 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095142+0000", 2023-11-25T04:24:38.517 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1899999999999998e-07 2023-11-25T04:24:38.517 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.517 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.517 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:38.517 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095143+0000", 2023-11-25T04:24:38.517 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7770000000000001e-06 2023-11-25T04:24:38.517 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.518 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.518 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:38.518 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095149+0000", 2023-11-25T04:24:38.518 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9909999999999999e-06 2023-11-25T04:24:38.518 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.518 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.518 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:38.518 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095154+0000", 2023-11-25T04:24:38.518 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3880000000000002e-06 2023-11-25T04:24:38.519 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.519 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.519 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:38.519 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107736+0000", 2023-11-25T04:24:38.519 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012582477999999999 2023-11-25T04:24:38.519 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.520 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.520 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:38.520 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107868+0000", 2023-11-25T04:24:38.520 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013134400000000001 2023-11-25T04:24:38.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.520 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.521 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:38.521 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107959+0000", 2023-11-25T04:24:38.521 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1253000000000002e-05 2023-11-25T04:24:38.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.521 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.521 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:38.522 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109325+0000", 2023-11-25T04:24:38.522 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0013662520000000001 2023-11-25T04:24:38.522 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.522 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.522 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:38.522 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110023+0000", 2023-11-25T04:24:38.522 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00069838799999999998 2023-11-25T04:24:38.523 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.523 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.523 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:38.523 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110061+0000", 2023-11-25T04:24:38.523 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7867999999999997e-05 2023-11-25T04:24:38.523 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.523 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.524 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:38.524 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110091+0000", 2023-11-25T04:24:38.524 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9312e-05 2023-11-25T04:24:38.524 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:38.524 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:38.524 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:38.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.525 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.525 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2883 4.3e 4:7d82be99:::10000001133.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:38.525 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095942+0000", 2023-11-25T04:24:38.525 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.267251318, 2023-11-25T04:24:38.525 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014290573000000001, 2023-11-25T04:24:38.525 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:38.526 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:38.526 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:38.526 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.526 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:38.526 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095942+0000", 2023-11-25T04:24:38.526 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:38.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.527 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.527 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:38.527 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095942+0000", 2023-11-25T04:24:38.527 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:38.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.528 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.528 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:38.528 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095944+0000", 2023-11-25T04:24:38.528 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.632e-06 2023-11-25T04:24:38.528 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.528 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.528 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:38.529 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095947+0000", 2023-11-25T04:24:38.529 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7760000000000002e-06 2023-11-25T04:24:38.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.529 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.529 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:38.530 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095954+0000", 2023-11-25T04:24:38.530 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5379999999999997e-06 2023-11-25T04:24:38.530 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.530 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.530 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:38.530 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095960+0000", 2023-11-25T04:24:38.530 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9209999999999997e-06 2023-11-25T04:24:38.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.531 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.531 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:38.531 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108809+0000", 2023-11-25T04:24:38.531 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012848972 2023-11-25T04:24:38.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.531 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.532 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:38.532 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108921+0000", 2023-11-25T04:24:38.532 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000111767 2023-11-25T04:24:38.532 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.532 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.532 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:38.532 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109008+0000", 2023-11-25T04:24:38.533 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6576999999999995e-05 2023-11-25T04:24:38.533 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.533 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.533 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:38.533 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110124+0000", 2023-11-25T04:24:38.533 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011163709999999999 2023-11-25T04:24:38.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.534 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.534 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:38.534 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110155+0000", 2023-11-25T04:24:38.534 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1248999999999998e-05 2023-11-25T04:24:38.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.534 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.535 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:38.535 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110202+0000", 2023-11-25T04:24:38.535 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7097e-05 2023-11-25T04:24:38.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.535 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:38.536 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110233+0000", 2023-11-25T04:24:38.536 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0673000000000003e-05 2023-11-25T04:24:38.536 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:38.536 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:38.536 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:38.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.536 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.536 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2886 4.25 4:a579baeb:::10000001139.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:38.536 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095994+0000", 2023-11-25T04:24:38.536 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 13.26719999, 2023-11-25T04:24:38.537 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013825356, 2023-11-25T04:24:38.537 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:38.537 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:38.537 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:38.537 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.537 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:38.537 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095994+0000", 2023-11-25T04:24:38.537 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:38.537 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.537 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.537 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:38.537 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095994+0000", 2023-11-25T04:24:38.538 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:38.538 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.538 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.538 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:38.538 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095996+0000", 2023-11-25T04:24:38.538 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8780000000000001e-06 2023-11-25T04:24:38.538 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.538 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.538 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:38.538 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095998+0000", 2023-11-25T04:24:38.538 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1560000000000002e-06 2023-11-25T04:24:38.538 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.539 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:38.539 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.096003+0000", 2023-11-25T04:24:38.539 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1080000000000001e-06 2023-11-25T04:24:38.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.539 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:38.539 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.096010+0000", 2023-11-25T04:24:38.539 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0650000000000001e-06 2023-11-25T04:24:38.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.539 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:38.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107742+0000", 2023-11-25T04:24:38.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011731691000000001 2023-11-25T04:24:38.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.540 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.540 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:38.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108063+0000", 2023-11-25T04:24:38.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00032168600000000001 2023-11-25T04:24:38.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.540 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.540 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:38.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109689+0000", 2023-11-25T04:24:38.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001625556 2023-11-25T04:24:38.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.541 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.541 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:38.541 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109740+0000", 2023-11-25T04:24:38.541 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1125000000000003e-05 2023-11-25T04:24:38.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:38.541 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:38.541 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:38.541 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109819+0000", 2023-11-25T04:24:38.541 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9091000000000006e-05 2023-11-25T04:24:38.541 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:38.541 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:38.542 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:38.542 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:38.542 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:38.542 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:38.543 INFO:tasks.workunit.client.0.smithi067.stdout: [CC] mmapcat 2023-11-25T04:24:38.544 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:24:38.544 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:38.541+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:38.546 INFO:tasks.workunit.client.0.smithi067.stdout: [CC] append_reader 2023-11-25T04:24:38.585 INFO:tasks.workunit.client.0.smithi067.stdout: [CC] append_writer 2023-11-25T04:24:38.588 INFO:tasks.workunit.client.0.smithi067.stdout: [CC] dirperf 2023-11-25T04:24:38.644 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:38.642+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:38.651 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:38.652 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:24:38.652 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2023-11-25T04:24:38.652 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:38.661 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:24:38.692 INFO:tasks.workunit.client.0.smithi067.stdout: [CC] metaperf 2023-11-25T04:24:38.745 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:38.743+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:38.786 INFO:tasks.workunit.client.0.smithi067.stdout: [CC] devzero 2023-11-25T04:24:38.810 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:38.810 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:24:38.810 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2023-11-25T04:24:38.810 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2023-11-25T04:24:38.810 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:38.817 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:24:38.822 INFO:tasks.workunit.client.0.smithi067.stdout: [CC] feature 2023-11-25T04:24:38.860 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:38.859+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:38.924 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:38.924 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2023-11-25T04:24:38.924 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2023-11-25T04:24:38.924 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2023-11-25T04:24:38.924 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.924 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:38.925 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:24:38.925 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 111.889954166, 2023-11-25T04:24:38.925 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01072943, 2023-11-25T04:24:38.925 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:38.925 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:38.925 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:38.925 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.925 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:38.925 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:24:38.925 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:38.925 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.925 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.926 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:38.926 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:24:38.926 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:38.926 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.926 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.926 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:38.926 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031829+0000", 2023-11-25T04:24:38.926 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.666e-06 2023-11-25T04:24:38.926 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.926 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.926 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:38.926 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031839+0000", 2023-11-25T04:24:38.927 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9490000000000008e-06 2023-11-25T04:24:38.927 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.927 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.927 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:38.927 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031857+0000", 2023-11-25T04:24:38.928 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8093000000000001e-05 2023-11-25T04:24:38.928 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.928 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.928 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:38.928 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031882+0000", 2023-11-25T04:24:38.928 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4294000000000002e-05 2023-11-25T04:24:38.928 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.928 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.928 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:38.928 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033130+0000", 2023-11-25T04:24:38.928 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001248264 2023-11-25T04:24:38.928 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.929 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.930 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:38.930 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033426+0000", 2023-11-25T04:24:38.930 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029620999999999998 2023-11-25T04:24:38.930 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.930 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.931 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:38.931 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033814+0000", 2023-11-25T04:24:38.931 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000387742 2023-11-25T04:24:38.931 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.931 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.931 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:38.931 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036793+0000", 2023-11-25T04:24:38.932 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002979598 2023-11-25T04:24:38.932 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.932 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.932 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:38.932 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042455+0000", 2023-11-25T04:24:38.932 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0056619260000000003 2023-11-25T04:24:38.932 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.932 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.932 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:38.932 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042502+0000", 2023-11-25T04:24:38.933 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7139999999999999e-05 2023-11-25T04:24:38.933 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.933 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.933 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:38.933 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042557+0000", 2023-11-25T04:24:38.933 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4548e-05 2023-11-25T04:24:38.933 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:38.933 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:38.933 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:38.933 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.933 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.934 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:38.934 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:24:38.934 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 111.888755631, 2023-11-25T04:24:38.934 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011505202000000001, 2023-11-25T04:24:38.934 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:38.934 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:38.934 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:38.934 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.934 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:38.934 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:24:38.934 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:38.935 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.935 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.935 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:38.935 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:24:38.935 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:38.935 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.935 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.935 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:38.935 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033027+0000", 2023-11-25T04:24:38.935 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0699999999999999e-06 2023-11-25T04:24:38.935 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.935 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.935 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:38.936 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033035+0000", 2023-11-25T04:24:38.936 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0830000000000004e-06 2023-11-25T04:24:38.936 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.936 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.936 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:38.936 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033059+0000", 2023-11-25T04:24:38.936 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3975e-05 2023-11-25T04:24:38.936 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.936 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.936 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:38.936 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033482+0000", 2023-11-25T04:24:38.936 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042253700000000001 2023-11-25T04:24:38.937 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.937 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.937 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:38.937 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034520+0000", 2023-11-25T04:24:38.937 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001037991 2023-11-25T04:24:38.937 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.937 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.937 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:38.937 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034611+0000", 2023-11-25T04:24:38.937 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1113999999999995e-05 2023-11-25T04:24:38.937 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.937 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.938 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:38.938 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034769+0000", 2023-11-25T04:24:38.938 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000157641 2023-11-25T04:24:38.938 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.938 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.938 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:38.938 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038292+0000", 2023-11-25T04:24:38.938 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035231160000000002 2023-11-25T04:24:38.938 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.938 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.938 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:38.938 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044440+0000", 2023-11-25T04:24:38.939 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0061486980000000002 2023-11-25T04:24:38.939 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.939 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.939 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:38.939 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044495+0000", 2023-11-25T04:24:38.939 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4339000000000003e-05 2023-11-25T04:24:38.939 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.939 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.939 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:38.939 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044531+0000", 2023-11-25T04:24:38.939 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6637999999999997e-05 2023-11-25T04:24:38.939 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:38.940 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:38.940 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:38.940 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.940 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.940 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:26 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:38.940 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:38.940 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 111.888166869, 2023-11-25T04:24:38.940 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011086945000000001, 2023-11-25T04:24:38.940 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:38.940 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:38.940 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:38.940 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.941 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:38.941 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:38.941 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:38.941 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.941 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.941 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:38.941 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:38.941 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:38.941 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.941 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.941 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:38.941 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:38.941 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1699999999999998e-07 2023-11-25T04:24:38.942 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.942 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.942 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:38.942 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033627+0000", 2023-11-25T04:24:38.942 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.163e-05 2023-11-25T04:24:38.942 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.942 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.942 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:38.942 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033636+0000", 2023-11-25T04:24:38.943 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6859999999999995e-06 2023-11-25T04:24:38.943 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.943 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.943 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:38.945 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033642+0000", 2023-11-25T04:24:38.945 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3149999999999997e-06 2023-11-25T04:24:38.945 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.945 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.945 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:38.945 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035414+0000", 2023-11-25T04:24:38.945 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0017716380000000001 2023-11-25T04:24:38.945 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.945 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.946 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:38.946 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035539+0000", 2023-11-25T04:24:38.946 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000125477 2023-11-25T04:24:38.946 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.946 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.946 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:38.946 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035672+0000", 2023-11-25T04:24:38.946 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000133316 2023-11-25T04:24:38.946 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.946 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.946 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:38.947 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038649+0000", 2023-11-25T04:24:38.947 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029764679999999999 2023-11-25T04:24:38.947 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.947 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.947 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:38.947 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044578+0000", 2023-11-25T04:24:38.947 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0059293510000000002 2023-11-25T04:24:38.947 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.947 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.947 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:38.947 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044651+0000", 2023-11-25T04:24:38.947 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2513000000000002e-05 2023-11-25T04:24:38.948 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.948 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.948 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:38.948 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044702+0000", 2023-11-25T04:24:38.948 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1034e-05 2023-11-25T04:24:38.948 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:38.948 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:38.948 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:38.948 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.948 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.948 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:29 3.b 3:d0630e4c:::mds0_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:38.948 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:38.949 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 111.88812894900001, 2023-11-25T04:24:38.949 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010980227, 2023-11-25T04:24:38.949 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:38.949 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:38.949 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:38.949 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.949 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:38.949 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:38.949 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:38.949 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.949 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.949 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:38.950 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:38.950 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:38.950 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.950 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.950 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:38.950 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:38.950 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3499999999999996e-07 2023-11-25T04:24:38.950 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.950 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.950 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:38.950 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033658+0000", 2023-11-25T04:24:38.950 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1280000000000001e-06 2023-11-25T04:24:38.951 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.951 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.951 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:38.951 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033667+0000", 2023-11-25T04:24:38.951 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1430000000000004e-06 2023-11-25T04:24:38.951 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.951 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.951 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:38.951 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033678+0000", 2023-11-25T04:24:38.951 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0906e-05 2023-11-25T04:24:38.951 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.951 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.952 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:38.952 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034456+0000", 2023-11-25T04:24:38.952 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00077833899999999996 2023-11-25T04:24:38.952 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.952 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.952 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:38.952 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034575+0000", 2023-11-25T04:24:38.952 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011926000000000001 2023-11-25T04:24:38.952 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.952 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.952 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:38.952 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035001+0000", 2023-11-25T04:24:38.952 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042550999999999998 2023-11-25T04:24:38.953 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.953 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.953 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:38.953 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.037923+0000", 2023-11-25T04:24:38.953 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029219670000000001 2023-11-25T04:24:38.953 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.953 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.953 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:38.953 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044538+0000", 2023-11-25T04:24:38.953 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0066152349999999997 2023-11-25T04:24:38.953 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.953 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.953 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:38.954 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044594+0000", 2023-11-25T04:24:38.954 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5593000000000001e-05 2023-11-25T04:24:38.954 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.954 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.954 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:38.954 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044633+0000", 2023-11-25T04:24:38.954 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9511000000000002e-05 2023-11-25T04:24:38.954 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:38.954 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:38.954 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:38.954 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.954 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.955 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:34 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:38.955 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:24:38.955 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 111.887878836, 2023-11-25T04:24:38.955 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010920445000000001, 2023-11-25T04:24:38.955 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:38.955 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:38.955 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:38.955 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.955 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:38.955 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:24:38.955 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:38.955 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.956 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.956 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:38.956 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:24:38.956 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:38.956 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.956 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.956 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:38.956 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033904+0000", 2023-11-25T04:24:38.956 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6599999999999994e-07 2023-11-25T04:24:38.956 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.956 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.956 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:38.957 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033909+0000", 2023-11-25T04:24:38.957 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5639999999999996e-06 2023-11-25T04:24:38.957 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.957 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.957 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:38.957 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033923+0000", 2023-11-25T04:24:38.957 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3028e-05 2023-11-25T04:24:38.957 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.957 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.957 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:38.957 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034043+0000", 2023-11-25T04:24:38.957 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000120328 2023-11-25T04:24:38.957 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.958 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.958 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:38.958 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035576+0000", 2023-11-25T04:24:38.958 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015333720000000001 2023-11-25T04:24:38.958 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.958 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.958 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:38.958 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035699+0000", 2023-11-25T04:24:38.958 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000123001 2023-11-25T04:24:38.958 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.958 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.958 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:38.959 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044823+0000", 2023-11-25T04:24:38.959 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0091241859999999994 2023-11-25T04:24:38.959 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:38.959 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:38.959 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:38.959 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.959 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.959 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:118 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals,omap-rm-keys] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e44)", 2023-11-25T04:24:38.959 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:17.112648+0000", 2023-11-25T04:24:38.959 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 21.809133495000001, 2023-11-25T04:24:38.959 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011250779000000001, 2023-11-25T04:24:38.959 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:38.960 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:38.960 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:38.960 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.960 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:38.960 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.112648+0000", 2023-11-25T04:24:38.960 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:38.960 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.960 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.960 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:38.960 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.112648+0000", 2023-11-25T04:24:38.960 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:38.960 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.960 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.961 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:38.961 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.112649+0000", 2023-11-25T04:24:38.961 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0070000000000001e-06 2023-11-25T04:24:38.961 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.961 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.961 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:38.961 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113182+0000", 2023-11-25T04:24:38.961 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00053268100000000002 2023-11-25T04:24:38.961 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.961 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.961 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:38.961 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113196+0000", 2023-11-25T04:24:38.962 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3742e-05 2023-11-25T04:24:38.962 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.962 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.962 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:38.962 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113398+0000", 2023-11-25T04:24:38.962 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000202726 2023-11-25T04:24:38.962 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.962 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.962 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:38.962 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113447+0000", 2023-11-25T04:24:38.962 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8655000000000001e-05 2023-11-25T04:24:38.963 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.963 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.963 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:38.963 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113529+0000", 2023-11-25T04:24:38.963 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1891999999999995e-05 2023-11-25T04:24:38.963 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.963 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.963 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:38.963 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123819+0000", 2023-11-25T04:24:38.963 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010290192 2023-11-25T04:24:38.963 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.963 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.964 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:38.964 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123852+0000", 2023-11-25T04:24:38.964 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2425999999999998e-05 2023-11-25T04:24:38.964 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.964 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.964 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:38.964 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123899+0000", 2023-11-25T04:24:38.964 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7457999999999999e-05 2023-11-25T04:24:38.964 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:38.964 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:38.964 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:38.964 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.964 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.965 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:154 3.1e 3:7bf7e7b5:::200.00000006:head [write 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:38.965 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:21.886430+0000", 2023-11-25T04:24:38.965 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 17.035352229000001, 2023-11-25T04:24:38.965 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.017800107999999999, 2023-11-25T04:24:38.965 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:38.965 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:38.965 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:38.965 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.965 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:38.965 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.886430+0000", 2023-11-25T04:24:38.965 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:38.965 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.966 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.966 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:38.966 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.886430+0000", 2023-11-25T04:24:38.966 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:38.966 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.966 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.966 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:38.966 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.886431+0000", 2023-11-25T04:24:38.966 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.178e-06 2023-11-25T04:24:38.966 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.966 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.966 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:38.967 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.892277+0000", 2023-11-25T04:24:38.967 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0058467409999999999 2023-11-25T04:24:38.967 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.967 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.967 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:38.967 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.892288+0000", 2023-11-25T04:24:38.967 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0183000000000001e-05 2023-11-25T04:24:38.967 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.967 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.967 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:38.967 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.892292+0000", 2023-11-25T04:24:38.967 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7829999999999998e-06 2023-11-25T04:24:38.967 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.968 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.968 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:38.968 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.892362+0000", 2023-11-25T04:24:38.968 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0034000000000001e-05 2023-11-25T04:24:38.968 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.968 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.968 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:38.968 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.892432+0000", 2023-11-25T04:24:38.968 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9648999999999997e-05 2023-11-25T04:24:38.968 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.968 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.968 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:38.968 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.892550+0000", 2023-11-25T04:24:38.969 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011825 2023-11-25T04:24:38.969 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.969 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.969 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:38.969 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.899106+0000", 2023-11-25T04:24:38.969 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0065556759999999999 2023-11-25T04:24:38.969 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.969 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.969 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:38.969 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.904145+0000", 2023-11-25T04:24:38.969 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0050393490000000003 2023-11-25T04:24:38.969 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.969 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.970 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:38.970 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.904200+0000", 2023-11-25T04:24:38.970 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4234999999999998e-05 2023-11-25T04:24:38.970 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.970 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.970 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:38.970 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.904230+0000", 2023-11-25T04:24:38.970 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0029999999999999e-05 2023-11-25T04:24:38.970 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:38.970 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:38.970 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:38.970 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.971 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.971 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:576 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals,omap-rm-keys] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:38.971 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:22.111407+0000", 2023-11-25T04:24:38.971 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 16.810374623000001, 2023-11-25T04:24:38.971 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.018987317, 2023-11-25T04:24:38.971 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:38.971 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:38.971 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:38.971 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.971 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:38.971 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111407+0000", 2023-11-25T04:24:38.971 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:38.972 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.972 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.972 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:38.972 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111407+0000", 2023-11-25T04:24:38.972 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:38.972 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.972 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.972 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:38.972 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111408+0000", 2023-11-25T04:24:38.972 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8000000000000004e-07 2023-11-25T04:24:38.972 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.972 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.972 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:38.973 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111659+0000", 2023-11-25T04:24:38.973 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000250537 2023-11-25T04:24:38.973 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.973 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.973 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:38.973 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111666+0000", 2023-11-25T04:24:38.973 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.973e-06 2023-11-25T04:24:38.973 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.973 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.973 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:38.973 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111670+0000", 2023-11-25T04:24:38.973 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.668e-06 2023-11-25T04:24:38.973 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.974 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.974 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:38.974 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111709+0000", 2023-11-25T04:24:38.974 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8800000000000001e-05 2023-11-25T04:24:38.974 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.974 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.974 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:38.974 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111761+0000", 2023-11-25T04:24:38.974 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1495000000000001e-05 2023-11-25T04:24:38.974 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.974 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.974 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:38.975 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.130255+0000", 2023-11-25T04:24:38.975 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.018494345999999998 2023-11-25T04:24:38.975 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.975 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.975 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:38.975 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.130311+0000", 2023-11-25T04:24:38.975 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5902000000000001e-05 2023-11-25T04:24:38.975 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.975 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.975 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:38.975 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.130394+0000", 2023-11-25T04:24:38.975 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3715999999999996e-05 2023-11-25T04:24:38.976 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:38.976 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:38.976 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:38.976 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.976 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.976 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:1067 4.21 4:869a7600:::100000011fa.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:38.976 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:22.155038+0000", 2023-11-25T04:24:38.976 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 16.766743285, 2023-11-25T04:24:38.976 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011313148, 2023-11-25T04:24:38.976 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:38.976 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:38.976 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:38.977 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.977 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:38.977 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155038+0000", 2023-11-25T04:24:38.977 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:38.977 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.977 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.977 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:38.977 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155038+0000", 2023-11-25T04:24:38.977 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:38.977 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.977 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.977 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:38.977 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155039+0000", 2023-11-25T04:24:38.978 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0200000000000002e-07 2023-11-25T04:24:38.978 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.978 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.978 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:38.978 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155049+0000", 2023-11-25T04:24:38.978 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0295000000000001e-05 2023-11-25T04:24:38.978 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.978 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.978 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:38.978 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155058+0000", 2023-11-25T04:24:38.978 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.208e-06 2023-11-25T04:24:38.978 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.979 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.979 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:38.979 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155064+0000", 2023-11-25T04:24:38.979 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4489999999999996e-06 2023-11-25T04:24:38.979 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.979 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.979 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:38.979 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155104+0000", 2023-11-25T04:24:38.979 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0164999999999999e-05 2023-11-25T04:24:38.979 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.979 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.979 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:38.980 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155147+0000", 2023-11-25T04:24:38.980 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2576e-05 2023-11-25T04:24:38.980 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.980 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.980 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:38.980 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155212+0000", 2023-11-25T04:24:38.980 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5516000000000001e-05 2023-11-25T04:24:38.980 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.980 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.980 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:38.980 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.156025+0000", 2023-11-25T04:24:38.980 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00081248599999999996 2023-11-25T04:24:38.980 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.981 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.981 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:38.981 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.166250+0000", 2023-11-25T04:24:38.981 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010224736999999999 2023-11-25T04:24:38.981 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.981 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.981 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:38.981 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.166319+0000", 2023-11-25T04:24:38.981 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9145000000000003e-05 2023-11-25T04:24:38.981 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.981 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.981 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:38.982 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.166352+0000", 2023-11-25T04:24:38.982 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2969000000000002e-05 2023-11-25T04:24:38.982 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:38.982 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:38.982 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:38.982 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.982 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.982 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:1102 4.2f 4:f543b94c:::1000000121d.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:38.982 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:22.158246+0000", 2023-11-25T04:24:38.982 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 16.763535762, 2023-11-25T04:24:38.982 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010094396, 2023-11-25T04:24:38.982 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:38.983 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:38.983 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:38.983 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.983 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:38.983 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158246+0000", 2023-11-25T04:24:38.983 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:38.983 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.983 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.983 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:38.983 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158246+0000", 2023-11-25T04:24:38.983 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:38.983 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.983 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.984 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:38.984 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158246+0000", 2023-11-25T04:24:38.984 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5299999999999999e-07 2023-11-25T04:24:38.984 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.984 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.984 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:38.984 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158249+0000", 2023-11-25T04:24:38.984 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2840000000000001e-06 2023-11-25T04:24:38.984 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.985 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.985 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:38.985 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158254+0000", 2023-11-25T04:24:38.985 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3449999999999996e-06 2023-11-25T04:24:38.985 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.985 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.985 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:38.985 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158258+0000", 2023-11-25T04:24:38.985 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5030000000000002e-06 2023-11-25T04:24:38.985 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.985 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.985 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:38.986 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158302+0000", 2023-11-25T04:24:38.986 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3908e-05 2023-11-25T04:24:38.986 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.986 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.986 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:38.986 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158398+0000", 2023-11-25T04:24:38.986 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6898000000000006e-05 2023-11-25T04:24:38.986 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.986 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.986 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:38.986 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158480+0000", 2023-11-25T04:24:38.986 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1578999999999999e-05 2023-11-25T04:24:38.987 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.987 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.987 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:38.987 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.159229+0000", 2023-11-25T04:24:38.987 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000749049 2023-11-25T04:24:38.987 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.987 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.987 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:38.987 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.168228+0000", 2023-11-25T04:24:38.987 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0089992930000000002 2023-11-25T04:24:38.987 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.987 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.987 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:38.988 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.168310+0000", 2023-11-25T04:24:38.988 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1348999999999997e-05 2023-11-25T04:24:38.988 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.988 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.988 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:38.988 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.168340+0000", 2023-11-25T04:24:38.988 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0735000000000003e-05 2023-11-25T04:24:38.988 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:38.988 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:38.988 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:38.988 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.989 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.989 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:1105 4.23 4:c6ff07b4:::10000001220.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:38.989 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:22.158714+0000", 2023-11-25T04:24:38.989 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 16.763067648, 2023-11-25T04:24:38.989 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010412490999999999, 2023-11-25T04:24:38.989 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:38.989 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:38.989 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:38.989 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.989 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:38.989 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158714+0000", 2023-11-25T04:24:38.989 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:38.989 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.990 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.990 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:38.990 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158714+0000", 2023-11-25T04:24:38.990 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:38.990 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.990 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.990 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:38.990 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158715+0000", 2023-11-25T04:24:38.990 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4099999999999998e-07 2023-11-25T04:24:38.990 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.990 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.990 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:38.991 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158718+0000", 2023-11-25T04:24:38.991 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4319999999999999e-06 2023-11-25T04:24:38.991 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.991 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.991 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:38.991 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158729+0000", 2023-11-25T04:24:38.991 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.078e-05 2023-11-25T04:24:38.991 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.991 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.991 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:38.991 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158736+0000", 2023-11-25T04:24:38.991 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4649999999999997e-06 2023-11-25T04:24:38.992 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.992 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.992 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:38.992 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158779+0000", 2023-11-25T04:24:38.992 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3049000000000001e-05 2023-11-25T04:24:38.992 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.992 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.992 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:38.992 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158865+0000", 2023-11-25T04:24:38.992 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5138999999999999e-05 2023-11-25T04:24:38.992 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.992 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.992 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:38.993 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158983+0000", 2023-11-25T04:24:38.993 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000118658 2023-11-25T04:24:38.993 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.993 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.993 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:38.993 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.159645+0000", 2023-11-25T04:24:38.993 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00066193999999999997 2023-11-25T04:24:38.993 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.993 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.993 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:38.993 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.169046+0000", 2023-11-25T04:24:38.993 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0094005400000000006 2023-11-25T04:24:38.994 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.994 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.994 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:38.994 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.169092+0000", 2023-11-25T04:24:38.994 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5840000000000002e-05 2023-11-25T04:24:38.994 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.994 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.994 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:38.994 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.169127+0000", 2023-11-25T04:24:38.994 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5006999999999999e-05 2023-11-25T04:24:38.994 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:38.994 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:38.994 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:38.995 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.995 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.995 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2841 4.4 4:22111d42:::100000010df.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:38.995 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.092102+0000", 2023-11-25T04:24:38.995 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 13.829680173, 2023-11-25T04:24:38.995 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.016449783999999999, 2023-11-25T04:24:38.995 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:38.995 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:38.995 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:38.995 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.995 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:38.995 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092102+0000", 2023-11-25T04:24:38.996 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:38.996 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.996 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.996 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:38.996 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092102+0000", 2023-11-25T04:24:38.996 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:38.996 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.996 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.996 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:38.996 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092102+0000", 2023-11-25T04:24:38.996 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3700000000000001e-07 2023-11-25T04:24:38.996 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.997 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.997 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:38.997 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092104+0000", 2023-11-25T04:24:38.997 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0679999999999999e-06 2023-11-25T04:24:38.997 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.997 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.997 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:38.997 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092111+0000", 2023-11-25T04:24:38.997 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0589999999999997e-06 2023-11-25T04:24:38.997 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.997 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.997 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:38.997 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092130+0000", 2023-11-25T04:24:38.998 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8686999999999999e-05 2023-11-25T04:24:38.998 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.998 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.998 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:38.998 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092443+0000", 2023-11-25T04:24:38.998 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000313519 2023-11-25T04:24:38.998 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.998 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.998 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:38.998 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092553+0000", 2023-11-25T04:24:38.998 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010932100000000001 2023-11-25T04:24:38.998 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.999 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.999 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:38.999 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092639+0000", 2023-11-25T04:24:38.999 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6094999999999995e-05 2023-11-25T04:24:38.999 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.999 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.999 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:38.999 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094679+0000", 2023-11-25T04:24:38.999 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0020402580000000001 2023-11-25T04:24:38.999 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:38.999 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:38.999 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:39.000 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108462+0000", 2023-11-25T04:24:39.000 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013783358000000001 2023-11-25T04:24:39.000 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.000 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.000 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:39.000 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108516+0000", 2023-11-25T04:24:39.000 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3483000000000003e-05 2023-11-25T04:24:39.000 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.000 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.000 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:39.000 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108551+0000", 2023-11-25T04:24:39.000 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5499e-05 2023-11-25T04:24:39.001 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:39.001 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:39.001 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:39.001 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.001 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.001 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2852 4.2d 4:b745ac76:::100000010f5.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:39.001 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.093477+0000", 2023-11-25T04:24:39.001 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 13.828305061, 2023-11-25T04:24:39.001 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.015423084, 2023-11-25T04:24:39.001 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:39.001 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:39.001 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:39.001 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.002 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:39.002 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093477+0000", 2023-11-25T04:24:39.002 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:39.002 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.002 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.002 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:39.002 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093477+0000", 2023-11-25T04:24:39.002 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:39.002 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.002 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.002 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:39.002 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093477+0000", 2023-11-25T04:24:39.003 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2500000000000005e-07 2023-11-25T04:24:39.003 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.003 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.003 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:39.003 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093480+0000", 2023-11-25T04:24:39.003 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0189999999999998e-06 2023-11-25T04:24:39.003 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.003 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.003 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:39.003 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093642+0000", 2023-11-25T04:24:39.003 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000162072 2023-11-25T04:24:39.003 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.004 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.004 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:39.004 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093649+0000", 2023-11-25T04:24:39.004 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9659999999999999e-06 2023-11-25T04:24:39.004 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.004 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.004 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:39.004 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093693+0000", 2023-11-25T04:24:39.004 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3794000000000001e-05 2023-11-25T04:24:39.004 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.004 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.004 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:39.004 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093764+0000", 2023-11-25T04:24:39.005 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0981000000000004e-05 2023-11-25T04:24:39.005 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.005 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.005 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:39.005 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093850+0000", 2023-11-25T04:24:39.005 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5536000000000002e-05 2023-11-25T04:24:39.005 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.005 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.005 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:39.005 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094710+0000", 2023-11-25T04:24:39.005 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00086070299999999999 2023-11-25T04:24:39.005 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.006 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.006 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:39.006 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108776+0000", 2023-11-25T04:24:39.006 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01406542 2023-11-25T04:24:39.006 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.006 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.006 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:39.006 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108829+0000", 2023-11-25T04:24:39.006 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2936999999999998e-05 2023-11-25T04:24:39.006 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.006 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.006 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:39.006 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108900+0000", 2023-11-25T04:24:39.007 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1031000000000005e-05 2023-11-25T04:24:39.007 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:39.007 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:39.007 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:39.007 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.007 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.007 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2854 4.2d 4:b5c259cf:::100000010f9.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:39.007 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.093674+0000", 2023-11-25T04:24:39.007 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 13.828107401, 2023-11-25T04:24:39.007 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.015970380999999999, 2023-11-25T04:24:39.009 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:39.009 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:39.009 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:39.009 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.009 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:39.009 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093674+0000", 2023-11-25T04:24:39.009 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:39.009 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.009 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.009 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:39.009 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093674+0000", 2023-11-25T04:24:39.010 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:39.010 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.010 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.010 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:39.010 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093675+0000", 2023-11-25T04:24:39.010 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8400000000000004e-07 2023-11-25T04:24:39.010 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.010 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.010 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:39.010 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093678+0000", 2023-11-25T04:24:39.010 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2229999999999999e-06 2023-11-25T04:24:39.010 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.010 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.011 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:39.011 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093686+0000", 2023-11-25T04:24:39.011 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9990000000000008e-06 2023-11-25T04:24:39.011 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.011 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.011 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:39.011 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093696+0000", 2023-11-25T04:24:39.011 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6479999999999998e-06 2023-11-25T04:24:39.011 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.011 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.011 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:39.011 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094104+0000", 2023-11-25T04:24:39.012 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00040823900000000002 2023-11-25T04:24:39.012 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.012 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.012 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:39.012 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094172+0000", 2023-11-25T04:24:39.012 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8023999999999998e-05 2023-11-25T04:24:39.012 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.012 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.012 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:39.012 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094256+0000", 2023-11-25T04:24:39.012 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3418999999999995e-05 2023-11-25T04:24:39.012 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.013 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.013 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:39.013 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095160+0000", 2023-11-25T04:24:39.013 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00090418599999999997 2023-11-25T04:24:39.013 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.013 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.013 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:39.013 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109551+0000", 2023-11-25T04:24:39.013 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014391571000000001 2023-11-25T04:24:39.013 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.013 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.013 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:39.013 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109601+0000", 2023-11-25T04:24:39.014 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9478000000000003e-05 2023-11-25T04:24:39.014 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.014 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.014 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:39.014 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109645+0000", 2023-11-25T04:24:39.014 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.401e-05 2023-11-25T04:24:39.014 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:39.014 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:39.014 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:39.014 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.014 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.015 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2861 4.4 4:214027a9:::10000001107.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:39.015 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.093829+0000", 2023-11-25T04:24:39.015 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 13.827952982999999, 2023-11-25T04:24:39.015 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014848917999999999, 2023-11-25T04:24:39.015 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:39.015 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:39.015 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:39.015 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.015 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:39.015 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093829+0000", 2023-11-25T04:24:39.015 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:39.015 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.015 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.016 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:39.016 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093829+0000", 2023-11-25T04:24:39.016 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:39.016 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.016 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.016 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:39.016 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093830+0000", 2023-11-25T04:24:39.016 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1999999999999998e-07 2023-11-25T04:24:39.016 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.016 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.016 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:39.016 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093834+0000", 2023-11-25T04:24:39.017 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.34e-06 2023-11-25T04:24:39.017 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.017 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.017 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:39.017 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093844+0000", 2023-11-25T04:24:39.017 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6690000000000001e-06 2023-11-25T04:24:39.017 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.017 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.017 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:39.017 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093857+0000", 2023-11-25T04:24:39.017 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3434999999999999e-05 2023-11-25T04:24:39.017 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.018 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.018 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:39.018 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093940+0000", 2023-11-25T04:24:39.018 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3047999999999995e-05 2023-11-25T04:24:39.018 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.018 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.018 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:39.018 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094052+0000", 2023-11-25T04:24:39.018 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000111538 2023-11-25T04:24:39.018 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.018 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.019 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:39.019 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094466+0000", 2023-11-25T04:24:39.019 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00041464600000000001 2023-11-25T04:24:39.019 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.019 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.019 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:39.019 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095570+0000", 2023-11-25T04:24:39.019 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001103981 2023-11-25T04:24:39.019 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.019 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.019 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:39.019 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108598+0000", 2023-11-25T04:24:39.019 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013027164000000001 2023-11-25T04:24:39.020 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.020 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.020 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:39.020 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108646+0000", 2023-11-25T04:24:39.020 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8090999999999997e-05 2023-11-25T04:24:39.020 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.020 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.020 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:39.020 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108678+0000", 2023-11-25T04:24:39.020 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2085999999999997e-05 2023-11-25T04:24:39.020 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:39.020 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:39.020 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:39.021 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.021 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.021 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2863 4.12 4:4b84c733:::1000000110b.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:39.021 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.093877+0000", 2023-11-25T04:24:39.021 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 13.827904655999999, 2023-11-25T04:24:39.021 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014682755, 2023-11-25T04:24:39.021 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:39.021 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:39.021 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:39.021 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.021 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:39.021 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093877+0000", 2023-11-25T04:24:39.022 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:39.022 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.022 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.022 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:39.022 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093877+0000", 2023-11-25T04:24:39.022 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:39.022 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.022 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.022 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:39.022 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093878+0000", 2023-11-25T04:24:39.022 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4199999999999996e-07 2023-11-25T04:24:39.022 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.022 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.022 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:39.023 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093880+0000", 2023-11-25T04:24:39.023 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6189999999999998e-06 2023-11-25T04:24:39.023 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.023 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.023 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:39.023 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093888+0000", 2023-11-25T04:24:39.023 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5510000000000002e-06 2023-11-25T04:24:39.023 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.023 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.023 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:39.023 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093893+0000", 2023-11-25T04:24:39.023 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4179999999999996e-06 2023-11-25T04:24:39.024 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.024 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.024 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:39.024 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093934+0000", 2023-11-25T04:24:39.024 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0648000000000001e-05 2023-11-25T04:24:39.024 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.024 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.024 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:39.024 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094081+0000", 2023-11-25T04:24:39.024 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014753499999999999 2023-11-25T04:24:39.024 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.024 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.025 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:39.025 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094172+0000", 2023-11-25T04:24:39.025 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.0722000000000003e-05 2023-11-25T04:24:39.025 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.025 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.025 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:39.025 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094742+0000", 2023-11-25T04:24:39.025 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00056948600000000002 2023-11-25T04:24:39.025 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.025 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.025 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:39.025 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108491+0000", 2023-11-25T04:24:39.025 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013749568 2023-11-25T04:24:39.026 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.026 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.026 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:39.026 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108525+0000", 2023-11-25T04:24:39.026 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3683999999999998e-05 2023-11-25T04:24:39.026 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.026 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.026 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:39.026 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108560+0000", 2023-11-25T04:24:39.026 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4981999999999998e-05 2023-11-25T04:24:39.026 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:39.026 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:39.027 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:39.027 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.027 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.027 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2880 4.21 4:855834a4:::1000000112d.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:39.027 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.095098+0000", 2023-11-25T04:24:39.027 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 13.826683962000001, 2023-11-25T04:24:39.027 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013847544, 2023-11-25T04:24:39.027 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:39.027 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:39.027 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:39.027 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.027 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:39.027 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095098+0000", 2023-11-25T04:24:39.028 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:39.028 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.028 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.028 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:39.028 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095098+0000", 2023-11-25T04:24:39.028 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:39.028 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.028 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.028 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:39.028 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095098+0000", 2023-11-25T04:24:39.028 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3000000000000002e-07 2023-11-25T04:24:39.028 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.029 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.029 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:39.029 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095099+0000", 2023-11-25T04:24:39.029 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.331e-06 2023-11-25T04:24:39.029 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.029 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.029 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:39.029 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095103+0000", 2023-11-25T04:24:39.029 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8319999999999999e-06 2023-11-25T04:24:39.029 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.029 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.029 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:39.029 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095108+0000", 2023-11-25T04:24:39.030 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4179999999999999e-06 2023-11-25T04:24:39.030 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.030 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.030 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:39.030 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095153+0000", 2023-11-25T04:24:39.030 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4799000000000003e-05 2023-11-25T04:24:39.030 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.030 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.030 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:39.030 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095268+0000", 2023-11-25T04:24:39.030 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000115437 2023-11-25T04:24:39.030 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.031 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.031 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:39.031 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095363+0000", 2023-11-25T04:24:39.031 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.5278000000000005e-05 2023-11-25T04:24:39.031 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.031 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.031 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:39.031 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095845+0000", 2023-11-25T04:24:39.031 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00048164600000000002 2023-11-25T04:24:39.031 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.031 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.031 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:39.032 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108854+0000", 2023-11-25T04:24:39.032 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013009289 2023-11-25T04:24:39.032 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.032 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.032 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:39.032 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108907+0000", 2023-11-25T04:24:39.032 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2528999999999998e-05 2023-11-25T04:24:39.032 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.032 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.032 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:39.032 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108945+0000", 2023-11-25T04:24:39.032 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8655000000000002e-05 2023-11-25T04:24:39.032 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:39.033 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:39.033 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:39.033 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.033 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.033 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2884 4.37 4:ef909cdc:::10000001135.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:39.033 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.095518+0000", 2023-11-25T04:24:39.033 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 13.826264034999999, 2023-11-25T04:24:39.033 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014149988, 2023-11-25T04:24:39.033 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:39.033 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:39.033 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:39.033 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.033 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:39.034 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095518+0000", 2023-11-25T04:24:39.034 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:39.034 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.034 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.034 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:39.034 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095518+0000", 2023-11-25T04:24:39.034 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:39.034 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.034 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.034 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:39.034 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095518+0000", 2023-11-25T04:24:39.034 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2900000000000003e-07 2023-11-25T04:24:39.035 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.035 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.035 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:39.035 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095522+0000", 2023-11-25T04:24:39.035 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2119999999999999e-06 2023-11-25T04:24:39.035 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.035 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.035 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:39.035 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095532+0000", 2023-11-25T04:24:39.035 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0375e-05 2023-11-25T04:24:39.035 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.035 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.035 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:39.036 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095537+0000", 2023-11-25T04:24:39.036 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5909999999999998e-06 2023-11-25T04:24:39.036 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.036 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.036 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:39.036 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095584+0000", 2023-11-25T04:24:39.036 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7169999999999997e-05 2023-11-25T04:24:39.036 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.036 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.036 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:39.036 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095695+0000", 2023-11-25T04:24:39.036 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011078699999999999 2023-11-25T04:24:39.037 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.037 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.037 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:39.037 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095777+0000", 2023-11-25T04:24:39.037 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2089999999999995e-05 2023-11-25T04:24:39.037 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.037 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.037 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:39.037 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096286+0000", 2023-11-25T04:24:39.037 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000509247 2023-11-25T04:24:39.037 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.037 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.037 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:39.038 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109596+0000", 2023-11-25T04:24:39.038 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013309872 2023-11-25T04:24:39.038 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.038 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.038 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:39.038 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109638+0000", 2023-11-25T04:24:39.038 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2381e-05 2023-11-25T04:24:39.038 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.038 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.038 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:39.038 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109668+0000", 2023-11-25T04:24:39.039 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9634000000000001e-05 2023-11-25T04:24:39.039 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:39.039 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:39.039 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:39.039 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.039 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.039 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2890 4.12 4:4a2ad640:::10000001141.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:39.039 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.096274+0000", 2023-11-25T04:24:39.039 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 13.825507972, 2023-11-25T04:24:39.039 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013647259, 2023-11-25T04:24:39.039 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:39.039 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:39.040 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:39.040 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.040 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:39.040 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096274+0000", 2023-11-25T04:24:39.040 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:39.040 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.040 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.040 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:39.040 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096274+0000", 2023-11-25T04:24:39.040 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:39.040 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.040 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.041 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:39.041 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096275+0000", 2023-11-25T04:24:39.041 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0299999999999998e-07 2023-11-25T04:24:39.041 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.041 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.041 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:39.041 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096279+0000", 2023-11-25T04:24:39.041 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9580000000000001e-06 2023-11-25T04:24:39.041 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.041 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.041 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:39.041 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096287+0000", 2023-11-25T04:24:39.042 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6929999999999996e-06 2023-11-25T04:24:39.042 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.042 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.042 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:39.042 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096294+0000", 2023-11-25T04:24:39.042 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.348e-06 2023-11-25T04:24:39.042 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.042 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.042 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:39.042 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096342+0000", 2023-11-25T04:24:39.042 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8068000000000001e-05 2023-11-25T04:24:39.042 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.042 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.043 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:39.043 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096462+0000", 2023-11-25T04:24:39.043 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000119918 2023-11-25T04:24:39.043 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.043 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.043 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:39.043 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096557+0000", 2023-11-25T04:24:39.043 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.5340999999999994e-05 2023-11-25T04:24:39.043 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.043 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.043 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:39.043 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096971+0000", 2023-11-25T04:24:39.044 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00041398100000000001 2023-11-25T04:24:39.044 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.044 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.044 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:39.044 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109849+0000", 2023-11-25T04:24:39.044 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.012877985999999999 2023-11-25T04:24:39.044 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.044 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.044 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:39.044 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109894+0000", 2023-11-25T04:24:39.044 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5595e-05 2023-11-25T04:24:39.044 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.044 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.045 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:39.045 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109921+0000", 2023-11-25T04:24:39.045 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6568e-05 2023-11-25T04:24:39.045 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:39.045 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:39.045 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:39.045 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.045 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.045 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2891 4.23 4:c4054dd3:::10000001143.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:39.045 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.096713+0000", 2023-11-25T04:24:39.045 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 13.825069278000001, 2023-11-25T04:24:39.045 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013148692, 2023-11-25T04:24:39.046 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:39.046 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:39.046 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:39.046 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.046 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:39.046 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096713+0000", 2023-11-25T04:24:39.046 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:39.046 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.046 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.046 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:39.046 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096713+0000", 2023-11-25T04:24:39.046 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:39.047 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.047 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.047 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:39.047 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096713+0000", 2023-11-25T04:24:39.047 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9299999999999998e-07 2023-11-25T04:24:39.047 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.047 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.047 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:39.047 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096716+0000", 2023-11-25T04:24:39.047 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.988e-06 2023-11-25T04:24:39.047 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.047 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.047 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:39.048 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096722+0000", 2023-11-25T04:24:39.048 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0499999999999997e-06 2023-11-25T04:24:39.048 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.048 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.048 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:39.048 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096727+0000", 2023-11-25T04:24:39.048 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3380000000000004e-06 2023-11-25T04:24:39.048 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.048 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.048 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:39.048 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096770+0000", 2023-11-25T04:24:39.048 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2938000000000001e-05 2023-11-25T04:24:39.048 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.049 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.049 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:39.049 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096904+0000", 2023-11-25T04:24:39.049 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000133759 2023-11-25T04:24:39.049 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.049 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.049 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:39.049 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096983+0000", 2023-11-25T04:24:39.049 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8559999999999993e-05 2023-11-25T04:24:39.049 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.049 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.049 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:39.049 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.097437+0000", 2023-11-25T04:24:39.050 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00045439200000000002 2023-11-25T04:24:39.050 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.050 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.050 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:39.050 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109737+0000", 2023-11-25T04:24:39.050 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.012300190000000001 2023-11-25T04:24:39.050 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.050 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.050 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:39.050 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109782+0000", 2023-11-25T04:24:39.050 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4546999999999999e-05 2023-11-25T04:24:39.050 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.051 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.051 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:39.051 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109861+0000", 2023-11-25T04:24:39.051 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9337000000000003e-05 2023-11-25T04:24:39.051 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:39.051 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:39.051 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:39.051 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:39.051 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:39.051 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:39.052 INFO:tasks.workunit.client.0.smithi067.stdout: [CC] alloc 2023-11-25T04:24:39.053 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:24:39.053 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:39.051+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:39.074 INFO:tasks.workunit.client.0.smithi067.stdout: [CC] fault 2023-11-25T04:24:39.101 INFO:tasks.workunit.client.0.smithi067.stdout: [CC] fstest 2023-11-25T04:24:39.145 INFO:tasks.workunit.client.0.smithi067.stdout: [CC] t_access_root 2023-11-25T04:24:39.147 INFO:tasks.workunit.client.0.smithi067.stderr:alloc.c: In function 'main': 2023-11-25T04:24:39.147 INFO:tasks.workunit.client.0.smithi067.stderr:alloc.c:154:1: error: 'bozo' undeclared (first use in this function) 2023-11-25T04:24:39.147 INFO:tasks.workunit.client.0.smithi067.stderr: 154 | bozo! 2023-11-25T04:24:39.147 INFO:tasks.workunit.client.0.smithi067.stderr: | ^~~~ 2023-11-25T04:24:39.147 INFO:tasks.workunit.client.0.smithi067.stderr:alloc.c:154:1: note: each undeclared identifier is reported only once for each function it appears in 2023-11-25T04:24:39.147 INFO:tasks.workunit.client.0.smithi067.stderr:alloc.c:154:5: error: expected ';' before '!' token 2023-11-25T04:24:39.147 INFO:tasks.workunit.client.0.smithi067.stderr: 154 | bozo! 2023-11-25T04:24:39.147 INFO:tasks.workunit.client.0.smithi067.stderr: | ^ 2023-11-25T04:24:39.147 INFO:tasks.workunit.client.0.smithi067.stderr: | ; 2023-11-25T04:24:39.148 INFO:tasks.workunit.client.0.smithi067.stderr:alloc.c:184:25: error: 'rflag' undeclared (first use in this function); did you mean 'nflag'? 2023-11-25T04:24:39.148 INFO:tasks.workunit.client.0.smithi067.stderr: 184 | rflag = 1; 2023-11-25T04:24:39.148 INFO:tasks.workunit.client.0.smithi067.stderr: | ^~~~~ 2023-11-25T04:24:39.148 INFO:tasks.workunit.client.0.smithi067.stderr: | nflag 2023-11-25T04:24:39.148 INFO:tasks.workunit.client.0.smithi067.stderr:alloc.c:320:5: error: expected ';' before '!' token 2023-11-25T04:24:39.148 INFO:tasks.workunit.client.0.smithi067.stderr: 320 | bozo! 2023-11-25T04:24:39.148 INFO:tasks.workunit.client.0.smithi067.stderr: | ^ 2023-11-25T04:24:39.148 INFO:tasks.workunit.client.0.smithi067.stderr: | ; 2023-11-25T04:24:39.149 INFO:tasks.workunit.client.0.smithi067.stderr:alloc.c:128:25: warning: variable 'f' set but not used [-Wunused-but-set-variable] 2023-11-25T04:24:39.149 INFO:tasks.workunit.client.0.smithi067.stderr: 128 | struct flock64 f; 2023-11-25T04:24:39.149 INFO:tasks.workunit.client.0.smithi067.stderr: | ^ 2023-11-25T04:24:39.151 INFO:tasks.workunit.client.0.smithi067.stderr:gmake[3]: *** [Makefile:83: alloc] Error 1 2023-11-25T04:24:39.151 INFO:tasks.workunit.client.0.smithi067.stderr:gmake[3]: *** Waiting for unfinished jobs.... 2023-11-25T04:24:39.153 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:39.152+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:39.161 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:39.161 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:24:39.161 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2023-11-25T04:24:39.161 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:39.169 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:24:39.254 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:39.252+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:39.274 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:39.274 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:24:39.274 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2023-11-25T04:24:39.274 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2023-11-25T04:24:39.274 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:39.282 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:24:39.333 INFO:tasks.workunit.client.0.smithi067.stderr:fstest.c: In function 'run_child': 2023-11-25T04:24:39.333 INFO:tasks.workunit.client.0.smithi067.stderr:fstest.c:95:27: warning: '/file' directive writing 5 bytes into a region of size between 1 and 1024 [-Wformat-overflow=] 2023-11-25T04:24:39.334 INFO:tasks.workunit.client.0.smithi067.stderr: 95 | sprintf(fname, "%s/file%d", dir, fnum); 2023-11-25T04:24:39.334 INFO:tasks.workunit.client.0.smithi067.stderr: | ^~~~~ 2023-11-25T04:24:39.334 INFO:tasks.workunit.client.0.smithi067.stderr:fstest.c:95:24: note: directive argument in the range [0, 2147483647] 2023-11-25T04:24:39.334 INFO:tasks.workunit.client.0.smithi067.stderr: 95 | sprintf(fname, "%s/file%d", dir, fnum); 2023-11-25T04:24:39.334 INFO:tasks.workunit.client.0.smithi067.stderr: | ^~~~~~~~~~~ 2023-11-25T04:24:39.334 INFO:tasks.workunit.client.0.smithi067.stderr:fstest.c:95:9: note: 'sprintf' output between 7 and 1039 bytes into a destination of size 1024 2023-11-25T04:24:39.334 INFO:tasks.workunit.client.0.smithi067.stderr: 95 | sprintf(fname, "%s/file%d", dir, fnum); 2023-11-25T04:24:39.334 INFO:tasks.workunit.client.0.smithi067.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-11-25T04:24:39.334 INFO:tasks.workunit.client.0.smithi067.stderr:fstest.c:166:27: warning: '/file' directive writing 5 bytes into a region of size between 1 and 1024 [-Wformat-overflow=] 2023-11-25T04:24:39.334 INFO:tasks.workunit.client.0.smithi067.stderr: 166 | sprintf(fname, "%s/file%d", dir, fnum); 2023-11-25T04:24:39.335 INFO:tasks.workunit.client.0.smithi067.stderr: | ^~~~~ 2023-11-25T04:24:39.335 INFO:tasks.workunit.client.0.smithi067.stderr:fstest.c:166:24: note: directive argument in the range [0, 2147483647] 2023-11-25T04:24:39.335 INFO:tasks.workunit.client.0.smithi067.stderr: 166 | sprintf(fname, "%s/file%d", dir, fnum); 2023-11-25T04:24:39.335 INFO:tasks.workunit.client.0.smithi067.stderr: | ^~~~~~~~~~~ 2023-11-25T04:24:39.335 INFO:tasks.workunit.client.0.smithi067.stderr:fstest.c:166:9: note: 'sprintf' output between 7 and 1039 bytes into a destination of size 1024 2023-11-25T04:24:39.335 INFO:tasks.workunit.client.0.smithi067.stderr: 166 | sprintf(fname, "%s/file%d", dir, fnum); 2023-11-25T04:24:39.336 INFO:tasks.workunit.client.0.smithi067.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-11-25T04:24:39.368 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:39.365+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:39.418 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-11-25T04:24:39.418 INFO:tasks.workunit.client.0.smithi067.stderr:gmake[2]: *** [include/buildrules:30: src] Error 2 2023-11-25T04:24:39.418 INFO:tasks.workunit.client.0.smithi067.stderr:make[1]: *** [Makefile:53: default] Error 2 2023-11-25T04:24:39.418 INFO:tasks.workunit.client.0.smithi067.stderr:make: *** [Makefile:51: default] Error 2 2023-11-25T04:24:39.419 INFO:tasks.workunit:Stopping ['suites/fsx.sh'] on client.0... 2023-11-25T04:24:39.419 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2023-11-25T04:24:39.431 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:39.432 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2023-11-25T04:24:39.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2023-11-25T04:24:39.432 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2023-11-25T04:24:39.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.433 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:152 3.1a 3:5e41ab69:::200.00000005:head [write 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:39.433 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:21.237401+0000", 2023-11-25T04:24:39.433 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.191345982000001, 2023-11-25T04:24:39.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015253802, 2023-11-25T04:24:39.434 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:39.434 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:39.434 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:39.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.434 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:39.434 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.237401+0000", 2023-11-25T04:24:39.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:39.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:39.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.237401+0000", 2023-11-25T04:24:39.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:39.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:39.435 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.237402+0000", 2023-11-25T04:24:39.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3699999999999999e-07 2023-11-25T04:24:39.435 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:39.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243182+0000", 2023-11-25T04:24:39.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057801270000000004 2023-11-25T04:24:39.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:39.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243193+0000", 2023-11-25T04:24:39.436 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1221e-05 2023-11-25T04:24:39.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.436 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.436 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:39.436 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243198+0000", 2023-11-25T04:24:39.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3739999999999997e-06 2023-11-25T04:24:39.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:39.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243246+0000", 2023-11-25T04:24:39.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7667999999999998e-05 2023-11-25T04:24:39.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:39.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243292+0000", 2023-11-25T04:24:39.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5794000000000002e-05 2023-11-25T04:24:39.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.437 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.437 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:39.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243360+0000", 2023-11-25T04:24:39.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8479000000000006e-05 2023-11-25T04:24:39.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.438 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:39.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.250667+0000", 2023-11-25T04:24:39.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.007306618 2023-11-25T04:24:39.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.438 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:39.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.252598+0000", 2023-11-25T04:24:39.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019315129999999999 2023-11-25T04:24:39.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.439 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:39.439 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.252631+0000", 2023-11-25T04:24:39.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2925000000000001e-05 2023-11-25T04:24:39.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.439 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:39.439 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.252655+0000", 2023-11-25T04:24:39.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3246000000000001e-05 2023-11-25T04:24:39.439 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:39.439 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:39.440 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:39.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.440 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:2841 4.4 e47/17 4:22111d42:::100000010df.00000000:head v 47'139, mlcod=47'137)", 2023-11-25T04:24:39.440 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.093610+0000", 2023-11-25T04:24:39.440 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.335137119000001, 2023-11-25T04:24:39.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014758871999999999, 2023-11-25T04:24:39.440 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:39.440 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:39.440 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:39.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:39.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093610+0000", 2023-11-25T04:24:39.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:39.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:39.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093610+0000", 2023-11-25T04:24:39.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:39.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:39.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093610+0000", 2023-11-25T04:24:39.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1799999999999999e-07 2023-11-25T04:24:39.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:39.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093621+0000", 2023-11-25T04:24:39.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1107999999999999e-05 2023-11-25T04:24:39.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:39.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093627+0000", 2023-11-25T04:24:39.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0310000000000004e-06 2023-11-25T04:24:39.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:39.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093633+0000", 2023-11-25T04:24:39.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2700000000000001e-06 2023-11-25T04:24:39.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:39.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093722+0000", 2023-11-25T04:24:39.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8417000000000004e-05 2023-11-25T04:24:39.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.443 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:39.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093737+0000", 2023-11-25T04:24:39.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5781e-05 2023-11-25T04:24:39.443 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:39.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108203+0000", 2023-11-25T04:24:39.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014465844 2023-11-25T04:24:39.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.444 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:39.444 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108368+0000", 2023-11-25T04:24:39.444 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000165203 2023-11-25T04:24:39.444 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:39.444 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:39.444 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:39.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.445 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2860 4.3d 4:bf0d7877:::10000001105.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:39.445 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.093823+0000", 2023-11-25T04:24:39.445 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.334923951, 2023-11-25T04:24:39.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015582848, 2023-11-25T04:24:39.445 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:39.445 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:39.445 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:39.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.445 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:39.445 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093823+0000", 2023-11-25T04:24:39.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:39.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:39.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093823+0000", 2023-11-25T04:24:39.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:39.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:39.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093823+0000", 2023-11-25T04:24:39.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9100000000000004e-07 2023-11-25T04:24:39.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:39.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093826+0000", 2023-11-25T04:24:39.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4499999999999998e-06 2023-11-25T04:24:39.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.447 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:39.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093835+0000", 2023-11-25T04:24:39.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8179999999999993e-06 2023-11-25T04:24:39.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.447 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:39.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093840+0000", 2023-11-25T04:24:39.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0819999999999998e-06 2023-11-25T04:24:39.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:39.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093973+0000", 2023-11-25T04:24:39.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013308999999999999 2023-11-25T04:24:39.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:39.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094277+0000", 2023-11-25T04:24:39.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00030403299999999999 2023-11-25T04:24:39.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:39.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108608+0000", 2023-11-25T04:24:39.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014330888 2023-11-25T04:24:39.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.449 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:39.449 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108676+0000", 2023-11-25T04:24:39.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8577000000000004e-05 2023-11-25T04:24:39.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.449 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:39.449 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109406+0000", 2023-11-25T04:24:39.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00072931899999999997 2023-11-25T04:24:39.449 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:39.450 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:39.450 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:39.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.450 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2862 4.13 4:cb2e645e:::10000001109.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:39.450 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.093852+0000", 2023-11-25T04:24:39.450 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.33489499, 2023-11-25T04:24:39.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015190636, 2023-11-25T04:24:39.450 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:39.450 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:39.450 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:39.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.451 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:39.451 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093852+0000", 2023-11-25T04:24:39.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:39.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.451 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:39.451 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093852+0000", 2023-11-25T04:24:39.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:39.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.451 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:39.451 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093852+0000", 2023-11-25T04:24:39.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0299999999999999e-07 2023-11-25T04:24:39.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:39.452 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093854+0000", 2023-11-25T04:24:39.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.846e-06 2023-11-25T04:24:39.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:39.452 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093861+0000", 2023-11-25T04:24:39.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5520000000000001e-06 2023-11-25T04:24:39.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:39.453 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093865+0000", 2023-11-25T04:24:39.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7870000000000001e-06 2023-11-25T04:24:39.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.453 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:39.453 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093940+0000", 2023-11-25T04:24:39.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4799e-05 2023-11-25T04:24:39.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.453 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:39.453 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094052+0000", 2023-11-25T04:24:39.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011156500000000001 2023-11-25T04:24:39.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.454 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:39.454 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108928+0000", 2023-11-25T04:24:39.454 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014876245 2023-11-25T04:24:39.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.454 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:39.454 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108972+0000", 2023-11-25T04:24:39.454 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4477999999999997e-05 2023-11-25T04:24:39.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:39.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109042+0000", 2023-11-25T04:24:39.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9861000000000001e-05 2023-11-25T04:24:39.455 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:39.455 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:39.455 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:39.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.455 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2865 4.36 4:6ebc5ac6:::1000000110f.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:39.455 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.093906+0000", 2023-11-25T04:24:39.455 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.334840717000001, 2023-11-25T04:24:39.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014962628, 2023-11-25T04:24:39.456 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:39.456 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:39.456 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:39.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:39.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093906+0000", 2023-11-25T04:24:39.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:39.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:39.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093906+0000", 2023-11-25T04:24:39.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:39.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:39.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093906+0000", 2023-11-25T04:24:39.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4400000000000001e-07 2023-11-25T04:24:39.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:39.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093908+0000", 2023-11-25T04:24:39.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8640000000000001e-06 2023-11-25T04:24:39.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.458 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.458 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:39.458 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093914+0000", 2023-11-25T04:24:39.458 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0170000000000002e-06 2023-11-25T04:24:39.458 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.458 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.458 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:39.458 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093921+0000", 2023-11-25T04:24:39.459 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6529999999999997e-06 2023-11-25T04:24:39.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.459 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:39.459 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094258+0000", 2023-11-25T04:24:39.459 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00033703599999999998 2023-11-25T04:24:39.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.459 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:39.459 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094340+0000", 2023-11-25T04:24:39.459 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2279000000000003e-05 2023-11-25T04:24:39.459 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:39.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094416+0000", 2023-11-25T04:24:39.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6248999999999995e-05 2023-11-25T04:24:39.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:39.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.096134+0000", 2023-11-25T04:24:39.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001717611 2023-11-25T04:24:39.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:39.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108809+0000", 2023-11-25T04:24:39.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012675327 2023-11-25T04:24:39.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:39.461 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108836+0000", 2023-11-25T04:24:39.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6509999999999999e-05 2023-11-25T04:24:39.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:39.461 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108869+0000", 2023-11-25T04:24:39.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2737999999999997e-05 2023-11-25T04:24:39.461 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:39.461 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:39.462 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:39.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.462 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:2852 4.2d e47/17 4:b745ac76:::100000010f5.00000000:head v 47'126, mlcod=47'124)", 2023-11-25T04:24:39.462 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.093987+0000", 2023-11-25T04:24:39.462 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.334760156, 2023-11-25T04:24:39.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014613256, 2023-11-25T04:24:39.462 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:39.462 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:39.462 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:39.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.462 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:39.462 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093987+0000", 2023-11-25T04:24:39.463 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:39.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.463 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.463 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:39.463 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093987+0000", 2023-11-25T04:24:39.463 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:39.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.463 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.463 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:39.463 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093987+0000", 2023-11-25T04:24:39.463 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7000000000000001e-07 2023-11-25T04:24:39.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.463 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.464 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:39.464 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093990+0000", 2023-11-25T04:24:39.464 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8729999999999999e-06 2023-11-25T04:24:39.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.464 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:39.464 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093997+0000", 2023-11-25T04:24:39.464 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.973e-06 2023-11-25T04:24:39.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.464 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:39.464 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094001+0000", 2023-11-25T04:24:39.465 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7360000000000001e-06 2023-11-25T04:24:39.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.465 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:39.465 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094263+0000", 2023-11-25T04:24:39.465 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026173400000000001 2023-11-25T04:24:39.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.465 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:39.465 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094278+0000", 2023-11-25T04:24:39.465 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4681999999999999e-05 2023-11-25T04:24:39.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.465 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:39.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108563+0000", 2023-11-25T04:24:39.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014284738 2023-11-25T04:24:39.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:39.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108600+0000", 2023-11-25T04:24:39.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7249999999999997e-05 2023-11-25T04:24:39.466 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:39.466 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:39.466 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:39.466 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.467 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2866 4.36 4:6f6907fb:::10000001111.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:39.467 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.094033+0000", 2023-11-25T04:24:39.467 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.334713990999999, 2023-11-25T04:24:39.467 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014885591, 2023-11-25T04:24:39.467 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:39.467 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:39.467 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:39.467 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.467 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:39.467 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094033+0000", 2023-11-25T04:24:39.467 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:39.467 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.467 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.468 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:39.468 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094033+0000", 2023-11-25T04:24:39.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:39.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.468 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:39.468 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094033+0000", 2023-11-25T04:24:39.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.06e-07 2023-11-25T04:24:39.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.468 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:39.468 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094036+0000", 2023-11-25T04:24:39.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.508e-06 2023-11-25T04:24:39.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.469 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.469 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:39.469 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094044+0000", 2023-11-25T04:24:39.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5909999999999996e-06 2023-11-25T04:24:39.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.469 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.469 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:39.469 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094051+0000", 2023-11-25T04:24:39.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0319999999999998e-06 2023-11-25T04:24:39.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.469 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.469 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:39.470 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094727+0000", 2023-11-25T04:24:39.470 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00067523300000000002 2023-11-25T04:24:39.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.470 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:39.470 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094795+0000", 2023-11-25T04:24:39.470 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8156000000000003e-05 2023-11-25T04:24:39.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.470 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:39.470 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094898+0000", 2023-11-25T04:24:39.470 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103329 2023-11-25T04:24:39.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.471 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:39.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108803+0000", 2023-11-25T04:24:39.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013904385 2023-11-25T04:24:39.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.471 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:39.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108875+0000", 2023-11-25T04:24:39.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1958999999999993e-05 2023-11-25T04:24:39.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.471 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:39.472 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108899+0000", 2023-11-25T04:24:39.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4019000000000002e-05 2023-11-25T04:24:39.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.472 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:39.472 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108918+0000", 2023-11-25T04:24:39.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9772999999999999e-05 2023-11-25T04:24:39.472 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:39.472 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:39.472 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:39.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.472 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:2854 4.2d e47/17 4:b5c259cf:::100000010f9.00000000:head v 47'127, mlcod=47'124)", 2023-11-25T04:24:39.473 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.094793+0000", 2023-11-25T04:24:39.473 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.333953702000001, 2023-11-25T04:24:39.473 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014650764, 2023-11-25T04:24:39.473 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:39.473 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:39.473 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:39.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.473 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:39.473 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094793+0000", 2023-11-25T04:24:39.473 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:39.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.473 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:39.473 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094793+0000", 2023-11-25T04:24:39.474 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:39.474 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.474 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.474 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:39.474 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094793+0000", 2023-11-25T04:24:39.474 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3700000000000001e-07 2023-11-25T04:24:39.474 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.474 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.474 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:39.474 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094798+0000", 2023-11-25T04:24:39.474 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1200000000000004e-06 2023-11-25T04:24:39.474 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.474 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.475 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:39.475 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094805+0000", 2023-11-25T04:24:39.475 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9759999999999998e-06 2023-11-25T04:24:39.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.475 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.475 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:39.475 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094811+0000", 2023-11-25T04:24:39.475 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1410000000000002e-06 2023-11-25T04:24:39.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.475 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.475 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:39.475 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094838+0000", 2023-11-25T04:24:39.475 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6995000000000002e-05 2023-11-25T04:24:39.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.476 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:39.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094848+0000", 2023-11-25T04:24:39.476 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0115999999999999e-05 2023-11-25T04:24:39.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.476 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:39.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109408+0000", 2023-11-25T04:24:39.476 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014559825 2023-11-25T04:24:39.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.476 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:39.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109444+0000", 2023-11-25T04:24:39.476 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6154e-05 2023-11-25T04:24:39.477 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:39.477 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:39.477 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:39.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.477 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2869 4.6 4:63719fb0:::10000001117.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:39.477 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.094910+0000", 2023-11-25T04:24:39.477 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.333836744999999, 2023-11-25T04:24:39.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014132570000000001, 2023-11-25T04:24:39.477 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:39.477 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:39.477 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:39.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.477 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:39.478 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094910+0000", 2023-11-25T04:24:39.478 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:39.478 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.478 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.478 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:39.478 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094910+0000", 2023-11-25T04:24:39.478 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:39.478 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.478 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.478 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:39.478 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094910+0000", 2023-11-25T04:24:39.478 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.58e-07 2023-11-25T04:24:39.478 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.479 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.479 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:39.479 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094913+0000", 2023-11-25T04:24:39.479 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.198e-06 2023-11-25T04:24:39.479 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.479 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.479 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:39.479 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094921+0000", 2023-11-25T04:24:39.479 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1629999999999996e-06 2023-11-25T04:24:39.479 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.479 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.479 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:39.479 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094926+0000", 2023-11-25T04:24:39.480 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7559999999999999e-06 2023-11-25T04:24:39.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.480 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.480 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:39.480 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094966+0000", 2023-11-25T04:24:39.480 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0151999999999998e-05 2023-11-25T04:24:39.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.480 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.480 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:39.480 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095073+0000", 2023-11-25T04:24:39.480 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107657 2023-11-25T04:24:39.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.480 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.480 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:39.481 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095158+0000", 2023-11-25T04:24:39.481 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5136999999999995e-05 2023-11-25T04:24:39.481 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.481 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:39.481 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108924+0000", 2023-11-25T04:24:39.481 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013765727 2023-11-25T04:24:39.481 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.481 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:39.481 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108958+0000", 2023-11-25T04:24:39.481 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3417000000000002e-05 2023-11-25T04:24:39.482 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.482 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.482 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:39.482 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109012+0000", 2023-11-25T04:24:39.482 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3939e-05 2023-11-25T04:24:39.482 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.482 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.482 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:39.482 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109043+0000", 2023-11-25T04:24:39.482 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0966000000000001e-05 2023-11-25T04:24:39.482 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:39.482 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:39.482 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:39.483 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.483 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.483 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2871 4.3f 4:fff10e11:::1000000111b.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:39.483 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.094936+0000", 2023-11-25T04:24:39.483 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.333810907, 2023-11-25T04:24:39.483 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014261726000000001, 2023-11-25T04:24:39.483 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:39.483 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:39.483 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:39.483 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.483 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:39.483 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094936+0000", 2023-11-25T04:24:39.483 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:39.484 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.484 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.484 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:39.484 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094936+0000", 2023-11-25T04:24:39.484 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:39.484 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.484 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.484 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:39.484 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094936+0000", 2023-11-25T04:24:39.484 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9900000000000001e-07 2023-11-25T04:24:39.484 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.484 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.484 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:39.485 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094938+0000", 2023-11-25T04:24:39.485 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9599999999999999e-06 2023-11-25T04:24:39.485 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.485 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.485 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:39.485 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094944+0000", 2023-11-25T04:24:39.485 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2219999999999999e-06 2023-11-25T04:24:39.485 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.485 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.485 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:39.485 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094950+0000", 2023-11-25T04:24:39.485 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1329999999999998e-06 2023-11-25T04:24:39.486 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.486 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.486 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:39.486 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094993+0000", 2023-11-25T04:24:39.486 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3305e-05 2023-11-25T04:24:39.486 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.486 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.486 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:39.486 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095107+0000", 2023-11-25T04:24:39.486 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011449400000000001 2023-11-25T04:24:39.486 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.486 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.486 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:39.487 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095176+0000", 2023-11-25T04:24:39.487 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8330000000000005e-05 2023-11-25T04:24:39.487 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.487 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.487 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:39.487 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107939+0000", 2023-11-25T04:24:39.487 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012763801 2023-11-25T04:24:39.487 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.487 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.487 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:39.487 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109128+0000", 2023-11-25T04:24:39.487 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011884770000000001 2023-11-25T04:24:39.487 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.487 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.488 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:39.488 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109166+0000", 2023-11-25T04:24:39.488 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7948999999999999e-05 2023-11-25T04:24:39.488 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.488 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.488 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:39.488 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109198+0000", 2023-11-25T04:24:39.488 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1656000000000003e-05 2023-11-25T04:24:39.488 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:39.488 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:39.488 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:39.488 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.488 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.488 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2872 4.32 4:4fb002bc:::1000000111d.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:39.489 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.094961+0000", 2023-11-25T04:24:39.489 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.333786203000001, 2023-11-25T04:24:39.489 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014631564999999999, 2023-11-25T04:24:39.489 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:39.489 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:39.489 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:39.489 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.489 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:39.489 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094961+0000", 2023-11-25T04:24:39.489 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:39.489 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.489 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.489 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:39.490 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094961+0000", 2023-11-25T04:24:39.490 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:39.490 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.490 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.490 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:39.490 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094961+0000", 2023-11-25T04:24:39.490 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9299999999999999e-07 2023-11-25T04:24:39.490 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.490 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.490 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:39.490 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094963+0000", 2023-11-25T04:24:39.490 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9209999999999999e-06 2023-11-25T04:24:39.490 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.490 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.491 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:39.491 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094968+0000", 2023-11-25T04:24:39.491 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.502e-06 2023-11-25T04:24:39.491 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.491 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.491 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:39.491 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094973+0000", 2023-11-25T04:24:39.491 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8559999999999998e-06 2023-11-25T04:24:39.491 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.491 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.491 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:39.491 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107722+0000", 2023-11-25T04:24:39.491 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012748534000000001 2023-11-25T04:24:39.492 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.492 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.492 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:39.492 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107844+0000", 2023-11-25T04:24:39.492 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012233400000000001 2023-11-25T04:24:39.492 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.492 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.492 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:39.492 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109469+0000", 2023-11-25T04:24:39.492 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001625175 2023-11-25T04:24:39.492 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.492 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.492 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:39.493 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109518+0000", 2023-11-25T04:24:39.493 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8980000000000002e-05 2023-11-25T04:24:39.493 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.493 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.493 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:39.493 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109592+0000", 2023-11-25T04:24:39.493 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3869999999999996e-05 2023-11-25T04:24:39.493 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:39.493 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:39.493 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:39.493 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.493 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.493 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2873 4.13 4:cae4f303:::1000000111f.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:39.494 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.094991+0000", 2023-11-25T04:24:39.494 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.333755611000001, 2023-11-25T04:24:39.494 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015101692, 2023-11-25T04:24:39.494 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:39.494 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:39.494 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:39.494 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.494 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:39.494 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094991+0000", 2023-11-25T04:24:39.494 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:39.494 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.495 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.495 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:39.495 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094991+0000", 2023-11-25T04:24:39.495 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:39.495 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.495 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.495 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:39.495 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094992+0000", 2023-11-25T04:24:39.495 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7e-07 2023-11-25T04:24:39.495 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.495 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.495 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:39.495 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094994+0000", 2023-11-25T04:24:39.496 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2110000000000001e-06 2023-11-25T04:24:39.496 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.496 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.496 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:39.496 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095000+0000", 2023-11-25T04:24:39.496 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.241e-06 2023-11-25T04:24:39.496 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.496 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.496 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:39.496 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095006+0000", 2023-11-25T04:24:39.496 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3659999999999997e-06 2023-11-25T04:24:39.496 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.496 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.497 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:39.497 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108369+0000", 2023-11-25T04:24:39.497 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01336247 2023-11-25T04:24:39.497 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.497 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.497 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:39.497 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108492+0000", 2023-11-25T04:24:39.497 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000123192 2023-11-25T04:24:39.497 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.497 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.498 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:39.498 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109975+0000", 2023-11-25T04:24:39.498 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001482894 2023-11-25T04:24:39.498 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.498 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.498 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:39.498 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110024+0000", 2023-11-25T04:24:39.498 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9097000000000001e-05 2023-11-25T04:24:39.498 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.498 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.498 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:39.498 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110093+0000", 2023-11-25T04:24:39.498 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8751000000000006e-05 2023-11-25T04:24:39.499 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:39.499 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:39.499 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:39.499 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.499 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.499 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2874 4.14 4:2b557d2e:::10000001121.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:39.499 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095019+0000", 2023-11-25T04:24:39.499 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.333728085000001, 2023-11-25T04:24:39.499 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014998296, 2023-11-25T04:24:39.499 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:39.499 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:39.499 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:39.499 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.500 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:39.500 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095019+0000", 2023-11-25T04:24:39.500 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:39.500 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.500 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.500 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:39.500 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095019+0000", 2023-11-25T04:24:39.500 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:39.500 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.500 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.500 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:39.500 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095019+0000", 2023-11-25T04:24:39.500 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7300000000000001e-07 2023-11-25T04:24:39.501 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.501 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.501 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:39.501 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095021+0000", 2023-11-25T04:24:39.501 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0099999999999998e-06 2023-11-25T04:24:39.501 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.501 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.501 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:39.501 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095028+0000", 2023-11-25T04:24:39.501 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7870000000000004e-06 2023-11-25T04:24:39.501 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.501 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.501 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:39.502 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095033+0000", 2023-11-25T04:24:39.502 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2039999999999996e-06 2023-11-25T04:24:39.502 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.502 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.502 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:39.502 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107740+0000", 2023-11-25T04:24:39.502 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012706693 2023-11-25T04:24:39.502 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.502 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.502 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:39.502 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107817+0000", 2023-11-25T04:24:39.502 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7599e-05 2023-11-25T04:24:39.503 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.503 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.503 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:39.503 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107961+0000", 2023-11-25T04:24:39.503 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000143194 2023-11-25T04:24:39.503 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.503 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.503 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:39.503 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109364+0000", 2023-11-25T04:24:39.503 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001403226 2023-11-25T04:24:39.503 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.503 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.503 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:39.504 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109507+0000", 2023-11-25T04:24:39.504 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000142991 2023-11-25T04:24:39.504 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.504 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.504 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:39.504 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109986+0000", 2023-11-25T04:24:39.504 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00047906899999999999 2023-11-25T04:24:39.504 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.504 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.504 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:39.504 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110017+0000", 2023-11-25T04:24:39.504 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1050000000000003e-05 2023-11-25T04:24:39.505 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:39.505 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:39.505 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:39.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.505 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.505 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2875 4.d 4:b17d5dc6:::10000001123.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:39.505 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095049+0000", 2023-11-25T04:24:39.505 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.333698160999999, 2023-11-25T04:24:39.505 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014882886, 2023-11-25T04:24:39.505 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:39.505 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:39.505 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:39.505 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.506 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:39.506 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095049+0000", 2023-11-25T04:24:39.506 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:39.506 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.506 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.506 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:39.506 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095049+0000", 2023-11-25T04:24:39.506 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:39.506 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.506 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.506 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:39.506 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095049+0000", 2023-11-25T04:24:39.507 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6100000000000001e-07 2023-11-25T04:24:39.507 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.507 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.507 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:39.507 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095051+0000", 2023-11-25T04:24:39.507 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1449999999999998e-06 2023-11-25T04:24:39.507 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.507 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.507 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:39.507 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095058+0000", 2023-11-25T04:24:39.507 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.601e-06 2023-11-25T04:24:39.507 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.507 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.508 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:39.508 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095064+0000", 2023-11-25T04:24:39.508 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1079999999999998e-06 2023-11-25T04:24:39.508 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.508 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.508 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:39.508 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108015+0000", 2023-11-25T04:24:39.508 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012951582 2023-11-25T04:24:39.508 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.508 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.508 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:39.508 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108260+0000", 2023-11-25T04:24:39.509 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024445199999999998 2023-11-25T04:24:39.509 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.509 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.509 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:39.509 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109825+0000", 2023-11-25T04:24:39.509 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0015651759999999999 2023-11-25T04:24:39.509 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.509 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.509 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:39.509 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109866+0000", 2023-11-25T04:24:39.510 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0629e-05 2023-11-25T04:24:39.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.510 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.510 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:39.510 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109931+0000", 2023-11-25T04:24:39.510 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5732e-05 2023-11-25T04:24:39.510 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:39.510 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:39.510 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:39.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.510 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.510 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2876 4.3b 4:dc871776:::10000001125.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:39.511 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095077+0000", 2023-11-25T04:24:39.511 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.333669540000001, 2023-11-25T04:24:39.511 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014500944, 2023-11-25T04:24:39.511 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:39.511 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:39.511 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:39.511 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.511 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:39.511 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095077+0000", 2023-11-25T04:24:39.511 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:39.511 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.511 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.511 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:39.512 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095077+0000", 2023-11-25T04:24:39.512 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:39.512 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.512 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.512 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:39.512 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095078+0000", 2023-11-25T04:24:39.512 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9799999999999999e-07 2023-11-25T04:24:39.512 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.512 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.512 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:39.512 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095079+0000", 2023-11-25T04:24:39.512 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.877e-06 2023-11-25T04:24:39.513 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.513 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.513 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:39.513 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095086+0000", 2023-11-25T04:24:39.513 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2439999999999998e-06 2023-11-25T04:24:39.513 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.513 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.513 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:39.513 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095094+0000", 2023-11-25T04:24:39.513 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9710000000000004e-06 2023-11-25T04:24:39.513 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.513 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.513 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:39.514 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107732+0000", 2023-11-25T04:24:39.514 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012638104000000001 2023-11-25T04:24:39.514 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.514 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.514 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:39.514 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107854+0000", 2023-11-25T04:24:39.514 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012261600000000001 2023-11-25T04:24:39.514 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.514 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.514 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:39.514 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107941+0000", 2023-11-25T04:24:39.514 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6935999999999996e-05 2023-11-25T04:24:39.514 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.515 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.515 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:39.515 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108871+0000", 2023-11-25T04:24:39.515 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00092955499999999999 2023-11-25T04:24:39.515 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.515 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.515 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:39.515 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109483+0000", 2023-11-25T04:24:39.515 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00061206100000000001 2023-11-25T04:24:39.515 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.515 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.515 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:39.516 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109527+0000", 2023-11-25T04:24:39.516 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4112e-05 2023-11-25T04:24:39.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.516 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.516 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:39.516 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109578+0000", 2023-11-25T04:24:39.516 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1069999999999997e-05 2023-11-25T04:24:39.516 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:39.516 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:39.516 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:39.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.516 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.516 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2878 4.3e 4:7ff98b9d:::10000001129.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:39.517 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095101+0000", 2023-11-25T04:24:39.517 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.333645756999999, 2023-11-25T04:24:39.517 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015015137, 2023-11-25T04:24:39.517 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:39.517 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:39.517 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:39.517 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.517 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:39.517 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095101+0000", 2023-11-25T04:24:39.517 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:39.517 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.517 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.517 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:39.518 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095101+0000", 2023-11-25T04:24:39.518 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:39.518 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.518 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.518 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:39.518 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095101+0000", 2023-11-25T04:24:39.518 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7399999999999999e-07 2023-11-25T04:24:39.518 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.518 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.518 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:39.518 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095103+0000", 2023-11-25T04:24:39.518 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7600000000000001e-06 2023-11-25T04:24:39.518 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.519 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.519 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:39.519 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095109+0000", 2023-11-25T04:24:39.519 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8189999999999997e-06 2023-11-25T04:24:39.519 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.519 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.519 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:39.519 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095114+0000", 2023-11-25T04:24:39.519 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6770000000000003e-06 2023-11-25T04:24:39.519 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.519 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.519 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:39.519 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107724+0000", 2023-11-25T04:24:39.520 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012610909999999999 2023-11-25T04:24:39.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.520 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.520 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:39.520 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108483+0000", 2023-11-25T04:24:39.520 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000758407 2023-11-25T04:24:39.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.520 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.520 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:39.520 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108568+0000", 2023-11-25T04:24:39.520 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4707000000000001e-05 2023-11-25T04:24:39.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.520 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.521 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:39.521 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109687+0000", 2023-11-25T04:24:39.521 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011192750000000001 2023-11-25T04:24:39.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.521 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.521 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:39.521 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110049+0000", 2023-11-25T04:24:39.521 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00036210600000000001 2023-11-25T04:24:39.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.521 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.521 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:39.521 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110086+0000", 2023-11-25T04:24:39.522 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7500999999999999e-05 2023-11-25T04:24:39.522 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.522 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.522 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:39.522 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110116+0000", 2023-11-25T04:24:39.522 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9601e-05 2023-11-25T04:24:39.522 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:39.522 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:39.522 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:39.522 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.522 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.522 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2879 4.2a 4:5524ace2:::1000000112b.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:39.522 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095120+0000", 2023-11-25T04:24:39.523 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.333626977, 2023-11-25T04:24:39.523 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014587937, 2023-11-25T04:24:39.523 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:39.523 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:39.523 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:39.523 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.523 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:39.523 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095120+0000", 2023-11-25T04:24:39.523 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:39.523 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.523 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.523 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:39.524 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095120+0000", 2023-11-25T04:24:39.524 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:39.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.524 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.524 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:39.524 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095120+0000", 2023-11-25T04:24:39.524 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5600000000000001e-07 2023-11-25T04:24:39.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.524 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.524 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:39.524 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095122+0000", 2023-11-25T04:24:39.524 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.694e-06 2023-11-25T04:24:39.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.525 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.525 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:39.525 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095127+0000", 2023-11-25T04:24:39.525 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5099999999999998e-06 2023-11-25T04:24:39.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.525 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.525 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:39.525 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095132+0000", 2023-11-25T04:24:39.525 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6210000000000003e-06 2023-11-25T04:24:39.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.525 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.525 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:39.526 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107729+0000", 2023-11-25T04:24:39.526 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012596931 2023-11-25T04:24:39.526 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.526 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.526 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:39.526 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107856+0000", 2023-11-25T04:24:39.526 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012707699999999999 2023-11-25T04:24:39.526 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.526 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.526 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:39.526 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109600+0000", 2023-11-25T04:24:39.526 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001744298 2023-11-25T04:24:39.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.527 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.527 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:39.527 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109641+0000", 2023-11-25T04:24:39.527 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0769000000000002e-05 2023-11-25T04:24:39.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.527 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.527 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:39.527 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109708+0000", 2023-11-25T04:24:39.527 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6680999999999994e-05 2023-11-25T04:24:39.527 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:39.527 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:39.527 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:39.528 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.528 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.528 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2882 4.1c 4:3a7c2e07:::10000001131.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:39.528 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095141+0000", 2023-11-25T04:24:39.528 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.333605602, 2023-11-25T04:24:39.528 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014949469999999999, 2023-11-25T04:24:39.528 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:39.528 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:39.528 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:39.528 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.528 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:39.528 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095141+0000", 2023-11-25T04:24:39.529 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:39.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.529 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.529 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:39.529 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095141+0000", 2023-11-25T04:24:39.529 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:39.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.529 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.529 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:39.529 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095142+0000", 2023-11-25T04:24:39.529 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1899999999999998e-07 2023-11-25T04:24:39.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.529 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.530 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:39.530 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095143+0000", 2023-11-25T04:24:39.530 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7770000000000001e-06 2023-11-25T04:24:39.530 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.530 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.530 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:39.530 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095149+0000", 2023-11-25T04:24:39.530 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9909999999999999e-06 2023-11-25T04:24:39.530 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.530 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.530 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:39.530 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095154+0000", 2023-11-25T04:24:39.531 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3880000000000002e-06 2023-11-25T04:24:39.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.531 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.531 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:39.531 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107736+0000", 2023-11-25T04:24:39.531 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012582477999999999 2023-11-25T04:24:39.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.531 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.531 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:39.531 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107868+0000", 2023-11-25T04:24:39.531 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013134400000000001 2023-11-25T04:24:39.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.531 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.532 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:39.532 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107959+0000", 2023-11-25T04:24:39.532 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1253000000000002e-05 2023-11-25T04:24:39.532 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.532 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.532 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:39.532 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109325+0000", 2023-11-25T04:24:39.532 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0013662520000000001 2023-11-25T04:24:39.532 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.532 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.532 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:39.532 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110023+0000", 2023-11-25T04:24:39.532 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00069838799999999998 2023-11-25T04:24:39.533 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.533 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.533 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:39.533 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110061+0000", 2023-11-25T04:24:39.533 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7867999999999997e-05 2023-11-25T04:24:39.533 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.533 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.533 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:39.533 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110091+0000", 2023-11-25T04:24:39.533 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9312e-05 2023-11-25T04:24:39.533 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:39.533 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:39.534 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:39.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.534 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.534 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2883 4.3e 4:7d82be99:::10000001133.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:39.534 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095942+0000", 2023-11-25T04:24:39.534 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.332804316000001, 2023-11-25T04:24:39.534 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014290573000000001, 2023-11-25T04:24:39.534 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:39.534 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:39.534 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:39.534 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.534 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:39.535 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095942+0000", 2023-11-25T04:24:39.535 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:39.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.535 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:39.535 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095942+0000", 2023-11-25T04:24:39.535 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:39.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.535 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.535 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:39.535 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095944+0000", 2023-11-25T04:24:39.535 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.632e-06 2023-11-25T04:24:39.535 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.536 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.536 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:39.536 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095947+0000", 2023-11-25T04:24:39.536 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7760000000000002e-06 2023-11-25T04:24:39.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.536 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.536 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:39.536 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095954+0000", 2023-11-25T04:24:39.536 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5379999999999997e-06 2023-11-25T04:24:39.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.536 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.536 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:39.537 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095960+0000", 2023-11-25T04:24:39.537 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9209999999999997e-06 2023-11-25T04:24:39.537 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.537 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.537 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:39.537 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108809+0000", 2023-11-25T04:24:39.537 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012848972 2023-11-25T04:24:39.537 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.537 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.537 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:39.537 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108921+0000", 2023-11-25T04:24:39.537 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000111767 2023-11-25T04:24:39.537 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.538 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.538 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:39.538 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109008+0000", 2023-11-25T04:24:39.538 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6576999999999995e-05 2023-11-25T04:24:39.538 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.538 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.538 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:39.538 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110124+0000", 2023-11-25T04:24:39.538 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011163709999999999 2023-11-25T04:24:39.538 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.538 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.538 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:39.539 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110155+0000", 2023-11-25T04:24:39.539 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1248999999999998e-05 2023-11-25T04:24:39.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.539 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:39.539 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110202+0000", 2023-11-25T04:24:39.539 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7097e-05 2023-11-25T04:24:39.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.539 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:39.539 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110233+0000", 2023-11-25T04:24:39.539 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0673000000000003e-05 2023-11-25T04:24:39.539 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:39.540 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:39.540 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:39.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.540 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.540 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2886 4.25 4:a579baeb:::10000001139.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:39.540 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095994+0000", 2023-11-25T04:24:39.540 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 14.332752987999999, 2023-11-25T04:24:39.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013825356, 2023-11-25T04:24:39.540 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:39.540 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:39.540 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:39.540 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.540 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:39.541 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095994+0000", 2023-11-25T04:24:39.541 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:39.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.541 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.541 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:39.541 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095994+0000", 2023-11-25T04:24:39.541 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:39.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.541 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.541 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:39.541 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095996+0000", 2023-11-25T04:24:39.541 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8780000000000001e-06 2023-11-25T04:24:39.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.542 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.542 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:39.542 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095998+0000", 2023-11-25T04:24:39.542 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1560000000000002e-06 2023-11-25T04:24:39.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.542 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.542 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:39.542 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.096003+0000", 2023-11-25T04:24:39.542 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1080000000000001e-06 2023-11-25T04:24:39.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.542 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.542 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:39.542 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.096010+0000", 2023-11-25T04:24:39.543 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0650000000000001e-06 2023-11-25T04:24:39.543 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.543 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.543 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:39.543 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107742+0000", 2023-11-25T04:24:39.543 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011731691000000001 2023-11-25T04:24:39.543 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.543 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.543 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:39.543 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108063+0000", 2023-11-25T04:24:39.543 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00032168600000000001 2023-11-25T04:24:39.543 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.543 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.543 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:39.544 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109689+0000", 2023-11-25T04:24:39.544 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001625556 2023-11-25T04:24:39.544 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.544 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.544 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:39.544 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109740+0000", 2023-11-25T04:24:39.544 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1125000000000003e-05 2023-11-25T04:24:39.544 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:39.544 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:39.544 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:39.544 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109819+0000", 2023-11-25T04:24:39.544 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9091000000000006e-05 2023-11-25T04:24:39.544 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:39.544 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:39.545 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:39.545 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:39.545 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:39.545 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:39.546 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:24:39.547 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:39.544+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:39.647 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:39.645+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:39.655 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:39.655 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:24:39.656 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2023-11-25T04:24:39.656 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:39.665 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:24:39.704 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_ceph_ceph-c_ec928368edcc5fc71884a56fd9fd821b148f4e30/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/github.com_ceph_ceph-c_ec928368edcc5fc71884a56fd9fd821b148f4e30/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/github.com_ceph_ceph-c_ec928368edcc5fc71884a56fd9fd821b148f4e30/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/fsx.sh) on smithi067 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ec928368edcc5fc71884a56fd9fd821b148f4e30 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh' 2023-11-25T04:24:39.840 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=8e4f33aab3754c9bb578e9a1b80e8d3e Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_ceph_ceph-c_ec928368edcc5fc71884a56fd9fd821b148f4e30/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/github.com_ceph_ceph-c_ec928368edcc5fc71884a56fd9fd821b148f4e30/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/github.com_ceph_ceph-c_ec928368edcc5fc71884a56fd9fd821b148f4e30/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/fsx.sh) on smithi067 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ec928368edcc5fc71884a56fd9fd821b148f4e30 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh' 2023-11-25T04:24:39.842 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2023-11-25T04:24:39.851 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2023-11-25T04:24:39.852 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-11-25T04:24:39.852 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-25T04:24:39.867 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:39.865+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:39.868 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:39.868 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:24:39.868 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2023-11-25T04:24:39.869 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2023-11-25T04:24:39.869 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:39.869 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:24:39.881 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-11-25T04:24:39.882 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-25T04:24:39.938 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-25T04:24:39.967 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:39.965+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:39.983 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:39.984 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2023-11-25T04:24:39.984 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2023-11-25T04:24:39.984 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2023-11-25T04:24:39.984 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.984 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:39.984 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:24:39.985 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 112.949485, 2023-11-25T04:24:39.985 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01072943, 2023-11-25T04:24:39.985 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:39.985 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:39.985 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:39.985 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.986 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:39.986 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:24:39.986 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:39.986 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.986 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.986 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:39.986 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:24:39.987 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:39.987 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.987 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.987 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:39.987 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031829+0000", 2023-11-25T04:24:39.987 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.666e-06 2023-11-25T04:24:39.988 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.988 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.988 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:39.988 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031839+0000", 2023-11-25T04:24:39.988 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9490000000000008e-06 2023-11-25T04:24:39.988 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.988 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.988 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:39.989 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031857+0000", 2023-11-25T04:24:39.989 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8093000000000001e-05 2023-11-25T04:24:39.989 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.989 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.989 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:39.989 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031882+0000", 2023-11-25T04:24:39.990 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4294000000000002e-05 2023-11-25T04:24:39.990 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.990 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.990 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:39.990 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033130+0000", 2023-11-25T04:24:39.990 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001248264 2023-11-25T04:24:39.990 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.991 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.991 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:39.991 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033426+0000", 2023-11-25T04:24:39.991 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029620999999999998 2023-11-25T04:24:39.991 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.991 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.991 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:39.991 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033814+0000", 2023-11-25T04:24:39.992 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000387742 2023-11-25T04:24:39.992 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.992 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.992 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:39.992 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036793+0000", 2023-11-25T04:24:39.992 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002979598 2023-11-25T04:24:39.992 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.992 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.993 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:39.993 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042455+0000", 2023-11-25T04:24:39.993 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0056619260000000003 2023-11-25T04:24:39.993 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.993 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.993 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:39.993 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042502+0000", 2023-11-25T04:24:39.993 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7139999999999999e-05 2023-11-25T04:24:39.994 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.994 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.994 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:39.994 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042557+0000", 2023-11-25T04:24:39.994 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4548e-05 2023-11-25T04:24:39.994 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:39.994 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:39.995 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:39.995 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.995 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.995 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:39.995 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:24:39.995 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 112.948286465, 2023-11-25T04:24:39.995 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011505202000000001, 2023-11-25T04:24:39.995 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:39.996 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:39.996 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:39.996 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.996 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:39.996 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:24:39.996 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:39.996 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.996 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.997 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:39.997 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:24:39.997 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:39.997 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.997 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.997 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:39.997 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033027+0000", 2023-11-25T04:24:39.998 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0699999999999999e-06 2023-11-25T04:24:39.998 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.998 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.998 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:39.998 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033035+0000", 2023-11-25T04:24:39.998 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0830000000000004e-06 2023-11-25T04:24:39.998 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.999 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.999 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:39.999 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033059+0000", 2023-11-25T04:24:39.999 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3975e-05 2023-11-25T04:24:39.999 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:39.999 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:39.999 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:39.999 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033482+0000", 2023-11-25T04:24:40.000 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042253700000000001 2023-11-25T04:24:40.000 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.000 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.000 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:40.000 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034520+0000", 2023-11-25T04:24:40.000 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001037991 2023-11-25T04:24:40.000 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.000 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.001 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:40.001 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034611+0000", 2023-11-25T04:24:40.001 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1113999999999995e-05 2023-11-25T04:24:40.001 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.001 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.001 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:40.001 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034769+0000", 2023-11-25T04:24:40.001 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000157641 2023-11-25T04:24:40.001 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.002 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.002 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:40.002 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038292+0000", 2023-11-25T04:24:40.002 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035231160000000002 2023-11-25T04:24:40.002 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.002 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.002 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:40.002 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044440+0000", 2023-11-25T04:24:40.003 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0061486980000000002 2023-11-25T04:24:40.003 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.003 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.003 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:40.003 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044495+0000", 2023-11-25T04:24:40.003 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4339000000000003e-05 2023-11-25T04:24:40.003 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.003 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.004 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:40.004 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044531+0000", 2023-11-25T04:24:40.004 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6637999999999997e-05 2023-11-25T04:24:40.004 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:40.004 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:40.004 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:40.004 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.004 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.005 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:26 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:40.005 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:40.005 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 112.947697703, 2023-11-25T04:24:40.005 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011086945000000001, 2023-11-25T04:24:40.005 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:40.005 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:40.005 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:40.005 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.006 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:40.006 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:40.006 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:40.006 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.006 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.006 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:40.006 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:40.006 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:40.007 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.007 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.007 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:40.007 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:40.007 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1699999999999998e-07 2023-11-25T04:24:40.007 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.007 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.007 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:40.008 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033627+0000", 2023-11-25T04:24:40.008 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.163e-05 2023-11-25T04:24:40.008 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.008 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.008 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:40.008 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033636+0000", 2023-11-25T04:24:40.008 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6859999999999995e-06 2023-11-25T04:24:40.008 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.009 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.009 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:40.009 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033642+0000", 2023-11-25T04:24:40.009 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3149999999999997e-06 2023-11-25T04:24:40.009 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.009 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.009 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:40.010 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035414+0000", 2023-11-25T04:24:40.010 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0017716380000000001 2023-11-25T04:24:40.010 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.010 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.010 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:40.010 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035539+0000", 2023-11-25T04:24:40.010 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000125477 2023-11-25T04:24:40.010 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.011 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.011 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:40.011 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035672+0000", 2023-11-25T04:24:40.011 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000133316 2023-11-25T04:24:40.011 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.011 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.011 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:40.011 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038649+0000", 2023-11-25T04:24:40.012 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029764679999999999 2023-11-25T04:24:40.012 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.012 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.012 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:40.012 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044578+0000", 2023-11-25T04:24:40.012 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0059293510000000002 2023-11-25T04:24:40.012 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.012 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.013 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:40.013 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044651+0000", 2023-11-25T04:24:40.013 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2513000000000002e-05 2023-11-25T04:24:40.013 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.013 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.013 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:40.013 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044702+0000", 2023-11-25T04:24:40.013 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1034e-05 2023-11-25T04:24:40.014 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:40.014 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:40.014 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:40.014 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.014 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.014 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:29 3.b 3:d0630e4c:::mds0_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:40.014 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:40.014 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 112.94765978300001, 2023-11-25T04:24:40.015 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010980227, 2023-11-25T04:24:40.015 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:40.015 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:40.015 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:40.015 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.015 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:40.015 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:40.015 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:40.015 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.016 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.016 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:40.016 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:40.016 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:40.016 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.016 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.016 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:40.016 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:40.016 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3499999999999996e-07 2023-11-25T04:24:40.017 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.017 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.017 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:40.017 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033658+0000", 2023-11-25T04:24:40.017 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1280000000000001e-06 2023-11-25T04:24:40.017 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.018 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.018 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:40.018 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033667+0000", 2023-11-25T04:24:40.018 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1430000000000004e-06 2023-11-25T04:24:40.018 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.018 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.018 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:40.018 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033678+0000", 2023-11-25T04:24:40.019 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0906e-05 2023-11-25T04:24:40.019 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.019 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.019 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:40.019 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034456+0000", 2023-11-25T04:24:40.019 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00077833899999999996 2023-11-25T04:24:40.019 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.019 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.020 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:40.020 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034575+0000", 2023-11-25T04:24:40.020 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011926000000000001 2023-11-25T04:24:40.020 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.021 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.021 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:40.021 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035001+0000", 2023-11-25T04:24:40.021 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042550999999999998 2023-11-25T04:24:40.022 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.022 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.022 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:40.022 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.037923+0000", 2023-11-25T04:24:40.022 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029219670000000001 2023-11-25T04:24:40.023 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.023 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.023 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:40.023 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044538+0000", 2023-11-25T04:24:40.023 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0066152349999999997 2023-11-25T04:24:40.023 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.023 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.024 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:40.024 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044594+0000", 2023-11-25T04:24:40.024 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5593000000000001e-05 2023-11-25T04:24:40.024 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.024 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.024 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:40.024 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044633+0000", 2023-11-25T04:24:40.025 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9511000000000002e-05 2023-11-25T04:24:40.025 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:40.025 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:40.025 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:40.025 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.025 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.026 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:34 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:40.026 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:24:40.026 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 112.94740967, 2023-11-25T04:24:40.026 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010920445000000001, 2023-11-25T04:24:40.026 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:40.026 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:40.026 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:40.027 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.027 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:40.027 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:24:40.027 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:40.027 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.027 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.029 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:40.029 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:24:40.029 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:40.029 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.029 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.029 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:40.030 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033904+0000", 2023-11-25T04:24:40.030 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6599999999999994e-07 2023-11-25T04:24:40.030 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.030 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.030 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:40.030 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033909+0000", 2023-11-25T04:24:40.030 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5639999999999996e-06 2023-11-25T04:24:40.031 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.031 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.031 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:40.031 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033923+0000", 2023-11-25T04:24:40.031 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3028e-05 2023-11-25T04:24:40.031 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.031 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.031 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:40.032 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034043+0000", 2023-11-25T04:24:40.032 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000120328 2023-11-25T04:24:40.032 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.032 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.032 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:40.032 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035576+0000", 2023-11-25T04:24:40.032 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015333720000000001 2023-11-25T04:24:40.033 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.033 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.033 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:40.033 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035699+0000", 2023-11-25T04:24:40.033 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000123001 2023-11-25T04:24:40.033 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.033 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.033 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:40.034 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044823+0000", 2023-11-25T04:24:40.034 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0091241859999999994 2023-11-25T04:24:40.034 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:40.034 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:40.034 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:40.034 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.034 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.034 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:118 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals,omap-rm-keys] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e44)", 2023-11-25T04:24:40.035 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:17.112648+0000", 2023-11-25T04:24:40.035 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 22.868664329000001, 2023-11-25T04:24:40.035 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011250779000000001, 2023-11-25T04:24:40.035 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:40.035 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:40.035 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:40.035 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.035 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:40.035 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.112648+0000", 2023-11-25T04:24:40.036 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:40.036 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.036 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.036 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:40.036 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.112648+0000", 2023-11-25T04:24:40.036 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:40.036 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.036 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.037 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:40.037 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.112649+0000", 2023-11-25T04:24:40.037 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0070000000000001e-06 2023-11-25T04:24:40.037 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.037 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.037 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:40.037 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113182+0000", 2023-11-25T04:24:40.037 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00053268100000000002 2023-11-25T04:24:40.038 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.038 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.038 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:40.038 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113196+0000", 2023-11-25T04:24:40.038 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3742e-05 2023-11-25T04:24:40.038 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.038 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.038 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:40.039 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113398+0000", 2023-11-25T04:24:40.039 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000202726 2023-11-25T04:24:40.039 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.039 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.039 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:40.039 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113447+0000", 2023-11-25T04:24:40.039 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8655000000000001e-05 2023-11-25T04:24:40.039 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.039 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.040 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:40.040 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113529+0000", 2023-11-25T04:24:40.040 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1891999999999995e-05 2023-11-25T04:24:40.040 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.040 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.040 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:40.040 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123819+0000", 2023-11-25T04:24:40.040 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010290192 2023-11-25T04:24:40.041 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.041 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.041 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:40.041 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123852+0000", 2023-11-25T04:24:40.041 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2425999999999998e-05 2023-11-25T04:24:40.041 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.041 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.041 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:40.042 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123899+0000", 2023-11-25T04:24:40.042 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7457999999999999e-05 2023-11-25T04:24:40.042 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:40.042 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:40.042 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:40.042 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.042 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.042 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:154 3.1e 3:7bf7e7b5:::200.00000006:head [write 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:40.043 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:21.886430+0000", 2023-11-25T04:24:40.043 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 18.094883063000001, 2023-11-25T04:24:40.043 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.017800107999999999, 2023-11-25T04:24:40.043 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:40.043 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:40.043 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:40.043 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.043 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:40.044 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.886430+0000", 2023-11-25T04:24:40.044 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:40.044 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.044 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.044 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:40.044 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.886430+0000", 2023-11-25T04:24:40.044 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:40.044 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.044 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.045 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:40.045 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.886431+0000", 2023-11-25T04:24:40.045 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.178e-06 2023-11-25T04:24:40.045 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.045 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.045 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:40.045 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.892277+0000", 2023-11-25T04:24:40.045 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0058467409999999999 2023-11-25T04:24:40.046 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.046 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.046 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:40.046 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.892288+0000", 2023-11-25T04:24:40.046 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0183000000000001e-05 2023-11-25T04:24:40.046 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.046 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.049 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:40.049 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.892292+0000", 2023-11-25T04:24:40.049 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7829999999999998e-06 2023-11-25T04:24:40.049 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.050 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.050 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:40.050 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.892362+0000", 2023-11-25T04:24:40.050 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0034000000000001e-05 2023-11-25T04:24:40.050 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.051 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.051 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:40.051 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.892432+0000", 2023-11-25T04:24:40.051 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9648999999999997e-05 2023-11-25T04:24:40.051 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.051 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.051 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:40.051 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.892550+0000", 2023-11-25T04:24:40.052 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011825 2023-11-25T04:24:40.052 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.052 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.052 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:40.052 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.899106+0000", 2023-11-25T04:24:40.052 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0065556759999999999 2023-11-25T04:24:40.052 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.052 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.053 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:40.053 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.904145+0000", 2023-11-25T04:24:40.053 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0050393490000000003 2023-11-25T04:24:40.053 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.053 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.053 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:40.053 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.904200+0000", 2023-11-25T04:24:40.053 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4234999999999998e-05 2023-11-25T04:24:40.054 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.054 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.054 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:40.054 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.904230+0000", 2023-11-25T04:24:40.054 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0029999999999999e-05 2023-11-25T04:24:40.054 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:40.054 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:40.054 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:40.054 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.055 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.055 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:576 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals,omap-rm-keys] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:40.055 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:22.111407+0000", 2023-11-25T04:24:40.055 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 17.869905457000002, 2023-11-25T04:24:40.055 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.018987317, 2023-11-25T04:24:40.055 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:40.055 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:40.055 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:40.056 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.056 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:40.056 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111407+0000", 2023-11-25T04:24:40.056 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:40.056 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.056 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.056 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:40.056 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111407+0000", 2023-11-25T04:24:40.057 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:40.057 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.057 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.057 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:40.057 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111408+0000", 2023-11-25T04:24:40.057 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8000000000000004e-07 2023-11-25T04:24:40.057 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.057 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.058 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:40.058 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111659+0000", 2023-11-25T04:24:40.058 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000250537 2023-11-25T04:24:40.058 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.058 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.058 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:40.058 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111666+0000", 2023-11-25T04:24:40.058 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.973e-06 2023-11-25T04:24:40.059 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.059 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.059 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:40.059 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111670+0000", 2023-11-25T04:24:40.059 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.668e-06 2023-11-25T04:24:40.059 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.059 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.059 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:40.059 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111709+0000", 2023-11-25T04:24:40.060 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8800000000000001e-05 2023-11-25T04:24:40.060 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.060 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.060 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:40.060 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111761+0000", 2023-11-25T04:24:40.060 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1495000000000001e-05 2023-11-25T04:24:40.060 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.060 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.061 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:40.061 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.130255+0000", 2023-11-25T04:24:40.061 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.018494345999999998 2023-11-25T04:24:40.061 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.061 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.061 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:40.061 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.130311+0000", 2023-11-25T04:24:40.061 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5902000000000001e-05 2023-11-25T04:24:40.061 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.062 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.062 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:40.062 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.130394+0000", 2023-11-25T04:24:40.062 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3715999999999996e-05 2023-11-25T04:24:40.062 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:40.062 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:40.062 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:40.062 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.063 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.063 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:1067 4.21 4:869a7600:::100000011fa.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:40.063 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:22.155038+0000", 2023-11-25T04:24:40.063 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 17.826274119000001, 2023-11-25T04:24:40.063 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011313148, 2023-11-25T04:24:40.063 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:40.063 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:40.063 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:40.064 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.064 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:40.064 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155038+0000", 2023-11-25T04:24:40.064 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:40.064 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.064 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.064 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:40.064 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155038+0000", 2023-11-25T04:24:40.065 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:40.065 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.065 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.065 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:40.065 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155039+0000", 2023-11-25T04:24:40.065 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0200000000000002e-07 2023-11-25T04:24:40.065 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.065 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.066 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:40.066 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155049+0000", 2023-11-25T04:24:40.066 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0295000000000001e-05 2023-11-25T04:24:40.066 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.066 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.066 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:40.066 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155058+0000", 2023-11-25T04:24:40.066 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.208e-06 2023-11-25T04:24:40.066 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.067 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.067 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:40.067 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155064+0000", 2023-11-25T04:24:40.067 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4489999999999996e-06 2023-11-25T04:24:40.067 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.067 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.067 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:40.067 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155104+0000", 2023-11-25T04:24:40.068 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0164999999999999e-05 2023-11-25T04:24:40.068 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.068 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.068 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:40.068 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155147+0000", 2023-11-25T04:24:40.068 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2576e-05 2023-11-25T04:24:40.068 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.068 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.069 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:40.069 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155212+0000", 2023-11-25T04:24:40.069 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5516000000000001e-05 2023-11-25T04:24:40.069 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.069 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.069 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:40.069 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.156025+0000", 2023-11-25T04:24:40.069 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00081248599999999996 2023-11-25T04:24:40.070 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.138 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.139 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:40.139 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.166250+0000", 2023-11-25T04:24:40.139 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010224736999999999 2023-11-25T04:24:40.139 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.139 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.139 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:40.139 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.166319+0000", 2023-11-25T04:24:40.139 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9145000000000003e-05 2023-11-25T04:24:40.140 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.140 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.140 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:40.140 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.166352+0000", 2023-11-25T04:24:40.140 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2969000000000002e-05 2023-11-25T04:24:40.140 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:40.140 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:40.140 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:40.140 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.141 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.141 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:1102 4.2f 4:f543b94c:::1000000121d.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:40.141 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:22.158246+0000", 2023-11-25T04:24:40.141 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 17.823066596, 2023-11-25T04:24:40.141 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010094396, 2023-11-25T04:24:40.141 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:40.141 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:40.141 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:40.142 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.142 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:40.142 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158246+0000", 2023-11-25T04:24:40.142 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:40.142 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.142 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.142 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:40.142 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158246+0000", 2023-11-25T04:24:40.143 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:40.143 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.143 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.143 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:40.143 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158246+0000", 2023-11-25T04:24:40.143 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5299999999999999e-07 2023-11-25T04:24:40.143 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.143 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.143 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:40.144 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158249+0000", 2023-11-25T04:24:40.144 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2840000000000001e-06 2023-11-25T04:24:40.144 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.144 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.144 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:40.144 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158254+0000", 2023-11-25T04:24:40.144 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3449999999999996e-06 2023-11-25T04:24:40.144 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.145 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.145 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:40.145 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158258+0000", 2023-11-25T04:24:40.145 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5030000000000002e-06 2023-11-25T04:24:40.145 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.145 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.145 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:40.145 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158302+0000", 2023-11-25T04:24:40.145 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3908e-05 2023-11-25T04:24:40.146 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.146 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.146 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:40.146 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158398+0000", 2023-11-25T04:24:40.146 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6898000000000006e-05 2023-11-25T04:24:40.146 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.146 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.146 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:40.147 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158480+0000", 2023-11-25T04:24:40.147 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1578999999999999e-05 2023-11-25T04:24:40.147 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.147 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.147 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:40.147 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.159229+0000", 2023-11-25T04:24:40.147 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000749049 2023-11-25T04:24:40.147 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.147 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.148 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:40.148 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.168228+0000", 2023-11-25T04:24:40.148 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0089992930000000002 2023-11-25T04:24:40.148 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.148 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.148 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:40.148 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.168310+0000", 2023-11-25T04:24:40.148 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1348999999999997e-05 2023-11-25T04:24:40.148 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.149 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.149 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:40.149 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.168340+0000", 2023-11-25T04:24:40.149 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0735000000000003e-05 2023-11-25T04:24:40.149 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:40.149 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:40.149 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:40.149 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.150 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.150 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:1105 4.23 4:c6ff07b4:::10000001220.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:40.150 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:22.158714+0000", 2023-11-25T04:24:40.150 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 17.822598482, 2023-11-25T04:24:40.150 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010412490999999999, 2023-11-25T04:24:40.150 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:40.150 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:40.150 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:40.151 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.151 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:40.151 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158714+0000", 2023-11-25T04:24:40.151 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:40.151 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.151 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.151 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:40.151 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158714+0000", 2023-11-25T04:24:40.151 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:40.152 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.152 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.152 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:40.152 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158715+0000", 2023-11-25T04:24:40.152 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4099999999999998e-07 2023-11-25T04:24:40.152 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.152 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.152 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:40.152 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158718+0000", 2023-11-25T04:24:40.153 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4319999999999999e-06 2023-11-25T04:24:40.153 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.153 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.153 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:40.153 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158729+0000", 2023-11-25T04:24:40.153 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.078e-05 2023-11-25T04:24:40.153 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.153 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.154 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:40.154 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158736+0000", 2023-11-25T04:24:40.154 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4649999999999997e-06 2023-11-25T04:24:40.154 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.154 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.154 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:40.154 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158779+0000", 2023-11-25T04:24:40.154 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3049000000000001e-05 2023-11-25T04:24:40.155 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.155 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.155 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:40.155 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158865+0000", 2023-11-25T04:24:40.155 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5138999999999999e-05 2023-11-25T04:24:40.155 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.155 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.155 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:40.156 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158983+0000", 2023-11-25T04:24:40.156 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000118658 2023-11-25T04:24:40.156 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.156 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.156 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:40.156 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.159645+0000", 2023-11-25T04:24:40.156 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00066193999999999997 2023-11-25T04:24:40.157 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.157 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.157 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:40.157 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.169046+0000", 2023-11-25T04:24:40.157 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0094005400000000006 2023-11-25T04:24:40.157 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.157 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.157 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:40.157 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.169092+0000", 2023-11-25T04:24:40.158 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5840000000000002e-05 2023-11-25T04:24:40.158 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.158 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.158 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:40.158 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.169127+0000", 2023-11-25T04:24:40.158 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5006999999999999e-05 2023-11-25T04:24:40.158 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:40.158 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:40.159 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:40.159 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.159 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.159 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2841 4.4 4:22111d42:::100000010df.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:40.159 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.092102+0000", 2023-11-25T04:24:40.159 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 14.889211007, 2023-11-25T04:24:40.159 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.016449783999999999, 2023-11-25T04:24:40.159 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:40.159 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:40.160 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:40.160 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.160 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:40.160 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092102+0000", 2023-11-25T04:24:40.160 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:40.160 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.160 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.160 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:40.161 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092102+0000", 2023-11-25T04:24:40.161 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:40.161 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.161 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.161 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:40.161 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092102+0000", 2023-11-25T04:24:40.161 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3700000000000001e-07 2023-11-25T04:24:40.161 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.161 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.162 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:40.162 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092104+0000", 2023-11-25T04:24:40.194 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0679999999999999e-06 2023-11-25T04:24:40.194 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.194 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.194 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:40.195 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092111+0000", 2023-11-25T04:24:40.195 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0589999999999997e-06 2023-11-25T04:24:40.195 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.195 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.195 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:40.195 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092130+0000", 2023-11-25T04:24:40.195 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8686999999999999e-05 2023-11-25T04:24:40.195 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.196 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.196 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:40.196 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092443+0000", 2023-11-25T04:24:40.196 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000313519 2023-11-25T04:24:40.196 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.196 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.196 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:40.197 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092553+0000", 2023-11-25T04:24:40.197 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010932100000000001 2023-11-25T04:24:40.197 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.197 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.197 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:40.197 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092639+0000", 2023-11-25T04:24:40.197 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6094999999999995e-05 2023-11-25T04:24:40.197 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.198 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.198 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:40.198 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094679+0000", 2023-11-25T04:24:40.198 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0020402580000000001 2023-11-25T04:24:40.198 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.198 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.198 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:40.198 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108462+0000", 2023-11-25T04:24:40.199 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013783358000000001 2023-11-25T04:24:40.199 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.199 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.199 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:40.199 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108516+0000", 2023-11-25T04:24:40.199 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3483000000000003e-05 2023-11-25T04:24:40.199 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.199 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.200 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:40.200 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108551+0000", 2023-11-25T04:24:40.200 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5499e-05 2023-11-25T04:24:40.200 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:40.200 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:40.200 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:40.200 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.200 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.201 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2852 4.2d 4:b745ac76:::100000010f5.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:40.201 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.093477+0000", 2023-11-25T04:24:40.201 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 14.887835895, 2023-11-25T04:24:40.201 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.015423084, 2023-11-25T04:24:40.201 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:40.201 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:40.201 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:40.201 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.201 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:40.202 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093477+0000", 2023-11-25T04:24:40.202 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:40.202 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.202 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.202 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:40.202 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093477+0000", 2023-11-25T04:24:40.202 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:40.202 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.203 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.203 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:40.203 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093477+0000", 2023-11-25T04:24:40.203 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2500000000000005e-07 2023-11-25T04:24:40.203 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.203 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.203 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:40.203 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093480+0000", 2023-11-25T04:24:40.204 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0189999999999998e-06 2023-11-25T04:24:40.204 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.204 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.204 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:40.204 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093642+0000", 2023-11-25T04:24:40.204 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000162072 2023-11-25T04:24:40.204 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.204 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.205 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:40.205 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093649+0000", 2023-11-25T04:24:40.206 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9659999999999999e-06 2023-11-25T04:24:40.206 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.206 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.206 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:40.206 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093693+0000", 2023-11-25T04:24:40.206 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3794000000000001e-05 2023-11-25T04:24:40.206 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.206 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.207 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:40.207 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093764+0000", 2023-11-25T04:24:40.207 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0981000000000004e-05 2023-11-25T04:24:40.207 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.207 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.207 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:40.207 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093850+0000", 2023-11-25T04:24:40.207 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5536000000000002e-05 2023-11-25T04:24:40.207 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.208 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.208 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:40.208 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094710+0000", 2023-11-25T04:24:40.208 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00086070299999999999 2023-11-25T04:24:40.208 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.208 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.208 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:40.208 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108776+0000", 2023-11-25T04:24:40.208 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01406542 2023-11-25T04:24:40.209 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.209 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.209 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:40.209 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108829+0000", 2023-11-25T04:24:40.209 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2936999999999998e-05 2023-11-25T04:24:40.209 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.209 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.209 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:40.210 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108900+0000", 2023-11-25T04:24:40.210 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1031000000000005e-05 2023-11-25T04:24:40.210 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:40.210 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:40.210 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:40.210 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.210 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.210 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2854 4.2d 4:b5c259cf:::100000010f9.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:40.211 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.093674+0000", 2023-11-25T04:24:40.211 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 14.887638235000001, 2023-11-25T04:24:40.211 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.015970380999999999, 2023-11-25T04:24:40.211 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:40.212 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:40.212 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:40.212 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.212 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:40.212 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093674+0000", 2023-11-25T04:24:40.213 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:40.213 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.213 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.213 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:40.213 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093674+0000", 2023-11-25T04:24:40.214 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:40.214 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.214 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.214 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:40.214 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093675+0000", 2023-11-25T04:24:40.214 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8400000000000004e-07 2023-11-25T04:24:40.214 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.215 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.215 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:40.215 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093678+0000", 2023-11-25T04:24:40.215 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2229999999999999e-06 2023-11-25T04:24:40.215 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.216 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.216 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:40.216 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093686+0000", 2023-11-25T04:24:40.216 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9990000000000008e-06 2023-11-25T04:24:40.216 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.216 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.216 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:40.216 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093696+0000", 2023-11-25T04:24:40.217 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6479999999999998e-06 2023-11-25T04:24:40.217 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.217 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.217 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:40.217 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094104+0000", 2023-11-25T04:24:40.217 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00040823900000000002 2023-11-25T04:24:40.217 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.217 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.218 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:40.218 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094172+0000", 2023-11-25T04:24:40.218 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8023999999999998e-05 2023-11-25T04:24:40.218 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.218 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.218 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:40.219 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094256+0000", 2023-11-25T04:24:40.219 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3418999999999995e-05 2023-11-25T04:24:40.219 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.219 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.219 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:40.248 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095160+0000", 2023-11-25T04:24:40.249 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00090418599999999997 2023-11-25T04:24:40.249 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.249 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.249 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:40.249 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109551+0000", 2023-11-25T04:24:40.249 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014391571000000001 2023-11-25T04:24:40.249 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.249 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.250 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:40.250 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109601+0000", 2023-11-25T04:24:40.250 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9478000000000003e-05 2023-11-25T04:24:40.250 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.250 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.250 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:40.250 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109645+0000", 2023-11-25T04:24:40.250 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.401e-05 2023-11-25T04:24:40.251 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:40.251 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:40.251 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:40.251 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.251 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.251 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2861 4.4 4:214027a9:::10000001107.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:40.251 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.093829+0000", 2023-11-25T04:24:40.252 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 14.887483817, 2023-11-25T04:24:40.252 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014848917999999999, 2023-11-25T04:24:40.252 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:40.252 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:40.252 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:40.252 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.252 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:40.253 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093829+0000", 2023-11-25T04:24:40.253 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:40.253 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.253 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.253 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:40.253 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093829+0000", 2023-11-25T04:24:40.253 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:40.253 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.254 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.254 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:40.254 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093830+0000", 2023-11-25T04:24:40.254 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1999999999999998e-07 2023-11-25T04:24:40.254 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.254 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.255 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:40.256 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093834+0000", 2023-11-25T04:24:40.256 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.34e-06 2023-11-25T04:24:40.257 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.257 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.257 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:40.257 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093844+0000", 2023-11-25T04:24:40.257 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6690000000000001e-06 2023-11-25T04:24:40.257 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.258 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.258 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:40.258 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093857+0000", 2023-11-25T04:24:40.258 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3434999999999999e-05 2023-11-25T04:24:40.258 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.258 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.258 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:40.258 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093940+0000", 2023-11-25T04:24:40.259 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3047999999999995e-05 2023-11-25T04:24:40.259 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.259 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.259 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:40.259 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094052+0000", 2023-11-25T04:24:40.259 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000111538 2023-11-25T04:24:40.259 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.318 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.318 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:40.318 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094466+0000", 2023-11-25T04:24:40.318 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00041464600000000001 2023-11-25T04:24:40.318 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.318 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.319 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:40.319 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095570+0000", 2023-11-25T04:24:40.319 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001103981 2023-11-25T04:24:40.319 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.319 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.319 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:40.319 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108598+0000", 2023-11-25T04:24:40.319 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013027164000000001 2023-11-25T04:24:40.319 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.320 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.320 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:40.320 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108646+0000", 2023-11-25T04:24:40.320 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8090999999999997e-05 2023-11-25T04:24:40.320 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.320 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.320 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:40.320 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108678+0000", 2023-11-25T04:24:40.321 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2085999999999997e-05 2023-11-25T04:24:40.321 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:40.321 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:40.321 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:40.321 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.321 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.321 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2863 4.12 4:4b84c733:::1000000110b.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:40.321 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.093877+0000", 2023-11-25T04:24:40.321 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 14.88743549, 2023-11-25T04:24:40.322 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014682755, 2023-11-25T04:24:40.322 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:40.322 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:40.322 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:40.322 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.322 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:40.322 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093877+0000", 2023-11-25T04:24:40.322 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:40.323 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.323 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.323 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:40.323 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093877+0000", 2023-11-25T04:24:40.323 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:40.323 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.323 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.323 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:40.323 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093878+0000", 2023-11-25T04:24:40.324 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4199999999999996e-07 2023-11-25T04:24:40.324 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.324 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.324 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:40.324 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093880+0000", 2023-11-25T04:24:40.324 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6189999999999998e-06 2023-11-25T04:24:40.324 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.324 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.325 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:40.325 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093888+0000", 2023-11-25T04:24:40.325 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5510000000000002e-06 2023-11-25T04:24:40.325 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.325 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.325 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:40.325 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093893+0000", 2023-11-25T04:24:40.325 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4179999999999996e-06 2023-11-25T04:24:40.325 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.326 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.351 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:40.351 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093934+0000", 2023-11-25T04:24:40.351 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0648000000000001e-05 2023-11-25T04:24:40.351 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.351 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.351 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:40.352 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094081+0000", 2023-11-25T04:24:40.352 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014753499999999999 2023-11-25T04:24:40.352 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.352 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.352 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:40.352 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094172+0000", 2023-11-25T04:24:40.352 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.0722000000000003e-05 2023-11-25T04:24:40.352 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.352 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.353 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:40.353 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094742+0000", 2023-11-25T04:24:40.353 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00056948600000000002 2023-11-25T04:24:40.353 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.353 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.353 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:40.353 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108491+0000", 2023-11-25T04:24:40.353 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013749568 2023-11-25T04:24:40.354 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.354 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.354 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:40.354 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108525+0000", 2023-11-25T04:24:40.354 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3683999999999998e-05 2023-11-25T04:24:40.354 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.354 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.355 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:40.355 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108560+0000", 2023-11-25T04:24:40.355 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4981999999999998e-05 2023-11-25T04:24:40.355 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:40.355 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:40.355 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:40.355 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.355 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.356 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2880 4.21 4:855834a4:::1000000112d.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:40.356 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.095098+0000", 2023-11-25T04:24:40.356 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 14.886214796000001, 2023-11-25T04:24:40.356 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013847544, 2023-11-25T04:24:40.356 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:40.356 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:40.356 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:40.356 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.356 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:40.357 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095098+0000", 2023-11-25T04:24:40.357 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:40.357 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.357 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.357 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:40.358 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095098+0000", 2023-11-25T04:24:40.358 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:40.358 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.358 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.358 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:40.358 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095098+0000", 2023-11-25T04:24:40.358 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3000000000000002e-07 2023-11-25T04:24:40.358 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.359 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.359 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:40.359 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095099+0000", 2023-11-25T04:24:40.359 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.331e-06 2023-11-25T04:24:40.359 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.359 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.359 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:40.359 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095103+0000", 2023-11-25T04:24:40.360 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8319999999999999e-06 2023-11-25T04:24:40.360 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.360 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.360 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:40.360 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095108+0000", 2023-11-25T04:24:40.360 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4179999999999999e-06 2023-11-25T04:24:40.360 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.360 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.360 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:40.361 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095153+0000", 2023-11-25T04:24:40.361 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4799000000000003e-05 2023-11-25T04:24:40.361 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.361 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.392 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:40.392 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095268+0000", 2023-11-25T04:24:40.392 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000115437 2023-11-25T04:24:40.392 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.392 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.393 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:40.393 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095363+0000", 2023-11-25T04:24:40.393 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.5278000000000005e-05 2023-11-25T04:24:40.393 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.393 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.393 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:40.393 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095845+0000", 2023-11-25T04:24:40.393 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00048164600000000002 2023-11-25T04:24:40.394 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.394 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.394 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:40.394 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108854+0000", 2023-11-25T04:24:40.394 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013009289 2023-11-25T04:24:40.394 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.394 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.395 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:40.395 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108907+0000", 2023-11-25T04:24:40.395 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2528999999999998e-05 2023-11-25T04:24:40.395 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.395 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.395 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:40.395 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108945+0000", 2023-11-25T04:24:40.395 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8655000000000002e-05 2023-11-25T04:24:40.396 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:40.396 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:40.396 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:40.396 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.396 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.396 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2884 4.37 4:ef909cdc:::10000001135.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:40.396 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.095518+0000", 2023-11-25T04:24:40.397 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 14.885794869, 2023-11-25T04:24:40.397 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014149988, 2023-11-25T04:24:40.397 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:40.397 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:40.397 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:40.397 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.397 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:40.398 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095518+0000", 2023-11-25T04:24:40.398 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:40.398 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.398 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.398 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:40.398 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095518+0000", 2023-11-25T04:24:40.398 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:40.399 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.399 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.399 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:40.399 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095518+0000", 2023-11-25T04:24:40.399 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2900000000000003e-07 2023-11-25T04:24:40.400 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.400 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.400 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:40.400 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095522+0000", 2023-11-25T04:24:40.400 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2119999999999999e-06 2023-11-25T04:24:40.400 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.400 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.400 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:40.400 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095532+0000", 2023-11-25T04:24:40.401 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0375e-05 2023-11-25T04:24:40.401 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.401 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.401 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:40.401 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095537+0000", 2023-11-25T04:24:40.401 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5909999999999998e-06 2023-11-25T04:24:40.401 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.402 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.402 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:40.402 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095584+0000", 2023-11-25T04:24:40.404 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7169999999999997e-05 2023-11-25T04:24:40.432 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.433 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.433 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:40.433 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095695+0000", 2023-11-25T04:24:40.433 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011078699999999999 2023-11-25T04:24:40.433 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.433 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.433 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:40.433 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095777+0000", 2023-11-25T04:24:40.434 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2089999999999995e-05 2023-11-25T04:24:40.434 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.434 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.434 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:40.434 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096286+0000", 2023-11-25T04:24:40.434 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000509247 2023-11-25T04:24:40.434 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.434 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.434 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:40.435 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109596+0000", 2023-11-25T04:24:40.435 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013309872 2023-11-25T04:24:40.435 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.435 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.435 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:40.435 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109638+0000", 2023-11-25T04:24:40.435 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2381e-05 2023-11-25T04:24:40.435 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.436 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.436 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:40.436 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109668+0000", 2023-11-25T04:24:40.436 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9634000000000001e-05 2023-11-25T04:24:40.436 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:40.436 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:40.436 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:40.436 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.436 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.437 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2890 4.12 4:4a2ad640:::10000001141.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:40.437 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.096274+0000", 2023-11-25T04:24:40.437 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 14.885038806000001, 2023-11-25T04:24:40.437 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013647259, 2023-11-25T04:24:40.437 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:40.437 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:40.437 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:40.437 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.438 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:40.438 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096274+0000", 2023-11-25T04:24:40.438 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:40.438 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.438 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.438 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:40.438 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096274+0000", 2023-11-25T04:24:40.438 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:40.438 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.439 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.439 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:40.439 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096275+0000", 2023-11-25T04:24:40.439 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0299999999999998e-07 2023-11-25T04:24:40.439 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.439 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.439 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:40.439 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096279+0000", 2023-11-25T04:24:40.439 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9580000000000001e-06 2023-11-25T04:24:40.440 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.440 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.440 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:40.440 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096287+0000", 2023-11-25T04:24:40.440 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6929999999999996e-06 2023-11-25T04:24:40.440 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.440 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.440 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:40.441 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096294+0000", 2023-11-25T04:24:40.441 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.348e-06 2023-11-25T04:24:40.441 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.441 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.441 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:40.441 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096342+0000", 2023-11-25T04:24:40.441 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8068000000000001e-05 2023-11-25T04:24:40.441 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.441 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.442 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:40.442 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096462+0000", 2023-11-25T04:24:40.442 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000119918 2023-11-25T04:24:40.442 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.442 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.442 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:40.442 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096557+0000", 2023-11-25T04:24:40.442 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.5340999999999994e-05 2023-11-25T04:24:40.442 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.443 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.443 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:40.443 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096971+0000", 2023-11-25T04:24:40.443 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00041398100000000001 2023-11-25T04:24:40.443 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.443 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.443 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:40.443 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109849+0000", 2023-11-25T04:24:40.444 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.012877985999999999 2023-11-25T04:24:40.444 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.444 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.444 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:40.444 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109894+0000", 2023-11-25T04:24:40.444 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5595e-05 2023-11-25T04:24:40.444 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.444 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.444 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:40.445 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109921+0000", 2023-11-25T04:24:40.445 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6568e-05 2023-11-25T04:24:40.445 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:40.445 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:40.445 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:40.445 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.445 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.445 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2891 4.23 4:c4054dd3:::10000001143.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:40.445 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.096713+0000", 2023-11-25T04:24:40.446 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 14.884600111999999, 2023-11-25T04:24:40.446 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013148692, 2023-11-25T04:24:40.446 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:40.446 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:40.446 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:40.446 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.446 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:40.446 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096713+0000", 2023-11-25T04:24:40.447 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:40.447 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.447 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.447 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:40.447 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096713+0000", 2023-11-25T04:24:40.447 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:40.447 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.447 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.447 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:40.448 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096713+0000", 2023-11-25T04:24:40.448 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9299999999999998e-07 2023-11-25T04:24:40.448 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.448 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.448 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:40.448 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096716+0000", 2023-11-25T04:24:40.448 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.988e-06 2023-11-25T04:24:40.449 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.449 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.449 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:40.449 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096722+0000", 2023-11-25T04:24:40.449 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0499999999999997e-06 2023-11-25T04:24:40.449 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.449 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.449 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:40.449 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096727+0000", 2023-11-25T04:24:40.450 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3380000000000004e-06 2023-11-25T04:24:40.450 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.450 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.450 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:40.450 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096770+0000", 2023-11-25T04:24:40.450 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2938000000000001e-05 2023-11-25T04:24:40.450 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.450 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.451 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:40.451 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096904+0000", 2023-11-25T04:24:40.451 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000133759 2023-11-25T04:24:40.451 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.451 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.451 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:40.451 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096983+0000", 2023-11-25T04:24:40.451 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8559999999999993e-05 2023-11-25T04:24:40.451 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.452 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.452 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:40.452 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.097437+0000", 2023-11-25T04:24:40.452 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00045439200000000002 2023-11-25T04:24:40.452 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.452 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.452 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:40.452 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109737+0000", 2023-11-25T04:24:40.452 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.012300190000000001 2023-11-25T04:24:40.453 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.453 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.453 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:40.453 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109782+0000", 2023-11-25T04:24:40.453 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4546999999999999e-05 2023-11-25T04:24:40.453 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:40.453 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:40.453 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:40.454 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109861+0000", 2023-11-25T04:24:40.454 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9337000000000003e-05 2023-11-25T04:24:40.454 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:40.454 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:40.454 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:40.454 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:40.454 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:40.454 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:40.455 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-11-25T04:24:40.455 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2023-11-25T04:24:40.457 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:24:40.458 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":52,"fsid":"a677f511-1143-4e76-8da6-5ddfadaff077","created":"2023-11-25T04:22:25.140474+0000","modified":"2023-11-25T04:24:38.193878+0000","last_up_change":"2023-11-25T04:23:08.103191+0000","last_in_change":"2023-11-25T04:22:25.985372+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-25T04:22:35.548359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":3.0299999713897705,"score_stable":3.0299999713897705,"optimal_score":0.33000001311302185,"raw_score_acting":1,"raw_score_stable":1,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-25T04:22:40.218527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.8700000047683716,"score_stable":1.8700000047683716,"optimal_score":0.67000001668930054,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-25T04:22:44.844906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.7799999713897705,"score_stable":1.7799999713897705,"optimal_score":0.67000001668930054,"raw_score_acting":1.190000057220459,"raw_score_stable":1.190000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-25T04:22:45.365536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.4900000095367432,"score_stable":1.4900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":1,"raw_score_stable":1,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bc397d35-53ea-430c-a96a-63a4160e64c3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":17,"down_at":21,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":181847435},{"type":"v1","addr":"172.21.15.76:6801","nonce":181847435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":181847435},{"type":"v1","addr":"172.21.15.76:6803","nonce":181847435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":181847435},{"type":"v1","addr":"172.21.15.76:6807","nonce":181847435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":181847435},{"type":"v1","addr":"172.21.15.76:6805","nonce":181847435}]},"public_addr":"172.21.15.76:6801/181847435","cluster_addr":"172.21.15.76:6803/181847435","heartbeat_back_addr":"172.21.15.76:6807/181847435","heartbeat_front_addr":"172.21.15.76:6805/181847435","state":["exists"]},{"osd":1,"uuid":"d4f1b88a-bbd4-4e58-9519-569489ced7b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6801","nonce":3130474108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6803","nonce":3130474108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6807","nonce":3130474108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6805","nonce":3130474108}]},"public_addr":"172.21.15.92:6801/3130474108","cluster_addr":"172.21.15.92:6803/3130474108","heartbeat_back_addr":"172.21.15.92:6807/3130474108","heartbeat_front_addr":"172.21.15.92:6805/3130474108","state":["exists","up"]},{"osd":2,"uuid":"55c82801-e090-4c10-91e5-5fa9dbfba36d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6801","nonce":4099978417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6803","nonce":4099978417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6807","nonce":4099978417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6805","nonce":4099978417}]},"public_addr":"172.21.15.195:6801/4099978417","cluster_addr":"172.21.15.195:6803/4099978417","heartbeat_back_addr":"172.21.15.195:6807/4099978417","heartbeat_front_addr":"172.21.15.195:6805/4099978417","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-25T04:22:31.039867+0000","dead_epoch":20},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-25T04:22:31.021863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-25T04:22:31.009821+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-25T04:24:40.458 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:40.454+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:40.459 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-25T04:22:35.548359+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 3.0299999713897705, 'score_stable': 3.0299999713897705, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 1, 'raw_score_stable': 1, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-25T04:22:40.218527+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.8700000047683716, 'score_stable': 1.8700000047683716, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-25T04:22:44.844906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.7799999713897705, 'score_stable': 1.7799999713897705, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 1.190000057220459, 'raw_score_stable': 1.190000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-25T04:22:45.365536+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.4900000095367432, 'score_stable': 1.4900000095367432, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 1, 'raw_score_stable': 1, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-25T04:24:40.459 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-25T04:24:40.460 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:24:40.556 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:40.554+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:40.573 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:40.573 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:24:40.573 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2023-11-25T04:24:40.574 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:40.582 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:24:40.615 INFO:teuthology.orchestra.run.smithi067.stderr:Error ENXIO: problem getting command descriptions from osd.0 2023-11-25T04:24:40.634 INFO:teuthology.orchestra.run.smithi067.stdout:osd.1: {} 2023-11-25T04:24:40.634 INFO:teuthology.orchestra.run.smithi067.stderr:osd.1: osd_enable_op_tracker = '' 2023-11-25T04:24:40.653 INFO:teuthology.orchestra.run.smithi067.stdout:osd.2: {} 2023-11-25T04:24:40.654 INFO:teuthology.orchestra.run.smithi067.stderr:osd.2: osd_enable_op_tracker = '' 2023-11-25T04:24:40.657 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-11-25T04:24:40.657 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-11-25T04:24:40.658 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:40.656+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:40.727 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:40.727 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:40.735 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:24:40.758 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:40.756+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:40.840 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:40.840 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:40.850 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:24:40.866 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2023-11-25T04:24:40.879 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-25T04:24:40.895 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:40.893+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:40.963 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:40.962+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:40.965 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:40.965 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:40.974 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:24:41.065 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:41.063+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:41.120 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:41.121 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:41.129 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:24:41.173 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:41.172+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:41.174 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 8 2023-11-25T04:24:41.187 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-25T04:24:41.212 INFO:teuthology.orchestra.run.smithi067.stderr:noscrub is set 2023-11-25T04:24:41.237 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:41.237 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:41.247 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:24:41.290 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:41.288+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:41.355 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:41.355 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:41.364 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:24:41.368 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:41.367+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:41.469 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:41.467+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:41.488 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2023-11-25T04:24:41.501 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-25T04:24:41.510 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:41.510 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:41.519 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:24:41.570 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:41.569+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:41.664 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:41.665 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:41.671 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:41.669+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:41.673 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:24:41.772 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:41.771+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:41.780 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:41.781 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:41.790 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:24:41.795 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2023-11-25T04:24:41.808 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-25T04:24:41.875 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:41.873+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:41.947 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:41.947 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:41.956 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:24:41.975 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:41.973+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:42.065 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:42.066 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:42.075 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:24:42.076 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:42.074+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:42.120 INFO:teuthology.orchestra.run.smithi067.stderr:listed 0 entries 2023-11-25T04:24:42.137 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2023-11-25T04:24:42.137 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-25T04:24:42.153 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-25T04:24:42.176 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:42.174+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:42.182 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:42.183 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:42.191 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:24:42.278 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:42.276+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:42.336 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:42.336 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:42.344 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:24:42.385 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:42.383+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:42.483 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:42.481+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:42.489 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:42.489 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:42.490 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:24:42.491 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":53,"fsid":"a677f511-1143-4e76-8da6-5ddfadaff077","created":"2023-11-25T04:22:25.140474+0000","modified":"2023-11-25T04:24:41.203657+0000","last_up_change":"2023-11-25T04:23:08.103191+0000","last_in_change":"2023-11-25T04:22:25.985372+0000","flags":"noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5801984,"flags_set":["noscrub","pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-25T04:22:35.548359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":3.0299999713897705,"score_stable":3.0299999713897705,"optimal_score":0.33000001311302185,"raw_score_acting":1,"raw_score_stable":1,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-25T04:22:40.218527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.8700000047683716,"score_stable":1.8700000047683716,"optimal_score":0.67000001668930054,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-25T04:22:44.844906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.7799999713897705,"score_stable":1.7799999713897705,"optimal_score":0.67000001668930054,"raw_score_acting":1.190000057220459,"raw_score_stable":1.190000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-25T04:22:45.365536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.4900000095367432,"score_stable":1.4900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":1,"raw_score_stable":1,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bc397d35-53ea-430c-a96a-63a4160e64c3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":17,"down_at":21,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":181847435},{"type":"v1","addr":"172.21.15.76:6801","nonce":181847435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":181847435},{"type":"v1","addr":"172.21.15.76:6803","nonce":181847435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":181847435},{"type":"v1","addr":"172.21.15.76:6807","nonce":181847435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":181847435},{"type":"v1","addr":"172.21.15.76:6805","nonce":181847435}]},"public_addr":"172.21.15.76:6801/181847435","cluster_addr":"172.21.15.76:6803/181847435","heartbeat_back_addr":"172.21.15.76:6807/181847435","heartbeat_front_addr":"172.21.15.76:6805/181847435","state":["exists"]},{"osd":1,"uuid":"d4f1b88a-bbd4-4e58-9519-569489ced7b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6801","nonce":3130474108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6803","nonce":3130474108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6807","nonce":3130474108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6805","nonce":3130474108}]},"public_addr":"172.21.15.92:6801/3130474108","cluster_addr":"172.21.15.92:6803/3130474108","heartbeat_back_addr":"172.21.15.92:6807/3130474108","heartbeat_front_addr":"172.21.15.92:6805/3130474108","state":["exists","up"]},{"osd":2,"uuid":"55c82801-e090-4c10-91e5-5fa9dbfba36d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6801","nonce":4099978417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6803","nonce":4099978417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6807","nonce":4099978417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6805","nonce":4099978417}]},"public_addr":"172.21.15.195:6801/4099978417","cluster_addr":"172.21.15.195:6803/4099978417","heartbeat_back_addr":"172.21.15.195:6807/4099978417","heartbeat_front_addr":"172.21.15.195:6805/4099978417","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-25T04:22:31.039867+0000","dead_epoch":20},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-25T04:22:31.021863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-25T04:22:31.009821+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-25T04:24:42.498 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:24:42.506 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-25T04:22:35.548359+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 3.0299999713897705, 'score_stable': 3.0299999713897705, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 1, 'raw_score_stable': 1, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-25T04:22:40.218527+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.8700000047683716, 'score_stable': 1.8700000047683716, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-25T04:22:44.844906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.7799999713897705, 'score_stable': 1.7799999713897705, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 1.190000057220459, 'raw_score_stable': 1.190000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-25T04:22:45.365536+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.4900000095367432, 'score_stable': 1.4900000095367432, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 1, 'raw_score_stable': 1, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-25T04:24:42.507 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-25T04:24:42.584 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:42.582+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:42.605 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:42.606 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:42.614 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:24:42.658 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-11-25T04:24:42.685 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:42.683+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:42.720 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:42.721 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:42.729 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:24:42.742 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~94s 2023-11-25T04:24:42.771 INFO:teuthology.orchestra.run.smithi067.stderr:Error ENXIO: problem getting command descriptions from osd.0 2023-11-25T04:24:42.786 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:42.784+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:42.789 INFO:teuthology.orchestra.run.smithi067.stdout:osd.1: {} 2023-11-25T04:24:42.789 INFO:teuthology.orchestra.run.smithi067.stderr:osd.1: osd_enable_op_tracker = '' 2023-11-25T04:24:42.806 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2023-11-25T04:24:42.807 INFO:teuthology.orchestra.run.smithi067.stdout:osd.2: {} 2023-11-25T04:24:42.807 INFO:teuthology.orchestra.run.smithi067.stderr:osd.2: osd_enable_op_tracker = '' 2023-11-25T04:24:42.811 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-11-25T04:24:42.811 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-11-25T04:24:42.820 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-25T04:24:42.878 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:42.878 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2023-11-25T04:24:42.878 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2023-11-25T04:24:42.878 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2023-11-25T04:24:42.878 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.879 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:42.879 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:24:42.879 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 115.84374063999999, 2023-11-25T04:24:42.879 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01072943, 2023-11-25T04:24:42.879 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:42.879 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:42.879 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:42.880 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.880 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:42.880 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:24:42.880 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:42.880 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.880 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.880 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:42.880 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:24:42.881 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:42.881 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.881 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.881 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:42.881 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031829+0000", 2023-11-25T04:24:42.881 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.666e-06 2023-11-25T04:24:42.881 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.882 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.882 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:42.882 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031839+0000", 2023-11-25T04:24:42.882 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9490000000000008e-06 2023-11-25T04:24:42.882 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.882 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.883 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:42.883 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031857+0000", 2023-11-25T04:24:42.883 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8093000000000001e-05 2023-11-25T04:24:42.883 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.883 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.883 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:42.884 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031882+0000", 2023-11-25T04:24:42.884 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4294000000000002e-05 2023-11-25T04:24:42.884 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.884 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.884 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:42.884 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033130+0000", 2023-11-25T04:24:42.884 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001248264 2023-11-25T04:24:42.885 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.885 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.885 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:42.885 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033426+0000", 2023-11-25T04:24:42.885 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029620999999999998 2023-11-25T04:24:42.885 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.885 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.885 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:42.886 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033814+0000", 2023-11-25T04:24:42.886 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000387742 2023-11-25T04:24:42.886 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.886 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.886 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:42.886 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036793+0000", 2023-11-25T04:24:42.886 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002979598 2023-11-25T04:24:42.886 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.887 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.887 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:42.887 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042455+0000", 2023-11-25T04:24:42.887 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0056619260000000003 2023-11-25T04:24:42.887 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.887 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.887 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:42.887 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042502+0000", 2023-11-25T04:24:42.888 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7139999999999999e-05 2023-11-25T04:24:42.888 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.888 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.888 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:42.888 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042557+0000", 2023-11-25T04:24:42.888 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4548e-05 2023-11-25T04:24:42.888 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:42.888 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:42.889 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:42.889 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.889 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.889 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:42.889 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:24:42.889 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 115.84254210500001, 2023-11-25T04:24:42.889 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011505202000000001, 2023-11-25T04:24:42.890 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:42.890 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:42.890 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:42.890 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.890 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:42.890 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:24:42.890 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:42.890 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.891 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.891 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:42.891 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:24:42.891 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:42.891 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.891 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.892 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:42.892 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033027+0000", 2023-11-25T04:24:42.893 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0699999999999999e-06 2023-11-25T04:24:42.893 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.894 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.894 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:42.895 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033035+0000", 2023-11-25T04:24:42.895 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0830000000000004e-06 2023-11-25T04:24:42.895 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.895 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.895 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:42.895 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033059+0000", 2023-11-25T04:24:42.895 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3975e-05 2023-11-25T04:24:42.896 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.896 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.896 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:42.896 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033482+0000", 2023-11-25T04:24:42.896 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042253700000000001 2023-11-25T04:24:42.896 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.896 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.897 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:42.897 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034520+0000", 2023-11-25T04:24:42.897 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001037991 2023-11-25T04:24:42.897 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.897 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.897 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:42.897 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034611+0000", 2023-11-25T04:24:42.897 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1113999999999995e-05 2023-11-25T04:24:42.898 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.898 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.898 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:42.898 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034769+0000", 2023-11-25T04:24:42.898 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000157641 2023-11-25T04:24:42.898 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.899 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.899 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:42.899 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038292+0000", 2023-11-25T04:24:42.899 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035231160000000002 2023-11-25T04:24:42.899 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.899 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.899 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:42.900 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044440+0000", 2023-11-25T04:24:42.900 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0061486980000000002 2023-11-25T04:24:42.900 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.900 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.900 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:42.900 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044495+0000", 2023-11-25T04:24:42.900 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4339000000000003e-05 2023-11-25T04:24:42.901 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.901 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.901 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:42.901 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044531+0000", 2023-11-25T04:24:42.901 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6637999999999997e-05 2023-11-25T04:24:42.901 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:42.901 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:42.901 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:42.902 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.902 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.902 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:26 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:42.902 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:42.902 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 115.841953343, 2023-11-25T04:24:42.902 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011086945000000001, 2023-11-25T04:24:42.902 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:42.903 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:42.903 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:42.903 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.903 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:42.903 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:42.903 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:42.903 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.903 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.904 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:42.904 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:42.904 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:42.904 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.904 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.905 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:42.905 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:42.905 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1699999999999998e-07 2023-11-25T04:24:42.905 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.905 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.905 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:42.905 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033627+0000", 2023-11-25T04:24:42.906 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.163e-05 2023-11-25T04:24:42.906 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.906 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.906 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:42.906 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033636+0000", 2023-11-25T04:24:42.906 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6859999999999995e-06 2023-11-25T04:24:42.906 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.907 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.907 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:42.907 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033642+0000", 2023-11-25T04:24:42.907 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3149999999999997e-06 2023-11-25T04:24:42.907 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.907 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.907 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:42.907 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035414+0000", 2023-11-25T04:24:42.908 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0017716380000000001 2023-11-25T04:24:42.908 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.908 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.908 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:42.908 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035539+0000", 2023-11-25T04:24:42.908 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000125477 2023-11-25T04:24:42.908 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.908 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.909 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:42.909 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035672+0000", 2023-11-25T04:24:42.909 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000133316 2023-11-25T04:24:42.909 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.909 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.909 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:42.909 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038649+0000", 2023-11-25T04:24:42.909 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029764679999999999 2023-11-25T04:24:42.910 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.910 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.910 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:42.910 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044578+0000", 2023-11-25T04:24:42.910 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0059293510000000002 2023-11-25T04:24:42.910 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.910 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.910 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:42.911 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044651+0000", 2023-11-25T04:24:42.911 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2513000000000002e-05 2023-11-25T04:24:42.911 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.911 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.911 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:42.911 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044702+0000", 2023-11-25T04:24:42.911 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1034e-05 2023-11-25T04:24:42.911 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:42.912 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:42.912 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:42.912 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.912 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.912 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:29 3.b 3:d0630e4c:::mds0_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:42.912 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:42.913 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 115.841915423, 2023-11-25T04:24:42.913 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010980227, 2023-11-25T04:24:42.913 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:42.913 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:42.914 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:42.914 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.914 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:42.914 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:42.914 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:42.915 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.915 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.915 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:42.915 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:42.915 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:42.915 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.915 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.915 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:42.916 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:42.916 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3499999999999996e-07 2023-11-25T04:24:42.916 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.916 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.916 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:42.916 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033658+0000", 2023-11-25T04:24:42.916 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1280000000000001e-06 2023-11-25T04:24:42.916 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.917 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.917 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:42.917 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033667+0000", 2023-11-25T04:24:42.917 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1430000000000004e-06 2023-11-25T04:24:42.917 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.917 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.917 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:42.917 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033678+0000", 2023-11-25T04:24:42.918 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0906e-05 2023-11-25T04:24:42.918 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.918 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.918 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:42.918 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034456+0000", 2023-11-25T04:24:42.918 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00077833899999999996 2023-11-25T04:24:42.918 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.919 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.919 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:42.919 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034575+0000", 2023-11-25T04:24:42.919 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011926000000000001 2023-11-25T04:24:42.919 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.919 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.919 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:42.919 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035001+0000", 2023-11-25T04:24:42.920 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042550999999999998 2023-11-25T04:24:42.920 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.920 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.920 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:42.920 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.037923+0000", 2023-11-25T04:24:42.920 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029219670000000001 2023-11-25T04:24:42.920 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.920 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.920 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:42.921 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044538+0000", 2023-11-25T04:24:42.921 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0066152349999999997 2023-11-25T04:24:42.921 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.921 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.921 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:42.921 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044594+0000", 2023-11-25T04:24:42.921 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5593000000000001e-05 2023-11-25T04:24:42.922 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.922 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.922 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:42.922 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044633+0000", 2023-11-25T04:24:42.922 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9511000000000002e-05 2023-11-25T04:24:42.923 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:42.923 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:42.923 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:42.923 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.923 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.923 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:34 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:42.923 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:24:42.923 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 115.84166531, 2023-11-25T04:24:42.924 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010920445000000001, 2023-11-25T04:24:42.924 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:42.924 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:42.924 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:42.924 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.924 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:42.924 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:24:42.925 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:42.925 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.925 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.925 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:42.925 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:24:42.925 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:42.925 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.925 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.926 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:42.926 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033904+0000", 2023-11-25T04:24:42.926 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6599999999999994e-07 2023-11-25T04:24:42.926 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.926 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.926 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:42.926 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033909+0000", 2023-11-25T04:24:42.926 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5639999999999996e-06 2023-11-25T04:24:42.927 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.927 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.927 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:42.927 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033923+0000", 2023-11-25T04:24:42.927 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3028e-05 2023-11-25T04:24:42.927 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.927 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.927 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:42.928 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034043+0000", 2023-11-25T04:24:42.928 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000120328 2023-11-25T04:24:42.928 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.928 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.928 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:42.928 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035576+0000", 2023-11-25T04:24:42.928 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015333720000000001 2023-11-25T04:24:42.928 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.929 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.929 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:42.929 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035699+0000", 2023-11-25T04:24:42.929 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000123001 2023-11-25T04:24:42.929 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.929 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.929 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:42.929 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044823+0000", 2023-11-25T04:24:42.930 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0091241859999999994 2023-11-25T04:24:42.930 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:42.930 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:42.930 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:42.930 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.930 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.930 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:118 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals,omap-rm-keys] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e44)", 2023-11-25T04:24:42.931 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:17.112648+0000", 2023-11-25T04:24:42.931 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 25.762919968999999, 2023-11-25T04:24:42.931 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011250779000000001, 2023-11-25T04:24:42.931 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:42.931 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:42.931 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:42.931 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.931 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:42.932 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.112648+0000", 2023-11-25T04:24:42.932 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:42.932 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.932 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.932 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:42.932 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.112648+0000", 2023-11-25T04:24:42.932 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:42.932 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.932 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.933 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:42.933 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.112649+0000", 2023-11-25T04:24:42.933 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0070000000000001e-06 2023-11-25T04:24:42.933 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.933 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.933 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:42.933 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113182+0000", 2023-11-25T04:24:42.934 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00053268100000000002 2023-11-25T04:24:42.934 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.934 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.934 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:42.934 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113196+0000", 2023-11-25T04:24:42.934 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3742e-05 2023-11-25T04:24:42.934 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.934 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.935 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:42.935 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113398+0000", 2023-11-25T04:24:42.935 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000202726 2023-11-25T04:24:42.935 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.935 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.935 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:42.935 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113447+0000", 2023-11-25T04:24:42.935 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8655000000000001e-05 2023-11-25T04:24:42.935 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.936 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.936 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:42.936 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113529+0000", 2023-11-25T04:24:42.936 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1891999999999995e-05 2023-11-25T04:24:42.936 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.936 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.936 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:42.936 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123819+0000", 2023-11-25T04:24:42.937 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010290192 2023-11-25T04:24:42.937 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.937 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.937 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:42.937 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123852+0000", 2023-11-25T04:24:42.937 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2425999999999998e-05 2023-11-25T04:24:42.937 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.937 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.938 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:42.938 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123899+0000", 2023-11-25T04:24:42.938 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7457999999999999e-05 2023-11-25T04:24:42.938 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:42.938 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:42.938 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:42.938 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.938 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.939 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:154 3.1e 3:7bf7e7b5:::200.00000006:head [write 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:42.939 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:21.886430+0000", 2023-11-25T04:24:42.939 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 20.989138702999998, 2023-11-25T04:24:42.939 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.017800107999999999, 2023-11-25T04:24:42.939 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:42.939 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:42.939 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:42.939 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.940 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:42.940 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.886430+0000", 2023-11-25T04:24:42.940 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:42.940 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.940 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.940 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:42.940 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.886430+0000", 2023-11-25T04:24:42.940 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:42.941 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.941 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.941 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:42.941 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.886431+0000", 2023-11-25T04:24:42.941 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.178e-06 2023-11-25T04:24:42.941 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.941 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.941 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:42.942 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.892277+0000", 2023-11-25T04:24:42.942 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0058467409999999999 2023-11-25T04:24:42.942 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.942 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.942 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:42.942 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.892288+0000", 2023-11-25T04:24:42.942 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0183000000000001e-05 2023-11-25T04:24:42.942 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.943 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.943 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:42.943 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.892292+0000", 2023-11-25T04:24:42.943 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7829999999999998e-06 2023-11-25T04:24:42.943 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.943 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.943 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:42.943 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.892362+0000", 2023-11-25T04:24:42.944 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0034000000000001e-05 2023-11-25T04:24:42.944 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.944 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.944 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:42.944 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.892432+0000", 2023-11-25T04:24:42.944 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9648999999999997e-05 2023-11-25T04:24:42.944 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.944 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.945 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:42.945 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.892550+0000", 2023-11-25T04:24:42.945 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011825 2023-11-25T04:24:42.945 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.945 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.945 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:42.945 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.899106+0000", 2023-11-25T04:24:42.945 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0065556759999999999 2023-11-25T04:24:42.946 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.946 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.946 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:42.946 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.904145+0000", 2023-11-25T04:24:42.946 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0050393490000000003 2023-11-25T04:24:42.946 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.946 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.946 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:42.947 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.904200+0000", 2023-11-25T04:24:42.947 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4234999999999998e-05 2023-11-25T04:24:42.947 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.947 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.947 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:42.947 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.904230+0000", 2023-11-25T04:24:42.947 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0029999999999999e-05 2023-11-25T04:24:42.947 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:42.948 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:42.948 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:42.948 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.948 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.948 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:576 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals,omap-rm-keys] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:42.948 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:22.111407+0000", 2023-11-25T04:24:42.948 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 20.764161096999999, 2023-11-25T04:24:42.948 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.018987317, 2023-11-25T04:24:42.949 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:42.949 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:42.949 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:42.949 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.949 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:42.949 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111407+0000", 2023-11-25T04:24:42.949 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:42.949 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.950 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.950 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:42.950 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111407+0000", 2023-11-25T04:24:42.950 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:42.950 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.950 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.950 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:42.951 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111408+0000", 2023-11-25T04:24:42.951 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8000000000000004e-07 2023-11-25T04:24:42.951 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.951 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.951 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:42.951 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111659+0000", 2023-11-25T04:24:42.951 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000250537 2023-11-25T04:24:42.951 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.952 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.952 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:42.952 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111666+0000", 2023-11-25T04:24:42.952 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.973e-06 2023-11-25T04:24:42.952 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.952 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.952 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:42.953 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111670+0000", 2023-11-25T04:24:42.953 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.668e-06 2023-11-25T04:24:42.953 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.953 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.953 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:42.953 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111709+0000", 2023-11-25T04:24:42.953 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8800000000000001e-05 2023-11-25T04:24:42.953 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.954 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.954 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:42.954 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111761+0000", 2023-11-25T04:24:42.954 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1495000000000001e-05 2023-11-25T04:24:42.954 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.954 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.954 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:42.954 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.130255+0000", 2023-11-25T04:24:42.955 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.018494345999999998 2023-11-25T04:24:42.955 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.955 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.955 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:42.955 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.130311+0000", 2023-11-25T04:24:42.955 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5902000000000001e-05 2023-11-25T04:24:42.955 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.955 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.956 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:42.956 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.130394+0000", 2023-11-25T04:24:42.956 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3715999999999996e-05 2023-11-25T04:24:42.956 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:42.956 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:42.956 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:42.956 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.956 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.957 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:1067 4.21 4:869a7600:::100000011fa.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:42.957 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:22.155038+0000", 2023-11-25T04:24:42.957 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 20.720529759000001, 2023-11-25T04:24:42.957 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011313148, 2023-11-25T04:24:42.957 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:42.957 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:42.957 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:42.957 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.958 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:42.958 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155038+0000", 2023-11-25T04:24:42.958 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:42.958 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.958 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.958 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:42.958 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155038+0000", 2023-11-25T04:24:42.959 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:42.959 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.959 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.959 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:42.959 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155039+0000", 2023-11-25T04:24:42.959 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0200000000000002e-07 2023-11-25T04:24:42.959 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.959 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.960 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:42.960 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155049+0000", 2023-11-25T04:24:42.960 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0295000000000001e-05 2023-11-25T04:24:42.960 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.960 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.960 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:42.960 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155058+0000", 2023-11-25T04:24:42.960 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.208e-06 2023-11-25T04:24:42.960 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.961 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.961 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:42.961 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155064+0000", 2023-11-25T04:24:42.961 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4489999999999996e-06 2023-11-25T04:24:42.961 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.961 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.961 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:42.961 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155104+0000", 2023-11-25T04:24:42.962 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0164999999999999e-05 2023-11-25T04:24:42.962 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.962 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.962 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:42.962 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155147+0000", 2023-11-25T04:24:42.962 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2576e-05 2023-11-25T04:24:42.962 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.963 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.963 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:42.963 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155212+0000", 2023-11-25T04:24:42.963 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5516000000000001e-05 2023-11-25T04:24:42.963 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.963 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.963 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:42.963 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.156025+0000", 2023-11-25T04:24:42.963 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00081248599999999996 2023-11-25T04:24:42.964 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.964 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.964 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:42.964 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.166250+0000", 2023-11-25T04:24:42.964 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010224736999999999 2023-11-25T04:24:42.964 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.964 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.964 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:42.965 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.166319+0000", 2023-11-25T04:24:42.965 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9145000000000003e-05 2023-11-25T04:24:42.965 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.965 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.965 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:42.965 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.166352+0000", 2023-11-25T04:24:42.965 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2969000000000002e-05 2023-11-25T04:24:42.965 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:42.966 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:42.966 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:42.966 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.966 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.966 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:1102 4.2f 4:f543b94c:::1000000121d.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:42.966 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:22.158246+0000", 2023-11-25T04:24:42.966 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 20.717322236000001, 2023-11-25T04:24:42.967 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010094396, 2023-11-25T04:24:42.967 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:42.967 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:42.967 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:42.967 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.967 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:42.967 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158246+0000", 2023-11-25T04:24:42.967 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:42.968 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.968 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.968 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:42.968 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158246+0000", 2023-11-25T04:24:42.968 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:42.968 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.968 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.968 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:42.969 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158246+0000", 2023-11-25T04:24:42.969 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5299999999999999e-07 2023-11-25T04:24:42.969 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.969 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.969 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:42.969 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158249+0000", 2023-11-25T04:24:42.969 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2840000000000001e-06 2023-11-25T04:24:42.969 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.970 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.970 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:42.970 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158254+0000", 2023-11-25T04:24:42.970 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3449999999999996e-06 2023-11-25T04:24:42.970 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.970 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.970 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:42.970 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158258+0000", 2023-11-25T04:24:42.971 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5030000000000002e-06 2023-11-25T04:24:42.971 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.971 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.971 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:42.971 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158302+0000", 2023-11-25T04:24:42.971 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3908e-05 2023-11-25T04:24:42.971 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.971 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.972 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:42.972 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158398+0000", 2023-11-25T04:24:42.972 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6898000000000006e-05 2023-11-25T04:24:42.972 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.972 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.972 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:42.972 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158480+0000", 2023-11-25T04:24:42.972 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1578999999999999e-05 2023-11-25T04:24:42.973 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.973 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.973 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:42.973 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.159229+0000", 2023-11-25T04:24:42.973 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000749049 2023-11-25T04:24:42.973 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.973 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.973 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:42.974 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.168228+0000", 2023-11-25T04:24:42.974 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0089992930000000002 2023-11-25T04:24:42.974 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.974 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.974 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:42.974 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.168310+0000", 2023-11-25T04:24:42.974 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1348999999999997e-05 2023-11-25T04:24:42.974 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.975 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.975 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:42.975 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.168340+0000", 2023-11-25T04:24:42.975 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0735000000000003e-05 2023-11-25T04:24:42.975 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:42.975 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:42.975 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:42.976 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.976 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.976 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:1105 4.23 4:c6ff07b4:::10000001220.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:42.976 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:22.158714+0000", 2023-11-25T04:24:42.976 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 20.716854122000001, 2023-11-25T04:24:42.976 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010412490999999999, 2023-11-25T04:24:42.976 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:42.976 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:42.977 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:42.977 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.977 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:42.977 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158714+0000", 2023-11-25T04:24:42.977 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:42.977 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.977 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.977 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:42.978 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158714+0000", 2023-11-25T04:24:42.978 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:42.978 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.978 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.978 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:42.978 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158715+0000", 2023-11-25T04:24:42.978 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4099999999999998e-07 2023-11-25T04:24:42.978 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.979 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.979 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:42.979 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158718+0000", 2023-11-25T04:24:42.979 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4319999999999999e-06 2023-11-25T04:24:42.979 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.979 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.979 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:42.979 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158729+0000", 2023-11-25T04:24:42.980 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.078e-05 2023-11-25T04:24:42.980 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.980 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.980 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:42.980 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158736+0000", 2023-11-25T04:24:42.980 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4649999999999997e-06 2023-11-25T04:24:42.980 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.980 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.981 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:42.981 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158779+0000", 2023-11-25T04:24:42.981 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3049000000000001e-05 2023-11-25T04:24:42.981 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.981 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.981 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:42.981 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158865+0000", 2023-11-25T04:24:42.982 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5138999999999999e-05 2023-11-25T04:24:42.982 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.982 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.982 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:42.982 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158983+0000", 2023-11-25T04:24:42.982 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000118658 2023-11-25T04:24:42.982 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.982 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.983 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:42.983 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.159645+0000", 2023-11-25T04:24:42.983 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00066193999999999997 2023-11-25T04:24:42.983 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.983 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.983 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:42.983 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.169046+0000", 2023-11-25T04:24:42.984 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0094005400000000006 2023-11-25T04:24:42.984 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.984 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.984 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:42.984 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.169092+0000", 2023-11-25T04:24:42.984 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5840000000000002e-05 2023-11-25T04:24:42.984 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.984 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.985 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:42.985 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.169127+0000", 2023-11-25T04:24:42.985 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5006999999999999e-05 2023-11-25T04:24:42.985 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:42.985 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:42.985 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:42.985 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.985 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.986 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2841 4.4 4:22111d42:::100000010df.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:42.986 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.092102+0000", 2023-11-25T04:24:42.986 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 17.783466647000001, 2023-11-25T04:24:42.986 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.016449783999999999, 2023-11-25T04:24:42.986 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:42.986 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:42.986 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:42.986 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.987 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:42.987 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092102+0000", 2023-11-25T04:24:42.987 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:42.987 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.987 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.987 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:42.987 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092102+0000", 2023-11-25T04:24:42.988 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:42.988 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.988 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.988 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:42.988 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092102+0000", 2023-11-25T04:24:42.988 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3700000000000001e-07 2023-11-25T04:24:42.988 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.988 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.989 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:42.989 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092104+0000", 2023-11-25T04:24:42.989 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0679999999999999e-06 2023-11-25T04:24:42.989 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.989 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.989 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:42.989 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092111+0000", 2023-11-25T04:24:42.989 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0589999999999997e-06 2023-11-25T04:24:42.990 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.990 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.990 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:42.990 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092130+0000", 2023-11-25T04:24:42.990 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8686999999999999e-05 2023-11-25T04:24:42.990 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.990 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.991 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:42.991 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092443+0000", 2023-11-25T04:24:42.991 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000313519 2023-11-25T04:24:42.991 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.991 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.991 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:42.991 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092553+0000", 2023-11-25T04:24:42.991 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010932100000000001 2023-11-25T04:24:42.991 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.992 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.992 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:42.992 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092639+0000", 2023-11-25T04:24:42.992 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6094999999999995e-05 2023-11-25T04:24:42.992 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.992 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.992 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:42.992 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094679+0000", 2023-11-25T04:24:42.993 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0020402580000000001 2023-11-25T04:24:42.993 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.993 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.993 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:42.993 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108462+0000", 2023-11-25T04:24:42.993 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013783358000000001 2023-11-25T04:24:42.993 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.993 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.994 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:42.994 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108516+0000", 2023-11-25T04:24:42.994 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3483000000000003e-05 2023-11-25T04:24:42.994 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.994 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.994 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:42.995 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108551+0000", 2023-11-25T04:24:42.995 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5499e-05 2023-11-25T04:24:42.995 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:42.995 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:42.995 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:42.995 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.995 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.995 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2852 4.2d 4:b745ac76:::100000010f5.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:42.996 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.093477+0000", 2023-11-25T04:24:42.996 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 17.782091534999999, 2023-11-25T04:24:42.996 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.015423084, 2023-11-25T04:24:42.996 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:42.996 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:42.996 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:42.996 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.997 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:42.997 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093477+0000", 2023-11-25T04:24:42.997 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:42.997 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.997 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.997 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:42.997 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093477+0000", 2023-11-25T04:24:42.997 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:42.998 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.998 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.998 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:42.998 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093477+0000", 2023-11-25T04:24:42.998 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2500000000000005e-07 2023-11-25T04:24:42.998 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.998 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.998 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:42.999 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093480+0000", 2023-11-25T04:24:42.999 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0189999999999998e-06 2023-11-25T04:24:42.999 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:42.999 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:42.999 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:42.999 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093642+0000", 2023-11-25T04:24:42.999 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000162072 2023-11-25T04:24:42.999 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.000 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.000 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:43.000 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093649+0000", 2023-11-25T04:24:43.000 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9659999999999999e-06 2023-11-25T04:24:43.000 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.000 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.000 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:43.000 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093693+0000", 2023-11-25T04:24:43.001 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3794000000000001e-05 2023-11-25T04:24:43.001 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.001 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.001 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:43.001 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093764+0000", 2023-11-25T04:24:43.001 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0981000000000004e-05 2023-11-25T04:24:43.001 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.001 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.002 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:43.002 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093850+0000", 2023-11-25T04:24:43.002 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5536000000000002e-05 2023-11-25T04:24:43.002 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.002 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.002 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:43.002 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094710+0000", 2023-11-25T04:24:43.003 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00086070299999999999 2023-11-25T04:24:43.003 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.003 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.003 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:43.003 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108776+0000", 2023-11-25T04:24:43.003 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01406542 2023-11-25T04:24:43.003 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.003 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.004 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:43.004 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108829+0000", 2023-11-25T04:24:43.004 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2936999999999998e-05 2023-11-25T04:24:43.004 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.004 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.004 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:43.004 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108900+0000", 2023-11-25T04:24:43.004 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1031000000000005e-05 2023-11-25T04:24:43.005 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:43.005 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:43.005 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:43.005 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.005 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.005 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2854 4.2d 4:b5c259cf:::100000010f9.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:43.005 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.093674+0000", 2023-11-25T04:24:43.005 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 17.781893875000002, 2023-11-25T04:24:43.006 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.015970380999999999, 2023-11-25T04:24:43.006 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:43.006 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:43.006 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:43.006 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.006 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:43.006 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093674+0000", 2023-11-25T04:24:43.006 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:43.007 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.007 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.007 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:43.007 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093674+0000", 2023-11-25T04:24:43.007 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:43.007 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.008 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.008 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:43.008 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093675+0000", 2023-11-25T04:24:43.008 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8400000000000004e-07 2023-11-25T04:24:43.008 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.008 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.008 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:43.009 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093678+0000", 2023-11-25T04:24:43.009 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2229999999999999e-06 2023-11-25T04:24:43.009 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.009 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.009 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:43.009 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093686+0000", 2023-11-25T04:24:43.009 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9990000000000008e-06 2023-11-25T04:24:43.009 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.010 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.010 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:43.010 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093696+0000", 2023-11-25T04:24:43.010 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6479999999999998e-06 2023-11-25T04:24:43.010 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.010 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.010 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:43.010 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094104+0000", 2023-11-25T04:24:43.011 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00040823900000000002 2023-11-25T04:24:43.011 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.011 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.011 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:43.011 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094172+0000", 2023-11-25T04:24:43.011 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8023999999999998e-05 2023-11-25T04:24:43.011 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.011 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.012 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:43.012 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094256+0000", 2023-11-25T04:24:43.012 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3418999999999995e-05 2023-11-25T04:24:43.012 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.012 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.012 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:43.012 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095160+0000", 2023-11-25T04:24:43.013 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00090418599999999997 2023-11-25T04:24:43.013 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.013 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.013 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:43.013 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109551+0000", 2023-11-25T04:24:43.013 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014391571000000001 2023-11-25T04:24:43.013 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.013 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.014 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:43.014 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109601+0000", 2023-11-25T04:24:43.014 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9478000000000003e-05 2023-11-25T04:24:43.014 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.014 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.014 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:43.014 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109645+0000", 2023-11-25T04:24:43.014 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.401e-05 2023-11-25T04:24:43.015 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:43.015 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:43.015 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:43.015 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.015 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.015 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2861 4.4 4:214027a9:::10000001107.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:43.015 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.093829+0000", 2023-11-25T04:24:43.016 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 17.781739457, 2023-11-25T04:24:43.016 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014848917999999999, 2023-11-25T04:24:43.016 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:43.016 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:43.016 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:43.016 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.016 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:43.016 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093829+0000", 2023-11-25T04:24:43.017 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:43.017 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.017 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.017 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:43.017 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093829+0000", 2023-11-25T04:24:43.017 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:43.017 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.018 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.018 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:43.018 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093830+0000", 2023-11-25T04:24:43.018 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1999999999999998e-07 2023-11-25T04:24:43.018 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.018 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.018 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:43.018 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093834+0000", 2023-11-25T04:24:43.019 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.34e-06 2023-11-25T04:24:43.019 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.019 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.019 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:43.019 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093844+0000", 2023-11-25T04:24:43.019 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6690000000000001e-06 2023-11-25T04:24:43.019 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.019 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.020 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:43.020 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093857+0000", 2023-11-25T04:24:43.020 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3434999999999999e-05 2023-11-25T04:24:43.020 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.020 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.020 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:43.020 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093940+0000", 2023-11-25T04:24:43.020 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3047999999999995e-05 2023-11-25T04:24:43.021 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.021 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.021 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:43.021 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094052+0000", 2023-11-25T04:24:43.021 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000111538 2023-11-25T04:24:43.021 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.021 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.021 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:43.022 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094466+0000", 2023-11-25T04:24:43.022 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00041464600000000001 2023-11-25T04:24:43.022 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.022 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.022 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:43.022 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095570+0000", 2023-11-25T04:24:43.022 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001103981 2023-11-25T04:24:43.023 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.023 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.023 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:43.023 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108598+0000", 2023-11-25T04:24:43.023 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013027164000000001 2023-11-25T04:24:43.023 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.023 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.023 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:43.024 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108646+0000", 2023-11-25T04:24:43.024 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8090999999999997e-05 2023-11-25T04:24:43.024 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.024 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.024 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:43.024 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108678+0000", 2023-11-25T04:24:43.024 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2085999999999997e-05 2023-11-25T04:24:43.024 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:43.025 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:43.025 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:43.025 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.025 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.025 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2863 4.12 4:4b84c733:::1000000110b.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:43.025 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.093877+0000", 2023-11-25T04:24:43.025 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 17.781691129999999, 2023-11-25T04:24:43.025 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014682755, 2023-11-25T04:24:43.026 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:43.026 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:43.026 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:43.026 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.026 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:43.026 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093877+0000", 2023-11-25T04:24:43.026 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:43.027 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.027 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.027 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:43.027 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093877+0000", 2023-11-25T04:24:43.027 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:43.027 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.027 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.027 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:43.028 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093878+0000", 2023-11-25T04:24:43.028 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4199999999999996e-07 2023-11-25T04:24:43.028 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.028 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.028 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:43.028 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093880+0000", 2023-11-25T04:24:43.028 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6189999999999998e-06 2023-11-25T04:24:43.028 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.029 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.029 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:43.029 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093888+0000", 2023-11-25T04:24:43.029 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5510000000000002e-06 2023-11-25T04:24:43.029 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.029 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.029 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:43.029 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093893+0000", 2023-11-25T04:24:43.030 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4179999999999996e-06 2023-11-25T04:24:43.030 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.030 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.030 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:43.030 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093934+0000", 2023-11-25T04:24:43.030 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0648000000000001e-05 2023-11-25T04:24:43.030 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.031 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.031 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:43.031 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094081+0000", 2023-11-25T04:24:43.031 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014753499999999999 2023-11-25T04:24:43.031 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.031 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.031 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:43.031 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094172+0000", 2023-11-25T04:24:43.032 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.0722000000000003e-05 2023-11-25T04:24:43.032 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.032 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.032 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:43.032 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094742+0000", 2023-11-25T04:24:43.032 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00056948600000000002 2023-11-25T04:24:43.032 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.032 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.033 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:43.033 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108491+0000", 2023-11-25T04:24:43.033 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013749568 2023-11-25T04:24:43.033 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.033 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.033 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:43.033 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108525+0000", 2023-11-25T04:24:43.033 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3683999999999998e-05 2023-11-25T04:24:43.034 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.034 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.034 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:43.034 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108560+0000", 2023-11-25T04:24:43.034 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4981999999999998e-05 2023-11-25T04:24:43.034 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:43.034 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:43.034 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:43.035 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.035 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.035 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2880 4.21 4:855834a4:::1000000112d.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:43.035 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.095098+0000", 2023-11-25T04:24:43.035 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 17.780470436000002, 2023-11-25T04:24:43.035 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013847544, 2023-11-25T04:24:43.035 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:43.035 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:43.036 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:43.036 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.036 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:43.036 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095098+0000", 2023-11-25T04:24:43.036 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:43.036 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.036 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.036 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:43.037 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095098+0000", 2023-11-25T04:24:43.037 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:43.037 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.037 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.037 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:43.037 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095098+0000", 2023-11-25T04:24:43.037 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3000000000000002e-07 2023-11-25T04:24:43.038 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.038 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.038 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:43.038 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095099+0000", 2023-11-25T04:24:43.038 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.331e-06 2023-11-25T04:24:43.038 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.038 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.038 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:43.039 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095103+0000", 2023-11-25T04:24:43.039 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8319999999999999e-06 2023-11-25T04:24:43.039 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.039 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.039 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:43.039 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095108+0000", 2023-11-25T04:24:43.039 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4179999999999999e-06 2023-11-25T04:24:43.040 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.040 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.040 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:43.040 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095153+0000", 2023-11-25T04:24:43.040 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4799000000000003e-05 2023-11-25T04:24:43.040 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.040 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.040 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:43.041 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095268+0000", 2023-11-25T04:24:43.041 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000115437 2023-11-25T04:24:43.041 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.041 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.041 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:43.041 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095363+0000", 2023-11-25T04:24:43.041 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.5278000000000005e-05 2023-11-25T04:24:43.041 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.042 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.042 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:43.042 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095845+0000", 2023-11-25T04:24:43.042 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00048164600000000002 2023-11-25T04:24:43.042 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.042 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.042 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:43.042 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108854+0000", 2023-11-25T04:24:43.043 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013009289 2023-11-25T04:24:43.043 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.043 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.043 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:43.043 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108907+0000", 2023-11-25T04:24:43.043 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2528999999999998e-05 2023-11-25T04:24:43.043 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.043 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.044 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:43.044 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108945+0000", 2023-11-25T04:24:43.044 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8655000000000002e-05 2023-11-25T04:24:43.044 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:43.044 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:43.044 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:43.044 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.044 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.045 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2884 4.37 4:ef909cdc:::10000001135.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:43.045 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.095518+0000", 2023-11-25T04:24:43.045 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 17.780050508999999, 2023-11-25T04:24:43.045 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014149988, 2023-11-25T04:24:43.045 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:43.045 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:43.045 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:43.045 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.046 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:43.046 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095518+0000", 2023-11-25T04:24:43.046 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:43.046 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.046 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.046 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:43.046 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095518+0000", 2023-11-25T04:24:43.047 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:43.047 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.047 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.047 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:43.047 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095518+0000", 2023-11-25T04:24:43.047 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2900000000000003e-07 2023-11-25T04:24:43.047 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.047 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.047 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:43.048 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095522+0000", 2023-11-25T04:24:43.048 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2119999999999999e-06 2023-11-25T04:24:43.048 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.048 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.048 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:43.048 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095532+0000", 2023-11-25T04:24:43.048 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0375e-05 2023-11-25T04:24:43.048 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.049 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.049 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:43.049 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095537+0000", 2023-11-25T04:24:43.049 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5909999999999998e-06 2023-11-25T04:24:43.049 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.049 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.049 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:43.049 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095584+0000", 2023-11-25T04:24:43.050 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7169999999999997e-05 2023-11-25T04:24:43.050 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.050 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.050 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:43.050 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095695+0000", 2023-11-25T04:24:43.050 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011078699999999999 2023-11-25T04:24:43.050 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.051 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.051 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:43.051 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095777+0000", 2023-11-25T04:24:43.051 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2089999999999995e-05 2023-11-25T04:24:43.051 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.051 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.051 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:43.051 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096286+0000", 2023-11-25T04:24:43.052 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000509247 2023-11-25T04:24:43.052 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.052 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.052 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:43.052 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109596+0000", 2023-11-25T04:24:43.052 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013309872 2023-11-25T04:24:43.052 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.052 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.053 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:43.053 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109638+0000", 2023-11-25T04:24:43.053 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2381e-05 2023-11-25T04:24:43.053 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.053 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.053 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:43.053 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109668+0000", 2023-11-25T04:24:43.053 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9634000000000001e-05 2023-11-25T04:24:43.054 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:43.054 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:43.054 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:43.054 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.054 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.054 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2890 4.12 4:4a2ad640:::10000001141.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:43.054 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.096274+0000", 2023-11-25T04:24:43.055 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 17.779294446000002, 2023-11-25T04:24:43.055 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013647259, 2023-11-25T04:24:43.055 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:43.055 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:43.055 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:43.055 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.055 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:43.055 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096274+0000", 2023-11-25T04:24:43.056 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:43.056 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.056 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.056 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:43.056 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096274+0000", 2023-11-25T04:24:43.056 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:43.056 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.057 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.057 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:43.057 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096275+0000", 2023-11-25T04:24:43.057 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0299999999999998e-07 2023-11-25T04:24:43.057 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.057 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.057 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:43.058 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096279+0000", 2023-11-25T04:24:43.058 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9580000000000001e-06 2023-11-25T04:24:43.058 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.058 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.058 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:43.058 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096287+0000", 2023-11-25T04:24:43.058 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6929999999999996e-06 2023-11-25T04:24:43.058 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.059 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.059 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:43.059 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096294+0000", 2023-11-25T04:24:43.059 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.348e-06 2023-11-25T04:24:43.059 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.059 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.059 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:43.059 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096342+0000", 2023-11-25T04:24:43.060 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8068000000000001e-05 2023-11-25T04:24:43.060 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.060 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.060 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:43.060 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096462+0000", 2023-11-25T04:24:43.060 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000119918 2023-11-25T04:24:43.060 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.060 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.061 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:43.061 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096557+0000", 2023-11-25T04:24:43.061 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.5340999999999994e-05 2023-11-25T04:24:43.061 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.061 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.061 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:43.061 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096971+0000", 2023-11-25T04:24:43.062 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00041398100000000001 2023-11-25T04:24:43.062 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.062 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.062 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:43.062 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109849+0000", 2023-11-25T04:24:43.062 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.012877985999999999 2023-11-25T04:24:43.062 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.063 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.063 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:43.063 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109894+0000", 2023-11-25T04:24:43.063 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5595e-05 2023-11-25T04:24:43.063 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.063 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.063 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:43.063 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109921+0000", 2023-11-25T04:24:43.064 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6568e-05 2023-11-25T04:24:43.064 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:43.064 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:43.064 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:43.064 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.064 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.064 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2891 4.23 4:c4054dd3:::10000001143.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:43.064 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.096713+0000", 2023-11-25T04:24:43.065 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 17.778855751999998, 2023-11-25T04:24:43.065 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013148692, 2023-11-25T04:24:43.065 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:43.065 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:43.065 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:43.065 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.065 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:43.065 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096713+0000", 2023-11-25T04:24:43.066 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:43.066 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.066 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.066 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:43.066 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096713+0000", 2023-11-25T04:24:43.066 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:43.066 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.066 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.067 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:43.067 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096713+0000", 2023-11-25T04:24:43.067 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9299999999999998e-07 2023-11-25T04:24:43.067 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.067 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.067 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:43.067 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096716+0000", 2023-11-25T04:24:43.067 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.988e-06 2023-11-25T04:24:43.068 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.068 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.068 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:43.068 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096722+0000", 2023-11-25T04:24:43.068 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0499999999999997e-06 2023-11-25T04:24:43.068 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.068 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.068 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:43.069 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096727+0000", 2023-11-25T04:24:43.069 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3380000000000004e-06 2023-11-25T04:24:43.069 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.069 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.069 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:43.069 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096770+0000", 2023-11-25T04:24:43.069 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2938000000000001e-05 2023-11-25T04:24:43.069 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.070 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.070 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:43.070 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096904+0000", 2023-11-25T04:24:43.070 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000133759 2023-11-25T04:24:43.070 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.070 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.070 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:43.071 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096983+0000", 2023-11-25T04:24:43.071 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8559999999999993e-05 2023-11-25T04:24:43.071 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.071 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.071 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:43.071 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.097437+0000", 2023-11-25T04:24:43.071 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00045439200000000002 2023-11-25T04:24:43.071 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.072 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.072 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:43.072 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109737+0000", 2023-11-25T04:24:43.072 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.012300190000000001 2023-11-25T04:24:43.072 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.072 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.072 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:43.072 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109782+0000", 2023-11-25T04:24:43.073 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4546999999999999e-05 2023-11-25T04:24:43.073 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:43.073 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:43.073 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:43.073 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109861+0000", 2023-11-25T04:24:43.073 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9337000000000003e-05 2023-11-25T04:24:43.073 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:43.073 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:43.074 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:43.074 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:43.074 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:43.074 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:43.075 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:24:43.076 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:43.074+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:43.158 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 8 2023-11-25T04:24:43.171 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-25T04:24:43.175 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:43.173+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:43.184 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:43.185 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:24:43.185 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2023-11-25T04:24:43.185 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:43.195 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:24:43.229 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2023-11-25T04:24:43.280 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:43.278+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:43.340 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:43.340 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:24:43.340 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2023-11-25T04:24:43.340 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2023-11-25T04:24:43.340 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:43.348 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:24:43.389 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:43.387+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:43.479 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:43.477+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:43.496 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:43.497 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2023-11-25T04:24:43.497 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2023-11-25T04:24:43.497 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2023-11-25T04:24:43.497 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.497 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:152 3.1a 3:5e41ab69:::200.00000005:head [write 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:43.498 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:21.237401+0000", 2023-11-25T04:24:43.498 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 22.255965330999999, 2023-11-25T04:24:43.498 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015253802, 2023-11-25T04:24:43.498 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:43.498 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:43.498 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:43.499 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.499 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:43.499 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.237401+0000", 2023-11-25T04:24:43.499 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:43.499 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.499 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.499 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:43.500 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.237401+0000", 2023-11-25T04:24:43.500 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:43.500 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.500 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.500 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:43.500 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.237402+0000", 2023-11-25T04:24:43.501 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.3699999999999999e-07 2023-11-25T04:24:43.501 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.501 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.501 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:43.501 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243182+0000", 2023-11-25T04:24:43.501 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0057801270000000004 2023-11-25T04:24:43.502 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.502 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.502 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:43.502 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243193+0000", 2023-11-25T04:24:43.502 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1221e-05 2023-11-25T04:24:43.502 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.502 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.503 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:43.503 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243198+0000", 2023-11-25T04:24:43.503 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3739999999999997e-06 2023-11-25T04:24:43.503 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.505 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.506 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:43.506 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243246+0000", 2023-11-25T04:24:43.506 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7667999999999998e-05 2023-11-25T04:24:43.506 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.506 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.507 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:43.507 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243292+0000", 2023-11-25T04:24:43.507 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5794000000000002e-05 2023-11-25T04:24:43.507 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.507 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.508 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:43.508 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.243360+0000", 2023-11-25T04:24:43.508 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8479000000000006e-05 2023-11-25T04:24:43.508 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.508 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.508 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:43.509 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.250667+0000", 2023-11-25T04:24:43.509 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.007306618 2023-11-25T04:24:43.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.510 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.510 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:43.510 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.252598+0000", 2023-11-25T04:24:43.510 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0019315129999999999 2023-11-25T04:24:43.511 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.511 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.511 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:43.511 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.252631+0000", 2023-11-25T04:24:43.511 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2925000000000001e-05 2023-11-25T04:24:43.512 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.512 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.512 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:43.512 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:21.252655+0000", 2023-11-25T04:24:43.512 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.3246000000000001e-05 2023-11-25T04:24:43.512 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:43.513 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:43.513 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:43.513 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.513 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.513 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:2841 4.4 e47/17 4:22111d42:::100000010df.00000000:head v 47'139, mlcod=47'137)", 2023-11-25T04:24:43.513 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.093610+0000", 2023-11-25T04:24:43.514 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.399756468, 2023-11-25T04:24:43.514 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014758871999999999, 2023-11-25T04:24:43.514 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:43.514 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:43.514 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:43.515 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.515 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:43.515 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093610+0000", 2023-11-25T04:24:43.515 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:43.515 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.516 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.516 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:43.516 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093610+0000", 2023-11-25T04:24:43.516 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:43.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.516 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.517 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:43.517 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093610+0000", 2023-11-25T04:24:43.517 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1799999999999999e-07 2023-11-25T04:24:43.517 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.517 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.517 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:43.518 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093621+0000", 2023-11-25T04:24:43.518 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.1107999999999999e-05 2023-11-25T04:24:43.518 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.518 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.518 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:43.518 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093627+0000", 2023-11-25T04:24:43.518 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0310000000000004e-06 2023-11-25T04:24:43.518 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.519 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.519 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:43.519 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093633+0000", 2023-11-25T04:24:43.519 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2700000000000001e-06 2023-11-25T04:24:43.519 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.519 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.519 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:43.519 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093722+0000", 2023-11-25T04:24:43.520 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8417000000000004e-05 2023-11-25T04:24:43.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.520 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.520 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:43.520 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093737+0000", 2023-11-25T04:24:43.520 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5781e-05 2023-11-25T04:24:43.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.520 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.521 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:43.521 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108203+0000", 2023-11-25T04:24:43.521 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014465844 2023-11-25T04:24:43.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.521 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.521 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:43.521 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108368+0000", 2023-11-25T04:24:43.521 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000165203 2023-11-25T04:24:43.522 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:43.522 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:43.522 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:43.522 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.522 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.522 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2860 4.3d 4:bf0d7877:::10000001105.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:43.522 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.093823+0000", 2023-11-25T04:24:43.522 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.399543300000001, 2023-11-25T04:24:43.523 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015582848, 2023-11-25T04:24:43.523 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:43.523 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:43.523 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:43.523 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.523 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:43.523 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093823+0000", 2023-11-25T04:24:43.523 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:43.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.524 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.524 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:43.524 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093823+0000", 2023-11-25T04:24:43.524 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:43.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.524 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.524 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:43.525 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093823+0000", 2023-11-25T04:24:43.525 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9100000000000004e-07 2023-11-25T04:24:43.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.525 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.525 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:43.525 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093826+0000", 2023-11-25T04:24:43.525 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4499999999999998e-06 2023-11-25T04:24:43.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.526 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.526 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:43.526 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093835+0000", 2023-11-25T04:24:43.526 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.8179999999999993e-06 2023-11-25T04:24:43.526 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.526 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.526 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:43.527 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093840+0000", 2023-11-25T04:24:43.527 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0819999999999998e-06 2023-11-25T04:24:43.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.527 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.527 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:43.527 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093973+0000", 2023-11-25T04:24:43.527 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013308999999999999 2023-11-25T04:24:43.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.528 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.528 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:43.528 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094277+0000", 2023-11-25T04:24:43.528 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00030403299999999999 2023-11-25T04:24:43.528 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.528 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.528 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:43.529 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108608+0000", 2023-11-25T04:24:43.529 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014330888 2023-11-25T04:24:43.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.529 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.529 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:43.529 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108676+0000", 2023-11-25T04:24:43.530 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8577000000000004e-05 2023-11-25T04:24:43.530 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.530 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.530 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:43.530 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109406+0000", 2023-11-25T04:24:43.530 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00072931899999999997 2023-11-25T04:24:43.530 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:43.531 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:43.531 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:43.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.531 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.531 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2862 4.13 4:cb2e645e:::10000001109.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:43.531 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.093852+0000", 2023-11-25T04:24:43.531 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.399514339, 2023-11-25T04:24:43.531 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015190636, 2023-11-25T04:24:43.532 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:43.532 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:43.532 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:43.532 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.532 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:43.532 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093852+0000", 2023-11-25T04:24:43.532 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:43.536 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.537 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.537 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:43.537 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093852+0000", 2023-11-25T04:24:43.537 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:43.537 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.537 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.537 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:43.538 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093852+0000", 2023-11-25T04:24:43.538 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0299999999999999e-07 2023-11-25T04:24:43.538 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.538 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.538 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:43.538 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093854+0000", 2023-11-25T04:24:43.538 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.846e-06 2023-11-25T04:24:43.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.539 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:43.539 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093861+0000", 2023-11-25T04:24:43.539 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5520000000000001e-06 2023-11-25T04:24:43.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.540 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.540 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:43.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093865+0000", 2023-11-25T04:24:43.540 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7870000000000001e-06 2023-11-25T04:24:43.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.540 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.540 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:43.540 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093940+0000", 2023-11-25T04:24:43.541 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.4799e-05 2023-11-25T04:24:43.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.541 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.541 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:43.541 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094052+0000", 2023-11-25T04:24:43.541 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011156500000000001 2023-11-25T04:24:43.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.541 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.542 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:43.542 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108928+0000", 2023-11-25T04:24:43.542 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014876245 2023-11-25T04:24:43.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.542 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.542 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:43.542 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108972+0000", 2023-11-25T04:24:43.543 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4477999999999997e-05 2023-11-25T04:24:43.543 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.543 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.543 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:43.543 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109042+0000", 2023-11-25T04:24:43.543 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9861000000000001e-05 2023-11-25T04:24:43.543 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:43.543 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:43.543 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:43.544 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.544 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.544 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2865 4.36 4:6ebc5ac6:::1000000110f.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:43.544 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.093906+0000", 2023-11-25T04:24:43.544 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.399460066, 2023-11-25T04:24:43.544 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014962628, 2023-11-25T04:24:43.544 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:43.545 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:43.545 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:43.545 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.545 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:43.545 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093906+0000", 2023-11-25T04:24:43.545 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:43.545 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.545 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.546 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:43.546 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093906+0000", 2023-11-25T04:24:43.546 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:43.546 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.547 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.547 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:43.547 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093906+0000", 2023-11-25T04:24:43.547 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.4400000000000001e-07 2023-11-25T04:24:43.547 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.547 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.547 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:43.547 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093908+0000", 2023-11-25T04:24:43.548 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8640000000000001e-06 2023-11-25T04:24:43.548 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.548 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.548 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:43.548 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093914+0000", 2023-11-25T04:24:43.548 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.0170000000000002e-06 2023-11-25T04:24:43.548 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.549 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.549 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:43.549 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093921+0000", 2023-11-25T04:24:43.549 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6529999999999997e-06 2023-11-25T04:24:43.549 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.549 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.549 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:43.549 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094258+0000", 2023-11-25T04:24:43.550 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00033703599999999998 2023-11-25T04:24:43.550 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.550 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.550 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:43.550 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094340+0000", 2023-11-25T04:24:43.550 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.2279000000000003e-05 2023-11-25T04:24:43.550 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.551 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.551 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:43.551 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094416+0000", 2023-11-25T04:24:43.551 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.6248999999999995e-05 2023-11-25T04:24:43.551 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.551 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.551 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:43.551 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.096134+0000", 2023-11-25T04:24:43.551 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001717611 2023-11-25T04:24:43.552 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.552 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.552 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:43.552 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108809+0000", 2023-11-25T04:24:43.552 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012675327 2023-11-25T04:24:43.552 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.552 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.553 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:43.553 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108836+0000", 2023-11-25T04:24:43.553 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6509999999999999e-05 2023-11-25T04:24:43.553 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.553 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.553 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:43.553 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108869+0000", 2023-11-25T04:24:43.553 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.2737999999999997e-05 2023-11-25T04:24:43.554 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:43.554 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:43.554 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:43.554 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.554 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.554 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:2852 4.2d e47/17 4:b745ac76:::100000010f5.00000000:head v 47'126, mlcod=47'124)", 2023-11-25T04:24:43.554 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.093987+0000", 2023-11-25T04:24:43.554 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.399379504999999, 2023-11-25T04:24:43.555 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014613256, 2023-11-25T04:24:43.555 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:43.555 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:43.555 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:43.555 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.555 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:43.555 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093987+0000", 2023-11-25T04:24:43.555 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:43.556 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.556 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.556 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:43.556 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093987+0000", 2023-11-25T04:24:43.556 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:43.556 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.556 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.557 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:43.557 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093987+0000", 2023-11-25T04:24:43.557 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7000000000000001e-07 2023-11-25T04:24:43.557 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.557 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.557 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:43.557 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093990+0000", 2023-11-25T04:24:43.557 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.8729999999999999e-06 2023-11-25T04:24:43.558 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.558 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.558 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:43.558 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.093997+0000", 2023-11-25T04:24:43.558 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.973e-06 2023-11-25T04:24:43.558 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.558 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.558 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:43.559 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094001+0000", 2023-11-25T04:24:43.559 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7360000000000001e-06 2023-11-25T04:24:43.559 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.559 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.559 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:43.559 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094263+0000", 2023-11-25T04:24:43.559 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00026173400000000001 2023-11-25T04:24:43.560 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.560 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.560 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:43.560 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094278+0000", 2023-11-25T04:24:43.560 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.4681999999999999e-05 2023-11-25T04:24:43.560 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.560 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.560 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:43.561 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108563+0000", 2023-11-25T04:24:43.561 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014284738 2023-11-25T04:24:43.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.561 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.561 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:43.561 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108600+0000", 2023-11-25T04:24:43.561 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7249999999999997e-05 2023-11-25T04:24:43.562 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:43.562 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:43.562 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:43.562 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.562 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.562 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2866 4.36 4:6f6907fb:::10000001111.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:43.562 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.094033+0000", 2023-11-25T04:24:43.563 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.399333339999998, 2023-11-25T04:24:43.564 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014885591, 2023-11-25T04:24:43.564 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:43.564 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:43.564 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:43.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.564 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:43.564 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094033+0000", 2023-11-25T04:24:43.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:43.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.565 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:43.565 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094033+0000", 2023-11-25T04:24:43.565 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:43.565 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.565 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.566 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:43.566 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094033+0000", 2023-11-25T04:24:43.566 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.06e-07 2023-11-25T04:24:43.566 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.566 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.566 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:43.566 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094036+0000", 2023-11-25T04:24:43.566 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.508e-06 2023-11-25T04:24:43.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.567 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.567 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:43.567 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094044+0000", 2023-11-25T04:24:43.567 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5909999999999996e-06 2023-11-25T04:24:43.567 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:43.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094051+0000", 2023-11-25T04:24:43.568 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0319999999999998e-06 2023-11-25T04:24:43.568 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.568 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.568 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:43.568 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094727+0000", 2023-11-25T04:24:43.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00067523300000000002 2023-11-25T04:24:43.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.569 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:43.569 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094795+0000", 2023-11-25T04:24:43.569 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8156000000000003e-05 2023-11-25T04:24:43.569 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.569 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.570 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:43.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094898+0000", 2023-11-25T04:24:43.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103329 2023-11-25T04:24:43.570 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.570 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.570 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:43.570 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108803+0000", 2023-11-25T04:24:43.570 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013904385 2023-11-25T04:24:43.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:43.571 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108875+0000", 2023-11-25T04:24:43.571 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1958999999999993e-05 2023-11-25T04:24:43.571 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.571 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.571 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:43.572 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108899+0000", 2023-11-25T04:24:43.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.4019000000000002e-05 2023-11-25T04:24:43.572 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.572 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.572 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:43.572 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108918+0000", 2023-11-25T04:24:43.572 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9772999999999999e-05 2023-11-25T04:24:43.572 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:43.573 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:43.573 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:43.573 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.573 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.573 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_repop(mds.0.3:2854 4.2d e47/17 4:b5c259cf:::100000010f9.00000000:head v 47'127, mlcod=47'124)", 2023-11-25T04:24:43.573 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.094793+0000", 2023-11-25T04:24:43.573 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.398573051, 2023-11-25T04:24:43.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014650764, 2023-11-25T04:24:43.574 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:43.574 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:43.574 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:43.574 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.574 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:43.574 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094793+0000", 2023-11-25T04:24:43.574 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:43.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:43.575 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094793+0000", 2023-11-25T04:24:43.575 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:43.575 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.575 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.575 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:43.576 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094793+0000", 2023-11-25T04:24:43.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3700000000000001e-07 2023-11-25T04:24:43.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.576 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.576 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:43.576 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094798+0000", 2023-11-25T04:24:43.576 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1200000000000004e-06 2023-11-25T04:24:43.576 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.577 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:43.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094805+0000", 2023-11-25T04:24:43.577 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9759999999999998e-06 2023-11-25T04:24:43.577 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.577 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.577 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:43.577 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094811+0000", 2023-11-25T04:24:43.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1410000000000002e-06 2023-11-25T04:24:43.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.578 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.578 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:43.578 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094838+0000", 2023-11-25T04:24:43.578 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.6995000000000002e-05 2023-11-25T04:24:43.578 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:43.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094848+0000", 2023-11-25T04:24:43.579 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.0115999999999999e-05 2023-11-25T04:24:43.579 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.579 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.579 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:43.579 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109408+0000", 2023-11-25T04:24:43.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014559825 2023-11-25T04:24:43.580 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.580 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.580 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:43.580 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109444+0000", 2023-11-25T04:24:43.580 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6154e-05 2023-11-25T04:24:43.580 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:43.580 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:43.581 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:43.581 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.581 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.581 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2869 4.6 4:63719fb0:::10000001117.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:43.581 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.094910+0000", 2023-11-25T04:24:43.581 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.398456094, 2023-11-25T04:24:43.581 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014132570000000001, 2023-11-25T04:24:43.581 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:43.582 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:43.582 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:43.582 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.582 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:43.582 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094910+0000", 2023-11-25T04:24:43.582 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:43.582 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.583 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:43.583 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094910+0000", 2023-11-25T04:24:43.583 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:43.583 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.583 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.583 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:43.583 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094910+0000", 2023-11-25T04:24:43.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.58e-07 2023-11-25T04:24:43.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.584 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:43.584 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094913+0000", 2023-11-25T04:24:43.584 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.198e-06 2023-11-25T04:24:43.584 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.584 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:43.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094921+0000", 2023-11-25T04:24:43.585 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.1629999999999996e-06 2023-11-25T04:24:43.585 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.585 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.585 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:43.585 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094926+0000", 2023-11-25T04:24:43.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7559999999999999e-06 2023-11-25T04:24:43.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.586 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:43.586 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094966+0000", 2023-11-25T04:24:43.586 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0151999999999998e-05 2023-11-25T04:24:43.586 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.586 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.587 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:43.587 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095073+0000", 2023-11-25T04:24:43.587 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000107657 2023-11-25T04:24:43.587 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.587 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.587 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:43.587 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095158+0000", 2023-11-25T04:24:43.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.5136999999999995e-05 2023-11-25T04:24:43.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.588 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:43.588 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108924+0000", 2023-11-25T04:24:43.588 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013765727 2023-11-25T04:24:43.588 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.588 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:43.589 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108958+0000", 2023-11-25T04:24:43.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3417000000000002e-05 2023-11-25T04:24:43.589 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.589 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.589 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:43.589 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109012+0000", 2023-11-25T04:24:43.589 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3939e-05 2023-11-25T04:24:43.590 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.590 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.590 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:43.590 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109043+0000", 2023-11-25T04:24:43.590 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0966000000000001e-05 2023-11-25T04:24:43.590 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:43.590 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:43.590 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:43.591 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.591 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.591 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2871 4.3f 4:fff10e11:::1000000111b.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:43.591 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.094936+0000", 2023-11-25T04:24:43.591 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.398430256000001, 2023-11-25T04:24:43.591 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014261726000000001, 2023-11-25T04:24:43.591 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:43.592 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:43.592 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:43.592 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.592 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:43.592 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094936+0000", 2023-11-25T04:24:43.592 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:43.592 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.592 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:43.593 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094936+0000", 2023-11-25T04:24:43.593 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:43.593 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.593 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.593 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:43.593 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094936+0000", 2023-11-25T04:24:43.593 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9900000000000001e-07 2023-11-25T04:24:43.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.594 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:43.594 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094938+0000", 2023-11-25T04:24:43.594 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9599999999999999e-06 2023-11-25T04:24:43.594 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.594 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:43.595 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094944+0000", 2023-11-25T04:24:43.595 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2219999999999999e-06 2023-11-25T04:24:43.595 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.595 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.595 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:43.595 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094950+0000", 2023-11-25T04:24:43.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1329999999999998e-06 2023-11-25T04:24:43.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.596 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.596 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:43.596 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094993+0000", 2023-11-25T04:24:43.596 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3305e-05 2023-11-25T04:24:43.596 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.596 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.597 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:43.597 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095107+0000", 2023-11-25T04:24:43.597 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011449400000000001 2023-11-25T04:24:43.597 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.597 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.597 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:43.597 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095176+0000", 2023-11-25T04:24:43.597 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8330000000000005e-05 2023-11-25T04:24:43.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.598 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:43.598 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107939+0000", 2023-11-25T04:24:43.598 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012763801 2023-11-25T04:24:43.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.598 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:43.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109128+0000", 2023-11-25T04:24:43.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011884770000000001 2023-11-25T04:24:43.599 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.599 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.599 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:43.599 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109166+0000", 2023-11-25T04:24:43.599 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7948999999999999e-05 2023-11-25T04:24:43.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.600 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.600 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:43.600 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109198+0000", 2023-11-25T04:24:43.600 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1656000000000003e-05 2023-11-25T04:24:43.600 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:43.600 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:43.600 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:43.601 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.601 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.601 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2872 4.32 4:4fb002bc:::1000000111d.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:43.601 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.094961+0000", 2023-11-25T04:24:43.601 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.398405552, 2023-11-25T04:24:43.601 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014631564999999999, 2023-11-25T04:24:43.601 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:43.601 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:43.602 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:43.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:43.602 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094961+0000", 2023-11-25T04:24:43.602 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:43.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.602 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.602 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:43.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094961+0000", 2023-11-25T04:24:43.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:43.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.603 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.603 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:43.603 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094961+0000", 2023-11-25T04:24:43.603 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9299999999999999e-07 2023-11-25T04:24:43.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:43.604 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094963+0000", 2023-11-25T04:24:43.604 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9209999999999999e-06 2023-11-25T04:24:43.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.604 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.604 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:43.605 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094968+0000", 2023-11-25T04:24:43.605 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.502e-06 2023-11-25T04:24:43.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.606 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.606 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:43.606 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094973+0000", 2023-11-25T04:24:43.606 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8559999999999998e-06 2023-11-25T04:24:43.606 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:43.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107722+0000", 2023-11-25T04:24:43.607 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012748534000000001 2023-11-25T04:24:43.607 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.607 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.607 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:43.607 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107844+0000", 2023-11-25T04:24:43.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012233400000000001 2023-11-25T04:24:43.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.608 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.608 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:43.608 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109469+0000", 2023-11-25T04:24:43.608 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001625175 2023-11-25T04:24:43.608 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.609 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:43.609 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109518+0000", 2023-11-25T04:24:43.609 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8980000000000002e-05 2023-11-25T04:24:43.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.609 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.610 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:43.610 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109592+0000", 2023-11-25T04:24:43.610 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.3869999999999996e-05 2023-11-25T04:24:43.610 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:43.610 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:43.610 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:43.610 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.611 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.611 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2873 4.13 4:cae4f303:::1000000111f.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:43.611 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.094991+0000", 2023-11-25T04:24:43.611 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.398374960000002, 2023-11-25T04:24:43.611 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015101692, 2023-11-25T04:24:43.611 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:43.612 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:43.612 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:43.612 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.612 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:43.612 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094991+0000", 2023-11-25T04:24:43.612 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:43.613 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.613 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.613 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:43.613 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094991+0000", 2023-11-25T04:24:43.613 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:43.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:43.614 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094992+0000", 2023-11-25T04:24:43.614 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7e-07 2023-11-25T04:24:43.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.614 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.614 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:43.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.094994+0000", 2023-11-25T04:24:43.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.2110000000000001e-06 2023-11-25T04:24:43.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.615 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.615 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:43.615 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095000+0000", 2023-11-25T04:24:43.615 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.241e-06 2023-11-25T04:24:43.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:43.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095006+0000", 2023-11-25T04:24:43.616 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3659999999999997e-06 2023-11-25T04:24:43.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.616 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.616 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:43.616 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108369+0000", 2023-11-25T04:24:43.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.01336247 2023-11-25T04:24:43.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.617 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:43.617 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108492+0000", 2023-11-25T04:24:43.617 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000123192 2023-11-25T04:24:43.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.617 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.618 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:43.618 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109975+0000", 2023-11-25T04:24:43.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001482894 2023-11-25T04:24:43.618 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.618 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.618 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:43.618 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110024+0000", 2023-11-25T04:24:43.618 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9097000000000001e-05 2023-11-25T04:24:43.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.619 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.619 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:43.619 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110093+0000", 2023-11-25T04:24:43.619 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.8751000000000006e-05 2023-11-25T04:24:43.619 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:43.619 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:43.619 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:43.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.620 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.620 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2874 4.14 4:2b557d2e:::10000001121.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:43.620 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095019+0000", 2023-11-25T04:24:43.620 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.398347434000001, 2023-11-25T04:24:43.620 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014998296, 2023-11-25T04:24:43.620 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:43.621 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:43.621 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:43.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.621 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:43.621 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095019+0000", 2023-11-25T04:24:43.621 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:43.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.621 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.622 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:43.622 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095019+0000", 2023-11-25T04:24:43.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:43.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.622 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.622 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:43.622 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095019+0000", 2023-11-25T04:24:43.622 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7300000000000001e-07 2023-11-25T04:24:43.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.623 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:43.623 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095021+0000", 2023-11-25T04:24:43.623 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0099999999999998e-06 2023-11-25T04:24:43.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.623 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.623 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:43.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095028+0000", 2023-11-25T04:24:43.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.7870000000000004e-06 2023-11-25T04:24:43.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.624 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:43.624 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095033+0000", 2023-11-25T04:24:43.624 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2039999999999996e-06 2023-11-25T04:24:43.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:43.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107740+0000", 2023-11-25T04:24:43.625 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012706693 2023-11-25T04:24:43.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.625 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:43.625 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107817+0000", 2023-11-25T04:24:43.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7599e-05 2023-11-25T04:24:43.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.626 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:43.626 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107961+0000", 2023-11-25T04:24:43.626 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000143194 2023-11-25T04:24:43.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.626 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.627 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:43.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109364+0000", 2023-11-25T04:24:43.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001403226 2023-11-25T04:24:43.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.627 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.627 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:43.627 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109507+0000", 2023-11-25T04:24:43.627 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000142991 2023-11-25T04:24:43.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:43.628 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109986+0000", 2023-11-25T04:24:43.628 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00047906899999999999 2023-11-25T04:24:43.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.628 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.628 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:43.629 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110017+0000", 2023-11-25T04:24:43.629 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1050000000000003e-05 2023-11-25T04:24:43.629 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:43.629 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:43.629 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:43.629 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.629 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.629 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2875 4.d 4:b17d5dc6:::10000001123.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:43.630 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095049+0000", 2023-11-25T04:24:43.630 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.398317509999998, 2023-11-25T04:24:43.630 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014882886, 2023-11-25T04:24:43.630 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:43.630 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:43.630 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:43.630 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.630 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:43.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095049+0000", 2023-11-25T04:24:43.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:43.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.631 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.631 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:43.631 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095049+0000", 2023-11-25T04:24:43.631 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:43.631 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:43.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095049+0000", 2023-11-25T04:24:43.632 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6100000000000001e-07 2023-11-25T04:24:43.632 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.632 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.632 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:43.632 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095051+0000", 2023-11-25T04:24:43.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1449999999999998e-06 2023-11-25T04:24:43.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.633 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:43.633 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095058+0000", 2023-11-25T04:24:43.633 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.601e-06 2023-11-25T04:24:43.633 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.633 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.634 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:43.634 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095064+0000", 2023-11-25T04:24:43.634 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1079999999999998e-06 2023-11-25T04:24:43.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.635 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:43.635 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108015+0000", 2023-11-25T04:24:43.635 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012951582 2023-11-25T04:24:43.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.635 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:43.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108260+0000", 2023-11-25T04:24:43.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00024445199999999998 2023-11-25T04:24:43.636 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.636 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.636 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:43.636 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109825+0000", 2023-11-25T04:24:43.636 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0015651759999999999 2023-11-25T04:24:43.637 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.637 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.637 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:43.638 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109866+0000", 2023-11-25T04:24:43.638 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0629e-05 2023-11-25T04:24:43.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.639 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.639 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:43.639 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109931+0000", 2023-11-25T04:24:43.640 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5732e-05 2023-11-25T04:24:43.640 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:43.640 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:43.641 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:43.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.641 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.641 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2876 4.3b 4:dc871776:::10000001125.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:43.641 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095077+0000", 2023-11-25T04:24:43.641 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.398288889, 2023-11-25T04:24:43.641 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014500944, 2023-11-25T04:24:43.641 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:43.642 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:43.642 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:43.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.642 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:43.642 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095077+0000", 2023-11-25T04:24:43.642 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:43.642 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.642 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:43.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095077+0000", 2023-11-25T04:24:43.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:43.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.643 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.643 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:43.643 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095078+0000", 2023-11-25T04:24:43.643 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9799999999999999e-07 2023-11-25T04:24:43.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.644 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:43.644 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095079+0000", 2023-11-25T04:24:43.644 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.877e-06 2023-11-25T04:24:43.644 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.644 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.644 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:43.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095086+0000", 2023-11-25T04:24:43.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2439999999999998e-06 2023-11-25T04:24:43.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.645 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.645 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:43.645 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095094+0000", 2023-11-25T04:24:43.645 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9710000000000004e-06 2023-11-25T04:24:43.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:43.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107732+0000", 2023-11-25T04:24:43.646 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012638104000000001 2023-11-25T04:24:43.646 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.646 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.646 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:43.646 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107854+0000", 2023-11-25T04:24:43.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012261600000000001 2023-11-25T04:24:43.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.647 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:43.647 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107941+0000", 2023-11-25T04:24:43.647 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6935999999999996e-05 2023-11-25T04:24:43.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.647 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.648 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:43.648 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108871+0000", 2023-11-25T04:24:43.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00092955499999999999 2023-11-25T04:24:43.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.648 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.648 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:43.648 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109483+0000", 2023-11-25T04:24:43.648 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00061206100000000001 2023-11-25T04:24:43.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.649 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:43.649 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109527+0000", 2023-11-25T04:24:43.649 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.4112e-05 2023-11-25T04:24:43.649 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.649 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.649 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:43.650 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109578+0000", 2023-11-25T04:24:43.650 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1069999999999997e-05 2023-11-25T04:24:43.650 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:43.650 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:43.650 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:43.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.650 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.650 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2878 4.3e 4:7ff98b9d:::10000001129.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:43.651 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095101+0000", 2023-11-25T04:24:43.651 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.398265106, 2023-11-25T04:24:43.651 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.015015137, 2023-11-25T04:24:43.651 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:43.651 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:43.651 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:43.651 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.651 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:43.652 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095101+0000", 2023-11-25T04:24:43.652 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:43.652 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.652 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.652 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:43.652 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095101+0000", 2023-11-25T04:24:43.652 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:43.652 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.653 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:43.653 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095101+0000", 2023-11-25T04:24:43.653 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7399999999999999e-07 2023-11-25T04:24:43.653 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.653 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.653 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:43.653 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095103+0000", 2023-11-25T04:24:43.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7600000000000001e-06 2023-11-25T04:24:43.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.654 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:43.654 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095109+0000", 2023-11-25T04:24:43.654 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.8189999999999997e-06 2023-11-25T04:24:43.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.654 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:43.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095114+0000", 2023-11-25T04:24:43.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6770000000000003e-06 2023-11-25T04:24:43.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.655 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.655 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:43.655 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107724+0000", 2023-11-25T04:24:43.655 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012610909999999999 2023-11-25T04:24:43.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.656 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:43.656 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108483+0000", 2023-11-25T04:24:43.656 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000758407 2023-11-25T04:24:43.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.656 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.656 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:43.657 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108568+0000", 2023-11-25T04:24:43.657 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.4707000000000001e-05 2023-11-25T04:24:43.657 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.657 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.657 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:43.657 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109687+0000", 2023-11-25T04:24:43.657 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011192750000000001 2023-11-25T04:24:43.657 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.657 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.658 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:43.658 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110049+0000", 2023-11-25T04:24:43.658 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00036210600000000001 2023-11-25T04:24:43.658 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.658 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.658 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:43.658 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110086+0000", 2023-11-25T04:24:43.659 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7500999999999999e-05 2023-11-25T04:24:43.659 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.659 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.659 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:43.659 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110116+0000", 2023-11-25T04:24:43.659 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9601e-05 2023-11-25T04:24:43.659 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:43.659 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:43.659 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:43.660 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.660 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.660 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2879 4.2a 4:5524ace2:::1000000112b.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:43.660 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095120+0000", 2023-11-25T04:24:43.660 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.398246325999999, 2023-11-25T04:24:43.660 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014587937, 2023-11-25T04:24:43.660 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:43.660 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:43.661 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:43.661 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.661 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:43.661 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095120+0000", 2023-11-25T04:24:43.661 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:43.661 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.661 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.661 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:43.662 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095120+0000", 2023-11-25T04:24:43.662 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:43.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.662 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.662 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:43.662 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095120+0000", 2023-11-25T04:24:43.662 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5600000000000001e-07 2023-11-25T04:24:43.662 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:43.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095122+0000", 2023-11-25T04:24:43.663 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.694e-06 2023-11-25T04:24:43.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.663 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:43.663 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095127+0000", 2023-11-25T04:24:43.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.5099999999999998e-06 2023-11-25T04:24:43.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.664 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:43.664 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095132+0000", 2023-11-25T04:24:43.664 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6210000000000003e-06 2023-11-25T04:24:43.664 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.664 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.665 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:43.665 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107729+0000", 2023-11-25T04:24:43.665 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012596931 2023-11-25T04:24:43.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.665 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.665 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:43.665 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107856+0000", 2023-11-25T04:24:43.665 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012707699999999999 2023-11-25T04:24:43.666 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.666 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.666 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:43.666 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109600+0000", 2023-11-25T04:24:43.666 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001744298 2023-11-25T04:24:43.666 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.666 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.666 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:43.667 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109641+0000", 2023-11-25T04:24:43.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.0769000000000002e-05 2023-11-25T04:24:43.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.667 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.667 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:43.667 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109708+0000", 2023-11-25T04:24:43.667 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6680999999999994e-05 2023-11-25T04:24:43.667 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:43.668 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:43.668 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:43.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.668 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.668 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2882 4.1c 4:3a7c2e07:::10000001131.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:43.668 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095141+0000", 2023-11-25T04:24:43.668 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.398224951, 2023-11-25T04:24:43.668 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014949469999999999, 2023-11-25T04:24:43.669 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:43.669 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:43.669 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:43.669 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.669 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:43.669 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095141+0000", 2023-11-25T04:24:43.669 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:43.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.670 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:43.670 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095141+0000", 2023-11-25T04:24:43.670 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:43.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.670 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.670 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:43.670 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095142+0000", 2023-11-25T04:24:43.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1899999999999998e-07 2023-11-25T04:24:43.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.671 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:43.671 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095143+0000", 2023-11-25T04:24:43.671 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7770000000000001e-06 2023-11-25T04:24:43.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.671 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.672 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:43.672 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095149+0000", 2023-11-25T04:24:43.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9909999999999999e-06 2023-11-25T04:24:43.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.672 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.672 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:43.672 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095154+0000", 2023-11-25T04:24:43.672 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3880000000000002e-06 2023-11-25T04:24:43.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.673 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.673 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:43.673 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107736+0000", 2023-11-25T04:24:43.673 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012582477999999999 2023-11-25T04:24:43.673 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.673 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.673 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:43.673 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107868+0000", 2023-11-25T04:24:43.674 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00013134400000000001 2023-11-25T04:24:43.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.674 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.674 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:43.674 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107959+0000", 2023-11-25T04:24:43.674 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.1253000000000002e-05 2023-11-25T04:24:43.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.674 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.675 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:43.675 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109325+0000", 2023-11-25T04:24:43.675 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0013662520000000001 2023-11-25T04:24:43.675 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.675 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.675 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:43.676 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110023+0000", 2023-11-25T04:24:43.676 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00069838799999999998 2023-11-25T04:24:43.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.676 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.676 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:43.676 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110061+0000", 2023-11-25T04:24:43.676 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7867999999999997e-05 2023-11-25T04:24:43.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.677 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.677 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:43.677 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110091+0000", 2023-11-25T04:24:43.677 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.9312e-05 2023-11-25T04:24:43.677 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:43.677 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:43.677 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:43.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.678 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.678 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2883 4.3e 4:7d82be99:::10000001133.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:43.678 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095942+0000", 2023-11-25T04:24:43.678 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.397423665000002, 2023-11-25T04:24:43.678 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.014290573000000001, 2023-11-25T04:24:43.678 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:43.678 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:43.678 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:43.679 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.679 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:43.679 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095942+0000", 2023-11-25T04:24:43.679 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:43.679 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.679 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.679 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:43.679 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095942+0000", 2023-11-25T04:24:43.680 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:43.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.680 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.680 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:43.680 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095944+0000", 2023-11-25T04:24:43.680 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.632e-06 2023-11-25T04:24:43.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.680 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.680 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:43.681 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095947+0000", 2023-11-25T04:24:43.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.7760000000000002e-06 2023-11-25T04:24:43.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.681 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.681 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:43.681 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095954+0000", 2023-11-25T04:24:43.681 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.5379999999999997e-06 2023-11-25T04:24:43.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:43.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095960+0000", 2023-11-25T04:24:43.682 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9209999999999997e-06 2023-11-25T04:24:43.682 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.682 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.682 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:43.682 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108809+0000", 2023-11-25T04:24:43.683 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.012848972 2023-11-25T04:24:43.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.683 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:43.683 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108921+0000", 2023-11-25T04:24:43.683 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000111767 2023-11-25T04:24:43.683 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.683 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.684 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "waiting for subops from 2", 2023-11-25T04:24:43.684 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109008+0000", 2023-11-25T04:24:43.684 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.6576999999999995e-05 2023-11-25T04:24:43.684 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.684 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.684 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:43.684 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110124+0000", 2023-11-25T04:24:43.684 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0011163709999999999 2023-11-25T04:24:43.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.685 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.685 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:43.685 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110155+0000", 2023-11-25T04:24:43.685 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.1248999999999998e-05 2023-11-25T04:24:43.685 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.685 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.685 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:43.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110202+0000", 2023-11-25T04:24:43.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7097e-05 2023-11-25T04:24:43.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.686 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.686 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:43.686 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.110233+0000", 2023-11-25T04:24:43.686 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0673000000000003e-05 2023-11-25T04:24:43.686 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:43.687 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:43.687 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:43.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.687 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.687 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:2886 4.25 4:a579baeb:::10000001139.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:43.687 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:25.095994+0000", 2023-11-25T04:24:43.687 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 18.397372337, 2023-11-25T04:24:43.687 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.013825356, 2023-11-25T04:24:43.688 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:43.688 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:43.688 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:43.688 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.688 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:43.688 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095994+0000", 2023-11-25T04:24:43.688 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:43.688 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.689 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:43.689 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095994+0000", 2023-11-25T04:24:43.689 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:43.689 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.689 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:43.689 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095996+0000", 2023-11-25T04:24:43.690 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.8780000000000001e-06 2023-11-25T04:24:43.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.690 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.690 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:43.690 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.095998+0000", 2023-11-25T04:24:43.690 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.1560000000000002e-06 2023-11-25T04:24:43.690 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.690 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.691 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:43.691 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.096003+0000", 2023-11-25T04:24:43.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1080000000000001e-06 2023-11-25T04:24:43.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.691 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.691 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:43.691 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.096010+0000", 2023-11-25T04:24:43.691 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0650000000000001e-06 2023-11-25T04:24:43.691 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.692 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.692 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:43.692 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.107742+0000", 2023-11-25T04:24:43.692 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.011731691000000001 2023-11-25T04:24:43.692 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.692 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.692 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:43.692 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.108063+0000", 2023-11-25T04:24:43.693 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00032168600000000001 2023-11-25T04:24:43.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.693 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "op_commit", 2023-11-25T04:24:43.693 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109689+0000", 2023-11-25T04:24:43.693 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001625556 2023-11-25T04:24:43.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.693 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.694 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "commit_sent", 2023-11-25T04:24:43.694 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109740+0000", 2023-11-25T04:24:43.694 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1125000000000003e-05 2023-11-25T04:24:43.694 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:43.694 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:43.694 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:43.694 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:25.109819+0000", 2023-11-25T04:24:43.694 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.9091000000000006e-05 2023-11-25T04:24:43.695 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:43.695 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:43.695 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:43.695 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:43.695 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:43.695 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:43.697 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2023-11-25T04:24:43.698 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:24:43.698 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:43.695+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:43.698 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-25T04:24:43.799 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:43.797+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:43.816 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:43.816 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:24:43.816 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2023-11-25T04:24:43.816 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:43.830 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:24:43.899 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:43.897+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:43.936 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:43.936 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:24:43.936 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2023-11-25T04:24:43.937 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2023-11-25T04:24:43.937 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:43.944 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:24:43.996 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2023-11-25T04:24:43.999 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:43.998+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:44.009 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-25T04:24:44.092 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:44.092 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2023-11-25T04:24:44.092 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2023-11-25T04:24:44.092 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2023-11-25T04:24:44.092 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.092 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:4 3.1f 3:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:44.093 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:24:44.093 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 117.05751128999999, 2023-11-25T04:24:44.093 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01072943, 2023-11-25T04:24:44.093 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:44.093 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:44.093 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:44.093 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.093 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:44.094 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:24:44.094 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:44.094 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.094 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.094 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:44.094 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031828+0000", 2023-11-25T04:24:44.094 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:44.095 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.095 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.095 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:44.095 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031829+0000", 2023-11-25T04:24:44.095 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.666e-06 2023-11-25T04:24:44.095 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.095 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.096 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:44.096 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031839+0000", 2023-11-25T04:24:44.096 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9490000000000008e-06 2023-11-25T04:24:44.096 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.096 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.096 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:44.096 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031857+0000", 2023-11-25T04:24:44.097 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8093000000000001e-05 2023-11-25T04:24:44.097 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.097 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.097 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:44.097 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.031882+0000", 2023-11-25T04:24:44.098 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4294000000000002e-05 2023-11-25T04:24:44.098 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.098 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.098 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:44.098 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033130+0000", 2023-11-25T04:24:44.098 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001248264 2023-11-25T04:24:44.098 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.098 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.099 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:44.099 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033426+0000", 2023-11-25T04:24:44.099 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029620999999999998 2023-11-25T04:24:44.099 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.100 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.100 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:44.100 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033814+0000", 2023-11-25T04:24:44.100 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000387742 2023-11-25T04:24:44.101 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.101 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.101 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:44.101 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.036793+0000", 2023-11-25T04:24:44.101 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002979598 2023-11-25T04:24:44.101 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.101 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.101 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:44.102 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042455+0000", 2023-11-25T04:24:44.102 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0056619260000000003 2023-11-25T04:24:44.102 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.102 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.102 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:44.102 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042502+0000", 2023-11-25T04:24:44.103 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7139999999999999e-05 2023-11-25T04:24:44.103 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.103 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.103 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:44.103 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.042557+0000", 2023-11-25T04:24:44.103 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4548e-05 2023-11-25T04:24:44.103 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:44.103 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:44.104 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:44.104 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.104 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.104 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:5 3.1f 3:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:44.104 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:24:44.104 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 117.05631275499999, 2023-11-25T04:24:44.104 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011505202000000001, 2023-11-25T04:24:44.104 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:44.105 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:44.105 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:44.105 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.105 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:44.105 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:24:44.105 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:44.105 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.105 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.106 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:44.106 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033026+0000", 2023-11-25T04:24:44.106 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:44.106 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.106 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.106 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:44.106 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033027+0000", 2023-11-25T04:24:44.106 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0699999999999999e-06 2023-11-25T04:24:44.107 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.107 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.107 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:44.107 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033035+0000", 2023-11-25T04:24:44.107 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0830000000000004e-06 2023-11-25T04:24:44.107 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.107 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.108 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:44.108 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033059+0000", 2023-11-25T04:24:44.108 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3975e-05 2023-11-25T04:24:44.108 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.108 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.109 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:44.109 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033482+0000", 2023-11-25T04:24:44.109 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042253700000000001 2023-11-25T04:24:44.109 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.109 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.109 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:44.109 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034520+0000", 2023-11-25T04:24:44.109 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001037991 2023-11-25T04:24:44.110 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.110 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.110 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:44.110 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034611+0000", 2023-11-25T04:24:44.110 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1113999999999995e-05 2023-11-25T04:24:44.110 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.110 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.111 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:44.111 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034769+0000", 2023-11-25T04:24:44.111 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000157641 2023-11-25T04:24:44.111 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.111 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.111 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:44.111 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038292+0000", 2023-11-25T04:24:44.111 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035231160000000002 2023-11-25T04:24:44.112 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.112 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.112 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:44.112 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044440+0000", 2023-11-25T04:24:44.112 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0061486980000000002 2023-11-25T04:24:44.112 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.112 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.112 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:44.113 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044495+0000", 2023-11-25T04:24:44.113 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4339000000000003e-05 2023-11-25T04:24:44.113 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.113 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.113 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:44.113 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044531+0000", 2023-11-25T04:24:44.113 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6637999999999997e-05 2023-11-25T04:24:44.113 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:44.114 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:44.114 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:44.114 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.114 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.114 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:26 3.1f 3:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:44.114 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:44.114 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 117.055723993, 2023-11-25T04:24:44.114 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011086945000000001, 2023-11-25T04:24:44.115 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:44.115 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:44.115 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:44.115 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.115 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:44.115 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:44.115 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:44.115 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.116 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.116 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:44.116 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:44.116 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:44.116 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.116 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.116 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:44.117 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033615+0000", 2023-11-25T04:24:44.117 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1699999999999998e-07 2023-11-25T04:24:44.117 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.117 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.117 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:44.118 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033627+0000", 2023-11-25T04:24:44.118 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.163e-05 2023-11-25T04:24:44.118 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.118 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.122 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:44.123 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033636+0000", 2023-11-25T04:24:44.123 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6859999999999995e-06 2023-11-25T04:24:44.123 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.123 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.123 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:44.123 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033642+0000", 2023-11-25T04:24:44.123 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3149999999999997e-06 2023-11-25T04:24:44.123 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.124 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.124 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:44.124 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035414+0000", 2023-11-25T04:24:44.124 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0017716380000000001 2023-11-25T04:24:44.124 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.124 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.124 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:44.124 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035539+0000", 2023-11-25T04:24:44.125 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000125477 2023-11-25T04:24:44.125 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.125 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.125 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:44.125 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035672+0000", 2023-11-25T04:24:44.125 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000133316 2023-11-25T04:24:44.125 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.126 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.126 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:44.126 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.038649+0000", 2023-11-25T04:24:44.126 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029764679999999999 2023-11-25T04:24:44.126 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.126 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.126 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:44.126 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044578+0000", 2023-11-25T04:24:44.127 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0059293510000000002 2023-11-25T04:24:44.127 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.127 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.127 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:44.127 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044651+0000", 2023-11-25T04:24:44.127 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2513000000000002e-05 2023-11-25T04:24:44.127 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.127 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.128 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:44.128 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044702+0000", 2023-11-25T04:24:44.128 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1034e-05 2023-11-25T04:24:44.128 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:44.128 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:44.128 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:44.128 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.128 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.129 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:29 3.b 3:d0630e4c:::mds0_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:44.129 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:44.129 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 117.055686073, 2023-11-25T04:24:44.129 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010980227, 2023-11-25T04:24:44.129 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:44.129 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:44.129 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:44.129 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.130 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:44.130 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:44.130 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:44.130 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.130 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.130 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:44.130 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:44.130 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:44.131 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.131 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.131 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:44.131 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033653+0000", 2023-11-25T04:24:44.131 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3499999999999996e-07 2023-11-25T04:24:44.131 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.131 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.131 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:44.132 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033658+0000", 2023-11-25T04:24:44.132 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1280000000000001e-06 2023-11-25T04:24:44.132 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.132 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.133 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:44.133 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033667+0000", 2023-11-25T04:24:44.133 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1430000000000004e-06 2023-11-25T04:24:44.133 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.133 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.133 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:44.133 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033678+0000", 2023-11-25T04:24:44.133 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0906e-05 2023-11-25T04:24:44.134 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.134 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.134 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:44.134 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034456+0000", 2023-11-25T04:24:44.134 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00077833899999999996 2023-11-25T04:24:44.134 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.134 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.135 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:44.135 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034575+0000", 2023-11-25T04:24:44.135 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011926000000000001 2023-11-25T04:24:44.135 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.135 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.135 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0", 2023-11-25T04:24:44.135 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035001+0000", 2023-11-25T04:24:44.135 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042550999999999998 2023-11-25T04:24:44.136 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.136 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.136 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:44.136 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.037923+0000", 2023-11-25T04:24:44.136 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029219670000000001 2023-11-25T04:24:44.136 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.136 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.136 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:44.136 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044538+0000", 2023-11-25T04:24:44.137 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0066152349999999997 2023-11-25T04:24:44.137 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.137 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.137 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:44.137 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044594+0000", 2023-11-25T04:24:44.137 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5593000000000001e-05 2023-11-25T04:24:44.137 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.138 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.138 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:44.138 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044633+0000", 2023-11-25T04:24:44.138 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9511000000000002e-05 2023-11-25T04:24:44.138 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:44.138 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:44.138 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:44.138 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.139 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.139 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:34 3.b 3:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e18)", 2023-11-25T04:24:44.139 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:24:44.139 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 117.05543596, 2023-11-25T04:24:44.139 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010920445000000001, 2023-11-25T04:24:44.139 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:44.139 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:44.139 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:44.140 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.140 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:44.140 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:24:44.140 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:44.140 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.140 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.140 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:44.140 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033903+0000", 2023-11-25T04:24:44.141 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:44.141 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.141 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.141 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:44.141 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033904+0000", 2023-11-25T04:24:44.141 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6599999999999994e-07 2023-11-25T04:24:44.141 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.141 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.142 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:44.142 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033909+0000", 2023-11-25T04:24:44.142 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5639999999999996e-06 2023-11-25T04:24:44.142 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.142 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.142 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:44.142 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.033923+0000", 2023-11-25T04:24:44.142 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3028e-05 2023-11-25T04:24:44.143 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.143 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.143 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:44.143 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.034043+0000", 2023-11-25T04:24:44.143 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000120328 2023-11-25T04:24:44.143 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.143 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.143 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:44.144 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035576+0000", 2023-11-25T04:24:44.144 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015333720000000001 2023-11-25T04:24:44.144 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.144 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.144 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:44.144 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.035699+0000", 2023-11-25T04:24:44.144 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000123001 2023-11-25T04:24:44.144 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.144 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.145 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:44.145 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:22:47.044823+0000", 2023-11-25T04:24:44.145 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0091241859999999994 2023-11-25T04:24:44.145 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:44.145 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:44.145 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:44.145 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.145 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.146 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:118 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals,omap-rm-keys] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e44)", 2023-11-25T04:24:44.146 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:17.112648+0000", 2023-11-25T04:24:44.146 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 26.976690618999999, 2023-11-25T04:24:44.146 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011250779000000001, 2023-11-25T04:24:44.146 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:44.146 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:44.146 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:44.147 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.147 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:44.147 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.112648+0000", 2023-11-25T04:24:44.147 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:44.147 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.147 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.147 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:44.147 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.112648+0000", 2023-11-25T04:24:44.147 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:44.148 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.148 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.148 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:44.148 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.112649+0000", 2023-11-25T04:24:44.148 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0070000000000001e-06 2023-11-25T04:24:44.148 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.148 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.149 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:44.149 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113182+0000", 2023-11-25T04:24:44.149 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00053268100000000002 2023-11-25T04:24:44.149 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.149 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.149 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:44.149 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113196+0000", 2023-11-25T04:24:44.149 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3742e-05 2023-11-25T04:24:44.150 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.150 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.150 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:44.150 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113398+0000", 2023-11-25T04:24:44.150 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000202726 2023-11-25T04:24:44.150 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.150 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.150 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:44.151 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113447+0000", 2023-11-25T04:24:44.151 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8655000000000001e-05 2023-11-25T04:24:44.151 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.151 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.151 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:44.151 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.113529+0000", 2023-11-25T04:24:44.151 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1891999999999995e-05 2023-11-25T04:24:44.151 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.152 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.152 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:44.152 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123819+0000", 2023-11-25T04:24:44.152 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010290192 2023-11-25T04:24:44.152 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.152 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.152 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:44.152 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123852+0000", 2023-11-25T04:24:44.153 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2425999999999998e-05 2023-11-25T04:24:44.153 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.153 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.153 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:44.153 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:17.123899+0000", 2023-11-25T04:24:44.153 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7457999999999999e-05 2023-11-25T04:24:44.153 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:44.153 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:44.154 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:44.154 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.154 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.154 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:154 3.1e 3:7bf7e7b5:::200.00000006:head [write 0~4194304 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:44.154 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:21.886430+0000", 2023-11-25T04:24:44.154 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 22.202909352999999, 2023-11-25T04:24:44.154 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.017800107999999999, 2023-11-25T04:24:44.154 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:44.155 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:44.155 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:44.155 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.155 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:44.155 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.886430+0000", 2023-11-25T04:24:44.155 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:44.155 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.155 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.156 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:44.156 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.886430+0000", 2023-11-25T04:24:44.156 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:44.156 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.156 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.156 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:44.156 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.886431+0000", 2023-11-25T04:24:44.156 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.178e-06 2023-11-25T04:24:44.157 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.157 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.157 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:44.157 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.892277+0000", 2023-11-25T04:24:44.158 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0058467409999999999 2023-11-25T04:24:44.158 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.158 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.158 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:44.158 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.892288+0000", 2023-11-25T04:24:44.158 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0183000000000001e-05 2023-11-25T04:24:44.160 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.160 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.160 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:44.160 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.892292+0000", 2023-11-25T04:24:44.160 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7829999999999998e-06 2023-11-25T04:24:44.160 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.161 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.161 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:44.161 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.892362+0000", 2023-11-25T04:24:44.161 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0034000000000001e-05 2023-11-25T04:24:44.161 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.161 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.161 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:44.162 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.892432+0000", 2023-11-25T04:24:44.162 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9648999999999997e-05 2023-11-25T04:24:44.162 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.162 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.162 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:44.162 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.892550+0000", 2023-11-25T04:24:44.162 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011825 2023-11-25T04:24:44.162 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.163 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.163 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:44.163 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.899106+0000", 2023-11-25T04:24:44.163 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0065556759999999999 2023-11-25T04:24:44.163 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.163 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.164 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:44.164 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.904145+0000", 2023-11-25T04:24:44.164 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0050393490000000003 2023-11-25T04:24:44.164 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.164 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.164 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:44.165 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.904200+0000", 2023-11-25T04:24:44.165 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4234999999999998e-05 2023-11-25T04:24:44.165 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.165 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.165 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:44.165 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:21.904230+0000", 2023-11-25T04:24:44.165 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0029999999999999e-05 2023-11-25T04:24:44.166 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:44.166 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:44.166 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:44.166 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.166 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.166 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:576 3.6 3:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals,omap-rm-keys] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:44.166 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:22.111407+0000", 2023-11-25T04:24:44.167 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 21.977931747, 2023-11-25T04:24:44.167 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.018987317, 2023-11-25T04:24:44.167 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:44.167 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:44.167 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:44.167 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.167 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:44.168 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111407+0000", 2023-11-25T04:24:44.168 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:44.168 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.168 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.168 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:44.168 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111407+0000", 2023-11-25T04:24:44.168 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:44.168 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.169 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.169 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:44.169 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111408+0000", 2023-11-25T04:24:44.169 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8000000000000004e-07 2023-11-25T04:24:44.169 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.169 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.169 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:44.169 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111659+0000", 2023-11-25T04:24:44.170 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000250537 2023-11-25T04:24:44.170 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.170 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.170 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:44.170 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111666+0000", 2023-11-25T04:24:44.170 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.973e-06 2023-11-25T04:24:44.172 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.172 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.172 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:44.172 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111670+0000", 2023-11-25T04:24:44.172 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.668e-06 2023-11-25T04:24:44.172 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.172 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.173 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:44.173 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111709+0000", 2023-11-25T04:24:44.173 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8800000000000001e-05 2023-11-25T04:24:44.173 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.173 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.173 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:44.173 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.111761+0000", 2023-11-25T04:24:44.173 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1495000000000001e-05 2023-11-25T04:24:44.174 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.174 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.174 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:44.174 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.130255+0000", 2023-11-25T04:24:44.174 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.018494345999999998 2023-11-25T04:24:44.174 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.174 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.175 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:44.175 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.130311+0000", 2023-11-25T04:24:44.175 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5902000000000001e-05 2023-11-25T04:24:44.175 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.175 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.175 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:44.175 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.130394+0000", 2023-11-25T04:24:44.175 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3715999999999996e-05 2023-11-25T04:24:44.176 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:44.176 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:44.176 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:44.176 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.176 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.176 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:1067 4.21 4:869a7600:::100000011fa.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:44.176 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:22.155038+0000", 2023-11-25T04:24:44.177 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 21.934300408999999, 2023-11-25T04:24:44.177 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011313148, 2023-11-25T04:24:44.177 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:44.177 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:44.177 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:44.177 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.177 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:44.177 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155038+0000", 2023-11-25T04:24:44.178 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:44.178 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.178 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.178 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:44.178 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155038+0000", 2023-11-25T04:24:44.178 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:44.178 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.178 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.179 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:44.179 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155039+0000", 2023-11-25T04:24:44.179 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0200000000000002e-07 2023-11-25T04:24:44.179 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.179 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.179 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:44.179 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155049+0000", 2023-11-25T04:24:44.179 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0295000000000001e-05 2023-11-25T04:24:44.180 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.180 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.180 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:44.180 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155058+0000", 2023-11-25T04:24:44.180 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.208e-06 2023-11-25T04:24:44.180 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.180 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.180 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:44.181 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155064+0000", 2023-11-25T04:24:44.181 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4489999999999996e-06 2023-11-25T04:24:44.181 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.181 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.181 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:44.181 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155104+0000", 2023-11-25T04:24:44.181 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0164999999999999e-05 2023-11-25T04:24:44.182 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.182 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.182 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:44.182 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155147+0000", 2023-11-25T04:24:44.182 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2576e-05 2023-11-25T04:24:44.182 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.182 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.183 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:44.183 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.155212+0000", 2023-11-25T04:24:44.183 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5516000000000001e-05 2023-11-25T04:24:44.183 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.183 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.183 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:44.184 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.156025+0000", 2023-11-25T04:24:44.184 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00081248599999999996 2023-11-25T04:24:44.184 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.184 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.184 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:44.184 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.166250+0000", 2023-11-25T04:24:44.184 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010224736999999999 2023-11-25T04:24:44.185 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.185 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.185 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:44.185 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.166319+0000", 2023-11-25T04:24:44.185 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9145000000000003e-05 2023-11-25T04:24:44.186 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.186 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.186 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:44.187 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.166352+0000", 2023-11-25T04:24:44.187 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2969000000000002e-05 2023-11-25T04:24:44.187 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:44.187 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:44.187 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:44.187 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.187 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.188 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:1102 4.2f 4:f543b94c:::1000000121d.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:44.188 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:22.158246+0000", 2023-11-25T04:24:44.188 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 21.931092885999998, 2023-11-25T04:24:44.188 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010094396, 2023-11-25T04:24:44.188 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:44.189 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:44.189 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:44.189 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.189 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:44.189 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158246+0000", 2023-11-25T04:24:44.189 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:44.189 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.190 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.190 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:44.190 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158246+0000", 2023-11-25T04:24:44.190 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:44.190 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.190 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.190 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:44.190 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158246+0000", 2023-11-25T04:24:44.191 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5299999999999999e-07 2023-11-25T04:24:44.191 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.191 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.191 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:44.191 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158249+0000", 2023-11-25T04:24:44.191 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2840000000000001e-06 2023-11-25T04:24:44.191 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.192 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.192 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:44.192 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158254+0000", 2023-11-25T04:24:44.192 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3449999999999996e-06 2023-11-25T04:24:44.192 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.192 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.192 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:44.193 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158258+0000", 2023-11-25T04:24:44.193 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5030000000000002e-06 2023-11-25T04:24:44.193 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.193 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.193 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:44.193 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158302+0000", 2023-11-25T04:24:44.193 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3908e-05 2023-11-25T04:24:44.194 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.194 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.194 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:44.194 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158398+0000", 2023-11-25T04:24:44.194 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6898000000000006e-05 2023-11-25T04:24:44.194 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.194 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.195 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:44.195 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158480+0000", 2023-11-25T04:24:44.195 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1578999999999999e-05 2023-11-25T04:24:44.195 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.195 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.195 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:44.195 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.159229+0000", 2023-11-25T04:24:44.195 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000749049 2023-11-25T04:24:44.196 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.196 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.196 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:44.196 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.168228+0000", 2023-11-25T04:24:44.196 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0089992930000000002 2023-11-25T04:24:44.196 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.196 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.197 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:44.197 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.168310+0000", 2023-11-25T04:24:44.198 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1348999999999997e-05 2023-11-25T04:24:44.198 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.198 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.198 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:44.199 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.168340+0000", 2023-11-25T04:24:44.199 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0735000000000003e-05 2023-11-25T04:24:44.199 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:44.199 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:44.199 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:44.199 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.199 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.200 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:1105 4.23 4:c6ff07b4:::10000001220.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e46)", 2023-11-25T04:24:44.200 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:22.158714+0000", 2023-11-25T04:24:44.200 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 21.930624772000002, 2023-11-25T04:24:44.200 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010412490999999999, 2023-11-25T04:24:44.200 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:44.200 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:44.200 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:44.200 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.201 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:44.201 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158714+0000", 2023-11-25T04:24:44.201 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:44.201 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.201 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.201 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:44.201 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158714+0000", 2023-11-25T04:24:44.202 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:44.202 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.202 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.202 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:44.202 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158715+0000", 2023-11-25T04:24:44.202 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4099999999999998e-07 2023-11-25T04:24:44.202 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.203 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.203 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:44.203 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158718+0000", 2023-11-25T04:24:44.203 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4319999999999999e-06 2023-11-25T04:24:44.203 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.203 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.203 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:44.204 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158729+0000", 2023-11-25T04:24:44.204 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.078e-05 2023-11-25T04:24:44.204 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.204 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.204 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:44.204 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158736+0000", 2023-11-25T04:24:44.204 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4649999999999997e-06 2023-11-25T04:24:44.204 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.205 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.205 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:44.205 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158779+0000", 2023-11-25T04:24:44.205 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3049000000000001e-05 2023-11-25T04:24:44.205 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.205 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.205 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:44.206 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158865+0000", 2023-11-25T04:24:44.206 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5138999999999999e-05 2023-11-25T04:24:44.206 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.206 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.206 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:44.206 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.158983+0000", 2023-11-25T04:24:44.206 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000118658 2023-11-25T04:24:44.207 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.207 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.207 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:44.207 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.159645+0000", 2023-11-25T04:24:44.207 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00066193999999999997 2023-11-25T04:24:44.207 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.208 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.208 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:44.208 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.169046+0000", 2023-11-25T04:24:44.208 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0094005400000000006 2023-11-25T04:24:44.208 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.208 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.209 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:44.209 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.169092+0000", 2023-11-25T04:24:44.209 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5840000000000002e-05 2023-11-25T04:24:44.209 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.210 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.210 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:44.210 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:22.169127+0000", 2023-11-25T04:24:44.210 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5006999999999999e-05 2023-11-25T04:24:44.210 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:44.210 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:44.211 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:44.211 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.211 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.211 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2841 4.4 4:22111d42:::100000010df.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:44.211 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.092102+0000", 2023-11-25T04:24:44.211 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 18.997237297000002, 2023-11-25T04:24:44.211 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.016449783999999999, 2023-11-25T04:24:44.211 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:44.212 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:44.212 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:44.212 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.212 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:44.212 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092102+0000", 2023-11-25T04:24:44.212 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:44.212 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.212 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.213 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:44.213 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092102+0000", 2023-11-25T04:24:44.213 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:44.213 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.213 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.213 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:44.213 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092102+0000", 2023-11-25T04:24:44.213 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3700000000000001e-07 2023-11-25T04:24:44.214 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.214 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.214 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:44.214 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092104+0000", 2023-11-25T04:24:44.214 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0679999999999999e-06 2023-11-25T04:24:44.214 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.214 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.215 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:44.215 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092111+0000", 2023-11-25T04:24:44.215 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0589999999999997e-06 2023-11-25T04:24:44.215 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.215 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.215 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:44.215 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092130+0000", 2023-11-25T04:24:44.215 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8686999999999999e-05 2023-11-25T04:24:44.216 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.216 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.216 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:44.216 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092443+0000", 2023-11-25T04:24:44.216 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000313519 2023-11-25T04:24:44.216 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.216 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.216 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:44.217 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092553+0000", 2023-11-25T04:24:44.217 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010932100000000001 2023-11-25T04:24:44.217 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.217 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.217 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:44.218 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.092639+0000", 2023-11-25T04:24:44.218 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6094999999999995e-05 2023-11-25T04:24:44.218 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.218 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.218 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:44.218 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094679+0000", 2023-11-25T04:24:44.219 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0020402580000000001 2023-11-25T04:24:44.219 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.219 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.219 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:44.219 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108462+0000", 2023-11-25T04:24:44.220 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013783358000000001 2023-11-25T04:24:44.220 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.220 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.220 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:44.220 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108516+0000", 2023-11-25T04:24:44.220 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3483000000000003e-05 2023-11-25T04:24:44.220 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.221 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.221 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:44.221 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108551+0000", 2023-11-25T04:24:44.221 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5499e-05 2023-11-25T04:24:44.221 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:44.221 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:44.221 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:44.222 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.222 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.222 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2852 4.2d 4:b745ac76:::100000010f5.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:44.222 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.093477+0000", 2023-11-25T04:24:44.222 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 18.995862185, 2023-11-25T04:24:44.222 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.015423084, 2023-11-25T04:24:44.222 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:44.223 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:44.223 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:44.223 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.223 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:44.223 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093477+0000", 2023-11-25T04:24:44.223 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:44.223 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.223 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.224 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:44.224 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093477+0000", 2023-11-25T04:24:44.224 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:44.225 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.225 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.225 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:44.225 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093477+0000", 2023-11-25T04:24:44.225 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2500000000000005e-07 2023-11-25T04:24:44.225 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.225 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.226 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:44.226 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093480+0000", 2023-11-25T04:24:44.226 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0189999999999998e-06 2023-11-25T04:24:44.226 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.226 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.226 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:44.226 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093642+0000", 2023-11-25T04:24:44.227 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000162072 2023-11-25T04:24:44.227 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.227 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.227 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:44.227 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093649+0000", 2023-11-25T04:24:44.227 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9659999999999999e-06 2023-11-25T04:24:44.227 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.227 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.228 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:44.228 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093693+0000", 2023-11-25T04:24:44.228 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3794000000000001e-05 2023-11-25T04:24:44.228 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.229 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.229 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:44.229 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093764+0000", 2023-11-25T04:24:44.229 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0981000000000004e-05 2023-11-25T04:24:44.229 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.230 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.230 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:44.230 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093850+0000", 2023-11-25T04:24:44.230 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5536000000000002e-05 2023-11-25T04:24:44.230 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.230 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.230 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:44.231 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094710+0000", 2023-11-25T04:24:44.231 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00086070299999999999 2023-11-25T04:24:44.231 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.231 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.231 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:44.231 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108776+0000", 2023-11-25T04:24:44.231 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01406542 2023-11-25T04:24:44.232 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.232 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.232 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:44.232 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108829+0000", 2023-11-25T04:24:44.232 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2936999999999998e-05 2023-11-25T04:24:44.232 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.232 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.233 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:44.233 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108900+0000", 2023-11-25T04:24:44.233 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1031000000000005e-05 2023-11-25T04:24:44.233 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:44.233 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:44.233 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:44.233 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.233 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.234 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2854 4.2d 4:b5c259cf:::100000010f9.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:44.234 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.093674+0000", 2023-11-25T04:24:44.234 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 18.995664524999999, 2023-11-25T04:24:44.234 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.015970380999999999, 2023-11-25T04:24:44.234 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:44.234 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:44.234 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:44.235 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.235 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:44.235 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093674+0000", 2023-11-25T04:24:44.235 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:44.235 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.235 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.235 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:44.236 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093674+0000", 2023-11-25T04:24:44.236 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:44.236 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.236 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.236 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:44.236 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093675+0000", 2023-11-25T04:24:44.236 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8400000000000004e-07 2023-11-25T04:24:44.236 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.237 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.237 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:44.237 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093678+0000", 2023-11-25T04:24:44.237 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2229999999999999e-06 2023-11-25T04:24:44.237 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.237 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.237 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:44.237 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093686+0000", 2023-11-25T04:24:44.238 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9990000000000008e-06 2023-11-25T04:24:44.238 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.238 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.238 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:44.238 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093696+0000", 2023-11-25T04:24:44.238 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6479999999999998e-06 2023-11-25T04:24:44.238 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.238 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.239 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:44.239 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094104+0000", 2023-11-25T04:24:44.239 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00040823900000000002 2023-11-25T04:24:44.239 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.239 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.239 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:44.239 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094172+0000", 2023-11-25T04:24:44.239 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8023999999999998e-05 2023-11-25T04:24:44.240 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.240 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.240 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:44.240 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094256+0000", 2023-11-25T04:24:44.240 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3418999999999995e-05 2023-11-25T04:24:44.241 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.241 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.241 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:44.241 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095160+0000", 2023-11-25T04:24:44.241 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00090418599999999997 2023-11-25T04:24:44.241 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.241 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.241 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:44.242 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109551+0000", 2023-11-25T04:24:44.242 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014391571000000001 2023-11-25T04:24:44.242 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.242 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.242 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:44.242 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109601+0000", 2023-11-25T04:24:44.242 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9478000000000003e-05 2023-11-25T04:24:44.243 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.243 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.243 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:44.243 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109645+0000", 2023-11-25T04:24:44.243 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.401e-05 2023-11-25T04:24:44.243 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:44.243 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:44.243 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:44.244 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.244 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.244 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2861 4.4 4:214027a9:::10000001107.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:44.244 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.093829+0000", 2023-11-25T04:24:44.244 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 18.995510107000001, 2023-11-25T04:24:44.244 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014848917999999999, 2023-11-25T04:24:44.244 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:44.244 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:44.245 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:44.245 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.245 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:44.245 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093829+0000", 2023-11-25T04:24:44.245 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:44.245 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.245 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.246 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:44.246 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093829+0000", 2023-11-25T04:24:44.246 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:44.246 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.246 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.246 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:44.246 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093830+0000", 2023-11-25T04:24:44.247 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1999999999999998e-07 2023-11-25T04:24:44.247 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.247 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.247 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:44.247 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093834+0000", 2023-11-25T04:24:44.247 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.34e-06 2023-11-25T04:24:44.247 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.247 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.248 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:44.248 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093844+0000", 2023-11-25T04:24:44.248 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6690000000000001e-06 2023-11-25T04:24:44.248 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.248 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.248 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:44.248 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093857+0000", 2023-11-25T04:24:44.248 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3434999999999999e-05 2023-11-25T04:24:44.249 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.249 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.249 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:44.249 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093940+0000", 2023-11-25T04:24:44.249 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3047999999999995e-05 2023-11-25T04:24:44.249 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.249 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.249 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:44.250 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094052+0000", 2023-11-25T04:24:44.250 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000111538 2023-11-25T04:24:44.250 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.250 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.250 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:44.251 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094466+0000", 2023-11-25T04:24:44.251 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00041464600000000001 2023-11-25T04:24:44.251 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.251 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.251 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:44.251 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095570+0000", 2023-11-25T04:24:44.251 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001103981 2023-11-25T04:24:44.251 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.252 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.252 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:44.252 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108598+0000", 2023-11-25T04:24:44.252 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013027164000000001 2023-11-25T04:24:44.252 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.252 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.252 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:44.252 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108646+0000", 2023-11-25T04:24:44.253 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8090999999999997e-05 2023-11-25T04:24:44.253 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.253 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.253 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:44.253 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108678+0000", 2023-11-25T04:24:44.253 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2085999999999997e-05 2023-11-25T04:24:44.253 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:44.254 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:44.254 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:44.254 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.254 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.254 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2863 4.12 4:4b84c733:::1000000110b.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:44.254 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.093877+0000", 2023-11-25T04:24:44.254 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 18.995461779999999, 2023-11-25T04:24:44.255 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014682755, 2023-11-25T04:24:44.255 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:44.255 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:44.255 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:44.255 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.255 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:44.255 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093877+0000", 2023-11-25T04:24:44.255 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:44.256 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.256 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.256 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:44.256 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093877+0000", 2023-11-25T04:24:44.256 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:44.256 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.256 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.256 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:44.257 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093878+0000", 2023-11-25T04:24:44.257 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4199999999999996e-07 2023-11-25T04:24:44.257 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.257 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.257 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:44.257 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093880+0000", 2023-11-25T04:24:44.257 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6189999999999998e-06 2023-11-25T04:24:44.257 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.258 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.258 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:44.258 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093888+0000", 2023-11-25T04:24:44.258 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5510000000000002e-06 2023-11-25T04:24:44.258 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.258 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.258 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:44.259 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093893+0000", 2023-11-25T04:24:44.259 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4179999999999996e-06 2023-11-25T04:24:44.259 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.259 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.259 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:44.259 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.093934+0000", 2023-11-25T04:24:44.259 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0648000000000001e-05 2023-11-25T04:24:44.260 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.260 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.260 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:44.260 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094081+0000", 2023-11-25T04:24:44.260 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014753499999999999 2023-11-25T04:24:44.260 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.260 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.261 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:44.261 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094172+0000", 2023-11-25T04:24:44.261 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.0722000000000003e-05 2023-11-25T04:24:44.261 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.261 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.262 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:44.262 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.094742+0000", 2023-11-25T04:24:44.262 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00056948600000000002 2023-11-25T04:24:44.262 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.262 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.262 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:44.262 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108491+0000", 2023-11-25T04:24:44.262 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013749568 2023-11-25T04:24:44.263 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.263 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.263 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:44.263 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108525+0000", 2023-11-25T04:24:44.263 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3683999999999998e-05 2023-11-25T04:24:44.263 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.263 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.264 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:44.264 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108560+0000", 2023-11-25T04:24:44.264 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4981999999999998e-05 2023-11-25T04:24:44.264 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:44.264 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:44.264 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:44.264 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.264 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.265 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2880 4.21 4:855834a4:::1000000112d.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:44.265 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.095098+0000", 2023-11-25T04:24:44.265 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 18.994241085999999, 2023-11-25T04:24:44.265 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013847544, 2023-11-25T04:24:44.265 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:44.265 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:44.265 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:44.266 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.266 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:44.266 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095098+0000", 2023-11-25T04:24:44.266 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:44.266 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.266 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.266 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:44.267 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095098+0000", 2023-11-25T04:24:44.267 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:44.267 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.267 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.267 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:44.267 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095098+0000", 2023-11-25T04:24:44.268 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3000000000000002e-07 2023-11-25T04:24:44.268 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.268 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.268 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:44.268 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095099+0000", 2023-11-25T04:24:44.268 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.331e-06 2023-11-25T04:24:44.268 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.269 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.269 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:44.269 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095103+0000", 2023-11-25T04:24:44.269 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8319999999999999e-06 2023-11-25T04:24:44.269 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.269 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.269 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:44.270 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095108+0000", 2023-11-25T04:24:44.270 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4179999999999999e-06 2023-11-25T04:24:44.270 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.270 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.270 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:44.270 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095153+0000", 2023-11-25T04:24:44.270 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4799000000000003e-05 2023-11-25T04:24:44.271 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.271 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.271 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:44.271 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095268+0000", 2023-11-25T04:24:44.272 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000115437 2023-11-25T04:24:44.272 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.272 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.272 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:44.272 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095363+0000", 2023-11-25T04:24:44.272 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.5278000000000005e-05 2023-11-25T04:24:44.272 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.272 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.273 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:44.273 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095845+0000", 2023-11-25T04:24:44.273 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00048164600000000002 2023-11-25T04:24:44.273 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.273 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.273 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:44.273 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108854+0000", 2023-11-25T04:24:44.273 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013009289 2023-11-25T04:24:44.274 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.274 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.274 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:44.274 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108907+0000", 2023-11-25T04:24:44.274 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2528999999999998e-05 2023-11-25T04:24:44.274 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.274 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.274 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:44.275 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.108945+0000", 2023-11-25T04:24:44.275 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8655000000000002e-05 2023-11-25T04:24:44.275 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:44.275 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:44.275 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:44.275 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.275 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.276 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2884 4.37 4:ef909cdc:::10000001135.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:44.276 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.095518+0000", 2023-11-25T04:24:44.276 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 18.993821158999999, 2023-11-25T04:24:44.276 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014149988, 2023-11-25T04:24:44.276 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:44.276 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:44.276 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:44.276 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.277 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:44.277 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095518+0000", 2023-11-25T04:24:44.277 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:44.277 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.277 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.277 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:44.277 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095518+0000", 2023-11-25T04:24:44.277 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:44.278 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.278 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.278 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:44.278 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095518+0000", 2023-11-25T04:24:44.278 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2900000000000003e-07 2023-11-25T04:24:44.278 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.278 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.278 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:44.279 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095522+0000", 2023-11-25T04:24:44.279 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2119999999999999e-06 2023-11-25T04:24:44.279 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.279 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.279 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:44.279 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095532+0000", 2023-11-25T04:24:44.279 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0375e-05 2023-11-25T04:24:44.279 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.280 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.280 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:44.280 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095537+0000", 2023-11-25T04:24:44.280 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5909999999999998e-06 2023-11-25T04:24:44.280 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.280 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.280 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:44.280 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095584+0000", 2023-11-25T04:24:44.281 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7169999999999997e-05 2023-11-25T04:24:44.281 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.281 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.281 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:44.281 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095695+0000", 2023-11-25T04:24:44.282 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011078699999999999 2023-11-25T04:24:44.282 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.282 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.282 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:44.282 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.095777+0000", 2023-11-25T04:24:44.282 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2089999999999995e-05 2023-11-25T04:24:44.283 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.283 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.283 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:44.283 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096286+0000", 2023-11-25T04:24:44.283 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000509247 2023-11-25T04:24:44.283 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.283 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.283 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:44.284 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109596+0000", 2023-11-25T04:24:44.284 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013309872 2023-11-25T04:24:44.284 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.284 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.284 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:44.284 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109638+0000", 2023-11-25T04:24:44.284 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2381e-05 2023-11-25T04:24:44.285 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.285 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.285 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:44.285 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109668+0000", 2023-11-25T04:24:44.285 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9634000000000001e-05 2023-11-25T04:24:44.285 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:44.286 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:44.286 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:44.286 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.286 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.286 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2890 4.12 4:4a2ad640:::10000001141.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:44.286 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.096274+0000", 2023-11-25T04:24:44.286 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 18.993065095999999, 2023-11-25T04:24:44.287 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013647259, 2023-11-25T04:24:44.287 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:44.287 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:44.287 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:44.287 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.287 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:44.288 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096274+0000", 2023-11-25T04:24:44.288 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:44.288 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.288 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.288 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:44.288 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096274+0000", 2023-11-25T04:24:44.288 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:44.289 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.289 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.289 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:44.289 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096275+0000", 2023-11-25T04:24:44.289 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0299999999999998e-07 2023-11-25T04:24:44.289 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.290 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.290 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:44.290 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096279+0000", 2023-11-25T04:24:44.290 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9580000000000001e-06 2023-11-25T04:24:44.290 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.290 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.290 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:44.291 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096287+0000", 2023-11-25T04:24:44.291 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6929999999999996e-06 2023-11-25T04:24:44.291 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.291 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.291 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:44.291 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096294+0000", 2023-11-25T04:24:44.291 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.348e-06 2023-11-25T04:24:44.292 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.292 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.292 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:44.292 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096342+0000", 2023-11-25T04:24:44.293 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8068000000000001e-05 2023-11-25T04:24:44.293 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.293 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.293 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:44.293 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096462+0000", 2023-11-25T04:24:44.293 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000119918 2023-11-25T04:24:44.294 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.294 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.294 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:44.294 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096557+0000", 2023-11-25T04:24:44.294 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.5340999999999994e-05 2023-11-25T04:24:44.294 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.295 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.295 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:44.295 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096971+0000", 2023-11-25T04:24:44.295 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00041398100000000001 2023-11-25T04:24:44.295 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.296 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.296 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:44.296 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109849+0000", 2023-11-25T04:24:44.296 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.012877985999999999 2023-11-25T04:24:44.296 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.296 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.296 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:44.297 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109894+0000", 2023-11-25T04:24:44.297 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5595e-05 2023-11-25T04:24:44.297 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.297 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.297 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:44.297 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109921+0000", 2023-11-25T04:24:44.297 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6568e-05 2023-11-25T04:24:44.297 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:44.298 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:44.298 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:44.298 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.298 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.298 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:2891 4.23 4:c4054dd3:::10000001143.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e47)", 2023-11-25T04:24:44.298 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:25.096713+0000", 2023-11-25T04:24:44.299 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 18.992626401999999, 2023-11-25T04:24:44.299 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013148692, 2023-11-25T04:24:44.299 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:44.299 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2023-11-25T04:24:44.299 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:44.299 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.299 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:44.299 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096713+0000", 2023-11-25T04:24:44.300 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:44.300 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.300 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.300 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:44.300 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096713+0000", 2023-11-25T04:24:44.300 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:44.301 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.301 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.301 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:44.301 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096713+0000", 2023-11-25T04:24:44.301 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9299999999999998e-07 2023-11-25T04:24:44.301 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.301 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.302 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:44.302 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096716+0000", 2023-11-25T04:24:44.302 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.988e-06 2023-11-25T04:24:44.302 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.303 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.303 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:44.303 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096722+0000", 2023-11-25T04:24:44.303 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0499999999999997e-06 2023-11-25T04:24:44.303 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.303 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.303 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:44.304 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096727+0000", 2023-11-25T04:24:44.304 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3380000000000004e-06 2023-11-25T04:24:44.304 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.304 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.304 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:44.304 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096770+0000", 2023-11-25T04:24:44.304 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2938000000000001e-05 2023-11-25T04:24:44.304 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.305 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.305 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:44.305 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096904+0000", 2023-11-25T04:24:44.305 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000133759 2023-11-25T04:24:44.305 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.305 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.305 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1", 2023-11-25T04:24:44.305 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.096983+0000", 2023-11-25T04:24:44.306 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8559999999999993e-05 2023-11-25T04:24:44.306 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.306 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.306 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2023-11-25T04:24:44.306 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.097437+0000", 2023-11-25T04:24:44.306 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00045439200000000002 2023-11-25T04:24:44.306 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.307 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.307 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2023-11-25T04:24:44.307 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109737+0000", 2023-11-25T04:24:44.307 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.012300190000000001 2023-11-25T04:24:44.307 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.308 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.308 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2023-11-25T04:24:44.308 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109782+0000", 2023-11-25T04:24:44.308 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4546999999999999e-05 2023-11-25T04:24:44.308 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:44.308 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:44.309 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:44.309 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:25.109861+0000", 2023-11-25T04:24:44.309 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9337000000000003e-05 2023-11-25T04:24:44.309 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:44.309 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:44.309 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:44.309 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:44.309 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:44.310 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:44.311 INFO:teuthology.orchestra.run.smithi067.stderr:nodeep-scrub is set 2023-11-25T04:24:44.312 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:24:44.313 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:44.309+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:44.323 INFO:teuthology.orchestra.run.smithi067.stderr:listed 0 entries 2023-11-25T04:24:44.337 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi067.front.sepia.ceph.com... 2023-11-25T04:24:44.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-25T04:24:44.337 DEBUG:teuthology.orchestra.run.smithi067:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-25T04:24:44.406 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi067.stderr:ceph-fuse[85907]: fuse finished with error 0 and tester_r 0 2023-11-25T04:24:44.412 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:44.410+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:44.415 INFO:teuthology.orchestra.run:waiting for 300 2023-11-25T04:24:44.425 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:44.425 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:24:44.426 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2023-11-25T04:24:44.426 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:44.436 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:24:44.513 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:44.512+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:44.586 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:44.586 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:24:44.586 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2023-11-25T04:24:44.586 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2023-11-25T04:24:44.586 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:44.595 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:24:44.638 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:44.636+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:44.700 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:44.701 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2023-11-25T04:24:44.701 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2023-11-25T04:24:44.701 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2023-11-25T04:24:44.701 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.701 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:3691 4.19 4:9ae7aaf4:::10000000051.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:44.701 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:44.433304+0000", 2023-11-25T04:24:44.702 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.26532946899999998, 2023-11-25T04:24:44.702 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056495450000000003, 2023-11-25T04:24:44.702 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:44.702 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:44.702 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:44.702 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.702 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:44.702 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.433304+0000", 2023-11-25T04:24:44.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:44.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.703 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.703 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:44.703 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.433304+0000", 2023-11-25T04:24:44.703 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:44.703 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.704 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.704 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:44.704 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.433305+0000", 2023-11-25T04:24:44.704 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8099999999999998e-07 2023-11-25T04:24:44.707 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.707 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.708 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:44.708 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.433306+0000", 2023-11-25T04:24:44.708 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.578e-06 2023-11-25T04:24:44.708 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.708 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.708 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:44.709 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.433310+0000", 2023-11-25T04:24:44.709 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.771e-06 2023-11-25T04:24:44.709 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.709 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.710 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:44.710 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.433313+0000", 2023-11-25T04:24:44.710 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.864e-06 2023-11-25T04:24:44.710 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.710 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.710 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:44.711 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.433350+0000", 2023-11-25T04:24:44.711 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7398999999999999e-05 2023-11-25T04:24:44.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.711 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.711 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:44.711 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.433454+0000", 2023-11-25T04:24:44.711 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103837 2023-11-25T04:24:44.711 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.712 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.712 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:44.712 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.489800+0000", 2023-11-25T04:24:44.712 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056345619999999999 2023-11-25T04:24:44.712 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:44.712 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:44.712 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:44.712 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.713 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.713 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:3696 4.28 4:166e0f51:::10000000056.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:44.713 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:44.433361+0000", 2023-11-25T04:24:44.713 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.26527249000000003, 2023-11-25T04:24:44.713 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055970330999999998, 2023-11-25T04:24:44.713 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:44.714 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:44.714 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:44.714 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.714 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:44.714 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.433361+0000", 2023-11-25T04:24:44.714 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:44.714 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.715 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.715 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:44.715 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.433361+0000", 2023-11-25T04:24:44.715 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:44.715 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.715 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.715 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:44.715 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.433362+0000", 2023-11-25T04:24:44.716 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6199999999999998e-07 2023-11-25T04:24:44.716 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.716 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.716 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:44.716 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.433364+0000", 2023-11-25T04:24:44.717 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.323e-06 2023-11-25T04:24:44.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.717 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.717 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:44.717 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.433371+0000", 2023-11-25T04:24:44.717 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5899999999999996e-06 2023-11-25T04:24:44.717 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.718 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:44.718 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.433376+0000", 2023-11-25T04:24:44.718 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9269999999999996e-06 2023-11-25T04:24:44.718 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.718 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.718 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:44.719 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.433418+0000", 2023-11-25T04:24:44.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2867000000000001e-05 2023-11-25T04:24:44.719 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.719 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.719 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:44.719 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.433522+0000", 2023-11-25T04:24:44.719 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103421 2023-11-25T04:24:44.720 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.720 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.720 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:44.720 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.489331+0000", 2023-11-25T04:24:44.720 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055809641 2023-11-25T04:24:44.720 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:44.720 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:44.720 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:44.721 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.721 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.721 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:3731 4.19 4:99ec5763:::10000000079.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:44.721 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:44.434107+0000", 2023-11-25T04:24:44.721 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.26452682, 2023-11-25T04:24:44.721 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055803469000000001, 2023-11-25T04:24:44.721 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:44.722 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:44.722 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:44.722 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.722 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:44.722 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.434107+0000", 2023-11-25T04:24:44.722 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:44.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.723 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:44.723 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.434107+0000", 2023-11-25T04:24:44.723 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:44.723 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.723 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.724 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:44.724 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.434107+0000", 2023-11-25T04:24:44.724 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9900000000000001e-07 2023-11-25T04:24:44.724 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.724 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.724 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:44.724 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.434109+0000", 2023-11-25T04:24:44.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.553e-06 2023-11-25T04:24:44.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.725 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.725 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:44.725 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.434114+0000", 2023-11-25T04:24:44.725 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7320000000000003e-06 2023-11-25T04:24:44.725 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.726 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:44.726 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.434119+0000", 2023-11-25T04:24:44.726 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1080000000000003e-06 2023-11-25T04:24:44.726 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.726 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.726 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:44.727 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.434370+0000", 2023-11-25T04:24:44.727 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00025150999999999998 2023-11-25T04:24:44.727 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.727 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.727 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:44.727 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.434475+0000", 2023-11-25T04:24:44.727 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000105016 2023-11-25T04:24:44.728 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.728 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.728 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:44.728 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.489910+0000", 2023-11-25T04:24:44.728 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055435151000000002 2023-11-25T04:24:44.728 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:44.728 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:44.728 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:44.729 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.729 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.729 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:3901 4.36 4:6f395059:::10000000123.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:44.729 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:44.440446+0000", 2023-11-25T04:24:44.729 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.25818713900000001, 2023-11-25T04:24:44.729 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.062374564, 2023-11-25T04:24:44.729 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:44.730 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:44.730 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:44.730 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.730 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:44.730 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.440446+0000", 2023-11-25T04:24:44.730 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:44.730 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.731 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:44.731 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.440446+0000", 2023-11-25T04:24:44.731 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:44.731 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.731 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.731 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:44.732 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.440447+0000", 2023-11-25T04:24:44.732 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7599999999999997e-07 2023-11-25T04:24:44.732 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.732 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.732 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:44.733 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.440449+0000", 2023-11-25T04:24:44.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9609999999999999e-06 2023-11-25T04:24:44.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.733 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.733 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:44.733 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.440455+0000", 2023-11-25T04:24:44.733 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.392e-06 2023-11-25T04:24:44.733 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.734 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:44.734 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.440460+0000", 2023-11-25T04:24:44.734 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.916e-06 2023-11-25T04:24:44.734 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.734 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.735 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:44.735 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.444413+0000", 2023-11-25T04:24:44.735 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039524959999999998 2023-11-25T04:24:44.735 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.735 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.735 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:44.735 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.444512+0000", 2023-11-25T04:24:44.736 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8876999999999995e-05 2023-11-25T04:24:44.736 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.736 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.736 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:44.736 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.502821+0000", 2023-11-25T04:24:44.736 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.058309446000000001 2023-11-25T04:24:44.737 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:44.737 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:44.737 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:44.737 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.737 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.737 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:3922 4.6 4:629cc6bc:::10000000138.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:44.737 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:44.441824+0000", 2023-11-25T04:24:44.738 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.25681005099999998, 2023-11-25T04:24:44.738 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061109164000000001, 2023-11-25T04:24:44.738 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:44.738 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:44.738 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:44.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.741 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:44.741 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.441824+0000", 2023-11-25T04:24:44.741 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:44.741 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.741 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.742 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:44.742 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.441824+0000", 2023-11-25T04:24:44.742 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:44.742 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.742 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.742 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:44.743 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.441826+0000", 2023-11-25T04:24:44.743 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.599e-06 2023-11-25T04:24:44.743 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.743 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.743 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:44.743 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.441829+0000", 2023-11-25T04:24:44.744 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.055e-06 2023-11-25T04:24:44.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.744 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:44.744 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.441835+0000", 2023-11-25T04:24:44.744 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.276e-06 2023-11-25T04:24:44.744 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.744 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:44.745 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.441840+0000", 2023-11-25T04:24:44.745 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9599999999999997e-06 2023-11-25T04:24:44.745 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.745 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.745 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:44.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.445570+0000", 2023-11-25T04:24:44.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003729951 2023-11-25T04:24:44.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.746 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.746 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:44.746 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.445680+0000", 2023-11-25T04:24:44.746 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000109807 2023-11-25T04:24:44.746 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.747 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.747 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:44.747 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.502933+0000", 2023-11-25T04:24:44.747 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057252516000000003 2023-11-25T04:24:44.747 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:44.747 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:44.747 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:44.747 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.748 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.748 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:3926 4.36 4:6c5ddb65:::1000000013c.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:44.748 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:44.443003+0000", 2023-11-25T04:24:44.748 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.25563038199999999, 2023-11-25T04:24:44.748 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.059937957, 2023-11-25T04:24:44.748 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:44.748 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:44.749 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:44.749 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.749 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:44.749 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.443003+0000", 2023-11-25T04:24:44.749 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:44.749 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.750 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.750 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:44.750 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.443003+0000", 2023-11-25T04:24:44.750 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:44.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:44.751 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.443004+0000", 2023-11-25T04:24:44.751 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6100000000000001e-07 2023-11-25T04:24:44.751 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.751 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.751 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:44.752 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.443012+0000", 2023-11-25T04:24:44.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0090000000000007e-06 2023-11-25T04:24:44.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.752 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.752 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:44.752 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.443019+0000", 2023-11-25T04:24:44.752 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0500000000000003e-06 2023-11-25T04:24:44.752 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.753 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.753 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:44.753 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.443025+0000", 2023-11-25T04:24:44.753 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1800000000000001e-06 2023-11-25T04:24:44.753 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.753 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.753 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:44.753 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.445746+0000", 2023-11-25T04:24:44.754 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0027212640000000001 2023-11-25T04:24:44.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.754 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:44.754 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.445841+0000", 2023-11-25T04:24:44.754 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4488999999999996e-05 2023-11-25T04:24:44.754 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.754 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.755 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:44.755 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.502941+0000", 2023-11-25T04:24:44.755 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057100404 2023-11-25T04:24:44.755 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:44.755 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:44.755 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:44.755 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.755 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.755 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:3933 4.2e 4:77f9b43c:::10000000143.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:44.756 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:44.443095+0000", 2023-11-25T04:24:44.756 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.25553910299999999, 2023-11-25T04:24:44.756 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.059948544999999999, 2023-11-25T04:24:44.756 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:44.756 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:44.756 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:44.756 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.756 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:44.757 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.443095+0000", 2023-11-25T04:24:44.757 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:44.757 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.758 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.758 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:44.758 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.443095+0000", 2023-11-25T04:24:44.758 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:44.758 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:44.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.443095+0000", 2023-11-25T04:24:44.759 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5499999999999998e-07 2023-11-25T04:24:44.759 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.759 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.759 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:44.759 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.443097+0000", 2023-11-25T04:24:44.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7549999999999999e-06 2023-11-25T04:24:44.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.760 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:44.760 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.443103+0000", 2023-11-25T04:24:44.760 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6449999999999999e-06 2023-11-25T04:24:44.760 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.760 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:44.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.443108+0000", 2023-11-25T04:24:44.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6829999999999999e-06 2023-11-25T04:24:44.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.761 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.761 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:44.761 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.445970+0000", 2023-11-25T04:24:44.761 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002862 2023-11-25T04:24:44.761 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.762 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.762 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:44.762 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446688+0000", 2023-11-25T04:24:44.762 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00071754200000000005 2023-11-25T04:24:44.762 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.763 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.763 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:44.763 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.503043+0000", 2023-11-25T04:24:44.763 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056355465 2023-11-25T04:24:44.763 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:44.763 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:44.763 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:44.763 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.764 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.764 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:3994 4.6 4:63fec8d1:::10000000180.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:44.764 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:44.444709+0000", 2023-11-25T04:24:44.764 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.25392463700000001, 2023-11-25T04:24:44.764 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.058338905000000003, 2023-11-25T04:24:44.764 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:44.764 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:44.764 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:44.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:44.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.444709+0000", 2023-11-25T04:24:44.765 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:44.765 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.765 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.765 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:44.765 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.444709+0000", 2023-11-25T04:24:44.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:44.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.766 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:44.766 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.444709+0000", 2023-11-25T04:24:44.766 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7e-07 2023-11-25T04:24:44.766 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.766 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.767 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:44.767 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.444711+0000", 2023-11-25T04:24:44.767 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.635e-06 2023-11-25T04:24:44.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.767 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.767 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:44.767 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.444724+0000", 2023-11-25T04:24:44.767 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2753e-05 2023-11-25T04:24:44.767 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.768 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:44.768 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.444734+0000", 2023-11-25T04:24:44.768 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.007e-05 2023-11-25T04:24:44.768 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.768 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.768 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:44.768 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447792+0000", 2023-11-25T04:24:44.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0030580630000000002 2023-11-25T04:24:44.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.769 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:44.769 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447909+0000", 2023-11-25T04:24:44.769 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011692 2023-11-25T04:24:44.769 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.769 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.770 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:44.770 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.503048+0000", 2023-11-25T04:24:44.770 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055139094 2023-11-25T04:24:44.770 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:44.770 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:44.770 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:44.770 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.770 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.771 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:3997 4.6 4:60954073:::10000000183.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:44.771 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:44.444741+0000", 2023-11-25T04:24:44.771 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.25389242200000001, 2023-11-25T04:24:44.771 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.058419077, 2023-11-25T04:24:44.771 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:44.771 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:44.771 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:44.771 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.772 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:44.772 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.444741+0000", 2023-11-25T04:24:44.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:44.772 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.772 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.772 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:44.772 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.444741+0000", 2023-11-25T04:24:44.772 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:44.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.773 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:44.773 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.444742+0000", 2023-11-25T04:24:44.773 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1100000000000001e-07 2023-11-25T04:24:44.773 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.773 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.773 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:44.773 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.444744+0000", 2023-11-25T04:24:44.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.013e-06 2023-11-25T04:24:44.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.774 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:44.774 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.444750+0000", 2023-11-25T04:24:44.774 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2299999999999996e-06 2023-11-25T04:24:44.774 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.774 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:44.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.444757+0000", 2023-11-25T04:24:44.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9870000000000002e-06 2023-11-25T04:24:44.775 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.775 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.775 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:44.775 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.448162+0000", 2023-11-25T04:24:44.775 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0034051020000000001 2023-11-25T04:24:44.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:44.776 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.448494+0000", 2023-11-25T04:24:44.776 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00033231800000000001 2023-11-25T04:24:44.776 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.776 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.776 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:44.777 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.503160+0000", 2023-11-25T04:24:44.777 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054666015999999998 2023-11-25T04:24:44.777 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:44.777 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:44.777 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:44.777 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.777 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.777 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:4032 4.2e 4:775b9fb2:::100000001a6.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:44.777 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:44.446660+0000", 2023-11-25T04:24:44.778 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.25197396799999999, 2023-11-25T04:24:44.778 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056503945, 2023-11-25T04:24:44.778 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:44.778 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:44.778 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:44.778 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.778 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:44.778 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446660+0000", 2023-11-25T04:24:44.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:44.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.779 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:44.779 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446660+0000", 2023-11-25T04:24:44.779 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:44.779 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.779 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.780 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:44.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446660+0000", 2023-11-25T04:24:44.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5499999999999998e-07 2023-11-25T04:24:44.780 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.780 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.780 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:44.780 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446662+0000", 2023-11-25T04:24:44.780 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.798e-06 2023-11-25T04:24:44.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:44.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446667+0000", 2023-11-25T04:24:44.781 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2349999999999999e-06 2023-11-25T04:24:44.781 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.781 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.781 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:44.781 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446670+0000", 2023-11-25T04:24:44.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0699999999999998e-06 2023-11-25T04:24:44.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.782 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.782 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:44.782 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.455057+0000", 2023-11-25T04:24:44.782 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0083872570000000004 2023-11-25T04:24:44.782 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:44.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.455340+0000", 2023-11-25T04:24:44.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028249500000000001 2023-11-25T04:24:44.783 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.783 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.783 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:44.783 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.503164+0000", 2023-11-25T04:24:44.783 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.047823635000000003 2023-11-25T04:24:44.784 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:44.784 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:44.784 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:44.784 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.784 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.784 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:4037 4.2e 4:76865fdc:::100000001ab.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:44.784 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:44.446688+0000", 2023-11-25T04:24:44.784 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.25194575499999999, 2023-11-25T04:24:44.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056587089, 2023-11-25T04:24:44.785 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:44.785 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:44.785 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:44.785 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.785 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:44.785 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446688+0000", 2023-11-25T04:24:44.785 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:44.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:44.786 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446688+0000", 2023-11-25T04:24:44.786 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:44.786 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.786 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.786 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:44.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446688+0000", 2023-11-25T04:24:44.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2399999999999999e-07 2023-11-25T04:24:44.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.787 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.787 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:44.787 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446690+0000", 2023-11-25T04:24:44.787 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.911e-06 2023-11-25T04:24:44.787 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:44.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446696+0000", 2023-11-25T04:24:44.788 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4859999999999997e-06 2023-11-25T04:24:44.788 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.788 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.788 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:44.788 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446700+0000", 2023-11-25T04:24:44.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7099999999999998e-06 2023-11-25T04:24:44.789 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.789 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.789 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:44.789 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.455754+0000", 2023-11-25T04:24:44.789 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0090534350000000003 2023-11-25T04:24:44.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.790 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:44.790 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.455881+0000", 2023-11-25T04:24:44.790 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000126818 2023-11-25T04:24:44.790 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.790 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.790 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:44.791 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.503275+0000", 2023-11-25T04:24:44.791 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.047394304999999998 2023-11-25T04:24:44.791 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:44.791 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:44.791 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:44.791 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.791 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.791 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:4044 4.2e 4:76ff63fc:::100000001b2.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:44.792 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:44.446860+0000", 2023-11-25T04:24:44.792 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.25177390999999999, 2023-11-25T04:24:44.792 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056538421999999998, 2023-11-25T04:24:44.792 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:44.792 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:44.792 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:44.792 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.792 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:44.792 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446860+0000", 2023-11-25T04:24:44.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:44.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.793 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:44.793 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446860+0000", 2023-11-25T04:24:44.793 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:44.793 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.793 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.794 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:44.794 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446860+0000", 2023-11-25T04:24:44.794 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8599999999999998e-07 2023-11-25T04:24:44.794 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.794 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.794 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:44.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446862+0000", 2023-11-25T04:24:44.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0650000000000001e-06 2023-11-25T04:24:44.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.795 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.795 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:44.795 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446869+0000", 2023-11-25T04:24:44.795 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1709999999999996e-06 2023-11-25T04:24:44.795 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:44.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446875+0000", 2023-11-25T04:24:44.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3870000000000003e-06 2023-11-25T04:24:44.796 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.796 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.796 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:44.796 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.456185+0000", 2023-11-25T04:24:44.796 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0093104260000000001 2023-11-25T04:24:44.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:44.797 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.456285+0000", 2023-11-25T04:24:44.797 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9393000000000002e-05 2023-11-25T04:24:44.797 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.797 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.797 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:44.798 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.503398+0000", 2023-11-25T04:24:44.798 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.047113493999999999 2023-11-25T04:24:44.798 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:44.798 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:44.798 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:44.798 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.798 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.798 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:4049 4.19 4:99a7d964:::100000001b7.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:44.799 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:44.446931+0000", 2023-11-25T04:24:44.799 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.251702641, 2023-11-25T04:24:44.799 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056039185999999998, 2023-11-25T04:24:44.799 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:44.799 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:44.799 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:44.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.799 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:44.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446931+0000", 2023-11-25T04:24:44.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:44.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.800 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:44.800 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446931+0000", 2023-11-25T04:24:44.800 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:44.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.800 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:44.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446931+0000", 2023-11-25T04:24:44.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.84e-07 2023-11-25T04:24:44.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.801 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.801 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:44.801 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446939+0000", 2023-11-25T04:24:44.801 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2010000000000002e-06 2023-11-25T04:24:44.801 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:44.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446945+0000", 2023-11-25T04:24:44.802 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5429999999999999e-06 2023-11-25T04:24:44.802 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.802 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.802 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:44.802 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446950+0000", 2023-11-25T04:24:44.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0560000000000004e-06 2023-11-25T04:24:44.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.803 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:44.803 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446999+0000", 2023-11-25T04:24:44.803 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8594000000000003e-05 2023-11-25T04:24:44.803 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.803 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.804 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:44.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447095+0000", 2023-11-25T04:24:44.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6638999999999994e-05 2023-11-25T04:24:44.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.804 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.804 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:44.804 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.502970+0000", 2023-11-25T04:24:44.804 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055874768999999998 2023-11-25T04:24:44.804 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:44.805 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:44.805 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:44.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.805 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.805 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:4052 4.36 4:6d15073c:::100000001ba.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:44.805 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:44.446994+0000", 2023-11-25T04:24:44.805 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.251639373, 2023-11-25T04:24:44.805 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056265880999999997, 2023-11-25T04:24:44.806 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:44.806 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:44.806 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:44.806 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.806 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:44.806 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446994+0000", 2023-11-25T04:24:44.806 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:44.806 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:44.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446994+0000", 2023-11-25T04:24:44.807 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:44.807 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.807 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.807 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:44.807 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446995+0000", 2023-11-25T04:24:44.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6300000000000001e-07 2023-11-25T04:24:44.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.808 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:44.808 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446996+0000", 2023-11-25T04:24:44.808 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.612e-06 2023-11-25T04:24:44.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.808 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:44.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447003+0000", 2023-11-25T04:24:44.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3069999999999999e-06 2023-11-25T04:24:44.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.809 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.809 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:44.809 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447007+0000", 2023-11-25T04:24:44.809 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3930000000000001e-06 2023-11-25T04:24:44.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:44.810 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.456139+0000", 2023-11-25T04:24:44.810 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0091321370000000002 2023-11-25T04:24:44.810 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.810 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.810 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:44.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.456249+0000", 2023-11-25T04:24:44.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000109491 2023-11-25T04:24:44.811 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.811 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.811 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:44.811 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.503260+0000", 2023-11-25T04:24:44.811 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.047011577999999998 2023-11-25T04:24:44.811 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:44.812 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:44.812 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:44.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.812 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.812 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:4054 4.19 4:9b54a4b5:::100000001bc.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:44.812 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:44.447013+0000", 2023-11-25T04:24:44.812 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.251620909, 2023-11-25T04:24:44.812 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056061487, 2023-11-25T04:24:44.813 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:44.813 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:44.813 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:44.813 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.813 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:44.813 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447013+0000", 2023-11-25T04:24:44.813 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:44.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:44.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447013+0000", 2023-11-25T04:24:44.814 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:44.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.814 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.814 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:44.814 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447013+0000", 2023-11-25T04:24:44.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7700000000000005e-07 2023-11-25T04:24:44.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.815 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:44.815 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447015+0000", 2023-11-25T04:24:44.815 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5939999999999999e-06 2023-11-25T04:24:44.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.815 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:44.816 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447020+0000", 2023-11-25T04:24:44.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6030000000000004e-06 2023-11-25T04:24:44.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.816 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.816 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:44.816 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447026+0000", 2023-11-25T04:24:44.816 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1420000000000001e-06 2023-11-25T04:24:44.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:44.817 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447490+0000", 2023-11-25T04:24:44.817 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00046441800000000002 2023-11-25T04:24:44.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.817 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.817 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:44.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447568+0000", 2023-11-25T04:24:44.818 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7751999999999997e-05 2023-11-25T04:24:44.818 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.818 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.818 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:44.818 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.503074+0000", 2023-11-25T04:24:44.819 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055506501 2023-11-25T04:24:44.819 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:44.819 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:44.819 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:44.819 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.819 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.819 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:4057 4.19 4:9ab6d71b:::100000001bf.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:44.820 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:44.447032+0000", 2023-11-25T04:24:44.820 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.25160184000000002, 2023-11-25T04:24:44.820 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056145928999999997, 2023-11-25T04:24:44.820 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:44.820 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:44.820 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:44.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:44.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447032+0000", 2023-11-25T04:24:44.821 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:44.821 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.821 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.821 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:44.821 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447032+0000", 2023-11-25T04:24:44.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:44.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.822 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:44.822 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447032+0000", 2023-11-25T04:24:44.822 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3500000000000002e-07 2023-11-25T04:24:44.822 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.822 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.823 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:44.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447034+0000", 2023-11-25T04:24:44.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.832e-06 2023-11-25T04:24:44.823 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.823 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.823 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:44.823 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447040+0000", 2023-11-25T04:24:44.823 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6779999999999997e-06 2023-11-25T04:24:44.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:44.824 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447044+0000", 2023-11-25T04:24:44.824 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.391e-06 2023-11-25T04:24:44.824 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.824 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.824 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:44.825 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447807+0000", 2023-11-25T04:24:44.825 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00076306799999999999 2023-11-25T04:24:44.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.825 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.825 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:44.826 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447931+0000", 2023-11-25T04:24:44.826 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012439999999999999 2023-11-25T04:24:44.826 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.826 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.826 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:44.826 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.503178+0000", 2023-11-25T04:24:44.827 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055246225000000003 2023-11-25T04:24:44.827 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:44.827 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:44.827 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:44.827 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.827 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.827 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:4058 4.19 4:98a8a8b3:::100000001c0.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:44.827 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:44.447050+0000", 2023-11-25T04:24:44.828 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.25158353900000002, 2023-11-25T04:24:44.828 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.05633759, 2023-11-25T04:24:44.828 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:44.828 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:44.828 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:44.828 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.828 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:44.828 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447050+0000", 2023-11-25T04:24:44.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:44.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.829 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:44.829 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447050+0000", 2023-11-25T04:24:44.829 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:44.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.829 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:44.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447051+0000", 2023-11-25T04:24:44.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1900000000000003e-07 2023-11-25T04:24:44.830 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.830 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.830 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:44.830 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447053+0000", 2023-11-25T04:24:44.830 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.108e-06 2023-11-25T04:24:44.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:44.831 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447058+0000", 2023-11-25T04:24:44.831 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3159999999999996e-06 2023-11-25T04:24:44.831 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.831 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.831 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:44.832 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447063+0000", 2023-11-25T04:24:44.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5449999999999997e-06 2023-11-25T04:24:44.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.832 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.832 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:44.832 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.448196+0000", 2023-11-25T04:24:44.832 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001133128 2023-11-25T04:24:44.832 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:44.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.448895+0000", 2023-11-25T04:24:44.833 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00069910700000000003 2023-11-25T04:24:44.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.833 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.833 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:44.833 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.503388+0000", 2023-11-25T04:24:44.834 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054492867 2023-11-25T04:24:44.834 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:44.834 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:44.834 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:44.835 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.835 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.835 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:4077 4.3e 4:7c32b2c0:::100000001d3.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:44.835 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:44.448674+0000", 2023-11-25T04:24:44.835 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.24995919599999999, 2023-11-25T04:24:44.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054710506999999999, 2023-11-25T04:24:44.836 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:44.836 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:44.836 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:44.836 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.836 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:44.836 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.448674+0000", 2023-11-25T04:24:44.836 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:44.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.837 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:44.837 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.448674+0000", 2023-11-25T04:24:44.837 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:44.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.837 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.837 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:44.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.448675+0000", 2023-11-25T04:24:44.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0199999999999998e-07 2023-11-25T04:24:44.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.838 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:44.838 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.448676+0000", 2023-11-25T04:24:44.838 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.646e-06 2023-11-25T04:24:44.838 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:44.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.448682+0000", 2023-11-25T04:24:44.839 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.981e-06 2023-11-25T04:24:44.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.839 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:44.839 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.448687+0000", 2023-11-25T04:24:44.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.853e-06 2023-11-25T04:24:44.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.840 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:44.840 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.456676+0000", 2023-11-25T04:24:44.840 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0079888660000000007 2023-11-25T04:24:44.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.840 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:44.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.456799+0000", 2023-11-25T04:24:44.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012312799999999999 2023-11-25T04:24:44.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.841 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.841 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:44.841 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.503385+0000", 2023-11-25T04:24:44.841 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046585730999999998 2023-11-25T04:24:44.842 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:44.842 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:44.843 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:44.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.843 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:4083 4.6 4:61ce4130:::100000001d9.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:44.844 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:44.448770+0000", 2023-11-25T04:24:44.844 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.24986410100000001, 2023-11-25T04:24:44.844 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054722232000000003, 2023-11-25T04:24:44.844 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:44.844 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:44.844 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:44.844 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.844 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:44.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.448770+0000", 2023-11-25T04:24:44.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:44.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.845 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.845 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:44.845 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.448770+0000", 2023-11-25T04:24:44.845 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:44.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:44.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.448770+0000", 2023-11-25T04:24:44.846 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5499999999999999e-07 2023-11-25T04:24:44.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.846 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.846 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:44.846 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.448772+0000", 2023-11-25T04:24:44.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.72e-06 2023-11-25T04:24:44.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.847 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:44.847 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.448779+0000", 2023-11-25T04:24:44.847 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2139999999999999e-06 2023-11-25T04:24:44.847 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.847 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.848 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:44.848 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.448784+0000", 2023-11-25T04:24:44.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7299999999999996e-06 2023-11-25T04:24:44.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.848 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.848 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:44.848 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.456849+0000", 2023-11-25T04:24:44.848 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0080652759999999997 2023-11-25T04:24:44.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:44.849 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.456950+0000", 2023-11-25T04:24:44.849 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000101551 2023-11-25T04:24:44.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.849 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.849 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:44.849 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.503492+0000", 2023-11-25T04:24:44.850 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046541385999999997 2023-11-25T04:24:44.850 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:44.850 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:44.850 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:44.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.850 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.850 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:4088 4.2e 4:77eaaa70:::100000001de.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:44.850 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:44.448828+0000", 2023-11-25T04:24:44.851 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 0.24980523099999999, 2023-11-25T04:24:44.851 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054684280000000002, 2023-11-25T04:24:44.851 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:44.851 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:44.851 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:44.851 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:44.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.448828+0000", 2023-11-25T04:24:44.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:44.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.852 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.852 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:44.852 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.448828+0000", 2023-11-25T04:24:44.852 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:44.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:44.853 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.448829+0000", 2023-11-25T04:24:44.853 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5999999999999999e-07 2023-11-25T04:24:44.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.853 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.853 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:44.854 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.448831+0000", 2023-11-25T04:24:44.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7400000000000001e-06 2023-11-25T04:24:44.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.854 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.854 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:44.854 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.448836+0000", 2023-11-25T04:24:44.854 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7309999999999998e-06 2023-11-25T04:24:44.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.855 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:44.855 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.448842+0000", 2023-11-25T04:24:44.855 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3979999999999998e-06 2023-11-25T04:24:44.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.855 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.856 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:44.856 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.457248+0000", 2023-11-25T04:24:44.856 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0084061819999999999 2023-11-25T04:24:44.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.856 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.856 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:44.857 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.457351+0000", 2023-11-25T04:24:44.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010357399999999999 2023-11-25T04:24:44.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:44.857 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:44.857 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:44.857 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.503513+0000", 2023-11-25T04:24:44.857 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046161294999999998 2023-11-25T04:24:44.857 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:44.857 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:44.858 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:44.858 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:44.858 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:44.858 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:44.859 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-11-25T04:24:44.860 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:44.858+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:44.860 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:24:44.960 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:44.959+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:44.967 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:44.967 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:24:44.967 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2023-11-25T04:24:44.967 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:44.973 INFO:teuthology.orchestra.run.smithi067.stderr:Error ENXIO: problem getting command descriptions from osd.0 2023-11-25T04:24:44.975 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:24:44.991 INFO:teuthology.orchestra.run.smithi067.stdout:osd.1: {} 2023-11-25T04:24:44.992 INFO:teuthology.orchestra.run.smithi067.stderr:osd.1: osd_enable_op_tracker = '' 2023-11-25T04:24:45.010 INFO:teuthology.orchestra.run.smithi067.stdout:osd.2: {} 2023-11-25T04:24:45.011 INFO:teuthology.orchestra.run.smithi067.stderr:osd.2: osd_enable_op_tracker = '' 2023-11-25T04:24:45.014 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-11-25T04:24:45.014 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-11-25T04:24:45.061 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:45.059+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:45.121 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:45.121 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:45.128 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:24:45.162 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:45.161+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:45.265 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:45.263+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:45.275 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:45.275 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:45.284 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:24:45.368 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:45.364+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:45.394 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:45.394 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:45.403 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:24:45.466 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:45.464+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:45.507 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:45.507 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:45.515 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:24:45.567 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:45.566+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:45.660 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:45.661 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:45.668 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:45.666+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:45.669 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:24:45.770 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:45.768+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:45.776 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:45.777 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:45.785 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:24:45.871 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:45.869+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:45.931 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:45.931 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:45.940 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:24:45.983 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:45.982+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:46.044 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:46.045 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:46.053 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:24:46.096 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:46.093+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:46.160 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:46.160 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:46.170 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:24:46.173 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:46.171+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:46.273 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:46.272+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:46.275 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:46.275 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:46.283 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:24:46.313 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2023-11-25T04:24:46.374 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:46.373+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:46.388 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:46.389 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:46.397 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:24:46.475 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:46.473+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:46.506 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:46.506 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:46.515 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:24:46.598 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:46.597+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:46.659 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:46.659 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:46.667 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:24:46.677 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:46.676+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:46.777 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:46.776+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:46.814 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:46.814 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:46.822 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:24:46.879 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:46.877+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:46.930 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:46.930 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:46.940 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:24:46.985 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:46.984+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:47.015 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-11-25T04:24:47.046 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:47.047 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:47.055 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:24:47.081 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:47.079+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:47.131 INFO:teuthology.orchestra.run.smithi067.stderr:Error ENXIO: problem getting command descriptions from osd.0 2023-11-25T04:24:47.150 INFO:teuthology.orchestra.run.smithi067.stdout:osd.1: {} 2023-11-25T04:24:47.151 INFO:teuthology.orchestra.run.smithi067.stderr:osd.1: osd_enable_op_tracker = '' 2023-11-25T04:24:47.170 INFO:teuthology.orchestra.run.smithi067.stdout:osd.2: {} 2023-11-25T04:24:47.171 INFO:teuthology.orchestra.run.smithi067.stderr:osd.2: osd_enable_op_tracker = '' 2023-11-25T04:24:47.172 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-11-25T04:24:47.173 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-11-25T04:24:47.182 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:47.180+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:47.203 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:47.203 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2023-11-25T04:24:47.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2023-11-25T04:24:47.204 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2023-11-25T04:24:47.204 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.204 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:3691 4.19 4:9ae7aaf4:::10000000051.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:47.204 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:44.433304+0000", 2023-11-25T04:24:47.204 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.7678823530000001, 2023-11-25T04:24:47.204 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056495450000000003, 2023-11-25T04:24:47.204 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:47.205 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:47.205 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:47.205 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.205 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:47.205 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.433304+0000", 2023-11-25T04:24:47.205 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:47.205 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:47.206 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.433304+0000", 2023-11-25T04:24:47.206 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:47.206 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.206 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.206 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:47.207 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.433305+0000", 2023-11-25T04:24:47.207 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8099999999999998e-07 2023-11-25T04:24:47.207 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.207 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.208 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:47.209 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.433306+0000", 2023-11-25T04:24:47.209 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.578e-06 2023-11-25T04:24:47.210 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.210 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.210 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:47.210 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.433310+0000", 2023-11-25T04:24:47.210 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.771e-06 2023-11-25T04:24:47.211 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.211 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.211 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:47.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.433313+0000", 2023-11-25T04:24:47.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.864e-06 2023-11-25T04:24:47.214 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.214 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.214 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:47.214 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.433350+0000", 2023-11-25T04:24:47.214 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7398999999999999e-05 2023-11-25T04:24:47.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.215 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.215 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:47.215 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.433454+0000", 2023-11-25T04:24:47.215 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103837 2023-11-25T04:24:47.215 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.216 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.216 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:47.216 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.489800+0000", 2023-11-25T04:24:47.216 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056345619999999999 2023-11-25T04:24:47.216 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:47.216 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:47.216 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:47.216 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.217 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:3696 4.28 4:166e0f51:::10000000056.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:47.217 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:44.433361+0000", 2023-11-25T04:24:47.217 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.7678253740000001, 2023-11-25T04:24:47.217 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055970330999999998, 2023-11-25T04:24:47.217 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:47.217 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:47.217 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:47.217 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.218 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:47.218 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.433361+0000", 2023-11-25T04:24:47.218 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:47.218 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.218 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.218 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:47.218 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.433361+0000", 2023-11-25T04:24:47.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:47.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.219 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:47.219 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.433362+0000", 2023-11-25T04:24:47.219 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6199999999999998e-07 2023-11-25T04:24:47.219 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.219 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:47.220 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.433364+0000", 2023-11-25T04:24:47.220 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.323e-06 2023-11-25T04:24:47.220 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.220 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.220 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:47.221 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.433371+0000", 2023-11-25T04:24:47.222 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5899999999999996e-06 2023-11-25T04:24:47.222 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.222 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.223 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:47.223 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.433376+0000", 2023-11-25T04:24:47.223 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9269999999999996e-06 2023-11-25T04:24:47.223 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.224 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.224 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:47.224 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.433418+0000", 2023-11-25T04:24:47.224 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2867000000000001e-05 2023-11-25T04:24:47.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.225 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.225 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:47.225 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.433522+0000", 2023-11-25T04:24:47.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103421 2023-11-25T04:24:47.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.226 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:47.226 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.489331+0000", 2023-11-25T04:24:47.226 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055809641 2023-11-25T04:24:47.227 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:47.227 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:47.227 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:47.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.227 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:3731 4.19 4:99ec5763:::10000000079.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:47.227 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:44.434107+0000", 2023-11-25T04:24:47.227 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.7670797039999999, 2023-11-25T04:24:47.228 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055803469000000001, 2023-11-25T04:24:47.228 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:47.228 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:47.228 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:47.228 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.228 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:47.229 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.434107+0000", 2023-11-25T04:24:47.229 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:47.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.229 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.230 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:47.230 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.434107+0000", 2023-11-25T04:24:47.230 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:47.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.231 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.231 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:47.231 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.434107+0000", 2023-11-25T04:24:47.232 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9900000000000001e-07 2023-11-25T04:24:47.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.232 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.232 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:47.232 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.434109+0000", 2023-11-25T04:24:47.233 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.553e-06 2023-11-25T04:24:47.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.233 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.233 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:47.233 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.434114+0000", 2023-11-25T04:24:47.234 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7320000000000003e-06 2023-11-25T04:24:47.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.234 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.234 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:47.234 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.434119+0000", 2023-11-25T04:24:47.235 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1080000000000003e-06 2023-11-25T04:24:47.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.235 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.235 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:47.236 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.434370+0000", 2023-11-25T04:24:47.236 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00025150999999999998 2023-11-25T04:24:47.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.236 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.236 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:47.237 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.434475+0000", 2023-11-25T04:24:47.237 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000105016 2023-11-25T04:24:47.237 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.237 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.238 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:47.238 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.489910+0000", 2023-11-25T04:24:47.238 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055435151000000002 2023-11-25T04:24:47.238 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:47.239 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:47.239 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:47.239 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.239 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.240 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:3901 4.36 4:6f395059:::10000000123.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:47.240 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:44.440446+0000", 2023-11-25T04:24:47.240 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.7607400229999999, 2023-11-25T04:24:47.240 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.062374564, 2023-11-25T04:24:47.241 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:47.241 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:47.241 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:47.242 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.242 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:47.242 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.440446+0000", 2023-11-25T04:24:47.242 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:47.243 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.243 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.243 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:47.243 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.440446+0000", 2023-11-25T04:24:47.244 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:47.244 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.244 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.244 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:47.245 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.440447+0000", 2023-11-25T04:24:47.245 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7599999999999997e-07 2023-11-25T04:24:47.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.245 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.246 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:47.246 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.440449+0000", 2023-11-25T04:24:47.246 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9609999999999999e-06 2023-11-25T04:24:47.246 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.246 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:47.247 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.440455+0000", 2023-11-25T04:24:47.247 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.392e-06 2023-11-25T04:24:47.247 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.247 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.247 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:47.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.440460+0000", 2023-11-25T04:24:47.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.916e-06 2023-11-25T04:24:47.248 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.248 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:47.248 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.444413+0000", 2023-11-25T04:24:47.248 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039524959999999998 2023-11-25T04:24:47.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.249 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.249 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:47.249 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.444512+0000", 2023-11-25T04:24:47.249 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8876999999999995e-05 2023-11-25T04:24:47.250 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.250 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.250 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:47.251 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.502821+0000", 2023-11-25T04:24:47.251 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.058309446000000001 2023-11-25T04:24:47.251 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:47.251 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:47.251 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:47.252 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.252 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.252 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:3922 4.6 4:629cc6bc:::10000000138.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:47.252 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:44.441824+0000", 2023-11-25T04:24:47.252 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.759362935, 2023-11-25T04:24:47.252 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061109164000000001, 2023-11-25T04:24:47.253 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:47.253 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:47.253 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:47.253 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.254 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:47.254 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.441824+0000", 2023-11-25T04:24:47.254 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:47.254 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.255 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:47.255 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.441824+0000", 2023-11-25T04:24:47.255 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:47.256 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.256 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.256 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:47.256 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.441826+0000", 2023-11-25T04:24:47.257 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.599e-06 2023-11-25T04:24:47.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.257 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.257 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:47.257 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.441829+0000", 2023-11-25T04:24:47.258 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.055e-06 2023-11-25T04:24:47.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.258 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.258 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:47.259 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.441835+0000", 2023-11-25T04:24:47.259 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.276e-06 2023-11-25T04:24:47.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.259 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.260 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:47.260 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.441840+0000", 2023-11-25T04:24:47.261 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9599999999999997e-06 2023-11-25T04:24:47.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.261 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.262 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:47.262 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.445570+0000", 2023-11-25T04:24:47.262 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003729951 2023-11-25T04:24:47.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.263 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.263 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:47.263 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.445680+0000", 2023-11-25T04:24:47.263 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000109807 2023-11-25T04:24:47.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.264 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:47.264 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.502933+0000", 2023-11-25T04:24:47.264 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057252516000000003 2023-11-25T04:24:47.264 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:47.265 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:47.265 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:47.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.265 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.266 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:3926 4.36 4:6c5ddb65:::1000000013c.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:47.266 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:44.443003+0000", 2023-11-25T04:24:47.266 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.7581832660000001, 2023-11-25T04:24:47.266 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.059937957, 2023-11-25T04:24:47.266 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:47.267 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:47.269 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:47.269 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.269 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:47.269 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.443003+0000", 2023-11-25T04:24:47.270 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:47.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.270 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.270 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:47.270 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.443003+0000", 2023-11-25T04:24:47.271 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:47.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.271 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.272 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:47.272 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.443004+0000", 2023-11-25T04:24:47.272 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6100000000000001e-07 2023-11-25T04:24:47.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.273 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.273 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:47.273 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.443012+0000", 2023-11-25T04:24:47.274 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0090000000000007e-06 2023-11-25T04:24:47.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.274 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.275 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:47.275 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.443019+0000", 2023-11-25T04:24:47.275 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0500000000000003e-06 2023-11-25T04:24:47.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.276 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:47.276 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.443025+0000", 2023-11-25T04:24:47.276 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1800000000000001e-06 2023-11-25T04:24:47.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.277 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:47.277 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.445746+0000", 2023-11-25T04:24:47.277 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0027212640000000001 2023-11-25T04:24:47.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.278 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.278 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:47.278 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.445841+0000", 2023-11-25T04:24:47.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4488999999999996e-05 2023-11-25T04:24:47.279 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.279 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.279 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:47.279 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.502941+0000", 2023-11-25T04:24:47.279 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057100404 2023-11-25T04:24:47.280 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:47.280 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:47.280 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:47.280 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.280 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.280 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:3933 4.2e 4:77f9b43c:::10000000143.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:47.280 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:44.443095+0000", 2023-11-25T04:24:47.281 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.7580919870000002, 2023-11-25T04:24:47.281 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.059948544999999999, 2023-11-25T04:24:47.281 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:47.281 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:47.281 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:47.281 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.282 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:47.282 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.443095+0000", 2023-11-25T04:24:47.282 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:47.282 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:47.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.443095+0000", 2023-11-25T04:24:47.283 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:47.283 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.283 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.283 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:47.283 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.443095+0000", 2023-11-25T04:24:47.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5499999999999998e-07 2023-11-25T04:24:47.284 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.284 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.284 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:47.284 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.443097+0000", 2023-11-25T04:24:47.284 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7549999999999999e-06 2023-11-25T04:24:47.285 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.286 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.286 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:47.286 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.443103+0000", 2023-11-25T04:24:47.287 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6449999999999999e-06 2023-11-25T04:24:47.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.287 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.287 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:47.288 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.443108+0000", 2023-11-25T04:24:47.288 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6829999999999999e-06 2023-11-25T04:24:47.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.289 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.289 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:47.289 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.445970+0000", 2023-11-25T04:24:47.289 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002862 2023-11-25T04:24:47.289 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.290 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:47.290 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446688+0000", 2023-11-25T04:24:47.290 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00071754200000000005 2023-11-25T04:24:47.290 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.290 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.290 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:47.290 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.503043+0000", 2023-11-25T04:24:47.291 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056355465 2023-11-25T04:24:47.291 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:47.291 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:47.291 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:47.291 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.291 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.291 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:3994 4.6 4:63fec8d1:::10000000180.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:47.291 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:44.444709+0000", 2023-11-25T04:24:47.292 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.7564775209999999, 2023-11-25T04:24:47.292 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.058338905000000003, 2023-11-25T04:24:47.292 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:47.292 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:47.292 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:47.292 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.292 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:47.292 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.444709+0000", 2023-11-25T04:24:47.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:47.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.293 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:47.293 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.444709+0000", 2023-11-25T04:24:47.293 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:47.293 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.293 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.293 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:47.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.444709+0000", 2023-11-25T04:24:47.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7e-07 2023-11-25T04:24:47.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.294 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.294 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:47.294 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.444711+0000", 2023-11-25T04:24:47.294 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.635e-06 2023-11-25T04:24:47.294 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:47.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.444724+0000", 2023-11-25T04:24:47.295 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2753e-05 2023-11-25T04:24:47.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.295 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.295 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:47.295 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.444734+0000", 2023-11-25T04:24:47.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.007e-05 2023-11-25T04:24:47.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.296 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:47.296 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447792+0000", 2023-11-25T04:24:47.296 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0030580630000000002 2023-11-25T04:24:47.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.296 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:47.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447909+0000", 2023-11-25T04:24:47.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011692 2023-11-25T04:24:47.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.297 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.297 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:47.297 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.503048+0000", 2023-11-25T04:24:47.297 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055139094 2023-11-25T04:24:47.298 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:47.298 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:47.298 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:47.298 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.298 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.298 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:3997 4.6 4:60954073:::10000000183.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:47.298 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:44.444741+0000", 2023-11-25T04:24:47.299 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.7564453059999998, 2023-11-25T04:24:47.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.058419077, 2023-11-25T04:24:47.299 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:47.299 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:47.299 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:47.299 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.299 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:47.299 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.444741+0000", 2023-11-25T04:24:47.299 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:47.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.300 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:47.300 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.444741+0000", 2023-11-25T04:24:47.300 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:47.300 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.300 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.300 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:47.301 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.444742+0000", 2023-11-25T04:24:47.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1100000000000001e-07 2023-11-25T04:24:47.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.301 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.301 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:47.301 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.444744+0000", 2023-11-25T04:24:47.301 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.013e-06 2023-11-25T04:24:47.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:47.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.444750+0000", 2023-11-25T04:24:47.302 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2299999999999996e-06 2023-11-25T04:24:47.302 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.302 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.302 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:47.302 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.444757+0000", 2023-11-25T04:24:47.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9870000000000002e-06 2023-11-25T04:24:47.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.303 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:47.303 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.448162+0000", 2023-11-25T04:24:47.303 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0034051020000000001 2023-11-25T04:24:47.303 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.303 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.303 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:47.304 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.448494+0000", 2023-11-25T04:24:47.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00033231800000000001 2023-11-25T04:24:47.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.304 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.304 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:47.304 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.503160+0000", 2023-11-25T04:24:47.304 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054666015999999998 2023-11-25T04:24:47.304 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:47.305 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:47.305 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:47.305 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.305 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.305 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:4032 4.2e 4:775b9fb2:::100000001a6.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:47.305 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:44.446660+0000", 2023-11-25T04:24:47.305 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.7545268520000001, 2023-11-25T04:24:47.305 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056503945, 2023-11-25T04:24:47.306 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:47.306 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:47.306 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:47.306 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.306 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:47.306 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446660+0000", 2023-11-25T04:24:47.306 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:47.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:47.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446660+0000", 2023-11-25T04:24:47.307 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:47.307 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.307 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.307 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:47.307 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446660+0000", 2023-11-25T04:24:47.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5499999999999998e-07 2023-11-25T04:24:47.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.308 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.308 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:47.308 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446662+0000", 2023-11-25T04:24:47.308 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.798e-06 2023-11-25T04:24:47.308 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.309 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:47.309 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446667+0000", 2023-11-25T04:24:47.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2349999999999999e-06 2023-11-25T04:24:47.309 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.309 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.309 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:47.309 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446670+0000", 2023-11-25T04:24:47.309 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0699999999999998e-06 2023-11-25T04:24:47.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:47.310 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.455057+0000", 2023-11-25T04:24:47.310 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0083872570000000004 2023-11-25T04:24:47.310 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.310 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.310 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:47.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.455340+0000", 2023-11-25T04:24:47.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028249500000000001 2023-11-25T04:24:47.311 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.311 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.311 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:47.311 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.503164+0000", 2023-11-25T04:24:47.311 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.047823635000000003 2023-11-25T04:24:47.311 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:47.312 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:47.312 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:47.312 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.312 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.312 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:4037 4.2e 4:76865fdc:::100000001ab.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:47.312 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:44.446688+0000", 2023-11-25T04:24:47.312 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.7544986389999999, 2023-11-25T04:24:47.312 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056587089, 2023-11-25T04:24:47.313 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:47.313 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:47.313 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:47.313 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.313 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:47.313 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446688+0000", 2023-11-25T04:24:47.313 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:47.313 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.314 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:47.314 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446688+0000", 2023-11-25T04:24:47.314 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:47.314 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.314 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.314 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:47.314 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446688+0000", 2023-11-25T04:24:47.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2399999999999999e-07 2023-11-25T04:24:47.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.315 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:47.315 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446690+0000", 2023-11-25T04:24:47.315 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.911e-06 2023-11-25T04:24:47.315 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.315 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.316 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:47.316 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446696+0000", 2023-11-25T04:24:47.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4859999999999997e-06 2023-11-25T04:24:47.316 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.316 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.316 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:47.316 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446700+0000", 2023-11-25T04:24:47.316 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7099999999999998e-06 2023-11-25T04:24:47.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.317 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:47.317 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.455754+0000", 2023-11-25T04:24:47.317 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0090534350000000003 2023-11-25T04:24:47.317 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.317 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.317 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:47.318 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.455881+0000", 2023-11-25T04:24:47.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000126818 2023-11-25T04:24:47.318 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.318 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.318 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:47.318 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.503275+0000", 2023-11-25T04:24:47.318 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.047394304999999998 2023-11-25T04:24:47.318 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:47.319 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:47.319 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:47.319 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.319 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.319 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:4044 4.2e 4:76ff63fc:::100000001b2.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:47.319 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:44.446860+0000", 2023-11-25T04:24:47.319 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.7543267939999998, 2023-11-25T04:24:47.319 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056538421999999998, 2023-11-25T04:24:47.320 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:47.320 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:47.320 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:47.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.320 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:47.320 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446860+0000", 2023-11-25T04:24:47.320 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:47.320 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.320 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:47.321 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446860+0000", 2023-11-25T04:24:47.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:47.321 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.321 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.321 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:47.321 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446860+0000", 2023-11-25T04:24:47.321 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8599999999999998e-07 2023-11-25T04:24:47.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:47.322 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446862+0000", 2023-11-25T04:24:47.322 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0650000000000001e-06 2023-11-25T04:24:47.322 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.322 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.322 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:47.323 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446869+0000", 2023-11-25T04:24:47.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1709999999999996e-06 2023-11-25T04:24:47.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.323 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.323 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:47.323 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446875+0000", 2023-11-25T04:24:47.323 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3870000000000003e-06 2023-11-25T04:24:47.323 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.324 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.324 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:47.324 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.456185+0000", 2023-11-25T04:24:47.324 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0093104260000000001 2023-11-25T04:24:47.324 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.324 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.324 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:47.324 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.456285+0000", 2023-11-25T04:24:47.324 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9393000000000002e-05 2023-11-25T04:24:47.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.325 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.325 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:47.325 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.503398+0000", 2023-11-25T04:24:47.325 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.047113493999999999 2023-11-25T04:24:47.325 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:47.325 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:47.325 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:47.325 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.326 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.326 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:4049 4.19 4:99a7d964:::100000001b7.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:47.326 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:44.446931+0000", 2023-11-25T04:24:47.326 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.754255525, 2023-11-25T04:24:47.326 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056039185999999998, 2023-11-25T04:24:47.326 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:47.326 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:47.326 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:47.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.327 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:47.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446931+0000", 2023-11-25T04:24:47.327 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:47.327 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.327 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.327 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:47.327 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446931+0000", 2023-11-25T04:24:47.328 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:47.328 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.328 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.328 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:47.328 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446931+0000", 2023-11-25T04:24:47.328 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.84e-07 2023-11-25T04:24:47.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.329 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.329 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:47.329 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446939+0000", 2023-11-25T04:24:47.329 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2010000000000002e-06 2023-11-25T04:24:47.329 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.330 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.330 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:47.331 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446945+0000", 2023-11-25T04:24:47.331 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5429999999999999e-06 2023-11-25T04:24:47.331 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.331 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.331 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:47.331 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446950+0000", 2023-11-25T04:24:47.331 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0560000000000004e-06 2023-11-25T04:24:47.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.332 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.332 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:47.332 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446999+0000", 2023-11-25T04:24:47.332 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8594000000000003e-05 2023-11-25T04:24:47.332 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.332 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.332 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:47.333 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447095+0000", 2023-11-25T04:24:47.333 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6638999999999994e-05 2023-11-25T04:24:47.333 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.333 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.333 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:47.333 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.502970+0000", 2023-11-25T04:24:47.333 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055874768999999998 2023-11-25T04:24:47.333 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:47.334 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:47.334 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:47.334 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.334 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.334 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:4052 4.36 4:6d15073c:::100000001ba.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:47.334 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:44.446994+0000", 2023-11-25T04:24:47.334 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.7541922570000001, 2023-11-25T04:24:47.334 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056265880999999997, 2023-11-25T04:24:47.335 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:47.335 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:47.335 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:47.335 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.335 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:47.335 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446994+0000", 2023-11-25T04:24:47.335 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:47.335 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.336 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.336 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:47.336 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446994+0000", 2023-11-25T04:24:47.336 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:47.336 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.336 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.336 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:47.336 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446995+0000", 2023-11-25T04:24:47.336 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6300000000000001e-07 2023-11-25T04:24:47.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.337 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:47.337 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446996+0000", 2023-11-25T04:24:47.337 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.612e-06 2023-11-25T04:24:47.337 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.337 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:47.338 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447003+0000", 2023-11-25T04:24:47.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3069999999999999e-06 2023-11-25T04:24:47.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.338 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:47.338 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447007+0000", 2023-11-25T04:24:47.338 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3930000000000001e-06 2023-11-25T04:24:47.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.339 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:47.339 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.456139+0000", 2023-11-25T04:24:47.339 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0091321370000000002 2023-11-25T04:24:47.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.339 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:47.339 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.456249+0000", 2023-11-25T04:24:47.339 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000109491 2023-11-25T04:24:47.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.340 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:47.340 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.503260+0000", 2023-11-25T04:24:47.340 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.047011577999999998 2023-11-25T04:24:47.340 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:47.340 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:47.340 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:47.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.341 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:4054 4.19 4:9b54a4b5:::100000001bc.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:47.341 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:44.447013+0000", 2023-11-25T04:24:47.341 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.7541737930000001, 2023-11-25T04:24:47.342 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056061487, 2023-11-25T04:24:47.342 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:47.342 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:47.342 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:47.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.343 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:47.343 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447013+0000", 2023-11-25T04:24:47.343 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:47.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.344 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:47.344 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447013+0000", 2023-11-25T04:24:47.344 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:47.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.345 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:47.345 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447013+0000", 2023-11-25T04:24:47.345 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7700000000000005e-07 2023-11-25T04:24:47.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.346 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:47.346 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447015+0000", 2023-11-25T04:24:47.346 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5939999999999999e-06 2023-11-25T04:24:47.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.347 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:47.347 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447020+0000", 2023-11-25T04:24:47.348 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6030000000000004e-06 2023-11-25T04:24:47.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.348 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:47.348 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447026+0000", 2023-11-25T04:24:47.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1420000000000001e-06 2023-11-25T04:24:47.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.349 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:47.349 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447490+0000", 2023-11-25T04:24:47.349 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00046441800000000002 2023-11-25T04:24:47.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.350 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:47.350 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447568+0000", 2023-11-25T04:24:47.350 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7751999999999997e-05 2023-11-25T04:24:47.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.350 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:47.350 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.503074+0000", 2023-11-25T04:24:47.351 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055506501 2023-11-25T04:24:47.351 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:47.351 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:47.351 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:47.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.351 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:4057 4.19 4:9ab6d71b:::100000001bf.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:47.351 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:44.447032+0000", 2023-11-25T04:24:47.351 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.7541547240000002, 2023-11-25T04:24:47.352 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056145928999999997, 2023-11-25T04:24:47.352 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:47.352 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:47.352 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:47.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.352 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:47.352 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447032+0000", 2023-11-25T04:24:47.352 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:47.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.353 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:47.353 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447032+0000", 2023-11-25T04:24:47.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:47.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.353 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.353 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:47.353 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447032+0000", 2023-11-25T04:24:47.353 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3500000000000002e-07 2023-11-25T04:24:47.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.354 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:47.354 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447034+0000", 2023-11-25T04:24:47.354 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.832e-06 2023-11-25T04:24:47.354 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.354 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.354 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:47.355 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447040+0000", 2023-11-25T04:24:47.355 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6779999999999997e-06 2023-11-25T04:24:47.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.355 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.355 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:47.355 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447044+0000", 2023-11-25T04:24:47.355 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.391e-06 2023-11-25T04:24:47.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.355 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.356 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:47.356 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447807+0000", 2023-11-25T04:24:47.356 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00076306799999999999 2023-11-25T04:24:47.356 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.356 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.356 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:47.356 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447931+0000", 2023-11-25T04:24:47.356 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012439999999999999 2023-11-25T04:24:47.356 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.357 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.357 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:47.357 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.503178+0000", 2023-11-25T04:24:47.357 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055246225000000003 2023-11-25T04:24:47.357 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:47.357 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:47.357 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:47.357 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.358 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:4058 4.19 4:98a8a8b3:::100000001c0.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:47.358 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:44.447050+0000", 2023-11-25T04:24:47.358 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.7541364229999998, 2023-11-25T04:24:47.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.05633759, 2023-11-25T04:24:47.358 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:47.358 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:47.358 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:47.358 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:47.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447050+0000", 2023-11-25T04:24:47.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:47.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.359 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:47.359 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447050+0000", 2023-11-25T04:24:47.359 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:47.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.360 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:47.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447051+0000", 2023-11-25T04:24:47.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1900000000000003e-07 2023-11-25T04:24:47.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.360 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.360 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:47.360 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447053+0000", 2023-11-25T04:24:47.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.108e-06 2023-11-25T04:24:47.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:47.361 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447058+0000", 2023-11-25T04:24:47.361 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3159999999999996e-06 2023-11-25T04:24:47.361 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.361 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.361 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:47.362 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447063+0000", 2023-11-25T04:24:47.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5449999999999997e-06 2023-11-25T04:24:47.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.362 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:47.362 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.448196+0000", 2023-11-25T04:24:47.362 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001133128 2023-11-25T04:24:47.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:47.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.448895+0000", 2023-11-25T04:24:47.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00069910700000000003 2023-11-25T04:24:47.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.363 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:47.363 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.503388+0000", 2023-11-25T04:24:47.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054492867 2023-11-25T04:24:47.364 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:47.364 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:47.364 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:47.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.364 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.364 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:4077 4.3e 4:7c32b2c0:::100000001d3.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:47.364 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:44.448674+0000", 2023-11-25T04:24:47.364 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.7525120799999998, 2023-11-25T04:24:47.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054710506999999999, 2023-11-25T04:24:47.365 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:47.365 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:47.365 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:47.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.365 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:47.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.448674+0000", 2023-11-25T04:24:47.365 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:47.365 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:47.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.448674+0000", 2023-11-25T04:24:47.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:47.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.366 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.366 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:47.366 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.448675+0000", 2023-11-25T04:24:47.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0199999999999998e-07 2023-11-25T04:24:47.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:47.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.448676+0000", 2023-11-25T04:24:47.367 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.646e-06 2023-11-25T04:24:47.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:47.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.448682+0000", 2023-11-25T04:24:47.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.981e-06 2023-11-25T04:24:47.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.368 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:47.368 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.448687+0000", 2023-11-25T04:24:47.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.853e-06 2023-11-25T04:24:47.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:47.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.456676+0000", 2023-11-25T04:24:47.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0079888660000000007 2023-11-25T04:24:47.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:47.369 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.456799+0000", 2023-11-25T04:24:47.369 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012312799999999999 2023-11-25T04:24:47.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.370 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:47.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.503385+0000", 2023-11-25T04:24:47.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046585730999999998 2023-11-25T04:24:47.370 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:47.370 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:47.370 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:47.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.371 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:4083 4.6 4:61ce4130:::100000001d9.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:47.371 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:44.448770+0000", 2023-11-25T04:24:47.371 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.752416985, 2023-11-25T04:24:47.371 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054722232000000003, 2023-11-25T04:24:47.371 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:47.371 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:47.371 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:47.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:47.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.448770+0000", 2023-11-25T04:24:47.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:47.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.372 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:47.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.448770+0000", 2023-11-25T04:24:47.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:47.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:47.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.448770+0000", 2023-11-25T04:24:47.373 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5499999999999999e-07 2023-11-25T04:24:47.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.373 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:47.373 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.448772+0000", 2023-11-25T04:24:47.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.72e-06 2023-11-25T04:24:47.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:47.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.448779+0000", 2023-11-25T04:24:47.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2139999999999999e-06 2023-11-25T04:24:47.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:47.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.448784+0000", 2023-11-25T04:24:47.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7299999999999996e-06 2023-11-25T04:24:47.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.375 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:47.375 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.456849+0000", 2023-11-25T04:24:47.375 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0080652759999999997 2023-11-25T04:24:47.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:47.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.456950+0000", 2023-11-25T04:24:47.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000101551 2023-11-25T04:24:47.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:47.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.503492+0000", 2023-11-25T04:24:47.376 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046541385999999997 2023-11-25T04:24:47.376 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:47.377 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:47.377 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:47.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.377 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:4088 4.2e 4:77eaaa70:::100000001de.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:47.377 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:44.448828+0000", 2023-11-25T04:24:47.377 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 2.7523581149999998, 2023-11-25T04:24:47.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054684280000000002, 2023-11-25T04:24:47.378 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:47.378 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:47.378 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:47.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.378 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:47.378 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.448828+0000", 2023-11-25T04:24:47.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:47.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.379 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:47.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.448828+0000", 2023-11-25T04:24:47.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:47.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.379 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:47.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.448829+0000", 2023-11-25T04:24:47.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5999999999999999e-07 2023-11-25T04:24:47.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.380 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:47.380 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.448831+0000", 2023-11-25T04:24:47.380 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7400000000000001e-06 2023-11-25T04:24:47.380 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:47.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.448836+0000", 2023-11-25T04:24:47.381 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7309999999999998e-06 2023-11-25T04:24:47.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.381 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:47.381 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.448842+0000", 2023-11-25T04:24:47.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3979999999999998e-06 2023-11-25T04:24:47.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.382 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:47.382 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.457248+0000", 2023-11-25T04:24:47.382 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0084061819999999999 2023-11-25T04:24:47.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:47.383 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.457351+0000", 2023-11-25T04:24:47.383 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010357399999999999 2023-11-25T04:24:47.383 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:47.383 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:47.383 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:47.384 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.503513+0000", 2023-11-25T04:24:47.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046161294999999998 2023-11-25T04:24:47.384 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:47.385 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:47.385 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:47.385 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:47.385 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:47.385 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:47.387 INFO:teuthology.orchestra.run.smithi067.stderr:noscrub is unset 2023-11-25T04:24:47.387 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:24:47.388 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:47.385+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:47.487 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:47.486+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:47.495 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:47.496 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:24:47.496 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2023-11-25T04:24:47.496 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:47.504 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:24:47.588 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:47.587+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:47.650 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:47.650 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:24:47.651 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2023-11-25T04:24:47.651 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2023-11-25T04:24:47.651 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:47.659 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:24:47.690 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:47.688+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:47.766 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:47.766 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2023-11-25T04:24:47.766 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2023-11-25T04:24:47.766 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2023-11-25T04:24:47.767 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.767 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:3624 4.37 4:eefe613f:::1000000000e.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:47.767 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:44.430400+0000", 2023-11-25T04:24:47.767 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.333664604, 2023-11-25T04:24:47.767 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.052686746, 2023-11-25T04:24:47.767 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:47.767 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:47.768 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:47.768 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.768 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:47.768 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.430400+0000", 2023-11-25T04:24:47.768 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:47.768 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.768 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.768 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:47.769 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.430400+0000", 2023-11-25T04:24:47.769 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:47.769 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.770 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.771 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:47.771 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.430400+0000", 2023-11-25T04:24:47.772 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6800000000000001e-07 2023-11-25T04:24:47.772 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.772 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.772 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:47.772 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.430403+0000", 2023-11-25T04:24:47.772 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7700000000000002e-06 2023-11-25T04:24:47.772 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.773 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.773 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:47.774 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.430414+0000", 2023-11-25T04:24:47.774 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0798000000000001e-05 2023-11-25T04:24:47.775 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.775 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.775 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:47.775 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.430419+0000", 2023-11-25T04:24:47.776 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8679999999999998e-06 2023-11-25T04:24:47.776 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.776 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.776 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:47.776 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.430455+0000", 2023-11-25T04:24:47.777 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6764999999999998e-05 2023-11-25T04:24:47.777 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.777 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.777 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:47.778 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.430568+0000", 2023-11-25T04:24:47.778 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000113008 2023-11-25T04:24:47.778 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.778 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.779 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:47.779 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.483087+0000", 2023-11-25T04:24:47.779 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.052518169000000003 2023-11-25T04:24:47.779 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:47.780 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:47.780 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:47.780 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.780 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.781 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:3625 4.12 4:49e35fbc:::1000000000f.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:47.781 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:44.430429+0000", 2023-11-25T04:24:47.781 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.3336359079999998, 2023-11-25T04:24:47.781 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053415323000000001, 2023-11-25T04:24:47.782 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:47.782 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:47.782 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:47.782 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.783 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:47.783 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.430429+0000", 2023-11-25T04:24:47.783 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:47.783 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.783 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.784 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:47.784 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.430429+0000", 2023-11-25T04:24:47.784 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:47.784 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.785 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.785 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:47.785 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.430429+0000", 2023-11-25T04:24:47.785 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0999999999999999e-07 2023-11-25T04:24:47.786 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.786 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.786 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:47.786 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.430431+0000", 2023-11-25T04:24:47.787 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3779999999999999e-06 2023-11-25T04:24:47.787 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.787 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.787 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:47.788 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.430438+0000", 2023-11-25T04:24:47.788 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3199999999999996e-06 2023-11-25T04:24:47.788 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.788 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.789 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:47.789 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.430449+0000", 2023-11-25T04:24:47.789 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0834e-05 2023-11-25T04:24:47.789 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.790 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.790 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:47.790 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.430514+0000", 2023-11-25T04:24:47.790 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5364999999999995e-05 2023-11-25T04:24:47.791 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.791 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.791 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:47.791 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.430621+0000", 2023-11-25T04:24:47.792 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000107416 2023-11-25T04:24:47.792 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.792 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.792 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:47.793 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.483844+0000", 2023-11-25T04:24:47.793 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053222499999999999 2023-11-25T04:24:47.793 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:47.793 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:47.793 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:47.794 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.794 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.794 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:3630 4.10 4:0b7da9c3:::10000000014.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:47.794 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:44.430468+0000", 2023-11-25T04:24:47.795 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.333596526, 2023-11-25T04:24:47.795 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053832480000000002, 2023-11-25T04:24:47.795 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:47.795 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:47.796 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:47.796 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.796 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:47.796 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.430468+0000", 2023-11-25T04:24:47.796 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:47.796 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.796 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.797 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:47.797 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.430468+0000", 2023-11-25T04:24:47.797 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:47.797 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.797 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.797 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:47.797 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.430468+0000", 2023-11-25T04:24:47.797 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0800000000000005e-07 2023-11-25T04:24:47.797 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.798 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.798 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:47.798 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.430471+0000", 2023-11-25T04:24:47.798 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5610000000000001e-06 2023-11-25T04:24:47.798 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.798 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.798 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:47.798 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.430492+0000", 2023-11-25T04:24:47.799 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1087e-05 2023-11-25T04:24:47.799 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.799 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.799 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:47.799 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.430498+0000", 2023-11-25T04:24:47.799 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9429999999999997e-06 2023-11-25T04:24:47.799 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.799 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.799 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:47.800 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.430550+0000", 2023-11-25T04:24:47.800 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1785e-05 2023-11-25T04:24:47.800 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.800 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.800 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:47.800 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.430666+0000", 2023-11-25T04:24:47.800 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000115926 2023-11-25T04:24:47.800 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.801 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.801 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:47.801 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.484300+0000", 2023-11-25T04:24:47.801 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053634670000000002 2023-11-25T04:24:47.801 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:47.801 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:47.801 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:47.801 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.801 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.802 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:3635 4.21 4:86d8200e:::10000000019.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:47.802 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:44.430901+0000", 2023-11-25T04:24:47.802 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.3331630969999999, 2023-11-25T04:24:47.802 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.057400916000000003, 2023-11-25T04:24:47.802 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:47.802 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:47.802 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:47.802 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.803 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:47.803 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.430901+0000", 2023-11-25T04:24:47.803 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:47.803 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.803 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.803 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:47.803 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.430901+0000", 2023-11-25T04:24:47.803 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:47.803 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.804 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.804 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:47.804 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.430902+0000", 2023-11-25T04:24:47.804 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.92e-07 2023-11-25T04:24:47.804 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.804 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.804 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:47.804 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.430906+0000", 2023-11-25T04:24:47.805 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5999999999999998e-06 2023-11-25T04:24:47.805 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.805 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.805 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:47.805 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.430914+0000", 2023-11-25T04:24:47.805 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9370000000000004e-06 2023-11-25T04:24:47.805 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.805 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.805 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:47.806 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.430920+0000", 2023-11-25T04:24:47.806 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3799999999999999e-06 2023-11-25T04:24:47.806 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.806 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.806 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:47.806 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.430971+0000", 2023-11-25T04:24:47.806 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0878999999999999e-05 2023-11-25T04:24:47.806 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.806 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.807 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:47.807 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.431092+0000", 2023-11-25T04:24:47.807 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000120951 2023-11-25T04:24:47.807 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.807 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.807 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:47.807 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.488302+0000", 2023-11-25T04:24:47.807 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.057210477000000003 2023-11-25T04:24:47.808 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:47.808 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:47.808 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:47.808 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.808 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.808 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:3646 4.2f 4:f6ddf183:::10000000024.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:47.808 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:44.431468+0000", 2023-11-25T04:24:47.808 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.332596347, 2023-11-25T04:24:47.808 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051625977000000003, 2023-11-25T04:24:47.809 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:47.809 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:47.809 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:47.809 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.809 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:47.809 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.431468+0000", 2023-11-25T04:24:47.809 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:47.809 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.809 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.810 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:47.810 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.431468+0000", 2023-11-25T04:24:47.810 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:47.810 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.810 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.810 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:47.810 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.431469+0000", 2023-11-25T04:24:47.810 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2399999999999998e-07 2023-11-25T04:24:47.811 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.811 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.811 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:47.811 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.431472+0000", 2023-11-25T04:24:47.811 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4570000000000001e-06 2023-11-25T04:24:47.811 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.811 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.811 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:47.811 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.431482+0000", 2023-11-25T04:24:47.812 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3070000000000008e-06 2023-11-25T04:24:47.812 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.812 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.812 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:47.812 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.431488+0000", 2023-11-25T04:24:47.812 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4239999999999998e-06 2023-11-25T04:24:47.812 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.812 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.812 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:47.813 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.431563+0000", 2023-11-25T04:24:47.813 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5179000000000006e-05 2023-11-25T04:24:47.813 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.813 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.813 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:47.813 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.431670+0000", 2023-11-25T04:24:47.813 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010675699999999999 2023-11-25T04:24:47.813 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.813 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.814 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:47.814 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.483094+0000", 2023-11-25T04:24:47.814 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051424229000000002 2023-11-25T04:24:47.814 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:47.814 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:47.814 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:47.814 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.814 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.815 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:3656 4.12 4:49ffa098:::1000000002e.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:47.815 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:44.431630+0000", 2023-11-25T04:24:47.815 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.3324344460000002, 2023-11-25T04:24:47.815 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.052311218, 2023-11-25T04:24:47.815 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:47.815 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:47.815 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:47.815 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.815 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:47.816 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.431630+0000", 2023-11-25T04:24:47.816 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:47.816 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.816 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.816 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:47.816 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.431630+0000", 2023-11-25T04:24:47.816 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:47.816 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.816 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.817 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:47.817 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.431631+0000", 2023-11-25T04:24:47.817 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9500000000000002e-07 2023-11-25T04:24:47.817 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.817 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.817 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:47.817 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.431633+0000", 2023-11-25T04:24:47.817 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9319999999999999e-06 2023-11-25T04:24:47.818 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.818 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.818 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:47.818 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.431638+0000", 2023-11-25T04:24:47.818 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4809999999999997e-06 2023-11-25T04:24:47.818 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.818 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.818 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:47.818 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.431643+0000", 2023-11-25T04:24:47.819 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4389999999999999e-06 2023-11-25T04:24:47.819 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.819 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.819 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:47.819 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.431695+0000", 2023-11-25T04:24:47.819 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1119999999999998e-05 2023-11-25T04:24:47.819 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.819 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.820 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:47.820 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.431795+0000", 2023-11-25T04:24:47.820 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000100696 2023-11-25T04:24:47.820 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.820 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.820 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:47.820 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.483941+0000", 2023-11-25T04:24:47.820 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.052145955000000001 2023-11-25T04:24:47.820 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:47.821 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:47.821 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:47.821 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.821 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.821 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:3657 4.23 4:c550f404:::1000000002f.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:47.821 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:44.432139+0000", 2023-11-25T04:24:47.821 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.3319259309999998, 2023-11-25T04:24:47.821 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051595467999999998, 2023-11-25T04:24:47.821 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:47.822 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:47.822 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:47.822 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.822 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:47.822 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432139+0000", 2023-11-25T04:24:47.822 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:47.822 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.823 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.823 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:47.823 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432139+0000", 2023-11-25T04:24:47.823 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:47.824 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.824 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.824 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:47.824 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432139+0000", 2023-11-25T04:24:47.824 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0200000000000002e-07 2023-11-25T04:24:47.824 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.824 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.824 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:47.825 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432143+0000", 2023-11-25T04:24:47.825 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3200000000000001e-06 2023-11-25T04:24:47.825 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.825 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.825 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:47.825 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432151+0000", 2023-11-25T04:24:47.825 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2219999999999996e-06 2023-11-25T04:24:47.825 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.826 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.826 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:47.826 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432162+0000", 2023-11-25T04:24:47.826 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1571e-05 2023-11-25T04:24:47.826 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.826 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.826 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:47.826 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432209+0000", 2023-11-25T04:24:47.827 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.6984000000000003e-05 2023-11-25T04:24:47.827 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.827 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.827 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:47.827 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432334+0000", 2023-11-25T04:24:47.827 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000124762 2023-11-25T04:24:47.827 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.827 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.828 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:47.828 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.483734+0000", 2023-11-25T04:24:47.828 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051400006999999998 2023-11-25T04:24:47.828 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:47.828 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:47.828 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:47.828 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.828 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.829 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:3674 4.23 4:c6a0a389:::10000000040.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:47.829 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:44.432419+0000", 2023-11-25T04:24:47.829 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.331645585, 2023-11-25T04:24:47.829 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051408338999999997, 2023-11-25T04:24:47.829 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:47.829 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:47.829 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:47.829 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.829 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:47.830 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432419+0000", 2023-11-25T04:24:47.830 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:47.830 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.830 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.830 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:47.830 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432419+0000", 2023-11-25T04:24:47.830 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:47.830 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.831 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.831 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:47.831 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432419+0000", 2023-11-25T04:24:47.831 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3399999999999999e-07 2023-11-25T04:24:47.831 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.831 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.831 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:47.831 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432421+0000", 2023-11-25T04:24:47.832 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9580000000000001e-06 2023-11-25T04:24:47.832 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.832 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.832 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:47.832 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432427+0000", 2023-11-25T04:24:47.832 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.7710000000000003e-06 2023-11-25T04:24:47.832 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.832 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.833 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:47.833 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432432+0000", 2023-11-25T04:24:47.833 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4680000000000003e-06 2023-11-25T04:24:47.833 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.833 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.833 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:47.833 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.433768+0000", 2023-11-25T04:24:47.834 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001336093 2023-11-25T04:24:47.834 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.834 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.834 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:47.834 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.433872+0000", 2023-11-25T04:24:47.834 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000104609 2023-11-25T04:24:47.834 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.835 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.835 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:47.835 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.483827+0000", 2023-11-25T04:24:47.835 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.049954906 2023-11-25T04:24:47.835 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:47.835 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:47.835 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:47.835 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.836 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.836 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:3682 4.12 4:4899516f:::10000000048.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:47.836 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:44.432556+0000", 2023-11-25T04:24:47.836 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.3315085079999998, 2023-11-25T04:24:47.836 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051716377000000001, 2023-11-25T04:24:47.836 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:47.836 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:47.836 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:47.837 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.837 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:47.837 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432556+0000", 2023-11-25T04:24:47.837 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:47.837 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.837 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.837 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:47.837 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432556+0000", 2023-11-25T04:24:47.838 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:47.838 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.838 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.838 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:47.838 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432556+0000", 2023-11-25T04:24:47.839 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.63e-07 2023-11-25T04:24:47.839 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.839 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.839 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:47.839 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432559+0000", 2023-11-25T04:24:47.840 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3269999999999999e-06 2023-11-25T04:24:47.840 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.840 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.840 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:47.840 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432566+0000", 2023-11-25T04:24:47.840 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9779999999999999e-06 2023-11-25T04:24:47.840 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.840 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.841 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:47.841 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432572+0000", 2023-11-25T04:24:47.841 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.161e-06 2023-11-25T04:24:47.841 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.841 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.841 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:47.842 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.433149+0000", 2023-11-25T04:24:47.842 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00057659900000000001 2023-11-25T04:24:47.842 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.842 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.842 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:47.842 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.433519+0000", 2023-11-25T04:24:47.842 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00037059899999999999 2023-11-25T04:24:47.843 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.843 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.843 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:47.843 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.484272+0000", 2023-11-25T04:24:47.843 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.05075325 2023-11-25T04:24:47.843 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:47.843 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:47.843 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:47.844 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.844 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.844 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:3686 4.21 4:844eae9e:::1000000004c.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:47.844 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:44.432639+0000", 2023-11-25T04:24:47.844 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.3314251590000001, 2023-11-25T04:24:47.844 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.056557505000000001, 2023-11-25T04:24:47.844 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:47.844 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:47.845 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:47.845 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.845 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:47.845 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432639+0000", 2023-11-25T04:24:47.845 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:47.845 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.845 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.845 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:47.846 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432639+0000", 2023-11-25T04:24:47.846 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:47.846 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.846 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.846 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:47.846 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432640+0000", 2023-11-25T04:24:47.846 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3900000000000005e-07 2023-11-25T04:24:47.846 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.847 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.847 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:47.847 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432642+0000", 2023-11-25T04:24:47.847 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1900000000000002e-06 2023-11-25T04:24:47.847 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.847 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.847 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:47.847 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432648+0000", 2023-11-25T04:24:47.848 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0619999999999997e-06 2023-11-25T04:24:47.848 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.848 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.848 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:47.848 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432653+0000", 2023-11-25T04:24:47.848 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5410000000000002e-06 2023-11-25T04:24:47.848 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.849 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.849 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:47.849 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432699+0000", 2023-11-25T04:24:47.849 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.6681000000000002e-05 2023-11-25T04:24:47.849 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.849 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.849 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:47.849 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432799+0000", 2023-11-25T04:24:47.850 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9264000000000004e-05 2023-11-25T04:24:47.850 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.850 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.850 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:47.850 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.489197+0000", 2023-11-25T04:24:47.850 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.056398228000000002 2023-11-25T04:24:47.850 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:47.850 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:47.851 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:47.851 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.851 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.851 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:3689 4.21 4:8660ae9f:::1000000004f.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:47.851 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:44.432696+0000", 2023-11-25T04:24:47.851 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.331368157, 2023-11-25T04:24:47.851 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.05686683, 2023-11-25T04:24:47.852 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:47.852 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:47.852 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:47.852 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.852 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:47.852 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432696+0000", 2023-11-25T04:24:47.852 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:47.852 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.853 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.853 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:47.853 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432696+0000", 2023-11-25T04:24:47.853 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:47.853 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.853 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.854 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:47.854 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432697+0000", 2023-11-25T04:24:47.854 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5499999999999999e-07 2023-11-25T04:24:47.854 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.854 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.854 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:47.854 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432699+0000", 2023-11-25T04:24:47.855 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0030000000000001e-06 2023-11-25T04:24:47.855 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.855 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.855 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:47.855 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432712+0000", 2023-11-25T04:24:47.855 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3803e-05 2023-11-25T04:24:47.855 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.855 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.856 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:47.856 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432717+0000", 2023-11-25T04:24:47.856 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7369999999999997e-06 2023-11-25T04:24:47.856 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.856 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.856 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:47.856 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.433424+0000", 2023-11-25T04:24:47.856 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00070664700000000003 2023-11-25T04:24:47.857 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.857 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.857 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:47.857 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.433561+0000", 2023-11-25T04:24:47.857 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013756400000000001 2023-11-25T04:24:47.857 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.858 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.858 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:47.858 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.489563+0000", 2023-11-25T04:24:47.858 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.056001720999999997 2023-11-25T04:24:47.858 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:47.858 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:47.858 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:47.859 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.859 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.859 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:3703 4.2d 4:b5236558:::1000000005d.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:47.859 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:44.432912+0000", 2023-11-25T04:24:47.859 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.3311521100000001, 2023-11-25T04:24:47.859 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051876347000000003, 2023-11-25T04:24:47.859 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:47.859 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:47.860 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:47.860 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.860 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:47.860 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432912+0000", 2023-11-25T04:24:47.860 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:47.860 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.860 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.860 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:47.861 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432912+0000", 2023-11-25T04:24:47.861 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:47.861 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.861 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.861 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:47.861 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432913+0000", 2023-11-25T04:24:47.861 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0699999999999998e-07 2023-11-25T04:24:47.861 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.862 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.862 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:47.862 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432915+0000", 2023-11-25T04:24:47.862 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7880000000000001e-06 2023-11-25T04:24:47.862 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.862 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.862 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:47.862 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432920+0000", 2023-11-25T04:24:47.863 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.7030000000000003e-06 2023-11-25T04:24:47.863 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.863 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.863 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:47.863 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432925+0000", 2023-11-25T04:24:47.863 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2889999999999999e-06 2023-11-25T04:24:47.863 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.863 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.863 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:47.864 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432968+0000", 2023-11-25T04:24:47.864 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3322999999999998e-05 2023-11-25T04:24:47.864 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.864 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.864 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:47.864 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.433069+0000", 2023-11-25T04:24:47.864 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000101005 2023-11-25T04:24:47.864 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.865 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.865 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:47.865 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.484789+0000", 2023-11-25T04:24:47.865 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051719832 2023-11-25T04:24:47.865 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:47.865 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:47.865 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:47.866 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.866 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.866 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:3704 4.21 4:870ac4b3:::1000000005e.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:47.866 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:44.432937+0000", 2023-11-25T04:24:47.866 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.3311276059999999, 2023-11-25T04:24:47.866 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.056731691000000001, 2023-11-25T04:24:47.866 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:47.866 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:47.867 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:47.867 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.867 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:47.867 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432937+0000", 2023-11-25T04:24:47.867 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:47.867 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.867 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.867 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:47.868 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432937+0000", 2023-11-25T04:24:47.868 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:47.868 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.868 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.868 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:47.868 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432937+0000", 2023-11-25T04:24:47.868 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1100000000000001e-07 2023-11-25T04:24:47.868 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.869 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.869 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:47.869 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432939+0000", 2023-11-25T04:24:47.869 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0480000000000001e-06 2023-11-25T04:24:47.869 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.869 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.869 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:47.870 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432947+0000", 2023-11-25T04:24:47.870 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6369999999999999e-06 2023-11-25T04:24:47.870 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.870 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.870 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:47.870 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432953+0000", 2023-11-25T04:24:47.870 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8459999999999996e-06 2023-11-25T04:24:47.871 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.871 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.871 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:47.871 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.433831+0000", 2023-11-25T04:24:47.871 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00087796300000000005 2023-11-25T04:24:47.871 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.871 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.871 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:47.871 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.433935+0000", 2023-11-25T04:24:47.872 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010438900000000001 2023-11-25T04:24:47.872 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.872 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.872 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:47.872 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.489669+0000", 2023-11-25T04:24:47.872 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055733396999999997 2023-11-25T04:24:47.872 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:47.872 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:47.873 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:47.873 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.873 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.873 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:3721 4.21 4:862b00fc:::1000000006f.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:47.873 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:44.434158+0000", 2023-11-25T04:24:47.873 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.3299068080000001, 2023-11-25T04:24:47.873 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055629563999999999, 2023-11-25T04:24:47.874 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:47.874 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:47.874 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:47.874 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.874 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:47.874 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.434158+0000", 2023-11-25T04:24:47.874 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:47.875 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.875 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.875 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:47.875 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.434158+0000", 2023-11-25T04:24:47.875 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:47.875 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.875 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.875 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:47.875 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.434158+0000", 2023-11-25T04:24:47.876 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3099999999999997e-07 2023-11-25T04:24:47.876 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.876 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.876 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:47.876 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.434162+0000", 2023-11-25T04:24:47.876 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2880000000000001e-06 2023-11-25T04:24:47.876 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.876 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.876 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:47.877 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.434174+0000", 2023-11-25T04:24:47.877 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1997000000000001e-05 2023-11-25T04:24:47.877 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.877 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.877 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:47.877 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.434180+0000", 2023-11-25T04:24:47.877 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7970000000000003e-06 2023-11-25T04:24:47.878 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.878 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.878 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:47.878 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.434546+0000", 2023-11-25T04:24:47.878 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00036599100000000003 2023-11-25T04:24:47.878 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.878 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.879 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:47.879 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.434647+0000", 2023-11-25T04:24:47.879 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000100472 2023-11-25T04:24:47.879 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.879 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.879 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:47.879 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.489787+0000", 2023-11-25T04:24:47.879 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055140387999999999 2023-11-25T04:24:47.880 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:47.880 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:47.880 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:47.880 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.880 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.880 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:3727 4.4 4:2045e544:::10000000075.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:47.880 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:44.434211+0000", 2023-11-25T04:24:47.880 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.3298530560000001, 2023-11-25T04:24:47.881 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.052705747999999997, 2023-11-25T04:24:47.881 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:47.881 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:47.881 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:47.881 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.881 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:47.881 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.434211+0000", 2023-11-25T04:24:47.881 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:47.883 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.883 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.883 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:47.884 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.434211+0000", 2023-11-25T04:24:47.884 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:47.884 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.884 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.884 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:47.884 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.434212+0000", 2023-11-25T04:24:47.884 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.27e-07 2023-11-25T04:24:47.884 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.885 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.885 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:47.885 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.434214+0000", 2023-11-25T04:24:47.885 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.854e-06 2023-11-25T04:24:47.885 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.885 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.885 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:47.885 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.434220+0000", 2023-11-25T04:24:47.886 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6760000000000001e-06 2023-11-25T04:24:47.886 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.886 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.886 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:47.886 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.434232+0000", 2023-11-25T04:24:47.886 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1642e-05 2023-11-25T04:24:47.886 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.886 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.887 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:47.887 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.434357+0000", 2023-11-25T04:24:47.887 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012530000000000001 2023-11-25T04:24:47.887 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.887 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.887 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:47.887 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.434461+0000", 2023-11-25T04:24:47.887 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000103775 2023-11-25T04:24:47.888 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.888 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.888 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:47.888 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.486917+0000", 2023-11-25T04:24:47.888 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.052456174000000001 2023-11-25T04:24:47.888 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:47.888 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:47.888 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:47.889 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.889 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.889 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:3742 4.4 4:21b748bd:::10000000084.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:47.889 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:44.434396+0000", 2023-11-25T04:24:47.889 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.3296683100000002, 2023-11-25T04:24:47.889 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053409643999999999, 2023-11-25T04:24:47.889 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:47.889 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:47.889 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:47.890 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.890 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:47.890 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.434396+0000", 2023-11-25T04:24:47.890 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:47.890 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.890 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.890 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:47.890 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.434396+0000", 2023-11-25T04:24:47.891 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:47.891 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.891 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.891 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:47.891 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.434397+0000", 2023-11-25T04:24:47.891 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2399999999999999e-07 2023-11-25T04:24:47.891 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.891 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.892 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:47.892 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.434398+0000", 2023-11-25T04:24:47.892 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.798e-06 2023-11-25T04:24:47.892 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.892 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.892 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:47.892 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.434405+0000", 2023-11-25T04:24:47.892 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4160000000000001e-06 2023-11-25T04:24:47.893 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.893 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.893 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:47.893 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.434409+0000", 2023-11-25T04:24:47.893 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2699999999999998e-06 2023-11-25T04:24:47.893 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.893 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.893 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:47.893 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.434774+0000", 2023-11-25T04:24:47.894 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00036455899999999998 2023-11-25T04:24:47.894 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.894 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.894 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:47.894 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.434857+0000", 2023-11-25T04:24:47.894 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3244999999999993e-05 2023-11-25T04:24:47.894 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.894 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.895 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:47.895 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.487806+0000", 2023-11-25T04:24:47.895 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.052948931999999997 2023-11-25T04:24:47.895 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:47.895 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:47.895 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:47.895 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.895 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.896 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:3743 4.10 4:0a074d4e:::10000000085.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:47.896 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:44.434415+0000", 2023-11-25T04:24:47.896 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.3296498400000001, 2023-11-25T04:24:47.896 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.052541022, 2023-11-25T04:24:47.896 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:47.896 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:47.896 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:47.896 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.897 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:47.897 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.434415+0000", 2023-11-25T04:24:47.897 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:47.897 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.897 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.897 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:47.897 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.434415+0000", 2023-11-25T04:24:47.898 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:47.898 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.898 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.898 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:47.898 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.434415+0000", 2023-11-25T04:24:47.898 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2399999999999999e-07 2023-11-25T04:24:47.898 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.898 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.899 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:47.899 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.434417+0000", 2023-11-25T04:24:47.899 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.716e-06 2023-11-25T04:24:47.899 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.899 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.899 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:47.899 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.434423+0000", 2023-11-25T04:24:47.900 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4309999999999999e-06 2023-11-25T04:24:47.900 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.900 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.900 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:47.900 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.434434+0000", 2023-11-25T04:24:47.900 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0447e-05 2023-11-25T04:24:47.900 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.900 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.901 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:47.901 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.462024+0000", 2023-11-25T04:24:47.901 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.027590248000000001 2023-11-25T04:24:47.901 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.901 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.901 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:47.901 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.462143+0000", 2023-11-25T04:24:47.902 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011916100000000001 2023-11-25T04:24:47.902 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.902 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.902 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:47.902 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.486956+0000", 2023-11-25T04:24:47.902 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.024812694999999999 2023-11-25T04:24:47.902 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:47.903 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:47.903 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:47.903 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.903 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.903 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:3771 4.4 4:23d81b1d:::100000000a1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:47.903 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:44.436259+0000", 2023-11-25T04:24:47.903 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.3278059209999999, 2023-11-25T04:24:47.903 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051988298000000002, 2023-11-25T04:24:47.904 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:47.904 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:47.904 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:47.904 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.904 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:47.904 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.436259+0000", 2023-11-25T04:24:47.904 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:47.905 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.905 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.905 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:47.905 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.436259+0000", 2023-11-25T04:24:47.905 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:47.905 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.905 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.906 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:47.906 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.436259+0000", 2023-11-25T04:24:47.906 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8200000000000001e-07 2023-11-25T04:24:47.906 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.906 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.906 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:47.906 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.436261+0000", 2023-11-25T04:24:47.906 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2050000000000001e-06 2023-11-25T04:24:47.907 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.907 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.907 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:47.908 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.436267+0000", 2023-11-25T04:24:47.908 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1879999999999999e-06 2023-11-25T04:24:47.908 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.909 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.909 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:47.909 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.436272+0000", 2023-11-25T04:24:47.909 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4120000000000003e-06 2023-11-25T04:24:47.910 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.910 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.910 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:47.910 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.463132+0000", 2023-11-25T04:24:47.910 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.026860520999999998 2023-11-25T04:24:47.910 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.910 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.910 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:47.911 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.463238+0000", 2023-11-25T04:24:47.911 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000105766 2023-11-25T04:24:47.911 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.911 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.911 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:47.911 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.488247+0000", 2023-11-25T04:24:47.911 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.025008823999999999 2023-11-25T04:24:47.911 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:47.912 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:47.912 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:47.912 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.912 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.912 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:3792 4.21 4:869f8614:::100000000b6.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:47.912 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:44.436622+0000", 2023-11-25T04:24:47.912 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.3274428110000001, 2023-11-25T04:24:47.913 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053301386999999999, 2023-11-25T04:24:47.913 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:47.913 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:47.913 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:47.913 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.913 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:47.913 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.436622+0000", 2023-11-25T04:24:47.913 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:47.914 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.914 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.914 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:47.914 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.436622+0000", 2023-11-25T04:24:47.914 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:47.914 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.914 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.915 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:47.915 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.436622+0000", 2023-11-25T04:24:47.915 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9500000000000002e-07 2023-11-25T04:24:47.915 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.915 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.915 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:47.915 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.436625+0000", 2023-11-25T04:24:47.915 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1949999999999999e-06 2023-11-25T04:24:47.916 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.916 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.916 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:47.916 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.436633+0000", 2023-11-25T04:24:47.916 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5950000000000001e-06 2023-11-25T04:24:47.916 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.916 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.916 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:47.917 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.436639+0000", 2023-11-25T04:24:47.917 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2169999999999999e-06 2023-11-25T04:24:47.917 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.917 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.917 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:47.917 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.462010+0000", 2023-11-25T04:24:47.917 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.025371123999999998 2023-11-25T04:24:47.917 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.918 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.918 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:47.918 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.462132+0000", 2023-11-25T04:24:47.918 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000121477 2023-11-25T04:24:47.918 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.918 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.918 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:47.919 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.489923+0000", 2023-11-25T04:24:47.919 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.027791184 2023-11-25T04:24:47.919 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:47.919 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:47.919 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:47.919 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.919 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.920 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:3797 4.4 4:21fe71c4:::100000000bb.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:47.920 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:44.436674+0000", 2023-11-25T04:24:47.920 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.3273909129999999, 2023-11-25T04:24:47.920 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051685999000000003, 2023-11-25T04:24:47.920 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:47.920 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:47.920 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:47.921 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.921 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:47.921 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.436674+0000", 2023-11-25T04:24:47.921 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:47.921 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.921 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.921 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:47.921 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.436674+0000", 2023-11-25T04:24:47.922 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:47.922 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.922 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.922 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:47.923 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.436676+0000", 2023-11-25T04:24:47.923 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1789999999999998e-06 2023-11-25T04:24:47.923 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.923 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.923 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:47.923 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.436678+0000", 2023-11-25T04:24:47.924 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1509999999999998e-06 2023-11-25T04:24:47.924 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.924 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.924 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:47.925 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.436683+0000", 2023-11-25T04:24:47.926 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.809e-06 2023-11-25T04:24:47.926 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.926 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.926 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:47.927 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.436691+0000", 2023-11-25T04:24:47.927 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0520000000000002e-06 2023-11-25T04:24:47.927 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.928 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.928 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:47.928 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.465065+0000", 2023-11-25T04:24:47.928 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.028374391999999998 2023-11-25T04:24:47.928 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.928 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.929 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:47.929 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.465170+0000", 2023-11-25T04:24:47.929 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000104676 2023-11-25T04:24:47.929 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:47.929 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:47.929 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:47.929 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.488360+0000", 2023-11-25T04:24:47.930 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.02318974 2023-11-25T04:24:47.930 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:47.930 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:47.930 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:47.930 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:47.930 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:47.930 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:47.932 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:47.930+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:47.932 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:24:48.034 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:48.031+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:48.046 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:48.047 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:24:48.047 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2023-11-25T04:24:48.047 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:48.056 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:24:48.140 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:48.139+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:48.202 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:48.202 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:24:48.202 INFO:teuthology.orchestra.run.smithi092.stdout: "complaint_time": 30, 2023-11-25T04:24:48.202 INFO:teuthology.orchestra.run.smithi092.stdout: "num_blocked_ops": 0 2023-11-25T04:24:48.203 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:48.210 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:24:48.234 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:48.233+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:48.335 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:48.334+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:48.357 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:48.358 INFO:teuthology.orchestra.run.smithi092.stdout: "size": 20, 2023-11-25T04:24:48.358 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 600, 2023-11-25T04:24:48.359 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [ 2023-11-25T04:24:48.359 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.359 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:3691 4.19 4:9ae7aaf4:::10000000051.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:48.360 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:44.433304+0000", 2023-11-25T04:24:48.360 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.92153375, 2023-11-25T04:24:48.360 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056495450000000003, 2023-11-25T04:24:48.361 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:48.361 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:48.361 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:48.362 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.362 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:48.362 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.433304+0000", 2023-11-25T04:24:48.363 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:48.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.363 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.364 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:48.364 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.433304+0000", 2023-11-25T04:24:48.364 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:48.364 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.365 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.365 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:48.365 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.433305+0000", 2023-11-25T04:24:48.366 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.8099999999999998e-07 2023-11-25T04:24:48.366 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.367 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:48.367 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.433306+0000", 2023-11-25T04:24:48.368 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.578e-06 2023-11-25T04:24:48.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.369 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:48.370 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.433310+0000", 2023-11-25T04:24:48.370 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.771e-06 2023-11-25T04:24:48.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.371 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:48.372 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.433313+0000", 2023-11-25T04:24:48.372 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.864e-06 2023-11-25T04:24:48.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.374 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:48.374 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.433350+0000", 2023-11-25T04:24:48.374 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7398999999999999e-05 2023-11-25T04:24:48.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.376 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:48.376 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.433454+0000", 2023-11-25T04:24:48.377 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103837 2023-11-25T04:24:48.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.378 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:48.379 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.489800+0000", 2023-11-25T04:24:48.379 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056345619999999999 2023-11-25T04:24:48.380 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:48.380 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:48.381 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:48.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.382 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:3696 4.28 4:166e0f51:::10000000056.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:48.383 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:44.433361+0000", 2023-11-25T04:24:48.383 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.921476771, 2023-11-25T04:24:48.384 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055970330999999998, 2023-11-25T04:24:48.384 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:48.384 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:48.385 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:48.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.385 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:48.385 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.433361+0000", 2023-11-25T04:24:48.386 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:48.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.386 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:48.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.433361+0000", 2023-11-25T04:24:48.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:48.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.387 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:48.387 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.433362+0000", 2023-11-25T04:24:48.387 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6199999999999998e-07 2023-11-25T04:24:48.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.388 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:48.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.433364+0000", 2023-11-25T04:24:48.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.323e-06 2023-11-25T04:24:48.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.388 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:48.388 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.433371+0000", 2023-11-25T04:24:48.388 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5899999999999996e-06 2023-11-25T04:24:48.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:48.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.433376+0000", 2023-11-25T04:24:48.389 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.9269999999999996e-06 2023-11-25T04:24:48.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.389 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:48.389 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.433418+0000", 2023-11-25T04:24:48.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2867000000000001e-05 2023-11-25T04:24:48.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.390 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:48.390 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.433522+0000", 2023-11-25T04:24:48.390 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000103421 2023-11-25T04:24:48.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.391 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:48.391 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.489331+0000", 2023-11-25T04:24:48.391 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055809641 2023-11-25T04:24:48.391 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:48.391 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:48.391 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:48.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.392 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:3731 4.19 4:99ec5763:::10000000079.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:48.392 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:44.434107+0000", 2023-11-25T04:24:48.392 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.9207311009999999, 2023-11-25T04:24:48.392 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055803469000000001, 2023-11-25T04:24:48.392 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:48.392 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:48.392 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:48.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:48.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.434107+0000", 2023-11-25T04:24:48.393 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:48.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.393 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:48.393 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.434107+0000", 2023-11-25T04:24:48.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:48.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.394 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:48.394 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.434107+0000", 2023-11-25T04:24:48.394 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.9900000000000001e-07 2023-11-25T04:24:48.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.395 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:48.395 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.434109+0000", 2023-11-25T04:24:48.395 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.553e-06 2023-11-25T04:24:48.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:48.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.434114+0000", 2023-11-25T04:24:48.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7320000000000003e-06 2023-11-25T04:24:48.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.396 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:48.396 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.434119+0000", 2023-11-25T04:24:48.396 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1080000000000003e-06 2023-11-25T04:24:48.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:48.397 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.434370+0000", 2023-11-25T04:24:48.397 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00025150999999999998 2023-11-25T04:24:48.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.397 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:48.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.434475+0000", 2023-11-25T04:24:48.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000105016 2023-11-25T04:24:48.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.398 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:48.398 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.489910+0000", 2023-11-25T04:24:48.398 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055435151000000002 2023-11-25T04:24:48.398 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:48.398 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:48.399 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:48.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.399 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.399 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:3901 4.36 4:6f395059:::10000000123.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:48.399 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:44.440446+0000", 2023-11-25T04:24:48.399 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.9143914199999998, 2023-11-25T04:24:48.399 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.062374564, 2023-11-25T04:24:48.399 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:48.399 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:48.400 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:48.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:48.400 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.440446+0000", 2023-11-25T04:24:48.400 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:48.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.400 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.400 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:48.401 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.440446+0000", 2023-11-25T04:24:48.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:48.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.401 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.401 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:48.401 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.440447+0000", 2023-11-25T04:24:48.401 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7599999999999997e-07 2023-11-25T04:24:48.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:48.402 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.440449+0000", 2023-11-25T04:24:48.402 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.9609999999999999e-06 2023-11-25T04:24:48.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.402 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.402 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:48.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.440455+0000", 2023-11-25T04:24:48.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.392e-06 2023-11-25T04:24:48.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.403 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:48.403 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.440460+0000", 2023-11-25T04:24:48.403 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.916e-06 2023-11-25T04:24:48.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.403 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:48.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.444413+0000", 2023-11-25T04:24:48.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0039524959999999998 2023-11-25T04:24:48.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.404 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.404 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:48.404 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.444512+0000", 2023-11-25T04:24:48.404 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.8876999999999995e-05 2023-11-25T04:24:48.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.405 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.405 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:48.405 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.502821+0000", 2023-11-25T04:24:48.405 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.058309446000000001 2023-11-25T04:24:48.405 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:48.405 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:48.406 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:48.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.406 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.406 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:3922 4.6 4:629cc6bc:::10000000138.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:48.406 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:44.441824+0000", 2023-11-25T04:24:48.406 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.9130143319999999, 2023-11-25T04:24:48.406 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.061109164000000001, 2023-11-25T04:24:48.407 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:48.407 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:48.407 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:48.407 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.407 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:48.407 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.441824+0000", 2023-11-25T04:24:48.407 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:48.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:48.408 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.441824+0000", 2023-11-25T04:24:48.408 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:48.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.408 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.408 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:48.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.441826+0000", 2023-11-25T04:24:48.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.599e-06 2023-11-25T04:24:48.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.409 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:48.409 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.441829+0000", 2023-11-25T04:24:48.409 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.055e-06 2023-11-25T04:24:48.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:48.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.441835+0000", 2023-11-25T04:24:48.410 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.276e-06 2023-11-25T04:24:48.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.410 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:48.410 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.441840+0000", 2023-11-25T04:24:48.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.9599999999999997e-06 2023-11-25T04:24:48.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.411 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:48.411 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.445570+0000", 2023-11-25T04:24:48.411 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.003729951 2023-11-25T04:24:48.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.412 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:48.412 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.445680+0000", 2023-11-25T04:24:48.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000109807 2023-11-25T04:24:48.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.412 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:48.412 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.502933+0000", 2023-11-25T04:24:48.412 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057252516000000003 2023-11-25T04:24:48.413 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:48.413 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:48.413 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:48.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.413 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:3926 4.36 4:6c5ddb65:::1000000013c.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:48.413 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:44.443003+0000", 2023-11-25T04:24:48.413 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.911834663, 2023-11-25T04:24:48.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.059937957, 2023-11-25T04:24:48.414 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:48.414 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:48.414 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:48.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.414 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:48.414 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.443003+0000", 2023-11-25T04:24:48.414 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:48.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:48.415 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.443003+0000", 2023-11-25T04:24:48.415 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:48.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.415 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:48.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.443004+0000", 2023-11-25T04:24:48.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6100000000000001e-07 2023-11-25T04:24:48.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.416 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:48.416 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.443012+0000", 2023-11-25T04:24:48.416 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 8.0090000000000007e-06 2023-11-25T04:24:48.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:48.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.443019+0000", 2023-11-25T04:24:48.417 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.0500000000000003e-06 2023-11-25T04:24:48.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.417 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:48.417 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.443025+0000", 2023-11-25T04:24:48.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.1800000000000001e-06 2023-11-25T04:24:48.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.418 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:48.418 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.445746+0000", 2023-11-25T04:24:48.418 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0027212640000000001 2023-11-25T04:24:48.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:48.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.445841+0000", 2023-11-25T04:24:48.419 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.4488999999999996e-05 2023-11-25T04:24:48.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.419 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:48.419 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.502941+0000", 2023-11-25T04:24:48.420 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.057100404 2023-11-25T04:24:48.420 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:48.420 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:48.420 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:48.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.420 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:3933 4.2e 4:77f9b43c:::10000000143.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:48.420 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:44.443095+0000", 2023-11-25T04:24:48.421 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.9117433840000002, 2023-11-25T04:24:48.421 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.059948544999999999, 2023-11-25T04:24:48.421 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:48.421 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:48.421 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:48.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.421 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:48.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.443095+0000", 2023-11-25T04:24:48.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:48.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.422 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:48.422 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.443095+0000", 2023-11-25T04:24:48.422 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:48.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:48.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.443095+0000", 2023-11-25T04:24:48.423 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5499999999999998e-07 2023-11-25T04:24:48.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.423 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:48.423 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.443097+0000", 2023-11-25T04:24:48.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7549999999999999e-06 2023-11-25T04:24:48.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.424 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:48.424 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.443103+0000", 2023-11-25T04:24:48.424 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.6449999999999999e-06 2023-11-25T04:24:48.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:48.425 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.443108+0000", 2023-11-25T04:24:48.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.6829999999999999e-06 2023-11-25T04:24:48.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.425 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:48.425 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.445970+0000", 2023-11-25T04:24:48.425 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.002862 2023-11-25T04:24:48.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:48.426 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446688+0000", 2023-11-25T04:24:48.426 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00071754200000000005 2023-11-25T04:24:48.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.426 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:48.427 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.503043+0000", 2023-11-25T04:24:48.427 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056355465 2023-11-25T04:24:48.427 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:48.427 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:48.427 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:48.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.427 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.427 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:3994 4.6 4:63fec8d1:::10000000180.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:48.428 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:44.444709+0000", 2023-11-25T04:24:48.428 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.9101289179999998, 2023-11-25T04:24:48.428 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.058338905000000003, 2023-11-25T04:24:48.428 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:48.428 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:48.428 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:48.428 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.428 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:48.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.444709+0000", 2023-11-25T04:24:48.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:48.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.429 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.429 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:48.429 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.444709+0000", 2023-11-25T04:24:48.429 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:48.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.430 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:48.430 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.444709+0000", 2023-11-25T04:24:48.430 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.7e-07 2023-11-25T04:24:48.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.430 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.430 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:48.430 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.444711+0000", 2023-11-25T04:24:48.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.635e-06 2023-11-25T04:24:48.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.431 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:48.431 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.444724+0000", 2023-11-25T04:24:48.431 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.2753e-05 2023-11-25T04:24:48.431 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.431 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:48.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.444734+0000", 2023-11-25T04:24:48.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.007e-05 2023-11-25T04:24:48.432 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.432 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.432 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:48.432 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447792+0000", 2023-11-25T04:24:48.432 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0030580630000000002 2023-11-25T04:24:48.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:48.433 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447909+0000", 2023-11-25T04:24:48.433 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00011692 2023-11-25T04:24:48.433 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.433 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.433 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:48.434 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.503048+0000", 2023-11-25T04:24:48.434 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055139094 2023-11-25T04:24:48.434 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:48.434 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:48.434 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:48.434 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.434 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.434 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:3997 4.6 4:60954073:::10000000183.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:48.435 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:44.444741+0000", 2023-11-25T04:24:48.435 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.9100967029999998, 2023-11-25T04:24:48.435 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.058419077, 2023-11-25T04:24:48.435 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:48.435 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:48.435 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:48.435 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.435 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:48.437 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.444741+0000", 2023-11-25T04:24:48.437 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:48.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.438 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:48.438 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.444741+0000", 2023-11-25T04:24:48.438 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:48.438 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.438 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.438 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:48.439 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.444742+0000", 2023-11-25T04:24:48.439 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.1100000000000001e-07 2023-11-25T04:24:48.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.439 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.439 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:48.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.444744+0000", 2023-11-25T04:24:48.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.013e-06 2023-11-25T04:24:48.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.440 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.440 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:48.440 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.444750+0000", 2023-11-25T04:24:48.440 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.2299999999999996e-06 2023-11-25T04:24:48.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:48.441 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.444757+0000", 2023-11-25T04:24:48.441 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.9870000000000002e-06 2023-11-25T04:24:48.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.441 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.441 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:48.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.448162+0000", 2023-11-25T04:24:48.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0034051020000000001 2023-11-25T04:24:48.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.442 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:48.442 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.448494+0000", 2023-11-25T04:24:48.442 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00033231800000000001 2023-11-25T04:24:48.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.442 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.443 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:48.443 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.503160+0000", 2023-11-25T04:24:48.443 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054666015999999998 2023-11-25T04:24:48.443 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:48.443 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:48.444 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:48.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.444 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.444 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:4032 4.2e 4:775b9fb2:::100000001a6.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:48.444 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:44.446660+0000", 2023-11-25T04:24:48.444 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.9081782490000001, 2023-11-25T04:24:48.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056503945, 2023-11-25T04:24:48.445 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:48.445 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:48.445 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:48.445 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.445 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:48.445 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446660+0000", 2023-11-25T04:24:48.445 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:48.445 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:48.446 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446660+0000", 2023-11-25T04:24:48.446 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:48.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.446 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.446 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:48.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446660+0000", 2023-11-25T04:24:48.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5499999999999998e-07 2023-11-25T04:24:48.447 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.447 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.447 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:48.447 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446662+0000", 2023-11-25T04:24:48.447 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.798e-06 2023-11-25T04:24:48.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.448 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.448 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:48.448 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446667+0000", 2023-11-25T04:24:48.448 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.2349999999999999e-06 2023-11-25T04:24:48.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.449 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:48.449 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446670+0000", 2023-11-25T04:24:48.449 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0699999999999998e-06 2023-11-25T04:24:48.449 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.449 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.449 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:48.449 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.455057+0000", 2023-11-25T04:24:48.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0083872570000000004 2023-11-25T04:24:48.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.450 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:48.450 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.455340+0000", 2023-11-25T04:24:48.450 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00028249500000000001 2023-11-25T04:24:48.450 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.450 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.450 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:48.451 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.503164+0000", 2023-11-25T04:24:48.451 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.047823635000000003 2023-11-25T04:24:48.451 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:48.451 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:48.451 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:48.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.451 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.451 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:4037 4.2e 4:76865fdc:::100000001ab.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:48.452 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:44.446688+0000", 2023-11-25T04:24:48.452 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.9081500359999999, 2023-11-25T04:24:48.452 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056587089, 2023-11-25T04:24:48.452 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:48.452 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:48.452 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:48.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.452 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:48.452 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446688+0000", 2023-11-25T04:24:48.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:48.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.453 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:48.453 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446688+0000", 2023-11-25T04:24:48.453 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:48.453 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.453 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.454 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:48.454 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446688+0000", 2023-11-25T04:24:48.454 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.2399999999999999e-07 2023-11-25T04:24:48.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.454 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.454 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:48.454 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446690+0000", 2023-11-25T04:24:48.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.911e-06 2023-11-25T04:24:48.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.455 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:48.455 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446696+0000", 2023-11-25T04:24:48.455 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.4859999999999997e-06 2023-11-25T04:24:48.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.455 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:48.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446700+0000", 2023-11-25T04:24:48.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7099999999999998e-06 2023-11-25T04:24:48.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.456 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.456 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:48.456 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.455754+0000", 2023-11-25T04:24:48.456 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0090534350000000003 2023-11-25T04:24:48.456 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:48.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.455881+0000", 2023-11-25T04:24:48.457 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000126818 2023-11-25T04:24:48.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.457 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.457 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:48.457 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.503275+0000", 2023-11-25T04:24:48.458 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.047394304999999998 2023-11-25T04:24:48.458 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:48.458 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:48.458 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:48.458 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.458 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.458 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:4044 4.2e 4:76ff63fc:::100000001b2.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:48.458 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:44.446860+0000", 2023-11-25T04:24:48.459 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.9079781909999998, 2023-11-25T04:24:48.459 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056538421999999998, 2023-11-25T04:24:48.459 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:48.459 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:48.459 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:48.459 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.459 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:48.459 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446860+0000", 2023-11-25T04:24:48.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:48.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.460 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:48.460 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446860+0000", 2023-11-25T04:24:48.460 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:48.460 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.460 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:48.461 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446860+0000", 2023-11-25T04:24:48.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8599999999999998e-07 2023-11-25T04:24:48.461 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.461 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.461 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:48.461 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446862+0000", 2023-11-25T04:24:48.461 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.0650000000000001e-06 2023-11-25T04:24:48.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.462 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:48.462 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446869+0000", 2023-11-25T04:24:48.462 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.1709999999999996e-06 2023-11-25T04:24:48.462 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.462 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.462 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:48.463 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446875+0000", 2023-11-25T04:24:48.463 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3870000000000003e-06 2023-11-25T04:24:48.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.463 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.463 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:48.463 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.456185+0000", 2023-11-25T04:24:48.463 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0093104260000000001 2023-11-25T04:24:48.463 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.464 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:48.464 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.456285+0000", 2023-11-25T04:24:48.464 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.9393000000000002e-05 2023-11-25T04:24:48.464 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.464 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.464 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:48.464 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.503398+0000", 2023-11-25T04:24:48.464 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.047113493999999999 2023-11-25T04:24:48.465 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:48.465 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:48.465 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:48.465 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.465 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.465 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:4049 4.19 4:99a7d964:::100000001b7.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:48.465 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:44.446931+0000", 2023-11-25T04:24:48.465 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.907906922, 2023-11-25T04:24:48.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056039185999999998, 2023-11-25T04:24:48.466 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:48.466 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:48.466 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:48.466 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.466 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:48.466 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446931+0000", 2023-11-25T04:24:48.466 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:48.467 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.467 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.467 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:48.467 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446931+0000", 2023-11-25T04:24:48.467 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:48.467 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.467 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.467 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:48.468 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446931+0000", 2023-11-25T04:24:48.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.84e-07 2023-11-25T04:24:48.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.468 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:48.468 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446939+0000", 2023-11-25T04:24:48.468 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2010000000000002e-06 2023-11-25T04:24:48.468 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.468 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.469 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:48.469 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446945+0000", 2023-11-25T04:24:48.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.5429999999999999e-06 2023-11-25T04:24:48.469 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.469 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.469 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:48.469 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446950+0000", 2023-11-25T04:24:48.469 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.0560000000000004e-06 2023-11-25T04:24:48.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.470 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:48.470 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446999+0000", 2023-11-25T04:24:48.470 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.8594000000000003e-05 2023-11-25T04:24:48.470 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.470 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.470 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:48.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447095+0000", 2023-11-25T04:24:48.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 9.6638999999999994e-05 2023-11-25T04:24:48.471 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.471 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.471 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:48.471 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.502970+0000", 2023-11-25T04:24:48.471 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055874768999999998 2023-11-25T04:24:48.471 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:48.471 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:48.472 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:48.472 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.472 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.472 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:4052 4.36 4:6d15073c:::100000001ba.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:48.472 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:44.446994+0000", 2023-11-25T04:24:48.472 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.9078436540000001, 2023-11-25T04:24:48.472 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056265880999999997, 2023-11-25T04:24:48.472 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:48.473 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:48.473 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:48.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.473 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:48.473 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446994+0000", 2023-11-25T04:24:48.473 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:48.473 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.473 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.473 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:48.474 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446994+0000", 2023-11-25T04:24:48.474 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:48.474 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.474 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.474 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:48.474 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446995+0000", 2023-11-25T04:24:48.474 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.6300000000000001e-07 2023-11-25T04:24:48.474 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.475 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.475 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:48.475 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.446996+0000", 2023-11-25T04:24:48.475 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.612e-06 2023-11-25T04:24:48.475 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.475 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.475 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:48.475 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447003+0000", 2023-11-25T04:24:48.476 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 6.3069999999999999e-06 2023-11-25T04:24:48.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.476 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:48.476 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447007+0000", 2023-11-25T04:24:48.476 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.3930000000000001e-06 2023-11-25T04:24:48.476 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.476 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.476 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:48.477 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.456139+0000", 2023-11-25T04:24:48.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0091321370000000002 2023-11-25T04:24:48.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.477 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.477 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:48.477 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.456249+0000", 2023-11-25T04:24:48.477 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000109491 2023-11-25T04:24:48.477 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.478 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.478 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:48.478 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.503260+0000", 2023-11-25T04:24:48.478 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.047011577999999998 2023-11-25T04:24:48.478 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:48.478 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:48.478 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:48.478 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.479 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.479 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:4054 4.19 4:9b54a4b5:::100000001bc.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:48.479 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:44.447013+0000", 2023-11-25T04:24:48.479 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.9078251900000001, 2023-11-25T04:24:48.479 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056061487, 2023-11-25T04:24:48.479 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:48.479 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:48.479 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:48.480 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.480 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:48.480 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447013+0000", 2023-11-25T04:24:48.480 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:48.480 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.480 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.480 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:48.480 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447013+0000", 2023-11-25T04:24:48.480 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:48.481 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.481 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:48.481 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447013+0000", 2023-11-25T04:24:48.481 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7700000000000005e-07 2023-11-25T04:24:48.481 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.481 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.481 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:48.482 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447015+0000", 2023-11-25T04:24:48.482 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.5939999999999999e-06 2023-11-25T04:24:48.482 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.482 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.482 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:48.482 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447020+0000", 2023-11-25T04:24:48.482 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6030000000000004e-06 2023-11-25T04:24:48.482 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.483 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.483 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:48.483 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447026+0000", 2023-11-25T04:24:48.483 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1420000000000001e-06 2023-11-25T04:24:48.483 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.483 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.483 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:48.483 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447490+0000", 2023-11-25T04:24:48.484 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00046441800000000002 2023-11-25T04:24:48.484 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.484 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.484 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:48.484 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447568+0000", 2023-11-25T04:24:48.484 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.7751999999999997e-05 2023-11-25T04:24:48.484 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.484 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.484 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:48.485 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.503074+0000", 2023-11-25T04:24:48.485 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055506501 2023-11-25T04:24:48.485 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:48.485 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:48.485 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:48.485 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.485 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.485 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:4057 4.19 4:9ab6d71b:::100000001bf.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:48.486 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:44.447032+0000", 2023-11-25T04:24:48.486 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.9078061210000001, 2023-11-25T04:24:48.486 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.056145928999999997, 2023-11-25T04:24:48.486 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:48.486 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:48.486 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:48.486 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.486 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:48.487 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447032+0000", 2023-11-25T04:24:48.487 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:48.487 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.487 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.487 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:48.487 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447032+0000", 2023-11-25T04:24:48.487 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:48.487 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.487 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.488 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:48.488 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447032+0000", 2023-11-25T04:24:48.488 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.3500000000000002e-07 2023-11-25T04:24:48.488 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.488 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.488 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:48.488 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447034+0000", 2023-11-25T04:24:48.488 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.832e-06 2023-11-25T04:24:48.489 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.489 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.489 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:48.489 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447040+0000", 2023-11-25T04:24:48.489 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.6779999999999997e-06 2023-11-25T04:24:48.489 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.489 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.489 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:48.490 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447044+0000", 2023-11-25T04:24:48.490 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.391e-06 2023-11-25T04:24:48.490 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.490 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.490 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:48.490 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447807+0000", 2023-11-25T04:24:48.490 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00076306799999999999 2023-11-25T04:24:48.490 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.491 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.491 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:48.491 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447931+0000", 2023-11-25T04:24:48.491 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012439999999999999 2023-11-25T04:24:48.491 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.491 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.491 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:48.491 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.503178+0000", 2023-11-25T04:24:48.492 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.055246225000000003 2023-11-25T04:24:48.492 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:48.492 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:48.492 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:48.492 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.492 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.492 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:4058 4.19 4:98a8a8b3:::100000001c0.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:48.492 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:44.447050+0000", 2023-11-25T04:24:48.492 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.9077878200000002, 2023-11-25T04:24:48.493 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.05633759, 2023-11-25T04:24:48.493 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:48.493 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:48.493 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:48.493 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.493 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:48.493 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447050+0000", 2023-11-25T04:24:48.493 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:48.494 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.494 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.494 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:48.494 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447050+0000", 2023-11-25T04:24:48.494 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:48.494 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.494 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.494 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:48.495 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447051+0000", 2023-11-25T04:24:48.495 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.1900000000000003e-07 2023-11-25T04:24:48.495 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.495 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.495 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:48.495 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447053+0000", 2023-11-25T04:24:48.495 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 2.108e-06 2023-11-25T04:24:48.495 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.496 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.496 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:48.496 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447058+0000", 2023-11-25T04:24:48.496 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3159999999999996e-06 2023-11-25T04:24:48.496 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.496 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.496 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:48.496 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.447063+0000", 2023-11-25T04:24:48.497 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.5449999999999997e-06 2023-11-25T04:24:48.497 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.497 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.497 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:48.497 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.448196+0000", 2023-11-25T04:24:48.497 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.001133128 2023-11-25T04:24:48.497 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.497 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.498 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:48.498 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.448895+0000", 2023-11-25T04:24:48.498 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00069910700000000003 2023-11-25T04:24:48.498 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.498 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.498 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:48.498 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.503388+0000", 2023-11-25T04:24:48.498 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054492867 2023-11-25T04:24:48.499 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:48.499 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:48.499 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:48.499 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.499 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.499 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:4077 4.3e 4:7c32b2c0:::100000001d3.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:48.499 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:44.448674+0000", 2023-11-25T04:24:48.499 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.9061634769999998, 2023-11-25T04:24:48.500 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054710506999999999, 2023-11-25T04:24:48.500 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:48.500 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:48.500 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:48.500 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.500 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:48.500 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.448674+0000", 2023-11-25T04:24:48.500 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:48.500 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.501 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.501 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:48.501 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.448674+0000", 2023-11-25T04:24:48.501 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:48.501 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.501 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.501 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:48.501 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.448675+0000", 2023-11-25T04:24:48.502 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.0199999999999998e-07 2023-11-25T04:24:48.502 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.502 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.502 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:48.502 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.448676+0000", 2023-11-25T04:24:48.502 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.646e-06 2023-11-25T04:24:48.502 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.502 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.503 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:48.503 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.448682+0000", 2023-11-25T04:24:48.503 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.981e-06 2023-11-25T04:24:48.503 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.503 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.503 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:48.503 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.448687+0000", 2023-11-25T04:24:48.504 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.853e-06 2023-11-25T04:24:48.504 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.504 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.504 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:48.504 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.456676+0000", 2023-11-25T04:24:48.504 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0079888660000000007 2023-11-25T04:24:48.504 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.504 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.504 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:48.505 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.456799+0000", 2023-11-25T04:24:48.505 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00012312799999999999 2023-11-25T04:24:48.505 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.505 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.505 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:48.505 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.503385+0000", 2023-11-25T04:24:48.505 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046585730999999998 2023-11-25T04:24:48.505 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:48.506 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:48.506 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:48.506 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.506 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.506 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:4083 4.6 4:61ce4130:::100000001d9.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:48.506 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:44.448770+0000", 2023-11-25T04:24:48.506 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.9060683819999999, 2023-11-25T04:24:48.506 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054722232000000003, 2023-11-25T04:24:48.507 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:48.507 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:48.507 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:48.507 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.507 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:48.507 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.448770+0000", 2023-11-25T04:24:48.507 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:48.507 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.507 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.508 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:48.508 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.448770+0000", 2023-11-25T04:24:48.508 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:48.508 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.508 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.508 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:48.508 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.448770+0000", 2023-11-25T04:24:48.508 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5499999999999999e-07 2023-11-25T04:24:48.509 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.509 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.509 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:48.509 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.448772+0000", 2023-11-25T04:24:48.509 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.72e-06 2023-11-25T04:24:48.509 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.509 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.509 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:48.510 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.448779+0000", 2023-11-25T04:24:48.510 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 7.2139999999999999e-06 2023-11-25T04:24:48.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.510 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.510 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:48.510 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.448784+0000", 2023-11-25T04:24:48.510 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 4.7299999999999996e-06 2023-11-25T04:24:48.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.510 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.511 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:48.511 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.456849+0000", 2023-11-25T04:24:48.511 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0080652759999999997 2023-11-25T04:24:48.511 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.511 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.511 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:48.511 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.456950+0000", 2023-11-25T04:24:48.511 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.000101551 2023-11-25T04:24:48.512 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.512 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.512 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:48.512 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.503492+0000", 2023-11-25T04:24:48.512 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046541385999999997 2023-11-25T04:24:48.512 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:48.512 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:48.512 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:48.513 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.513 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.513 INFO:teuthology.orchestra.run.smithi092.stdout: "description": "osd_op(mds.0.3:4088 4.2e 4:77eaaa70:::100000001de.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:48.513 INFO:teuthology.orchestra.run.smithi092.stdout: "initiated_at": "2023-11-25T04:24:44.448828+0000", 2023-11-25T04:24:48.513 INFO:teuthology.orchestra.run.smithi092.stdout: "age": 3.9060095119999998, 2023-11-25T04:24:48.513 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.054684280000000002, 2023-11-25T04:24:48.513 INFO:teuthology.orchestra.run.smithi092.stdout: "type_data": { 2023-11-25T04:24:48.513 INFO:teuthology.orchestra.run.smithi092.stdout: "flag_point": "started", 2023-11-25T04:24:48.513 INFO:teuthology.orchestra.run.smithi092.stdout: "events": [ 2023-11-25T04:24:48.514 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.514 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "initiated", 2023-11-25T04:24:48.514 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.448828+0000", 2023-11-25T04:24:48.514 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:48.514 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.514 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.514 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "header_read", 2023-11-25T04:24:48.514 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.448828+0000", 2023-11-25T04:24:48.515 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0 2023-11-25T04:24:48.515 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.515 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.515 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "throttled", 2023-11-25T04:24:48.515 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.448829+0000", 2023-11-25T04:24:48.515 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 3.5999999999999999e-07 2023-11-25T04:24:48.515 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.515 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.516 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "all_read", 2023-11-25T04:24:48.516 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.448831+0000", 2023-11-25T04:24:48.516 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 1.7400000000000001e-06 2023-11-25T04:24:48.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.516 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.516 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "dispatched", 2023-11-25T04:24:48.516 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.448836+0000", 2023-11-25T04:24:48.516 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.7309999999999998e-06 2023-11-25T04:24:48.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.517 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.517 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "queued_for_pg", 2023-11-25T04:24:48.517 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.448842+0000", 2023-11-25T04:24:48.517 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 5.3979999999999998e-06 2023-11-25T04:24:48.517 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.517 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.517 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "reached_pg", 2023-11-25T04:24:48.517 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.457248+0000", 2023-11-25T04:24:48.517 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.0084061819999999999 2023-11-25T04:24:48.518 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.518 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.518 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "started", 2023-11-25T04:24:48.518 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.457351+0000", 2023-11-25T04:24:48.518 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.00010357399999999999 2023-11-25T04:24:48.518 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-11-25T04:24:48.518 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-11-25T04:24:48.518 INFO:teuthology.orchestra.run.smithi092.stdout: "event": "done", 2023-11-25T04:24:48.519 INFO:teuthology.orchestra.run.smithi092.stdout: "time": "2023-11-25T04:24:44.503513+0000", 2023-11-25T04:24:48.519 INFO:teuthology.orchestra.run.smithi092.stdout: "duration": 0.046161294999999998 2023-11-25T04:24:48.519 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:48.519 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:48.519 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:48.519 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-11-25T04:24:48.519 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-11-25T04:24:48.519 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:48.521 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:24:48.521 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:48.519+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:48.621 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:48.620+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:48.633 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:48.634 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:24:48.634 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2023-11-25T04:24:48.634 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:48.644 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:24:48.721 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~100s 2023-11-25T04:24:48.727 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:48.726+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:48.790 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:48.790 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2023-11-25T04:24:48.790 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2023-11-25T04:24:48.790 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2023-11-25T04:24:48.790 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:48.798 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:24:48.823 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:48.822+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:48.924 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:48.923+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:48.945 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2023-11-25T04:24:48.945 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2023-11-25T04:24:48.945 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2023-11-25T04:24:48.945 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2023-11-25T04:24:48.945 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.946 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:3624 4.37 4:eefe613f:::1000000000e.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:48.946 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:44.430400+0000", 2023-11-25T04:24:48.946 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.5127263659999999, 2023-11-25T04:24:48.946 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.052686746, 2023-11-25T04:24:48.946 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:48.946 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:48.946 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:48.947 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.947 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:48.947 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.430400+0000", 2023-11-25T04:24:48.947 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:48.947 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:48.947 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.947 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:48.947 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.430400+0000", 2023-11-25T04:24:48.948 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:48.948 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:48.948 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.948 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:48.948 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.430400+0000", 2023-11-25T04:24:48.948 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6800000000000001e-07 2023-11-25T04:24:48.948 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:48.948 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.948 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:48.949 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.430403+0000", 2023-11-25T04:24:48.949 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7700000000000002e-06 2023-11-25T04:24:48.949 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:48.949 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.949 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:48.949 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.430414+0000", 2023-11-25T04:24:48.949 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0798000000000001e-05 2023-11-25T04:24:48.949 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:48.950 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.950 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:48.950 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.430419+0000", 2023-11-25T04:24:48.950 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8679999999999998e-06 2023-11-25T04:24:48.950 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:48.950 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.950 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:48.950 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.430455+0000", 2023-11-25T04:24:48.951 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6764999999999998e-05 2023-11-25T04:24:48.951 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:48.951 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.951 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:48.951 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.430568+0000", 2023-11-25T04:24:48.951 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000113008 2023-11-25T04:24:48.951 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:48.951 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.952 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:48.952 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.483087+0000", 2023-11-25T04:24:48.952 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.052518169000000003 2023-11-25T04:24:48.952 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:48.952 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:48.952 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:48.952 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:48.952 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.953 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:3625 4.12 4:49e35fbc:::1000000000f.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:48.954 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:44.430429+0000", 2023-11-25T04:24:48.954 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.5126976699999997, 2023-11-25T04:24:48.954 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053415323000000001, 2023-11-25T04:24:48.954 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:48.955 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:48.956 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:48.956 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.957 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:48.957 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.430429+0000", 2023-11-25T04:24:48.957 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:48.957 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:48.957 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.957 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:48.957 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.430429+0000", 2023-11-25T04:24:48.958 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:48.958 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:48.958 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.958 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:48.958 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.430429+0000", 2023-11-25T04:24:48.958 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0999999999999999e-07 2023-11-25T04:24:48.958 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:48.958 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.959 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:48.959 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.430431+0000", 2023-11-25T04:24:48.959 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3779999999999999e-06 2023-11-25T04:24:48.959 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:48.959 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.959 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:48.960 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.430438+0000", 2023-11-25T04:24:48.960 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3199999999999996e-06 2023-11-25T04:24:48.960 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:48.960 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.960 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:48.961 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.430449+0000", 2023-11-25T04:24:48.961 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0834e-05 2023-11-25T04:24:48.961 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:48.961 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.961 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:48.961 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.430514+0000", 2023-11-25T04:24:48.961 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5364999999999995e-05 2023-11-25T04:24:48.961 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:48.962 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.962 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:48.962 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.430621+0000", 2023-11-25T04:24:48.962 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000107416 2023-11-25T04:24:48.963 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:48.963 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.963 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:48.963 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.483844+0000", 2023-11-25T04:24:48.963 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053222499999999999 2023-11-25T04:24:48.963 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:48.963 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:48.963 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:48.964 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:48.964 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.964 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:3630 4.10 4:0b7da9c3:::10000000014.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:48.964 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:44.430468+0000", 2023-11-25T04:24:48.964 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.5126582879999999, 2023-11-25T04:24:48.964 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053832480000000002, 2023-11-25T04:24:48.964 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:48.964 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:48.965 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:48.965 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.965 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:48.965 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.430468+0000", 2023-11-25T04:24:48.965 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:48.965 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:48.965 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.965 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:48.966 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.430468+0000", 2023-11-25T04:24:48.966 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:48.966 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:48.966 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.966 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:48.966 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.430468+0000", 2023-11-25T04:24:48.966 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0800000000000005e-07 2023-11-25T04:24:48.966 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:48.967 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.967 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:48.967 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.430471+0000", 2023-11-25T04:24:48.967 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5610000000000001e-06 2023-11-25T04:24:48.967 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:48.967 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.967 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:48.967 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.430492+0000", 2023-11-25T04:24:48.968 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1087e-05 2023-11-25T04:24:48.968 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:48.968 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.968 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:48.968 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.430498+0000", 2023-11-25T04:24:48.968 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9429999999999997e-06 2023-11-25T04:24:48.968 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:48.968 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.968 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:48.969 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.430550+0000", 2023-11-25T04:24:48.969 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1785e-05 2023-11-25T04:24:48.969 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:48.969 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.969 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:48.969 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.430666+0000", 2023-11-25T04:24:48.969 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000115926 2023-11-25T04:24:48.970 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:48.970 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.970 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:48.970 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.484300+0000", 2023-11-25T04:24:48.970 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053634670000000002 2023-11-25T04:24:48.970 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:48.970 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:48.970 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:48.971 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:48.971 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.971 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:3635 4.21 4:86d8200e:::10000000019.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:48.971 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:44.430901+0000", 2023-11-25T04:24:48.971 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.5122248589999998, 2023-11-25T04:24:48.971 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.057400916000000003, 2023-11-25T04:24:48.971 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:48.971 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:48.971 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:48.972 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.972 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:48.972 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.430901+0000", 2023-11-25T04:24:48.972 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:48.972 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:48.972 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.972 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:48.972 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.430901+0000", 2023-11-25T04:24:48.973 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:48.973 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:48.973 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.973 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:48.973 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.430902+0000", 2023-11-25T04:24:48.973 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.92e-07 2023-11-25T04:24:48.973 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:48.973 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.974 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:48.974 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.430906+0000", 2023-11-25T04:24:48.974 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5999999999999998e-06 2023-11-25T04:24:48.974 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:48.974 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.974 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:48.974 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.430914+0000", 2023-11-25T04:24:48.975 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9370000000000004e-06 2023-11-25T04:24:48.975 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:48.975 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.975 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:48.975 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.430920+0000", 2023-11-25T04:24:48.975 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3799999999999999e-06 2023-11-25T04:24:48.975 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:48.975 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.975 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:48.976 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.430971+0000", 2023-11-25T04:24:48.976 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0878999999999999e-05 2023-11-25T04:24:48.976 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:48.976 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.976 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:48.976 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.431092+0000", 2023-11-25T04:24:48.977 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000120951 2023-11-25T04:24:48.977 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:48.977 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.977 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:48.977 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.488302+0000", 2023-11-25T04:24:48.977 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.057210477000000003 2023-11-25T04:24:48.977 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:48.978 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:48.978 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:48.978 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:48.978 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.978 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:3646 4.2f 4:f6ddf183:::10000000024.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:48.978 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:44.431468+0000", 2023-11-25T04:24:48.978 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.5116581089999999, 2023-11-25T04:24:48.978 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051625977000000003, 2023-11-25T04:24:48.979 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:48.979 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:48.979 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:48.979 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.979 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:48.979 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.431468+0000", 2023-11-25T04:24:48.979 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:48.979 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:48.980 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.980 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:48.980 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.431468+0000", 2023-11-25T04:24:48.980 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:48.980 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:48.980 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.980 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:48.980 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.431469+0000", 2023-11-25T04:24:48.981 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2399999999999998e-07 2023-11-25T04:24:48.981 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:48.981 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.981 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:48.981 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.431472+0000", 2023-11-25T04:24:48.981 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4570000000000001e-06 2023-11-25T04:24:48.981 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:48.981 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.981 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:48.982 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.431482+0000", 2023-11-25T04:24:48.982 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3070000000000008e-06 2023-11-25T04:24:48.982 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:48.982 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.982 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:48.982 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.431488+0000", 2023-11-25T04:24:48.983 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4239999999999998e-06 2023-11-25T04:24:48.983 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:48.983 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.983 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:48.984 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.431563+0000", 2023-11-25T04:24:48.984 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5179000000000006e-05 2023-11-25T04:24:48.984 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:48.984 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.984 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:48.984 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.431670+0000", 2023-11-25T04:24:48.984 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010675699999999999 2023-11-25T04:24:48.984 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:48.985 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.985 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:48.985 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.483094+0000", 2023-11-25T04:24:48.985 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051424229000000002 2023-11-25T04:24:48.985 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:48.985 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:48.985 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:48.985 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:48.986 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.986 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:3656 4.12 4:49ffa098:::1000000002e.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:48.986 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:44.431630+0000", 2023-11-25T04:24:48.986 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.5114962079999996, 2023-11-25T04:24:48.986 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.052311218, 2023-11-25T04:24:48.986 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:48.986 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:48.986 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:48.987 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.987 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:48.987 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.431630+0000", 2023-11-25T04:24:48.987 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:48.987 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:48.987 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.987 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:48.987 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.431630+0000", 2023-11-25T04:24:48.988 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:48.988 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:48.988 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.988 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:48.988 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.431631+0000", 2023-11-25T04:24:48.988 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9500000000000002e-07 2023-11-25T04:24:48.988 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:48.989 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.989 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:48.989 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.431633+0000", 2023-11-25T04:24:48.989 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9319999999999999e-06 2023-11-25T04:24:48.989 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:48.989 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.989 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:48.989 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.431638+0000", 2023-11-25T04:24:48.990 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4809999999999997e-06 2023-11-25T04:24:48.990 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:48.990 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.990 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:48.990 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.431643+0000", 2023-11-25T04:24:48.990 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4389999999999999e-06 2023-11-25T04:24:48.990 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:48.990 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.990 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:48.991 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.431695+0000", 2023-11-25T04:24:48.991 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1119999999999998e-05 2023-11-25T04:24:48.991 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:48.991 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.991 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:48.991 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.431795+0000", 2023-11-25T04:24:48.991 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000100696 2023-11-25T04:24:48.991 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:48.991 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.992 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:48.992 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.483941+0000", 2023-11-25T04:24:48.992 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.052145955000000001 2023-11-25T04:24:48.992 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:48.992 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:48.992 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:48.992 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:48.992 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.993 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:3657 4.23 4:c550f404:::1000000002f.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:48.993 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:44.432139+0000", 2023-11-25T04:24:48.993 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.5109876929999997, 2023-11-25T04:24:48.993 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051595467999999998, 2023-11-25T04:24:48.993 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:48.993 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:48.994 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:48.994 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.994 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:48.994 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432139+0000", 2023-11-25T04:24:48.994 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:48.994 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:48.994 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.994 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:48.995 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432139+0000", 2023-11-25T04:24:48.995 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:48.995 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:48.995 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.995 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:48.995 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432139+0000", 2023-11-25T04:24:48.995 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0200000000000002e-07 2023-11-25T04:24:48.995 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:48.995 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.996 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:48.996 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432143+0000", 2023-11-25T04:24:48.996 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3200000000000001e-06 2023-11-25T04:24:48.996 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:48.996 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.996 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:48.996 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432151+0000", 2023-11-25T04:24:48.996 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2219999999999996e-06 2023-11-25T04:24:48.997 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:48.997 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.997 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:48.997 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432162+0000", 2023-11-25T04:24:48.997 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1571e-05 2023-11-25T04:24:48.997 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:48.997 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.997 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:48.998 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432209+0000", 2023-11-25T04:24:48.998 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.6984000000000003e-05 2023-11-25T04:24:48.998 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:48.998 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.998 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:48.998 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432334+0000", 2023-11-25T04:24:48.998 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000124762 2023-11-25T04:24:48.998 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:48.999 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:48.999 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:48.999 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.483734+0000", 2023-11-25T04:24:48.999 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051400006999999998 2023-11-25T04:24:48.999 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:48.999 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:48.999 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:48.999 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.000 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.000 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:3674 4.23 4:c6a0a389:::10000000040.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:49.000 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:44.432419+0000", 2023-11-25T04:24:49.000 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.5107073470000003, 2023-11-25T04:24:49.000 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051408338999999997, 2023-11-25T04:24:49.000 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:49.000 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:49.000 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:49.000 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.001 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:49.001 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432419+0000", 2023-11-25T04:24:49.001 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:49.001 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.001 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.001 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:49.001 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432419+0000", 2023-11-25T04:24:49.001 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:49.002 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.002 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.002 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:49.002 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432419+0000", 2023-11-25T04:24:49.002 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3399999999999999e-07 2023-11-25T04:24:49.002 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.002 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.002 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:49.003 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432421+0000", 2023-11-25T04:24:49.003 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9580000000000001e-06 2023-11-25T04:24:49.003 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.003 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.003 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:49.003 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432427+0000", 2023-11-25T04:24:49.004 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.7710000000000003e-06 2023-11-25T04:24:49.004 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.004 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.004 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:49.004 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432432+0000", 2023-11-25T04:24:49.004 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4680000000000003e-06 2023-11-25T04:24:49.004 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.004 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.005 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:49.005 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.433768+0000", 2023-11-25T04:24:49.005 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001336093 2023-11-25T04:24:49.005 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.005 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.005 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:49.005 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.433872+0000", 2023-11-25T04:24:49.005 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000104609 2023-11-25T04:24:49.005 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.006 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.006 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:49.006 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.483827+0000", 2023-11-25T04:24:49.006 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.049954906 2023-11-25T04:24:49.006 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:49.006 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:49.006 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:49.006 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.007 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.007 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:3682 4.12 4:4899516f:::10000000048.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:49.007 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:44.432556+0000", 2023-11-25T04:24:49.007 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.5105702699999997, 2023-11-25T04:24:49.007 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051716377000000001, 2023-11-25T04:24:49.007 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:49.007 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:49.007 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:49.007 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.008 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:49.008 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432556+0000", 2023-11-25T04:24:49.008 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:49.008 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.008 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.008 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:49.008 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432556+0000", 2023-11-25T04:24:49.008 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:49.009 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.009 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.009 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:49.009 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432556+0000", 2023-11-25T04:24:49.009 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.63e-07 2023-11-25T04:24:49.009 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.009 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.009 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:49.009 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432559+0000", 2023-11-25T04:24:49.010 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3269999999999999e-06 2023-11-25T04:24:49.010 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.010 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.010 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:49.010 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432566+0000", 2023-11-25T04:24:49.010 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9779999999999999e-06 2023-11-25T04:24:49.010 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.010 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.010 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:49.011 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432572+0000", 2023-11-25T04:24:49.011 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.161e-06 2023-11-25T04:24:49.011 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.011 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.011 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:49.011 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.433149+0000", 2023-11-25T04:24:49.011 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00057659900000000001 2023-11-25T04:24:49.011 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.012 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.012 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:49.012 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.433519+0000", 2023-11-25T04:24:49.012 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00037059899999999999 2023-11-25T04:24:49.012 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.012 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.012 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:49.012 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.484272+0000", 2023-11-25T04:24:49.012 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.05075325 2023-11-25T04:24:49.013 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:49.013 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:49.013 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:49.013 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.013 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.013 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:3686 4.21 4:844eae9e:::1000000004c.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:49.013 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:44.432639+0000", 2023-11-25T04:24:49.013 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.510486921, 2023-11-25T04:24:49.014 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.056557505000000001, 2023-11-25T04:24:49.014 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:49.014 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:49.014 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:49.014 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.014 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:49.014 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432639+0000", 2023-11-25T04:24:49.015 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:49.015 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.015 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.015 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:49.015 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432639+0000", 2023-11-25T04:24:49.015 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:49.015 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.015 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.016 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:49.016 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432640+0000", 2023-11-25T04:24:49.016 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3900000000000005e-07 2023-11-25T04:24:49.016 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.016 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.016 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:49.016 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432642+0000", 2023-11-25T04:24:49.016 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1900000000000002e-06 2023-11-25T04:24:49.016 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.017 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.017 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:49.017 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432648+0000", 2023-11-25T04:24:49.017 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0619999999999997e-06 2023-11-25T04:24:49.017 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.017 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.017 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:49.018 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432653+0000", 2023-11-25T04:24:49.018 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5410000000000002e-06 2023-11-25T04:24:49.018 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.018 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.018 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:49.018 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432699+0000", 2023-11-25T04:24:49.018 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.6681000000000002e-05 2023-11-25T04:24:49.018 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.019 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.019 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:49.019 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432799+0000", 2023-11-25T04:24:49.019 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9264000000000004e-05 2023-11-25T04:24:49.019 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.019 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.019 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:49.019 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.489197+0000", 2023-11-25T04:24:49.019 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.056398228000000002 2023-11-25T04:24:49.020 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:49.020 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:49.020 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:49.020 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.020 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.020 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:3689 4.21 4:8660ae9f:::1000000004f.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:49.020 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:44.432696+0000", 2023-11-25T04:24:49.020 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.5104299189999999, 2023-11-25T04:24:49.020 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.05686683, 2023-11-25T04:24:49.021 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:49.021 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:49.021 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:49.021 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.021 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:49.021 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432696+0000", 2023-11-25T04:24:49.021 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:49.021 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.022 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.022 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:49.022 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432696+0000", 2023-11-25T04:24:49.022 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:49.022 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.022 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.022 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:49.022 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432697+0000", 2023-11-25T04:24:49.022 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5499999999999999e-07 2023-11-25T04:24:49.023 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.023 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.023 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:49.023 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432699+0000", 2023-11-25T04:24:49.023 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0030000000000001e-06 2023-11-25T04:24:49.023 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.023 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.023 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:49.024 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432712+0000", 2023-11-25T04:24:49.024 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3803e-05 2023-11-25T04:24:49.024 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.024 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.024 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:49.024 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432717+0000", 2023-11-25T04:24:49.024 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7369999999999997e-06 2023-11-25T04:24:49.024 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.025 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.025 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:49.025 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.433424+0000", 2023-11-25T04:24:49.025 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00070664700000000003 2023-11-25T04:24:49.025 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.025 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.025 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:49.025 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.433561+0000", 2023-11-25T04:24:49.025 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013756400000000001 2023-11-25T04:24:49.026 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.026 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.026 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:49.026 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.489563+0000", 2023-11-25T04:24:49.026 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.056001720999999997 2023-11-25T04:24:49.026 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:49.026 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:49.026 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:49.027 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.027 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.027 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:3703 4.2d 4:b5236558:::1000000005d.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:49.027 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:44.432912+0000", 2023-11-25T04:24:49.027 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.5102138719999996, 2023-11-25T04:24:49.027 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051876347000000003, 2023-11-25T04:24:49.027 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:49.027 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:49.027 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:49.028 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.028 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:49.028 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432912+0000", 2023-11-25T04:24:49.028 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:49.028 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.028 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.028 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:49.028 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432912+0000", 2023-11-25T04:24:49.029 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:49.029 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.029 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.029 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:49.029 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432913+0000", 2023-11-25T04:24:49.029 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0699999999999998e-07 2023-11-25T04:24:49.029 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.029 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.029 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:49.030 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432915+0000", 2023-11-25T04:24:49.030 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7880000000000001e-06 2023-11-25T04:24:49.030 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.030 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.030 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:49.030 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432920+0000", 2023-11-25T04:24:49.030 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.7030000000000003e-06 2023-11-25T04:24:49.030 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.031 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.031 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:49.031 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432925+0000", 2023-11-25T04:24:49.031 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2889999999999999e-06 2023-11-25T04:24:49.031 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.031 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.031 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:49.031 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432968+0000", 2023-11-25T04:24:49.031 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3322999999999998e-05 2023-11-25T04:24:49.032 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.032 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.032 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:49.032 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.433069+0000", 2023-11-25T04:24:49.032 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000101005 2023-11-25T04:24:49.032 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.032 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.032 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:49.032 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.484789+0000", 2023-11-25T04:24:49.033 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051719832 2023-11-25T04:24:49.033 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:49.033 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:49.033 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:49.033 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.033 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.033 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:3704 4.21 4:870ac4b3:::1000000005e.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:49.033 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:44.432937+0000", 2023-11-25T04:24:49.034 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.5101893679999998, 2023-11-25T04:24:49.034 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.056731691000000001, 2023-11-25T04:24:49.034 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:49.034 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:49.034 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:49.034 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.034 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:49.035 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432937+0000", 2023-11-25T04:24:49.035 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:49.035 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.035 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.035 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:49.035 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432937+0000", 2023-11-25T04:24:49.035 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:49.035 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.035 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.036 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:49.036 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432937+0000", 2023-11-25T04:24:49.036 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1100000000000001e-07 2023-11-25T04:24:49.036 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.036 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.036 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:49.036 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432939+0000", 2023-11-25T04:24:49.037 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0480000000000001e-06 2023-11-25T04:24:49.037 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.037 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.037 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:49.037 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432947+0000", 2023-11-25T04:24:49.037 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6369999999999999e-06 2023-11-25T04:24:49.037 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.037 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.038 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:49.038 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.432953+0000", 2023-11-25T04:24:49.038 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8459999999999996e-06 2023-11-25T04:24:49.038 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.038 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.038 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:49.038 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.433831+0000", 2023-11-25T04:24:49.038 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00087796300000000005 2023-11-25T04:24:49.039 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.039 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.039 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:49.039 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.433935+0000", 2023-11-25T04:24:49.039 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010438900000000001 2023-11-25T04:24:49.039 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.039 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.039 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:49.039 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.489669+0000", 2023-11-25T04:24:49.040 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055733396999999997 2023-11-25T04:24:49.040 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:49.040 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:49.040 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:49.040 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.040 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.041 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:3721 4.21 4:862b00fc:::1000000006f.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:49.041 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:44.434158+0000", 2023-11-25T04:24:49.041 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.5089685700000004, 2023-11-25T04:24:49.041 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055629563999999999, 2023-11-25T04:24:49.041 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:49.041 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:49.041 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:49.041 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.042 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:49.042 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.434158+0000", 2023-11-25T04:24:49.042 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:49.042 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.042 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.042 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:49.042 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.434158+0000", 2023-11-25T04:24:49.042 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:49.043 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.043 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.043 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:49.043 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.434158+0000", 2023-11-25T04:24:49.043 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3099999999999997e-07 2023-11-25T04:24:49.043 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.043 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.043 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:49.044 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.434162+0000", 2023-11-25T04:24:49.044 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2880000000000001e-06 2023-11-25T04:24:49.044 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.044 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.044 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:49.044 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.434174+0000", 2023-11-25T04:24:49.044 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1997000000000001e-05 2023-11-25T04:24:49.044 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.045 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.045 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:49.045 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.434180+0000", 2023-11-25T04:24:49.045 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7970000000000003e-06 2023-11-25T04:24:49.045 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.045 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.045 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:49.045 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.434546+0000", 2023-11-25T04:24:49.045 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00036599100000000003 2023-11-25T04:24:49.046 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.046 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.046 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:49.046 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.434647+0000", 2023-11-25T04:24:49.046 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000100472 2023-11-25T04:24:49.046 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.046 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.047 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:49.047 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.489787+0000", 2023-11-25T04:24:49.047 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.055140387999999999 2023-11-25T04:24:49.047 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:49.047 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:49.047 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:49.048 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.048 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.048 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:3727 4.4 4:2045e544:::10000000075.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:49.048 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:44.434211+0000", 2023-11-25T04:24:49.048 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.508914818, 2023-11-25T04:24:49.048 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.052705747999999997, 2023-11-25T04:24:49.048 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:49.048 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:49.048 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:49.049 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.049 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:49.049 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.434211+0000", 2023-11-25T04:24:49.049 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:49.049 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.049 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.049 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:49.049 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.434211+0000", 2023-11-25T04:24:49.050 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:49.050 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.050 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.050 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:49.050 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.434212+0000", 2023-11-25T04:24:49.050 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.27e-07 2023-11-25T04:24:49.050 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.050 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.051 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:49.051 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.434214+0000", 2023-11-25T04:24:49.051 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.854e-06 2023-11-25T04:24:49.051 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.051 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.051 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:49.051 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.434220+0000", 2023-11-25T04:24:49.051 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6760000000000001e-06 2023-11-25T04:24:49.052 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.052 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.052 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:49.052 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.434232+0000", 2023-11-25T04:24:49.052 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1642e-05 2023-11-25T04:24:49.052 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.052 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.052 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:49.053 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.434357+0000", 2023-11-25T04:24:49.053 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012530000000000001 2023-11-25T04:24:49.053 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.053 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.053 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:49.053 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.434461+0000", 2023-11-25T04:24:49.053 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000103775 2023-11-25T04:24:49.053 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.054 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.054 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:49.054 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.486917+0000", 2023-11-25T04:24:49.054 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.052456174000000001 2023-11-25T04:24:49.054 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:49.054 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:49.054 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:49.054 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.055 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.055 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:3742 4.4 4:21b748bd:::10000000084.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:49.055 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:44.434396+0000", 2023-11-25T04:24:49.055 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.5087300719999996, 2023-11-25T04:24:49.055 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053409643999999999, 2023-11-25T04:24:49.055 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:49.055 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:49.055 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:49.056 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.056 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:49.056 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.434396+0000", 2023-11-25T04:24:49.056 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:49.056 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.056 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.056 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:49.056 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.434396+0000", 2023-11-25T04:24:49.057 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:49.057 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.057 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.057 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:49.057 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.434397+0000", 2023-11-25T04:24:49.057 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2399999999999999e-07 2023-11-25T04:24:49.057 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.057 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.058 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:49.058 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.434398+0000", 2023-11-25T04:24:49.058 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.798e-06 2023-11-25T04:24:49.058 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.058 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.058 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:49.058 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.434405+0000", 2023-11-25T04:24:49.058 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4160000000000001e-06 2023-11-25T04:24:49.059 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.059 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.059 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:49.059 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.434409+0000", 2023-11-25T04:24:49.059 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2699999999999998e-06 2023-11-25T04:24:49.059 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.059 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.059 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:49.060 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.434774+0000", 2023-11-25T04:24:49.060 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00036455899999999998 2023-11-25T04:24:49.060 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.060 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.060 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:49.060 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.434857+0000", 2023-11-25T04:24:49.060 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3244999999999993e-05 2023-11-25T04:24:49.060 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.061 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.061 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:49.061 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.487806+0000", 2023-11-25T04:24:49.061 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.052948931999999997 2023-11-25T04:24:49.061 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:49.061 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:49.061 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:49.061 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.061 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.062 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:3743 4.10 4:0a074d4e:::10000000085.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:49.062 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:44.434415+0000", 2023-11-25T04:24:49.062 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.508711602, 2023-11-25T04:24:49.062 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.052541022, 2023-11-25T04:24:49.062 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:49.062 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:49.062 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:49.062 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.063 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:49.063 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.434415+0000", 2023-11-25T04:24:49.063 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:49.063 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.063 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.063 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:49.063 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.434415+0000", 2023-11-25T04:24:49.063 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:49.064 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.064 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.064 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:49.064 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.434415+0000", 2023-11-25T04:24:49.064 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2399999999999999e-07 2023-11-25T04:24:49.064 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.064 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.064 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:49.065 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.434417+0000", 2023-11-25T04:24:49.065 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.716e-06 2023-11-25T04:24:49.065 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.065 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.065 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:49.065 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.434423+0000", 2023-11-25T04:24:49.065 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4309999999999999e-06 2023-11-25T04:24:49.065 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.066 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.066 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:49.066 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.434434+0000", 2023-11-25T04:24:49.066 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0447e-05 2023-11-25T04:24:49.066 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.066 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.066 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:49.066 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.462024+0000", 2023-11-25T04:24:49.067 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.027590248000000001 2023-11-25T04:24:49.067 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.067 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.067 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:49.067 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.462143+0000", 2023-11-25T04:24:49.067 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011916100000000001 2023-11-25T04:24:49.067 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.067 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.068 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:49.068 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.486956+0000", 2023-11-25T04:24:49.068 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.024812694999999999 2023-11-25T04:24:49.068 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:49.068 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:49.068 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:49.068 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.068 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.069 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:3771 4.4 4:23d81b1d:::100000000a1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:49.069 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:44.436259+0000", 2023-11-25T04:24:49.069 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.5068676830000003, 2023-11-25T04:24:49.069 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051988298000000002, 2023-11-25T04:24:49.069 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:49.069 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:49.069 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:49.069 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.070 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:49.070 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.436259+0000", 2023-11-25T04:24:49.070 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:49.070 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.070 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.070 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:49.070 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.436259+0000", 2023-11-25T04:24:49.070 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:49.071 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.071 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.071 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:49.071 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.436259+0000", 2023-11-25T04:24:49.071 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8200000000000001e-07 2023-11-25T04:24:49.071 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.071 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.071 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:49.072 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.436261+0000", 2023-11-25T04:24:49.072 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2050000000000001e-06 2023-11-25T04:24:49.072 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.072 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.072 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:49.072 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.436267+0000", 2023-11-25T04:24:49.072 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1879999999999999e-06 2023-11-25T04:24:49.072 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.073 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.073 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:49.073 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.436272+0000", 2023-11-25T04:24:49.073 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4120000000000003e-06 2023-11-25T04:24:49.073 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.073 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.073 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:49.073 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.463132+0000", 2023-11-25T04:24:49.074 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.026860520999999998 2023-11-25T04:24:49.074 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.074 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.074 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:49.074 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.463238+0000", 2023-11-25T04:24:49.074 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000105766 2023-11-25T04:24:49.075 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.075 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.075 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:49.075 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.488247+0000", 2023-11-25T04:24:49.075 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.025008823999999999 2023-11-25T04:24:49.075 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:49.075 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:49.075 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:49.076 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.076 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.076 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:3792 4.21 4:869f8614:::100000000b6.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:49.076 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:44.436622+0000", 2023-11-25T04:24:49.076 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.506504573, 2023-11-25T04:24:49.076 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.053301386999999999, 2023-11-25T04:24:49.076 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:49.076 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:49.076 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:49.077 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.077 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:49.077 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.436622+0000", 2023-11-25T04:24:49.077 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:49.077 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.077 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.077 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:49.077 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.436622+0000", 2023-11-25T04:24:49.078 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:49.078 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.078 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.078 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:49.078 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.436622+0000", 2023-11-25T04:24:49.078 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9500000000000002e-07 2023-11-25T04:24:49.078 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.078 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.079 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:49.079 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.436625+0000", 2023-11-25T04:24:49.079 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1949999999999999e-06 2023-11-25T04:24:49.079 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.079 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.079 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:49.079 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.436633+0000", 2023-11-25T04:24:49.079 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5950000000000001e-06 2023-11-25T04:24:49.080 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.080 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.080 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:49.080 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.436639+0000", 2023-11-25T04:24:49.080 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2169999999999999e-06 2023-11-25T04:24:49.081 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.081 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.081 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:49.081 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.462010+0000", 2023-11-25T04:24:49.081 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.025371123999999998 2023-11-25T04:24:49.081 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.081 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.081 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:49.082 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.462132+0000", 2023-11-25T04:24:49.082 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000121477 2023-11-25T04:24:49.082 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.082 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.082 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:49.082 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.489923+0000", 2023-11-25T04:24:49.082 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.027791184 2023-11-25T04:24:49.082 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:49.083 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:49.083 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:49.083 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.083 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.083 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(mds.0.3:3797 4.4 4:21fe71c4:::100000000bb.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e54)", 2023-11-25T04:24:49.083 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2023-11-25T04:24:44.436674+0000", 2023-11-25T04:24:49.083 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.5064526750000002, 2023-11-25T04:24:49.083 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.051685999000000003, 2023-11-25T04:24:49.084 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2023-11-25T04:24:49.084 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2023-11-25T04:24:49.084 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2023-11-25T04:24:49.084 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.084 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2023-11-25T04:24:49.084 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.436674+0000", 2023-11-25T04:24:49.084 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:49.084 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.085 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.085 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2023-11-25T04:24:49.085 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.436674+0000", 2023-11-25T04:24:49.085 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2023-11-25T04:24:49.085 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.085 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.085 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2023-11-25T04:24:49.085 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.436676+0000", 2023-11-25T04:24:49.086 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1789999999999998e-06 2023-11-25T04:24:49.086 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.086 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.086 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2023-11-25T04:24:49.086 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.436678+0000", 2023-11-25T04:24:49.086 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1509999999999998e-06 2023-11-25T04:24:49.086 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.087 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.087 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2023-11-25T04:24:49.087 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.436683+0000", 2023-11-25T04:24:49.087 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.809e-06 2023-11-25T04:24:49.087 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.087 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.087 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2023-11-25T04:24:49.087 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.436691+0000", 2023-11-25T04:24:49.088 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0520000000000002e-06 2023-11-25T04:24:49.088 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.088 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.088 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2023-11-25T04:24:49.088 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.465065+0000", 2023-11-25T04:24:49.088 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.028374391999999998 2023-11-25T04:24:49.088 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.088 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.089 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2023-11-25T04:24:49.089 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.465170+0000", 2023-11-25T04:24:49.089 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000104676 2023-11-25T04:24:49.089 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2023-11-25T04:24:49.089 INFO:teuthology.orchestra.run.smithi195.stdout: { 2023-11-25T04:24:49.089 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2023-11-25T04:24:49.089 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2023-11-25T04:24:44.488360+0000", 2023-11-25T04:24:49.089 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.02318974 2023-11-25T04:24:49.090 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:49.090 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:49.090 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:49.090 INFO:teuthology.orchestra.run.smithi195.stdout: } 2023-11-25T04:24:49.090 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2023-11-25T04:24:49.090 INFO:teuthology.orchestra.run.smithi195.stdout:} 2023-11-25T04:24:49.092 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:24:49.092 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:49.090+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:49.174 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-11-25T04:24:49.193 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:49.191+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:49.202 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-11-25T04:24:49.202 INFO:teuthology.orchestra.run.smithi092.stdout: "ops": [], 2023-11-25T04:24:49.202 INFO:teuthology.orchestra.run.smithi092.stdout: "num_ops": 0 2023-11-25T04:24:49.202 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-11-25T04:24:49.210 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:24:49.294 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:49.293+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:49.298 INFO:teuthology.orchestra.run.smithi067.stderr:Error ENXIO: problem getting command descriptions from osd.0 2023-11-25T04:24:49.317 INFO:teuthology.orchestra.run.smithi067.stdout:osd.1: {} 2023-11-25T04:24:49.318 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:49.318 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:49.318 INFO:teuthology.orchestra.run.smithi067.stderr:osd.1: osd_enable_op_tracker = '' 2023-11-25T04:24:49.325 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:24:49.337 INFO:teuthology.orchestra.run.smithi067.stdout:osd.2: {} 2023-11-25T04:24:49.338 INFO:teuthology.orchestra.run.smithi067.stderr:osd.2: osd_enable_op_tracker = '' 2023-11-25T04:24:49.339 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-11-25T04:24:49.340 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-11-25T04:24:49.389 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2023-11-25T04:24:49.410 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:49.409+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:49.471 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:49.471 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:49.480 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:24:49.521 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:49.520+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:49.590 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:49.590 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:49.596 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:49.595+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:49.599 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:24:49.698 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:49.697+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:49.705 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:49.705 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:49.714 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:24:49.799 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:49.798+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:49.858 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:49.858 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:49.867 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:24:49.901 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:49.899+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:49.973 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:49.974 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:49.982 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:24:50.001 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:50.000+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:50.086 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:50.086 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:50.095 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:24:50.138 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:50.137+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:50.199 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:50.200 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:50.203 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:50.202+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:50.209 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:24:50.304 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:50.302+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:50.315 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:50.315 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:50.324 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:24:50.361 INFO:teuthology.orchestra.run.smithi067.stderr:nodeep-scrub is unset 2023-11-25T04:24:50.404 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:50.403+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:50.429 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:50.429 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:50.441 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:24:50.505 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-11-25T04:24:50.504+0000 7f0d7c5ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 82875) UID: 0 2023-11-25T04:24:50.516 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi067.front.sepia.ceph.com 2023-11-25T04:24:50.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-25T04:24:50.517 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-25T04:24:50.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-25T04:24:50.533 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list 2023-11-25T04:24:50.546 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:50.547 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:50.555 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-11-25T04:24:50.590 INFO:teuthology.orchestra.run.smithi067.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-25T04:24:50.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-25T04:24:50.591 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-25T04:24:50.606 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:50.605+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:50.660 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:50.661 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:50.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-25T04:24:50.663 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link delete ceph-brx 2023-11-25T04:24:50.669 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-11-25T04:24:50.707 INFO:tasks.ceph.osd.2.smithi195.stderr:2023-11-25T04:24:50.705+0000 7f3a7bf7c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 83175) UID: 0 2023-11-25T04:24:50.757 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2023-11-25T04:24:50.766 INFO:tasks.thrashosds:joining thrashosds 2023-11-25T04:24:50.773 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:50.773 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:50.781 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-11-25T04:24:50.885 INFO:teuthology.orchestra.run.smithi092.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:50.886 INFO:teuthology.orchestra.run.smithi092.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:50.894 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-11-25T04:24:51.002 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:51.002 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:51.010 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-11-25T04:24:51.114 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:51.115 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:51.122 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-11-25T04:24:51.227 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2023-11-25T04:24:51.227 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-11-25T04:24:52.376 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2023-11-25T04:24:53.370 INFO:teuthology.orchestra.run.smithi067.stderr:noscrub is unset 2023-11-25T04:24:53.385 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2023-11-25T04:24:54.372 INFO:teuthology.orchestra.run.smithi067.stderr:nodeep-scrub is unset 2023-11-25T04:24:54.528 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~106s 2023-11-25T04:25:00.337 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~111s 2023-11-25T04:25:06.145 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~117s 2023-11-25T04:25:11.949 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~123s 2023-11-25T04:25:17.757 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~129s 2023-11-25T04:25:23.565 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~135s 2023-11-25T04:25:29.374 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~140s 2023-11-25T04:25:35.181 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~146s 2023-11-25T04:25:37.022 DEBUG:teuthology.orchestra.console.smithi076:expect before: [Enter `^Ec?' for help] \x80 \x80 \x80 Initializing Intel(R) Boot Agent XE v2.1.60 PXE 2.1 Build 088 (WfM 2.0)  Press Ctrl+S to enter the Setup Menu.                      Press Ctrl+S to enter the Setup Menu..  \x80 \x80 Initializing Intel(R) Boot Agent XE v2.1.60 PXE 2.1 Build 088 (WfM 2.0)  Press Ctrl+S to enter the Setup Menu.                      Press Ctrl+S to enter the Setup Menu..  \x80 \x80 Initializing Intel(R) Boot Agent GE v1.5.72 PXE 2.1 Build 092 (WfM 2.0) Press Ctrl+S to enter the Setup Menu.                       Press Ctrl+S to enter the Setup Menu..  \x80 \x80  el(R) Boot Agent GE v1.5.72 PXE 2.1 Build 092 (WfM 2.0) Press Ctrl+S to enter the Setup Menu.                       Initializing Intel(R) Boot Agent GE v1.5.72 Press Ctrl+S to enter the Setup Menu..  \x80 \x80  Intel(R) Boot Agent XE v2.1.60 Copyright (C) 1997-2009, Intel Corporation  CLIENT MAC ADDR: 0C C4 7A 8F 47 E1 GUID: 00000000 0000 0000 0000 0CC47AD8DA66 DHCP./                    DHCP.| DHCP./ DHCP.- DHCP.\ DHCP.| DHCP./ CLIENT IP: 172.21.15.76 MASK: 255.255.240.0 DHCP IP: 172.21.0.10 GATEWAY IP: 172.21.15.254 PXE->EB: !PXE at 95D3:0070, entry point at 95D3:0106  UNDI code segment 95D3:3E70, data segment 8E94:73F0 (570-615kB)  UNDI device is PCI 03:00.1, type DIX+802.3  570kB free base memory after PXE unload iPXE initialising devices... iPXE initialising devices...ok iPXE 1.20.1+ (g4bd0) -- Open Source Network Boot Firmware -- http://ipxe.org Features: DNS FTP HTTP HTTPS iSCSI NFS TFTP VLAN AoE ELF MBOOT PXE bzImage Menu PXEXT Configuring (net0 0c:c4:7a:8f:47:e1)... Configuring (net0 0c:c4:7a:8f:47:e1).... Configuring (net0 0c:c4:7a:8f:47:e1)..... Configuring (net0 0c:c4:7a:8f:47:e1)...... ok .    [?25lUse the ^ and v keys to change the selection. Press 'e' to edit the selected item, or 'c' for a command prompt.   CentOS Stream (5.14.0-386.el9.x86_64) 9  CentOS Stream (5.14.0-361.el9.x86_64) 9  CentOS Stream (5.14.0-239.el9.x86_64) 9  CentOS Stream (0-rescue-15949e3c88704300882482eddfd7baae) 9                The selected entry will be started automatically in 5s.           Use the * and * keys to change the selection.  Press 'e' to edit the selected item, or 'c' for a command prompt.  The selected entry will be started automatically in 5s.   CentOS Stream (5.14.0-386.el9.x86_64) 9   CentOS Stream (5.14.0-361.el9.x86_64) 9  CentOS Stream (5.14.0-239.el9.x86_64) 9  CentOS Stream (0-rescue-15949e3c88704300882482eddfd7baae) 9  The selected entry will be started automatically in 4s.  The selected entry will be started automatically in 4s.  The selected entry will be started automatically in 3s.  The selected entry will be started automatically in 3s.  The selected entry will be started automatically in 2s.  The selected entry will be started automatically in 2s.  The selected entry will be started automatically in 1s.  The selected entry will be started automatically in 1s.  The selected entry will be started automatically in 0s. [?25h       [ 0.000000] Linux version 5.14.0-386.el9.x86_64 (mockbuild@x86-05.stream.rdu2.redhat.com) (gcc (GCC) 11.4.1 20230605 (Red Hat 11.4.1-2), GNU ld version 2.35.2-42.el9) #1 SMP PREEMPT_DYNAMIC Wed Nov 15 10:38:22 UTC 2023 [ 0.000000] The list of certified hardware and cloud instances for Red Hat Enterprise Linux 9 can be viewed at the Red Hat Ecosystem Catalog, https://catalog.redhat.com. [ 0.000000] Command line: BOOT_IMAGE=(hd0,msdos1)/boot/vmlinuz-5.14.0-386.el9.x86_64 root=UUID=0b261d40-7844-4e73-a652-6bb884dfb0a5 ro crashkernel=1G-4G:192M,4G-64G:256M,64G-:512M console=ttyS1,115200 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000008abff] usable [ 0.000000] BIOS-e820: [mem 0x00000000000e0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x0000000078ec5fff] usable [ 0.000000] BIOS-e820: [mem 0x0000000078f78000-0x00000000798a7fff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000798a8000-0x0000000079d44fff] ACPI NVS [ 0.000000] BIOS-e820: [mem 0x0000000079d45000-0x000000008fffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed44fff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000ff000000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000087fffffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 3.0 present. [ 0.000000] DMI: Supermicro SYS-5018R-WR/X10SRW-F, BIOS 2.0a 08/02/2016 [ 0.000000] tsc: Fast TSC calibration using PIT [ 0.000000] tsc: Detected 3499.911 MHz processor [ 0.000489] last_pfn = 0x880000 max_arch_pfn = 0x400000000 [ 0.000493] MTRR map: 5 entries (3 fixed + 2 variable; max 23), built from 10 variable MTRRs [ 0.000495] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000821] last_pfn = 0x78ec6 max_arch_pfn = 0x400000000 [ 0.006521] found SMP MP-table at [mem 0x000fcd40-0x000fcd4f] [ 0.006531] Using GB pages for direct mapping [ 0.006657] RAMDISK: [mem 0x3436a000-0x361acfff] [ 0.006659] ACPI: Early table checksum verification disabled [ 0.006661] ACPI: RSDP 0x00000000000F05B0 000024 (v02 SUPERM) [ 0.006664] ACPI: XSDT 0x00000000798A80A0 0000BC (v01 01072009 AMI 00010013) [ 0.006668] ACPI: FACP 0x00000000798DB838 00010C (v05 SUPERM SMCI--MB 01072009 AMI 00010013) [ 0.006672] ACPI: DSDT 0x00000000798A81F0 033641 (v02 SUPERM SMCI--MB 01072009 INTL 20091013) [ 0.006674] ACPI: FACS 0x0000000079D43F80 000040 [ 0.006676] ACPI: APIC 0x00000000798DB948 0000C8 (v03 SUPERM SMCI--MB 01072009 AMI 00010013) [ 0.006678] ACPI: FPDT 0x00000000798DBA10 000044 (v01 SUPERM SMCI--MB 01072009 AMI 00010013) [ 0.006680] ACPI: FIDT 0x00000000798DBA58 00009C (v01 SUPERM SMCI--MB 01072009 AMI 00010013) [ 0.006682] ACPI: SPMI 0x00000000798DBAF8 000040 (v05 SUPERM SMCI--MB 00000000 AMI. 00000000) [ 0.006684] ACPI: MCFG 0x00000000798DBB38 00003C (v01 SUPERM SMCI--MB 01072009 MSFT 00000097) [ 0.006687] ACPI: UEFI 0x00000000798DBB78 000042 (v01 SUPERM SMCI--MB 01072009 00000000) [ 0.006689] ACPI: HPET 0x00000000798DBBC0 000038 (v01 SUPERM SMCI--MB 00000001 INTL 20091013) [ 0.006691] ACPI: WDDT 0x00000000798DBBF8 000040 (v01 SUPERM SMCI--MB 00000000 INTL 20091013) [ 0.006693] ACPI: SSDT 0x00000000798DBC38 01717F (v02 SUPERM PmMgt 00000001 INTL 20120913) [ 0.006695] ACPI: NITR 0x00000000798F2DB8 000071 (v02 SUPERM SMCI--MB 00000001 INTL 20091013) [ 0.006697] ACPI: SSDT 0x00000000798F2E30 00264C (v02 SUPERM SpsNm 00000002 INTL 20120913) [ 0.006699] ACPI: SSDT 0x00000000798F5480 000064 (v02 SUPERM SpsNvs 00000002 INTL 20120913) [ 0.006701] ACPI: PRAD 0x00000000798F54E8 000102 (v02 SUPERM SMCI--MB 00000002 INTL 20120913) [ 0.006703] ACPI: DMAR 0x00000000798F55F0 0000BC (v01 SUPERM SMCI--MB 00000001 INTL 20091013) [ 0.006705] ACPI: HEST 0x00000000798F56B0 00027C (v01 SUPERM SMCI--MB 00000001 INTL 00000001) [ 0.006707] ACPI: BERT 0x00000000798F5930 000030 (v01 SUPERM SMCI--MB 00000001 INTL 00000001) [ 0.006709] ACPI: ERST 0x00000000798F5960 000230 (v01 SUPERM SMCI--MB 00000001 INTL 00000001) [ 0.006711] ACPI: EINJ 0x00000000798F5B90 000130 (v01 SUPERM SMCI--MB 00000001 INTL 00000001) [ 0.006713] ACPI: Reserving FACP table memory at [mem 0x798db838-0x798db943] [ 0.006714] ACPI: Reserving DSDT table memory at [mem 0x798a81f0-0x798db830] [ 0.006715] ACPI: Reserving FACS table memory at [mem 0x79d43f80-0x79d43fbf] [ 0.006715] ACPI: Reserving APIC table memory at [mem 0x798db948-0x798dba0f] [ 0.006716] ACPI: Reserving FPDT table memory at [mem 0x798dba10-0x798dba53] [ 0.006716] ACPI: Reserving FIDT table memory at [mem 0x798dba58-0x798dbaf3] [ 0.006717] ACPI: Reserving SPMI table memory at [mem 0x798dbaf8-0x798dbb37] [ 0.006718] ACPI: Reserving MCFG table memory at [mem 0x798dbb38-0x798dbb73] [ 0.006718] ACPI: Reserving UEFI table memory at [mem 0x798dbb78-0x798dbbb9] [ 0.006719] ACPI: Reserving HPET table memory at [mem 0x798dbbc0-0x798dbbf7] [ 0.006719] ACPI: Reserving WDDT table memory at [mem 0x798dbbf8-0x798dbc37] [ 0.006720] ACPI: Reserving SSDT table memory at [mem 0x798dbc38-0x798f2db6] [ 0.006721] ACPI: Reserving NITR table memory at [mem 0x798f2db8-0x798f2e28] [ 0.006721] ACPI: Reserving SSDT table memory at [mem 0x798f2e30-0x798f547b] [ 0.006722] ACPI: Reserving SSDT table memory at [mem 0x798f5480-0x798f54e3] [ 0.006722] ACPI: Reserving PRAD table memory at [mem 0x798f54e8-0x798f55e9] [ 0.006723] ACPI: Reserving DMAR table memory at [mem 0x798f55f0-0x798f56ab] [ 0.006724] ACPI: Reserving HEST table memory at [mem 0x798f56b0-0x798f592b] [ 0.006724] ACPI: Reserving BERT table memory at [mem 0x798f5930-0x798f595f] [ 0.006725] ACPI: Reserving ERST table memory at [mem 0x798f5960-0x798f5b8f] [ 0.006726] ACPI: Reserving EINJ table memory at [mem 0x798f5b90-0x798f5cbf] [ 0.006769] No NUMA configuration found [ 0.006770] Faking a node at [mem 0x0000000000000000-0x000000087fffffff] [ 0.006778] NODE_DATA(0) allocated [mem 0x87ffd5000-0x87fffffff] [ 0.006975] Reserving 256MB of memory at 1664MB for crashkernel (System RAM: 32654MB) [ 0.007010] Zone ranges: [ 0.007011] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.007012] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.007013] Normal [mem 0x0000000100000000-0x000000087fffffff] [ 0.007015] Device empty [ 0.007015] Movable zone start for each node [ 0.007018] Early memory node ranges [ 0.007018] node 0: [mem 0x0000000000001000-0x0000000000089fff] [ 0.007019] node 0: [mem 0x0000000000100000-0x0000000078ec5fff] [ 0.007020] node 0: [mem 0x0000000100000000-0x000000087fffffff] [ 0.007023] Initmem setup node 0 [mem 0x0000000000001000-0x000000087fffffff] [ 0.007027] On node 0, zone DMA: 1 pages in unavailable ranges [ 0.007064] On node 0, zone DMA: 118 pages in unavailable ranges [ 0.011664] On node 0, zone Normal: 28986 pages in unavailable ranges [ 0.011775] ACPI: PM-Timer IO Port: 0x408 [ 0.011781] ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1]) [ 0.011782] ACPI: LAPIC_NMI (acpi_id[0x02] high edge lint[0x1]) [ 0.011783] ACPI: LAPIC_NMI (acpi_id[0x04] high edge lint[0x1]) [ 0.011783] ACPI: LAPIC_NMI (acpi_id[0x06] high edge lint[0x1]) [ 0.011784] ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1]) [ 0.011785] ACPI: LAPIC_NMI (acpi_id[0x03] high edge lint[0x1]) [ 0.011785] ACPI: LAPIC_NMI (acpi_id[0x05] high edge lint[0x1]) [ 0.011786] ACPI: LAPIC_NMI (acpi_id[0x07] high edge lint[0x1]) [ 0.011794] IOAPIC[0]: apic_id 1, version 32, address 0xfec00000, GSI 0-23 [ 0.011798] IOAPIC[1]: apic_id 2, version 32, address 0xfec01000, GSI 24-47 [ 0.011800] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.011802] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.011805] ACPI: Using ACPI (MADT) for SMP configuration information [ 0.011805] ACPI: HPET id: 0x8086a701 base: 0xfed00000 [ 0.011810] [Firmware Bug]: TSC_DEADLINE disabled due to Errata; please update microcode to version: 0xb000020 (or later) [ 0.011811] smpboot: Allowing 8 CPUs, 0 hotplug CPUs [ 0.011820] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.011822] PM: hibernation: Registered nosave memory: [mem 0x0008a000-0x000dffff] [ 0.011822] PM: hibernation: Registered nosave memory: [mem 0x000e0000-0x000fffff] [ 0.011823] PM: hibernation: Registered nosave memory: [mem 0x78ec6000-0x78f77fff] [ 0.011824] PM: hibernation: Registered nosave memory: [mem 0x78f78000-0x798a7fff] [ 0.011825] PM: hibernation: Registered nosave memory: [mem 0x798a8000-0x79d44fff] [ 0.011825] PM: hibernation: Registered nosave memory: [mem 0x79d45000-0x8fffffff] [ 0.011826] PM: hibernation: Registered nosave memory: [mem 0x90000000-0xfed1bfff] [ 0.011826] PM: hibernation: Registered nosave memory: [mem 0xfed1c000-0xfed44fff] [ 0.011827] PM: hibernation: Registered nosave memory: [mem 0xfed45000-0xfeffffff] [ 0.011827] PM: hibernation: Registered nosave memory: [mem 0xff000000-0xffffffff] [ 0.011829] [mem 0x90000000-0xfed1bfff] available for PCI devices [ 0.011830] Booting paravirtualized kernel on bare hardware [ 0.011832] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.016162] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:8 nr_cpu_ids:8 nr_node_ids:1 [ 0.016377] percpu: Embedded 62 pages/cpu s217088 r8192 d28672 u262144 [ 0.016404] Fallback order for Node 0: 0 [ 0.016406] Built 1 zonelists, mobility grouping on. Total pages: 8228746 [ 0.016407] Policy zone: Normal [ 0.016408] Kernel command line: BOOT_IMAGE=(hd0,msdos1)/boot/vmlinuz-5.14.0-386.el9.x86_64 root=UUID=0b261d40-7844-4e73-a652-6bb884dfb0a5 ro crashkernel=1G-4G:192M,4G-64G:256M,64G-:512M console=ttyS1,115200 [ 0.016458] Unknown kernel command line parameters "BOOT_IMAGE=(hd0,msdos1)/boot/vmlinuz-5.14.0-386.el9.x86_64", will be passed to user space. [ 0.016474] random: crng init done [ 0.018656] Dentry cache hash table entries: 4194304 (order: 13, 33554432 bytes, linear) [ 0.019796] Inode-cache hash table entries: 2097152 (order: 12, 16777216 bytes, linear) [ 0.020016] mem auto-init: stack:off, heap alloc:off, heap free:off [ 0.020023] software IO TLB: area num 8. [ 0.045494] Memory: 1752328K/33438012K available (16384K kernel code, 5606K rwdata, 11524K rodata, 3868K init, 6044K bss, 986992K reserved, 0K cma-reserved) [ 0.045597] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=8, Nodes=1 [ 0.045605] Kernel/User page tables isolation: enabled [ 0.045619] ftrace: allocating 46498 entries in 182 pages [ 0.063390] ftrace: allocated 182 pages with 5 groups [ 0.063890] Dynamic Preempt: voluntary [ 0.063918] rcu: Preemptible hierarchical RCU implementation. [ 0.063919] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=8. [ 0.063920] Trampoline variant of Tasks RCU enabled. [ 0.063920] Rude variant of Tasks RCU enabled. [ 0.063921] Tracing variant of Tasks RCU enabled. [ 0.063921] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.063922] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=8 [ 0.067248] NR_IRQS: 524544, nr_irqs: 896, preallocated irqs: 16 [ 0.067410] rcu: srcu_init: Setting srcu_struct sizes based on contention. [ 0.067500] kfence: initialized - using 2097152 bytes for 255 objects at 0x00000000cef9ed2c-0x0000000036291727 [ 0.072775] Console: colour VGA+ 80x25 [ 1.145682] printk: console [ttyS1] enabled [ 1.149869] ACPI: Core revision 20221020 [ 1.153957] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 133484882848 ns [ 1.163085] APIC: Switch to symmetric I/O mode setup [ 1.168056] DMAR: Host address width 46 [ 1.171900] DMAR: DRHD base: 0x000000fbffc000 flags: 0x1 [ 1.177214] DMAR: dmar0: reg_base_addr fbffc000 ver 1:0 cap 8d2078c106f0466 ecap f020de [ 1.185209] DMAR: RMRR base: 0x0000007ba6e000 end: 0x0000007ba7dfff [ 1.191477] DMAR: ATSR flags: 0x0 [ 1.194796] DMAR: RHSA base: 0x000000fbffc000 proximity domain: 0x0 [ 1.201063] DMAR-IR: IOAPIC id 1 under DRHD base 0xfbffc000 IOMMU 0 [ 1.207416] DMAR-IR: IOAPIC id 2 under DRHD base 0xfbffc000 IOMMU 0 [ 1.213769] DMAR-IR: HPET id 0 under DRHD base 0xfbffc000 [ 1.219167] DMAR-IR: x2apic is disabled because BIOS sets x2apic opt out bit. [ 1.219168] DMAR-IR: Use 'intremap=no_x2apic_optout' to override the BIOS setting. [ 1.234190] DMAR-IR: Enabled IRQ remapping in xapic mode [ 1.239494] x2apic: IRQ remapping doesn't support X2APIC mode [ 1.245711] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 1.256087] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x3272fcf80f8, max_idle_ns: 440795261698 ns [ 1.266599] Calibrating delay loop (skipped), value calculated using timer frequency.. 6999.82 BogoMIPS (lpj=3499911) [ 1.267598] pid_max: default: 32768 minimum: 301 [ 1.268617] LSM: initializing lsm=lockdown,capability,yama,integrity,selinux,bpf [ 1.269603] Yama: becoming mindful. [ 1.270601] SELinux: Initializing. [ 1.271615] LSM support for eBPF active [ 1.273640] Mount-cache hash table entries: 65536 (order: 7, 524288 bytes, linear) [ 1.274652] Mountpoint-cache hash table entries: 65536 (order: 7, 524288 bytes, linear) [ 1.275760] CPU0: Thermal monitoring enabled (TM1) [ 1.276654] process: using mwait in idle threads [ 1.277599] Last level iTLB entries: 4KB 64, 2MB 8, 4MB 8 [ 1.278598] Last level dTLB entries: 4KB 64, 2MB 0, 4MB 0, 1GB 4 [ 1.279600] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization [ 1.280599] Spectre V2 : Mitigation: Retpolines [ 1.281598] Spectre V2 : Spectre v2 / SpectreRSB mitigation: Filling RSB on context switch [ 1.282598] Spectre V2 : Spectre v2 / SpectreRSB : Filling RSB on VMEXIT [ 1.283598] Speculative Store Bypass: Vulnerable [ 1.284604] MDS: Vulnerable: Clear CPU buffers attempted, no microcode [ 1.285598] TAA: Vulnerable: Clear CPU buffers attempted, no microcode [ 1.286598] MMIO Stale Data: Vulnerable: Clear CPU buffers attempted, no microcode [ 1.298858] Freeing SMP alternatives memory: 36K [ 1.300645] smpboot: Estimated ratio of average max frequency by base frequency (times 1024): 1053 [ 1.404603] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz (family: 0x6, model: 0x4f, stepping: 0x1) [ 1.406695] cblist_init_generic: Setting adjustable number of callback queues. [ 1.407598] cblist_init_generic: Setting shift to 3 and lim to 1. [ 1.408609] cblist_init_generic: Setting shift to 3 and lim to 1. [ 1.409608] cblist_init_generic: Setting shift to 3 and lim to 1. [ 1.410606] Performance Events: PEBS fmt2+, Broadwell events, 16-deep LBR, full-width counters, Intel PMU driver. [ 1.411598] ... version: 3 [ 1.412598] ... bit width: 48 [ 1.413598] ... generic registers: 4 [ 1.414598] ... value mask: 0000ffffffffffff [ 1.415598] ... max period: 00007fffffffffff [ 1.416598] ... fixed-purpose events: 3 [ 1.417598] ... event mask: 000000070000000f [ 1.418707] rcu: Hierarchical SRCU implementation. [ 1.418708] rcu: Max phase no-delay instances is 400. [ 1.419598] printk: console [ttyS1] printing thread started [ 1.423834] NMI watchdog: Enabled. Permanently consumes one hw-PMU counter. [ 1.423900] smp: Bringing up secondary CPUs ... [ 1.423967] x86: Booting SMP configuration: [ 1.423968] .... node #0, CPUs: #1 #2 #3 #4 [ 1.429699] MDS CPU bug present and SMT on, data leak possible. See https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/mds.html for more details. [ 1.429699] TAA CPU bug present and SMT on, data leak possible. See https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/tsx_async_abort.html for more details. [ 1.429699] MMIO Stale Data CPU bug present and SMT on, data leak possible. See https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/processor_mmio_stale_data.html for more details. [ 1.429699] #5 #6 #7 [ 1.436130] smp: Brought up 1 node, 8 CPUs [ 1.436130] smpboot: Max logical packages: 1 [ 1.436130] smpboot: Total of 8 processors activated (55998.57 BogoMIPS) [ 1.478909] node 0 deferred pages initialised in 39ms [ 1.480723] devtmpfs: initialized [ 1.480723] x86/mm: Memory block size: 128MB [ 1.481872] ACPI: PM: Registering ACPI NVS region [mem 0x798a8000-0x79d44fff] (4837376 bytes) [ 1.481872] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 1.481872] futex hash table entries: 2048 (order: 5, 131072 bytes, linear) [ 1.481872] pinctrl core: initialized pinctrl subsystem [ 1.500967] NET: Registered PF_NETLINK/PF_ROUTE protocol family [ 1.501051] DMA: preallocated 4096 KiB GFP_KERNEL pool for atomic allocations [ 1.501061] DMA: preallocated 4096 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 1.501068] DMA: preallocated 4096 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 1.501076] audit: initializing netlink subsys (disabled) [ 1.501610] audit: type=2000 audit(1700886309.255:1): state=initialized audit_enabled=0 res=1 [ 1.501664] thermal_sys: Registered thermal governor 'fair_share' [ 1.501665] thermal_sys: Registered thermal governor 'step_wise' [ 1.501666] thermal_sys: Registered thermal governor 'user_space' [ 1.510600] cpuidle: using governor menu [ 1.517626] ACPI FADT declares the system doesn't support PCIe ASPM, so disable it [ 1.517627] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 1.517673] PCI: MMCONFIG for domain 0000 [bus 00-ff] at [mem 0x80000000-0x8fffffff] (base 0x80000000) [ 1.517676] PCI: MMCONFIG at [mem 0x80000000-0x8fffffff] reserved as E820 entry [ 1.517684] PCI: Using configuration type 1 for base access [ 1.528625] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible. [ 1.528629] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages [ 1.528629] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page [ 1.528629] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages [ 1.528629] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page [ 1.536634] cryptd: max_cpu_qlen set to 1000 [ 1.565648] ACPI: Added _OSI(Module Device) [ 1.565649] ACPI: Added _OSI(Processor Device) [ 1.565649] ACPI: Added _OSI(3.0 _SCP Extensions) [ 1.565650] ACPI: Added _OSI(Processor Aggregator Device) [ 1.565650] ACPI: Added _OSI(Linux-Dell-Video) [ 1.565651] ACPI: Added _OSI(Linux-Lenovo-NV-HDMI-Audio) [ 1.565653] ACPI: Added _OSI(Linux-HPI-Hybrid-Graphics) [ 1.595006] ACPI: 4 ACPI AML tables successfully acquired and loaded [ 1.597162] ACPI: [Firmware Bug]: BIOS _OSI(Linux) query ignored [ 1.599690] ACPI: Dynamic OEM Table Load: [ 1.613266] ACPI: Interpreter enabled [ 1.613275] ACPI: PM: (supports S0 S4 S5) [ 1.613276] ACPI: Using IOAPIC for interrupt routing [ 1.614723] HEST: Enabling Firmware First mode for corrected errors. [ 1.656447] HEST: Table parsing has been initialized. [ 1.656449] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 1.656450] PCI: Using E820 reservations for host bridge windows [ 1.656890] ACPI: Enabled 5 GPEs in block 00 to 3F [ 1.677246] ACPI: PCI Root Bridge [UNC0] (domain 0000 [bus ff]) [ 1.677250] acpi PNP0A03:03: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI EDR HPX-Type3] [ 1.678354] acpi PNP0A03:03: _OSC: platform does not support [SHPCHotplug AER LTR DPC] [ 1.678894] acpi PNP0A03:03: _OSC: OS now controls [PCIeHotplug PME PCIeCapability] [ 1.678895] acpi PNP0A03:03: FADT indicates ASPM is unsupported, using BIOS configuration [ 1.678922] PCI host bridge to bus 0000:ff [ 1.678924] pci_bus 0000:ff: root bus resource [bus ff] [ 1.678934] pci 0000:ff:0b.0: [8086:6f81] type 00 class 0x088000 [ 1.678976] pci 0000:ff:0b.1: [8086:6f36] type 00 class 0x110100 [ 1.679008] pci 0000:ff:0b.2: [8086:6f37] type 00 class 0x110100 [ 1.679037] pci 0000:ff:0b.3: [8086:6f76] type 00 class 0x088000 [ 1.679066] pci 0000:ff:0c.0: [8086:6fe0] type 00 class 0x088000 [ 1.679094] pci 0000:ff:0c.1: [8086:6fe1] type 00 class 0x088000 [ 1.679122] pci 0000:ff:0c.2: [8086:6fe2] type 00 class 0x088000 [ 1.679151] pci 0000:ff:0c.3: [8086:6fe3] type 00 class 0x088000 [ 1.679181] pci 0000:ff:0f.0: [8086:6ff8] type 00 class 0x088000 [ 1.679208] pci 0000:ff:0f.1: [8086:6ff9] type 00 class 0x088000 [ 1.679236] pci 0000:ff:0f.4: [8086:6ffc] type 00 class 0x088000 [ 1.679264] pci 0000:ff:0f.5: [8086:6ffd] type 00 class 0x088000 [ 1.679292] pci 0000:ff:0f.6: [8086:6ffe] type 00 class 0x088000 [ 1.679321] pci 0000:ff:10.0: [8086:6f1d] type 00 class 0x088000 [ 1.679349] pci 0000:ff:10.1: [8086:6f34] type 00 class 0x110100 [ 1.679378] pci 0000:ff:10.5: [8086:6f1e] type 00 class 0x088000 [ 1.679406] pci 0000:ff:10.6: [8086:6f7d] type 00 class 0x110100 [ 1.679433] pci 0000:ff:10.7: [8086:6f1f] type 00 class 0x088000 [ 1.679461] pci 0000:ff:12.0: [8086:6fa0] type 00 class 0x088000 [ 1.679481] pci 0000:ff:12.1: [8086:6f30] type 00 class 0x110100 [ 1.679515] pci 0000:ff:13.0: [8086:6fa8] type 00 class 0x088000 [ 1.679576] pci 0000:ff:13.1: [8086:6f71] type 00 class 0x088000 [ 1.679623] pci 0000:ff:13.2: [8086:6faa] type 00 class 0x088000 [ 1.679663] pci 0000:ff:13.3: [8086:6fab] type 00 class 0x088000 [ 1.679703] pci 0000:ff:13.4: [8086:6fac] type 00 class 0x088000 [ 1.679743] pci 0000:ff:13.5: [8086:6fad] type 00 class 0x088000 [ 1.679781] pci 0000:ff:13.6: [8086:6fae] type 00 class 0x088000 [ 1.679811] pci 0000:ff:13.7: [8086:6faf] type 00 class 0x088000 [ 1.679842] pci 0000:ff:14.0: [8086:6fb0] type 00 class 0x088000 [ 1.679885] pci 0000:ff:14.1: [8086:6fb1] type 00 class 0x088000 [ 1.679925] pci 0000:ff:14.2: [8086:6fb2] type 00 class 0x088000 [ 1.679965] pci 0000:ff:14.3: [8086:6fb3] type 00 class 0x088000 [ 1.680005] pci 0000:ff:14.4: [8086:6fbc] type 00 class 0x088000 [ 1.680035] pci 0000:ff:14.5: [8086:6fbd] type 00 class 0x088000 [ 1.680065] pci 0000:ff:14.6: [8086:6fbe] type 00 class 0x088000 [ 1.680095] pci 0000:ff:14.7: [8086:6fbf] type 00 class 0x088000 [ 1.680126] pci 0000:ff:15.0: [8086:6fb4] type 00 class 0x088000 [ 1.680166] pci 0000:ff:15.1: [8086:6fb5] type 00 class 0x088000 [ 1.680206] pci 0000:ff:15.2: [8086:6fb6] type 00 class 0x088000 [ 1.680248] pci 0000:ff:15.3: [8086:6fb7] type 00 class 0x088000 [ 1.680289] pci 0000:ff:16.0: [8086:6f68] type 00 class 0x088000 [ 1.680324] pci 0000:ff:16.6: [8086:6f6e] type 00 class 0x088000 [ 1.680353] pci 0000:ff:16.7: [8086:6f6f] type 00 class 0x088000 [ 1.680384] pci 0000:ff:17.0: [8086:6fd0] type 00 class 0x088000 [ 1.680417] pci 0000:ff:17.4: [8086:6fb8] type 00 class 0x088000 [ 1.680447] pci 0000:ff:17.5: [8086:6fb9] type 00 class 0x088000 [ 1.680477] pci 0000:ff:17.6: [8086:6fba] type 00 class 0x088000 [ 1.680508] pci 0000:ff:17.7: [8086:6fbb] type 00 class 0x088000 [ 1.680543] pci 0000:ff:1e.0: [8086:6f98] type 00 class 0x088000 [ 1.680572] pci 0000:ff:1e.1: [8086:6f99] type 00 class 0x088000 [ 1.680603] pci 0000:ff:1e.2: [8086:6f9a] type 00 class 0x088000 [ 1.680633] pci 0000:ff:1e.3: [8086:6fc0] type 00 class 0x088000 [ 1.680654] pci 0000:ff:1e.4: [8086:6f9c] type 00 class 0x088000 [ 1.680684] pci 0000:ff:1f.0: [8086:6f88] type 00 cla 0x088000 [ 1.680714] pci 0000:ff:1f.2: [8086:6f8a] type 00 class 0x088000 [ 1.682895] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-fe]) [ 1.682898] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI EDR HPX-Type3] [ 1.683029] acpi PNP0A08:00: _OSC: platform does not support [SHPCHotplug AER LTR DPC] [ 1.683269] acpi PNP0A08:00: _OSC: OS now controls [PCIeHotplug PME PCIeCapability] [ 1.683269] acpi PNP0A08:00: FADT indicates ASPM is unsupported, using BIOS configuration [ 1.683666] PCI host bridge to bus 0000:00 [ 1.683668] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 1.683669] pci_bus 0000:00: root bus resource [io 0x1000-0xffff window] [ 1.683670] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 1.683671] pci_bus 0000:00: root bus resource [mem 0xfedb0000-0xfedb000f window] [ 1.683672] pci_bus 0000:00: root bus resource [mem 0xfedc0000-0xfedc000f window] [ 1.683672] pci_bus 0000:00: root bus resource [mem 0x90000000-0xfbffbfff window] [ 1.683673] pci_bus 0000:00: root bus resource [bus 00-fe] [ 1.683692] pci 0000:00:00.0: [8086:6f00] type 00 class 0x060000 [ 1.683791] pci 0000:00:01.0: [8086:6f02] type 01 class 0x060400 [ 1.683834] pci 0000:00:01.0: PME# supported from D0 D3hot D3cold [ 1.683923] pci 0000:00:02.0: [8086:6f04] type 01 class 0x060400 [ 1.683965] pci 0000:00:02.0: PME# supported from D0 D3hot D3cold [ 1.684050] pci 0000:00:03.0: [8086:6f08] type 01 class 0x060400 [ 1.684091] pci 0000:00:03.0: PME# supported from D0 D3hot D3cold [ 1.684171] pci 0000:00:04.0: [8086:6f20] type 00 class 0x088000 [ 1.684179] pci 0000:00:04.0: reg 0x10: [mem 0xfb42c000-0xfb42ffff 64bit] [ 1.684253] pci 0000:00:04.1: [8086:6f21] type 00 class 0x088000 [ 1.684260] pci 0000:00:04.1: reg 0x10: [mem 0xfb428000-0xfb42bfff 64bit] [ 1.684332] pci 0000:00:04.2: [8086:6f22] type 00 class 0x088000 [ 1.684339] pci 0000:00:04.2: reg 0x10: [mem 0xfb424000-0xfb427fff 64bit] [ 1.684411] pci 0000:00:04.3: [8086:6f23] type 00 class 0x088000 [ 1.684418] pci 0000:00:04.3: reg 0x10: [mem 0xfb420000-0xfb423fff 64bit] [ 1.684490] pci 0000:00:04.4: [8086:6f24] type 00 class 0x088000 [ 1.684497] pci 0000:00:04.4: reg 0x10: [mem 0xfb41c000-0xfb41ffff 64bit] [ 1.684568] pci 0000:00:04.5: [8086:6f25] type 00 class 0x088000 [ 1.684575] pci 0000:00:04.5: reg 0x10: [mem 0xfb418000-0xfb41bfff 64bit] [ 1.684652] pci 0000:00:04.6: [8086:6f26] type 00 class 0x088000 [ 1.684660] pci 0000:00:04.6: reg 0x10: [mem 0xfb414000-0xfb417fff 64bit] [ 1.684737] pci 0000:00:04.7: [8086:6f27] type 00 class 0x088000 [ 1.684744] pci 0000:00:04.7: reg 0x10: [mem 0xfb410000-0xfb413fff 64bit] [ 1.684813] pci 0000:00:05.0: [8086:6f28] type 00 class 0x088000 [ 1.684873] pci 0000:00:05.1: [8086:6f29] type 00 class 0x088000 [ 1.684953] pci 0000:00:05.2: [8086:6f2a] type 00 class 0x088000 [ 1.685007] pci 0000:00:05.4: [8086:6f2c] type 00 class 0x080020 [ 1.685013] pci 0000:00:05.4: reg 0x10: [mem 0xfb439000-0xfb439fff] [ 1.685090] pci 0000:00:11.0: [8086:8d7c] type 00 class 0xff0000 [ 1.685327] pci 0000:00:11.4: [8086:8d62] type 00 class 0x010601 [ 1.685346] pci 0000:00:11.4: reg 0x10: [io 0xf110-0xf117] [ 1.685357] pci 0000:00:11.4: reg 0x14: [io 0xf100-0xf103] [ 1.685368] pci 0000:00:11.4: reg 0x18: [io 0xf0f0-0xf0f7] [ 1.685379] pci 0000:00:11.4: reg 0x1c: [io 0xf0e0-0xf0e3] [ 1.685391] pci 0000:00:11.4: reg 0x20: [io 0xf020-0xf03f] [ 1.685402] pci 0000:00:11.4: reg 0x24: [mem 0xfb438000-0xfb4387ff] [ 1.685459] pci 0000:00:11.4: PME# supported from D3hot [ 1.685536] pci 0000:00:14.0: [8086:8d31] type 00 class 0x0c0330 [ 1.685560] pci 0000:00:14.0: reg 0x10: [mem 0xfb400000-0xfb40ffff 64bit] [ 1.685646] pci 0000:00:14.0: PME# supported from D3hot D3cold [ 1.685731] pci 0000:00:16.0: [8086:8d3a] type 00 class 0x078000 [ 1.685758] pci 0000:00:16.0: reg 0x10: [mem 0xfb437000-0xfb43700f 64bit] [ 1.685845] pci 0000:00:16.0: PME# supported from D0 D3hot D3cold [ 1.685909] pci 0000:00:16.1: [8086:8d3b] type 00 class 0x078000 [ 1.685932] pci 0000:00:16.1: reg 0x10: [mem 0xfb436000-0xfb43600f 64bit] [ 1.686019] pci 0000:00:16.1: PME# supported from D0 D3hot D3cold [ 1.686101] pci 0000:00:1a.0: [8086:8d2d] type 00 class 0x0c0320 [ 1.686123] pci 0000:00:1a.0: reg 0x10: [mem 0xfb434000-0xfb4343ff] [ 1.686237] pci 0000:00:1a.0: PME# supported from D0 D3hot D3cold [ 1.686315] pci 0000:00:1c.0: [8086:8d10] type 01 class 0x060400 [ 1.686431] pci 0000:00:1c.0: PME# supported from D0 D3hot D3cold [ 1.686528] pci 0000:00:1c.2: [8086:8d14] type 01 class 0x060400 [ 1.686650] pci 0000:00:1c.2: PME# supported from D0 D3hot D3cold [ 1.686748] pci 0000:00:1c.4: [8086:8d18] type 01 class 0x060400 [ 1.686867] pci 0000:00:1c.4: PME# supported from D0 D3hot D3cold [ 1.686971] pci 0000:00:1d.0: [8086:8d26] type 00 class 0x0c0320 [ 1.686993] pci 0000:00:1d.0: reg 0x10: [mem 0xfb43 pci 0000:00:1d.cold [ 1.687186] pci 0000:00:1f.0: [8086:8d44] type 00 class 0x060100 [ 1.687386] pci 0000:00:1f.2: [8086:8d02] type 00 class 0x010601 [ 1.687403] pci 0000:00:1f.2: reg 0x10: [io 0xf070-0xf077] [ 1.687415] pci 0000:00:1f.2: reg 0x14: [io 0xf060-0xf063] [ 1.687425] pci 0000:00:1f.2: reg 0x18: [io 0xf050-0xf057] [ 1.687435] pci 0000:00:1f.2: reg 0x1c: [io 0xf040-0xf043] [ 1.687445] pci 0000:00:1f.2: reg 0x20: [io 0xf000-0xf01f] [ 1.687456] pci 0000:00:1f.2: reg 0x24: [mem 0xfb432000-0xfb4327ff] [ 1.687509] pci 0000:00:1f.2: PME# supported from D3hot [ 1.687568] pci 0000:00:1f.3: [8086:8d22] type 00 class 0x0c0500 [ 1.687592] pci 0000:00:1f.3: reg 0x10: [mem 0xfb431000-0xfb4310ff 64bit] [ 1.687623] pci 0000:00:1f.3: reg 0x20: [io 0x0580-0x059f] [ 1.687785] acpiphp: Slot [1] registered [ 1.687796] pci 0000:00:01.0: PCI 1.687897] pci 00 class 0x010800: reg 0x10: [mem 0xfb310000-0xfb313fff 64bit] [ 1.687922] pci 0000:02:00.0: reg 0x30: [mem 0xfb300000-0xfb30ffff pref] [ 1.687925] pci 0000:02:00.0: enabling Extended Tags [ 1.688026] pci 0000:00:02.0: PCI bridge to [bus 02] [ 1.688029] pci 0000:00:02.0: bridge window [mem 0xfb300000-0xfb3fffff] [ 1.688133] pci 0000:03:00.0: [8086:10fb] type 00 class 0x020000 [ 1.688143] pci 0000:03:00.0: reg 0x10: [mem 0xfbd80000-0xfbdfffff 64bit pref] [ 1.688148] pci 0000:03:00.0: reg 0x18: [io 0xe020-0xe03f] [ 1.688158] pci 0000:03:00.0: reg 0x20: [mem 0xfbe04000-0xfbe07fff 64bit pref] [ 1.688162] pci 0000:03:00.0: reg 0x30: [mem 0xfb280000-0xfb2fffff pref] [ 1.688198] pci 0000:03:00.0: PME# supported from D0 D3hot D3cold [ 1.688214] pci 0000:03:00.0: reg 0x184: [mem 0x00000000-0x00003fff 64bit pref] [ 1.688215] pci 0000:03:00.0: VF(n) BAR0 space: [mem 0x00000000-0x000fffff 64bit pref] (contains BAR0 for 64 VFs) [ 1.688224] pci 0000:03:00.0: reg 0x190: [mem 0x000000001.688225] pci 0000:03:00.0: VF(n) BAR3 space: [mem 0x00000000-0x000fffff 64bit pns BAR3 for 64 VFs) [ 1.688377] pci 0000:03:00.1: [8086:10fb] type 00 class 0x020000 [ 1.688386] pci 0000:03:00.1: reg 0x10: [mem 0xfbd00000-0xfbd7ffff 64bit pref] [ 1.688391] pci 0000:03:00.1: reg 0x18: [io 0xe000-0xe01f] [ 1.688401] pci 0000:03:00.1: reg 0x20: [mem 0xfbe00000-0xfbe03fff 64bit pref] [ 1.688405] pci 0000:03:00.1: reg 0x30: [mem 0xfb200000-0xfb27ffff pref] [ 1.688440] pci 0000:03:00.1: PME# supported from D0 D3hot D3cold [ 1.688454] pci 0000:03:00.1: reg 0x184: [mem 0x00000000-0x00003fff 64bit pref] [ 1.688455] pci 0000:03:00.1: space: [mem 0x00000000-0x000fffff 64bit pref] (contains BAR0 for 64 VFs) [ 1.688463] pci 0000:03:00.1: reg 0x190: [mem 0x00000000-0x00003fff 64bit pref] [ 1.688464] pci 0000:03:00.1: VF(n) BAR3 space: [mem 0x00000000-0x000fffff 64bit pref] (contains BAR3 for 64 VFs) [ 1.688605] pci 0000:00:03.0: PCI bridge to [bus 03] [ 1.688606] pci 0000:00:03.0: bridge window [io 0xe000-0xefff] [ 1.688608] pci 0000:00:03.0: bridge window [mem 0xfb200000-0xfb2fffff] [ 1.688610] pci 0000:00:03.0: bridge window [mem 0xfbd00000-0xfbefffff 64bit pref] [ 1.688655] pci 0000:00:1c.0: PCI bridge to [bus 04] [ 1.688747] pci 0000:05:00.0: [1a03:1150] type 01 class 0x060400 [ 1.688861] pci 0000:05:00.0: enabling Extended Tags [ 1.688993] pci 0000:05:00.0: supports D1 D2 [ 1.688993] pci 0000:05:00.0: PME# supported from D0 D1 D2 D3hot D3cold [ 1.689167] pci 0000:00:1c.2: PCI bridge to [bus 05-06] [ 1.689171] pci 0000:00:1c.2: bridge window [io 0xd000-0xdfff] [ 1.689174] pci 0000:00:1c.2: bridge window [mem 0xfa000000-0xfb0fffff] [ 1.689256] pci_bus 0000:06: extended config space not accessible [ 1.689289] pci 0000:06:00.0: [1a03:2000] type 00 class 0x030000 [ 1.689325] pci 0000:06:00.0: reg 0x10: [mem 0xfa000000-0xfaffffff] [ 1.689345] pci 0000:06:00.0: reg 0x14: [mem 0xfb000000-0xfb01ffff] [ 1.689364] pci 0000:06:00.0: reg 0x18: [io 0xd000-0xd07f] [ 1.689463] pci 0000:06:00.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff] [ 1.689533] pci 0000:06:00.0: supports D1 D2 [ 1.689534] pci 0000:06:00.0: PME# supported from D0 D1 D2 D3hot D3cold [ 1.689686] pci 0000:05:00.0: PCI bridge to [bus 06] [ 1.689696] pci 0000:05:00.0: bridge window [io 0xd000-0xdfff] [ 1.689702] pci 0000:05:00.0: bridge window [mem 0xfa000000-0xfb0fffff] [ 1.689830] pci 0000:07:00.0: [8086:1521] type 00 class 0x020000 [ 1.689862] pci 0000:07:00.0: reg 0x10: [mem 0xfb120000-0xfb13ffff] [ 1.689899] pci 0000:07:00.0: reg 0x18: [io 0xc020-0xc03f] [ 1.689917] pci 0000:07:00.0: reg 0x1c: [mem 0xfb144000-0xfb147fff] [ 1.690136] pci 0000:07:00.0: PME# supported from D0 D3hot D3cold [ 1.690201] pci 0000:07:00.0: reg 0x184: [mem 0x00000000-0x00003fff 64bit pref] [ 1.690202] pci 0000:07:00.0: VF(n) BAR0 space: [mem 0x00000000-0x0001ffff 64bit pref] (contains BAR0 for 8 VFs) [ 1.690238] pci 0000:07:00.0: reg 0x190: [mem 0x00000000-0x00003fff 64bit pref] [ 1.690238] pci 0000:07:00.0: VF(n) BAR3 space: [mem 0x00000000-0x0001ffff 64bit pref] (contains BAR3 for 8 VFs) [ 1.690569] pci 0000:07:00.1: [8086:1521] type 00 class 0x020000 [ 1.690602] pci 0000:07:00.1: reg 0x10: [mem 0xfb100000-0xfb11ffff] [ 1.690638] pci 0000:07:00.1: reg 0x18: [io 0xc000-0xc01f] [ 1.690656] pci 0000:07:00.1: reg 0x1c: [mem 0xfb140000-0xfb143fff] [ 1.690874] pci 0000:07:00.1: PME# supported from D0 D3hot D3cold [ 1.690935] pci 0000:07:00.1: reg 0x184: [mem 0x00000000-0x00003fff 64bit pref] [ 1.690936] pci 0000:07:00.1: VF(n) BAR0 space: [mem 0x00000000-0x0001ffff 64bit pref] (contains BAR0 for 8 VFs) [ 1.690969] pci 0000:07:00.1: reg 0x190: [mem 0x00000000-0x00003fff 64bit pref] [ 1.690970] pci 0000:07:00.1: VF(n) BAR3 space: [mem 0x00000000-0x0001ffff 64bit pref] (contains BAR3 for 8 VFs) [ 1.691268] pci 0000:00:1c.4: PCI bridge to [bus 07] [ 1.691271] pci 0000:00:1c.4: bridge window [io 0xc000-0xcfff] [ 1.691274] pci 0000:00:1c.4: bridge window [mem 0xfb100000-0xfb1fffff] [ 1.691280] pci 0000:00:1c.4: bridge has subordinate 07 but max busn 08 [ 1.691938] ACPI: PCI: Interrupt link LNKA configured for IRQ 11 [ 1.691968] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 1.691998] ACPI: PCI: Interrupt link LNKC configured for IRQ 5 [ 1.692029] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 1.692058] ACPI: PCI: Interrupt link LNKE configured for IRQ 0 [ 1.692058] ACPI: PCI: Interrupt link LNKE disabled [ 1.692088] ACPI: PCI: Interrupt link LNKF configured for IRQ 0 [ 1.692088] ACPI: PCI: Interrupt link LNKF disabled [ 1.692118] ACPI: PCI: Interrupt link LNKG configured for IRQ 0 [ 1.692118] ACPI: PCI: Interrupt link LNKG disabled [ 1.692147] ACPI: PCI: Interrupt link LNKH configured for IRQ 0 [ 1.692148] ACPI: PCI: Interrupt link LNKH disabled [ 1.692611] iommu: Default domain type: Translated [ 1.692611] iommu: DMA domain TLB invalidation policy: lazy mode [ 1.692678] SCSI subsystem initialized [ 1.692693] ACPI: bus type USB registered [ 1.692705] usbcore: registered new interface driver usbfs [ 1.692710] usbcore: registered new interface driver hub [ 1.692715] usbcore: registered new device driver usb [ 1.692734] pps_core: LinuxPPS API ver. 1 registered [ 1.692734] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 1.692737] PTP clock support registered [ 1.692749] EDAC MC: Ver: 3.0.0 [ 1.692749] NetLabel: Initializing [ 1.692749] NetLabel: domain hash size = 128 [ 1.692749] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 1.692756] NetLabel: unlabeled traffic allowed by default [ 1.692757] PCI: Using ACPI for IRQ routing [ 1.703619] pci 0000:06:00.0: vgaarb: setting as boot VGA device [ 1.703620] pci 0000:06:00.0: vgaarb: bridge control possible [ 1.703621] pci 0000:06:00.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none [ 1.703622] vgaarb: loaded [ 1.703646] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0, 0, 0, 0, 0, 0 [ 1.703651] hpet0: 8 comparators, 64-bit 14.318180 MHz counter [ 1.707370] clocksource: Switched to clocksource tsc-early [ 1.715028] VFS: Disk quotas dquot_6.6.0 [ 1.715037] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 1.715084] pnp: PnP ACPI init [ 1.715396] system 00:01: [io 0x0500-0x057f] has been reserved [ 1.715398] system 00:01: [io 0x0400-0x047f] has been reserved [ 1.715399] system 00:01: [io 0x0580-0x059f] has been reserved [ 1.715400] system 00:01: [io 0x0600-0x061f] has been reserved [ 1.715401] system 00:01: [io 0x0880-0x0883] has been reserved [ 1.715402] system 00:01: [io 0x0800-0x081f] has been reserved [ 1.715403] system 00:01: [mem 0xfed1c000-0xfed3ffff] has been reserved [ 1.715404] system 00:01: [mem 0xfed45000-0xfed8bfff] has been reserved [ 1.715405] system 00:01: [mem 0xff000000-0xffffffff] has been reserved [ 1.715406] system 00:01: [mem 0xfee00000-0xfeefffff] has been reserved [ 1.715407] system 00:01: [mem 0xfed12000-0xfed1200f] has been reserved [ 1.715408] system 00:01: [mem 0xfed12010-0xfed1201f] has been reserved [ 1.715409] system 00:01: [mem 0xfed1b000-0xfed1bfff] has been reserved [ 1.715549] system 00:02: [io 0x0a00-0x0a0f] has been reserved [ 1.715550] system 00:02: [io 0x0a10-0x0a1f] has been reserved [ 1.715551] system 00:02: [io 0x0a20-0x0a2f] has been reserved [ 1.715551] system 00:02: [io 0x0a30-0x0a3f] has been reserved [ 1.715552] system 00:02: [io 0x0a40-0x0a4f] has been reserved [ 1.716212] pnp: PnP ACPI: found 5 devices [ 1.721697] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 1.721747] NET: Registered PF_INET protocol family [ 1.721916] IP idents hash table entries: 262144 (order: 9, 2097152 bytes, linear) [ 1.724189] tcp_listen_portaddr_hash hash table entries: 16384 (order: 6, 262144 bytes, linear) [ 1.724216] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) [ 1.724221] TCP established hash table entries: 262144 (order: 9, 2097152 bytes, linear) [ 1.724436] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes, linear) [ 1.724545] TCP: Hash tables configured (established 262144 bind 65536) [ 1.724659] MPTCP token hash table entries: 32768 (order: 7, 786432 bytes, linear) [ 1.724744] UDP hash table entries: 16384 (order: 7, 524288 bytes, linear) [ 1.724798] UDP-Lite hash table entries: 16384 (order: 7, 524288 bytes, linear) [ 1.724877] NET: Registered PF_UNIX/PF_LOCAL protocol family [ 1.724882] NET: Registered PF_XDP protocol family [ 1.724902] pci 0000:00:1c.4: bridge window [mem 0x00100000-0x000fffff 64bit pref] to [bus 07] add_size 100000 add_align 100000 [ 1.724912] pci 0000:00:1c.4: BAR 15: assigned [mem 0x90000000-0x900fffff 64bit pref] [ 1.724914] pci 0000:00:01.0: PCI bridge to [bus 01] [ 1.724919] pci 0000:00:02.0: PCI bridge to [bus 02] [ 1.724921] pci 0000:00:02.0: bridge window [mem 0xfb300000-0xfb3fffff] [ 1.724926] pci 0000:03:00.0: BAR 7: no space for [mem size 0x00100000 64bit pref] [ 1.724927] pci 0000:03:00.0: BAR 7: failed to assign [mem size 0x00100000 64bit pref] [ 1.724929] pci 0000:03:00.0: BAR 10: no space for [mem size 0x00100000 64bit pref] [ 1.724930] pci 0000:03:00.0: BAR 10: failed to assign [mem size 0x00100000 64bit pref] [ 1.724931] pci 0000:03:00.1: BAR 7: no space for [mem size 0x00100000 64bit pref] [ 1.724932] pci 0000:03:00.1: BAR 7: failed to assign [mem size 0x00100000 64bit pref] [ 1.724933] pci 0000:03:00.1: BAR 10: no space for [mem size 0x00100000 64bit pref] [ 1.724934] pci 0000:03:00.1: BAR 10: failed to assign [mem size 0x00100000 64bit pref] [ 1.724935] pci 0000:03:00.0: BAR 7: no space for [mem size 0x00100000 64bit pref] [ 1.724936] pci 0000:03:00.0: BAR 7: failed to assign [mem size 0x00100000 64bit pref] [ 1.724937] pci 0000:03:00.0: BAR 10: no space for [mem size 0x00100000 64bit pref] [ 1.724938] pci 0000:03:00.0: BAR 10: failed to assign [mem size 0x00100000 64bit pref] [ 1.724939] pci 0000:03:00.1: BAR 7: no space for [mem size 0x00100000 64bit pref] [ 1.724940] pci 0000:03:00.1: BAR 7: failed to assign [mem size 0x00100000 64bit pref] [ 1.724941] pci 0000:03:00.1: BAR 10: no space for [mem size 0x00100000 64bit pref] [ 1.724942] pci 0000:03:00.1: BAR 10: failed to assign [mem size 0x00100000 64bit pref] [ 1.724943] pci 0000:00:03.0: PCI bridge to [bus 03] [ 1.724944] pci 0000:00:03.0: bridge window [io 0xe000-0xefff] [ 1.724946] pci 0000:00:03.0: bridge window [mem 0xfb200000-0xfb2fffff] [ 1.724948] pci 0000:00:03.0: bridge window [mem 0xfbd00000-0xfbefffff 64bit pref] [ 1.724951] pci 0000:00:1c.0: PCI bridge to [bus 04] [ 1.724963] pci 0000:05:00.0: PCI bridge to [bus 06] [ 1.724966] pci 0000:05:00.0: bridge window [io 0xd000-0xdfff] [ 1.724974] pci 0000:05:00.0: bridge window [mem 0xfa000000-0xfb0fffff] [ 1.724990] pci 0000:00:1c.2: PCI bridge to [bus 05-06] [ 1.724992] pci 0000:00:1c.2: bridge window [io 0xd000-0xdfff] [ 1.724997] pci 0000:00:1c.2: bridge window [mem 0xfa000000-0xfb0fffff] [ 1.725006] pci 0000:07:00.0: BAR 7: assigned [mem 0x90000000-0x9001ffff 64bit pref] [ 1.725012] pci 0000:07:00.0: BAR 10: assigned [mem 0x90020000-0x9003ffff 64bit pref] [ 1.725018] pci 0000:07:00.1: BAR 7: assigned [mem 0x90040000-0x9005ffff 64bit pref] [ 1.725024] pci 0000:07:00.1: BAR 10: assigned [mem 0x90060000-0x9007ffff 64bit pref] [ 1.725032] pci 0000:00:1c.4: PCI bridge to [bus 07] [ 1.725033] pci 0000:00:1c.4: bridge window [io 0xc000-0xcfff] [ 1.725039] pci 0000:00:1c.4: bridge window [mem 0xfb100000-0xfb1fffff] [ 1.725042] pci 0000:00:1c.4: bridge window [mem 0x90000000-0x900fffff 64bit pref] [ 1.725048] pci_bus 0000:00: Some PCI device resources are unassigned, try booting with pci=realloc [ 1.725048] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 1.725049] pci_bus 0000:00: resource 5 [io 0x1000-0xffff window] [ 1.725050] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 1.725051] pci_bus 0000:00: resource 7 [mem 0xfedb0000-0xfedb000f window] [ 1.725052] pci_bus 0000:00: resource 8 [mem 0xfedc0000-0xfedc000f window] [ 1.725053] pci_bus 0000:00: resource 9 [mem 0x90000000-0xfbffbfff window] [ 1.725054] pci_bus 0000:02: resource 1 [mem 0xfb300000-0xfb3fffff] [ 1.725055] pci_bus 0000:03: resource 0 [io 0xe000-0xefff] [ 1.725055] pci_bus 0000:03: resource 1 [mem 0xfb200000-0xfb2fffff] [ 1.725056] pci_bus 0000:03: resource 2 [mem 0xfbd00000-0xfbefffff 64bit pref] [ 1.725057] pci_bus 0000:05: resource 0 [io 0xd000-0xdfff] [ 1.725058] pci_bus 0000:05: resource 1 [mem 0xfa000000-0xfb0fffff] [ 1.725058] pci_bus 0000:06: resource 0 [io 0xd000-0xdfff] [ 1.725059] pci_bus 0000:06: resource 1 [mem 0xfa000000-0xfb0fffff] [ 1.725060] pci_bus 0000:07: resource 0 [io 0xc000-0xcfff] [ 1.725061] pci_bus 0000:07: resource 1 [mem 0xfb100000-0xfb1fffff] [ 1.725061] pci_bus 0000:07: resource 2 [mem 0x90000000-0x900fffff 64bit pref] [ 1.725218] pci 0000:00:05.0: disabled boot interrupts on device [8086:6f28] [ 1.736642] pci 0000:00:1a.0: quirk_usb_early_handoff+0x0/0x140 took 10985 usecs [ 1.748639] pci 0000:00:1d.0: quirk_usb_early_handoff+0x0/0x140 took 11702 usecs [ 1.748651] pci 0000:02:00.0: CLS mismatch (64 != 32), using 64 bytes [ 1.748692] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 1.748693] software IO TLB: mapped [mem 0x0000000064000000-0x0000000068000000] (64MB) [ 1.748707] Trying to unpack rootfs image as initramfs... [ 1.748708] ACPI: bus type thunderbolt registered [ 1.810480] Initialise system trusted keyrings [ 1.810492] Key type blacklist registered [ 1.810534] workingset: timestamp_bits=36 max_order=23 bucket_order=0 [ 1.812999] zbud: loaded [ 1.873501] integrity: Platform Keyring initialized [ 1.873505] integrity: Machine keyring initialized [ 1.881704] NET: Registered PF_ALG protocol family [ 1.881707] xor: automatically using best checksumming function avx [ 1.881709] Key type asymmetric registered [ 1.881710] Asymmetric key parser 'x509' registered [ 1.881711] Running certificate verification selftests [ 1.930377] Loaded X.509 cert 'Certificate verification self-testing key: f58703bb33ce1b73ee02eccdee5b8817518fe3db' [ 1.932613] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 246) [ 1.932640] io scheduler mq-deadline registered [ 1.932641] io scheduler kyber registered [ 1.932680] io scheduler bfq registered [ 1.951558] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 1.951918] pcieport 0000:00:01.0: PME: Signaling with IRQ 25 [ 1.952108] pcieport 0000:00:02.0: PME: Signaling with IRQ 27 [ 1.952282] pcieport 0000:00:03.0: PME: Signaling with IRQ 29 [ 1.952440] pcieport 0000:00:1c.0: PME: Signaling with IRQ 30 [ 1.952620] pcieport 0000:00:1c.2: PME: Signaling with IRQ 31 [ 1.952787] pcieport 0000:00:1c.4: PME: Signaling with IRQ 32 [ 1.952972] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 1.953049] ACPI: \_SB_.SCK0.CP00: Found 2 idle states [ 1.957962] input: Power Button as /devices/LNXSYSTM:00/LNXSYBUS:00/PNP0C0C:00/input/input0 [ 1.957980] ACPI: button: Power Button [PWRB] [ 1.958003] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input1 [ 1.958028] ACPI: button: Power Button [PWRF] [ 1.976086] ERST: Error Record Serialization Table (ERST) support is initialized. [ 1.976088] pstore: Registered erst as persistent store backend [ 1.976268] GHES: APEI firmware first mode is enabled by APEI bit and WHEA _OSC. [ 1.976384] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 1.976476] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 1.981499] 00:04: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 1.987959] Non-volatile memory driver v1.3 [ 1.993865] rdac: device handler registered [ 1.993893] hp_sw: device handler registered [ 1.993894] emc: device handler registered [ 1.993912] alua: device handler registered [ 1.994235] ehci-pci 0000:00:1d.0: EHCI Host Controller [ 1.994267] ehci-pci 0000:00:1d.0: new USB bus registered, assigned bus number 1 [ 1.994280] ehci-pci 0000:00:1d.0: debug port 2 [ 1.998206] ehci-pci 0000:00:1d.0: irq 18, io mem 0xfb433000 [ 2.004631] ehci-pci 0000:00:1d.0: USB 2.0 started, EHCI 1.00 [ 2.004695] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 5.14 [ 2.004697] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 2.004698] usb usb1: Product: EHCI Host Controller [ 2.004699] usb usb1: Manufacturer: Linux 5.14.0-386.el9.x86_64 ehci_hcd [ 2.004699] usb usb1: SerialNumber: 0000:00:1d.0 [ 2.004791] hub 1-0:1.0: USB hub found [ 2.004799] hub 1-0:1.0: 2 ports detected [ 2.004917] xhci_hcd 0000:00:14.0: xHCI Host Controller [ 2.004956] xhci_hcd 0000:00:14.0: new USB bus registered, assigned bus number 2 [ 2.006050] xhci_hcd 0000:00:14.0: hcc params 0x200077c1 hci version 0x100 quirks 0x0000000000009810 [ 2.006279] ehci-pci 0000:00:1a.0: EHCI Host Controller [ 2.006306] xhci_hcd 0000:00:14.0: xHCI Host Controller [ 2.006328] ehci-pci 0000:00:1a.0: new USB bus registered, assigned bus number 3 [ 2.006333] xhci_hcd 0000:00:14.0: new USB bus registered, assigned bus number 4 [ 2.006336] xhci_hcd 0000:00:14.0: Host supports USB 3.0 SuperSpeed [ 2.006344] ehci-pci 0000:00:1a.0: debug port 2 [ 2.006374] usb usb2: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 5.14 [ 2.006377] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 2.006378] usb usb2: Product: xHCI Host Controller [ 2.006380] usb usb2: Manufacturer: Linux 5.14.0-386.el9.x86_64 xhci-hcd [ 2.006381] usb usb2: SerialNumber: 0000:00:14.0 [ 2.006508] hub 2-0:1.0: USB hub found [ 2.006540] hub 2-0:1.0: 15 ports detected [ 2.009739] usb usb4: New USB device found, idVendor=1d6b, idProduct=0003, bcdDevice= 5.14 [ 2.009742] usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 2.009744] usb usb4: Product: xHCI Host Controller [ 2.009745] usb usb4: Manufacturer: Linux 5.14.0-386.el9.x86_64 xhci-hcd [ 2.009746] usb usb4: SerialNumber: 0000:00:14.0 [ 2.009840] hub 4-0:1.0: USB hub found [ 2.009859] hub 4-0:1.0: 6 ports detected [ 2.010246] ehci-pci 0000:00:1a.0: irq 18, io mem 0xfb434000 [ 2.011556] usbcore: registered new interface driver usbserial_generic [ 2.011563] usbserial: USB Serial support registered for generic [ 2.011596] i8042: PNP: No PS/2 controller found. [ 2.011633] mousedev: PS/2 mouse device common for all mice [ 2.011707] rtc_cmos 00:00: RTC can wake from S4 [ 2.012039] rtc_cmos 00:00: registered as rtc0 [ 2.012089] rtc_cmos 00:00: setting system clock to 2023-11-25T04:25:11 UTC (1700886311) [ 2.012118] rtc_cmos 00:00: alarms up to one month, y3k, 114 bytes nvram [ 2.016607] ehci-pci 0000:00:1a.0: USB 2.0 started, EHCI 1.00 [ 2.016651] usb usb3: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 5.14 [ 2.016653] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 2.016654] usb usb3: Product: EHCI Host Controller [ 2.016655] usb usb3: Manufacturer: Linux 5.14.0-386.el9.x86_64 ehci_hcd [ 2.016656] usb usb3: SerialNumber: 0000:00:1a.0 [ 2.016746] hub 3-0:1.0: USB hub found [ 2.016750] hub 3-0:1.0: 2 ports detected [ 2.023590] intel_pstate: Intel P-state driver initializing [ 2.042125] hid: raw HID events driver (C) Jiri Kosina [ 2.042158] usbcore: registered new interface driver usbhid [ 2.042159] usbhid: USB HID core driver [ 2.042183] drop_monitor: Initializing network drop monitor service [ 2.051479] Initializing XFRM netlink socket [ 2.051562] NET: Registered PF_INET6 protocol family [ 2.099032] Freeing initrd memory: 30988K [ 2.103453] Segment Routing with IPv6 [ 2.103478] NET: Registered PF_PACKET protocol family [ 2.103503] mpls_gso: MPLS GSO support [ 2.113710] microcode: Microcode Update Driver: v2.2. [ 2.119729] resctrl: L3 allocation detected [ 2.119729] resctrl: L3 monitoring detected [ 2.119733] IPI shorthand broadcast: enabled [ 2.119739] AVX2 version of gcm_enc/dec engaged. [ 2.155921] AES CTR mode by8 optimization enabled [ 2.155978] sched_clock: Marking stable (976439139, 1179159147)->(2519025861, -363427575) [ 2.173356] registered taskstats version 1 [ 2.200127] Loading compiled-in X.509 certificates [ 2.200620] Loaded X.509 cert 'The CentOS Project: CentOS Stream kernel signing key: af40eac79a5fac646a63bef7f5fbf3e105934bfc' [ 2.200906] Loaded X.509 cert 'Red Hat Enterprise Linux Driver Update Program (key 3): bf57f3e87362bc7229d9f465321773dfd1f77a80' [ 2.201184] Loaded X.509 cert 'Red Hat Enterprise Linux kpatch signing key: 4d38fd864ebe18c5f0b72e3852e2014c3a676fc8' [ 2.201329] Loaded X.509 cert 'RH-IMA-CA: Red Hat IMA CA: fb31825dd0e073685b264e3038963673f753959a' [ 2.201488] zswap: loaded using pool lzo/zbud [ 2.214545] page_owner is disabled [ 2.214739] pstore: Using crash dump compression: deflate [ 2.214752] Key type big_key registered [ 2.254600] usb 1-1: new high-speed USB device number 2 using ehci-pci [ 2.255634] usb 2-12: new high-speed USB device number 2 using xhci_hcd [ 2.266622] usb 3-1: new high-speed USB device number 2 using ehci-pci [ 2.318492] Key type encrypted registered [ 2.318498] ima: No TPM chip found, activating TPM-bypass! [ 2.318500] Loading compiled-in module X.509 certificates [ 2.318788] Loaded X.509 cert 'The CentOS Project: CentOS Stream kernel signing key: af40eac79a5fac646a63bef7f5fbf3e105934bfc' [ 2.318791] ima: Allocated hash algorithm: sha256 [ 2.318799] ima: No architecture policies found [ 2.318807] evm: Initialising EVM extended attributes: [ 2.318807] evm: security.selinux [ 2.318807] evm: security.SMACK64 (disabled) [ 2.318808] evm: security.SMACK64EXEC (disabled) [ 2.318808] evm: security.SMACK64TRANSMUTE (disabled) [ 2.318809] evm: security.SMACK64MMAP (disabled) [ 2.318809] evm: security.apparmor (disabled) [ 2.318809] evm: security.ima [ 2.318809] evm: security.capability [ 2.318810] evm: HMAC attrs: 0x1 [ 2.382823] usb 2-12: New USB device found, idVendor=0557, idProduct=7000, bcdDevice= 0.00 [ 2.382826] usb 2-12: New USB device strings: Mfr=0, Product=0, SerialNumber=0 [ 2.383008] usb 1-1: New USB device found, idVendor=8087, idProduct=8002, bcdDevice= 0.05 [ 2.383010] usb 1-1: New USB device strings: Mfr=0, Product=0, SerialNumber=0 [ 2.383169] hub 1-1:1.0: USB hub found [ 2.383227] hub 2-12:1.0: USB hub found [ 2.383258] hub 1-1:1.0: 8 ports detected [ 2.383303] hub 2-12:1.0: 4 ports detected [ 2.395006] usb 3-1: New USB device found, idVendor=8087, idProduct=800a, bcdDevice= 0.05 [ 2.395008] usb 3-1: New USB device strings: Mfr=0, Product=0, SerialNumber=0 [ 2.395163] hub 3-1:1.0: USB hub found [ 2.395256] hub 3-1:1.0: 6 ports detected [ 2.673624] usb 2-12.1: new low-speed USB device number 3 using xhci_hcd [ 2.749607] tsc: Refined TSC clocksource calibration: 3499.995 MHz [ 2.749610] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x32734c8180a, max_idle_ns: 440795326857 ns [ 2.751700] clocksource: Switched to clocksource tsc [ 2.778043] usb 2-12.1: New USB device found, idVendor=0557, idProduct=2419, bcdDevice= 1.00 [ 2.778046] usb 2-12.1: New USB device strings: Mfr=0, Product=0, SerialNumber=0 [ 2.783085] input: HID 0557:2419 as /devices/pci0000:00/0000:00:14.0/usb2/2-12/2-12.1/2-12.1:1.0/0003:0557:2419.0001/input/input2 [ 2.834706] hid-generic 0003:0557:2419.0001: input,hidraw0: USB HID v1.00 Keyboard [HID 0557:2419] on usb-0000:00:14.0-12.1/input0 [ 2.837422] input: HID 0557:2419 as /devices/pci0000:00/0000:00:14.0/usb2/2-12/2-12.1/2-12.1:1.1/0003:0557:2419.0002/input/input3 [ 2.837481] hid-generic 0003:0557:2419.0002: input,hidraw1: USB HID v1.00 Mouse [HID 0557:2419] on usb-0000:00:14.0-12.1/input1 [ 5.046672] Freeing unused decrypted memory: 2028K [ 5.047122] Freeing unused kernel image (initmem) memory: 3868K [ 5.058655] Write protecting the kernel read-only data: 28672k [ 5.058842] Freeing unused kernel image (rodata/data gap) memory: 764K [ 5.096316] x86/mm: Checked W+X mappings: passed, no W+X pages found. [ 5.096317] x86/mm: Checking user space page tables [ 5.134011] x86/mm: Checked W+X mappings: passed, no W+X pages found. [ 5.134021] Run /init as init process [ 5.147475] systemd[1]: systemd 252-18.el9 running in system mode (+PAM +AUDIT +SELINUX -APPARMOR +IMA +SMACK +SECCOMP +GCRYPT +GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS -FIDO2 +IDN2 -IDN -IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 -PWQUALITY +P11KIT -QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD -BPF_FRAMEWORK +XKBCOMMON +UTMP +SYSVINIT default-hierarchy=unified) Welcome to [0[ 5.179753] systemd[1]: Detected architecture x86-64. [ 5.179755] systemd[1]: Running in initrd. ;31mCentOS Stream 9 dracut-057-44.git20230822.el9 (Initramfs)! [ 5.198723] systemd[1]: Hostname set to . [ OK [[ 5.244076] systemd[1]: Queued start job for default target Initrd Default Target. [ 5.244188] systemd[1]: Reached target Initrd /usr File System. 0m] Reached target Initrd /usr File System. [ OK [[ 5.265752] systemd[1]: Reached target Local File Systems. 0m] Reached target Local File Systems. [ OK [[ 5.279733] systemd[1]: Reached target Slice Units. 0m] Reached target Slice Units. [ OK [[ 5.291672] systemd[1]: Reached target Swaps. 0m] Reached target Swaps. [ OK [[ 5.302731] systemd[1]: Reached target Timer Units. 0m] Reached target Timer Units. [ OK [[ 5.314744] systemd[1]: Listening on D-Bus System Message Bus Socket. 0m] Listening on D-Bus System Message Bus Socket. [ OK [[ 5.329832] systemd[1]: Listening on Journal Socket (/dev/log). 0m] Listening on Journal Socket (/dev/log). [ OK [[ 5.343740] systemd[1]: Listening on Journal Socket. 0m] Listening on Journal Socket. [ OK [[ 5.355929] systemd[1]: Listening on udev Control Socket. 0m] Listening on udev Control Socket. [ OK [[ 5.369730] systemd[1]: Listening on udev Kernel Socket. 0m] Listening on udev Kernel Socket. [ OK [[ 5.382669] systemd[1]: Reached target Socket Units. 0m] Reached target Socket Units. Startin[ 5.395621] systemd[1]: Starting Create List of Static Device Nodes... g Create List of Static Device Nodes... Startin[ 5.411706] systemd[1]: Starting Journal Service... g Journal Service... Startin[ 5.424379] systemd[1]: Starting Load Kernel Modules... g Load Kernel Modules... Startin[ 5.437176] systemd[1]: Starting Create System Users... g Create System Users... [ 5.445651] fuse: init (API version 7.36) Startin[ 5.451783] systemd[1]: Starting Setup Virtual Console... g Setup Virtual Console... [ OK [[ 5.464347] systemd[1]: Started Journal Service. 0m] Started Journal Service. [ OK ] Finished Create List of Static Device Nodes. [ OK ] Finished Load Kernel Modules. Starting Apply Kernel Variables... [ OK ] Finished Apply Kernel Variables. [ OK ] Finished Setup Virtual Console. Starting dracut cmdline hook... [ OK ] Finished Create System Users. Starting Create Static Device Nodes in /dev... Starting Create Volatile Files and Directories... [ OK ] Finished dracut cmdline hook. [ OK ] Finished Create Static Device Nodes in /dev. [ OK ] Finished Create Volatile Files and Directories. Starting dracut pre-udev hook... [ OK ] Finished dracut pre-udev hook. Starting Rule-based Manage…for Device Events and Files... [ OK ] Started Rule-based Manager for Device Events and Files. Starting Coldplug All udev Devices... [ OK ] Finished Coldplug All udev Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Network. Starting dracut initqueue hook... Starting Show Plymouth Boot Screen... [ 5.819851] dca service started, version 1.12.1 [ OK ] Started Show Plymouth Boot Screen. [ OK ] Started Forward Password R…s to Plymouth Directory Watch. [ OK ] Reached target Path Units. [ OK ] Reached target Basic System. [ 5.872069] ixgbe: Intel(R) 10 Gigabit PCI Express Network Driver [ 5.872072] ixgbe: Copyright (c) 1999-2016 Intel Corporation. [ 5.887639] igb: Intel(R) Gigabit Ethernet Network Driver [ 5.887641] igb: Copyright (c) 2007-2014 Intel Corporation. [ 5.903748] ahci 0000:00:11.4: AHCI 0001.0300 32 slots 4 ports 6 Gbps 0xf impl SATA mode [ 5.903751] ahci 0000:00:11.4: flags: 64bit ncq pm led clo pio slum part ems apst [ 5.903825] ACPI: bus type drm_connector registered [ 5.903857] nvme nvme0: pci function 0000:02:00.0 [ 5.918840] scsi host0: ahci [ 5.918959] scsi host1: ahci [ 5.919071] scsi host2: ahci [ 5.919203] scsi host3: ahci [ 5.919246] ata1: SATA max UDMA/133 abar m2048@0xfb438000 port 0xfb438100 irq 43 [ 5.919249] ata2: SATA max UDMA/133 abar m2048@0xfb438000 port 0xfb438180 irq 43 [ 5.919252] ata3: SATA max UDMA/133 abar m2048@0xfb438000 port 0xfb438200 irq 43 [ 5.919254] ata4: SATA max UDMA/133 abar m2048@0xfb438000 port 0xfb438280 irq 43 [ 5.919469] ahci 0000:00:1f.2: AHCI 0001.0300 32 slots 6 ports 6 Gbps 0x3f impl SATA mode [ 5.919472] ahci 0000:00:1f.2: flags: 64bit ncq pm led clo pio slum part ems apst [ 5.921680] nvme nvme0: 8/0/0 default/read/poll queues [ 5.937786] scsi host4: ahci [ 5.938197] scsi host5: ahci [ 5.938603] scsi host6: ahci [ 5.939038] scsi host7: ahci [ 5.939450] scsi host8: ahci [ 5.939868] scsi host9: ahci [ 5.939993] ata5: SATA max UDMA/133 abar m2048@0xfb432000 port 0xfb432100 irq 45 [ 5.939996] ata6: SATA max UDMA/133 abar m2048@0xfb432000 port 0xfb432180 irq 45 [ 5.939999] ata7: SATA max UDMA/133 abar m2048@0xfb432000 port 0xfb432200 irq 45 [ 5.940001] ata8: SATA max UDMA/133 abar m2048@0xfb432000 port 0xfb432280 irq 45 [ 5.940004] ata9: SATA max UDMA/133 abar m2048@0xfb432000 port 0xfb432300 irq 45 [ 5.940007] ata10: SATA max UDMA/133 abar m2048@0xfb432000 port 0xfb432380 irq 45 [ 5.944161] igb 0000:07:00.0: added PHC on eth0 [ 5.944163] igb 0000:07:00.0: Intel(R) Gigabit Ethernet Network Connection [ 5.944163] igb 0000:07:00.0: eth0: (PCIe:5.0Gb/s:Width x4) 0c:c4:7a:d8:da:66 [ 5.944241] igb 0000:07:00.0: eth0: PBA No: 050B00-000 [ 5.944241] igb 0000:07:00.0: Using MSI-X interrupts. 8 rx queue(s), 8 tx queue(s) [ 5.975832] ast 0000:06:00.0: [drm] Using P2A bridge for configuration [ 5.975837] ast 0000:06:00.0: [drm] AST 2400 detected [ 5.975854] ast 0000:06:00.0: [drm] Using analog VGA [ 5.975863] ast 0000:06:00.0: [drm] dram MCLK=408 Mhz type=1 bus_width=16 [ 5..0 20120228 for 0000:06:00.0 on minor 0 [ 6.000078] igb 0000:07:00.1: added PHC on eth1 [ 6.000079] igb 0000:07:00.1: Intel(R) Gigabit Ethernet Network Connection [ 6.000080] igb 0000:07:00.1: eth1: (PCIe:5.0Gb/s:Width x4) 0c:c4:7a:d8:da:67 [ 6.000162] igb 0000:07:00.1: eth1: PBA No: 050B00-000 [ 6.000163] igb 0000:07:00.1: Using MSI-X interrupts. 8 rx queue(s), 8 tx queue(s) [ 6.001438] igb 0000:07:00.0 eno1: renamed from eth0 [ 6.009722] igb 0000:07:00.1 eno2: renamed from eth1 [ 6.016997] fbcon: astdrmfb (fb0) is primary device [ 6.065902] Console: switching to colour frame buffer device 128x48 [ 6.102657] ast 0000:06:00.0: [drm] fb0: astdrmfb frame buffer device [ 6.226970] ata3: SATA link down (SStatus 0 SControl 300) [ 6.227059] ata4: SATA link down (SStatus 0 SControl 300) [ 6.227395] ata2: SATA link down (SStatus 0 SControl 300) [ 6.228719] ata1: SATA link down (SStatus 0 SControl 300) [ 6.251081] ata5: SATA link up 6.0 Gbps (SStatus 133 SControl 300) [ 6.251210] ata9: SATA link down (SStatus 0 SControl 300) [ 6.251941] ata10: SATA link down (SStatus 0 SControl 300) [ 6.251960] ata7: SATA link down (SStatus 0 SControl 300) [ 6.252119] ata5.00: ATA-9: ST1000NM0033-9ZM173, SN06, max UDMA/133 [ 6.252CQ Send/Recv Log6.252680] ata5.0ors, multi 16: LBA48 NCQ (depth 32), AA [ 6.252799] ata8: SATA link down (SStatus 0 SControl 300) [ 6.254092] ata5.00: NCQ Send/Recv Log not supported [ 6.254096] ata5.00: configured for UDMA/133 [ 6.254198] scsi 4:0:0:0: Direct-Access ATA ST1000NM0033-9ZM SN06 PQ: 0 ANSI: 5 [ 6.256872] ata6: SATA link down (SStatus 0 SControl 300) [ 6.344352] scsi 4:0:0:0: Attached scsi generic sg0 type 0 [ 6.412514] sd 4:0:0:0: [sda] 1953525168 512-byte logical blocks: (1.00 TB/932 GiB) [ 6.412526] sd 4:0:0:0: [sda] Write Protect is off [ 6.412545] sd 4:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA [ 6.412566] sd 4:0:0:0: [sda] Preferred minimum I/O size 512 bytes [ 6.426300] sda: sda1 [ 6.426351] sd 4:0:0:0: [sda] Attached SCSI disk [ OK ] Found device ST1000NM0033-9ZM173 1. [ OK ] Reached target Initrd Root Device. [ 7.010844] ixgbe 0000:03:00.0: Multiqueue Enabled: Rx Queue count = 8, Tx Queue count = 8 XDP Queue count = 0 [ 7.011131] ixgbe 0000:03:00.0: 32.000 Gb/s available PCIe bandwidth (5.0 GT/s PCIe x8 link) [ 7.011213] ixgbe 0000:03:00.0: MAC: 2, PHY: 1, PBA No: 0210FF-0FF [ 7.011214] ixgbe 0000:03:00.0: 0c:c4:7a:8f:47:e0 [ 7.013636] ixgbe 0000:03:00.0: Intel(R) 10 Gigabit Network Connection [ 7.169847] ixgbe 0000:03:00.1: Multiqueue Enabled: Rx Queue count = 8, Tx Queue count = 8 XDP Queue count = 0 [ 7.170146] ixgbe 0000:03:00.1: 32.000 Gb/s available PCIe bandwidth (5.0 GT/s PCIe x8 link) [ 7.170228] ixgbe 0000:03:00.1: MAC: 2, PHY: 14, SFP+: 4, PBA No: 0210FF-0FF [ 7.170229] ixgbe 0000:03:00.1: 0c:c4:7a:8f:47:e1 [ 7.173391] ixgbe 0000:03:00.1: Intel(R) 10 Gigabit Network Connection [ 7.181346] ixgbe 0000:03:00.1 enp3s0f1: renamed from eth1 [ 7.214017] ixgbe 0000:03:00.0 enp3s0f0: renamed from eth0 [ OK ] Finished dracut initqueue hook. [ OK ] Reached target Preparation for Remote File Systems. [ OK ] Reached target Remote File Systems. Starting File System Check…7844-4e73-a652-6bb884dfb0a5... [ OK ] Finished File System Check…0-7844-4e73-a652-6bb884dfb0a5. Mounting /sysroot... [ 7.440161] EXT4-fs (sda1): mounted filesystem with ordered data mode. Quota mode: none. [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Mountpoints Configured in the Real Root... [ OK ] Finished Mountpoints Configured in the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Finished dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped target Network. [ OK ] Stopped target Timer Units. [ OK ] Closed D-Bus System Message Bus Socket. [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Basic System. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Initrd /usr File System. [ OK ] Stopped target Path Units. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Preparation for Remote File Systems. [ OK ] Stopped target Slice Units. [ OK ] Stopped target Socket Units. [ OK ] Stopped target System Initialization. [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Swaps. [ OK ] Stopped dracut initqueue hook. Starting Plymouth switch root service... [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped Load Kernel Modules. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped Coldplug All udev Devices. Stopping Rule-based Manage…for Device Events and Files... [ OK ] Stopped Rule-based Manager for Device Events and Files. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. Starting Cleanup udev Database... [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create List of Static Device Nodes. [ OK ] Stopped Create System Users. [ OK ] Finished Cleaning Up and Shutting Down Daemons. [ OK ] Finished Cleanup udev Database. [ OK ] Reached target Switch Root. [ OK ] Finished Plymouth switch root service. Starting Switch Root... [ 7.791875] systemd-journald[279]: Received SIGTERM from PID 1 (systemd). [ 8.279963] SELinux: policy capability network_peer_controls=1 [ 8.279966] SELinux: policy capability open_perms=1 [ 8.279966] SELinux: policy capability extended_socket_class=1 [ 8.279967] SELinux: policy capability always_check_network=0 [ 8.279967] SELinux: policy capability cgroup_seclabel=1 [ 8.279967] SELinux: policy capability nnp_nosuid_transition=1 [ 8.279968] SELinux: policy capability genfs_seclabel_symlinks=1 [ 8.323890] audit: type=1403 audit(1700886317.810:2): auid=4294967295 ses=4294967295 lsm=selinux res=1 [ 8.326082] systemd[1]: Successfully loaded SELinux policy in 115.983ms. [ 8.426749] systemd[1]: Relabelled /dev, /dev/shm, /run, /sys/fs/cgroup in 30.317ms. [ 8.435448] systemd[1]: systemd 252-18.el9 running in system mode (+PAM +AUDIT +SELINUX -APPARMOR +IMA +SMACK +SECCOMP +GCRYPT +GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS -FIDO2 +IDN2 -IDN -IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 -PWQUALITY +P11KIT -QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD -BPF_FRAMEWORK +XKBCOMMON +UTMP +SYSVINIT default-hierarchy=unified) [ 8.446675] systemd[1]: Detected architecture x86-64. Welcome to CentOS Stream 9! [ OK [[ 9.410191] systemd[1]: initrd-switch-root.service: Deactivated successfully. [ 9.410289] systemd[1]: Stopped Switch Root. 0m] Stopped [0;[ 9.410672] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 9.410969] systemd[1]: Created slice Slice /system/getty. [ 9.419316] systemd[1]: Created slice Slice /system/modprobe. [ 9.423630] systemd[1]: Created slice Slice /system/serial-getty. [ 9.433817] systemd[1]: Created slice Slice /system/sshd-keygen. 1;39mSwitch Root[ 9.457134] systemd[1]: Created slice User and Session Slice. [ 9.457165] systemd[1]: Dispatch Password Requests to Console Directory Watch was skipped because of an unmet condition check (ConditionPathExists=!/run/plymouth/pid). [ 9.457259] systemd[1]: Started Forward Password Requests to Wall Directory Watch. [ 9.457608] systemd[1]: Set up automount Arbitrary Executable File Formats File System Automount Point. [ 9.478978] systemd[1]: Reached target ceph target allowing to start/stop all ceph-mon@.service instances at once. . [[ 9.506222] systemd[1]: Reached target ceph target allowing to start/stop all ceph-mds@.service instances at once. [ 9.506245] systemd[1]: Reached target ceph target allowing to start/stop all ceph-mgr@.service instances at once. [ 9.506262] systemd[1]: Reached target ceph target allowing to start/stop all ceph-osd@.service instances at once. [ 9.506278] systemd[1]: Reached target ceph target allowing to start/stop all ceph-radosgw@.service instances at once. [ 9.506304] systemd[1]: Reached target ceph target allowing to start/stop all ceph-rbd-mirror@.service instances at once. [ 9.506321] systemd[1]: Reached target Local Encrypted Volumes. [ 9.506346] systemd[1]: Stopped target Switch Root. [ 9.506365] systemd[1]: Stopped target Initrd File Systems. [ 9.506379] systemd[1]: Stopped target Initrd Root File System. [ 9.506395] systemd[1]: Reached target Local Integrity Protected Volumes. [ 9.506430] systemd[1]: Reached target Path Units. [ 9.506461] systemd[1]: Reached target Slice Units. OK ] Crea[ 9.506485] systemd[1]: Reached target Swaps. [ 9.506504] systemd[1]: Reached target Local Verity Protected Volumes. [ 9.506865] systemd[1]: Listening on Device-mapper event daemon FIFOs. [ 9.608173] systemd[1]: Listening on LVM2 poll daemon socket. ted slice Slice /system/getty. [ OK ] Created slice Slice /[ 9.629745] systemd[1]: Listening on RPCbind Server Activation Socket. [ 9.629787] systemd[1]: Reached target RPC Port Mapper. [ 9.633731] systemd[1]: Listening on Process Core Dump Socket. [ 9.643626] systemd[1]: Listening on initctl Compatibility Named Pipe. system/modprobe[ 9.656552] systemd[1]: Listening on udev Control Socket. [ 9.656808] systemd[1]: Listening on udev Kernel Socket. [0m. [ OK ] Creat[ 9.668801] systemd[1]: Mounting Huge Pages File System... [ 9.669691] systemd[1]: Mounting POSIX Message Queue File System... ed slice [0;1;3[ 9.683235] systemd[1]: Mounting Kernel Debug File System... [ 9.684041] systemd[1]: Mounting Kernel Trace File System... 9mSlice /system/[ 9.695248] systemd[1]: Kernel Module supporting RPCSEC_GSS was skipped because of an unmet condition check (ConditionPathExists=/etc/krb5.keytab). [ 9.703545] systemd[1]: Starting Create List of Static Device Nodes... serial-getty. [ OK[ 9.717063] systemd[1]: Starting Monitoring of LVM2 mirrors, snapshots etc. using dmeventd or progress polling... [ 9.717894] systemd[1]: Starting Load Kernel Module configfs... ] Created slice S[ 9.735967] systemd[1]: Starting Load Kernel Module drm... [ 9.736711] systemd[1]: Starting Load Kernel Module fuse... lice /system/sshd-keygen. [ OK ] Created slice User and Session Slice. [ OK ] Started Forward Password R…uests to Wall Directory Watch. [[ 9.764827] systemd[1]: Starting Read and set NIS domainname from /etc/sysconfig/network... [ 9.764892] systemd[1]: plymouth-switch-root.service: Deactivated successfully. OK ] Set [ 9.764939] systemd[1]: Stopped Plymouth switch root service. [ 9.765112] systemd[1]: Stopped Journal Service. up automount [0[ 9.794947] systemd[1]: Starting Journal Service... [ 9.796664] systemd[1]: Starting Load Kernel Modules... ;1;39mArbitrary…s File System [ 9.806752] systemd[1]: Starting Generate network units from Kernel command line... [ 9.807540] systemd[1]: Starting Remount Root and Kernel File Systems... Automount Point[ 9.807620] systemd[1]: Repartition Root Disk was skipped because no trigger condition checks were met. [ 9.808361] systemd[1]: Starting Coldplug All udev Devices... [ 9.824155] systemd[1]: Mounted Huge Pages File System. [ 9.839827] systemd[1]: Mounted POSIX Message Queue File System. [0m. [ [ 9.850952] systemd[1]: Mounted Kernel Debug File System. OK ] Reach[ 9.851106] systemd[1]: Mounted Kernel Trace File System. [ 9.851484] systemd[1]: Finished Create List of Static Device Nodes. ed target [0;1;[ 9.851843] systemd[1]: Finished Read and set NIS domainname from /etc/sysconfig/network. [ 9.857637] EXT4-fs (sda1): re-mounted. Quota mode: none. [ 9.858284] systemd[1]: Finished Generate network units from Kernel command line. [ 9.864646] systemd[1]: Finished Remount Root and Kernel File Systems. 39mceph target\xe2\x80[ 9.870943] systemd[1]: Started Journal Service. \xa6on@.service instances at once. [ OK ] Reached target ceph target…ds@.service instances at once. [ OK ] Reached target ceph target…gr@.service instances at once. [ OK ] Reached target [ 9.928419] systemd-journald[567]: Received client request to flush runtime journal. ceph target…sd@.service instances at once. [ OK ] Reached target ceph target…gw@.service instances at once. [ OK ] Reached target ceph target…or@.service instances at once. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd File Systems. [ OK ] Stopped target Initrd Root File System. [ OK ] Reached target Local Integrity Protected Volumes. [ OK ] Reached target Path Units. [ OK ] Reached target Slice Units. [ OK ] Reached target Swaps. [ OK ] Reached target Local Verity Protected Volumes. [ OK ] Listening on Device-mapper event daemon FIFOs. [ OK ] Listening on LVM2 poll daemon socket. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Listening on Process Core Dump Socket. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Listening on udev Control Socket. [ OK ] Listening on udev Kernel Socket. Mounting Huge Pages File System... Mounting POSIX Message Queue File System... Mounting Kernel Debug File System... Mounting Kernel Trace File System... Starting Create List of Static Device Nodes... Starting Monitoring of LVM…meventd or progress polling... Starting Load Kernel Module configfs... Starting Load Kernel Module drm... Starting Load Kernel Module fuse... Starting Read and set NIS …from /etc/sysconfig/network... [ OK ] Stopped Plymouth switch root service. [ OK ] Stopped Journal Service. Starting Journal Service... Starting Load Kernel Modules... Starting Generate network …ts from Kernel command line... Starting Remount Root and Kernel File Systems... Starting Coldplug All udev Devices... [ OK ] Mounted Huge Pages File System. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Mounted Kernel Debug File System. [ OK ] Mounted Kernel Trace File System. [ OK ] Finished Create List of Static Device Nodes.[ 10.192002] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. [ 10.192020] device-mapper: uevent: version 1.0.3 [ 10.192105] device-mapper: ioctl: 4.48.0-ioctl (2023-03-01) initialised: dm-devel@redhat.com [ OK ] Finished Read and set NIS …e from /etc/sysconfig/network. [ OK ] Finished Generate network units from Kernel command line. [ OK ] Finished Remount Root and Kernel File Systems. [ OK ] Started Journal Service. [ OK ] Reached target Preparation for Network. Starting Flush Journal to Persistent Storage... Starting Load/Save Random Seed... Starting Create Static Device Nodes in /dev... [ OK ] Finished Flush Journal to Persistent Storage. [ OK ] Finished Coldplug All udev Devices. [ OK ] Finished Load Kernel Module drm. [ OK ] Finished Load Kernel Module configfs. [ OK ] Finished Load Kernel Module fuse. Mounting FUSE Control File System... Mounting Kernel Configuration File System... [ OK ] Mounted FUSE Control File System. [ OK ] Mounted Kernel Configuration File System. [ OK ] Finished Load Kernel Modules. Starting Apply Kernel Variables... [ OK ] Finished Load/Save Random Seed. [ OK ] Finished Apply Kernel Variables. [ OK ] Finished Monitoring of LVM… dmeventd or progress polling. [ OK ] Finished Create Static Device Nodes in /dev. Starting Rule-based Manage…for Device Events and Files... [ OK ] Started Rule-based Manager for Device Events and Files. [ OK ] Reached target Preparation for Local File Systems. [ OK ] Reached target Local File Systems. Starting Tell Plymouth To Write Out Runtime Data... Starting Automatic Boot Loader Update... Starting Create Volatile Files and Directories... Starting Load Kernel Module configfs... Starting Load Kernel Module fuse... [ OK ] Finished Load Kernel Module fuse. [ OK ] Finished Load Kernel Module configfs. [ OK ] Finished Automatic Boot Loader Update. [ OK ] Finished Tell Plymouth To Write Out Runtime Data. [ 10.573002] ioatdma: Intel(R) QuickData Technology Driver 5.00 [ 10.582401] igb 0000:07:00.0: DCA enabled [ 10.582416] igb 0000:07:00.1: DCA enabled [ 10.614214] mei_me 0000:00:16.0: Device doesn't have valid ME Interface [ 10.639621] i801_smbus 0000:00:1f.3: SMBus using PCI interrupt [ 10.642127] i2c i2c-3: 2/8 memory slots populated (from DMI) [ 10.642130] i2c i2c-3: Systems with more than 4 memory slots not supported yet, not instantiating SPD [ 10.661946] IPMI message handler: version 39.2 [ 10.667158] ipmi device interface [ 10.681228] ipmi_si: IPMI System Interface driver [ 10.684684] ipmi_si dmi-ipmi-si.0: ipmi_platform: probing via SMBIOS [ 10.684687] ipmi_platform: ipmi_si: SMBIOS: io 0xca2 regsize 1 spacing 1 irq 0 [ 10.684690] ipmi_si: Adding SMBIOS-specified kcs state machine [ 10.684750] ipmi_si IPI0001:00: ipmi_platform: probing via ACPI [ 10.684882] ipmi_si IPI0001:00: ipmi_platform: [io 0x0ca2] regsize 1 spacing 1 irq 0 [ 10.717835] power_meter ACPI000D:00: Found ACPI power meter. [ 10.717862] power_meter ACPI000D:00: Ignoring unsafe software power cap! [ 10.717867] power_meter ACPI000D:00: hwmon_device_register() is deprecated. Please convert the driver to use hwmon_device_register_with_info(). [ OK ] Started [0;[ 10.745094] input: PC Speaker as /devices/platform/pcspkr/input/input4 1;39m/usr/sbin/l[ 10.747241] ipmi_si dmi-ipmi-si.0: Removing SMBIOS-specified kcs state machine in favor of ACPI vm vgch…-autoa[ 10.747244] ipmi_si: Adding ACPI-specified kcs state machine ctivation event [ 10.747340] iTCO_vendor_support: vendor-support=0 vg_nvme. [ 10.748787] ipmi_si: Trying ACPI-specified kcs state machine at i/o address 0xca2, slave address 0x20, irq 0 [ 10.754732] iTCO_wdt iTCO_wdt.1.auto: Found a Wellsburg TCO device (Version=2, TCOBASE=0x0460) [ OK ] Finished [0[ 10.755157] iTCO_wdt iTCO_wdt.1.auto: initialized. heartbeat=30 sec (nowayout=0) ;1;39mCreate Volatile Files and Directories.[ 10.810407] RAPL PMU: API unit is 2^-32 Joules, 2 fixed counters, 655360 ms ovfl timer [ OK [ 10.810410] RAPL PMU: hw unit of domain package 2^-14 Joules [ 10.810411] RAPL PMU: hw unit of domain dram 2^-16 Joules ] Found device /de[ 10.833607] ipmi_si IPI0001:00: The BMC does not support clearing the recv irq bit, compensating, but the BMC needs to be fixed. v/vg_nvme/lv_5. Starting Security Auditing Service... Startin[ 10.855924] ipmi_si IPI0001:00: IPMI message handler: Found new BMC (man_id: 0x002a7c, prod_id: 0x0833, dev_id: 0x20) g RPC Bind... Mounting /var/lib/ceph... Mounting RPC Pipe File System... [ 10.935048] ipmi_si IPI0001:00: IPMI kcs interface initialized [ 10.943124] ipmi_ssif: IPMI SSIF Interface driver [ 11.040173] RPC: Registered named UNIX socket transport module. [ OK [[ 11.040175] RPC: Registered udp transport module. 0m] Mounted [0;[ 11.040175] RPC: Registered tcp transport module. 1;39mRPC Pipe File System. [ OK ] Reached targ[ 11.040176] RPC: Registered tcp NFSv4.1 backchannel transport module. et rpc_pipefs.target. [ OK ] Started RPC Bind. [ 11.260244] SGI XFS with ACLs, security attributes, scrub, quota, no debug enabled [ 11.268870] XFS (dm-4): Deprecated V4 format (crc=0) will not be supported after September 2030. [ 11.268975] XFS (dm-4): Mounting V4 Filesystem [ OK [[ 11.273548] XFS (dm-4): Ending clean mount [ 11.273584] xfs filesystem being mounted at /var/lib/ceph supports timestamps until 2038 (0x7fffffff) 0m] Mounted /var/lib/ceph. [ 11.336511] EDAC MC0: Giving out device to module sb_edac controller Broadwell SrcID#0_Ha#0: DEV 0000:ff:12.0 (INTERRUPT) [ 11.336512] EDAC sbridge: Ver: 1.1.2 [ 11.421117] intel_rapl_common: Found RAPL domain package [ 11.421123] intel_rapl_common: Found RAPL domain dram [ OK ] Started Security Auditing Service. Starting Record System Boot/Shutdown in UTMP... [ OK ] Finished Record System Boot/Shutdown in UTMP. [ OK ] Reached target System Initialization. [ OK ] Started Daily rotation of log files. [ OK ] Started Updates mlocate database every day. [ OK ] Started Run system activit…ounting tool every 10 minutes. [ OK ] Started Generate summary o…esterday's process accounting. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Reached target Timer Units. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Listening on SSSD Kerberos…ache Manager responder socket. [ OK ] Reached target Socket Units. Starting D-Bus System Message Bus... [ OK ] Started D-Bus System Message Bus. [ OK ] Reached target Basic System. Starting Network Manager... [ OK ] Started Ceph crash dump collector. [ OK ] Reached target ceph target…h*@.service instances at once. Starting NTP client/server... Starting Restore /run/initramfs on shutdown... [ OK ] Started irqbalance daemon. [ OK ] Started libstoragemgmt plug-in server daemon. [ OK ] Started Machine Check Exception Logging Daemon. Starting Authorization Manager... Starting Self Monitoring a…g Technology (SMART) Daemon... [ OK ] Reached target sshd-keygen.target. [ OK ] Reached target User and Group Name Lookups. Starting Resets System Activity Logs... Starting User Login Management... [ OK ] Finished Restore /run/initramfs on shutdown. [ OK ] Started User Login Management. [ OK ] Finished Resets System Activity Logs. [ OK ] Started Self Monitoring an…ing Technology (SMART) Daemon. [ OK ] Started NTP client/server. Starting Hostname Service... [ OK ] Started Hostname Service. [ OK ] Listening on Load/Save RF …itch Status /dev/rfkill Watch. [ OK ] Started Authorization Manager. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ 13.265068] ixgbe 0000:03:00.0: registered PHC device on enp3s0f0 [ 13.414513] ixgbe 0000:03:00.1: registered PHC device on enp3s0f1 [ 13.642294] ixgbe 0000:03:00.1 enp3s0f1: detected SFP+: 4 [ 13.725736] ixgbe 0000:03:00.1 enp3s0f1: NIC Link is Up 10 Gbps, Flow Control: RX/TX [ 13.725867] IPv6: ADDRCONF(NETDEV_CHANGE): enp3s0f1: link becomes ready [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting Network Manager Wait Online... Starting GSSAPI Proxy Daemon... Starting OpenSSH server daemon... Starting Dynamic System Tuning Daemon... [ OK ] Started OpenSSH server daemon. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Preparation for Remote File Systems. [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Finished Permit User Sessions. [ OK ] Started Deferred execution scheduler. [ OK ] Started Command Scheduler. [ OK ] Started Dynamic System Tuning Daemon. [* ] A start job is running for Network Manager Wait Online (9s / no limit) M [** ] A start job is running for Network Manager Wait Online (9s / no limit) M [ OK ] Finished Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Crash recovery kernel arming... [ OK ] Started Nagios Remote Plugin Executor. Starting /etc/rc.d/rc.local Compatibility... Starting Notify NFS peers of a restart... Starting System Logging Service... + '[' '!' -f /.cephlab_net_configured ']' + set +e + attempts=0 + myips= + '[' '' '!=' '' ']' + '[' 0 -ge 10 ']' ++ ip -4 addr ++ grep -oP '(?<=inet\s)\d+(\.\d+){3}' ++ grep -v '127.0.0.1\|127.0.1.1' [ OK ] Started Notify NFS peers of a restart. + myips=172.21.15.76 + attempts=1 + sleep 1 [ OK ] Started System Logging Service. Starting SETroubleshoot da…ing new SELinux denial logs... + '[' 172.21.15.76 '!=' '' ']' + set -e + '[' -n 172.21.15.76 ']' + for ip in $myips + timeout 1s ping -I 172.21.15.76 -nq -c1 172.21.0.1 ++ dig +short -x 172.21.15.76 @172.21.0.1 ++ sed 's/\.com.*/\.com/g' + newhostname=smithi076.front.sepia.ceph.com + '[' -n smithi076.front.sepia.ceph.com ']' + hostname smithi076.front.sepia.ceph.com ++ hostname -d + newdomain=front.sepia.ceph.com ++ hostname -s + shorthostname=smithi076 + echo smithi076 + grep -q front.sepia.ceph.com /etc/hosts + sed -i 's/.*front.sepia.ceph.com.*/172.21.15.76 smithi076.front.sepia.ceph.com smithi076/g' /etc/hosts + break + command -v zypper + command -v apt-get + '[' -e /.cephlab_rc_local ']' + exit 0 [ OK ] Started /etc/rc.d/rc.local Compatibility. Starting Hold until boot process finishes up... Starting Terminate Plymouth Boot Screen... [ 24.154264] block sda: the capability attribute has been deprecated. CentOS Stream 9 Kernel 5.14.0-386.el9.x86_64 on an x86_64 Activate the web console with: systemctl enable --now cockpit.socket 2023-11-25T04:25:37.023 DEBUG:teuthology.orchestra.console.smithi076:expect after: smithi076 login: 2023-11-25T04:25:37.075 DEBUG:teuthology.orchestra.console.smithi076:console disconnect output: [Enter `^Ec?' for help] \x80 \x80 \x80 Initializing Intel(R) Boot Agent XE v2.1.60 PXE 2.1 Build 088 (WfM 2.0)  Press Ctrl+S to enter the Setup Menu.                      Press Ctrl+S to enter the Setup Menu..  \x80 \x80 Initializing Intel(R) Boot Agent XE v2.1.60 PXE 2.1 Build 088 (WfM 2.0)  Press Ctrl+S to enter the Setup Menu.                      Press Ctrl+S to enter the Setup Menu..  \x80 \x80 Initializing Intel(R) Boot Agent GE v1.5.72 PXE 2.1 Build 092 (WfM 2.0) Press Ctrl+S to enter the Setup Menu.                       Press Ctrl+S to enter the Setup Menu..  \x80 \x80  el(R) Boot Agent GE v1.5.72 PXE 2.1 Build 092 (WfM 2.0) Press Ctrl+S to enter the Setup Menu.                       Initializing Intel(R) Boot Agent GE v1.5.72 Press Ctrl+S to enter the Setup Menu..  \x80 \x80  Intel(R) Boot Agent XE v2.1.60 Copyright (C) 1997-2009, Intel Corporation  CLIENT MAC ADDR: 0C C4 7A 8F 47 E1 GUID: 00000000 0000 0000 0000 0CC47AD8DA66 DHCP./                    DHCP.| DHCP./ DHCP.- DHCP.\ DHCP.| DHCP./ CLIENT IP: 172.21.15.76 MASK: 255.255.240.0 DHCP IP: 172.21.0.10 GATEWAY IP: 172.21.15.254 PXE->EB: !PXE at 95D3:0070, entry point at 95D3:0106  UNDI code segment 95D3:3E70, data segment 8E94:73F0 (570-615kB)  UNDI device is PCI 03:00.1, type DIX+802.3  570kB free base memory after PXE unload iPXE initialising devices... iPXE initialising devices...ok iPXE 1.20.1+ (g4bd0) -- Open Source Network Boot Firmware -- http://ipxe.org Features: DNS FTP HTTP HTTPS iSCSI NFS TFTP VLAN AoE ELF MBOOT PXE bzImage Menu PXEXT Configuring (net0 0c:c4:7a:8f:47:e1)... Configuring (net0 0c:c4:7a:8f:47:e1).... Configuring (net0 0c:c4:7a:8f:47:e1)..... Configuring (net0 0c:c4:7a:8f:47:e1)...... ok .    [?25lUse the ^ and v keys to change the selection. Press 'e' to edit the selected item, or 'c' for a command prompt.   CentOS Stream (5.14.0-386.el9.x86_64) 9  CentOS Stream (5.14.0-361.el9.x86_64) 9  CentOS Stream (5.14.0-239.el9.x86_64) 9  CentOS Stream (0-rescue-15949e3c88704300882482eddfd7baae) 9                The selected entry will be started automatically in 5s.           Use the * and * keys to change the selection.  Press 'e' to edit the selected item, or 'c' for a command prompt.  The selected entry will be started automatically in 5s.   CentOS Stream (5.14.0-386.el9.x86_64) 9   CentOS Stream (5.14.0-361.el9.x86_64) 9  CentOS Stream (5.14.0-239.el9.x86_64) 9  CentOS Stream (0-rescue-15949e3c88704300882482eddfd7baae) 9  The selected entry will be started automatically in 4s.  The selected entry will be started automatically in 4s.  The selected entry will be started automatically in 3s.  The selected entry will be started automatically in 3s.  The selected entry will be started automatically in 2s.  The selected entry will be started automatically in 2s.  The selected entry will be started automatically in 1s.  The selected entry will be started automatically in 1s.  The selected entry will be started automatically in 0s. [?25h       [ 0.000000] Linux version 5.14.0-386.el9.x86_64 (mockbuild@x86-05.stream.rdu2.redhat.com) (gcc (GCC) 11.4.1 20230605 (Red Hat 11.4.1-2), GNU ld version 2.35.2-42.el9) #1 SMP PREEMPT_DYNAMIC Wed Nov 15 10:38:22 UTC 2023 [ 0.000000] The list of certified hardware and cloud instances for Red Hat Enterprise Linux 9 can be viewed at the Red Hat Ecosystem Catalog, https://catalog.redhat.com. [ 0.000000] Command line: BOOT_IMAGE=(hd0,msdos1)/boot/vmlinuz-5.14.0-386.el9.x86_64 root=UUID=0b261d40-7844-4e73-a652-6bb884dfb0a5 ro crashkernel=1G-4G:192M,4G-64G:256M,64G-:512M console=ttyS1,115200 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000008abff] usable [ 0.000000] BIOS-e820: [mem 0x00000000000e0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x0000000078ec5fff] usable [ 0.000000] BIOS-e820: [mem 0x0000000078f78000-0x00000000798a7fff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000798a8000-0x0000000079d44fff] ACPI NVS [ 0.000000] BIOS-e820: [mem 0x0000000079d45000-0x000000008fffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed44fff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000ff000000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000087fffffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 3.0 present. [ 0.000000] DMI: Supermicro SYS-5018R-WR/X10SRW-F, BIOS 2.0a 08/02/2016 [ 0.000000] tsc: Fast TSC calibration using PIT [ 0.000000] tsc: Detected 3499.911 MHz processor [ 0.000489] last_pfn = 0x880000 max_arch_pfn = 0x400000000 [ 0.000493] MTRR map: 5 entries (3 fixed + 2 variable; max 23), built from 10 variable MTRRs [ 0.000495] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000821] last_pfn = 0x78ec6 max_arch_pfn = 0x400000000 [ 0.006521] found SMP MP-table at [mem 0x000fcd40-0x000fcd4f] [ 0.006531] Using GB pages for direct mapping [ 0.006657] RAMDISK: [mem 0x3436a000-0x361acfff] [ 0.006659] ACPI: Early table checksum verification disabled [ 0.006661] ACPI: RSDP 0x00000000000F05B0 000024 (v02 SUPERM) [ 0.006664] ACPI: XSDT 0x00000000798A80A0 0000BC (v01 01072009 AMI 00010013) [ 0.006668] ACPI: FACP 0x00000000798DB838 00010C (v05 SUPERM SMCI--MB 01072009 AMI 00010013) [ 0.006672] ACPI: DSDT 0x00000000798A81F0 033641 (v02 SUPERM SMCI--MB 01072009 INTL 20091013) [ 0.006674] ACPI: FACS 0x0000000079D43F80 000040 [ 0.006676] ACPI: APIC 0x00000000798DB948 0000C8 (v03 SUPERM SMCI--MB 01072009 AMI 00010013) [ 0.006678] ACPI: FPDT 0x00000000798DBA10 000044 (v01 SUPERM SMCI--MB 01072009 AMI 00010013) [ 0.006680] ACPI: FIDT 0x00000000798DBA58 00009C (v01 SUPERM SMCI--MB 01072009 AMI 00010013) [ 0.006682] ACPI: SPMI 0x00000000798DBAF8 000040 (v05 SUPERM SMCI--MB 00000000 AMI. 00000000) [ 0.006684] ACPI: MCFG 0x00000000798DBB38 00003C (v01 SUPERM SMCI--MB 01072009 MSFT 00000097) [ 0.006687] ACPI: UEFI 0x00000000798DBB78 000042 (v01 SUPERM SMCI--MB 01072009 00000000) [ 0.006689] ACPI: HPET 0x00000000798DBBC0 000038 (v01 SUPERM SMCI--MB 00000001 INTL 20091013) [ 0.006691] ACPI: WDDT 0x00000000798DBBF8 000040 (v01 SUPERM SMCI--MB 00000000 INTL 20091013) [ 0.006693] ACPI: SSDT 0x00000000798DBC38 01717F (v02 SUPERM PmMgt 00000001 INTL 20120913) [ 0.006695] ACPI: NITR 0x00000000798F2DB8 000071 (v02 SUPERM SMCI--MB 00000001 INTL 20091013) [ 0.006697] ACPI: SSDT 0x00000000798F2E30 00264C (v02 SUPERM SpsNm 00000002 INTL 20120913) [ 0.006699] ACPI: SSDT 0x00000000798F5480 000064 (v02 SUPERM SpsNvs 00000002 INTL 20120913) [ 0.006701] ACPI: PRAD 0x00000000798F54E8 000102 (v02 SUPERM SMCI--MB 00000002 INTL 20120913) [ 0.006703] ACPI: DMAR 0x00000000798F55F0 0000BC (v01 SUPERM SMCI--MB 00000001 INTL 20091013) [ 0.006705] ACPI: HEST 0x00000000798F56B0 00027C (v01 SUPERM SMCI--MB 00000001 INTL 00000001) [ 0.006707] ACPI: BERT 0x00000000798F5930 000030 (v01 SUPERM SMCI--MB 00000001 INTL 00000001) [ 0.006709] ACPI: ERST 0x00000000798F5960 000230 (v01 SUPERM SMCI--MB 00000001 INTL 00000001) [ 0.006711] ACPI: EINJ 0x00000000798F5B90 000130 (v01 SUPERM SMCI--MB 00000001 INTL 00000001) [ 0.006713] ACPI: Reserving FACP table memory at [mem 0x798db838-0x798db943] [ 0.006714] ACPI: Reserving DSDT table memory at [mem 0x798a81f0-0x798db830] [ 0.006715] ACPI: Reserving FACS table memory at [mem 0x79d43f80-0x79d43fbf] [ 0.006715] ACPI: Reserving APIC table memory at [mem 0x798db948-0x798dba0f] [ 0.006716] ACPI: Reserving FPDT table memory at [mem 0x798dba10-0x798dba53] [ 0.006716] ACPI: Reserving FIDT table memory at [mem 0x798dba58-0x798dbaf3] [ 0.006717] ACPI: Reserving SPMI table memory at [mem 0x798dbaf8-0x798dbb37] [ 0.006718] ACPI: Reserving MCFG table memory at [mem 0x798dbb38-0x798dbb73] [ 0.006718] ACPI: Reserving UEFI table memory at [mem 0x798dbb78-0x798dbbb9] [ 0.006719] ACPI: Reserving HPET table memory at [mem 0x798dbbc0-0x798dbbf7] [ 0.006719] ACPI: Reserving WDDT table memory at [mem 0x798dbbf8-0x798dbc37] [ 0.006720] ACPI: Reserving SSDT table memory at [mem 0x798dbc38-0x798f2db6] [ 0.006721] ACPI: Reserving NITR table memory at [mem 0x798f2db8-0x798f2e28] [ 0.006721] ACPI: Reserving SSDT table memory at [mem 0x798f2e30-0x798f547b] [ 0.006722] ACPI: Reserving SSDT table memory at [mem 0x798f5480-0x798f54e3] [ 0.006722] ACPI: Reserving PRAD table memory at [mem 0x798f54e8-0x798f55e9] [ 0.006723] ACPI: Reserving DMAR table memory at [mem 0x798f55f0-0x798f56ab] [ 0.006724] ACPI: Reserving HEST table memory at [mem 0x798f56b0-0x798f592b] [ 0.006724] ACPI: Reserving BERT table memory at [mem 0x798f5930-0x798f595f] [ 0.006725] ACPI: Reserving ERST table memory at [mem 0x798f5960-0x798f5b8f] [ 0.006726] ACPI: Reserving EINJ table memory at [mem 0x798f5b90-0x798f5cbf] [ 0.006769] No NUMA configuration found [ 0.006770] Faking a node at [mem 0x0000000000000000-0x000000087fffffff] [ 0.006778] NODE_DATA(0) allocated [mem 0x87ffd5000-0x87fffffff] [ 0.006975] Reserving 256MB of memory at 1664MB for crashkernel (System RAM: 32654MB) [ 0.007010] Zone ranges: [ 0.007011] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.007012] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.007013] Normal [mem 0x0000000100000000-0x000000087fffffff] [ 0.007015] Device empty [ 0.007015] Movable zone start for each node [ 0.007018] Early memory node ranges [ 0.007018] node 0: [mem 0x0000000000001000-0x0000000000089fff] [ 0.007019] node 0: [mem 0x0000000000100000-0x0000000078ec5fff] [ 0.007020] node 0: [mem 0x0000000100000000-0x000000087fffffff] [ 0.007023] Initmem setup node 0 [mem 0x0000000000001000-0x000000087fffffff] [ 0.007027] On node 0, zone DMA: 1 pages in unavailable ranges [ 0.007064] On node 0, zone DMA: 118 pages in unavailable ranges [ 0.011664] On node 0, zone Normal: 28986 pages in unavailable ranges [ 0.011775] ACPI: PM-Timer IO Port: 0x408 [ 0.011781] ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1]) [ 0.011782] ACPI: LAPIC_NMI (acpi_id[0x02] high edge lint[0x1]) [ 0.011783] ACPI: LAPIC_NMI (acpi_id[0x04] high edge lint[0x1]) [ 0.011783] ACPI: LAPIC_NMI (acpi_id[0x06] high edge lint[0x1]) [ 0.011784] ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1]) [ 0.011785] ACPI: LAPIC_NMI (acpi_id[0x03] high edge lint[0x1]) [ 0.011785] ACPI: LAPIC_NMI (acpi_id[0x05] high edge lint[0x1]) [ 0.011786] ACPI: LAPIC_NMI (acpi_id[0x07] high edge lint[0x1]) [ 0.011794] IOAPIC[0]: apic_id 1, version 32, address 0xfec00000, GSI 0-23 [ 0.011798] IOAPIC[1]: apic_id 2, version 32, address 0xfec01000, GSI 24-47 [ 0.011800] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.011802] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.011805] ACPI: Using ACPI (MADT) for SMP configuration information [ 0.011805] ACPI: HPET id: 0x8086a701 base: 0xfed00000 [ 0.011810] [Firmware Bug]: TSC_DEADLINE disabled due to Errata; please update microcode to version: 0xb000020 (or later) [ 0.011811] smpboot: Allowing 8 CPUs, 0 hotplug CPUs [ 0.011820] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.011822] PM: hibernation: Registered nosave memory: [mem 0x0008a000-0x000dffff] [ 0.011822] PM: hibernation: Registered nosave memory: [mem 0x000e0000-0x000fffff] [ 0.011823] PM: hibernation: Registered nosave memory: [mem 0x78ec6000-0x78f77fff] [ 0.011824] PM: hibernation: Registered nosave memory: [mem 0x78f78000-0x798a7fff] [ 0.011825] PM: hibernation: Registered nosave memory: [mem 0x798a8000-0x79d44fff] [ 0.011825] PM: hibernation: Registered nosave memory: [mem 0x79d45000-0x8fffffff] [ 0.011826] PM: hibernation: Registered nosave memory: [mem 0x90000000-0xfed1bfff] [ 0.011826] PM: hibernation: Registered nosave memory: [mem 0xfed1c000-0xfed44fff] [ 0.011827] PM: hibernation: Registered nosave memory: [mem 0xfed45000-0xfeffffff] [ 0.011827] PM: hibernation: Registered nosave memory: [mem 0xff000000-0xffffffff] [ 0.011829] [mem 0x90000000-0xfed1bfff] available for PCI devices [ 0.011830] Booting paravirtualized kernel on bare hardware [ 0.011832] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.016162] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:8 nr_cpu_ids:8 nr_node_ids:1 [ 0.016377] percpu: Embedded 62 pages/cpu s217088 r8192 d28672 u262144 [ 0.016404] Fallback order for Node 0: 0 [ 0.016406] Built 1 zonelists, mobility grouping on. Total pages: 8228746 [ 0.016407] Policy zone: Normal [ 0.016408] Kernel command line: BOOT_IMAGE=(hd0,msdos1)/boot/vmlinuz-5.14.0-386.el9.x86_64 root=UUID=0b261d40-7844-4e73-a652-6bb884dfb0a5 ro crashkernel=1G-4G:192M,4G-64G:256M,64G-:512M console=ttyS1,115200 [ 0.016458] Unknown kernel command line parameters "BOOT_IMAGE=(hd0,msdos1)/boot/vmlinuz-5.14.0-386.el9.x86_64", will be passed to user space. [ 0.016474] random: crng init done [ 0.018656] Dentry cache hash table entries: 4194304 (order: 13, 33554432 bytes, linear) [ 0.019796] Inode-cache hash table entries: 2097152 (order: 12, 16777216 bytes, linear) [ 0.020016] mem auto-init: stack:off, heap alloc:off, heap free:off [ 0.020023] software IO TLB: area num 8. [ 0.045494] Memory: 1752328K/33438012K available (16384K kernel code, 5606K rwdata, 11524K rodata, 3868K init, 6044K bss, 986992K reserved, 0K cma-reserved) [ 0.045597] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=8, Nodes=1 [ 0.045605] Kernel/User page tables isolation: enabled [ 0.045619] ftrace: allocating 46498 entries in 182 pages [ 0.063390] ftrace: allocated 182 pages with 5 groups [ 0.063890] Dynamic Preempt: voluntary [ 0.063918] rcu: Preemptible hierarchical RCU implementation. [ 0.063919] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=8. [ 0.063920] Trampoline variant of Tasks RCU enabled. [ 0.063920] Rude variant of Tasks RCU enabled. [ 0.063921] Tracing variant of Tasks RCU enabled. [ 0.063921] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.063922] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=8 [ 0.067248] NR_IRQS: 524544, nr_irqs: 896, preallocated irqs: 16 [ 0.067410] rcu: srcu_init: Setting srcu_struct sizes based on contention. [ 0.067500] kfence: initialized - using 2097152 bytes for 255 objects at 0x00000000cef9ed2c-0x0000000036291727 [ 0.072775] Console: colour VGA+ 80x25 [ 1.145682] printk: console [ttyS1] enabled [ 1.149869] ACPI: Core revision 20221020 [ 1.153957] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 133484882848 ns [ 1.163085] APIC: Switch to symmetric I/O mode setup [ 1.168056] DMAR: Host address width 46 [ 1.171900] DMAR: DRHD base: 0x000000fbffc000 flags: 0x1 [ 1.177214] DMAR: dmar0: reg_base_addr fbffc000 ver 1:0 cap 8d2078c106f0466 ecap f020de [ 1.185209] DMAR: RMRR base: 0x0000007ba6e000 end: 0x0000007ba7dfff [ 1.191477] DMAR: ATSR flags: 0x0 [ 1.194796] DMAR: RHSA base: 0x000000fbffc000 proximity domain: 0x0 [ 1.201063] DMAR-IR: IOAPIC id 1 under DRHD base 0xfbffc000 IOMMU 0 [ 1.207416] DMAR-IR: IOAPIC id 2 under DRHD base 0xfbffc000 IOMMU 0 [ 1.213769] DMAR-IR: HPET id 0 under DRHD base 0xfbffc000 [ 1.219167] DMAR-IR: x2apic is disabled because BIOS sets x2apic opt out bit. [ 1.219168] DMAR-IR: Use 'intremap=no_x2apic_optout' to override the BIOS setting. [ 1.234190] DMAR-IR: Enabled IRQ remapping in xapic mode [ 1.239494] x2apic: IRQ remapping doesn't support X2APIC mode [ 1.245711] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 1.256087] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x3272fcf80f8, max_idle_ns: 440795261698 ns [ 1.266599] Calibrating delay loop (skipped), value calculated using timer frequency.. 6999.82 BogoMIPS (lpj=3499911) [ 1.267598] pid_max: default: 32768 minimum: 301 [ 1.268617] LSM: initializing lsm=lockdown,capability,yama,integrity,selinux,bpf [ 1.269603] Yama: becoming mindful. [ 1.270601] SELinux: Initializing. [ 1.271615] LSM support for eBPF active [ 1.273640] Mount-cache hash table entries: 65536 (order: 7, 524288 bytes, linear) [ 1.274652] Mountpoint-cache hash table entries: 65536 (order: 7, 524288 bytes, linear) [ 1.275760] CPU0: Thermal monitoring enabled (TM1) [ 1.276654] process: using mwait in idle threads [ 1.277599] Last level iTLB entries: 4KB 64, 2MB 8, 4MB 8 [ 1.278598] Last level dTLB entries: 4KB 64, 2MB 0, 4MB 0, 1GB 4 [ 1.279600] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization [ 1.280599] Spectre V2 : Mitigation: Retpolines [ 1.281598] Spectre V2 : Spectre v2 / SpectreRSB mitigation: Filling RSB on context switch [ 1.282598] Spectre V2 : Spectre v2 / SpectreRSB : Filling RSB on VMEXIT [ 1.283598] Speculative Store Bypass: Vulnerable [ 1.284604] MDS: Vulnerable: Clear CPU buffers attempted, no microcode [ 1.285598] TAA: Vulnerable: Clear CPU buffers attempted, no microcode [ 1.286598] MMIO Stale Data: Vulnerable: Clear CPU buffers attempted, no microcode [ 1.298858] Freeing SMP alternatives memory: 36K [ 1.300645] smpboot: Estimated ratio of average max frequency by base frequency (times 1024): 1053 [ 1.404603] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz (family: 0x6, model: 0x4f, stepping: 0x1) [ 1.406695] cblist_init_generic: Setting adjustable number of callback queues. [ 1.407598] cblist_init_generic: Setting shift to 3 and lim to 1. [ 1.408609] cblist_init_generic: Setting shift to 3 and lim to 1. [ 1.409608] cblist_init_generic: Setting shift to 3 and lim to 1. [ 1.410606] Performance Events: PEBS fmt2+, Broadwell events, 16-deep LBR, full-width counters, Intel PMU driver. [ 1.411598] ... version: 3 [ 1.412598] ... bit width: 48 [ 1.413598] ... generic registers: 4 [ 1.414598] ... value mask: 0000ffffffffffff [ 1.415598] ... max period: 00007fffffffffff [ 1.416598] ... fixed-purpose events: 3 [ 1.417598] ... event mask: 000000070000000f [ 1.418707] rcu: Hierarchical SRCU implementation. [ 1.418708] rcu: Max phase no-delay instances is 400. [ 1.419598] printk: console [ttyS1] printing thread started [ 1.423834] NMI watchdog: Enabled. Permanently consumes one hw-PMU counter. [ 1.423900] smp: Bringing up secondary CPUs ... [ 1.423967] x86: Booting SMP configuration: [ 1.423968] .... node #0, CPUs: #1 #2 #3 #4 [ 1.429699] MDS CPU bug present and SMT on, data leak possible. See https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/mds.html for more details. [ 1.429699] TAA CPU bug present and SMT on, data leak possible. See https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/tsx_async_abort.html for more details. [ 1.429699] MMIO Stale Data CPU bug present and SMT on, data leak possible. See https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/processor_mmio_stale_data.html for more details. [ 1.429699] #5 #6 #7 [ 1.436130] smp: Brought up 1 node, 8 CPUs [ 1.436130] smpboot: Max logical packages: 1 [ 1.436130] smpboot: Total of 8 processors activated (55998.57 BogoMIPS) [ 1.478909] node 0 deferred pages initialised in 39ms [ 1.480723] devtmpfs: initialized [ 1.480723] x86/mm: Memory block size: 128MB [ 1.481872] ACPI: PM: Registering ACPI NVS region [mem 0x798a8000-0x79d44fff] (4837376 bytes) [ 1.481872] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 1.481872] futex hash table entries: 2048 (order: 5, 131072 bytes, linear) [ 1.481872] pinctrl core: initialized pinctrl subsystem [ 1.500967] NET: Registered PF_NETLINK/PF_ROUTE protocol family [ 1.501051] DMA: preallocated 4096 KiB GFP_KERNEL pool for atomic allocations [ 1.501061] DMA: preallocated 4096 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 1.501068] DMA: preallocated 4096 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 1.501076] audit: initializing netlink subsys (disabled) [ 1.501610] audit: type=2000 audit(1700886309.255:1): state=initialized audit_enabled=0 res=1 [ 1.501664] thermal_sys: Registered thermal governor 'fair_share' [ 1.501665] thermal_sys: Registered thermal governor 'step_wise' [ 1.501666] thermal_sys: Registered thermal governor 'user_space' [ 1.510600] cpuidle: using governor menu [ 1.517626] ACPI FADT declares the system doesn't support PCIe ASPM, so disable it [ 1.517627] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 1.517673] PCI: MMCONFIG for domain 0000 [bus 00-ff] at [mem 0x80000000-0x8fffffff] (base 0x80000000) [ 1.517676] PCI: MMCONFIG at [mem 0x80000000-0x8fffffff] reserved as E820 entry [ 1.517684] PCI: Using configuration type 1 for base access [ 1.528625] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible. [ 1.528629] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages [ 1.528629] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page [ 1.528629] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages [ 1.528629] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page [ 1.536634] cryptd: max_cpu_qlen set to 1000 [ 1.565648] ACPI: Added _OSI(Module Device) [ 1.565649] ACPI: Added _OSI(Processor Device) [ 1.565649] ACPI: Added _OSI(3.0 _SCP Extensions) [ 1.565650] ACPI: Added _OSI(Processor Aggregator Device) [ 1.565650] ACPI: Added _OSI(Linux-Dell-Video) [ 1.565651] ACPI: Added _OSI(Linux-Lenovo-NV-HDMI-Audio) [ 1.565653] ACPI: Added _OSI(Linux-HPI-Hybrid-Graphics) [ 1.595006] ACPI: 4 ACPI AML tables successfully acquired and loaded [ 1.597162] ACPI: [Firmware Bug]: BIOS _OSI(Linux) query ignored [ 1.599690] ACPI: Dynamic OEM Table Load: [ 1.613266] ACPI: Interpreter enabled [ 1.613275] ACPI: PM: (supports S0 S4 S5) [ 1.613276] ACPI: Using IOAPIC for interrupt routing [ 1.614723] HEST: Enabling Firmware First mode for corrected errors. [ 1.656447] HEST: Table parsing has been initialized. [ 1.656449] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 1.656450] PCI: Using E820 reservations for host bridge windows [ 1.656890] ACPI: Enabled 5 GPEs in block 00 to 3F [ 1.677246] ACPI: PCI Root Bridge [UNC0] (domain 0000 [bus ff]) [ 1.677250] acpi PNP0A03:03: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI EDR HPX-Type3] [ 1.678354] acpi PNP0A03:03: _OSC: platform does not support [SHPCHotplug AER LTR DPC] [ 1.678894] acpi PNP0A03:03: _OSC: OS now controls [PCIeHotplug PME PCIeCapability] [ 1.678895] acpi PNP0A03:03: FADT indicates ASPM is unsupported, using BIOS configuration [ 1.678922] PCI host bridge to bus 0000:ff [ 1.678924] pci_bus 0000:ff: root bus resource [bus ff] [ 1.678934] pci 0000:ff:0b.0: [8086:6f81] type 00 class 0x088000 [ 1.678976] pci 0000:ff:0b.1: [8086:6f36] type 00 class 0x110100 [ 1.679008] pci 0000:ff:0b.2: [8086:6f37] type 00 class 0x110100 [ 1.679037] pci 0000:ff:0b.3: [8086:6f76] type 00 class 0x088000 [ 1.679066] pci 0000:ff:0c.0: [8086:6fe0] type 00 class 0x088000 [ 1.679094] pci 0000:ff:0c.1: [8086:6fe1] type 00 class 0x088000 [ 1.679122] pci 0000:ff:0c.2: [8086:6fe2] type 00 class 0x088000 [ 1.679151] pci 0000:ff:0c.3: [8086:6fe3] type 00 class 0x088000 [ 1.679181] pci 0000:ff:0f.0: [8086:6ff8] type 00 class 0x088000 [ 1.679208] pci 0000:ff:0f.1: [8086:6ff9] type 00 class 0x088000 [ 1.679236] pci 0000:ff:0f.4: [8086:6ffc] type 00 class 0x088000 [ 1.679264] pci 0000:ff:0f.5: [8086:6ffd] type 00 class 0x088000 [ 1.679292] pci 0000:ff:0f.6: [8086:6ffe] type 00 class 0x088000 [ 1.679321] pci 0000:ff:10.0: [8086:6f1d] type 00 class 0x088000 [ 1.679349] pci 0000:ff:10.1: [8086:6f34] type 00 class 0x110100 [ 1.679378] pci 0000:ff:10.5: [8086:6f1e] type 00 class 0x088000 [ 1.679406] pci 0000:ff:10.6: [8086:6f7d] type 00 class 0x110100 [ 1.679433] pci 0000:ff:10.7: [8086:6f1f] type 00 class 0x088000 [ 1.679461] pci 0000:ff:12.0: [8086:6fa0] type 00 class 0x088000 [ 1.679481] pci 0000:ff:12.1: [8086:6f30] type 00 class 0x110100 [ 1.679515] pci 0000:ff:13.0: [8086:6fa8] type 00 class 0x088000 [ 1.679576] pci 0000:ff:13.1: [8086:6f71] type 00 class 0x088000 [ 1.679623] pci 0000:ff:13.2: [8086:6faa] type 00 class 0x088000 [ 1.679663] pci 0000:ff:13.3: [8086:6fab] type 00 class 0x088000 [ 1.679703] pci 0000:ff:13.4: [8086:6fac] type 00 class 0x088000 [ 1.679743] pci 0000:ff:13.5: [8086:6fad] type 00 class 0x088000 [ 1.679781] pci 0000:ff:13.6: [8086:6fae] type 00 class 0x088000 [ 1.679811] pci 0000:ff:13.7: [8086:6faf] type 00 class 0x088000 [ 1.679842] pci 0000:ff:14.0: [8086:6fb0] type 00 class 0x088000 [ 1.679885] pci 0000:ff:14.1: [8086:6fb1] type 00 class 0x088000 [ 1.679925] pci 0000:ff:14.2: [8086:6fb2] type 00 class 0x088000 [ 1.679965] pci 0000:ff:14.3: [8086:6fb3] type 00 class 0x088000 [ 1.680005] pci 0000:ff:14.4: [8086:6fbc] type 00 class 0x088000 [ 1.680035] pci 0000:ff:14.5: [8086:6fbd] type 00 class 0x088000 [ 1.680065] pci 0000:ff:14.6: [8086:6fbe] type 00 class 0x088000 [ 1.680095] pci 0000:ff:14.7: [8086:6fbf] type 00 class 0x088000 [ 1.680126] pci 0000:ff:15.0: [8086:6fb4] type 00 class 0x088000 [ 1.680166] pci 0000:ff:15.1: [8086:6fb5] type 00 class 0x088000 [ 1.680206] pci 0000:ff:15.2: [8086:6fb6] type 00 class 0x088000 [ 1.680248] pci 0000:ff:15.3: [8086:6fb7] type 00 class 0x088000 [ 1.680289] pci 0000:ff:16.0: [8086:6f68] type 00 class 0x088000 [ 1.680324] pci 0000:ff:16.6: [8086:6f6e] type 00 class 0x088000 [ 1.680353] pci 0000:ff:16.7: [8086:6f6f] type 00 class 0x088000 [ 1.680384] pci 0000:ff:17.0: [8086:6fd0] type 00 class 0x088000 [ 1.680417] pci 0000:ff:17.4: [8086:6fb8] type 00 class 0x088000 [ 1.680447] pci 0000:ff:17.5: [8086:6fb9] type 00 class 0x088000 [ 1.680477] pci 0000:ff:17.6: [8086:6fba] type 00 class 0x088000 [ 1.680508] pci 0000:ff:17.7: [8086:6fbb] type 00 class 0x088000 [ 1.680543] pci 0000:ff:1e.0: [8086:6f98] type 00 class 0x088000 [ 1.680572] pci 0000:ff:1e.1: [8086:6f99] type 00 class 0x088000 [ 1.680603] pci 0000:ff:1e.2: [8086:6f9a] type 00 class 0x088000 [ 1.680633] pci 0000:ff:1e.3: [8086:6fc0] type 00 class 0x088000 [ 1.680654] pci 0000:ff:1e.4: [8086:6f9c] type 00 class 0x088000 [ 1.680684] pci 0000:ff:1f.0: [8086:6f88] type 00 cla 0x088000 [ 1.680714] pci 0000:ff:1f.2: [8086:6f8a] type 00 class 0x088000 [ 1.682895] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-fe]) [ 1.682898] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI EDR HPX-Type3] [ 1.683029] acpi PNP0A08:00: _OSC: platform does not support [SHPCHotplug AER LTR DPC] [ 1.683269] acpi PNP0A08:00: _OSC: OS now controls [PCIeHotplug PME PCIeCapability] [ 1.683269] acpi PNP0A08:00: FADT indicates ASPM is unsupported, using BIOS configuration [ 1.683666] PCI host bridge to bus 0000:00 [ 1.683668] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 1.683669] pci_bus 0000:00: root bus resource [io 0x1000-0xffff window] [ 1.683670] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 1.683671] pci_bus 0000:00: root bus resource [mem 0xfedb0000-0xfedb000f window] [ 1.683672] pci_bus 0000:00: root bus resource [mem 0xfedc0000-0xfedc000f window] [ 1.683672] pci_bus 0000:00: root bus resource [mem 0x90000000-0xfbffbfff window] [ 1.683673] pci_bus 0000:00: root bus resource [bus 00-fe] [ 1.683692] pci 0000:00:00.0: [8086:6f00] type 00 class 0x060000 [ 1.683791] pci 0000:00:01.0: [8086:6f02] type 01 class 0x060400 [ 1.683834] pci 0000:00:01.0: PME# supported from D0 D3hot D3cold [ 1.683923] pci 0000:00:02.0: [8086:6f04] type 01 class 0x060400 [ 1.683965] pci 0000:00:02.0: PME# supported from D0 D3hot D3cold [ 1.684050] pci 0000:00:03.0: [8086:6f08] type 01 class 0x060400 [ 1.684091] pci 0000:00:03.0: PME# supported from D0 D3hot D3cold [ 1.684171] pci 0000:00:04.0: [8086:6f20] type 00 class 0x088000 [ 1.684179] pci 0000:00:04.0: reg 0x10: [mem 0xfb42c000-0xfb42ffff 64bit] [ 1.684253] pci 0000:00:04.1: [8086:6f21] type 00 class 0x088000 [ 1.684260] pci 0000:00:04.1: reg 0x10: [mem 0xfb428000-0xfb42bfff 64bit] [ 1.684332] pci 0000:00:04.2: [8086:6f22] type 00 class 0x088000 [ 1.684339] pci 0000:00:04.2: reg 0x10: [mem 0xfb424000-0xfb427fff 64bit] [ 1.684411] pci 0000:00:04.3: [8086:6f23] type 00 class 0x088000 [ 1.684418] pci 0000:00:04.3: reg 0x10: [mem 0xfb420000-0xfb423fff 64bit] [ 1.684490] pci 0000:00:04.4: [8086:6f24] type 00 class 0x088000 [ 1.684497] pci 0000:00:04.4: reg 0x10: [mem 0xfb41c000-0xfb41ffff 64bit] [ 1.684568] pci 0000:00:04.5: [8086:6f25] type 00 class 0x088000 [ 1.684575] pci 0000:00:04.5: reg 0x10: [mem 0xfb418000-0xfb41bfff 64bit] [ 1.684652] pci 0000:00:04.6: [8086:6f26] type 00 class 0x088000 [ 1.684660] pci 0000:00:04.6: reg 0x10: [mem 0xfb414000-0xfb417fff 64bit] [ 1.684737] pci 0000:00:04.7: [8086:6f27] type 00 class 0x088000 [ 1.684744] pci 0000:00:04.7: reg 0x10: [mem 0xfb410000-0xfb413fff 64bit] [ 1.684813] pci 0000:00:05.0: [8086:6f28] type 00 class 0x088000 [ 1.684873] pci 0000:00:05.1: [8086:6f29] type 00 class 0x088000 [ 1.684953] pci 0000:00:05.2: [8086:6f2a] type 00 class 0x088000 [ 1.685007] pci 0000:00:05.4: [8086:6f2c] type 00 class 0x080020 [ 1.685013] pci 0000:00:05.4: reg 0x10: [mem 0xfb439000-0xfb439fff] [ 1.685090] pci 0000:00:11.0: [8086:8d7c] type 00 class 0xff0000 [ 1.685327] pci 0000:00:11.4: [8086:8d62] type 00 class 0x010601 [ 1.685346] pci 0000:00:11.4: reg 0x10: [io 0xf110-0xf117] [ 1.685357] pci 0000:00:11.4: reg 0x14: [io 0xf100-0xf103] [ 1.685368] pci 0000:00:11.4: reg 0x18: [io 0xf0f0-0xf0f7] [ 1.685379] pci 0000:00:11.4: reg 0x1c: [io 0xf0e0-0xf0e3] [ 1.685391] pci 0000:00:11.4: reg 0x20: [io 0xf020-0xf03f] [ 1.685402] pci 0000:00:11.4: reg 0x24: [mem 0xfb438000-0xfb4387ff] [ 1.685459] pci 0000:00:11.4: PME# supported from D3hot [ 1.685536] pci 0000:00:14.0: [8086:8d31] type 00 class 0x0c0330 [ 1.685560] pci 0000:00:14.0: reg 0x10: [mem 0xfb400000-0xfb40ffff 64bit] [ 1.685646] pci 0000:00:14.0: PME# supported from D3hot D3cold [ 1.685731] pci 0000:00:16.0: [8086:8d3a] type 00 class 0x078000 [ 1.685758] pci 0000:00:16.0: reg 0x10: [mem 0xfb437000-0xfb43700f 64bit] [ 1.685845] pci 0000:00:16.0: PME# supported from D0 D3hot D3cold [ 1.685909] pci 0000:00:16.1: [8086:8d3b] type 00 class 0x078000 [ 1.685932] pci 0000:00:16.1: reg 0x10: [mem 0xfb436000-0xfb43600f 64bit] [ 1.686019] pci 0000:00:16.1: PME# supported from D0 D3hot D3cold [ 1.686101] pci 0000:00:1a.0: [8086:8d2d] type 00 class 0x0c0320 [ 1.686123] pci 0000:00:1a.0: reg 0x10: [mem 0xfb434000-0xfb4343ff] [ 1.686237] pci 0000:00:1a.0: PME# supported from D0 D3hot D3cold [ 1.686315] pci 0000:00:1c.0: [8086:8d10] type 01 class 0x060400 [ 1.686431] pci 0000:00:1c.0: PME# supported from D0 D3hot D3cold [ 1.686528] pci 0000:00:1c.2: [8086:8d14] type 01 class 0x060400 [ 1.686650] pci 0000:00:1c.2: PME# supported from D0 D3hot D3cold [ 1.686748] pci 0000:00:1c.4: [8086:8d18] type 01 class 0x060400 [ 1.686867] pci 0000:00:1c.4: PME# supported from D0 D3hot D3cold [ 1.686971] pci 0000:00:1d.0: [8086:8d26] type 00 class 0x0c0320 [ 1.686993] pci 0000:00:1d.0: reg 0x10: [mem 0xfb43 pci 0000:00:1d.cold [ 1.687186] pci 0000:00:1f.0: [8086:8d44] type 00 class 0x060100 [ 1.687386] pci 0000:00:1f.2: [8086:8d02] type 00 class 0x010601 [ 1.687403] pci 0000:00:1f.2: reg 0x10: [io 0xf070-0xf077] [ 1.687415] pci 0000:00:1f.2: reg 0x14: [io 0xf060-0xf063] [ 1.687425] pci 0000:00:1f.2: reg 0x18: [io 0xf050-0xf057] [ 1.687435] pci 0000:00:1f.2: reg 0x1c: [io 0xf040-0xf043] [ 1.687445] pci 0000:00:1f.2: reg 0x20: [io 0xf000-0xf01f] [ 1.687456] pci 0000:00:1f.2: reg 0x24: [mem 0xfb432000-0xfb4327ff] [ 1.687509] pci 0000:00:1f.2: PME# supported from D3hot [ 1.687568] pci 0000:00:1f.3: [8086:8d22] type 00 class 0x0c0500 [ 1.687592] pci 0000:00:1f.3: reg 0x10: [mem 0xfb431000-0xfb4310ff 64bit] [ 1.687623] pci 0000:00:1f.3: reg 0x20: [io 0x0580-0x059f] [ 1.687785] acpiphp: Slot [1] registered [ 1.687796] pci 0000:00:01.0: PCI 1.687897] pci 00 class 0x010800: reg 0x10: [mem 0xfb310000-0xfb313fff 64bit] [ 1.687922] pci 0000:02:00.0: reg 0x30: [mem 0xfb300000-0xfb30ffff pref] [ 1.687925] pci 0000:02:00.0: enabling Extended Tags [ 1.688026] pci 0000:00:02.0: PCI bridge to [bus 02] [ 1.688029] pci 0000:00:02.0: bridge window [mem 0xfb300000-0xfb3fffff] [ 1.688133] pci 0000:03:00.0: [8086:10fb] type 00 class 0x020000 [ 1.688143] pci 0000:03:00.0: reg 0x10: [mem 0xfbd80000-0xfbdfffff 64bit pref] [ 1.688148] pci 0000:03:00.0: reg 0x18: [io 0xe020-0xe03f] [ 1.688158] pci 0000:03:00.0: reg 0x20: [mem 0xfbe04000-0xfbe07fff 64bit pref] [ 1.688162] pci 0000:03:00.0: reg 0x30: [mem 0xfb280000-0xfb2fffff pref] [ 1.688198] pci 0000:03:00.0: PME# supported from D0 D3hot D3cold [ 1.688214] pci 0000:03:00.0: reg 0x184: [mem 0x00000000-0x00003fff 64bit pref] [ 1.688215] pci 0000:03:00.0: VF(n) BAR0 space: [mem 0x00000000-0x000fffff 64bit pref] (contains BAR0 for 64 VFs) [ 1.688224] pci 0000:03:00.0: reg 0x190: [mem 0x000000001.688225] pci 0000:03:00.0: VF(n) BAR3 space: [mem 0x00000000-0x000fffff 64bit pns BAR3 for 64 VFs) [ 1.688377] pci 0000:03:00.1: [8086:10fb] type 00 class 0x020000 [ 1.688386] pci 0000:03:00.1: reg 0x10: [mem 0xfbd00000-0xfbd7ffff 64bit pref] [ 1.688391] pci 0000:03:00.1: reg 0x18: [io 0xe000-0xe01f] [ 1.688401] pci 0000:03:00.1: reg 0x20: [mem 0xfbe00000-0xfbe03fff 64bit pref] [ 1.688405] pci 0000:03:00.1: reg 0x30: [mem 0xfb200000-0xfb27ffff pref] [ 1.688440] pci 0000:03:00.1: PME# supported from D0 D3hot D3cold [ 1.688454] pci 0000:03:00.1: reg 0x184: [mem 0x00000000-0x00003fff 64bit pref] [ 1.688455] pci 0000:03:00.1: space: [mem 0x00000000-0x000fffff 64bit pref] (contains BAR0 for 64 VFs) [ 1.688463] pci 0000:03:00.1: reg 0x190: [mem 0x00000000-0x00003fff 64bit pref] [ 1.688464] pci 0000:03:00.1: VF(n) BAR3 space: [mem 0x00000000-0x000fffff 64bit pref] (contains BAR3 for 64 VFs) [ 1.688605] pci 0000:00:03.0: PCI bridge to [bus 03] [ 1.688606] pci 0000:00:03.0: bridge window [io 0xe000-0xefff] [ 1.688608] pci 0000:00:03.0: bridge window [mem 0xfb200000-0xfb2fffff] [ 1.688610] pci 0000:00:03.0: bridge window [mem 0xfbd00000-0xfbefffff 64bit pref] [ 1.688655] pci 0000:00:1c.0: PCI bridge to [bus 04] [ 1.688747] pci 0000:05:00.0: [1a03:1150] type 01 class 0x060400 [ 1.688861] pci 0000:05:00.0: enabling Extended Tags [ 1.688993] pci 0000:05:00.0: supports D1 D2 [ 1.688993] pci 0000:05:00.0: PME# supported from D0 D1 D2 D3hot D3cold [ 1.689167] pci 0000:00:1c.2: PCI bridge to [bus 05-06] [ 1.689171] pci 0000:00:1c.2: bridge window [io 0xd000-0xdfff] [ 1.689174] pci 0000:00:1c.2: bridge window [mem 0xfa000000-0xfb0fffff] [ 1.689256] pci_bus 0000:06: extended config space not accessible [ 1.689289] pci 0000:06:00.0: [1a03:2000] type 00 class 0x030000 [ 1.689325] pci 0000:06:00.0: reg 0x10: [mem 0xfa000000-0xfaffffff] [ 1.689345] pci 0000:06:00.0: reg 0x14: [mem 0xfb000000-0xfb01ffff] [ 1.689364] pci 0000:06:00.0: reg 0x18: [io 0xd000-0xd07f] [ 1.689463] pci 0000:06:00.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff] [ 1.689533] pci 0000:06:00.0: supports D1 D2 [ 1.689534] pci 0000:06:00.0: PME# supported from D0 D1 D2 D3hot D3cold [ 1.689686] pci 0000:05:00.0: PCI bridge to [bus 06] [ 1.689696] pci 0000:05:00.0: bridge window [io 0xd000-0xdfff] [ 1.689702] pci 0000:05:00.0: bridge window [mem 0xfa000000-0xfb0fffff] [ 1.689830] pci 0000:07:00.0: [8086:1521] type 00 class 0x020000 [ 1.689862] pci 0000:07:00.0: reg 0x10: [mem 0xfb120000-0xfb13ffff] [ 1.689899] pci 0000:07:00.0: reg 0x18: [io 0xc020-0xc03f] [ 1.689917] pci 0000:07:00.0: reg 0x1c: [mem 0xfb144000-0xfb147fff] [ 1.690136] pci 0000:07:00.0: PME# supported from D0 D3hot D3cold [ 1.690201] pci 0000:07:00.0: reg 0x184: [mem 0x00000000-0x00003fff 64bit pref] [ 1.690202] pci 0000:07:00.0: VF(n) BAR0 space: [mem 0x00000000-0x0001ffff 64bit pref] (contains BAR0 for 8 VFs) [ 1.690238] pci 0000:07:00.0: reg 0x190: [mem 0x00000000-0x00003fff 64bit pref] [ 1.690238] pci 0000:07:00.0: VF(n) BAR3 space: [mem 0x00000000-0x0001ffff 64bit pref] (contains BAR3 for 8 VFs) [ 1.690569] pci 0000:07:00.1: [8086:1521] type 00 class 0x020000 [ 1.690602] pci 0000:07:00.1: reg 0x10: [mem 0xfb100000-0xfb11ffff] [ 1.690638] pci 0000:07:00.1: reg 0x18: [io 0xc000-0xc01f] [ 1.690656] pci 0000:07:00.1: reg 0x1c: [mem 0xfb140000-0xfb143fff] [ 1.690874] pci 0000:07:00.1: PME# supported from D0 D3hot D3cold [ 1.690935] pci 0000:07:00.1: reg 0x184: [mem 0x00000000-0x00003fff 64bit pref] [ 1.690936] pci 0000:07:00.1: VF(n) BAR0 space: [mem 0x00000000-0x0001ffff 64bit pref] (contains BAR0 for 8 VFs) [ 1.690969] pci 0000:07:00.1: reg 0x190: [mem 0x00000000-0x00003fff 64bit pref] [ 1.690970] pci 0000:07:00.1: VF(n) BAR3 space: [mem 0x00000000-0x0001ffff 64bit pref] (contains BAR3 for 8 VFs) [ 1.691268] pci 0000:00:1c.4: PCI bridge to [bus 07] [ 1.691271] pci 0000:00:1c.4: bridge window [io 0xc000-0xcfff] [ 1.691274] pci 0000:00:1c.4: bridge window [mem 0xfb100000-0xfb1fffff] [ 1.691280] pci 0000:00:1c.4: bridge has subordinate 07 but max busn 08 [ 1.691938] ACPI: PCI: Interrupt link LNKA configured for IRQ 11 [ 1.691968] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 1.691998] ACPI: PCI: Interrupt link LNKC configured for IRQ 5 [ 1.692029] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 1.692058] ACPI: PCI: Interrupt link LNKE configured for IRQ 0 [ 1.692058] ACPI: PCI: Interrupt link LNKE disabled [ 1.692088] ACPI: PCI: Interrupt link LNKF configured for IRQ 0 [ 1.692088] ACPI: PCI: Interrupt link LNKF disabled [ 1.692118] ACPI: PCI: Interrupt link LNKG configured for IRQ 0 [ 1.692118] ACPI: PCI: Interrupt link LNKG disabled [ 1.692147] ACPI: PCI: Interrupt link LNKH configured for IRQ 0 [ 1.692148] ACPI: PCI: Interrupt link LNKH disabled [ 1.692611] iommu: Default domain type: Translated [ 1.692611] iommu: DMA domain TLB invalidation policy: lazy mode [ 1.692678] SCSI subsystem initialized [ 1.692693] ACPI: bus type USB registered [ 1.692705] usbcore: registered new interface driver usbfs [ 1.692710] usbcore: registered new interface driver hub [ 1.692715] usbcore: registered new device driver usb [ 1.692734] pps_core: LinuxPPS API ver. 1 registered [ 1.692734] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 1.692737] PTP clock support registered [ 1.692749] EDAC MC: Ver: 3.0.0 [ 1.692749] NetLabel: Initializing [ 1.692749] NetLabel: domain hash size = 128 [ 1.692749] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 1.692756] NetLabel: unlabeled traffic allowed by default [ 1.692757] PCI: Using ACPI for IRQ routing [ 1.703619] pci 0000:06:00.0: vgaarb: setting as boot VGA device [ 1.703620] pci 0000:06:00.0: vgaarb: bridge control possible [ 1.703621] pci 0000:06:00.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none [ 1.703622] vgaarb: loaded [ 1.703646] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0, 0, 0, 0, 0, 0 [ 1.703651] hpet0: 8 comparators, 64-bit 14.318180 MHz counter [ 1.707370] clocksource: Switched to clocksource tsc-early [ 1.715028] VFS: Disk quotas dquot_6.6.0 [ 1.715037] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 1.715084] pnp: PnP ACPI init [ 1.715396] system 00:01: [io 0x0500-0x057f] has been reserved [ 1.715398] system 00:01: [io 0x0400-0x047f] has been reserved [ 1.715399] system 00:01: [io 0x0580-0x059f] has been reserved [ 1.715400] system 00:01: [io 0x0600-0x061f] has been reserved [ 1.715401] system 00:01: [io 0x0880-0x0883] has been reserved [ 1.715402] system 00:01: [io 0x0800-0x081f] has been reserved [ 1.715403] system 00:01: [mem 0xfed1c000-0xfed3ffff] has been reserved [ 1.715404] system 00:01: [mem 0xfed45000-0xfed8bfff] has been reserved [ 1.715405] system 00:01: [mem 0xff000000-0xffffffff] has been reserved [ 1.715406] system 00:01: [mem 0xfee00000-0xfeefffff] has been reserved [ 1.715407] system 00:01: [mem 0xfed12000-0xfed1200f] has been reserved [ 1.715408] system 00:01: [mem 0xfed12010-0xfed1201f] has been reserved [ 1.715409] system 00:01: [mem 0xfed1b000-0xfed1bfff] has been reserved [ 1.715549] system 00:02: [io 0x0a00-0x0a0f] has been reserved [ 1.715550] system 00:02: [io 0x0a10-0x0a1f] has been reserved [ 1.715551] system 00:02: [io 0x0a20-0x0a2f] has been reserved [ 1.715551] system 00:02: [io 0x0a30-0x0a3f] has been reserved [ 1.715552] system 00:02: [io 0x0a40-0x0a4f] has been reserved [ 1.716212] pnp: PnP ACPI: found 5 devices [ 1.721697] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 1.721747] NET: Registered PF_INET protocol family [ 1.721916] IP idents hash table entries: 262144 (order: 9, 2097152 bytes, linear) [ 1.724189] tcp_listen_portaddr_hash hash table entries: 16384 (order: 6, 262144 bytes, linear) [ 1.724216] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) [ 1.724221] TCP established hash table entries: 262144 (order: 9, 2097152 bytes, linear) [ 1.724436] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes, linear) [ 1.724545] TCP: Hash tables configured (established 262144 bind 65536) [ 1.724659] MPTCP token hash table entries: 32768 (order: 7, 786432 bytes, linear) [ 1.724744] UDP hash table entries: 16384 (order: 7, 524288 bytes, linear) [ 1.724798] UDP-Lite hash table entries: 16384 (order: 7, 524288 bytes, linear) [ 1.724877] NET: Registered PF_UNIX/PF_LOCAL protocol family [ 1.724882] NET: Registered PF_XDP protocol family [ 1.724902] pci 0000:00:1c.4: bridge window [mem 0x00100000-0x000fffff 64bit pref] to [bus 07] add_size 100000 add_align 100000 [ 1.724912] pci 0000:00:1c.4: BAR 15: assigned [mem 0x90000000-0x900fffff 64bit pref] [ 1.724914] pci 0000:00:01.0: PCI bridge to [bus 01] [ 1.724919] pci 0000:00:02.0: PCI bridge to [bus 02] [ 1.724921] pci 0000:00:02.0: bridge window [mem 0xfb300000-0xfb3fffff] [ 1.724926] pci 0000:03:00.0: BAR 7: no space for [mem size 0x00100000 64bit pref] [ 1.724927] pci 0000:03:00.0: BAR 7: failed to assign [mem size 0x00100000 64bit pref] [ 1.724929] pci 0000:03:00.0: BAR 10: no space for [mem size 0x00100000 64bit pref] [ 1.724930] pci 0000:03:00.0: BAR 10: failed to assign [mem size 0x00100000 64bit pref] [ 1.724931] pci 0000:03:00.1: BAR 7: no space for [mem size 0x00100000 64bit pref] [ 1.724932] pci 0000:03:00.1: BAR 7: failed to assign [mem size 0x00100000 64bit pref] [ 1.724933] pci 0000:03:00.1: BAR 10: no space for [mem size 0x00100000 64bit pref] [ 1.724934] pci 0000:03:00.1: BAR 10: failed to assign [mem size 0x00100000 64bit pref] [ 1.724935] pci 0000:03:00.0: BAR 7: no space for [mem size 0x00100000 64bit pref] [ 1.724936] pci 0000:03:00.0: BAR 7: failed to assign [mem size 0x00100000 64bit pref] [ 1.724937] pci 0000:03:00.0: BAR 10: no space for [mem size 0x00100000 64bit pref] [ 1.724938] pci 0000:03:00.0: BAR 10: failed to assign [mem size 0x00100000 64bit pref] [ 1.724939] pci 0000:03:00.1: BAR 7: no space for [mem size 0x00100000 64bit pref] [ 1.724940] pci 0000:03:00.1: BAR 7: failed to assign [mem size 0x00100000 64bit pref] [ 1.724941] pci 0000:03:00.1: BAR 10: no space for [mem size 0x00100000 64bit pref] [ 1.724942] pci 0000:03:00.1: BAR 10: failed to assign [mem size 0x00100000 64bit pref] [ 1.724943] pci 0000:00:03.0: PCI bridge to [bus 03] [ 1.724944] pci 0000:00:03.0: bridge window [io 0xe000-0xefff] [ 1.724946] pci 0000:00:03.0: bridge window [mem 0xfb200000-0xfb2fffff] [ 1.724948] pci 0000:00:03.0: bridge window [mem 0xfbd00000-0xfbefffff 64bit pref] [ 1.724951] pci 0000:00:1c.0: PCI bridge to [bus 04] [ 1.724963] pci 0000:05:00.0: PCI bridge to [bus 06] [ 1.724966] pci 0000:05:00.0: bridge window [io 0xd000-0xdfff] [ 1.724974] pci 0000:05:00.0: bridge window [mem 0xfa000000-0xfb0fffff] [ 1.724990] pci 0000:00:1c.2: PCI bridge to [bus 05-06] [ 1.724992] pci 0000:00:1c.2: bridge window [io 0xd000-0xdfff] [ 1.724997] pci 0000:00:1c.2: bridge window [mem 0xfa000000-0xfb0fffff] [ 1.725006] pci 0000:07:00.0: BAR 7: assigned [mem 0x90000000-0x9001ffff 64bit pref] [ 1.725012] pci 0000:07:00.0: BAR 10: assigned [mem 0x90020000-0x9003ffff 64bit pref] [ 1.725018] pci 0000:07:00.1: BAR 7: assigned [mem 0x90040000-0x9005ffff 64bit pref] [ 1.725024] pci 0000:07:00.1: BAR 10: assigned [mem 0x90060000-0x9007ffff 64bit pref] [ 1.725032] pci 0000:00:1c.4: PCI bridge to [bus 07] [ 1.725033] pci 0000:00:1c.4: bridge window [io 0xc000-0xcfff] [ 1.725039] pci 0000:00:1c.4: bridge window [mem 0xfb100000-0xfb1fffff] [ 1.725042] pci 0000:00:1c.4: bridge window [mem 0x90000000-0x900fffff 64bit pref] [ 1.725048] pci_bus 0000:00: Some PCI device resources are unassigned, try booting with pci=realloc [ 1.725048] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 1.725049] pci_bus 0000:00: resource 5 [io 0x1000-0xffff window] [ 1.725050] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 1.725051] pci_bus 0000:00: resource 7 [mem 0xfedb0000-0xfedb000f window] [ 1.725052] pci_bus 0000:00: resource 8 [mem 0xfedc0000-0xfedc000f window] [ 1.725053] pci_bus 0000:00: resource 9 [mem 0x90000000-0xfbffbfff window] [ 1.725054] pci_bus 0000:02: resource 1 [mem 0xfb300000-0xfb3fffff] [ 1.725055] pci_bus 0000:03: resource 0 [io 0xe000-0xefff] [ 1.725055] pci_bus 0000:03: resource 1 [mem 0xfb200000-0xfb2fffff] [ 1.725056] pci_bus 0000:03: resource 2 [mem 0xfbd00000-0xfbefffff 64bit pref] [ 1.725057] pci_bus 0000:05: resource 0 [io 0xd000-0xdfff] [ 1.725058] pci_bus 0000:05: resource 1 [mem 0xfa000000-0xfb0fffff] [ 1.725058] pci_bus 0000:06: resource 0 [io 0xd000-0xdfff] [ 1.725059] pci_bus 0000:06: resource 1 [mem 0xfa000000-0xfb0fffff] [ 1.725060] pci_bus 0000:07: resource 0 [io 0xc000-0xcfff] [ 1.725061] pci_bus 0000:07: resource 1 [mem 0xfb100000-0xfb1fffff] [ 1.725061] pci_bus 0000:07: resource 2 [mem 0x90000000-0x900fffff 64bit pref] [ 1.725218] pci 0000:00:05.0: disabled boot interrupts on device [8086:6f28] [ 1.736642] pci 0000:00:1a.0: quirk_usb_early_handoff+0x0/0x140 took 10985 usecs [ 1.748639] pci 0000:00:1d.0: quirk_usb_early_handoff+0x0/0x140 took 11702 usecs [ 1.748651] pci 0000:02:00.0: CLS mismatch (64 != 32), using 64 bytes [ 1.748692] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 1.748693] software IO TLB: mapped [mem 0x0000000064000000-0x0000000068000000] (64MB) [ 1.748707] Trying to unpack rootfs image as initramfs... [ 1.748708] ACPI: bus type thunderbolt registered [ 1.810480] Initialise system trusted keyrings [ 1.810492] Key type blacklist registered [ 1.810534] workingset: timestamp_bits=36 max_order=23 bucket_order=0 [ 1.812999] zbud: loaded [ 1.873501] integrity: Platform Keyring initialized [ 1.873505] integrity: Machine keyring initialized [ 1.881704] NET: Registered PF_ALG protocol family [ 1.881707] xor: automatically using best checksumming function avx [ 1.881709] Key type asymmetric registered [ 1.881710] Asymmetric key parser 'x509' registered [ 1.881711] Running certificate verification selftests [ 1.930377] Loaded X.509 cert 'Certificate verification self-testing key: f58703bb33ce1b73ee02eccdee5b8817518fe3db' [ 1.932613] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 246) [ 1.932640] io scheduler mq-deadline registered [ 1.932641] io scheduler kyber registered [ 1.932680] io scheduler bfq registered [ 1.951558] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 1.951918] pcieport 0000:00:01.0: PME: Signaling with IRQ 25 [ 1.952108] pcieport 0000:00:02.0: PME: Signaling with IRQ 27 [ 1.952282] pcieport 0000:00:03.0: PME: Signaling with IRQ 29 [ 1.952440] pcieport 0000:00:1c.0: PME: Signaling with IRQ 30 [ 1.952620] pcieport 0000:00:1c.2: PME: Signaling with IRQ 31 [ 1.952787] pcieport 0000:00:1c.4: PME: Signaling with IRQ 32 [ 1.952972] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 1.953049] ACPI: \_SB_.SCK0.CP00: Found 2 idle states [ 1.957962] input: Power Button as /devices/LNXSYSTM:00/LNXSYBUS:00/PNP0C0C:00/input/input0 [ 1.957980] ACPI: button: Power Button [PWRB] [ 1.958003] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input1 [ 1.958028] ACPI: button: Power Button [PWRF] [ 1.976086] ERST: Error Record Serialization Table (ERST) support is initialized. [ 1.976088] pstore: Registered erst as persistent store backend [ 1.976268] GHES: APEI firmware first mode is enabled by APEI bit and WHEA _OSC. [ 1.976384] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 1.976476] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 1.981499] 00:04: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 1.987959] Non-volatile memory driver v1.3 [ 1.993865] rdac: device handler registered [ 1.993893] hp_sw: device handler registered [ 1.993894] emc: device handler registered [ 1.993912] alua: device handler registered [ 1.994235] ehci-pci 0000:00:1d.0: EHCI Host Controller [ 1.994267] ehci-pci 0000:00:1d.0: new USB bus registered, assigned bus number 1 [ 1.994280] ehci-pci 0000:00:1d.0: debug port 2 [ 1.998206] ehci-pci 0000:00:1d.0: irq 18, io mem 0xfb433000 [ 2.004631] ehci-pci 0000:00:1d.0: USB 2.0 started, EHCI 1.00 [ 2.004695] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 5.14 [ 2.004697] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 2.004698] usb usb1: Product: EHCI Host Controller [ 2.004699] usb usb1: Manufacturer: Linux 5.14.0-386.el9.x86_64 ehci_hcd [ 2.004699] usb usb1: SerialNumber: 0000:00:1d.0 [ 2.004791] hub 1-0:1.0: USB hub found [ 2.004799] hub 1-0:1.0: 2 ports detected [ 2.004917] xhci_hcd 0000:00:14.0: xHCI Host Controller [ 2.004956] xhci_hcd 0000:00:14.0: new USB bus registered, assigned bus number 2 [ 2.006050] xhci_hcd 0000:00:14.0: hcc params 0x200077c1 hci version 0x100 quirks 0x0000000000009810 [ 2.006279] ehci-pci 0000:00:1a.0: EHCI Host Controller [ 2.006306] xhci_hcd 0000:00:14.0: xHCI Host Controller [ 2.006328] ehci-pci 0000:00:1a.0: new USB bus registered, assigned bus number 3 [ 2.006333] xhci_hcd 0000:00:14.0: new USB bus registered, assigned bus number 4 [ 2.006336] xhci_hcd 0000:00:14.0: Host supports USB 3.0 SuperSpeed [ 2.006344] ehci-pci 0000:00:1a.0: debug port 2 [ 2.006374] usb usb2: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 5.14 [ 2.006377] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 2.006378] usb usb2: Product: xHCI Host Controller [ 2.006380] usb usb2: Manufacturer: Linux 5.14.0-386.el9.x86_64 xhci-hcd [ 2.006381] usb usb2: SerialNumber: 0000:00:14.0 [ 2.006508] hub 2-0:1.0: USB hub found [ 2.006540] hub 2-0:1.0: 15 ports detected [ 2.009739] usb usb4: New USB device found, idVendor=1d6b, idProduct=0003, bcdDevice= 5.14 [ 2.009742] usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 2.009744] usb usb4: Product: xHCI Host Controller [ 2.009745] usb usb4: Manufacturer: Linux 5.14.0-386.el9.x86_64 xhci-hcd [ 2.009746] usb usb4: SerialNumber: 0000:00:14.0 [ 2.009840] hub 4-0:1.0: USB hub found [ 2.009859] hub 4-0:1.0: 6 ports detected [ 2.010246] ehci-pci 0000:00:1a.0: irq 18, io mem 0xfb434000 [ 2.011556] usbcore: registered new interface driver usbserial_generic [ 2.011563] usbserial: USB Serial support registered for generic [ 2.011596] i8042: PNP: No PS/2 controller found. [ 2.011633] mousedev: PS/2 mouse device common for all mice [ 2.011707] rtc_cmos 00:00: RTC can wake from S4 [ 2.012039] rtc_cmos 00:00: registered as rtc0 [ 2.012089] rtc_cmos 00:00: setting system clock to 2023-11-25T04:25:11 UTC (1700886311) [ 2.012118] rtc_cmos 00:00: alarms up to one month, y3k, 114 bytes nvram [ 2.016607] ehci-pci 0000:00:1a.0: USB 2.0 started, EHCI 1.00 [ 2.016651] usb usb3: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 5.14 [ 2.016653] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 2.016654] usb usb3: Product: EHCI Host Controller [ 2.016655] usb usb3: Manufacturer: Linux 5.14.0-386.el9.x86_64 ehci_hcd [ 2.016656] usb usb3: SerialNumber: 0000:00:1a.0 [ 2.016746] hub 3-0:1.0: USB hub found [ 2.016750] hub 3-0:1.0: 2 ports detected [ 2.023590] intel_pstate: Intel P-state driver initializing [ 2.042125] hid: raw HID events driver (C) Jiri Kosina [ 2.042158] usbcore: registered new interface driver usbhid [ 2.042159] usbhid: USB HID core driver [ 2.042183] drop_monitor: Initializing network drop monitor service [ 2.051479] Initializing XFRM netlink socket [ 2.051562] NET: Registered PF_INET6 protocol family [ 2.099032] Freeing initrd memory: 30988K [ 2.103453] Segment Routing with IPv6 [ 2.103478] NET: Registered PF_PACKET protocol family [ 2.103503] mpls_gso: MPLS GSO support [ 2.113710] microcode: Microcode Update Driver: v2.2. [ 2.119729] resctrl: L3 allocation detected [ 2.119729] resctrl: L3 monitoring detected [ 2.119733] IPI shorthand broadcast: enabled [ 2.119739] AVX2 version of gcm_enc/dec engaged. [ 2.155921] AES CTR mode by8 optimization enabled [ 2.155978] sched_clock: Marking stable (976439139, 1179159147)->(2519025861, -363427575) [ 2.173356] registered taskstats version 1 [ 2.200127] Loading compiled-in X.509 certificates [ 2.200620] Loaded X.509 cert 'The CentOS Project: CentOS Stream kernel signing key: af40eac79a5fac646a63bef7f5fbf3e105934bfc' [ 2.200906] Loaded X.509 cert 'Red Hat Enterprise Linux Driver Update Program (key 3): bf57f3e87362bc7229d9f465321773dfd1f77a80' [ 2.201184] Loaded X.509 cert 'Red Hat Enterprise Linux kpatch signing key: 4d38fd864ebe18c5f0b72e3852e2014c3a676fc8' [ 2.201329] Loaded X.509 cert 'RH-IMA-CA: Red Hat IMA CA: fb31825dd0e073685b264e3038963673f753959a' [ 2.201488] zswap: loaded using pool lzo/zbud [ 2.214545] page_owner is disabled [ 2.214739] pstore: Using crash dump compression: deflate [ 2.214752] Key type big_key registered [ 2.254600] usb 1-1: new high-speed USB device number 2 using ehci-pci [ 2.255634] usb 2-12: new high-speed USB device number 2 using xhci_hcd [ 2.266622] usb 3-1: new high-speed USB device number 2 using ehci-pci [ 2.318492] Key type encrypted registered [ 2.318498] ima: No TPM chip found, activating TPM-bypass! [ 2.318500] Loading compiled-in module X.509 certificates [ 2.318788] Loaded X.509 cert 'The CentOS Project: CentOS Stream kernel signing key: af40eac79a5fac646a63bef7f5fbf3e105934bfc' [ 2.318791] ima: Allocated hash algorithm: sha256 [ 2.318799] ima: No architecture policies found [ 2.318807] evm: Initialising EVM extended attributes: [ 2.318807] evm: security.selinux [ 2.318807] evm: security.SMACK64 (disabled) [ 2.318808] evm: security.SMACK64EXEC (disabled) [ 2.318808] evm: security.SMACK64TRANSMUTE (disabled) [ 2.318809] evm: security.SMACK64MMAP (disabled) [ 2.318809] evm: security.apparmor (disabled) [ 2.318809] evm: security.ima [ 2.318809] evm: security.capability [ 2.318810] evm: HMAC attrs: 0x1 [ 2.382823] usb 2-12: New USB device found, idVendor=0557, idProduct=7000, bcdDevice= 0.00 [ 2.382826] usb 2-12: New USB device strings: Mfr=0, Product=0, SerialNumber=0 [ 2.383008] usb 1-1: New USB device found, idVendor=8087, idProduct=8002, bcdDevice= 0.05 [ 2.383010] usb 1-1: New USB device strings: Mfr=0, Product=0, SerialNumber=0 [ 2.383169] hub 1-1:1.0: USB hub found [ 2.383227] hub 2-12:1.0: USB hub found [ 2.383258] hub 1-1:1.0: 8 ports detected [ 2.383303] hub 2-12:1.0: 4 ports detected [ 2.395006] usb 3-1: New USB device found, idVendor=8087, idProduct=800a, bcdDevice= 0.05 [ 2.395008] usb 3-1: New USB device strings: Mfr=0, Product=0, SerialNumber=0 [ 2.395163] hub 3-1:1.0: USB hub found [ 2.395256] hub 3-1:1.0: 6 ports detected [ 2.673624] usb 2-12.1: new low-speed USB device number 3 using xhci_hcd [ 2.749607] tsc: Refined TSC clocksource calibration: 3499.995 MHz [ 2.749610] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x32734c8180a, max_idle_ns: 440795326857 ns [ 2.751700] clocksource: Switched to clocksource tsc [ 2.778043] usb 2-12.1: New USB device found, idVendor=0557, idProduct=2419, bcdDevice= 1.00 [ 2.778046] usb 2-12.1: New USB device strings: Mfr=0, Product=0, SerialNumber=0 [ 2.783085] input: HID 0557:2419 as /devices/pci0000:00/0000:00:14.0/usb2/2-12/2-12.1/2-12.1:1.0/0003:0557:2419.0001/input/input2 [ 2.834706] hid-generic 0003:0557:2419.0001: input,hidraw0: USB HID v1.00 Keyboard [HID 0557:2419] on usb-0000:00:14.0-12.1/input0 [ 2.837422] input: HID 0557:2419 as /devices/pci0000:00/0000:00:14.0/usb2/2-12/2-12.1/2-12.1:1.1/0003:0557:2419.0002/input/input3 [ 2.837481] hid-generic 0003:0557:2419.0002: input,hidraw1: USB HID v1.00 Mouse [HID 0557:2419] on usb-0000:00:14.0-12.1/input1 [ 5.046672] Freeing unused decrypted memory: 2028K [ 5.047122] Freeing unused kernel image (initmem) memory: 3868K [ 5.058655] Write protecting the kernel read-only data: 28672k [ 5.058842] Freeing unused kernel image (rodata/data gap) memory: 764K [ 5.096316] x86/mm: Checked W+X mappings: passed, no W+X pages found. [ 5.096317] x86/mm: Checking user space page tables [ 5.134011] x86/mm: Checked W+X mappings: passed, no W+X pages found. [ 5.134021] Run /init as init process [ 5.147475] systemd[1]: systemd 252-18.el9 running in system mode (+PAM +AUDIT +SELINUX -APPARMOR +IMA +SMACK +SECCOMP +GCRYPT +GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS -FIDO2 +IDN2 -IDN -IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 -PWQUALITY +P11KIT -QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD -BPF_FRAMEWORK +XKBCOMMON +UTMP +SYSVINIT default-hierarchy=unified) Welcome to [0[ 5.179753] systemd[1]: Detected architecture x86-64. [ 5.179755] systemd[1]: Running in initrd. ;31mCentOS Stream 9 dracut-057-44.git20230822.el9 (Initramfs)! [ 5.198723] systemd[1]: Hostname set to . [ OK [[ 5.244076] systemd[1]: Queued start job for default target Initrd Default Target. [ 5.244188] systemd[1]: Reached target Initrd /usr File System. 0m] Reached target Initrd /usr File System. [ OK [[ 5.265752] systemd[1]: Reached target Local File Systems. 0m] Reached target Local File Systems. [ OK [[ 5.279733] systemd[1]: Reached target Slice Units. 0m] Reached target Slice Units. [ OK [[ 5.291672] systemd[1]: Reached target Swaps. 0m] Reached target Swaps. [ OK [[ 5.302731] systemd[1]: Reached target Timer Units. 0m] Reached target Timer Units. [ OK [[ 5.314744] systemd[1]: Listening on D-Bus System Message Bus Socket. 0m] Listening on D-Bus System Message Bus Socket. [ OK [[ 5.329832] systemd[1]: Listening on Journal Socket (/dev/log). 0m] Listening on Journal Socket (/dev/log). [ OK [[ 5.343740] systemd[1]: Listening on Journal Socket. 0m] Listening on Journal Socket. [ OK [[ 5.355929] systemd[1]: Listening on udev Control Socket. 0m] Listening on udev Control Socket. [ OK [[ 5.369730] systemd[1]: Listening on udev Kernel Socket. 0m] Listening on udev Kernel Socket. [ OK [[ 5.382669] systemd[1]: Reached target Socket Units. 0m] Reached target Socket Units. Startin[ 5.395621] systemd[1]: Starting Create List of Static Device Nodes... g Create List of Static Device Nodes... Startin[ 5.411706] systemd[1]: Starting Journal Service... g Journal Service... Startin[ 5.424379] systemd[1]: Starting Load Kernel Modules... g Load Kernel Modules... Startin[ 5.437176] systemd[1]: Starting Create System Users... g Create System Users... [ 5.445651] fuse: init (API version 7.36) Startin[ 5.451783] systemd[1]: Starting Setup Virtual Console... g Setup Virtual Console... [ OK [[ 5.464347] systemd[1]: Started Journal Service. 0m] Started Journal Service. [ OK ] Finished Create List of Static Device Nodes. [ OK ] Finished Load Kernel Modules. Starting Apply Kernel Variables... [ OK ] Finished Apply Kernel Variables. [ OK ] Finished Setup Virtual Console. Starting dracut cmdline hook... [ OK ] Finished Create System Users. Starting Create Static Device Nodes in /dev... Starting Create Volatile Files and Directories... [ OK ] Finished dracut cmdline hook. [ OK ] Finished Create Static Device Nodes in /dev. [ OK ] Finished Create Volatile Files and Directories. Starting dracut pre-udev hook... [ OK ] Finished dracut pre-udev hook. Starting Rule-based Manage…for Device Events and Files... [ OK ] Started Rule-based Manager for Device Events and Files. Starting Coldplug All udev Devices... [ OK ] Finished Coldplug All udev Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Network. Starting dracut initqueue hook... Starting Show Plymouth Boot Screen... [ 5.819851] dca service started, version 1.12.1 [ OK ] Started Show Plymouth Boot Screen. [ OK ] Started Forward Password R…s to Plymouth Directory Watch. [ OK ] Reached target Path Units. [ OK ] Reached target Basic System. [ 5.872069] ixgbe: Intel(R) 10 Gigabit PCI Express Network Driver [ 5.872072] ixgbe: Copyright (c) 1999-2016 Intel Corporation. [ 5.887639] igb: Intel(R) Gigabit Ethernet Network Driver [ 5.887641] igb: Copyright (c) 2007-2014 Intel Corporation. [ 5.903748] ahci 0000:00:11.4: AHCI 0001.0300 32 slots 4 ports 6 Gbps 0xf impl SATA mode [ 5.903751] ahci 0000:00:11.4: flags: 64bit ncq pm led clo pio slum part ems apst [ 5.903825] ACPI: bus type drm_connector registered [ 5.903857] nvme nvme0: pci function 0000:02:00.0 [ 5.918840] scsi host0: ahci [ 5.918959] scsi host1: ahci [ 5.919071] scsi host2: ahci [ 5.919203] scsi host3: ahci [ 5.919246] ata1: SATA max UDMA/133 abar m2048@0xfb438000 port 0xfb438100 irq 43 [ 5.919249] ata2: SATA max UDMA/133 abar m2048@0xfb438000 port 0xfb438180 irq 43 [ 5.919252] ata3: SATA max UDMA/133 abar m2048@0xfb438000 port 0xfb438200 irq 43 [ 5.919254] ata4: SATA max UDMA/133 abar m2048@0xfb438000 port 0xfb438280 irq 43 [ 5.919469] ahci 0000:00:1f.2: AHCI 0001.0300 32 slots 6 ports 6 Gbps 0x3f impl SATA mode [ 5.919472] ahci 0000:00:1f.2: flags: 64bit ncq pm led clo pio slum part ems apst [ 5.921680] nvme nvme0: 8/0/0 default/read/poll queues [ 5.937786] scsi host4: ahci [ 5.938197] scsi host5: ahci [ 5.938603] scsi host6: ahci [ 5.939038] scsi host7: ahci [ 5.939450] scsi host8: ahci [ 5.939868] scsi host9: ahci [ 5.939993] ata5: SATA max UDMA/133 abar m2048@0xfb432000 port 0xfb432100 irq 45 [ 5.939996] ata6: SATA max UDMA/133 abar m2048@0xfb432000 port 0xfb432180 irq 45 [ 5.939999] ata7: SATA max UDMA/133 abar m2048@0xfb432000 port 0xfb432200 irq 45 [ 5.940001] ata8: SATA max UDMA/133 abar m2048@0xfb432000 port 0xfb432280 irq 45 [ 5.940004] ata9: SATA max UDMA/133 abar m2048@0xfb432000 port 0xfb432300 irq 45 [ 5.940007] ata10: SATA max UDMA/133 abar m2048@0xfb432000 port 0xfb432380 irq 45 [ 5.944161] igb 0000:07:00.0: added PHC on eth0 [ 5.944163] igb 0000:07:00.0: Intel(R) Gigabit Ethernet Network Connection [ 5.944163] igb 0000:07:00.0: eth0: (PCIe:5.0Gb/s:Width x4) 0c:c4:7a:d8:da:66 [ 5.944241] igb 0000:07:00.0: eth0: PBA No: 050B00-000 [ 5.944241] igb 0000:07:00.0: Using MSI-X interrupts. 8 rx queue(s), 8 tx queue(s) [ 5.975832] ast 0000:06:00.0: [drm] Using P2A bridge for configuration [ 5.975837] ast 0000:06:00.0: [drm] AST 2400 detected [ 5.975854] ast 0000:06:00.0: [drm] Using analog VGA [ 5.975863] ast 0000:06:00.0: [drm] dram MCLK=408 Mhz type=1 bus_width=16 [ 5..0 20120228 for 0000:06:00.0 on minor 0 [ 6.000078] igb 0000:07:00.1: added PHC on eth1 [ 6.000079] igb 0000:07:00.1: Intel(R) Gigabit Ethernet Network Connection [ 6.000080] igb 0000:07:00.1: eth1: (PCIe:5.0Gb/s:Width x4) 0c:c4:7a:d8:da:67 [ 6.000162] igb 0000:07:00.1: eth1: PBA No: 050B00-000 [ 6.000163] igb 0000:07:00.1: Using MSI-X interrupts. 8 rx queue(s), 8 tx queue(s) [ 6.001438] igb 0000:07:00.0 eno1: renamed from eth0 [ 6.009722] igb 0000:07:00.1 eno2: renamed from eth1 [ 6.016997] fbcon: astdrmfb (fb0) is primary device [ 6.065902] Console: switching to colour frame buffer device 128x48 [ 6.102657] ast 0000:06:00.0: [drm] fb0: astdrmfb frame buffer device [ 6.226970] ata3: SATA link down (SStatus 0 SControl 300) [ 6.227059] ata4: SATA link down (SStatus 0 SControl 300) [ 6.227395] ata2: SATA link down (SStatus 0 SControl 300) [ 6.228719] ata1: SATA link down (SStatus 0 SControl 300) [ 6.251081] ata5: SATA link up 6.0 Gbps (SStatus 133 SControl 300) [ 6.251210] ata9: SATA link down (SStatus 0 SControl 300) [ 6.251941] ata10: SATA link down (SStatus 0 SControl 300) [ 6.251960] ata7: SATA link down (SStatus 0 SControl 300) [ 6.252119] ata5.00: ATA-9: ST1000NM0033-9ZM173, SN06, max UDMA/133 [ 6.252CQ Send/Recv Log6.252680] ata5.0ors, multi 16: LBA48 NCQ (depth 32), AA [ 6.252799] ata8: SATA link down (SStatus 0 SControl 300) [ 6.254092] ata5.00: NCQ Send/Recv Log not supported [ 6.254096] ata5.00: configured for UDMA/133 [ 6.254198] scsi 4:0:0:0: Direct-Access ATA ST1000NM0033-9ZM SN06 PQ: 0 ANSI: 5 [ 6.256872] ata6: SATA link down (SStatus 0 SControl 300) [ 6.344352] scsi 4:0:0:0: Attached scsi generic sg0 type 0 [ 6.412514] sd 4:0:0:0: [sda] 1953525168 512-byte logical blocks: (1.00 TB/932 GiB) [ 6.412526] sd 4:0:0:0: [sda] Write Protect is off [ 6.412545] sd 4:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA [ 6.412566] sd 4:0:0:0: [sda] Preferred minimum I/O size 512 bytes [ 6.426300] sda: sda1 [ 6.426351] sd 4:0:0:0: [sda] Attached SCSI disk [ OK ] Found device ST1000NM0033-9ZM173 1. [ OK ] Reached target Initrd Root Device. [ 7.010844] ixgbe 0000:03:00.0: Multiqueue Enabled: Rx Queue count = 8, Tx Queue count = 8 XDP Queue count = 0 [ 7.011131] ixgbe 0000:03:00.0: 32.000 Gb/s available PCIe bandwidth (5.0 GT/s PCIe x8 link) [ 7.011213] ixgbe 0000:03:00.0: MAC: 2, PHY: 1, PBA No: 0210FF-0FF [ 7.011214] ixgbe 0000:03:00.0: 0c:c4:7a:8f:47:e0 [ 7.013636] ixgbe 0000:03:00.0: Intel(R) 10 Gigabit Network Connection [ 7.169847] ixgbe 0000:03:00.1: Multiqueue Enabled: Rx Queue count = 8, Tx Queue count = 8 XDP Queue count = 0 [ 7.170146] ixgbe 0000:03:00.1: 32.000 Gb/s available PCIe bandwidth (5.0 GT/s PCIe x8 link) [ 7.170228] ixgbe 0000:03:00.1: MAC: 2, PHY: 14, SFP+: 4, PBA No: 0210FF-0FF [ 7.170229] ixgbe 0000:03:00.1: 0c:c4:7a:8f:47:e1 [ 7.173391] ixgbe 0000:03:00.1: Intel(R) 10 Gigabit Network Connection [ 7.181346] ixgbe 0000:03:00.1 enp3s0f1: renamed from eth1 [ 7.214017] ixgbe 0000:03:00.0 enp3s0f0: renamed from eth0 [ OK ] Finished dracut initqueue hook. [ OK ] Reached target Preparation for Remote File Systems. [ OK ] Reached target Remote File Systems. Starting File System Check…7844-4e73-a652-6bb884dfb0a5... [ OK ] Finished File System Check…0-7844-4e73-a652-6bb884dfb0a5. Mounting /sysroot... [ 7.440161] EXT4-fs (sda1): mounted filesystem with ordered data mode. Quota mode: none. [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Mountpoints Configured in the Real Root... [ OK ] Finished Mountpoints Configured in the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Finished dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped target Network. [ OK ] Stopped target Timer Units. [ OK ] Closed D-Bus System Message Bus Socket. [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Basic System. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Initrd /usr File System. [ OK ] Stopped target Path Units. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Preparation for Remote File Systems. [ OK ] Stopped target Slice Units. [ OK ] Stopped target Socket Units. [ OK ] Stopped target System Initialization. [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Swaps. [ OK ] Stopped dracut initqueue hook. Starting Plymouth switch root service... [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped Load Kernel Modules. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped Coldplug All udev Devices. Stopping Rule-based Manage…for Device Events and Files... [ OK ] Stopped Rule-based Manager for Device Events and Files. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. Starting Cleanup udev Database... [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create List of Static Device Nodes. [ OK ] Stopped Create System Users. [ OK ] Finished Cleaning Up and Shutting Down Daemons. [ OK ] Finished Cleanup udev Database. [ OK ] Reached target Switch Root. [ OK ] Finished Plymouth switch root service. Starting Switch Root... [ 7.791875] systemd-journald[279]: Received SIGTERM from PID 1 (systemd). [ 8.279963] SELinux: policy capability network_peer_controls=1 [ 8.279966] SELinux: policy capability open_perms=1 [ 8.279966] SELinux: policy capability extended_socket_class=1 [ 8.279967] SELinux: policy capability always_check_network=0 [ 8.279967] SELinux: policy capability cgroup_seclabel=1 [ 8.279967] SELinux: policy capability nnp_nosuid_transition=1 [ 8.279968] SELinux: policy capability genfs_seclabel_symlinks=1 [ 8.323890] audit: type=1403 audit(1700886317.810:2): auid=4294967295 ses=4294967295 lsm=selinux res=1 [ 8.326082] systemd[1]: Successfully loaded SELinux policy in 115.983ms. [ 8.426749] systemd[1]: Relabelled /dev, /dev/shm, /run, /sys/fs/cgroup in 30.317ms. [ 8.435448] systemd[1]: systemd 252-18.el9 running in system mode (+PAM +AUDIT +SELINUX -APPARMOR +IMA +SMACK +SECCOMP +GCRYPT +GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS -FIDO2 +IDN2 -IDN -IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 -PWQUALITY +P11KIT -QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD -BPF_FRAMEWORK +XKBCOMMON +UTMP +SYSVINIT default-hierarchy=unified) [ 8.446675] systemd[1]: Detected architecture x86-64. Welcome to CentOS Stream 9! [ OK [[ 9.410191] systemd[1]: initrd-switch-root.service: Deactivated successfully. [ 9.410289] systemd[1]: Stopped Switch Root. 0m] Stopped [0;[ 9.410672] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 9.410969] systemd[1]: Created slice Slice /system/getty. [ 9.419316] systemd[1]: Created slice Slice /system/modprobe. [ 9.423630] systemd[1]: Created slice Slice /system/serial-getty. [ 9.433817] systemd[1]: Created slice Slice /system/sshd-keygen. 1;39mSwitch Root[ 9.457134] systemd[1]: Created slice User and Session Slice. [ 9.457165] systemd[1]: Dispatch Password Requests to Console Directory Watch was skipped because of an unmet condition check (ConditionPathExists=!/run/plymouth/pid). [ 9.457259] systemd[1]: Started Forward Password Requests to Wall Directory Watch. [ 9.457608] systemd[1]: Set up automount Arbitrary Executable File Formats File System Automount Point. [ 9.478978] systemd[1]: Reached target ceph target allowing to start/stop all ceph-mon@.service instances at once. . [[ 9.506222] systemd[1]: Reached target ceph target allowing to start/stop all ceph-mds@.service instances at once. [ 9.506245] systemd[1]: Reached target ceph target allowing to start/stop all ceph-mgr@.service instances at once. [ 9.506262] systemd[1]: Reached target ceph target allowing to start/stop all ceph-osd@.service instances at once. [ 9.506278] systemd[1]: Reached target ceph target allowing to start/stop all ceph-radosgw@.service instances at once. [ 9.506304] systemd[1]: Reached target ceph target allowing to start/stop all ceph-rbd-mirror@.service instances at once. [ 9.506321] systemd[1]: Reached target Local Encrypted Volumes. [ 9.506346] systemd[1]: Stopped target Switch Root. [ 9.506365] systemd[1]: Stopped target Initrd File Systems. [ 9.506379] systemd[1]: Stopped target Initrd Root File System. [ 9.506395] systemd[1]: Reached target Local Integrity Protected Volumes. [ 9.506430] systemd[1]: Reached target Path Units. [ 9.506461] systemd[1]: Reached target Slice Units. OK ] Crea[ 9.506485] systemd[1]: Reached target Swaps. [ 9.506504] systemd[1]: Reached target Local Verity Protected Volumes. [ 9.506865] systemd[1]: Listening on Device-mapper event daemon FIFOs. [ 9.608173] systemd[1]: Listening on LVM2 poll daemon socket. ted slice Slice /system/getty. [ OK ] Created slice Slice /[ 9.629745] systemd[1]: Listening on RPCbind Server Activation Socket. [ 9.629787] systemd[1]: Reached target RPC Port Mapper. [ 9.633731] systemd[1]: Listening on Process Core Dump Socket. [ 9.643626] systemd[1]: Listening on initctl Compatibility Named Pipe. system/modprobe[ 9.656552] systemd[1]: Listening on udev Control Socket. [ 9.656808] systemd[1]: Listening on udev Kernel Socket. [0m. [ OK ] Creat[ 9.668801] systemd[1]: Mounting Huge Pages File System... [ 9.669691] systemd[1]: Mounting POSIX Message Queue File System... ed slice [0;1;3[ 9.683235] systemd[1]: Mounting Kernel Debug File System... [ 9.684041] systemd[1]: Mounting Kernel Trace File System... 9mSlice /system/[ 9.695248] systemd[1]: Kernel Module supporting RPCSEC_GSS was skipped because of an unmet condition check (ConditionPathExists=/etc/krb5.keytab). [ 9.703545] systemd[1]: Starting Create List of Static Device Nodes... serial-getty. [ OK[ 9.717063] systemd[1]: Starting Monitoring of LVM2 mirrors, snapshots etc. using dmeventd or progress polling... [ 9.717894] systemd[1]: Starting Load Kernel Module configfs... ] Created slice S[ 9.735967] systemd[1]: Starting Load Kernel Module drm... [ 9.736711] systemd[1]: Starting Load Kernel Module fuse... lice /system/sshd-keygen. [ OK ] Created slice User and Session Slice. [ OK ] Started Forward Password R…uests to Wall Directory Watch. [[ 9.764827] systemd[1]: Starting Read and set NIS domainname from /etc/sysconfig/network... [ 9.764892] systemd[1]: plymouth-switch-root.service: Deactivated successfully. OK ] Set [ 9.764939] systemd[1]: Stopped Plymouth switch root service. [ 9.765112] systemd[1]: Stopped Journal Service. up automount [0[ 9.794947] systemd[1]: Starting Journal Service... [ 9.796664] systemd[1]: Starting Load Kernel Modules... ;1;39mArbitrary…s File System [ 9.806752] systemd[1]: Starting Generate network units from Kernel command line... [ 9.807540] systemd[1]: Starting Remount Root and Kernel File Systems... Automount Point[ 9.807620] systemd[1]: Repartition Root Disk was skipped because no trigger condition checks were met. [ 9.808361] systemd[1]: Starting Coldplug All udev Devices... [ 9.824155] systemd[1]: Mounted Huge Pages File System. [ 9.839827] systemd[1]: Mounted POSIX Message Queue File System. [0m. [ [ 9.850952] systemd[1]: Mounted Kernel Debug File System. OK ] Reach[ 9.851106] systemd[1]: Mounted Kernel Trace File System. [ 9.851484] systemd[1]: Finished Create List of Static Device Nodes. ed target [0;1;[ 9.851843] systemd[1]: Finished Read and set NIS domainname from /etc/sysconfig/network. [ 9.857637] EXT4-fs (sda1): re-mounted. Quota mode: none. [ 9.858284] systemd[1]: Finished Generate network units from Kernel command line. [ 9.864646] systemd[1]: Finished Remount Root and Kernel File Systems. 39mceph target\xe2\x80[ 9.870943] systemd[1]: Started Journal Service. \xa6on@.service instances at once. [ OK ] Reached target ceph target…ds@.service instances at once. [ OK ] Reached target ceph target…gr@.service instances at once. [ OK ] Reached target [ 9.928419] systemd-journald[567]: Received client request to flush runtime journal. ceph target…sd@.service instances at once. [ OK ] Reached target ceph target…gw@.service instances at once. [ OK ] Reached target ceph target…or@.service instances at once. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd File Systems. [ OK ] Stopped target Initrd Root File System. [ OK ] Reached target Local Integrity Protected Volumes. [ OK ] Reached target Path Units. [ OK ] Reached target Slice Units. [ OK ] Reached target Swaps. [ OK ] Reached target Local Verity Protected Volumes. [ OK ] Listening on Device-mapper event daemon FIFOs. [ OK ] Listening on LVM2 poll daemon socket. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Listening on Process Core Dump Socket. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Listening on udev Control Socket. [ OK ] Listening on udev Kernel Socket. Mounting Huge Pages File System... Mounting POSIX Message Queue File System... Mounting Kernel Debug File System... Mounting Kernel Trace File System... Starting Create List of Static Device Nodes... Starting Monitoring of LVM…meventd or progress polling... Starting Load Kernel Module configfs... Starting Load Kernel Module drm... Starting Load Kernel Module fuse... Starting Read and set NIS …from /etc/sysconfig/network... [ OK ] Stopped Plymouth switch root service. [ OK ] Stopped Journal Service. Starting Journal Service... Starting Load Kernel Modules... Starting Generate network …ts from Kernel command line... Starting Remount Root and Kernel File Systems... Starting Coldplug All udev Devices... [ OK ] Mounted Huge Pages File System. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Mounted Kernel Debug File System. [ OK ] Mounted Kernel Trace File System. [ OK ] Finished Create List of Static Device Nodes.[ 10.192002] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. [ 10.192020] device-mapper: uevent: version 1.0.3 [ 10.192105] device-mapper: ioctl: 4.48.0-ioctl (2023-03-01) initialised: dm-devel@redhat.com [ OK ] Finished Read and set NIS …e from /etc/sysconfig/network. [ OK ] Finished Generate network units from Kernel command line. [ OK ] Finished Remount Root and Kernel File Systems. [ OK ] Started Journal Service. [ OK ] Reached target Preparation for Network. Starting Flush Journal to Persistent Storage... Starting Load/Save Random Seed... Starting Create Static Device Nodes in /dev... [ OK ] Finished Flush Journal to Persistent Storage. [ OK ] Finished Coldplug All udev Devices. [ OK ] Finished Load Kernel Module drm. [ OK ] Finished Load Kernel Module configfs. [ OK ] Finished Load Kernel Module fuse. Mounting FUSE Control File System... Mounting Kernel Configuration File System... [ OK ] Mounted FUSE Control File System. [ OK ] Mounted Kernel Configuration File System. [ OK ] Finished Load Kernel Modules. Starting Apply Kernel Variables... [ OK ] Finished Load/Save Random Seed. [ OK ] Finished Apply Kernel Variables. [ OK ] Finished Monitoring of LVM… dmeventd or progress polling. [ OK ] Finished Create Static Device Nodes in /dev. Starting Rule-based Manage…for Device Events and Files... [ OK ] Started Rule-based Manager for Device Events and Files. [ OK ] Reached target Preparation for Local File Systems. [ OK ] Reached target Local File Systems. Starting Tell Plymouth To Write Out Runtime Data... Starting Automatic Boot Loader Update... Starting Create Volatile Files and Directories... Starting Load Kernel Module configfs... Starting Load Kernel Module fuse... [ OK ] Finished Load Kernel Module fuse. [ OK ] Finished Load Kernel Module configfs. [ OK ] Finished Automatic Boot Loader Update. [ OK ] Finished Tell Plymouth To Write Out Runtime Data. [ 10.573002] ioatdma: Intel(R) QuickData Technology Driver 5.00 [ 10.582401] igb 0000:07:00.0: DCA enabled [ 10.582416] igb 0000:07:00.1: DCA enabled [ 10.614214] mei_me 0000:00:16.0: Device doesn't have valid ME Interface [ 10.639621] i801_smbus 0000:00:1f.3: SMBus using PCI interrupt [ 10.642127] i2c i2c-3: 2/8 memory slots populated (from DMI) [ 10.642130] i2c i2c-3: Systems with more than 4 memory slots not supported yet, not instantiating SPD [ 10.661946] IPMI message handler: version 39.2 [ 10.667158] ipmi device interface [ 10.681228] ipmi_si: IPMI System Interface driver [ 10.684684] ipmi_si dmi-ipmi-si.0: ipmi_platform: probing via SMBIOS [ 10.684687] ipmi_platform: ipmi_si: SMBIOS: io 0xca2 regsize 1 spacing 1 irq 0 [ 10.684690] ipmi_si: Adding SMBIOS-specified kcs state machine [ 10.684750] ipmi_si IPI0001:00: ipmi_platform: probing via ACPI [ 10.684882] ipmi_si IPI0001:00: ipmi_platform: [io 0x0ca2] regsize 1 spacing 1 irq 0 [ 10.717835] power_meter ACPI000D:00: Found ACPI power meter. [ 10.717862] power_meter ACPI000D:00: Ignoring unsafe software power cap! [ 10.717867] power_meter ACPI000D:00: hwmon_device_register() is deprecated. Please convert the driver to use hwmon_device_register_with_info(). [ OK ] Started [0;[ 10.745094] input: PC Speaker as /devices/platform/pcspkr/input/input4 1;39m/usr/sbin/l[ 10.747241] ipmi_si dmi-ipmi-si.0: Removing SMBIOS-specified kcs state machine in favor of ACPI vm vgch…-autoa[ 10.747244] ipmi_si: Adding ACPI-specified kcs state machine ctivation event [ 10.747340] iTCO_vendor_support: vendor-support=0 vg_nvme. [ 10.748787] ipmi_si: Trying ACPI-specified kcs state machine at i/o address 0xca2, slave address 0x20, irq 0 [ 10.754732] iTCO_wdt iTCO_wdt.1.auto: Found a Wellsburg TCO device (Version=2, TCOBASE=0x0460) [ OK ] Finished [0[ 10.755157] iTCO_wdt iTCO_wdt.1.auto: initialized. heartbeat=30 sec (nowayout=0) ;1;39mCreate Volatile Files and Directories.[ 10.810407] RAPL PMU: API unit is 2^-32 Joules, 2 fixed counters, 655360 ms ovfl timer [ OK [ 10.810410] RAPL PMU: hw unit of domain package 2^-14 Joules [ 10.810411] RAPL PMU: hw unit of domain dram 2^-16 Joules ] Found device /de[ 10.833607] ipmi_si IPI0001:00: The BMC does not support clearing the recv irq bit, compensating, but the BMC needs to be fixed. v/vg_nvme/lv_5. Starting Security Auditing Service... Startin[ 10.855924] ipmi_si IPI0001:00: IPMI message handler: Found new BMC (man_id: 0x002a7c, prod_id: 0x0833, dev_id: 0x20) g RPC Bind... Mounting /var/lib/ceph... Mounting RPC Pipe File System... [ 10.935048] ipmi_si IPI0001:00: IPMI kcs interface initialized [ 10.943124] ipmi_ssif: IPMI SSIF Interface driver [ 11.040173] RPC: Registered named UNIX socket transport module. [ OK [[ 11.040175] RPC: Registered udp transport module. 0m] Mounted [0;[ 11.040175] RPC: Registered tcp transport module. 1;39mRPC Pipe File System. [ OK ] Reached targ[ 11.040176] RPC: Registered tcp NFSv4.1 backchannel transport module. et rpc_pipefs.target. [ OK ] Started RPC Bind. [ 11.260244] SGI XFS with ACLs, security attributes, scrub, quota, no debug enabled [ 11.268870] XFS (dm-4): Deprecated V4 format (crc=0) will not be supported after September 2030. [ 11.268975] XFS (dm-4): Mounting V4 Filesystem [ OK [[ 11.273548] XFS (dm-4): Ending clean mount [ 11.273584] xfs filesystem being mounted at /var/lib/ceph supports timestamps until 2038 (0x7fffffff) 0m] Mounted /var/lib/ceph. [ 11.336511] EDAC MC0: Giving out device to module sb_edac controller Broadwell SrcID#0_Ha#0: DEV 0000:ff:12.0 (INTERRUPT) [ 11.336512] EDAC sbridge: Ver: 1.1.2 [ 11.421117] intel_rapl_common: Found RAPL domain package [ 11.421123] intel_rapl_common: Found RAPL domain dram [ OK ] Started Security Auditing Service. Starting Record System Boot/Shutdown in UTMP... [ OK ] Finished Record System Boot/Shutdown in UTMP. [ OK ] Reached target System Initialization. [ OK ] Started Daily rotation of log files. [ OK ] Started Updates mlocate database every day. [ OK ] Started Run system activit…ounting tool every 10 minutes. [ OK ] Started Generate summary o…esterday's process accounting. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Reached target Timer Units. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Listening on SSSD Kerberos…ache Manager responder socket. [ OK ] Reached target Socket Units. Starting D-Bus System Message Bus... [ OK ] Started D-Bus System Message Bus. [ OK ] Reached target Basic System. Starting Network Manager... [ OK ] Started Ceph crash dump collector. [ OK ] Reached target ceph target…h*@.service instances at once. Starting NTP client/server... Starting Restore /run/initramfs on shutdown... [ OK ] Started irqbalance daemon. [ OK ] Started libstoragemgmt plug-in server daemon. [ OK ] Started Machine Check Exception Logging Daemon. Starting Authorization Manager... Starting Self Monitoring a…g Technology (SMART) Daemon... [ OK ] Reached target sshd-keygen.target. [ OK ] Reached target User and Group Name Lookups. Starting Resets System Activity Logs... Starting User Login Management... [ OK ] Finished Restore /run/initramfs on shutdown. [ OK ] Started User Login Management. [ OK ] Finished Resets System Activity Logs. [ OK ] Started Self Monitoring an…ing Technology (SMART) Daemon. [ OK ] Started NTP client/server. Starting Hostname Service... [ OK ] Started Hostname Service. [ OK ] Listening on Load/Save RF …itch Status /dev/rfkill Watch. [ OK ] Started Authorization Manager. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ 13.265068] ixgbe 0000:03:00.0: registered PHC device on enp3s0f0 [ 13.414513] ixgbe 0000:03:00.1: registered PHC device on enp3s0f1 [ 13.642294] ixgbe 0000:03:00.1 enp3s0f1: detected SFP+: 4 [ 13.725736] ixgbe 0000:03:00.1 enp3s0f1: NIC Link is Up 10 Gbps, Flow Control: RX/TX [ 13.725867] IPv6: ADDRCONF(NETDEV_CHANGE): enp3s0f1: link becomes ready [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting Network Manager Wait Online... Starting GSSAPI Proxy Daemon... Starting OpenSSH server daemon... Starting Dynamic System Tuning Daemon... [ OK ] Started OpenSSH server daemon. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Preparation for Remote File Systems. [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Finished Permit User Sessions. [ OK ] Started Deferred execution scheduler. [ OK ] Started Command Scheduler. [ OK ] Started Dynamic System Tuning Daemon. [* ] A start job is running for Network Manager Wait Online (9s / no limit) M [** ] A start job is running for Network Manager Wait Online (9s / no limit) M [ OK ] Finished Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Crash recovery kernel arming... [ OK ] Started Nagios Remote Plugin Executor. Starting /etc/rc.d/rc.local Compatibility... Starting Notify NFS peers of a restart... Starting System Logging Service... + '[' '!' -f /.cephlab_net_configured ']' + set +e + attempts=0 + myips= + '[' '' '!=' '' ']' + '[' 0 -ge 10 ']' ++ ip -4 addr ++ grep -oP '(?<=inet\s)\d+(\.\d+){3}' ++ grep -v '127.0.0.1\|127.0.1.1' [ OK ] Started Notify NFS peers of a restart. + myips=172.21.15.76 + attempts=1 + sleep 1 [ OK ] Started System Logging Service. Starting SETroubleshoot da…ing new SELinux denial logs... + '[' 172.21.15.76 '!=' '' ']' + set -e + '[' -n 172.21.15.76 ']' + for ip in $myips + timeout 1s ping -I 172.21.15.76 -nq -c1 172.21.0.1 ++ dig +short -x 172.21.15.76 @172.21.0.1 ++ sed 's/\.com.*/\.com/g' + newhostname=smithi076.front.sepia.ceph.com + '[' -n smithi076.front.sepia.ceph.com ']' + hostname smithi076.front.sepia.ceph.com ++ hostname -d + newdomain=front.sepia.ceph.com ++ hostname -s + shorthostname=smithi076 + echo smithi076 + grep -q front.sepia.ceph.com /etc/hosts + sed -i 's/.*front.sepia.ceph.com.*/172.21.15.76 smithi076.front.sepia.ceph.com smithi076/g' /etc/hosts + break + command -v zypper + command -v apt-get + '[' -e /.cephlab_rc_local ']' + exit 0 [ OK ] Started /etc/rc.d/rc.local Compatibility. Starting Hold until boot process finishes up... Starting Terminate Plymouth Boot Screen... [ 24.154264] block sda: the capability attribute has been deprecated. CentOS Stream 9 Kernel 5.14.0-386.el9.x86_64 on an x86_64 Activate the web console with: systemctl enable --now cockpit.socket smithi076 login: [disconnect] 2023-11-25T04:25:37.177 INFO:teuthology.misc:Re-opening connections... 2023-11-25T04:25:37.177 INFO:teuthology.misc:trying to connect to ubuntu@smithi076.front.sepia.ceph.com 2023-11-25T04:25:37.177 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi076.front.sepia.ceph.com' 2023-11-25T04:25:37.178 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-25T04:25:37.843 DEBUG:teuthology.orchestra.run.smithi076:> true 2023-11-25T04:25:38.316 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi076.front.sepia.ceph.com' 2023-11-25T04:25:38.317 DEBUG:teuthology.misc:waited 1.1394672393798828 2023-11-25T04:25:39.318 DEBUG:tasks.ceph_manager:Mounting data for osd.0 on ubuntu@smithi076.front.sepia.ceph.com 2023-11-25T04:25:39.318 INFO:tasks.ceph_manager:Mounting osd.0: dev: ubuntu@smithi076.front.sepia.ceph.com, cluster: cephmountpoint: /var/lib/ceph/osd/ceph-0, type: xfs, options: ['noatime'] 2023-11-25T04:25:39.318 DEBUG:teuthology.orchestra.run.smithi076:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-11-25T04:25:39.412 DEBUG:teuthology.orchestra.run.smithi076:> sudo install -d -m0777 -- /var/run/ceph 2023-11-25T04:25:39.609 INFO:tasks.ceph.osd.0:Restarting daemon 2023-11-25T04:25:39.610 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-11-25T04:25:39.612 INFO:tasks.ceph.osd.0:Started 2023-11-25T04:25:39.612 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-11-25T04:25:39.718 INFO:teuthology.orchestra.run.smithi076.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2023-11-25T04:25:39.720 DEBUG:teuthology.orchestra.run:got remote process result: 22 2023-11-25T04:25:39.721 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-0, ['dump_ops_in_flight'] 2023-11-25T04:25:40.597 INFO:tasks.ceph.osd.0.smithi076.stderr:2023-11-25T04:25:40.596+0000 7f3e5c40c500 -1 Falling back to public interface 2023-11-25T04:25:41.089 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2023-11-25T04:25:44.300 INFO:tasks.ceph.osd.0.smithi076.stderr:2023-11-25T04:25:44.299+0000 7f3e5c40c500 -1 osd.0 20 log_to_monitors true 2023-11-25T04:25:44.722 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-11-25T04:25:44.832 INFO:teuthology.orchestra.run.smithi076.stderr:no valid command found; 10 closest matches: 2023-11-25T04:25:44.833 INFO:teuthology.orchestra.run.smithi076.stderr:0 2023-11-25T04:25:44.833 INFO:teuthology.orchestra.run.smithi076.stderr:1 2023-11-25T04:25:44.833 INFO:teuthology.orchestra.run.smithi076.stderr:2 2023-11-25T04:25:44.833 INFO:teuthology.orchestra.run.smithi076.stderr:abort 2023-11-25T04:25:44.833 INFO:teuthology.orchestra.run.smithi076.stderr:assert 2023-11-25T04:25:44.834 INFO:teuthology.orchestra.run.smithi076.stderr:bluefs debug_inject_read_zeros 2023-11-25T04:25:44.834 INFO:teuthology.orchestra.run.smithi076.stderr:bluefs files list 2023-11-25T04:25:44.834 INFO:teuthology.orchestra.run.smithi076.stderr:bluefs stats 2023-11-25T04:25:44.834 INFO:teuthology.orchestra.run.smithi076.stderr:bluestore allocator dump block 2023-11-25T04:25:44.834 INFO:teuthology.orchestra.run.smithi076.stderr:bluestore allocator fragmentation block 2023-11-25T04:25:44.834 INFO:teuthology.orchestra.run.smithi076.stderr:admin_socket: invalid command 2023-11-25T04:25:44.835 DEBUG:teuthology.orchestra.run:got remote process result: 22 2023-11-25T04:25:44.835 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-0, ['dump_ops_in_flight'] 2023-11-25T04:25:44.986 INFO:tasks.ceph.osd.0.smithi076.stderr:2023-11-25T04:25:44.985+0000 7f3e4fb48640 -1 osd.0 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-25T04:25:49.836 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-11-25T04:25:49.953 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2023-11-25T04:25:49.954 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2023-11-25T04:25:49.954 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2023-11-25T04:25:49.954 INFO:teuthology.orchestra.run.smithi076.stdout:} 2023-11-25T04:25:54.963 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2023-11-25T04:25:55.098 INFO:teuthology.orchestra.run.smithi067.stdout:mon.a: {} 2023-11-25T04:25:55.099 INFO:teuthology.orchestra.run.smithi067.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2023-11-25T04:25:55.107 INFO:teuthology.orchestra.run.smithi067.stdout:mon.b: {} 2023-11-25T04:25:55.107 INFO:teuthology.orchestra.run.smithi067.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2023-11-25T04:25:55.116 INFO:teuthology.orchestra.run.smithi067.stdout:mon.c: {} 2023-11-25T04:25:55.116 INFO:teuthology.orchestra.run.smithi067.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2023-11-25T04:25:55.127 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2023-11-25T04:25:55.947 INFO:teuthology.orchestra.run.smithi067.stderr:reweighted osd.1 to 1 (10000) 2023-11-25T04:25:55.962 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2023-11-25T04:25:56.954 INFO:teuthology.orchestra.run.smithi067.stderr:set osd.1 primary-affinity to 1 (8655362) 2023-11-25T04:25:56.970 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2023-11-25T04:25:57.951 INFO:teuthology.orchestra.run.smithi067.stderr:reweighted osd.2 to 1 (10000) 2023-11-25T04:25:57.970 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2023-11-25T04:25:58.966 INFO:teuthology.orchestra.run.smithi067.stderr:set osd.2 primary-affinity to 1 (8655362) 2023-11-25T04:25:58.983 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2023-11-25T04:25:59.979 INFO:teuthology.orchestra.run.smithi067.stderr:reweighted osd.0 to 1 (10000) 2023-11-25T04:25:59.996 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2023-11-25T04:26:00.987 INFO:teuthology.orchestra.run.smithi067.stderr:set osd.0 primary-affinity to 1 (8655362) 2023-11-25T04:26:01.005 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2023-11-25T04:26:01.005 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2023-11-25T04:26:01.006 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2023-11-25T04:26:01.006 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2023-11-25T04:26:01.006 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-11-25T04:26:01.006 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-25T04:26:01.306 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:26:01.307 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":75,"fsid":"a677f511-1143-4e76-8da6-5ddfadaff077","created":"2023-11-25T04:22:25.140474+0000","modified":"2023-11-25T04:26:00.979788+0000","last_up_change":"2023-11-25T04:25:45.970689+0000","last_in_change":"2023-11-25T04:22:25.985372+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-25T04:22:35.548359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-25T04:22:40.218527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-25T04:22:44.844906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-25T04:22:45.365536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bc397d35-53ea-430c-a96a-63a4160e64c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":20,"up_from":60,"up_thru":68,"down_at":21,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1420426974},{"type":"v1","addr":"172.21.15.76:6801","nonce":1420426974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1420426974},{"type":"v1","addr":"172.21.15.76:6803","nonce":1420426974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1420426974},{"type":"v1","addr":"172.21.15.76:6807","nonce":1420426974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1420426974},{"type":"v1","addr":"172.21.15.76:6805","nonce":1420426974}]},"public_addr":"172.21.15.76:6801/1420426974","cluster_addr":"172.21.15.76:6803/1420426974","heartbeat_back_addr":"172.21.15.76:6807/1420426974","heartbeat_front_addr":"172.21.15.76:6805/1420426974","state":["exists","up"]},{"osd":1,"uuid":"d4f1b88a-bbd4-4e58-9519-569489ced7b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6801","nonce":3130474108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6803","nonce":3130474108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6807","nonce":3130474108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6805","nonce":3130474108}]},"public_addr":"172.21.15.92:6801/3130474108","cluster_addr":"172.21.15.92:6803/3130474108","heartbeat_back_addr":"172.21.15.92:6807/3130474108","heartbeat_front_addr":"172.21.15.92:6805/3130474108","state":["exists","up"]},{"osd":2,"uuid":"55c82801-e090-4c10-91e5-5fa9dbfba36d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6801","nonce":4099978417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6803","nonce":4099978417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6807","nonce":4099978417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6805","nonce":4099978417}]},"public_addr":"172.21.15.195:6801/4099978417","cluster_addr":"172.21.15.195:6803/4099978417","heartbeat_back_addr":"172.21.15.195:6807/4099978417","heartbeat_front_addr":"172.21.15.195:6805/4099978417","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-25T04:22:31.039867+0000","dead_epoch":20},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-25T04:22:31.021863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-25T04:22:31.009821+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-25T04:26:01.319 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-11-25T04:26:01.319 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-25T04:26:01.615 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:26:01.615 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":75,"fsid":"a677f511-1143-4e76-8da6-5ddfadaff077","created":"2023-11-25T04:22:25.140474+0000","modified":"2023-11-25T04:26:00.979788+0000","last_up_change":"2023-11-25T04:25:45.970689+0000","last_in_change":"2023-11-25T04:22:25.985372+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-25T04:22:35.548359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-25T04:22:40.218527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-25T04:22:44.844906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-25T04:22:45.365536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bc397d35-53ea-430c-a96a-63a4160e64c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":20,"up_from":60,"up_thru":68,"down_at":21,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1420426974},{"type":"v1","addr":"172.21.15.76:6801","nonce":1420426974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1420426974},{"type":"v1","addr":"172.21.15.76:6803","nonce":1420426974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1420426974},{"type":"v1","addr":"172.21.15.76:6807","nonce":1420426974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1420426974},{"type":"v1","addr":"172.21.15.76:6805","nonce":1420426974}]},"public_addr":"172.21.15.76:6801/1420426974","cluster_addr":"172.21.15.76:6803/1420426974","heartbeat_back_addr":"172.21.15.76:6807/1420426974","heartbeat_front_addr":"172.21.15.76:6805/1420426974","state":["exists","up"]},{"osd":1,"uuid":"d4f1b88a-bbd4-4e58-9519-569489ced7b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6801","nonce":3130474108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6803","nonce":3130474108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6807","nonce":3130474108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6805","nonce":3130474108}]},"public_addr":"172.21.15.92:6801/3130474108","cluster_addr":"172.21.15.92:6803/3130474108","heartbeat_back_addr":"172.21.15.92:6807/3130474108","heartbeat_front_addr":"172.21.15.92:6805/3130474108","state":["exists","up"]},{"osd":2,"uuid":"55c82801-e090-4c10-91e5-5fa9dbfba36d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6801","nonce":4099978417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6803","nonce":4099978417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6807","nonce":4099978417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6805","nonce":4099978417}]},"public_addr":"172.21.15.195:6801/4099978417","cluster_addr":"172.21.15.195:6803/4099978417","heartbeat_back_addr":"172.21.15.195:6807/4099978417","heartbeat_front_addr":"172.21.15.195:6805/4099978417","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-25T04:22:31.039867+0000","dead_epoch":20},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-25T04:22:31.021863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-25T04:22:31.009821+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-25T04:26:01.628 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-11-25T04:26:01.629 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-11-25T04:26:01.629 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-11-25T04:26:01.778 INFO:teuthology.orchestra.run.smithi067.stdout:34359738414 2023-11-25T04:26:01.778 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-25T04:26:01.782 INFO:teuthology.orchestra.run.smithi067.stdout:257698037765 2023-11-25T04:26:01.783 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-25T04:26:01.783 INFO:teuthology.orchestra.run.smithi067.stdout:34359738414 2023-11-25T04:26:01.784 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-25T04:26:02.084 INFO:teuthology.orchestra.run.smithi067.stdout:34359738413 2023-11-25T04:26:02.098 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738414 got 34359738413 for osd.2 2023-11-25T04:26:02.132 INFO:teuthology.orchestra.run.smithi067.stdout:257698037764 2023-11-25T04:26:02.133 INFO:teuthology.orchestra.run.smithi067.stdout:34359738413 2023-11-25T04:26:02.146 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738414 got 34359738413 for osd.1 2023-11-25T04:26:02.147 INFO:tasks.ceph.ceph_manager.ceph:need seq 257698037765 got 257698037764 for osd.0 2023-11-25T04:26:03.099 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-25T04:26:03.148 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-25T04:26:03.148 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-25T04:26:03.417 INFO:teuthology.orchestra.run.smithi067.stdout:34359738413 2023-11-25T04:26:03.430 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738414 got 34359738413 for osd.2 2023-11-25T04:26:03.454 INFO:teuthology.orchestra.run.smithi067.stdout:34359738413 2023-11-25T04:26:03.461 INFO:teuthology.orchestra.run.smithi067.stdout:257698037764 2023-11-25T04:26:03.467 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738414 got 34359738413 for osd.1 2023-11-25T04:26:03.474 INFO:tasks.ceph.ceph_manager.ceph:need seq 257698037765 got 257698037764 for osd.0 2023-11-25T04:26:04.431 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-25T04:26:04.468 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-25T04:26:04.474 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-25T04:26:04.765 INFO:teuthology.orchestra.run.smithi067.stdout:34359738414 2023-11-25T04:26:04.783 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738414 got 34359738414 for osd.2 2023-11-25T04:26:04.783 DEBUG:teuthology.parallel:result is None 2023-11-25T04:26:04.798 INFO:teuthology.orchestra.run.smithi067.stdout:257698037765 2023-11-25T04:26:04.800 INFO:teuthology.orchestra.run.smithi067.stdout:34359738414 2023-11-25T04:26:04.812 INFO:tasks.ceph.ceph_manager.ceph:need seq 257698037765 got 257698037765 for osd.0 2023-11-25T04:26:04.812 DEBUG:teuthology.parallel:result is None 2023-11-25T04:26:04.814 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738414 got 34359738414 for osd.1 2023-11-25T04:26:04.814 DEBUG:teuthology.parallel:result is None 2023-11-25T04:26:04.815 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2023-11-25T04:26:04.815 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-25T04:26:05.155 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:26:05.156 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2023-11-25T04:26:05.166 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":178,"stamp":"2023-11-25T04:26:03.549604+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":67806771,"num_objects":3481,"num_object_clones":0,"num_object_copies":6962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3481,"num_whiteouts":0,"num_read":171,"num_read_kb":452,"num_write":11061,"num_write_kb":82472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4201,"num_bytes_recovered":45240620,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10838,"ondisk_log_size":10838,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":247712,"kb_used_data":153652,"kb_used_omap":10,"kb_used_meta":94005,"kb_avail":282867808,"statfs":{"total":289910292480,"available":289656635392,"internally_reserved":0,"allocated":157339648,"data_stored":136189732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10333,"internal_metadata":96262051},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.595710"},"pg_stats":[{"pgid":"4.3f","version":"54'166","reported_seq":234,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979173+0000","last_change":"2023-11-25T04:22:47.039847+0000","last_active":"2023-11-25T04:25:59.979173+0000","last_peered":"2023-11-25T04:25:59.979173+0000","last_clean":"2023-11-25T04:25:59.979173+0000","last_became_active":"2023-11-25T04:22:47.039490+0000","last_became_peered":"2023-11-25T04:22:47.039490+0000","last_unstale":"2023-11-25T04:25:59.979173+0000","last_undegraded":"2023-11-25T04:25:59.979173+0000","last_fullsized":"2023-11-25T04:25:59.979173+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:25:06.132517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":577866,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":174,"num_write_kb":756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3e","version":"54'212","reported_seq":280,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978984+0000","last_change":"2023-11-25T04:22:47.040836+0000","last_active":"2023-11-25T04:25:59.978984+0000","last_peered":"2023-11-25T04:25:59.978984+0000","last_clean":"2023-11-25T04:25:59.978984+0000","last_became_active":"2023-11-25T04:22:47.040511+0000","last_became_peered":"2023-11-25T04:22:47.040511+0000","last_unstale":"2023-11-25T04:25:59.978984+0000","last_undegraded":"2023-11-25T04:25:59.978984+0000","last_fullsized":"2023-11-25T04:25:59.978984+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:15:39.511043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":461579,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":236,"num_write_kb":841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3d","version":"54'167","reported_seq":233,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978931+0000","last_change":"2023-11-25T04:25:52.924678+0000","last_active":"2023-11-25T04:25:59.978931+0000","last_peered":"2023-11-25T04:25:59.978931+0000","last_clean":"2023-11-25T04:25:59.978931+0000","last_became_active":"2023-11-25T04:25:52.923459+0000","last_became_peered":"2023-11-25T04:25:52.923459+0000","last_unstale":"2023-11-25T04:25:59.978931+0000","last_undegraded":"2023-11-25T04:25:59.978931+0000","last_fullsized":"2023-11-25T04:25:59.978931+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:24:06.950438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":145844,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":175,"num_write_kb":312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":138765,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"54'124","reported_seq":191,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978386+0000","last_change":"2023-11-25T04:25:51.922843+0000","last_active":"2023-11-25T04:25:59.978386+0000","last_peered":"2023-11-25T04:25:59.978386+0000","last_clean":"2023-11-25T04:25:59.978386+0000","last_became_active":"2023-11-25T04:25:51.921633+0000","last_became_peered":"2023-11-25T04:25:51.921633+0000","last_unstale":"2023-11-25T04:25:59.978386+0000","last_undegraded":"2023-11-25T04:25:59.978386+0000","last_fullsized":"2023-11-25T04:25:59.978386+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:05:31.332951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61303,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119,"num_write_kb":161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":61303,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"54'169","reported_seq":237,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978640+0000","last_change":"2023-11-25T04:22:47.039499+0000","last_active":"2023-11-25T04:25:59.978640+0000","last_peered":"2023-11-25T04:25:59.978640+0000","last_clean":"2023-11-25T04:25:59.978640+0000","last_became_active":"2023-11-25T04:22:47.039165+0000","last_became_peered":"2023-11-25T04:22:47.039165+0000","last_unstale":"2023-11-25T04:25:59.978640+0000","last_undegraded":"2023-11-25T04:25:59.978640+0000","last_fullsized":"2023-11-25T04:25:59.978640+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:57:28.855151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":118227,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":185,"num_write_kb":351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3a","version":"54'169","reported_seq":234,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978863+0000","last_change":"2023-11-25T04:25:53.933198+0000","last_active":"2023-11-25T04:25:59.978863+0000","last_peered":"2023-11-25T04:25:59.978863+0000","last_clean":"2023-11-25T04:25:59.978863+0000","last_became_active":"2023-11-25T04:25:53.931982+0000","last_became_peered":"2023-11-25T04:25:53.931982+0000","last_unstale":"2023-11-25T04:25:59.978863+0000","last_undegraded":"2023-11-25T04:25:59.978863+0000","last_fullsized":"2023-11-25T04:25:59.978863+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:28:40.449148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100304,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":161,"num_write_kb":424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":100304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"54'168","reported_seq":237,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978380+0000","last_change":"2023-11-25T04:25:49.921033+0000","last_active":"2023-11-25T04:25:59.978380+0000","last_peered":"2023-11-25T04:25:59.978380+0000","last_clean":"2023-11-25T04:25:59.978380+0000","last_became_active":"2023-11-25T04:25:49.919354+0000","last_became_peered":"2023-11-25T04:25:49.919354+0000","last_unstale":"2023-11-25T04:25:59.978380+0000","last_undegraded":"2023-11-25T04:25:59.978380+0000","last_fullsized":"2023-11-25T04:25:59.978380+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:36:13.489397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":124558,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":167,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":111,"num_bytes_recovered":124533,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.38","version":"54'152","reported_seq":221,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978153+0000","last_change":"2023-11-25T04:25:49.920421+0000","last_active":"2023-11-25T04:25:59.978153+0000","last_peered":"2023-11-25T04:25:59.978153+0000","last_clean":"2023-11-25T04:25:59.978153+0000","last_became_active":"2023-11-25T04:25:49.919236+0000","last_became_peered":"2023-11-25T04:25:49.919236+0000","last_unstale":"2023-11-25T04:25:59.978153+0000","last_undegraded":"2023-11-25T04:25:59.978153+0000","last_fullsized":"2023-11-25T04:25:59.978153+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:32:41.956579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":120147,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":120121,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.37","version":"54'167","reported_seq":235,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978818+0000","last_change":"2023-11-25T04:22:47.038397+0000","last_active":"2023-11-25T04:25:59.978818+0000","last_peered":"2023-11-25T04:25:59.978818+0000","last_clean":"2023-11-25T04:25:59.978818+0000","last_became_active":"2023-11-25T04:22:47.037328+0000","last_became_peered":"2023-11-25T04:22:47.037328+0000","last_unstale":"2023-11-25T04:25:59.978818+0000","last_undegraded":"2023-11-25T04:25:59.978818+0000","last_fullsized":"2023-11-25T04:25:59.978818+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:56:59.689578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":174147,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":184,"num_write_kb":353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.36","version":"54'149","reported_seq":217,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979047+0000","last_change":"2023-11-25T04:22:47.040182+0000","last_active":"2023-11-25T04:25:59.979047+0000","last_peered":"2023-11-25T04:25:59.979047+0000","last_clean":"2023-11-25T04:25:59.979047+0000","last_became_active":"2023-11-25T04:22:47.039892+0000","last_became_peered":"2023-11-25T04:22:47.039892+0000","last_unstale":"2023-11-25T04:25:59.979047+0000","last_undegraded":"2023-11-25T04:25:59.979047+0000","last_fullsized":"2023-11-25T04:25:59.979047+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:38:09.176582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":233330,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"54'156","reported_seq":223,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979021+0000","last_change":"2023-11-25T04:25:51.923087+0000","last_active":"2023-11-25T04:25:59.979021+0000","last_peered":"2023-11-25T04:25:59.979021+0000","last_clean":"2023-11-25T04:25:59.979021+0000","last_became_active":"2023-11-25T04:25:51.921800+0000","last_became_peered":"2023-11-25T04:25:51.921800+0000","last_unstale":"2023-11-25T04:25:59.979021+0000","last_undegraded":"2023-11-25T04:25:59.979021+0000","last_fullsized":"2023-11-25T04:25:59.979021+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:29:47.766517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":302550,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":163,"num_write_kb":431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":300323,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.34","version":"54'136","reported_seq":400,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978706+0000","last_change":"2023-11-25T04:25:50.922843+0000","last_active":"2023-11-25T04:25:59.978706+0000","last_peered":"2023-11-25T04:25:59.978706+0000","last_clean":"2023-11-25T04:25:59.978706+0000","last_became_active":"2023-11-25T04:25:50.922309+0000","last_became_peered":"2023-11-25T04:25:50.922309+0000","last_unstale":"2023-11-25T04:25:59.978706+0000","last_undegraded":"2023-11-25T04:25:59.978706+0000","last_fullsized":"2023-11-25T04:25:59.978706+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:38:33.939562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":93053,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":131,"num_write_kb":248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":89,"num_bytes_recovered":93053,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"54'134","reported_seq":202,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978529+0000","last_change":"2023-11-25T04:22:47.040388+0000","last_active":"2023-11-25T04:25:59.978529+0000","last_peered":"2023-11-25T04:25:59.978529+0000","last_clean":"2023-11-25T04:25:59.978529+0000","last_became_active":"2023-11-25T04:22:47.040129+0000","last_became_peered":"2023-11-25T04:22:47.040129+0000","last_unstale":"2023-11-25T04:25:59.978529+0000","last_undegraded":"2023-11-25T04:25:59.978529+0000","last_fullsized":"2023-11-25T04:25:59.978529+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:31:54.857270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":149524,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":139,"num_write_kb":252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.32","version":"54'174","reported_seq":239,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978762+0000","last_change":"2023-11-25T04:25:53.934526+0000","last_active":"2023-11-25T04:25:59.978762+0000","last_peered":"2023-11-25T04:25:59.978762+0000","last_clean":"2023-11-25T04:25:59.978762+0000","last_became_active":"2023-11-25T04:25:53.933694+0000","last_became_peered":"2023-11-25T04:25:53.933694+0000","last_unstale":"2023-11-25T04:25:59.978762+0000","last_undegraded":"2023-11-25T04:25:59.978762+0000","last_fullsized":"2023-11-25T04:25:59.978762+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:01:16.858976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":95797,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":181,"num_write_kb":547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":103,"num_bytes_recovered":95797,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.31","version":"54'156","reported_seq":225,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978209+0000","last_change":"2023-11-25T04:25:49.922396+0000","last_active":"2023-11-25T04:25:59.978209+0000","last_peered":"2023-11-25T04:25:59.978209+0000","last_clean":"2023-11-25T04:25:59.978209+0000","last_became_active":"2023-11-25T04:25:49.921129+0000","last_became_peered":"2023-11-25T04:25:49.921129+0000","last_unstale":"2023-11-25T04:25:59.978209+0000","last_undegraded":"2023-11-25T04:25:59.978209+0000","last_fullsized":"2023-11-25T04:25:59.978209+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:57:02.387207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":163190,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":169,"num_write_kb":336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":154576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"54'174","reported_seq":239,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978986+0000","last_change":"2023-11-25T04:25:53.934614+0000","last_active":"2023-11-25T04:25:59.978986+0000","last_peered":"2023-11-25T04:25:59.978986+0000","last_clean":"2023-11-25T04:25:59.978986+0000","last_became_active":"2023-11-25T04:25:53.933395+0000","last_became_peered":"2023-11-25T04:25:53.933395+0000","last_unstale":"2023-11-25T04:25:59.978986+0000","last_undegraded":"2023-11-25T04:25:59.978986+0000","last_fullsized":"2023-11-25T04:25:59.978986+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:53:09.827809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1423587,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":182,"num_write_kb":1626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":1414508,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2f","version":"54'152","reported_seq":220,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979084+0000","last_change":"2023-11-25T04:22:47.039389+0000","last_active":"2023-11-25T04:25:59.979084+0000","last_peered":"2023-11-25T04:25:59.979084+0000","last_clean":"2023-11-25T04:25:59.979084+0000","last_became_active":"2023-11-25T04:22:47.039123+0000","last_became_peered":"2023-11-25T04:22:47.039123+0000","last_unstale":"2023-11-25T04:25:59.979084+0000","last_undegraded":"2023-11-25T04:25:59.979084+0000","last_fullsized":"2023-11-25T04:25:59.979084+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:09:11.570561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":122752,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":143,"num_write_kb":435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"54'148","reported_seq":216,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979169+0000","last_change":"2023-11-25T04:22:47.036946+0000","last_active":"2023-11-25T04:25:59.979169+0000","last_peered":"2023-11-25T04:25:59.979169+0000","last_clean":"2023-11-25T04:25:59.979169+0000","last_became_active":"2023-11-25T04:22:47.036411+0000","last_became_peered":"2023-11-25T04:22:47.036411+0000","last_unstale":"2023-11-25T04:25:59.979169+0000","last_undegraded":"2023-11-25T04:25:59.979169+0000","last_fullsized":"2023-11-25T04:25:59.979169+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:16:29.085050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4770899,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":140,"num_write_kb":4870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2d","version":"54'143","reported_seq":211,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978472+0000","last_change":"2023-11-25T04:22:47.038983+0000","last_active":"2023-11-25T04:25:59.978472+0000","last_peered":"2023-11-25T04:25:59.978472+0000","last_clean":"2023-11-25T04:25:59.978472+0000","last_became_active":"2023-11-25T04:22:47.038477+0000","last_became_peered":"2023-11-25T04:22:47.038477+0000","last_unstale":"2023-11-25T04:25:59.978472+0000","last_undegraded":"2023-11-25T04:25:59.978472+0000","last_fullsized":"2023-11-25T04:25:59.978472+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:30:04.568790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":115975,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"54'167","reported_seq":233,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978615+0000","last_change":"2023-11-25T04:25:52.923640+0000","last_active":"2023-11-25T04:25:59.978615+0000","last_peered":"2023-11-25T04:25:59.978615+0000","last_clean":"2023-11-25T04:25:59.978615+0000","last_became_active":"2023-11-25T04:25:52.922367+0000","last_became_peered":"2023-11-25T04:25:52.922367+0000","last_unstale":"2023-11-25T04:25:59.978615+0000","last_undegraded":"2023-11-25T04:25:59.978615+0000","last_fullsized":"2023-11-25T04:25:59.978615+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:08:24.794280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":747090,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":211,"num_write_kb":985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":732276,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"54'156","reported_seq":222,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978632+0000","last_change":"2023-11-25T04:25:52.924457+0000","last_active":"2023-11-25T04:25:59.978632+0000","last_peered":"2023-11-25T04:25:59.978632+0000","last_clean":"2023-11-25T04:25:59.978632+0000","last_became_active":"2023-11-25T04:25:52.923378+0000","last_became_peered":"2023-11-25T04:25:52.923378+0000","last_unstale":"2023-11-25T04:25:59.978632+0000","last_undegraded":"2023-11-25T04:25:59.978632+0000","last_fullsized":"2023-11-25T04:25:59.978632+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:34:30.942779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69580,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":142,"num_write_kb":315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":98,"num_bytes_recovered":69580,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2a","version":"54'171","reported_seq":467,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978347+0000","last_change":"2023-11-25T04:25:52.925532+0000","last_active":"2023-11-25T04:25:59.978347+0000","last_peered":"2023-11-25T04:25:59.978347+0000","last_clean":"2023-11-25T04:25:59.978347+0000","last_became_active":"2023-11-25T04:25:52.924844+0000","last_became_peered":"2023-11-25T04:25:52.924844+0000","last_unstale":"2023-11-25T04:25:59.978347+0000","last_undegraded":"2023-11-25T04:25:59.978347+0000","last_fullsized":"2023-11-25T04:25:59.978347+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:11:30.124326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":115997,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":162,"num_write_kb":261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":115997,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.29","version":"54'152","reported_seq":438,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.977803+0000","last_change":"2023-11-25T04:25:49.920367+0000","last_active":"2023-11-25T04:25:59.977803+0000","last_peered":"2023-11-25T04:25:59.977803+0000","last_clean":"2023-11-25T04:25:59.977803+0000","last_became_active":"2023-11-25T04:25:49.919886+0000","last_became_peered":"2023-11-25T04:25:49.919886+0000","last_unstale":"2023-11-25T04:25:59.977803+0000","last_undegraded":"2023-11-25T04:25:59.977803+0000","last_fullsized":"2023-11-25T04:25:59.977803+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:04:58.273982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":212878,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":212878,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"54'169","reported_seq":237,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978222+0000","last_change":"2023-11-25T04:22:47.036805+0000","last_active":"2023-11-25T04:25:59.978222+0000","last_peered":"2023-11-25T04:25:59.978222+0000","last_clean":"2023-11-25T04:25:59.978222+0000","last_became_active":"2023-11-25T04:22:47.036135+0000","last_became_peered":"2023-11-25T04:22:47.036135+0000","last_unstale":"2023-11-25T04:25:59.978222+0000","last_undegraded":"2023-11-25T04:25:59.978222+0000","last_fullsized":"2023-11-25T04:25:59.978222+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:25:21.730992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":102957,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":172,"num_write_kb":610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"54'167","reported_seq":465,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979220+0000","last_change":"2023-11-25T04:25:51.922925+0000","last_active":"2023-11-25T04:25:59.979220+0000","last_peered":"2023-11-25T04:25:59.979220+0000","last_clean":"2023-11-25T04:25:59.979220+0000","last_became_active":"2023-11-25T04:25:51.922364+0000","last_became_peered":"2023-11-25T04:25:51.922364+0000","last_unstale":"2023-11-25T04:25:59.979220+0000","last_undegraded":"2023-11-25T04:25:59.979220+0000","last_fullsized":"2023-11-25T04:25:59.979220+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:09:23.741871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":182683,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":171,"num_write_kb":791,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":182683,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"54'187","reported_seq":505,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978723+0000","last_change":"2023-11-25T04:25:51.922529+0000","last_active":"2023-11-25T04:25:59.978723+0000","last_peered":"2023-11-25T04:25:59.978723+0000","last_clean":"2023-11-25T04:25:59.978723+0000","last_became_active":"2023-11-25T04:25:51.922064+0000","last_became_peered":"2023-11-25T04:25:51.922064+0000","last_unstale":"2023-11-25T04:25:59.978723+0000","last_undegraded":"2023-11-25T04:25:59.978723+0000","last_fullsized":"2023-11-25T04:25:59.978723+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":187,"log_dups_size":0,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:01:39.413986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":86311,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":171,"num_write_kb":274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":116,"num_bytes_recovered":86311,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.25","version":"54'149","reported_seq":215,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978829+0000","last_change":"2023-11-25T04:25:52.925505+0000","last_active":"2023-11-25T04:25:59.978829+0000","last_peered":"2023-11-25T04:25:59.978829+0000","last_clean":"2023-11-25T04:25:59.978829+0000","last_became_active":"2023-11-25T04:25:52.924795+0000","last_became_peered":"2023-11-25T04:25:52.924795+0000","last_unstale":"2023-11-25T04:25:59.978829+0000","last_undegraded":"2023-11-25T04:25:59.978829+0000","last_fullsized":"2023-11-25T04:25:59.978829+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:30:46.532141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":128288,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":167,"num_write_kb":267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":120265,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.24","version":"54'151","reported_seq":427,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978903+0000","last_change":"2023-11-25T04:25:50.922751+0000","last_active":"2023-11-25T04:25:59.978903+0000","last_peered":"2023-11-25T04:25:59.978903+0000","last_clean":"2023-11-25T04:25:59.978903+0000","last_became_active":"2023-11-25T04:25:50.922216+0000","last_became_peered":"2023-11-25T04:25:50.922216+0000","last_unstale":"2023-11-25T04:25:59.978903+0000","last_undegraded":"2023-11-25T04:25:59.978903+0000","last_fullsized":"2023-11-25T04:25:59.978903+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:40:16.255665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":107507,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":148,"num_write_kb":288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":107494,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.23","version":"54'155","reported_seq":223,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978207+0000","last_change":"2023-11-25T04:22:47.037668+0000","last_active":"2023-11-25T04:25:59.978207+0000","last_peered":"2023-11-25T04:25:59.978207+0000","last_clean":"2023-11-25T04:25:59.978207+0000","last_became_active":"2023-11-25T04:22:47.036823+0000","last_became_peered":"2023-11-25T04:22:47.036823+0000","last_unstale":"2023-11-25T04:25:59.978207+0000","last_undegraded":"2023-11-25T04:25:59.978207+0000","last_fullsized":"2023-11-25T04:25:59.978207+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:02:07.088241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4304582,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":152,"num_write_kb":4331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.22","version":"54'189","reported_seq":517,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978625+0000","last_change":"2023-11-25T04:25:54.940559+0000","last_active":"2023-11-25T04:25:59.978625+0000","last_peered":"2023-11-25T04:25:59.978625+0000","last_clean":"2023-11-25T04:25:59.978625+0000","last_became_active":"2023-11-25T04:25:54.939584+0000","last_became_peered":"2023-11-25T04:25:54.939584+0000","last_unstale":"2023-11-25T04:25:59.978625+0000","last_undegraded":"2023-11-25T04:25:59.978625+0000","last_fullsized":"2023-11-25T04:25:59.978625+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:01:39.413986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":129145,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":185,"num_write_kb":432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":121,"num_bytes_recovered":129107,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"54'201","reported_seq":269,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.977881+0000","last_change":"2023-11-25T04:22:47.037786+0000","last_active":"2023-11-25T04:25:59.977881+0000","last_peered":"2023-11-25T04:25:59.977881+0000","last_clean":"2023-11-25T04:25:59.977881+0000","last_became_active":"2023-11-25T04:22:47.037206+0000","last_became_peered":"2023-11-25T04:22:47.037206+0000","last_unstale":"2023-11-25T04:25:59.977881+0000","last_undegraded":"2023-11-25T04:25:59.977881+0000","last_fullsized":"2023-11-25T04:25:59.977881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:11:40.532140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":215311,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":194,"num_write_kb":381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"54'180","reported_seq":245,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.977884+0000","last_change":"2023-11-25T04:25:53.933176+0000","last_active":"2023-11-25T04:25:59.977884+0000","last_peered":"2023-11-25T04:25:59.977884+0000","last_clean":"2023-11-25T04:25:59.977884+0000","last_became_active":"2023-11-25T04:25:53.931937+0000","last_became_peered":"2023-11-25T04:25:53.931937+0000","last_unstale":"2023-11-25T04:25:59.977884+0000","last_undegraded":"2023-11-25T04:25:59.977884+0000","last_fullsized":"2023-11-25T04:25:59.977884+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:41:10.697064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104603,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":172,"num_write_kb":267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":104603,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"54'137","reported_seq":205,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978811+0000","last_change":"2023-11-25T04:22:47.036689+0000","last_active":"2023-11-25T04:25:59.978811+0000","last_peered":"2023-11-25T04:25:59.978811+0000","last_clean":"2023-11-25T04:25:59.978811+0000","last_became_active":"2023-11-25T04:22:47.036107+0000","last_became_peered":"2023-11-25T04:22:47.036107+0000","last_unstale":"2023-11-25T04:25:59.978811+0000","last_undegraded":"2023-11-25T04:25:59.978811+0000","last_fullsized":"2023-11-25T04:25:59.978811+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:02:07.088241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":183586,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":127,"num_write_kb":284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"58'114","reported_seq":191,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978227+0000","last_change":"2023-11-25T04:25:49.920320+0000","last_active":"2023-11-25T04:25:59.978227+0000","last_peered":"2023-11-25T04:25:59.978227+0000","last_clean":"2023-11-25T04:25:59.978227+0000","last_became_active":"2023-11-25T04:25:49.920010+0000","last_became_peered":"2023-11-25T04:25:49.920010+0000","last_unstale":"2023-11-25T04:25:59.978227+0000","last_undegraded":"2023-11-25T04:25:59.978227+0000","last_fullsized":"2023-11-25T04:25:59.978227+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:48:58.746789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2816216,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":113,"num_write_kb":2816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":2816182,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"54'145","reported_seq":213,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978995+0000","last_change":"2023-11-25T04:22:47.037888+0000","last_active":"2023-11-25T04:25:59.978995+0000","last_peered":"2023-11-25T04:25:59.978995+0000","last_clean":"2023-11-25T04:25:59.978995+0000","last_became_active":"2023-11-25T04:22:47.037158+0000","last_became_peered":"2023-11-25T04:22:47.037158+0000","last_unstale":"2023-11-25T04:25:59.978995+0000","last_undegraded":"2023-11-25T04:25:59.978995+0000","last_fullsized":"2023-11-25T04:25:59.978995+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:56:59.689578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":205976,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.12","version":"49'2","reported_seq":82,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978301+0000","last_change":"2023-11-25T04:25:47.000916+0000","last_active":"2023-11-25T04:25:59.978301+0000","last_peered":"2023-11-25T04:25:59.978301+0000","last_clean":"2023-11-25T04:25:59.978301+0000","last_became_active":"2023-11-25T04:25:47.000516+0000","last_became_peered":"2023-11-25T04:25:47.000516+0000","last_unstale":"2023-11-25T04:25:59.978301+0000","last_undegraded":"2023-11-25T04:25:59.978301+0000","last_fullsized":"2023-11-25T04:25:59.978301+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:53:58.873447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"54'175","reported_seq":489,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978855+0000","last_change":"2023-11-25T04:25:52.923694+0000","last_active":"2023-11-25T04:25:59.978855+0000","last_peered":"2023-11-25T04:25:59.978855+0000","last_clean":"2023-11-25T04:25:59.978855+0000","last_became_active":"2023-11-25T04:25:52.923045+0000","last_became_peered":"2023-11-25T04:25:52.923045+0000","last_unstale":"2023-11-25T04:25:59.978855+0000","last_undegraded":"2023-11-25T04:25:59.978855+0000","last_fullsized":"2023-11-25T04:25:59.978855+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:46:12.914433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":171823,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":185,"num_write_kb":340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":166387,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":80,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978355+0000","last_change":"2023-11-25T04:25:46.994324+0000","last_active":"2023-11-25T04:25:59.978355+0000","last_peered":"2023-11-25T04:25:59.978355+0000","last_clean":"2023-11-25T04:25:59.978355+0000","last_became_active":"2023-11-25T04:25:46.993870+0000","last_became_peered":"2023-11-25T04:25:46.993870+0000","last_unstale":"2023-11-25T04:25:59.978355+0000","last_undegraded":"2023-11-25T04:25:59.978355+0000","last_fullsized":"2023-11-25T04:25:59.978355+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:51:09.827080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"54'152","reported_seq":435,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978827+0000","last_change":"2023-11-25T04:25:50.922922+0000","last_active":"2023-11-25T04:25:59.978827+0000","last_peered":"2023-11-25T04:25:59.978827+0000","last_clean":"2023-11-25T04:25:59.978827+0000","last_became_active":"2023-11-25T04:25:50.922511+0000","last_became_peered":"2023-11-25T04:25:50.922511+0000","last_unstale":"2023-11-25T04:25:59.978827+0000","last_undegraded":"2023-11-25T04:25:59.978827+0000","last_fullsized":"2023-11-25T04:25:59.978827+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:37:14.620534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":579704,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":97,"num_bytes_recovered":570044,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.10","version":"18'8","reported_seq":39,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.977878+0000","last_change":"2023-11-25T04:25:47.000499+0000","last_active":"2023-11-25T04:25:59.977878+0000","last_peered":"2023-11-25T04:25:59.977878+0000","last_clean":"2023-11-25T04:25:59.977878+0000","last_became_active":"2023-11-25T04:25:46.998778+0000","last_became_peered":"2023-11-25T04:25:46.998778+0000","last_unstale":"2023-11-25T04:25:59.977878+0000","last_undegraded":"2023-11-25T04:25:59.977878+0000","last_fullsized":"2023-11-25T04:25:59.977878+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:33:09.169689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"49'2","reported_seq":37,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979103+0000","last_change":"2023-11-25T04:25:46.998392+0000","last_active":"2023-11-25T04:25:59.979103+0000","last_peered":"2023-11-25T04:25:59.979103+0000","last_clean":"2023-11-25T04:25:59.979103+0000","last_became_active":"2023-11-25T04:25:46.997108+0000","last_became_peered":"2023-11-25T04:25:46.997108+0000","last_unstale":"2023-11-25T04:25:59.979103+0000","last_undegraded":"2023-11-25T04:25:59.979103+0000","last_fullsized":"2023-11-25T04:25:59.979103+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:02:00.073630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"54'159","reported_seq":443,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978233+0000","last_change":"2023-11-25T04:25:50.922282+0000","last_active":"2023-11-25T04:25:59.978233+0000","last_peered":"2023-11-25T04:25:59.978233+0000","last_clean":"2023-11-25T04:25:59.978233+0000","last_became_active":"2023-11-25T04:25:50.921772+0000","last_became_peered":"2023-11-25T04:25:50.921772+0000","last_unstale":"2023-11-25T04:25:59.978233+0000","last_undegraded":"2023-11-25T04:25:59.978233+0000","last_fullsized":"2023-11-25T04:25:59.978233+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:36:33.451689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112103,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":164,"num_write_kb":240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":99,"num_bytes_recovered":103472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"18'2","reported_seq":82,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979281+0000","last_change":"2023-11-25T04:25:46.994943+0000","last_active":"2023-11-25T04:25:59.979281+0000","last_peered":"2023-11-25T04:25:59.979281+0000","last_clean":"2023-11-25T04:25:59.979281+0000","last_became_active":"2023-11-25T04:25:46.994640+0000","last_became_peered":"2023-11-25T04:25:46.994640+0000","last_unstale":"2023-11-25T04:25:59.979281+0000","last_undegraded":"2023-11-25T04:25:59.979281+0000","last_fullsized":"2023-11-25T04:25:59.979281+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:08:12.601941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"54'173","reported_seq":241,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978244+0000","last_change":"2023-11-25T04:22:47.039685+0000","last_active":"2023-11-25T04:25:59.978244+0000","last_peered":"2023-11-25T04:25:59.978244+0000","last_clean":"2023-11-25T04:25:59.978244+0000","last_became_active":"2023-11-25T04:22:47.039296+0000","last_became_peered":"2023-11-25T04:22:47.039296+0000","last_unstale":"2023-11-25T04:25:59.978244+0000","last_undegraded":"2023-11-25T04:25:59.978244+0000","last_fullsized":"2023-11-25T04:25:59.978244+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:11:40.532140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154533,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":182,"num_write_kb":299,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"18'1","reported_seq":37,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978725+0000","last_change":"2023-11-25T04:25:47.000059+0000","last_active":"2023-11-25T04:25:59.978725+0000","last_peered":"2023-11-25T04:25:59.978725+0000","last_clean":"2023-11-25T04:25:59.978725+0000","last_became_active":"2023-11-25T04:25:46.999604+0000","last_became_peered":"2023-11-25T04:25:46.999604+0000","last_unstale":"2023-11-25T04:25:59.978725+0000","last_undegraded":"2023-11-25T04:25:59.978725+0000","last_fullsized":"2023-11-25T04:25:59.978725+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:14:43.431272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"54'147","reported_seq":211,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978982+0000","last_change":"2023-11-25T04:25:54.939274+0000","last_active":"2023-11-25T04:25:59.978982+0000","last_peered":"2023-11-25T04:25:59.978982+0000","last_clean":"2023-11-25T04:25:59.978982+0000","last_became_active":"2023-11-25T04:25:54.938236+0000","last_became_peered":"2023-11-25T04:25:54.938236+0000","last_unstale":"2023-11-25T04:25:59.978982+0000","last_undegraded":"2023-11-25T04:25:59.978982+0000","last_fullsized":"2023-11-25T04:25:59.978982+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":147,"log_dups_size":0,"ondisk_log_size":147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:11:43.785294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":87524,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":155,"num_write_kb":235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":87524,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978546+0000","last_change":"2023-11-25T04:25:46.997080+0000","last_active":"2023-11-25T04:25:59.978546+0000","last_peered":"2023-11-25T04:25:59.978546+0000","last_clean":"2023-11-25T04:25:59.978546+0000","last_became_active":"2023-11-25T04:25:46.995712+0000","last_became_peered":"2023-11-25T04:25:46.995712+0000","last_unstale":"2023-11-25T04:25:59.978546+0000","last_undegraded":"2023-11-25T04:25:59.978546+0000","last_fullsized":"2023-11-25T04:25:59.978546+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T16:01:00.922843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"54'155","reported_seq":441,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978274+0000","last_change":"2023-11-25T04:25:52.923096+0000","last_active":"2023-11-25T04:25:59.978274+0000","last_peered":"2023-11-25T04:25:59.978274+0000","last_clean":"2023-11-25T04:25:59.978274+0000","last_became_active":"2023-11-25T04:25:52.922549+0000","last_became_peered":"2023-11-25T04:25:52.922549+0000","last_unstale":"2023-11-25T04:25:59.978274+0000","last_undegraded":"2023-11-25T04:25:59.978274+0000","last_fullsized":"2023-11-25T04:25:59.978274+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:52:38.053808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":266919,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":264258,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"45'11","reported_seq":96,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978532+0000","last_change":"2023-11-25T04:25:47.092497+0000","last_active":"2023-11-25T04:25:59.978532+0000","last_peered":"2023-11-25T04:25:59.978532+0000","last_clean":"2023-11-25T04:25:59.978532+0000","last_became_active":"2023-11-25T04:25:47.000687+0000","last_became_peered":"2023-11-25T04:25:47.000687+0000","last_unstale":"2023-11-25T04:25:59.978532+0000","last_undegraded":"2023-11-25T04:25:59.978532+0000","last_fullsized":"2023-11-25T04:25:59.978532+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:47:09.798238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":4101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4194304,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"54'173","reported_seq":241,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978732+0000","last_change":"2023-11-25T04:25:50.923240+0000","last_active":"2023-11-25T04:25:59.978732+0000","last_peered":"2023-11-25T04:25:59.978732+0000","last_clean":"2023-11-25T04:25:59.978732+0000","last_became_active":"2023-11-25T04:25:50.922409+0000","last_became_peered":"2023-11-25T04:25:50.922409+0000","last_unstale":"2023-11-25T04:25:59.978732+0000","last_undegraded":"2023-11-25T04:25:59.978732+0000","last_fullsized":"2023-11-25T04:25:59.978732+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:53:30.920627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":553893,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":169,"num_write_kb":770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":553893,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978079+0000","last_change":"2023-11-25T04:25:46.998818+0000","last_active":"2023-11-25T04:25:59.978079+0000","last_peered":"2023-11-25T04:25:59.978079+0000","last_clean":"2023-11-25T04:25:59.978079+0000","last_became_active":"2023-11-25T04:25:46.997019+0000","last_became_peered":"2023-11-25T04:25:46.997019+0000","last_unstale":"2023-11-25T04:25:59.978079+0000","last_undegraded":"2023-11-25T04:25:59.978079+0000","last_fullsized":"2023-11-25T04:25:59.978079+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:51:47.882117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"54'163","reported_seq":228,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978685+0000","last_change":"2023-11-25T04:25:53.934183+0000","last_active":"2023-11-25T04:25:59.978685+0000","last_peered":"2023-11-25T04:25:59.978685+0000","last_clean":"2023-11-25T04:25:59.978685+0000","last_became_active":"2023-11-25T04:25:53.933471+0000","last_became_peered":"2023-11-25T04:25:53.933471+0000","last_unstale":"2023-11-25T04:25:59.978685+0000","last_undegraded":"2023-11-25T04:25:59.978685+0000","last_fullsized":"2023-11-25T04:25:59.978685+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:57:07.515414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":320867,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":190,"num_write_kb":577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":108,"num_bytes_recovered":311640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":80,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978412+0000","last_change":"2023-11-25T04:25:46.994480+0000","last_active":"2023-11-25T04:25:59.978412+0000","last_peered":"2023-11-25T04:25:59.978412+0000","last_clean":"2023-11-25T04:25:59.978412+0000","last_became_active":"2023-11-25T04:25:46.994101+0000","last_became_peered":"2023-11-25T04:25:46.994101+0000","last_unstale":"2023-11-25T04:25:59.978412+0000","last_undegraded":"2023-11-25T04:25:59.978412+0000","last_fullsized":"2023-11-25T04:25:59.978412+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:53:58.873447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"54'141","reported_seq":210,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979040+0000","last_change":"2023-11-25T04:25:49.921950+0000","last_active":"2023-11-25T04:25:59.979040+0000","last_peered":"2023-11-25T04:25:59.979040+0000","last_clean":"2023-11-25T04:25:59.979040+0000","last_became_active":"2023-11-25T04:25:49.920971+0000","last_became_peered":"2023-11-25T04:25:49.920971+0000","last_unstale":"2023-11-25T04:25:59.979040+0000","last_undegraded":"2023-11-25T04:25:59.979040+0000","last_fullsized":"2023-11-25T04:25:59.979040+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:22:38.004934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3502371,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":3718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":89,"num_bytes_recovered":3497669,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"49'1","reported_seq":70,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978352+0000","last_change":"2023-11-25T04:22:46.034509+0000","last_active":"2023-11-25T04:25:59.978352+0000","last_peered":"2023-11-25T04:25:59.978352+0000","last_clean":"2023-11-25T04:25:59.978352+0000","last_became_active":"2023-11-25T04:22:46.034158+0000","last_became_peered":"2023-11-25T04:22:46.034158+0000","last_unstale":"2023-11-25T04:25:59.978352+0000","last_undegraded":"2023-11-25T04:25:59.978352+0000","last_fullsized":"2023-11-25T04:25:59.978352+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:26:25.338117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"54'141","reported_seq":209,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978912+0000","last_change":"2023-11-25T04:22:47.038800+0000","last_active":"2023-11-25T04:25:59.978912+0000","last_peered":"2023-11-25T04:25:59.978912+0000","last_clean":"2023-11-25T04:25:59.978912+0000","last_became_active":"2023-11-25T04:22:47.038081+0000","last_became_peered":"2023-11-25T04:22:47.038081+0000","last_unstale":"2023-11-25T04:25:59.978912+0000","last_undegraded":"2023-11-25T04:25:59.978912+0000","last_fullsized":"2023-11-25T04:25:59.978912+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:45:40.065146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":174088,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":146,"num_write_kb":385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"12'157","reported_seq":232,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.977771+0000","last_change":"2023-11-25T04:25:47.000504+0000","last_active":"2023-11-25T04:25:59.977771+0000","last_peered":"2023-11-25T04:25:59.977771+0000","last_clean":"2023-11-25T04:25:59.977771+0000","last_became_active":"2023-11-25T04:25:46.999721+0000","last_became_peered":"2023-11-25T04:25:46.999721+0000","last_unstale":"2023-11-25T04:25:59.977771+0000","last_undegraded":"2023-11-25T04:25:59.977771+0000","last_fullsized":"2023-11-25T04:25:59.977771+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:35.919850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:35.919850+0000","last_clean_scrub_stamp":"2023-11-25T04:22:35.919850+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:26:33.607474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978082+0000","last_change":"2023-11-25T04:25:46.998960+0000","last_active":"2023-11-25T04:25:59.978082+0000","last_peered":"2023-11-25T04:25:59.978082+0000","last_clean":"2023-11-25T04:25:59.978082+0000","last_became_active":"2023-11-25T04:25:46.997693+0000","last_became_peered":"2023-11-25T04:25:46.997693+0000","last_unstale":"2023-11-25T04:25:59.978082+0000","last_undegraded":"2023-11-25T04:25:59.978082+0000","last_fullsized":"2023-11-25T04:25:59.978082+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:51:45.180957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"46'1","reported_seq":70,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978331+0000","last_change":"2023-11-25T04:22:46.037044+0000","last_active":"2023-11-25T04:25:59.978331+0000","last_peered":"2023-11-25T04:25:59.978331+0000","last_clean":"2023-11-25T04:25:59.978331+0000","last_became_active":"2023-11-25T04:22:46.035916+0000","last_became_peered":"2023-11-25T04:22:46.035916+0000","last_unstale":"2023-11-25T04:25:59.978331+0000","last_undegraded":"2023-11-25T04:25:59.978331+0000","last_fullsized":"2023-11-25T04:25:59.978331+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:38:20.501102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"54'146","reported_seq":213,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978804+0000","last_change":"2023-11-25T04:25:51.923452+0000","last_active":"2023-11-25T04:25:59.978804+0000","last_peered":"2023-11-25T04:25:59.978804+0000","last_clean":"2023-11-25T04:25:59.978804+0000","last_became_active":"2023-11-25T04:25:51.922490+0000","last_became_peered":"2023-11-25T04:25:51.922490+0000","last_unstale":"2023-11-25T04:25:59.978804+0000","last_undegraded":"2023-11-25T04:25:59.978804+0000","last_fullsized":"2023-11-25T04:25:59.978804+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:40:11.001062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":502154,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":170,"num_write_kb":687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":99,"num_bytes_recovered":502101,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978713+0000","last_change":"2023-11-25T04:25:47.000763+0000","last_active":"2023-11-25T04:25:59.978713+0000","last_peered":"2023-11-25T04:25:59.978713+0000","last_clean":"2023-11-25T04:25:59.978713+0000","last_became_active":"2023-11-25T04:25:47.000033+0000","last_became_peered":"2023-11-25T04:25:47.000033+0000","last_unstale":"2023-11-25T04:25:59.978713+0000","last_undegraded":"2023-11-25T04:25:59.978713+0000","last_fullsized":"2023-11-25T04:25:59.978713+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:53:59.131478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"50'37","reported_seq":156,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978116+0000","last_change":"2023-11-25T04:25:47.002009+0000","last_active":"2023-11-25T04:25:59.978116+0000","last_peered":"2023-11-25T04:25:59.978116+0000","last_clean":"2023-11-25T04:25:59.978116+0000","last_became_active":"2023-11-25T04:25:46.995637+0000","last_became_peered":"2023-11-25T04:25:46.995637+0000","last_unstale":"2023-11-25T04:25:59.978116+0000","last_undegraded":"2023-11-25T04:25:59.978116+0000","last_fullsized":"2023-11-25T04:25:59.978116+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:55:01.351319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":319160,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":35,"num_read_kb":334,"num_write":35,"num_write_kb":334,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":319160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"54'158","reported_seq":226,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979092+0000","last_change":"2023-11-25T04:22:47.039583+0000","last_active":"2023-11-25T04:25:59.979092+0000","last_peered":"2023-11-25T04:25:59.979092+0000","last_clean":"2023-11-25T04:25:59.979092+0000","last_became_active":"2023-11-25T04:22:47.039197+0000","last_became_peered":"2023-11-25T04:22:47.039197+0000","last_unstale":"2023-11-25T04:25:59.979092+0000","last_undegraded":"2023-11-25T04:25:59.979092+0000","last_fullsized":"2023-11-25T04:25:59.979092+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:37:30.959147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":169303,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":189,"num_write_kb":333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":72,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978222+0000","last_change":"2023-11-25T04:22:42.000382+0000","last_active":"2023-11-25T04:25:59.978222+0000","last_peered":"2023-11-25T04:25:59.978222+0000","last_clean":"2023-11-25T04:25:59.978222+0000","last_became_active":"2023-11-25T04:22:42.000041+0000","last_became_peered":"2023-11-25T04:22:42.000041+0000","last_unstale":"2023-11-25T04:25:59.978222+0000","last_undegraded":"2023-11-25T04:25:59.978222+0000","last_fullsized":"2023-11-25T04:25:59.978222+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:08:19.655864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"49'71","reported_seq":155,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978346+0000","last_change":"2023-11-25T04:25:47.109628+0000","last_active":"2023-11-25T04:25:59.978346+0000","last_peered":"2023-11-25T04:25:59.978346+0000","last_clean":"2023-11-25T04:25:59.978346+0000","last_became_active":"2023-11-25T04:25:47.001162+0000","last_became_peered":"2023-11-25T04:25:47.001162+0000","last_unstale":"2023-11-25T04:25:59.978346+0000","last_undegraded":"2023-11-25T04:25:59.978346+0000","last_fullsized":"2023-11-25T04:25:59.978346+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:03:33.658392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":69,"num_write_kb":4130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4194304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"54'158","reported_seq":226,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978945+0000","last_change":"2023-11-25T04:22:47.038454+0000","last_active":"2023-11-25T04:25:59.978945+0000","last_peered":"2023-11-25T04:25:59.978945+0000","last_clean":"2023-11-25T04:25:59.978945+0000","last_became_active":"2023-11-25T04:22:47.038046+0000","last_became_peered":"2023-11-25T04:22:47.038046+0000","last_unstale":"2023-11-25T04:25:59.978945+0000","last_undegraded":"2023-11-25T04:25:59.978945+0000","last_fullsized":"2023-11-25T04:25:59.978945+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:38:38.839941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":115501,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":163,"num_write_kb":243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.977941+0000","last_change":"2023-11-25T04:25:46.998739+0000","last_active":"2023-11-25T04:25:59.977941+0000","last_peered":"2023-11-25T04:25:59.977941+0000","last_clean":"2023-11-25T04:25:59.977941+0000","last_became_active":"2023-11-25T04:25:46.997391+0000","last_became_peered":"2023-11-25T04:25:46.997391+0000","last_unstale":"2023-11-25T04:25:59.977941+0000","last_undegraded":"2023-11-25T04:25:59.977941+0000","last_fullsized":"2023-11-25T04:25:59.977941+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:47:46.300596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"50'1","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978166+0000","last_change":"2023-11-25T04:25:46.999446+0000","last_active":"2023-11-25T04:25:59.978166+0000","last_peered":"2023-11-25T04:25:59.978166+0000","last_clean":"2023-11-25T04:25:59.978166+0000","last_became_active":"2023-11-25T04:25:46.998132+0000","last_became_peered":"2023-11-25T04:25:46.998132+0000","last_unstale":"2023-11-25T04:25:59.978166+0000","last_undegraded":"2023-11-25T04:25:59.978166+0000","last_fullsized":"2023-11-25T04:25:59.978166+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:42:51.166678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"54'157","reported_seq":225,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978638+0000","last_change":"2023-11-25T04:22:47.037355+0000","last_active":"2023-11-25T04:25:59.978638+0000","last_peered":"2023-11-25T04:25:59.978638+0000","last_clean":"2023-11-25T04:25:59.978638+0000","last_became_active":"2023-11-25T04:22:47.036924+0000","last_became_peered":"2023-11-25T04:22:47.036924+0000","last_unstale":"2023-11-25T04:25:59.978638+0000","last_undegraded":"2023-11-25T04:25:59.978638+0000","last_fullsized":"2023-11-25T04:25:59.978638+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:15:39.511043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":93321,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":148,"num_write_kb":360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978520+0000","last_change":"2023-11-25T04:25:46.999738+0000","last_active":"2023-11-25T04:25:59.978520+0000","last_peered":"2023-11-25T04:25:59.978520+0000","last_clean":"2023-11-25T04:25:59.978520+0000","last_became_active":"2023-11-25T04:25:46.999403+0000","last_became_peered":"2023-11-25T04:25:46.999403+0000","last_unstale":"2023-11-25T04:25:59.978520+0000","last_undegraded":"2023-11-25T04:25:59.978520+0000","last_fullsized":"2023-11-25T04:25:59.978520+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:45:53.304063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":72,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978705+0000","last_change":"2023-11-25T04:22:42.000204+0000","last_active":"2023-11-25T04:25:59.978705+0000","last_peered":"2023-11-25T04:25:59.978705+0000","last_clean":"2023-11-25T04:25:59.978705+0000","last_became_active":"2023-11-25T04:22:41.999915+0000","last_became_peered":"2023-11-25T04:22:41.999915+0000","last_unstale":"2023-11-25T04:25:59.978705+0000","last_undegraded":"2023-11-25T04:25:59.978705+0000","last_fullsized":"2023-11-25T04:25:59.978705+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:07:32.457905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"54'152","reported_seq":432,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978379+0000","last_change":"2023-11-25T04:25:53.933239+0000","last_active":"2023-11-25T04:25:59.978379+0000","last_peered":"2023-11-25T04:25:59.978379+0000","last_clean":"2023-11-25T04:25:59.978379+0000","last_became_active":"2023-11-25T04:25:53.932586+0000","last_became_peered":"2023-11-25T04:25:53.932586+0000","last_unstale":"2023-11-25T04:25:59.978379+0000","last_undegraded":"2023-11-25T04:25:59.978379+0000","last_fullsized":"2023-11-25T04:25:59.978379+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:54:51.339025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":329327,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":151,"num_write_kb":471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":97,"num_bytes_recovered":329278,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":69,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978632+0000","last_change":"2023-11-25T04:22:46.035885+0000","last_active":"2023-11-25T04:25:59.978632+0000","last_peered":"2023-11-25T04:25:59.978632+0000","last_clean":"2023-11-25T04:25:59.978632+0000","last_became_active":"2023-11-25T04:22:46.035405+0000","last_became_peered":"2023-11-25T04:22:46.035405+0000","last_unstale":"2023-11-25T04:25:59.978632+0000","last_undegraded":"2023-11-25T04:25:59.978632+0000","last_fullsized":"2023-11-25T04:25:59.978632+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:53:47.398746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":72,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979221+0000","last_change":"2023-11-25T04:22:41.999067+0000","last_active":"2023-11-25T04:25:59.979221+0000","last_peered":"2023-11-25T04:25:59.979221+0000","last_clean":"2023-11-25T04:25:59.979221+0000","last_became_active":"2023-11-25T04:22:41.998798+0000","last_became_peered":"2023-11-25T04:22:41.998798+0000","last_unstale":"2023-11-25T04:25:59.979221+0000","last_undegraded":"2023-11-25T04:25:59.979221+0000","last_fullsized":"2023-11-25T04:25:59.979221+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:00:04.266051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"54'172","reported_seq":470,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978595+0000","last_change":"2023-11-25T04:25:51.922403+0000","last_active":"2023-11-25T04:25:59.978595+0000","last_peered":"2023-11-25T04:25:59.978595+0000","last_clean":"2023-11-25T04:25:59.978595+0000","last_became_active":"2023-11-25T04:25:51.921858+0000","last_became_peered":"2023-11-25T04:25:51.921858+0000","last_unstale":"2023-11-25T04:25:59.978595+0000","last_undegraded":"2023-11-25T04:25:59.978595+0000","last_fullsized":"2023-11-25T04:25:59.978595+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:04:58.273982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":129965,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":166,"num_write_kb":290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":129965,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"18'2","reported_seq":71,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979088+0000","last_change":"2023-11-25T04:22:46.034606+0000","last_active":"2023-11-25T04:25:59.979088+0000","last_peered":"2023-11-25T04:25:59.979088+0000","last_clean":"2023-11-25T04:25:59.979088+0000","last_became_active":"2023-11-25T04:22:46.034229+0000","last_became_peered":"2023-11-25T04:22:46.034229+0000","last_unstale":"2023-11-25T04:25:59.979088+0000","last_undegraded":"2023-11-25T04:25:59.979088+0000","last_fullsized":"2023-11-25T04:25:59.979088+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:21:42.146264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":72,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978952+0000","last_change":"2023-11-25T04:22:42.000258+0000","last_active":"2023-11-25T04:25:59.978952+0000","last_peered":"2023-11-25T04:25:59.978952+0000","last_clean":"2023-11-25T04:25:59.978952+0000","last_became_active":"2023-11-25T04:22:41.999992+0000","last_became_peered":"2023-11-25T04:22:41.999992+0000","last_unstale":"2023-11-25T04:25:59.978952+0000","last_undegraded":"2023-11-25T04:25:59.978952+0000","last_fullsized":"2023-11-25T04:25:59.978952+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:31:13.715992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"54'150","reported_seq":436,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.977926+0000","last_change":"2023-11-25T04:25:54.938479+0000","last_active":"2023-11-25T04:25:59.977926+0000","last_peered":"2023-11-25T04:25:59.977926+0000","last_clean":"2023-11-25T04:25:59.977926+0000","last_became_active":"2023-11-25T04:25:54.937838+0000","last_became_peered":"2023-11-25T04:25:54.937838+0000","last_unstale":"2023-11-25T04:25:59.977926+0000","last_undegraded":"2023-11-25T04:25:59.977926+0000","last_fullsized":"2023-11-25T04:25:59.977926+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:51:14.394190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":159585,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":154224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"54'14","reported_seq":99,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978916+0000","last_change":"2023-11-25T04:25:47.026749+0000","last_active":"2023-11-25T04:25:59.978916+0000","last_peered":"2023-11-25T04:25:59.978916+0000","last_clean":"2023-11-25T04:25:59.978916+0000","last_became_active":"2023-11-25T04:25:46.997404+0000","last_became_peered":"2023-11-25T04:25:46.997404+0000","last_unstale":"2023-11-25T04:25:59.978916+0000","last_undegraded":"2023-11-25T04:25:59.978916+0000","last_fullsized":"2023-11-25T04:25:59.978916+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:20:31.719529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978930+0000","last_change":"2023-11-25T04:25:46.995200+0000","last_active":"2023-11-25T04:25:59.978930+0000","last_peered":"2023-11-25T04:25:59.978930+0000","last_clean":"2023-11-25T04:25:59.978930+0000","last_became_active":"2023-11-25T04:25:46.994834+0000","last_became_peered":"2023-11-25T04:25:46.994834+0000","last_unstale":"2023-11-25T04:25:59.978930+0000","last_undegraded":"2023-11-25T04:25:59.978930+0000","last_fullsized":"2023-11-25T04:25:59.978930+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:33:07.913911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"54'135","reported_seq":407,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978276+0000","last_change":"2023-11-25T04:25:49.922046+0000","last_active":"2023-11-25T04:25:59.978276+0000","last_peered":"2023-11-25T04:25:59.978276+0000","last_clean":"2023-11-25T04:25:59.978276+0000","last_became_active":"2023-11-25T04:25:49.921465+0000","last_became_peered":"2023-11-25T04:25:49.921465+0000","last_unstale":"2023-11-25T04:25:59.978276+0000","last_undegraded":"2023-11-25T04:25:59.978276+0000","last_fullsized":"2023-11-25T04:25:59.978276+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:12:54.077901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":99503,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":148,"num_write_kb":250,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":93,"num_bytes_recovered":99503,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"54'166","reported_seq":233,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978407+0000","last_change":"2023-11-25T04:25:51.923141+0000","last_active":"2023-11-25T04:25:59.978407+0000","last_peered":"2023-11-25T04:25:59.978407+0000","last_clean":"2023-11-25T04:25:59.978407+0000","last_became_active":"2023-11-25T04:25:51.922070+0000","last_became_peered":"2023-11-25T04:25:51.922070+0000","last_unstale":"2023-11-25T04:25:59.978407+0000","last_undegraded":"2023-11-25T04:25:59.978407+0000","last_fullsized":"2023-11-25T04:25:59.978407+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:34:53.021081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":185321,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":172,"num_write_kb":312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":164176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"58'50","reported_seq":53,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978227+0000","last_change":"2023-11-25T04:25:47.021705+0000","last_active":"2023-11-25T04:25:59.978227+0000","last_peered":"2023-11-25T04:25:59.978227+0000","last_clean":"2023-11-25T04:25:59.978227+0000","last_became_active":"2023-11-25T04:25:46.994997+0000","last_became_peered":"2023-11-25T04:25:46.994997+0000","last_unstale":"2023-11-25T04:25:59.978227+0000","last_undegraded":"2023-11-25T04:25:59.978227+0000","last_fullsized":"2023-11-25T04:25:59.978227+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:41:04.147372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":4119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4190183,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"54'153","reported_seq":221,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978107+0000","last_change":"2023-11-25T04:22:47.037181+0000","last_active":"2023-11-25T04:25:59.978107+0000","last_peered":"2023-11-25T04:25:59.978107+0000","last_clean":"2023-11-25T04:25:59.978107+0000","last_became_active":"2023-11-25T04:22:47.036821+0000","last_became_peered":"2023-11-25T04:22:47.036821+0000","last_unstale":"2023-11-25T04:25:59.978107+0000","last_undegraded":"2023-11-25T04:25:59.978107+0000","last_fullsized":"2023-11-25T04:25:59.978107+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:12:51.990448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":101117,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":161,"num_write_kb":242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"50'22","reported_seq":107,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979124+0000","last_change":"2023-11-25T04:25:47.028636+0000","last_active":"2023-11-25T04:25:59.979124+0000","last_peered":"2023-11-25T04:25:59.979124+0000","last_clean":"2023-11-25T04:25:59.979124+0000","last_became_active":"2023-11-25T04:25:47.000478+0000","last_became_peered":"2023-11-25T04:25:47.000478+0000","last_unstale":"2023-11-25T04:25:59.979124+0000","last_undegraded":"2023-11-25T04:25:59.979124+0000","last_fullsized":"2023-11-25T04:25:59.979124+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:50:13.896985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":90,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978907+0000","last_change":"2023-11-25T04:25:46.996182+0000","last_active":"2023-11-25T04:25:59.978907+0000","last_peered":"2023-11-25T04:25:59.978907+0000","last_clean":"2023-11-25T04:25:59.978907+0000","last_became_active":"2023-11-25T04:25:46.994525+0000","last_became_peered":"2023-11-25T04:25:46.994525+0000","last_unstale":"2023-11-25T04:25:59.978907+0000","last_undegraded":"2023-11-25T04:25:59.978907+0000","last_fullsized":"2023-11-25T04:25:59.978907+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:26:24.469530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"54'177","reported_seq":485,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.977915+0000","last_change":"2023-11-25T04:25:51.922466+0000","last_active":"2023-11-25T04:25:59.977915+0000","last_peered":"2023-11-25T04:25:59.977915+0000","last_clean":"2023-11-25T04:25:59.977915+0000","last_became_active":"2023-11-25T04:25:51.921938+0000","last_became_peered":"2023-11-25T04:25:51.921938+0000","last_unstale":"2023-11-25T04:25:59.977915+0000","last_undegraded":"2023-11-25T04:25:59.977915+0000","last_fullsized":"2023-11-25T04:25:59.977915+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:38:59.016525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":522494,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":168,"num_write_kb":749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":111,"num_bytes_recovered":522494,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"46'2","reported_seq":82,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979031+0000","last_change":"2023-11-25T04:25:46.998867+0000","last_active":"2023-11-25T04:25:59.979031+0000","last_peered":"2023-11-25T04:25:59.979031+0000","last_clean":"2023-11-25T04:25:59.979031+0000","last_became_active":"2023-11-25T04:25:46.998508+0000","last_became_peered":"2023-11-25T04:25:46.998508+0000","last_unstale":"2023-11-25T04:25:59.979031+0000","last_undegraded":"2023-11-25T04:25:59.979031+0000","last_fullsized":"2023-11-25T04:25:59.979031+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:50:13.896985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.10","version":"54'153","reported_seq":221,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.977904+0000","last_change":"2023-11-25T04:22:47.037790+0000","last_active":"2023-11-25T04:25:59.977904+0000","last_peered":"2023-11-25T04:25:59.977904+0000","last_clean":"2023-11-25T04:25:59.977904+0000","last_became_active":"2023-11-25T04:22:47.037149+0000","last_became_peered":"2023-11-25T04:22:47.037149+0000","last_unstale":"2023-11-25T04:25:59.977904+0000","last_undegraded":"2023-11-25T04:25:59.977904+0000","last_fullsized":"2023-11-25T04:25:59.977904+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:31:54.857270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":218073,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1f","version":"54'156","reported_seq":448,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979259+0000","last_change":"2023-11-25T04:25:49.921904+0000","last_active":"2023-11-25T04:25:59.979259+0000","last_peered":"2023-11-25T04:25:59.979259+0000","last_clean":"2023-11-25T04:25:59.979259+0000","last_became_active":"2023-11-25T04:25:49.921458+0000","last_became_peered":"2023-11-25T04:25:49.921458+0000","last_unstale":"2023-11-25T04:25:59.979259+0000","last_undegraded":"2023-11-25T04:25:59.979259+0000","last_fullsized":"2023-11-25T04:25:59.979259+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:59:27.602060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":107712,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":103,"num_bytes_recovered":107703,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"48'1","reported_seq":70,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978057+0000","last_change":"2023-11-25T04:22:46.036085+0000","last_active":"2023-11-25T04:25:59.978057+0000","last_peered":"2023-11-25T04:25:59.978057+0000","last_clean":"2023-11-25T04:25:59.978057+0000","last_became_active":"2023-11-25T04:22:46.035495+0000","last_became_peered":"2023-11-25T04:22:46.035495+0000","last_unstale":"2023-11-25T04:25:59.978057+0000","last_undegraded":"2023-11-25T04:25:59.978057+0000","last_fullsized":"2023-11-25T04:25:59.978057+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:14:56.735365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"54'152","reported_seq":430,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978757+0000","last_change":"2023-11-25T04:25:50.923882+0000","last_active":"2023-11-25T04:25:59.978757+0000","last_peered":"2023-11-25T04:25:59.978757+0000","last_clean":"2023-11-25T04:25:59.978757+0000","last_became_active":"2023-11-25T04:25:50.923375+0000","last_became_peered":"2023-11-25T04:25:50.923375+0000","last_unstale":"2023-11-25T04:25:59.978757+0000","last_undegraded":"2023-11-25T04:25:59.978757+0000","last_fullsized":"2023-11-25T04:25:59.978757+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:54:51.339025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":76435,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":76435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":80,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978069+0000","last_change":"2023-11-25T04:25:46.995228+0000","last_active":"2023-11-25T04:25:59.978069+0000","last_peered":"2023-11-25T04:25:59.978069+0000","last_clean":"2023-11-25T04:25:59.978069+0000","last_became_active":"2023-11-25T04:25:46.994871+0000","last_became_peered":"2023-11-25T04:25:46.994871+0000","last_unstale":"2023-11-25T04:25:59.978069+0000","last_undegraded":"2023-11-25T04:25:59.978069+0000","last_fullsized":"2023-11-25T04:25:59.978069+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:53:34.807075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"54'158","reported_seq":432,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978899+0000","last_change":"2023-11-25T04:25:52.925523+0000","last_active":"2023-11-25T04:25:59.978899+0000","last_peered":"2023-11-25T04:25:59.978899+0000","last_clean":"2023-11-25T04:25:59.978899+0000","last_became_active":"2023-11-25T04:25:52.925038+0000","last_became_peered":"2023-11-25T04:25:52.925038+0000","last_unstale":"2023-11-25T04:25:59.978899+0000","last_undegraded":"2023-11-25T04:25:59.978899+0000","last_fullsized":"2023-11-25T04:25:59.978899+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:37:14.620534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":209500,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":139,"num_write_kb":351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":209500,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1a","version":"48'4","reported_seq":73,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978665+0000","last_change":"2023-11-25T04:22:46.034340+0000","last_active":"2023-11-25T04:25:59.978665+0000","last_peered":"2023-11-25T04:25:59.978665+0000","last_clean":"2023-11-25T04:25:59.978665+0000","last_became_active":"2023-11-25T04:22:46.033944+0000","last_became_peered":"2023-11-25T04:22:46.033944+0000","last_unstale":"2023-11-25T04:25:59.978665+0000","last_undegraded":"2023-11-25T04:25:59.978665+0000","last_fullsized":"2023-11-25T04:25:59.978665+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:59:30.291004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"54'163","reported_seq":231,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978741+0000","last_change":"2023-11-25T04:22:47.039547+0000","last_active":"2023-11-25T04:25:59.978741+0000","last_peered":"2023-11-25T04:25:59.978741+0000","last_clean":"2023-11-25T04:25:59.978741+0000","last_became_active":"2023-11-25T04:22:47.039218+0000","last_became_peered":"2023-11-25T04:22:47.039218+0000","last_unstale":"2023-11-25T04:25:59.978741+0000","last_undegraded":"2023-11-25T04:25:59.978741+0000","last_fullsized":"2023-11-25T04:25:59.978741+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:44:48.553972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":165883,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"45'9","reported_seq":94,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978700+0000","last_change":"2023-11-25T04:25:47.023365+0000","last_active":"2023-11-25T04:25:59.978700+0000","last_peered":"2023-11-25T04:25:59.978700+0000","last_clean":"2023-11-25T04:25:59.978700+0000","last_became_active":"2023-11-25T04:25:46.999998+0000","last_became_peered":"2023-11-25T04:25:46.999998+0000","last_unstale":"2023-11-25T04:25:59.978700+0000","last_undegraded":"2023-11-25T04:25:59.978700+0000","last_fullsized":"2023-11-25T04:25:59.978700+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:53:34.807075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":4101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4194304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"54'149","reported_seq":217,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978573+0000","last_change":"2023-11-25T04:25:50.923041+0000","last_active":"2023-11-25T04:25:59.978573+0000","last_peered":"2023-11-25T04:25:59.978573+0000","last_clean":"2023-11-25T04:25:59.978573+0000","last_became_active":"2023-11-25T04:25:50.922164+0000","last_became_peered":"2023-11-25T04:25:50.922164+0000","last_unstale":"2023-11-25T04:25:59.978573+0000","last_undegraded":"2023-11-25T04:25:59.978573+0000","last_fullsized":"2023-11-25T04:25:59.978573+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:31:10.744814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":179506,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":155,"num_write_kb":317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":179506,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"18'2","reported_seq":37,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978409+0000","last_change":"2023-11-25T04:25:46.996675+0000","last_active":"2023-11-25T04:25:59.978409+0000","last_peered":"2023-11-25T04:25:59.978409+0000","last_clean":"2023-11-25T04:25:59.978409+0000","last_became_active":"2023-11-25T04:25:46.995160+0000","last_became_peered":"2023-11-25T04:25:46.995160+0000","last_unstale":"2023-11-25T04:25:59.978409+0000","last_undegraded":"2023-11-25T04:25:59.978409+0000","last_fullsized":"2023-11-25T04:25:59.978409+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:59:39.131605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1a","version":"54'162","reported_seq":226,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978332+0000","last_change":"2023-11-25T04:25:54.939182+0000","last_active":"2023-11-25T04:25:59.978332+0000","last_peered":"2023-11-25T04:25:59.978332+0000","last_clean":"2023-11-25T04:25:59.978332+0000","last_became_active":"2023-11-25T04:25:54.937821+0000","last_became_peered":"2023-11-25T04:25:54.937821+0000","last_unstale":"2023-11-25T04:25:59.978332+0000","last_undegraded":"2023-11-25T04:25:59.978332+0000","last_fullsized":"2023-11-25T04:25:59.978332+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:05:34.575966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":134672,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":169,"num_write_kb":311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":126291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"48'59","reported_seq":146,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979001+0000","last_change":"2023-11-25T04:25:47.077194+0000","last_active":"2023-11-25T04:25:59.979001+0000","last_peered":"2023-11-25T04:25:59.979001+0000","last_clean":"2023-11-25T04:25:59.979001+0000","last_became_active":"2023-11-25T04:25:47.000722+0000","last_became_peered":"2023-11-25T04:25:47.000722+0000","last_unstale":"2023-11-25T04:25:59.979001+0000","last_undegraded":"2023-11-25T04:25:59.979001+0000","last_fullsized":"2023-11-25T04:25:59.979001+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:55:01.351319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8388608,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"54'163","reported_seq":231,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978500+0000","last_change":"2023-11-25T04:22:47.036764+0000","last_active":"2023-11-25T04:25:59.978500+0000","last_peered":"2023-11-25T04:25:59.978500+0000","last_clean":"2023-11-25T04:25:59.978500+0000","last_became_active":"2023-11-25T04:22:47.036406+0000","last_became_peered":"2023-11-25T04:22:47.036406+0000","last_unstale":"2023-11-25T04:25:59.978500+0000","last_undegraded":"2023-11-25T04:25:59.978500+0000","last_fullsized":"2023-11-25T04:25:59.978500+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:09:55.357642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":527891,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":163,"num_write_kb":687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1e","version":"46'2","reported_seq":71,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978868+0000","last_change":"2023-11-25T04:22:46.035659+0000","last_active":"2023-11-25T04:25:59.978868+0000","last_peered":"2023-11-25T04:25:59.978868+0000","last_clean":"2023-11-25T04:25:59.978868+0000","last_became_active":"2023-11-25T04:22:46.035113+0000","last_became_peered":"2023-11-25T04:22:46.035113+0000","last_unstale":"2023-11-25T04:25:59.978868+0000","last_undegraded":"2023-11-25T04:25:59.978868+0000","last_fullsized":"2023-11-25T04:25:59.978868+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:26:25.338117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.18","version":"54'195","reported_seq":259,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978263+0000","last_change":"2023-11-25T04:25:54.938573+0000","last_active":"2023-11-25T04:25:59.978263+0000","last_peered":"2023-11-25T04:25:59.978263+0000","last_clean":"2023-11-25T04:25:59.978263+0000","last_became_active":"2023-11-25T04:25:54.937444+0000","last_became_peered":"2023-11-25T04:25:54.937444+0000","last_unstale":"2023-11-25T04:25:59.978263+0000","last_undegraded":"2023-11-25T04:25:59.978263+0000","last_fullsized":"2023-11-25T04:25:59.978263+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:20:15.814691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":128641,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":185,"num_write_kb":408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":122,"num_bytes_recovered":128641,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"50'17","reported_seq":102,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978969+0000","last_change":"2023-11-25T04:25:47.044228+0000","last_active":"2023-11-25T04:25:59.978969+0000","last_peered":"2023-11-25T04:25:59.978969+0000","last_clean":"2023-11-25T04:25:59.978969+0000","last_became_active":"2023-11-25T04:25:46.998870+0000","last_became_peered":"2023-11-25T04:25:46.998870+0000","last_unstale":"2023-11-25T04:25:59.978969+0000","last_undegraded":"2023-11-25T04:25:59.978969+0000","last_fullsized":"2023-11-25T04:25:59.978969+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:08:12.601941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":4103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4194304,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":26330855,"num_objects":3447,"num_object_clones":0,"num_object_copies":6894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10423,"num_write_kb":38550,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4189,"num_bytes_recovered":12749181,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":72376320,"data_stored":52324046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10246,"ondisk_log_size":10246,"up":128,"acting":128,"num_store_stats":3},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":40885548,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":35,"num_read_kb":334,"num_write":435,"num_write_kb":40462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":32491439,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":81833984,"data_stored":81771096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5548,"internal_metadata":0},"log_size":435,"ondisk_log_size":435,"up":64,"acting":64,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738414,"num_pgs":72,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73208,"kb_used_data":46188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94298632,"statfs":{"total":96636764160,"available":96561799168,"internally_reserved":0,"allocated":47296512,"data_stored":39787071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":1,"last update":"Sat Nov 25 04:25:40 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.433,"15min":0.44500000000000001},"min":{"1min":0.28899999999999998,"5min":0.216,"15min":0.216},"max":{"1min":0.60499999999999998,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.439,"15min":0.45100000000000001},"min":{"1min":0.25600000000000001,"5min":0.184,"15min":0.184},"max":{"1min":0.63400000000000001,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.63}]}]},{"osd":1,"up_from":8,"seq":34359738414,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86880,"kb_used_data":55700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94284960,"statfs":{"total":96636764160,"available":96547799040,"internally_reserved":0,"allocated":57036800,"data_stored":50261554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":2,"last update":"Sat Nov 25 04:25:42 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.40600000000000003,"15min":0.41099999999999998},"min":{"1min":0.29099999999999998,"5min":0.182,"15min":0.182},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.40200000000000002,"15min":0.40600000000000003},"min":{"1min":0.312,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.55500000000000005}]}]},{"osd":0,"up_from":60,"seq":257698037765,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87624,"kb_used_data":51764,"kb_used_omap":6,"kb_used_meta":35833,"kb_avail":94284216,"statfs":{"total":96636764160,"available":96547037184,"internally_reserved":0,"allocated":53006336,"data_stored":46141107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7143,"internal_metadata":36693017},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32530432,"data_stored":32496940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5548,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":29380608,"data_stored":29360982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":19922944,"data_stored":19913174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":19234816,"data_stored":12749181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":26415104,"data_stored":20005586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":26726400,"data_stored":19569279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-25T04:26:05.171 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-25T04:26:05.503 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:26:05.504 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2023-11-25T04:26:05.514 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":178,"stamp":"2023-11-25T04:26:03.549604+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":67806771,"num_objects":3481,"num_object_clones":0,"num_object_copies":6962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3481,"num_whiteouts":0,"num_read":171,"num_read_kb":452,"num_write":11061,"num_write_kb":82472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4201,"num_bytes_recovered":45240620,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10838,"ondisk_log_size":10838,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":247712,"kb_used_data":153652,"kb_used_omap":10,"kb_used_meta":94005,"kb_avail":282867808,"statfs":{"total":289910292480,"available":289656635392,"internally_reserved":0,"allocated":157339648,"data_stored":136189732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10333,"internal_metadata":96262051},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.595710"},"pg_stats":[{"pgid":"4.3f","version":"54'166","reported_seq":234,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979173+0000","last_change":"2023-11-25T04:22:47.039847+0000","last_active":"2023-11-25T04:25:59.979173+0000","last_peered":"2023-11-25T04:25:59.979173+0000","last_clean":"2023-11-25T04:25:59.979173+0000","last_became_active":"2023-11-25T04:22:47.039490+0000","last_became_peered":"2023-11-25T04:22:47.039490+0000","last_unstale":"2023-11-25T04:25:59.979173+0000","last_undegraded":"2023-11-25T04:25:59.979173+0000","last_fullsized":"2023-11-25T04:25:59.979173+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:25:06.132517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":577866,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":174,"num_write_kb":756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3e","version":"54'212","reported_seq":280,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978984+0000","last_change":"2023-11-25T04:22:47.040836+0000","last_active":"2023-11-25T04:25:59.978984+0000","last_peered":"2023-11-25T04:25:59.978984+0000","last_clean":"2023-11-25T04:25:59.978984+0000","last_became_active":"2023-11-25T04:22:47.040511+0000","last_became_peered":"2023-11-25T04:22:47.040511+0000","last_unstale":"2023-11-25T04:25:59.978984+0000","last_undegraded":"2023-11-25T04:25:59.978984+0000","last_fullsized":"2023-11-25T04:25:59.978984+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:15:39.511043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":461579,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":236,"num_write_kb":841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3d","version":"54'167","reported_seq":233,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978931+0000","last_change":"2023-11-25T04:25:52.924678+0000","last_active":"2023-11-25T04:25:59.978931+0000","last_peered":"2023-11-25T04:25:59.978931+0000","last_clean":"2023-11-25T04:25:59.978931+0000","last_became_active":"2023-11-25T04:25:52.923459+0000","last_became_peered":"2023-11-25T04:25:52.923459+0000","last_unstale":"2023-11-25T04:25:59.978931+0000","last_undegraded":"2023-11-25T04:25:59.978931+0000","last_fullsized":"2023-11-25T04:25:59.978931+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:24:06.950438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":145844,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":175,"num_write_kb":312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":138765,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"54'124","reported_seq":191,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978386+0000","last_change":"2023-11-25T04:25:51.922843+0000","last_active":"2023-11-25T04:25:59.978386+0000","last_peered":"2023-11-25T04:25:59.978386+0000","last_clean":"2023-11-25T04:25:59.978386+0000","last_became_active":"2023-11-25T04:25:51.921633+0000","last_became_peered":"2023-11-25T04:25:51.921633+0000","last_unstale":"2023-11-25T04:25:59.978386+0000","last_undegraded":"2023-11-25T04:25:59.978386+0000","last_fullsized":"2023-11-25T04:25:59.978386+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:05:31.332951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61303,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119,"num_write_kb":161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":61303,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"54'169","reported_seq":237,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978640+0000","last_change":"2023-11-25T04:22:47.039499+0000","last_active":"2023-11-25T04:25:59.978640+0000","last_peered":"2023-11-25T04:25:59.978640+0000","last_clean":"2023-11-25T04:25:59.978640+0000","last_became_active":"2023-11-25T04:22:47.039165+0000","last_became_peered":"2023-11-25T04:22:47.039165+0000","last_unstale":"2023-11-25T04:25:59.978640+0000","last_undegraded":"2023-11-25T04:25:59.978640+0000","last_fullsized":"2023-11-25T04:25:59.978640+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:57:28.855151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":118227,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":185,"num_write_kb":351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3a","version":"54'169","reported_seq":234,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978863+0000","last_change":"2023-11-25T04:25:53.933198+0000","last_active":"2023-11-25T04:25:59.978863+0000","last_peered":"2023-11-25T04:25:59.978863+0000","last_clean":"2023-11-25T04:25:59.978863+0000","last_became_active":"2023-11-25T04:25:53.931982+0000","last_became_peered":"2023-11-25T04:25:53.931982+0000","last_unstale":"2023-11-25T04:25:59.978863+0000","last_undegraded":"2023-11-25T04:25:59.978863+0000","last_fullsized":"2023-11-25T04:25:59.978863+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:28:40.449148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100304,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":161,"num_write_kb":424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":100304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"54'168","reported_seq":237,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978380+0000","last_change":"2023-11-25T04:25:49.921033+0000","last_active":"2023-11-25T04:25:59.978380+0000","last_peered":"2023-11-25T04:25:59.978380+0000","last_clean":"2023-11-25T04:25:59.978380+0000","last_became_active":"2023-11-25T04:25:49.919354+0000","last_became_peered":"2023-11-25T04:25:49.919354+0000","last_unstale":"2023-11-25T04:25:59.978380+0000","last_undegraded":"2023-11-25T04:25:59.978380+0000","last_fullsized":"2023-11-25T04:25:59.978380+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:36:13.489397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":124558,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":167,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":111,"num_bytes_recovered":124533,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.38","version":"54'152","reported_seq":221,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978153+0000","last_change":"2023-11-25T04:25:49.920421+0000","last_active":"2023-11-25T04:25:59.978153+0000","last_peered":"2023-11-25T04:25:59.978153+0000","last_clean":"2023-11-25T04:25:59.978153+0000","last_became_active":"2023-11-25T04:25:49.919236+0000","last_became_peered":"2023-11-25T04:25:49.919236+0000","last_unstale":"2023-11-25T04:25:59.978153+0000","last_undegraded":"2023-11-25T04:25:59.978153+0000","last_fullsized":"2023-11-25T04:25:59.978153+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:32:41.956579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":120147,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":120121,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.37","version":"54'167","reported_seq":235,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978818+0000","last_change":"2023-11-25T04:22:47.038397+0000","last_active":"2023-11-25T04:25:59.978818+0000","last_peered":"2023-11-25T04:25:59.978818+0000","last_clean":"2023-11-25T04:25:59.978818+0000","last_became_active":"2023-11-25T04:22:47.037328+0000","last_became_peered":"2023-11-25T04:22:47.037328+0000","last_unstale":"2023-11-25T04:25:59.978818+0000","last_undegraded":"2023-11-25T04:25:59.978818+0000","last_fullsized":"2023-11-25T04:25:59.978818+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:56:59.689578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":174147,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":184,"num_write_kb":353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.36","version":"54'149","reported_seq":217,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979047+0000","last_change":"2023-11-25T04:22:47.040182+0000","last_active":"2023-11-25T04:25:59.979047+0000","last_peered":"2023-11-25T04:25:59.979047+0000","last_clean":"2023-11-25T04:25:59.979047+0000","last_became_active":"2023-11-25T04:22:47.039892+0000","last_became_peered":"2023-11-25T04:22:47.039892+0000","last_unstale":"2023-11-25T04:25:59.979047+0000","last_undegraded":"2023-11-25T04:25:59.979047+0000","last_fullsized":"2023-11-25T04:25:59.979047+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:38:09.176582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":233330,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"54'156","reported_seq":223,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979021+0000","last_change":"2023-11-25T04:25:51.923087+0000","last_active":"2023-11-25T04:25:59.979021+0000","last_peered":"2023-11-25T04:25:59.979021+0000","last_clean":"2023-11-25T04:25:59.979021+0000","last_became_active":"2023-11-25T04:25:51.921800+0000","last_became_peered":"2023-11-25T04:25:51.921800+0000","last_unstale":"2023-11-25T04:25:59.979021+0000","last_undegraded":"2023-11-25T04:25:59.979021+0000","last_fullsized":"2023-11-25T04:25:59.979021+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:29:47.766517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":302550,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":163,"num_write_kb":431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":300323,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.34","version":"54'136","reported_seq":400,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978706+0000","last_change":"2023-11-25T04:25:50.922843+0000","last_active":"2023-11-25T04:25:59.978706+0000","last_peered":"2023-11-25T04:25:59.978706+0000","last_clean":"2023-11-25T04:25:59.978706+0000","last_became_active":"2023-11-25T04:25:50.922309+0000","last_became_peered":"2023-11-25T04:25:50.922309+0000","last_unstale":"2023-11-25T04:25:59.978706+0000","last_undegraded":"2023-11-25T04:25:59.978706+0000","last_fullsized":"2023-11-25T04:25:59.978706+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:38:33.939562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":93053,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":131,"num_write_kb":248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":89,"num_bytes_recovered":93053,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"54'134","reported_seq":202,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978529+0000","last_change":"2023-11-25T04:22:47.040388+0000","last_active":"2023-11-25T04:25:59.978529+0000","last_peered":"2023-11-25T04:25:59.978529+0000","last_clean":"2023-11-25T04:25:59.978529+0000","last_became_active":"2023-11-25T04:22:47.040129+0000","last_became_peered":"2023-11-25T04:22:47.040129+0000","last_unstale":"2023-11-25T04:25:59.978529+0000","last_undegraded":"2023-11-25T04:25:59.978529+0000","last_fullsized":"2023-11-25T04:25:59.978529+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:31:54.857270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":149524,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":139,"num_write_kb":252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.32","version":"54'174","reported_seq":239,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978762+0000","last_change":"2023-11-25T04:25:53.934526+0000","last_active":"2023-11-25T04:25:59.978762+0000","last_peered":"2023-11-25T04:25:59.978762+0000","last_clean":"2023-11-25T04:25:59.978762+0000","last_became_active":"2023-11-25T04:25:53.933694+0000","last_became_peered":"2023-11-25T04:25:53.933694+0000","last_unstale":"2023-11-25T04:25:59.978762+0000","last_undegraded":"2023-11-25T04:25:59.978762+0000","last_fullsized":"2023-11-25T04:25:59.978762+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:01:16.858976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":95797,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":181,"num_write_kb":547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":103,"num_bytes_recovered":95797,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.31","version":"54'156","reported_seq":225,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978209+0000","last_change":"2023-11-25T04:25:49.922396+0000","last_active":"2023-11-25T04:25:59.978209+0000","last_peered":"2023-11-25T04:25:59.978209+0000","last_clean":"2023-11-25T04:25:59.978209+0000","last_became_active":"2023-11-25T04:25:49.921129+0000","last_became_peered":"2023-11-25T04:25:49.921129+0000","last_unstale":"2023-11-25T04:25:59.978209+0000","last_undegraded":"2023-11-25T04:25:59.978209+0000","last_fullsized":"2023-11-25T04:25:59.978209+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:57:02.387207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":163190,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":169,"num_write_kb":336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":154576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"54'174","reported_seq":239,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978986+0000","last_change":"2023-11-25T04:25:53.934614+0000","last_active":"2023-11-25T04:25:59.978986+0000","last_peered":"2023-11-25T04:25:59.978986+0000","last_clean":"2023-11-25T04:25:59.978986+0000","last_became_active":"2023-11-25T04:25:53.933395+0000","last_became_peered":"2023-11-25T04:25:53.933395+0000","last_unstale":"2023-11-25T04:25:59.978986+0000","last_undegraded":"2023-11-25T04:25:59.978986+0000","last_fullsized":"2023-11-25T04:25:59.978986+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:53:09.827809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1423587,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":182,"num_write_kb":1626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":1414508,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2f","version":"54'152","reported_seq":220,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979084+0000","last_change":"2023-11-25T04:22:47.039389+0000","last_active":"2023-11-25T04:25:59.979084+0000","last_peered":"2023-11-25T04:25:59.979084+0000","last_clean":"2023-11-25T04:25:59.979084+0000","last_became_active":"2023-11-25T04:22:47.039123+0000","last_became_peered":"2023-11-25T04:22:47.039123+0000","last_unstale":"2023-11-25T04:25:59.979084+0000","last_undegraded":"2023-11-25T04:25:59.979084+0000","last_fullsized":"2023-11-25T04:25:59.979084+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:09:11.570561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":122752,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":143,"num_write_kb":435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"54'148","reported_seq":216,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979169+0000","last_change":"2023-11-25T04:22:47.036946+0000","last_active":"2023-11-25T04:25:59.979169+0000","last_peered":"2023-11-25T04:25:59.979169+0000","last_clean":"2023-11-25T04:25:59.979169+0000","last_became_active":"2023-11-25T04:22:47.036411+0000","last_became_peered":"2023-11-25T04:22:47.036411+0000","last_unstale":"2023-11-25T04:25:59.979169+0000","last_undegraded":"2023-11-25T04:25:59.979169+0000","last_fullsized":"2023-11-25T04:25:59.979169+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:16:29.085050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4770899,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":140,"num_write_kb":4870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2d","version":"54'143","reported_seq":211,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978472+0000","last_change":"2023-11-25T04:22:47.038983+0000","last_active":"2023-11-25T04:25:59.978472+0000","last_peered":"2023-11-25T04:25:59.978472+0000","last_clean":"2023-11-25T04:25:59.978472+0000","last_became_active":"2023-11-25T04:22:47.038477+0000","last_became_peered":"2023-11-25T04:22:47.038477+0000","last_unstale":"2023-11-25T04:25:59.978472+0000","last_undegraded":"2023-11-25T04:25:59.978472+0000","last_fullsized":"2023-11-25T04:25:59.978472+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:30:04.568790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":115975,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"54'167","reported_seq":233,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978615+0000","last_change":"2023-11-25T04:25:52.923640+0000","last_active":"2023-11-25T04:25:59.978615+0000","last_peered":"2023-11-25T04:25:59.978615+0000","last_clean":"2023-11-25T04:25:59.978615+0000","last_became_active":"2023-11-25T04:25:52.922367+0000","last_became_peered":"2023-11-25T04:25:52.922367+0000","last_unstale":"2023-11-25T04:25:59.978615+0000","last_undegraded":"2023-11-25T04:25:59.978615+0000","last_fullsized":"2023-11-25T04:25:59.978615+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:08:24.794280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":747090,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":211,"num_write_kb":985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":732276,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"54'156","reported_seq":222,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978632+0000","last_change":"2023-11-25T04:25:52.924457+0000","last_active":"2023-11-25T04:25:59.978632+0000","last_peered":"2023-11-25T04:25:59.978632+0000","last_clean":"2023-11-25T04:25:59.978632+0000","last_became_active":"2023-11-25T04:25:52.923378+0000","last_became_peered":"2023-11-25T04:25:52.923378+0000","last_unstale":"2023-11-25T04:25:59.978632+0000","last_undegraded":"2023-11-25T04:25:59.978632+0000","last_fullsized":"2023-11-25T04:25:59.978632+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:34:30.942779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69580,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":142,"num_write_kb":315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":98,"num_bytes_recovered":69580,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2a","version":"54'171","reported_seq":467,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978347+0000","last_change":"2023-11-25T04:25:52.925532+0000","last_active":"2023-11-25T04:25:59.978347+0000","last_peered":"2023-11-25T04:25:59.978347+0000","last_clean":"2023-11-25T04:25:59.978347+0000","last_became_active":"2023-11-25T04:25:52.924844+0000","last_became_peered":"2023-11-25T04:25:52.924844+0000","last_unstale":"2023-11-25T04:25:59.978347+0000","last_undegraded":"2023-11-25T04:25:59.978347+0000","last_fullsized":"2023-11-25T04:25:59.978347+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:11:30.124326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":115997,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":162,"num_write_kb":261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":115997,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.29","version":"54'152","reported_seq":438,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.977803+0000","last_change":"2023-11-25T04:25:49.920367+0000","last_active":"2023-11-25T04:25:59.977803+0000","last_peered":"2023-11-25T04:25:59.977803+0000","last_clean":"2023-11-25T04:25:59.977803+0000","last_became_active":"2023-11-25T04:25:49.919886+0000","last_became_peered":"2023-11-25T04:25:49.919886+0000","last_unstale":"2023-11-25T04:25:59.977803+0000","last_undegraded":"2023-11-25T04:25:59.977803+0000","last_fullsized":"2023-11-25T04:25:59.977803+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:04:58.273982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":212878,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":212878,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"54'169","reported_seq":237,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978222+0000","last_change":"2023-11-25T04:22:47.036805+0000","last_active":"2023-11-25T04:25:59.978222+0000","last_peered":"2023-11-25T04:25:59.978222+0000","last_clean":"2023-11-25T04:25:59.978222+0000","last_became_active":"2023-11-25T04:22:47.036135+0000","last_became_peered":"2023-11-25T04:22:47.036135+0000","last_unstale":"2023-11-25T04:25:59.978222+0000","last_undegraded":"2023-11-25T04:25:59.978222+0000","last_fullsized":"2023-11-25T04:25:59.978222+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:25:21.730992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":102957,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":172,"num_write_kb":610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"54'167","reported_seq":465,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979220+0000","last_change":"2023-11-25T04:25:51.922925+0000","last_active":"2023-11-25T04:25:59.979220+0000","last_peered":"2023-11-25T04:25:59.979220+0000","last_clean":"2023-11-25T04:25:59.979220+0000","last_became_active":"2023-11-25T04:25:51.922364+0000","last_became_peered":"2023-11-25T04:25:51.922364+0000","last_unstale":"2023-11-25T04:25:59.979220+0000","last_undegraded":"2023-11-25T04:25:59.979220+0000","last_fullsized":"2023-11-25T04:25:59.979220+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:09:23.741871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":182683,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":171,"num_write_kb":791,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":182683,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"54'187","reported_seq":505,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978723+0000","last_change":"2023-11-25T04:25:51.922529+0000","last_active":"2023-11-25T04:25:59.978723+0000","last_peered":"2023-11-25T04:25:59.978723+0000","last_clean":"2023-11-25T04:25:59.978723+0000","last_became_active":"2023-11-25T04:25:51.922064+0000","last_became_peered":"2023-11-25T04:25:51.922064+0000","last_unstale":"2023-11-25T04:25:59.978723+0000","last_undegraded":"2023-11-25T04:25:59.978723+0000","last_fullsized":"2023-11-25T04:25:59.978723+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":187,"log_dups_size":0,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:01:39.413986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":86311,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":171,"num_write_kb":274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":116,"num_bytes_recovered":86311,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.25","version":"54'149","reported_seq":215,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978829+0000","last_change":"2023-11-25T04:25:52.925505+0000","last_active":"2023-11-25T04:25:59.978829+0000","last_peered":"2023-11-25T04:25:59.978829+0000","last_clean":"2023-11-25T04:25:59.978829+0000","last_became_active":"2023-11-25T04:25:52.924795+0000","last_became_peered":"2023-11-25T04:25:52.924795+0000","last_unstale":"2023-11-25T04:25:59.978829+0000","last_undegraded":"2023-11-25T04:25:59.978829+0000","last_fullsized":"2023-11-25T04:25:59.978829+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:30:46.532141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":128288,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":167,"num_write_kb":267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":120265,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.24","version":"54'151","reported_seq":427,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978903+0000","last_change":"2023-11-25T04:25:50.922751+0000","last_active":"2023-11-25T04:25:59.978903+0000","last_peered":"2023-11-25T04:25:59.978903+0000","last_clean":"2023-11-25T04:25:59.978903+0000","last_became_active":"2023-11-25T04:25:50.922216+0000","last_became_peered":"2023-11-25T04:25:50.922216+0000","last_unstale":"2023-11-25T04:25:59.978903+0000","last_undegraded":"2023-11-25T04:25:59.978903+0000","last_fullsized":"2023-11-25T04:25:59.978903+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:40:16.255665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":107507,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":148,"num_write_kb":288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":107494,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.23","version":"54'155","reported_seq":223,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978207+0000","last_change":"2023-11-25T04:22:47.037668+0000","last_active":"2023-11-25T04:25:59.978207+0000","last_peered":"2023-11-25T04:25:59.978207+0000","last_clean":"2023-11-25T04:25:59.978207+0000","last_became_active":"2023-11-25T04:22:47.036823+0000","last_became_peered":"2023-11-25T04:22:47.036823+0000","last_unstale":"2023-11-25T04:25:59.978207+0000","last_undegraded":"2023-11-25T04:25:59.978207+0000","last_fullsized":"2023-11-25T04:25:59.978207+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:02:07.088241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4304582,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":152,"num_write_kb":4331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.22","version":"54'189","reported_seq":517,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978625+0000","last_change":"2023-11-25T04:25:54.940559+0000","last_active":"2023-11-25T04:25:59.978625+0000","last_peered":"2023-11-25T04:25:59.978625+0000","last_clean":"2023-11-25T04:25:59.978625+0000","last_became_active":"2023-11-25T04:25:54.939584+0000","last_became_peered":"2023-11-25T04:25:54.939584+0000","last_unstale":"2023-11-25T04:25:59.978625+0000","last_undegraded":"2023-11-25T04:25:59.978625+0000","last_fullsized":"2023-11-25T04:25:59.978625+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:01:39.413986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":129145,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":185,"num_write_kb":432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":121,"num_bytes_recovered":129107,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"54'201","reported_seq":269,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.977881+0000","last_change":"2023-11-25T04:22:47.037786+0000","last_active":"2023-11-25T04:25:59.977881+0000","last_peered":"2023-11-25T04:25:59.977881+0000","last_clean":"2023-11-25T04:25:59.977881+0000","last_became_active":"2023-11-25T04:22:47.037206+0000","last_became_peered":"2023-11-25T04:22:47.037206+0000","last_unstale":"2023-11-25T04:25:59.977881+0000","last_undegraded":"2023-11-25T04:25:59.977881+0000","last_fullsized":"2023-11-25T04:25:59.977881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:11:40.532140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":215311,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":194,"num_write_kb":381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"54'180","reported_seq":245,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.977884+0000","last_change":"2023-11-25T04:25:53.933176+0000","last_active":"2023-11-25T04:25:59.977884+0000","last_peered":"2023-11-25T04:25:59.977884+0000","last_clean":"2023-11-25T04:25:59.977884+0000","last_became_active":"2023-11-25T04:25:53.931937+0000","last_became_peered":"2023-11-25T04:25:53.931937+0000","last_unstale":"2023-11-25T04:25:59.977884+0000","last_undegraded":"2023-11-25T04:25:59.977884+0000","last_fullsized":"2023-11-25T04:25:59.977884+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:41:10.697064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104603,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":172,"num_write_kb":267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":104603,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"54'137","reported_seq":205,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978811+0000","last_change":"2023-11-25T04:22:47.036689+0000","last_active":"2023-11-25T04:25:59.978811+0000","last_peered":"2023-11-25T04:25:59.978811+0000","last_clean":"2023-11-25T04:25:59.978811+0000","last_became_active":"2023-11-25T04:22:47.036107+0000","last_became_peered":"2023-11-25T04:22:47.036107+0000","last_unstale":"2023-11-25T04:25:59.978811+0000","last_undegraded":"2023-11-25T04:25:59.978811+0000","last_fullsized":"2023-11-25T04:25:59.978811+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:02:07.088241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":183586,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":127,"num_write_kb":284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"58'114","reported_seq":191,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978227+0000","last_change":"2023-11-25T04:25:49.920320+0000","last_active":"2023-11-25T04:25:59.978227+0000","last_peered":"2023-11-25T04:25:59.978227+0000","last_clean":"2023-11-25T04:25:59.978227+0000","last_became_active":"2023-11-25T04:25:49.920010+0000","last_became_peered":"2023-11-25T04:25:49.920010+0000","last_unstale":"2023-11-25T04:25:59.978227+0000","last_undegraded":"2023-11-25T04:25:59.978227+0000","last_fullsized":"2023-11-25T04:25:59.978227+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:48:58.746789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2816216,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":113,"num_write_kb":2816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":2816182,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"54'145","reported_seq":213,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978995+0000","last_change":"2023-11-25T04:22:47.037888+0000","last_active":"2023-11-25T04:25:59.978995+0000","last_peered":"2023-11-25T04:25:59.978995+0000","last_clean":"2023-11-25T04:25:59.978995+0000","last_became_active":"2023-11-25T04:22:47.037158+0000","last_became_peered":"2023-11-25T04:22:47.037158+0000","last_unstale":"2023-11-25T04:25:59.978995+0000","last_undegraded":"2023-11-25T04:25:59.978995+0000","last_fullsized":"2023-11-25T04:25:59.978995+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:56:59.689578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":205976,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.12","version":"49'2","reported_seq":82,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978301+0000","last_change":"2023-11-25T04:25:47.000916+0000","last_active":"2023-11-25T04:25:59.978301+0000","last_peered":"2023-11-25T04:25:59.978301+0000","last_clean":"2023-11-25T04:25:59.978301+0000","last_became_active":"2023-11-25T04:25:47.000516+0000","last_became_peered":"2023-11-25T04:25:47.000516+0000","last_unstale":"2023-11-25T04:25:59.978301+0000","last_undegraded":"2023-11-25T04:25:59.978301+0000","last_fullsized":"2023-11-25T04:25:59.978301+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:53:58.873447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"54'175","reported_seq":489,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978855+0000","last_change":"2023-11-25T04:25:52.923694+0000","last_active":"2023-11-25T04:25:59.978855+0000","last_peered":"2023-11-25T04:25:59.978855+0000","last_clean":"2023-11-25T04:25:59.978855+0000","last_became_active":"2023-11-25T04:25:52.923045+0000","last_became_peered":"2023-11-25T04:25:52.923045+0000","last_unstale":"2023-11-25T04:25:59.978855+0000","last_undegraded":"2023-11-25T04:25:59.978855+0000","last_fullsized":"2023-11-25T04:25:59.978855+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:46:12.914433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":171823,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":185,"num_write_kb":340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":166387,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":80,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978355+0000","last_change":"2023-11-25T04:25:46.994324+0000","last_active":"2023-11-25T04:25:59.978355+0000","last_peered":"2023-11-25T04:25:59.978355+0000","last_clean":"2023-11-25T04:25:59.978355+0000","last_became_active":"2023-11-25T04:25:46.993870+0000","last_became_peered":"2023-11-25T04:25:46.993870+0000","last_unstale":"2023-11-25T04:25:59.978355+0000","last_undegraded":"2023-11-25T04:25:59.978355+0000","last_fullsized":"2023-11-25T04:25:59.978355+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:51:09.827080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"54'152","reported_seq":435,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978827+0000","last_change":"2023-11-25T04:25:50.922922+0000","last_active":"2023-11-25T04:25:59.978827+0000","last_peered":"2023-11-25T04:25:59.978827+0000","last_clean":"2023-11-25T04:25:59.978827+0000","last_became_active":"2023-11-25T04:25:50.922511+0000","last_became_peered":"2023-11-25T04:25:50.922511+0000","last_unstale":"2023-11-25T04:25:59.978827+0000","last_undegraded":"2023-11-25T04:25:59.978827+0000","last_fullsized":"2023-11-25T04:25:59.978827+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:37:14.620534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":579704,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":97,"num_bytes_recovered":570044,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.10","version":"18'8","reported_seq":39,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.977878+0000","last_change":"2023-11-25T04:25:47.000499+0000","last_active":"2023-11-25T04:25:59.977878+0000","last_peered":"2023-11-25T04:25:59.977878+0000","last_clean":"2023-11-25T04:25:59.977878+0000","last_became_active":"2023-11-25T04:25:46.998778+0000","last_became_peered":"2023-11-25T04:25:46.998778+0000","last_unstale":"2023-11-25T04:25:59.977878+0000","last_undegraded":"2023-11-25T04:25:59.977878+0000","last_fullsized":"2023-11-25T04:25:59.977878+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:33:09.169689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"49'2","reported_seq":37,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979103+0000","last_change":"2023-11-25T04:25:46.998392+0000","last_active":"2023-11-25T04:25:59.979103+0000","last_peered":"2023-11-25T04:25:59.979103+0000","last_clean":"2023-11-25T04:25:59.979103+0000","last_became_active":"2023-11-25T04:25:46.997108+0000","last_became_peered":"2023-11-25T04:25:46.997108+0000","last_unstale":"2023-11-25T04:25:59.979103+0000","last_undegraded":"2023-11-25T04:25:59.979103+0000","last_fullsized":"2023-11-25T04:25:59.979103+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:02:00.073630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"54'159","reported_seq":443,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978233+0000","last_change":"2023-11-25T04:25:50.922282+0000","last_active":"2023-11-25T04:25:59.978233+0000","last_peered":"2023-11-25T04:25:59.978233+0000","last_clean":"2023-11-25T04:25:59.978233+0000","last_became_active":"2023-11-25T04:25:50.921772+0000","last_became_peered":"2023-11-25T04:25:50.921772+0000","last_unstale":"2023-11-25T04:25:59.978233+0000","last_undegraded":"2023-11-25T04:25:59.978233+0000","last_fullsized":"2023-11-25T04:25:59.978233+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:36:33.451689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112103,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":164,"num_write_kb":240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":99,"num_bytes_recovered":103472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"18'2","reported_seq":82,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979281+0000","last_change":"2023-11-25T04:25:46.994943+0000","last_active":"2023-11-25T04:25:59.979281+0000","last_peered":"2023-11-25T04:25:59.979281+0000","last_clean":"2023-11-25T04:25:59.979281+0000","last_became_active":"2023-11-25T04:25:46.994640+0000","last_became_peered":"2023-11-25T04:25:46.994640+0000","last_unstale":"2023-11-25T04:25:59.979281+0000","last_undegraded":"2023-11-25T04:25:59.979281+0000","last_fullsized":"2023-11-25T04:25:59.979281+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:08:12.601941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"54'173","reported_seq":241,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978244+0000","last_change":"2023-11-25T04:22:47.039685+0000","last_active":"2023-11-25T04:25:59.978244+0000","last_peered":"2023-11-25T04:25:59.978244+0000","last_clean":"2023-11-25T04:25:59.978244+0000","last_became_active":"2023-11-25T04:22:47.039296+0000","last_became_peered":"2023-11-25T04:22:47.039296+0000","last_unstale":"2023-11-25T04:25:59.978244+0000","last_undegraded":"2023-11-25T04:25:59.978244+0000","last_fullsized":"2023-11-25T04:25:59.978244+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:11:40.532140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154533,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":182,"num_write_kb":299,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"18'1","reported_seq":37,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978725+0000","last_change":"2023-11-25T04:25:47.000059+0000","last_active":"2023-11-25T04:25:59.978725+0000","last_peered":"2023-11-25T04:25:59.978725+0000","last_clean":"2023-11-25T04:25:59.978725+0000","last_became_active":"2023-11-25T04:25:46.999604+0000","last_became_peered":"2023-11-25T04:25:46.999604+0000","last_unstale":"2023-11-25T04:25:59.978725+0000","last_undegraded":"2023-11-25T04:25:59.978725+0000","last_fullsized":"2023-11-25T04:25:59.978725+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:14:43.431272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"54'147","reported_seq":211,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978982+0000","last_change":"2023-11-25T04:25:54.939274+0000","last_active":"2023-11-25T04:25:59.978982+0000","last_peered":"2023-11-25T04:25:59.978982+0000","last_clean":"2023-11-25T04:25:59.978982+0000","last_became_active":"2023-11-25T04:25:54.938236+0000","last_became_peered":"2023-11-25T04:25:54.938236+0000","last_unstale":"2023-11-25T04:25:59.978982+0000","last_undegraded":"2023-11-25T04:25:59.978982+0000","last_fullsized":"2023-11-25T04:25:59.978982+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":147,"log_dups_size":0,"ondisk_log_size":147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:11:43.785294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":87524,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":155,"num_write_kb":235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":87524,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978546+0000","last_change":"2023-11-25T04:25:46.997080+0000","last_active":"2023-11-25T04:25:59.978546+0000","last_peered":"2023-11-25T04:25:59.978546+0000","last_clean":"2023-11-25T04:25:59.978546+0000","last_became_active":"2023-11-25T04:25:46.995712+0000","last_became_peered":"2023-11-25T04:25:46.995712+0000","last_unstale":"2023-11-25T04:25:59.978546+0000","last_undegraded":"2023-11-25T04:25:59.978546+0000","last_fullsized":"2023-11-25T04:25:59.978546+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T16:01:00.922843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"54'155","reported_seq":441,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978274+0000","last_change":"2023-11-25T04:25:52.923096+0000","last_active":"2023-11-25T04:25:59.978274+0000","last_peered":"2023-11-25T04:25:59.978274+0000","last_clean":"2023-11-25T04:25:59.978274+0000","last_became_active":"2023-11-25T04:25:52.922549+0000","last_became_peered":"2023-11-25T04:25:52.922549+0000","last_unstale":"2023-11-25T04:25:59.978274+0000","last_undegraded":"2023-11-25T04:25:59.978274+0000","last_fullsized":"2023-11-25T04:25:59.978274+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:52:38.053808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":266919,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":264258,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"45'11","reported_seq":96,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978532+0000","last_change":"2023-11-25T04:25:47.092497+0000","last_active":"2023-11-25T04:25:59.978532+0000","last_peered":"2023-11-25T04:25:59.978532+0000","last_clean":"2023-11-25T04:25:59.978532+0000","last_became_active":"2023-11-25T04:25:47.000687+0000","last_became_peered":"2023-11-25T04:25:47.000687+0000","last_unstale":"2023-11-25T04:25:59.978532+0000","last_undegraded":"2023-11-25T04:25:59.978532+0000","last_fullsized":"2023-11-25T04:25:59.978532+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:47:09.798238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":4101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4194304,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"54'173","reported_seq":241,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978732+0000","last_change":"2023-11-25T04:25:50.923240+0000","last_active":"2023-11-25T04:25:59.978732+0000","last_peered":"2023-11-25T04:25:59.978732+0000","last_clean":"2023-11-25T04:25:59.978732+0000","last_became_active":"2023-11-25T04:25:50.922409+0000","last_became_peered":"2023-11-25T04:25:50.922409+0000","last_unstale":"2023-11-25T04:25:59.978732+0000","last_undegraded":"2023-11-25T04:25:59.978732+0000","last_fullsized":"2023-11-25T04:25:59.978732+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:53:30.920627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":553893,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":169,"num_write_kb":770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":553893,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978079+0000","last_change":"2023-11-25T04:25:46.998818+0000","last_active":"2023-11-25T04:25:59.978079+0000","last_peered":"2023-11-25T04:25:59.978079+0000","last_clean":"2023-11-25T04:25:59.978079+0000","last_became_active":"2023-11-25T04:25:46.997019+0000","last_became_peered":"2023-11-25T04:25:46.997019+0000","last_unstale":"2023-11-25T04:25:59.978079+0000","last_undegraded":"2023-11-25T04:25:59.978079+0000","last_fullsized":"2023-11-25T04:25:59.978079+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:51:47.882117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"54'163","reported_seq":228,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978685+0000","last_change":"2023-11-25T04:25:53.934183+0000","last_active":"2023-11-25T04:25:59.978685+0000","last_peered":"2023-11-25T04:25:59.978685+0000","last_clean":"2023-11-25T04:25:59.978685+0000","last_became_active":"2023-11-25T04:25:53.933471+0000","last_became_peered":"2023-11-25T04:25:53.933471+0000","last_unstale":"2023-11-25T04:25:59.978685+0000","last_undegraded":"2023-11-25T04:25:59.978685+0000","last_fullsized":"2023-11-25T04:25:59.978685+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:57:07.515414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":320867,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":190,"num_write_kb":577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":108,"num_bytes_recovered":311640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":80,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978412+0000","last_change":"2023-11-25T04:25:46.994480+0000","last_active":"2023-11-25T04:25:59.978412+0000","last_peered":"2023-11-25T04:25:59.978412+0000","last_clean":"2023-11-25T04:25:59.978412+0000","last_became_active":"2023-11-25T04:25:46.994101+0000","last_became_peered":"2023-11-25T04:25:46.994101+0000","last_unstale":"2023-11-25T04:25:59.978412+0000","last_undegraded":"2023-11-25T04:25:59.978412+0000","last_fullsized":"2023-11-25T04:25:59.978412+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:53:58.873447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"54'141","reported_seq":210,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979040+0000","last_change":"2023-11-25T04:25:49.921950+0000","last_active":"2023-11-25T04:25:59.979040+0000","last_peered":"2023-11-25T04:25:59.979040+0000","last_clean":"2023-11-25T04:25:59.979040+0000","last_became_active":"2023-11-25T04:25:49.920971+0000","last_became_peered":"2023-11-25T04:25:49.920971+0000","last_unstale":"2023-11-25T04:25:59.979040+0000","last_undegraded":"2023-11-25T04:25:59.979040+0000","last_fullsized":"2023-11-25T04:25:59.979040+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:22:38.004934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3502371,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":3718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":89,"num_bytes_recovered":3497669,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"49'1","reported_seq":70,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978352+0000","last_change":"2023-11-25T04:22:46.034509+0000","last_active":"2023-11-25T04:25:59.978352+0000","last_peered":"2023-11-25T04:25:59.978352+0000","last_clean":"2023-11-25T04:25:59.978352+0000","last_became_active":"2023-11-25T04:22:46.034158+0000","last_became_peered":"2023-11-25T04:22:46.034158+0000","last_unstale":"2023-11-25T04:25:59.978352+0000","last_undegraded":"2023-11-25T04:25:59.978352+0000","last_fullsized":"2023-11-25T04:25:59.978352+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:26:25.338117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"54'141","reported_seq":209,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978912+0000","last_change":"2023-11-25T04:22:47.038800+0000","last_active":"2023-11-25T04:25:59.978912+0000","last_peered":"2023-11-25T04:25:59.978912+0000","last_clean":"2023-11-25T04:25:59.978912+0000","last_became_active":"2023-11-25T04:22:47.038081+0000","last_became_peered":"2023-11-25T04:22:47.038081+0000","last_unstale":"2023-11-25T04:25:59.978912+0000","last_undegraded":"2023-11-25T04:25:59.978912+0000","last_fullsized":"2023-11-25T04:25:59.978912+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:45:40.065146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":174088,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":146,"num_write_kb":385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"12'157","reported_seq":232,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.977771+0000","last_change":"2023-11-25T04:25:47.000504+0000","last_active":"2023-11-25T04:25:59.977771+0000","last_peered":"2023-11-25T04:25:59.977771+0000","last_clean":"2023-11-25T04:25:59.977771+0000","last_became_active":"2023-11-25T04:25:46.999721+0000","last_became_peered":"2023-11-25T04:25:46.999721+0000","last_unstale":"2023-11-25T04:25:59.977771+0000","last_undegraded":"2023-11-25T04:25:59.977771+0000","last_fullsized":"2023-11-25T04:25:59.977771+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:35.919850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:35.919850+0000","last_clean_scrub_stamp":"2023-11-25T04:22:35.919850+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:26:33.607474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978082+0000","last_change":"2023-11-25T04:25:46.998960+0000","last_active":"2023-11-25T04:25:59.978082+0000","last_peered":"2023-11-25T04:25:59.978082+0000","last_clean":"2023-11-25T04:25:59.978082+0000","last_became_active":"2023-11-25T04:25:46.997693+0000","last_became_peered":"2023-11-25T04:25:46.997693+0000","last_unstale":"2023-11-25T04:25:59.978082+0000","last_undegraded":"2023-11-25T04:25:59.978082+0000","last_fullsized":"2023-11-25T04:25:59.978082+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:51:45.180957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"46'1","reported_seq":70,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978331+0000","last_change":"2023-11-25T04:22:46.037044+0000","last_active":"2023-11-25T04:25:59.978331+0000","last_peered":"2023-11-25T04:25:59.978331+0000","last_clean":"2023-11-25T04:25:59.978331+0000","last_became_active":"2023-11-25T04:22:46.035916+0000","last_became_peered":"2023-11-25T04:22:46.035916+0000","last_unstale":"2023-11-25T04:25:59.978331+0000","last_undegraded":"2023-11-25T04:25:59.978331+0000","last_fullsized":"2023-11-25T04:25:59.978331+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:38:20.501102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"54'146","reported_seq":213,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978804+0000","last_change":"2023-11-25T04:25:51.923452+0000","last_active":"2023-11-25T04:25:59.978804+0000","last_peered":"2023-11-25T04:25:59.978804+0000","last_clean":"2023-11-25T04:25:59.978804+0000","last_became_active":"2023-11-25T04:25:51.922490+0000","last_became_peered":"2023-11-25T04:25:51.922490+0000","last_unstale":"2023-11-25T04:25:59.978804+0000","last_undegraded":"2023-11-25T04:25:59.978804+0000","last_fullsized":"2023-11-25T04:25:59.978804+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:40:11.001062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":502154,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":170,"num_write_kb":687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":99,"num_bytes_recovered":502101,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978713+0000","last_change":"2023-11-25T04:25:47.000763+0000","last_active":"2023-11-25T04:25:59.978713+0000","last_peered":"2023-11-25T04:25:59.978713+0000","last_clean":"2023-11-25T04:25:59.978713+0000","last_became_active":"2023-11-25T04:25:47.000033+0000","last_became_peered":"2023-11-25T04:25:47.000033+0000","last_unstale":"2023-11-25T04:25:59.978713+0000","last_undegraded":"2023-11-25T04:25:59.978713+0000","last_fullsized":"2023-11-25T04:25:59.978713+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:53:59.131478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"50'37","reported_seq":156,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978116+0000","last_change":"2023-11-25T04:25:47.002009+0000","last_active":"2023-11-25T04:25:59.978116+0000","last_peered":"2023-11-25T04:25:59.978116+0000","last_clean":"2023-11-25T04:25:59.978116+0000","last_became_active":"2023-11-25T04:25:46.995637+0000","last_became_peered":"2023-11-25T04:25:46.995637+0000","last_unstale":"2023-11-25T04:25:59.978116+0000","last_undegraded":"2023-11-25T04:25:59.978116+0000","last_fullsized":"2023-11-25T04:25:59.978116+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:55:01.351319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":319160,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":35,"num_read_kb":334,"num_write":35,"num_write_kb":334,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":319160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"54'158","reported_seq":226,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979092+0000","last_change":"2023-11-25T04:22:47.039583+0000","last_active":"2023-11-25T04:25:59.979092+0000","last_peered":"2023-11-25T04:25:59.979092+0000","last_clean":"2023-11-25T04:25:59.979092+0000","last_became_active":"2023-11-25T04:22:47.039197+0000","last_became_peered":"2023-11-25T04:22:47.039197+0000","last_unstale":"2023-11-25T04:25:59.979092+0000","last_undegraded":"2023-11-25T04:25:59.979092+0000","last_fullsized":"2023-11-25T04:25:59.979092+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:37:30.959147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":169303,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":189,"num_write_kb":333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":72,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978222+0000","last_change":"2023-11-25T04:22:42.000382+0000","last_active":"2023-11-25T04:25:59.978222+0000","last_peered":"2023-11-25T04:25:59.978222+0000","last_clean":"2023-11-25T04:25:59.978222+0000","last_became_active":"2023-11-25T04:22:42.000041+0000","last_became_peered":"2023-11-25T04:22:42.000041+0000","last_unstale":"2023-11-25T04:25:59.978222+0000","last_undegraded":"2023-11-25T04:25:59.978222+0000","last_fullsized":"2023-11-25T04:25:59.978222+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:08:19.655864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"49'71","reported_seq":155,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978346+0000","last_change":"2023-11-25T04:25:47.109628+0000","last_active":"2023-11-25T04:25:59.978346+0000","last_peered":"2023-11-25T04:25:59.978346+0000","last_clean":"2023-11-25T04:25:59.978346+0000","last_became_active":"2023-11-25T04:25:47.001162+0000","last_became_peered":"2023-11-25T04:25:47.001162+0000","last_unstale":"2023-11-25T04:25:59.978346+0000","last_undegraded":"2023-11-25T04:25:59.978346+0000","last_fullsized":"2023-11-25T04:25:59.978346+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:03:33.658392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":69,"num_write_kb":4130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4194304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"54'158","reported_seq":226,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978945+0000","last_change":"2023-11-25T04:22:47.038454+0000","last_active":"2023-11-25T04:25:59.978945+0000","last_peered":"2023-11-25T04:25:59.978945+0000","last_clean":"2023-11-25T04:25:59.978945+0000","last_became_active":"2023-11-25T04:22:47.038046+0000","last_became_peered":"2023-11-25T04:22:47.038046+0000","last_unstale":"2023-11-25T04:25:59.978945+0000","last_undegraded":"2023-11-25T04:25:59.978945+0000","last_fullsized":"2023-11-25T04:25:59.978945+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:38:38.839941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":115501,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":163,"num_write_kb":243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.977941+0000","last_change":"2023-11-25T04:25:46.998739+0000","last_active":"2023-11-25T04:25:59.977941+0000","last_peered":"2023-11-25T04:25:59.977941+0000","last_clean":"2023-11-25T04:25:59.977941+0000","last_became_active":"2023-11-25T04:25:46.997391+0000","last_became_peered":"2023-11-25T04:25:46.997391+0000","last_unstale":"2023-11-25T04:25:59.977941+0000","last_undegraded":"2023-11-25T04:25:59.977941+0000","last_fullsized":"2023-11-25T04:25:59.977941+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:47:46.300596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"50'1","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978166+0000","last_change":"2023-11-25T04:25:46.999446+0000","last_active":"2023-11-25T04:25:59.978166+0000","last_peered":"2023-11-25T04:25:59.978166+0000","last_clean":"2023-11-25T04:25:59.978166+0000","last_became_active":"2023-11-25T04:25:46.998132+0000","last_became_peered":"2023-11-25T04:25:46.998132+0000","last_unstale":"2023-11-25T04:25:59.978166+0000","last_undegraded":"2023-11-25T04:25:59.978166+0000","last_fullsized":"2023-11-25T04:25:59.978166+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:42:51.166678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"54'157","reported_seq":225,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978638+0000","last_change":"2023-11-25T04:22:47.037355+0000","last_active":"2023-11-25T04:25:59.978638+0000","last_peered":"2023-11-25T04:25:59.978638+0000","last_clean":"2023-11-25T04:25:59.978638+0000","last_became_active":"2023-11-25T04:22:47.036924+0000","last_became_peered":"2023-11-25T04:22:47.036924+0000","last_unstale":"2023-11-25T04:25:59.978638+0000","last_undegraded":"2023-11-25T04:25:59.978638+0000","last_fullsized":"2023-11-25T04:25:59.978638+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:15:39.511043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":93321,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":148,"num_write_kb":360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978520+0000","last_change":"2023-11-25T04:25:46.999738+0000","last_active":"2023-11-25T04:25:59.978520+0000","last_peered":"2023-11-25T04:25:59.978520+0000","last_clean":"2023-11-25T04:25:59.978520+0000","last_became_active":"2023-11-25T04:25:46.999403+0000","last_became_peered":"2023-11-25T04:25:46.999403+0000","last_unstale":"2023-11-25T04:25:59.978520+0000","last_undegraded":"2023-11-25T04:25:59.978520+0000","last_fullsized":"2023-11-25T04:25:59.978520+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:45:53.304063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":72,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978705+0000","last_change":"2023-11-25T04:22:42.000204+0000","last_active":"2023-11-25T04:25:59.978705+0000","last_peered":"2023-11-25T04:25:59.978705+0000","last_clean":"2023-11-25T04:25:59.978705+0000","last_became_active":"2023-11-25T04:22:41.999915+0000","last_became_peered":"2023-11-25T04:22:41.999915+0000","last_unstale":"2023-11-25T04:25:59.978705+0000","last_undegraded":"2023-11-25T04:25:59.978705+0000","last_fullsized":"2023-11-25T04:25:59.978705+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:07:32.457905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"54'152","reported_seq":432,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978379+0000","last_change":"2023-11-25T04:25:53.933239+0000","last_active":"2023-11-25T04:25:59.978379+0000","last_peered":"2023-11-25T04:25:59.978379+0000","last_clean":"2023-11-25T04:25:59.978379+0000","last_became_active":"2023-11-25T04:25:53.932586+0000","last_became_peered":"2023-11-25T04:25:53.932586+0000","last_unstale":"2023-11-25T04:25:59.978379+0000","last_undegraded":"2023-11-25T04:25:59.978379+0000","last_fullsized":"2023-11-25T04:25:59.978379+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:54:51.339025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":329327,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":151,"num_write_kb":471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":97,"num_bytes_recovered":329278,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":69,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978632+0000","last_change":"2023-11-25T04:22:46.035885+0000","last_active":"2023-11-25T04:25:59.978632+0000","last_peered":"2023-11-25T04:25:59.978632+0000","last_clean":"2023-11-25T04:25:59.978632+0000","last_became_active":"2023-11-25T04:22:46.035405+0000","last_became_peered":"2023-11-25T04:22:46.035405+0000","last_unstale":"2023-11-25T04:25:59.978632+0000","last_undegraded":"2023-11-25T04:25:59.978632+0000","last_fullsized":"2023-11-25T04:25:59.978632+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:53:47.398746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":72,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979221+0000","last_change":"2023-11-25T04:22:41.999067+0000","last_active":"2023-11-25T04:25:59.979221+0000","last_peered":"2023-11-25T04:25:59.979221+0000","last_clean":"2023-11-25T04:25:59.979221+0000","last_became_active":"2023-11-25T04:22:41.998798+0000","last_became_peered":"2023-11-25T04:22:41.998798+0000","last_unstale":"2023-11-25T04:25:59.979221+0000","last_undegraded":"2023-11-25T04:25:59.979221+0000","last_fullsized":"2023-11-25T04:25:59.979221+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:00:04.266051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"54'172","reported_seq":470,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978595+0000","last_change":"2023-11-25T04:25:51.922403+0000","last_active":"2023-11-25T04:25:59.978595+0000","last_peered":"2023-11-25T04:25:59.978595+0000","last_clean":"2023-11-25T04:25:59.978595+0000","last_became_active":"2023-11-25T04:25:51.921858+0000","last_became_peered":"2023-11-25T04:25:51.921858+0000","last_unstale":"2023-11-25T04:25:59.978595+0000","last_undegraded":"2023-11-25T04:25:59.978595+0000","last_fullsized":"2023-11-25T04:25:59.978595+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:04:58.273982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":129965,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":166,"num_write_kb":290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":129965,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"18'2","reported_seq":71,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979088+0000","last_change":"2023-11-25T04:22:46.034606+0000","last_active":"2023-11-25T04:25:59.979088+0000","last_peered":"2023-11-25T04:25:59.979088+0000","last_clean":"2023-11-25T04:25:59.979088+0000","last_became_active":"2023-11-25T04:22:46.034229+0000","last_became_peered":"2023-11-25T04:22:46.034229+0000","last_unstale":"2023-11-25T04:25:59.979088+0000","last_undegraded":"2023-11-25T04:25:59.979088+0000","last_fullsized":"2023-11-25T04:25:59.979088+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:21:42.146264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":72,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978952+0000","last_change":"2023-11-25T04:22:42.000258+0000","last_active":"2023-11-25T04:25:59.978952+0000","last_peered":"2023-11-25T04:25:59.978952+0000","last_clean":"2023-11-25T04:25:59.978952+0000","last_became_active":"2023-11-25T04:22:41.999992+0000","last_became_peered":"2023-11-25T04:22:41.999992+0000","last_unstale":"2023-11-25T04:25:59.978952+0000","last_undegraded":"2023-11-25T04:25:59.978952+0000","last_fullsized":"2023-11-25T04:25:59.978952+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:31:13.715992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"54'150","reported_seq":436,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.977926+0000","last_change":"2023-11-25T04:25:54.938479+0000","last_active":"2023-11-25T04:25:59.977926+0000","last_peered":"2023-11-25T04:25:59.977926+0000","last_clean":"2023-11-25T04:25:59.977926+0000","last_became_active":"2023-11-25T04:25:54.937838+0000","last_became_peered":"2023-11-25T04:25:54.937838+0000","last_unstale":"2023-11-25T04:25:59.977926+0000","last_undegraded":"2023-11-25T04:25:59.977926+0000","last_fullsized":"2023-11-25T04:25:59.977926+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:51:14.394190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":159585,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":154224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"54'14","reported_seq":99,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978916+0000","last_change":"2023-11-25T04:25:47.026749+0000","last_active":"2023-11-25T04:25:59.978916+0000","last_peered":"2023-11-25T04:25:59.978916+0000","last_clean":"2023-11-25T04:25:59.978916+0000","last_became_active":"2023-11-25T04:25:46.997404+0000","last_became_peered":"2023-11-25T04:25:46.997404+0000","last_unstale":"2023-11-25T04:25:59.978916+0000","last_undegraded":"2023-11-25T04:25:59.978916+0000","last_fullsized":"2023-11-25T04:25:59.978916+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:20:31.719529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978930+0000","last_change":"2023-11-25T04:25:46.995200+0000","last_active":"2023-11-25T04:25:59.978930+0000","last_peered":"2023-11-25T04:25:59.978930+0000","last_clean":"2023-11-25T04:25:59.978930+0000","last_became_active":"2023-11-25T04:25:46.994834+0000","last_became_peered":"2023-11-25T04:25:46.994834+0000","last_unstale":"2023-11-25T04:25:59.978930+0000","last_undegraded":"2023-11-25T04:25:59.978930+0000","last_fullsized":"2023-11-25T04:25:59.978930+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:33:07.913911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"54'135","reported_seq":407,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978276+0000","last_change":"2023-11-25T04:25:49.922046+0000","last_active":"2023-11-25T04:25:59.978276+0000","last_peered":"2023-11-25T04:25:59.978276+0000","last_clean":"2023-11-25T04:25:59.978276+0000","last_became_active":"2023-11-25T04:25:49.921465+0000","last_became_peered":"2023-11-25T04:25:49.921465+0000","last_unstale":"2023-11-25T04:25:59.978276+0000","last_undegraded":"2023-11-25T04:25:59.978276+0000","last_fullsized":"2023-11-25T04:25:59.978276+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:12:54.077901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":99503,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":148,"num_write_kb":250,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":93,"num_bytes_recovered":99503,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"54'166","reported_seq":233,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978407+0000","last_change":"2023-11-25T04:25:51.923141+0000","last_active":"2023-11-25T04:25:59.978407+0000","last_peered":"2023-11-25T04:25:59.978407+0000","last_clean":"2023-11-25T04:25:59.978407+0000","last_became_active":"2023-11-25T04:25:51.922070+0000","last_became_peered":"2023-11-25T04:25:51.922070+0000","last_unstale":"2023-11-25T04:25:59.978407+0000","last_undegraded":"2023-11-25T04:25:59.978407+0000","last_fullsized":"2023-11-25T04:25:59.978407+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:34:53.021081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":185321,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":172,"num_write_kb":312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":164176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"58'50","reported_seq":53,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978227+0000","last_change":"2023-11-25T04:25:47.021705+0000","last_active":"2023-11-25T04:25:59.978227+0000","last_peered":"2023-11-25T04:25:59.978227+0000","last_clean":"2023-11-25T04:25:59.978227+0000","last_became_active":"2023-11-25T04:25:46.994997+0000","last_became_peered":"2023-11-25T04:25:46.994997+0000","last_unstale":"2023-11-25T04:25:59.978227+0000","last_undegraded":"2023-11-25T04:25:59.978227+0000","last_fullsized":"2023-11-25T04:25:59.978227+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:41:04.147372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":4119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4190183,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"54'153","reported_seq":221,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978107+0000","last_change":"2023-11-25T04:22:47.037181+0000","last_active":"2023-11-25T04:25:59.978107+0000","last_peered":"2023-11-25T04:25:59.978107+0000","last_clean":"2023-11-25T04:25:59.978107+0000","last_became_active":"2023-11-25T04:22:47.036821+0000","last_became_peered":"2023-11-25T04:22:47.036821+0000","last_unstale":"2023-11-25T04:25:59.978107+0000","last_undegraded":"2023-11-25T04:25:59.978107+0000","last_fullsized":"2023-11-25T04:25:59.978107+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:12:51.990448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":101117,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":161,"num_write_kb":242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"50'22","reported_seq":107,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979124+0000","last_change":"2023-11-25T04:25:47.028636+0000","last_active":"2023-11-25T04:25:59.979124+0000","last_peered":"2023-11-25T04:25:59.979124+0000","last_clean":"2023-11-25T04:25:59.979124+0000","last_became_active":"2023-11-25T04:25:47.000478+0000","last_became_peered":"2023-11-25T04:25:47.000478+0000","last_unstale":"2023-11-25T04:25:59.979124+0000","last_undegraded":"2023-11-25T04:25:59.979124+0000","last_fullsized":"2023-11-25T04:25:59.979124+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:50:13.896985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":90,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978907+0000","last_change":"2023-11-25T04:25:46.996182+0000","last_active":"2023-11-25T04:25:59.978907+0000","last_peered":"2023-11-25T04:25:59.978907+0000","last_clean":"2023-11-25T04:25:59.978907+0000","last_became_active":"2023-11-25T04:25:46.994525+0000","last_became_peered":"2023-11-25T04:25:46.994525+0000","last_unstale":"2023-11-25T04:25:59.978907+0000","last_undegraded":"2023-11-25T04:25:59.978907+0000","last_fullsized":"2023-11-25T04:25:59.978907+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:26:24.469530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"54'177","reported_seq":485,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.977915+0000","last_change":"2023-11-25T04:25:51.922466+0000","last_active":"2023-11-25T04:25:59.977915+0000","last_peered":"2023-11-25T04:25:59.977915+0000","last_clean":"2023-11-25T04:25:59.977915+0000","last_became_active":"2023-11-25T04:25:51.921938+0000","last_became_peered":"2023-11-25T04:25:51.921938+0000","last_unstale":"2023-11-25T04:25:59.977915+0000","last_undegraded":"2023-11-25T04:25:59.977915+0000","last_fullsized":"2023-11-25T04:25:59.977915+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:38:59.016525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":522494,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":168,"num_write_kb":749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":111,"num_bytes_recovered":522494,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"46'2","reported_seq":82,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979031+0000","last_change":"2023-11-25T04:25:46.998867+0000","last_active":"2023-11-25T04:25:59.979031+0000","last_peered":"2023-11-25T04:25:59.979031+0000","last_clean":"2023-11-25T04:25:59.979031+0000","last_became_active":"2023-11-25T04:25:46.998508+0000","last_became_peered":"2023-11-25T04:25:46.998508+0000","last_unstale":"2023-11-25T04:25:59.979031+0000","last_undegraded":"2023-11-25T04:25:59.979031+0000","last_fullsized":"2023-11-25T04:25:59.979031+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:50:13.896985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.10","version":"54'153","reported_seq":221,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.977904+0000","last_change":"2023-11-25T04:22:47.037790+0000","last_active":"2023-11-25T04:25:59.977904+0000","last_peered":"2023-11-25T04:25:59.977904+0000","last_clean":"2023-11-25T04:25:59.977904+0000","last_became_active":"2023-11-25T04:22:47.037149+0000","last_became_peered":"2023-11-25T04:22:47.037149+0000","last_unstale":"2023-11-25T04:25:59.977904+0000","last_undegraded":"2023-11-25T04:25:59.977904+0000","last_fullsized":"2023-11-25T04:25:59.977904+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:31:54.857270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":218073,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1f","version":"54'156","reported_seq":448,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979259+0000","last_change":"2023-11-25T04:25:49.921904+0000","last_active":"2023-11-25T04:25:59.979259+0000","last_peered":"2023-11-25T04:25:59.979259+0000","last_clean":"2023-11-25T04:25:59.979259+0000","last_became_active":"2023-11-25T04:25:49.921458+0000","last_became_peered":"2023-11-25T04:25:49.921458+0000","last_unstale":"2023-11-25T04:25:59.979259+0000","last_undegraded":"2023-11-25T04:25:59.979259+0000","last_fullsized":"2023-11-25T04:25:59.979259+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:59:27.602060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":107712,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":103,"num_bytes_recovered":107703,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"48'1","reported_seq":70,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978057+0000","last_change":"2023-11-25T04:22:46.036085+0000","last_active":"2023-11-25T04:25:59.978057+0000","last_peered":"2023-11-25T04:25:59.978057+0000","last_clean":"2023-11-25T04:25:59.978057+0000","last_became_active":"2023-11-25T04:22:46.035495+0000","last_became_peered":"2023-11-25T04:22:46.035495+0000","last_unstale":"2023-11-25T04:25:59.978057+0000","last_undegraded":"2023-11-25T04:25:59.978057+0000","last_fullsized":"2023-11-25T04:25:59.978057+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:14:56.735365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"54'152","reported_seq":430,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978757+0000","last_change":"2023-11-25T04:25:50.923882+0000","last_active":"2023-11-25T04:25:59.978757+0000","last_peered":"2023-11-25T04:25:59.978757+0000","last_clean":"2023-11-25T04:25:59.978757+0000","last_became_active":"2023-11-25T04:25:50.923375+0000","last_became_peered":"2023-11-25T04:25:50.923375+0000","last_unstale":"2023-11-25T04:25:59.978757+0000","last_undegraded":"2023-11-25T04:25:59.978757+0000","last_fullsized":"2023-11-25T04:25:59.978757+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:54:51.339025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":76435,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":76435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":80,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978069+0000","last_change":"2023-11-25T04:25:46.995228+0000","last_active":"2023-11-25T04:25:59.978069+0000","last_peered":"2023-11-25T04:25:59.978069+0000","last_clean":"2023-11-25T04:25:59.978069+0000","last_became_active":"2023-11-25T04:25:46.994871+0000","last_became_peered":"2023-11-25T04:25:46.994871+0000","last_unstale":"2023-11-25T04:25:59.978069+0000","last_undegraded":"2023-11-25T04:25:59.978069+0000","last_fullsized":"2023-11-25T04:25:59.978069+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:53:34.807075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"54'158","reported_seq":432,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978899+0000","last_change":"2023-11-25T04:25:52.925523+0000","last_active":"2023-11-25T04:25:59.978899+0000","last_peered":"2023-11-25T04:25:59.978899+0000","last_clean":"2023-11-25T04:25:59.978899+0000","last_became_active":"2023-11-25T04:25:52.925038+0000","last_became_peered":"2023-11-25T04:25:52.925038+0000","last_unstale":"2023-11-25T04:25:59.978899+0000","last_undegraded":"2023-11-25T04:25:59.978899+0000","last_fullsized":"2023-11-25T04:25:59.978899+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:37:14.620534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":209500,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":139,"num_write_kb":351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":209500,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1a","version":"48'4","reported_seq":73,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978665+0000","last_change":"2023-11-25T04:22:46.034340+0000","last_active":"2023-11-25T04:25:59.978665+0000","last_peered":"2023-11-25T04:25:59.978665+0000","last_clean":"2023-11-25T04:25:59.978665+0000","last_became_active":"2023-11-25T04:22:46.033944+0000","last_became_peered":"2023-11-25T04:22:46.033944+0000","last_unstale":"2023-11-25T04:25:59.978665+0000","last_undegraded":"2023-11-25T04:25:59.978665+0000","last_fullsized":"2023-11-25T04:25:59.978665+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:59:30.291004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"54'163","reported_seq":231,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978741+0000","last_change":"2023-11-25T04:22:47.039547+0000","last_active":"2023-11-25T04:25:59.978741+0000","last_peered":"2023-11-25T04:25:59.978741+0000","last_clean":"2023-11-25T04:25:59.978741+0000","last_became_active":"2023-11-25T04:22:47.039218+0000","last_became_peered":"2023-11-25T04:22:47.039218+0000","last_unstale":"2023-11-25T04:25:59.978741+0000","last_undegraded":"2023-11-25T04:25:59.978741+0000","last_fullsized":"2023-11-25T04:25:59.978741+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:44:48.553972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":165883,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"45'9","reported_seq":94,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978700+0000","last_change":"2023-11-25T04:25:47.023365+0000","last_active":"2023-11-25T04:25:59.978700+0000","last_peered":"2023-11-25T04:25:59.978700+0000","last_clean":"2023-11-25T04:25:59.978700+0000","last_became_active":"2023-11-25T04:25:46.999998+0000","last_became_peered":"2023-11-25T04:25:46.999998+0000","last_unstale":"2023-11-25T04:25:59.978700+0000","last_undegraded":"2023-11-25T04:25:59.978700+0000","last_fullsized":"2023-11-25T04:25:59.978700+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:53:34.807075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":4101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4194304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"54'149","reported_seq":217,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978573+0000","last_change":"2023-11-25T04:25:50.923041+0000","last_active":"2023-11-25T04:25:59.978573+0000","last_peered":"2023-11-25T04:25:59.978573+0000","last_clean":"2023-11-25T04:25:59.978573+0000","last_became_active":"2023-11-25T04:25:50.922164+0000","last_became_peered":"2023-11-25T04:25:50.922164+0000","last_unstale":"2023-11-25T04:25:59.978573+0000","last_undegraded":"2023-11-25T04:25:59.978573+0000","last_fullsized":"2023-11-25T04:25:59.978573+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:31:10.744814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":179506,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":155,"num_write_kb":317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":179506,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"18'2","reported_seq":37,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978409+0000","last_change":"2023-11-25T04:25:46.996675+0000","last_active":"2023-11-25T04:25:59.978409+0000","last_peered":"2023-11-25T04:25:59.978409+0000","last_clean":"2023-11-25T04:25:59.978409+0000","last_became_active":"2023-11-25T04:25:46.995160+0000","last_became_peered":"2023-11-25T04:25:46.995160+0000","last_unstale":"2023-11-25T04:25:59.978409+0000","last_undegraded":"2023-11-25T04:25:59.978409+0000","last_fullsized":"2023-11-25T04:25:59.978409+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:59:39.131605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1a","version":"54'162","reported_seq":226,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978332+0000","last_change":"2023-11-25T04:25:54.939182+0000","last_active":"2023-11-25T04:25:59.978332+0000","last_peered":"2023-11-25T04:25:59.978332+0000","last_clean":"2023-11-25T04:25:59.978332+0000","last_became_active":"2023-11-25T04:25:54.937821+0000","last_became_peered":"2023-11-25T04:25:54.937821+0000","last_unstale":"2023-11-25T04:25:59.978332+0000","last_undegraded":"2023-11-25T04:25:59.978332+0000","last_fullsized":"2023-11-25T04:25:59.978332+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:05:34.575966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":134672,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":169,"num_write_kb":311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":126291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"48'59","reported_seq":146,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979001+0000","last_change":"2023-11-25T04:25:47.077194+0000","last_active":"2023-11-25T04:25:59.979001+0000","last_peered":"2023-11-25T04:25:59.979001+0000","last_clean":"2023-11-25T04:25:59.979001+0000","last_became_active":"2023-11-25T04:25:47.000722+0000","last_became_peered":"2023-11-25T04:25:47.000722+0000","last_unstale":"2023-11-25T04:25:59.979001+0000","last_undegraded":"2023-11-25T04:25:59.979001+0000","last_fullsized":"2023-11-25T04:25:59.979001+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:55:01.351319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8388608,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"54'163","reported_seq":231,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978500+0000","last_change":"2023-11-25T04:22:47.036764+0000","last_active":"2023-11-25T04:25:59.978500+0000","last_peered":"2023-11-25T04:25:59.978500+0000","last_clean":"2023-11-25T04:25:59.978500+0000","last_became_active":"2023-11-25T04:22:47.036406+0000","last_became_peered":"2023-11-25T04:22:47.036406+0000","last_unstale":"2023-11-25T04:25:59.978500+0000","last_undegraded":"2023-11-25T04:25:59.978500+0000","last_fullsized":"2023-11-25T04:25:59.978500+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:09:55.357642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":527891,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":163,"num_write_kb":687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1e","version":"46'2","reported_seq":71,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978868+0000","last_change":"2023-11-25T04:22:46.035659+0000","last_active":"2023-11-25T04:25:59.978868+0000","last_peered":"2023-11-25T04:25:59.978868+0000","last_clean":"2023-11-25T04:25:59.978868+0000","last_became_active":"2023-11-25T04:22:46.035113+0000","last_became_peered":"2023-11-25T04:22:46.035113+0000","last_unstale":"2023-11-25T04:25:59.978868+0000","last_undegraded":"2023-11-25T04:25:59.978868+0000","last_fullsized":"2023-11-25T04:25:59.978868+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:26:25.338117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.18","version":"54'195","reported_seq":259,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978263+0000","last_change":"2023-11-25T04:25:54.938573+0000","last_active":"2023-11-25T04:25:59.978263+0000","last_peered":"2023-11-25T04:25:59.978263+0000","last_clean":"2023-11-25T04:25:59.978263+0000","last_became_active":"2023-11-25T04:25:54.937444+0000","last_became_peered":"2023-11-25T04:25:54.937444+0000","last_unstale":"2023-11-25T04:25:59.978263+0000","last_undegraded":"2023-11-25T04:25:59.978263+0000","last_fullsized":"2023-11-25T04:25:59.978263+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:20:15.814691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":128641,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":185,"num_write_kb":408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":122,"num_bytes_recovered":128641,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"50'17","reported_seq":102,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978969+0000","last_change":"2023-11-25T04:25:47.044228+0000","last_active":"2023-11-25T04:25:59.978969+0000","last_peered":"2023-11-25T04:25:59.978969+0000","last_clean":"2023-11-25T04:25:59.978969+0000","last_became_active":"2023-11-25T04:25:46.998870+0000","last_became_peered":"2023-11-25T04:25:46.998870+0000","last_unstale":"2023-11-25T04:25:59.978969+0000","last_undegraded":"2023-11-25T04:25:59.978969+0000","last_fullsized":"2023-11-25T04:25:59.978969+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:08:12.601941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":4103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4194304,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":26330855,"num_objects":3447,"num_object_clones":0,"num_object_copies":6894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10423,"num_write_kb":38550,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4189,"num_bytes_recovered":12749181,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":72376320,"data_stored":52324046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10246,"ondisk_log_size":10246,"up":128,"acting":128,"num_store_stats":3},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":40885548,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":35,"num_read_kb":334,"num_write":435,"num_write_kb":40462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":32491439,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":81833984,"data_stored":81771096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5548,"internal_metadata":0},"log_size":435,"ondisk_log_size":435,"up":64,"acting":64,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738414,"num_pgs":72,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73208,"kb_used_data":46188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94298632,"statfs":{"total":96636764160,"available":96561799168,"internally_reserved":0,"allocated":47296512,"data_stored":39787071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":1,"last update":"Sat Nov 25 04:25:40 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.433,"15min":0.44500000000000001},"min":{"1min":0.28899999999999998,"5min":0.216,"15min":0.216},"max":{"1min":0.60499999999999998,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.439,"15min":0.45100000000000001},"min":{"1min":0.25600000000000001,"5min":0.184,"15min":0.184},"max":{"1min":0.63400000000000001,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.63}]}]},{"osd":1,"up_from":8,"seq":34359738414,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86880,"kb_used_data":55700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94284960,"statfs":{"total":96636764160,"available":96547799040,"internally_reserved":0,"allocated":57036800,"data_stored":50261554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":2,"last update":"Sat Nov 25 04:25:42 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.40600000000000003,"15min":0.41099999999999998},"min":{"1min":0.29099999999999998,"5min":0.182,"15min":0.182},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.40200000000000002,"15min":0.40600000000000003},"min":{"1min":0.312,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.55500000000000005}]}]},{"osd":0,"up_from":60,"seq":257698037765,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87624,"kb_used_data":51764,"kb_used_omap":6,"kb_used_meta":35833,"kb_avail":94284216,"statfs":{"total":96636764160,"available":96547037184,"internally_reserved":0,"allocated":53006336,"data_stored":46141107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7143,"internal_metadata":36693017},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32530432,"data_stored":32496940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5548,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":29380608,"data_stored":29360982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":19922944,"data_stored":19913174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":19234816,"data_stored":12749181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":26415104,"data_stored":20005586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":26726400,"data_stored":19569279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-25T04:26:05.519 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2023-11-25T04:26:05.519 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2023-11-25T04:26:05.866 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-11-25T04:26:05.878 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-25T04:26:05.878 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-25T04:26:06.180 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:26:06.182 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2023-11-25T04:26:06.191 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":179,"stamp":"2023-11-25T04:26:05.550158+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":67806771,"num_objects":3481,"num_object_clones":0,"num_object_copies":6962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3481,"num_whiteouts":0,"num_read":171,"num_read_kb":452,"num_write":11061,"num_write_kb":82472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4201,"num_bytes_recovered":45240620,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10838,"ondisk_log_size":10838,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":247712,"kb_used_data":153652,"kb_used_omap":10,"kb_used_meta":94005,"kb_avail":282867808,"statfs":{"total":289910292480,"available":289656635392,"internally_reserved":0,"allocated":157339648,"data_stored":136189732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10333,"internal_metadata":96262051},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.586462"},"pg_stats":[{"pgid":"4.3f","version":"54'166","reported_seq":234,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979173+0000","last_change":"2023-11-25T04:22:47.039847+0000","last_active":"2023-11-25T04:25:59.979173+0000","last_peered":"2023-11-25T04:25:59.979173+0000","last_clean":"2023-11-25T04:25:59.979173+0000","last_became_active":"2023-11-25T04:22:47.039490+0000","last_became_peered":"2023-11-25T04:22:47.039490+0000","last_unstale":"2023-11-25T04:25:59.979173+0000","last_undegraded":"2023-11-25T04:25:59.979173+0000","last_fullsized":"2023-11-25T04:25:59.979173+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:25:06.132517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":577866,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":174,"num_write_kb":756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3e","version":"54'212","reported_seq":280,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978984+0000","last_change":"2023-11-25T04:22:47.040836+0000","last_active":"2023-11-25T04:25:59.978984+0000","last_peered":"2023-11-25T04:25:59.978984+0000","last_clean":"2023-11-25T04:25:59.978984+0000","last_became_active":"2023-11-25T04:22:47.040511+0000","last_became_peered":"2023-11-25T04:22:47.040511+0000","last_unstale":"2023-11-25T04:25:59.978984+0000","last_undegraded":"2023-11-25T04:25:59.978984+0000","last_fullsized":"2023-11-25T04:25:59.978984+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:15:39.511043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":461579,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":236,"num_write_kb":841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3d","version":"54'167","reported_seq":233,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978931+0000","last_change":"2023-11-25T04:25:52.924678+0000","last_active":"2023-11-25T04:25:59.978931+0000","last_peered":"2023-11-25T04:25:59.978931+0000","last_clean":"2023-11-25T04:25:59.978931+0000","last_became_active":"2023-11-25T04:25:52.923459+0000","last_became_peered":"2023-11-25T04:25:52.923459+0000","last_unstale":"2023-11-25T04:25:59.978931+0000","last_undegraded":"2023-11-25T04:25:59.978931+0000","last_fullsized":"2023-11-25T04:25:59.978931+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:24:06.950438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":145844,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":175,"num_write_kb":312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":138765,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"54'124","reported_seq":191,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978386+0000","last_change":"2023-11-25T04:25:51.922843+0000","last_active":"2023-11-25T04:25:59.978386+0000","last_peered":"2023-11-25T04:25:59.978386+0000","last_clean":"2023-11-25T04:25:59.978386+0000","last_became_active":"2023-11-25T04:25:51.921633+0000","last_became_peered":"2023-11-25T04:25:51.921633+0000","last_unstale":"2023-11-25T04:25:59.978386+0000","last_undegraded":"2023-11-25T04:25:59.978386+0000","last_fullsized":"2023-11-25T04:25:59.978386+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:05:31.332951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61303,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119,"num_write_kb":161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":61303,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"54'169","reported_seq":237,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978640+0000","last_change":"2023-11-25T04:22:47.039499+0000","last_active":"2023-11-25T04:25:59.978640+0000","last_peered":"2023-11-25T04:25:59.978640+0000","last_clean":"2023-11-25T04:25:59.978640+0000","last_became_active":"2023-11-25T04:22:47.039165+0000","last_became_peered":"2023-11-25T04:22:47.039165+0000","last_unstale":"2023-11-25T04:25:59.978640+0000","last_undegraded":"2023-11-25T04:25:59.978640+0000","last_fullsized":"2023-11-25T04:25:59.978640+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:57:28.855151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":118227,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":185,"num_write_kb":351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3a","version":"54'169","reported_seq":234,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978863+0000","last_change":"2023-11-25T04:25:53.933198+0000","last_active":"2023-11-25T04:25:59.978863+0000","last_peered":"2023-11-25T04:25:59.978863+0000","last_clean":"2023-11-25T04:25:59.978863+0000","last_became_active":"2023-11-25T04:25:53.931982+0000","last_became_peered":"2023-11-25T04:25:53.931982+0000","last_unstale":"2023-11-25T04:25:59.978863+0000","last_undegraded":"2023-11-25T04:25:59.978863+0000","last_fullsized":"2023-11-25T04:25:59.978863+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:28:40.449148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100304,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":161,"num_write_kb":424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":100304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"54'168","reported_seq":237,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978380+0000","last_change":"2023-11-25T04:25:49.921033+0000","last_active":"2023-11-25T04:25:59.978380+0000","last_peered":"2023-11-25T04:25:59.978380+0000","last_clean":"2023-11-25T04:25:59.978380+0000","last_became_active":"2023-11-25T04:25:49.919354+0000","last_became_peered":"2023-11-25T04:25:49.919354+0000","last_unstale":"2023-11-25T04:25:59.978380+0000","last_undegraded":"2023-11-25T04:25:59.978380+0000","last_fullsized":"2023-11-25T04:25:59.978380+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:36:13.489397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":124558,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":167,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":111,"num_bytes_recovered":124533,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.38","version":"54'152","reported_seq":221,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978153+0000","last_change":"2023-11-25T04:25:49.920421+0000","last_active":"2023-11-25T04:25:59.978153+0000","last_peered":"2023-11-25T04:25:59.978153+0000","last_clean":"2023-11-25T04:25:59.978153+0000","last_became_active":"2023-11-25T04:25:49.919236+0000","last_became_peered":"2023-11-25T04:25:49.919236+0000","last_unstale":"2023-11-25T04:25:59.978153+0000","last_undegraded":"2023-11-25T04:25:59.978153+0000","last_fullsized":"2023-11-25T04:25:59.978153+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:32:41.956579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":120147,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":120121,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.37","version":"54'167","reported_seq":235,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978818+0000","last_change":"2023-11-25T04:22:47.038397+0000","last_active":"2023-11-25T04:25:59.978818+0000","last_peered":"2023-11-25T04:25:59.978818+0000","last_clean":"2023-11-25T04:25:59.978818+0000","last_became_active":"2023-11-25T04:22:47.037328+0000","last_became_peered":"2023-11-25T04:22:47.037328+0000","last_unstale":"2023-11-25T04:25:59.978818+0000","last_undegraded":"2023-11-25T04:25:59.978818+0000","last_fullsized":"2023-11-25T04:25:59.978818+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:56:59.689578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":174147,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":184,"num_write_kb":353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.36","version":"54'149","reported_seq":217,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979047+0000","last_change":"2023-11-25T04:22:47.040182+0000","last_active":"2023-11-25T04:25:59.979047+0000","last_peered":"2023-11-25T04:25:59.979047+0000","last_clean":"2023-11-25T04:25:59.979047+0000","last_became_active":"2023-11-25T04:22:47.039892+0000","last_became_peered":"2023-11-25T04:22:47.039892+0000","last_unstale":"2023-11-25T04:25:59.979047+0000","last_undegraded":"2023-11-25T04:25:59.979047+0000","last_fullsized":"2023-11-25T04:25:59.979047+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:38:09.176582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":233330,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"54'156","reported_seq":223,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979021+0000","last_change":"2023-11-25T04:25:51.923087+0000","last_active":"2023-11-25T04:25:59.979021+0000","last_peered":"2023-11-25T04:25:59.979021+0000","last_clean":"2023-11-25T04:25:59.979021+0000","last_became_active":"2023-11-25T04:25:51.921800+0000","last_became_peered":"2023-11-25T04:25:51.921800+0000","last_unstale":"2023-11-25T04:25:59.979021+0000","last_undegraded":"2023-11-25T04:25:59.979021+0000","last_fullsized":"2023-11-25T04:25:59.979021+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:29:47.766517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":302550,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":163,"num_write_kb":431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":300323,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.34","version":"54'136","reported_seq":400,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978706+0000","last_change":"2023-11-25T04:25:50.922843+0000","last_active":"2023-11-25T04:25:59.978706+0000","last_peered":"2023-11-25T04:25:59.978706+0000","last_clean":"2023-11-25T04:25:59.978706+0000","last_became_active":"2023-11-25T04:25:50.922309+0000","last_became_peered":"2023-11-25T04:25:50.922309+0000","last_unstale":"2023-11-25T04:25:59.978706+0000","last_undegraded":"2023-11-25T04:25:59.978706+0000","last_fullsized":"2023-11-25T04:25:59.978706+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:38:33.939562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":93053,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":131,"num_write_kb":248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":89,"num_bytes_recovered":93053,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"54'134","reported_seq":202,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978529+0000","last_change":"2023-11-25T04:22:47.040388+0000","last_active":"2023-11-25T04:25:59.978529+0000","last_peered":"2023-11-25T04:25:59.978529+0000","last_clean":"2023-11-25T04:25:59.978529+0000","last_became_active":"2023-11-25T04:22:47.040129+0000","last_became_peered":"2023-11-25T04:22:47.040129+0000","last_unstale":"2023-11-25T04:25:59.978529+0000","last_undegraded":"2023-11-25T04:25:59.978529+0000","last_fullsized":"2023-11-25T04:25:59.978529+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:31:54.857270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":149524,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":139,"num_write_kb":252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.32","version":"54'174","reported_seq":239,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978762+0000","last_change":"2023-11-25T04:25:53.934526+0000","last_active":"2023-11-25T04:25:59.978762+0000","last_peered":"2023-11-25T04:25:59.978762+0000","last_clean":"2023-11-25T04:25:59.978762+0000","last_became_active":"2023-11-25T04:25:53.933694+0000","last_became_peered":"2023-11-25T04:25:53.933694+0000","last_unstale":"2023-11-25T04:25:59.978762+0000","last_undegraded":"2023-11-25T04:25:59.978762+0000","last_fullsized":"2023-11-25T04:25:59.978762+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:01:16.858976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":95797,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":181,"num_write_kb":547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":103,"num_bytes_recovered":95797,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.31","version":"54'156","reported_seq":225,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978209+0000","last_change":"2023-11-25T04:25:49.922396+0000","last_active":"2023-11-25T04:25:59.978209+0000","last_peered":"2023-11-25T04:25:59.978209+0000","last_clean":"2023-11-25T04:25:59.978209+0000","last_became_active":"2023-11-25T04:25:49.921129+0000","last_became_peered":"2023-11-25T04:25:49.921129+0000","last_unstale":"2023-11-25T04:25:59.978209+0000","last_undegraded":"2023-11-25T04:25:59.978209+0000","last_fullsized":"2023-11-25T04:25:59.978209+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:57:02.387207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":163190,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":169,"num_write_kb":336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":154576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"54'174","reported_seq":239,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978986+0000","last_change":"2023-11-25T04:25:53.934614+0000","last_active":"2023-11-25T04:25:59.978986+0000","last_peered":"2023-11-25T04:25:59.978986+0000","last_clean":"2023-11-25T04:25:59.978986+0000","last_became_active":"2023-11-25T04:25:53.933395+0000","last_became_peered":"2023-11-25T04:25:53.933395+0000","last_unstale":"2023-11-25T04:25:59.978986+0000","last_undegraded":"2023-11-25T04:25:59.978986+0000","last_fullsized":"2023-11-25T04:25:59.978986+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:53:09.827809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1423587,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":182,"num_write_kb":1626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":1414508,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2f","version":"54'152","reported_seq":220,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979084+0000","last_change":"2023-11-25T04:22:47.039389+0000","last_active":"2023-11-25T04:25:59.979084+0000","last_peered":"2023-11-25T04:25:59.979084+0000","last_clean":"2023-11-25T04:25:59.979084+0000","last_became_active":"2023-11-25T04:22:47.039123+0000","last_became_peered":"2023-11-25T04:22:47.039123+0000","last_unstale":"2023-11-25T04:25:59.979084+0000","last_undegraded":"2023-11-25T04:25:59.979084+0000","last_fullsized":"2023-11-25T04:25:59.979084+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:09:11.570561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":122752,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":143,"num_write_kb":435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"54'148","reported_seq":216,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979169+0000","last_change":"2023-11-25T04:22:47.036946+0000","last_active":"2023-11-25T04:25:59.979169+0000","last_peered":"2023-11-25T04:25:59.979169+0000","last_clean":"2023-11-25T04:25:59.979169+0000","last_became_active":"2023-11-25T04:22:47.036411+0000","last_became_peered":"2023-11-25T04:22:47.036411+0000","last_unstale":"2023-11-25T04:25:59.979169+0000","last_undegraded":"2023-11-25T04:25:59.979169+0000","last_fullsized":"2023-11-25T04:25:59.979169+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:16:29.085050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4770899,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":140,"num_write_kb":4870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2d","version":"54'143","reported_seq":211,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978472+0000","last_change":"2023-11-25T04:22:47.038983+0000","last_active":"2023-11-25T04:25:59.978472+0000","last_peered":"2023-11-25T04:25:59.978472+0000","last_clean":"2023-11-25T04:25:59.978472+0000","last_became_active":"2023-11-25T04:22:47.038477+0000","last_became_peered":"2023-11-25T04:22:47.038477+0000","last_unstale":"2023-11-25T04:25:59.978472+0000","last_undegraded":"2023-11-25T04:25:59.978472+0000","last_fullsized":"2023-11-25T04:25:59.978472+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:30:04.568790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":115975,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"54'167","reported_seq":233,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978615+0000","last_change":"2023-11-25T04:25:52.923640+0000","last_active":"2023-11-25T04:25:59.978615+0000","last_peered":"2023-11-25T04:25:59.978615+0000","last_clean":"2023-11-25T04:25:59.978615+0000","last_became_active":"2023-11-25T04:25:52.922367+0000","last_became_peered":"2023-11-25T04:25:52.922367+0000","last_unstale":"2023-11-25T04:25:59.978615+0000","last_undegraded":"2023-11-25T04:25:59.978615+0000","last_fullsized":"2023-11-25T04:25:59.978615+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:08:24.794280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":747090,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":211,"num_write_kb":985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":732276,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"54'156","reported_seq":222,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978632+0000","last_change":"2023-11-25T04:25:52.924457+0000","last_active":"2023-11-25T04:25:59.978632+0000","last_peered":"2023-11-25T04:25:59.978632+0000","last_clean":"2023-11-25T04:25:59.978632+0000","last_became_active":"2023-11-25T04:25:52.923378+0000","last_became_peered":"2023-11-25T04:25:52.923378+0000","last_unstale":"2023-11-25T04:25:59.978632+0000","last_undegraded":"2023-11-25T04:25:59.978632+0000","last_fullsized":"2023-11-25T04:25:59.978632+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:34:30.942779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69580,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":142,"num_write_kb":315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":98,"num_bytes_recovered":69580,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2a","version":"54'171","reported_seq":467,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978347+0000","last_change":"2023-11-25T04:25:52.925532+0000","last_active":"2023-11-25T04:25:59.978347+0000","last_peered":"2023-11-25T04:25:59.978347+0000","last_clean":"2023-11-25T04:25:59.978347+0000","last_became_active":"2023-11-25T04:25:52.924844+0000","last_became_peered":"2023-11-25T04:25:52.924844+0000","last_unstale":"2023-11-25T04:25:59.978347+0000","last_undegraded":"2023-11-25T04:25:59.978347+0000","last_fullsized":"2023-11-25T04:25:59.978347+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:11:30.124326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":115997,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":162,"num_write_kb":261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":115997,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.29","version":"54'152","reported_seq":438,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.977803+0000","last_change":"2023-11-25T04:25:49.920367+0000","last_active":"2023-11-25T04:25:59.977803+0000","last_peered":"2023-11-25T04:25:59.977803+0000","last_clean":"2023-11-25T04:25:59.977803+0000","last_became_active":"2023-11-25T04:25:49.919886+0000","last_became_peered":"2023-11-25T04:25:49.919886+0000","last_unstale":"2023-11-25T04:25:59.977803+0000","last_undegraded":"2023-11-25T04:25:59.977803+0000","last_fullsized":"2023-11-25T04:25:59.977803+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:04:58.273982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":212878,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":212878,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"54'169","reported_seq":237,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978222+0000","last_change":"2023-11-25T04:22:47.036805+0000","last_active":"2023-11-25T04:25:59.978222+0000","last_peered":"2023-11-25T04:25:59.978222+0000","last_clean":"2023-11-25T04:25:59.978222+0000","last_became_active":"2023-11-25T04:22:47.036135+0000","last_became_peered":"2023-11-25T04:22:47.036135+0000","last_unstale":"2023-11-25T04:25:59.978222+0000","last_undegraded":"2023-11-25T04:25:59.978222+0000","last_fullsized":"2023-11-25T04:25:59.978222+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:25:21.730992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":102957,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":172,"num_write_kb":610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"54'167","reported_seq":465,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979220+0000","last_change":"2023-11-25T04:25:51.922925+0000","last_active":"2023-11-25T04:25:59.979220+0000","last_peered":"2023-11-25T04:25:59.979220+0000","last_clean":"2023-11-25T04:25:59.979220+0000","last_became_active":"2023-11-25T04:25:51.922364+0000","last_became_peered":"2023-11-25T04:25:51.922364+0000","last_unstale":"2023-11-25T04:25:59.979220+0000","last_undegraded":"2023-11-25T04:25:59.979220+0000","last_fullsized":"2023-11-25T04:25:59.979220+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:09:23.741871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":182683,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":171,"num_write_kb":791,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":182683,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"54'187","reported_seq":505,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978723+0000","last_change":"2023-11-25T04:25:51.922529+0000","last_active":"2023-11-25T04:25:59.978723+0000","last_peered":"2023-11-25T04:25:59.978723+0000","last_clean":"2023-11-25T04:25:59.978723+0000","last_became_active":"2023-11-25T04:25:51.922064+0000","last_became_peered":"2023-11-25T04:25:51.922064+0000","last_unstale":"2023-11-25T04:25:59.978723+0000","last_undegraded":"2023-11-25T04:25:59.978723+0000","last_fullsized":"2023-11-25T04:25:59.978723+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":187,"log_dups_size":0,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:01:39.413986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":86311,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":171,"num_write_kb":274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":116,"num_bytes_recovered":86311,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.25","version":"54'149","reported_seq":215,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978829+0000","last_change":"2023-11-25T04:25:52.925505+0000","last_active":"2023-11-25T04:25:59.978829+0000","last_peered":"2023-11-25T04:25:59.978829+0000","last_clean":"2023-11-25T04:25:59.978829+0000","last_became_active":"2023-11-25T04:25:52.924795+0000","last_became_peered":"2023-11-25T04:25:52.924795+0000","last_unstale":"2023-11-25T04:25:59.978829+0000","last_undegraded":"2023-11-25T04:25:59.978829+0000","last_fullsized":"2023-11-25T04:25:59.978829+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:30:46.532141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":128288,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":167,"num_write_kb":267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":120265,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.24","version":"54'151","reported_seq":427,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978903+0000","last_change":"2023-11-25T04:25:50.922751+0000","last_active":"2023-11-25T04:25:59.978903+0000","last_peered":"2023-11-25T04:25:59.978903+0000","last_clean":"2023-11-25T04:25:59.978903+0000","last_became_active":"2023-11-25T04:25:50.922216+0000","last_became_peered":"2023-11-25T04:25:50.922216+0000","last_unstale":"2023-11-25T04:25:59.978903+0000","last_undegraded":"2023-11-25T04:25:59.978903+0000","last_fullsized":"2023-11-25T04:25:59.978903+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:40:16.255665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":107507,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":148,"num_write_kb":288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":107494,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.23","version":"54'155","reported_seq":223,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978207+0000","last_change":"2023-11-25T04:22:47.037668+0000","last_active":"2023-11-25T04:25:59.978207+0000","last_peered":"2023-11-25T04:25:59.978207+0000","last_clean":"2023-11-25T04:25:59.978207+0000","last_became_active":"2023-11-25T04:22:47.036823+0000","last_became_peered":"2023-11-25T04:22:47.036823+0000","last_unstale":"2023-11-25T04:25:59.978207+0000","last_undegraded":"2023-11-25T04:25:59.978207+0000","last_fullsized":"2023-11-25T04:25:59.978207+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:02:07.088241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4304582,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":152,"num_write_kb":4331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.22","version":"54'189","reported_seq":517,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978625+0000","last_change":"2023-11-25T04:25:54.940559+0000","last_active":"2023-11-25T04:25:59.978625+0000","last_peered":"2023-11-25T04:25:59.978625+0000","last_clean":"2023-11-25T04:25:59.978625+0000","last_became_active":"2023-11-25T04:25:54.939584+0000","last_became_peered":"2023-11-25T04:25:54.939584+0000","last_unstale":"2023-11-25T04:25:59.978625+0000","last_undegraded":"2023-11-25T04:25:59.978625+0000","last_fullsized":"2023-11-25T04:25:59.978625+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:01:39.413986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":129145,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":185,"num_write_kb":432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":121,"num_bytes_recovered":129107,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"54'201","reported_seq":269,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.977881+0000","last_change":"2023-11-25T04:22:47.037786+0000","last_active":"2023-11-25T04:25:59.977881+0000","last_peered":"2023-11-25T04:25:59.977881+0000","last_clean":"2023-11-25T04:25:59.977881+0000","last_became_active":"2023-11-25T04:22:47.037206+0000","last_became_peered":"2023-11-25T04:22:47.037206+0000","last_unstale":"2023-11-25T04:25:59.977881+0000","last_undegraded":"2023-11-25T04:25:59.977881+0000","last_fullsized":"2023-11-25T04:25:59.977881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:11:40.532140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":215311,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":194,"num_write_kb":381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"54'180","reported_seq":245,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.977884+0000","last_change":"2023-11-25T04:25:53.933176+0000","last_active":"2023-11-25T04:25:59.977884+0000","last_peered":"2023-11-25T04:25:59.977884+0000","last_clean":"2023-11-25T04:25:59.977884+0000","last_became_active":"2023-11-25T04:25:53.931937+0000","last_became_peered":"2023-11-25T04:25:53.931937+0000","last_unstale":"2023-11-25T04:25:59.977884+0000","last_undegraded":"2023-11-25T04:25:59.977884+0000","last_fullsized":"2023-11-25T04:25:59.977884+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:41:10.697064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104603,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":172,"num_write_kb":267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":104603,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"54'137","reported_seq":205,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978811+0000","last_change":"2023-11-25T04:22:47.036689+0000","last_active":"2023-11-25T04:25:59.978811+0000","last_peered":"2023-11-25T04:25:59.978811+0000","last_clean":"2023-11-25T04:25:59.978811+0000","last_became_active":"2023-11-25T04:22:47.036107+0000","last_became_peered":"2023-11-25T04:22:47.036107+0000","last_unstale":"2023-11-25T04:25:59.978811+0000","last_undegraded":"2023-11-25T04:25:59.978811+0000","last_fullsized":"2023-11-25T04:25:59.978811+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:02:07.088241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":183586,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":127,"num_write_kb":284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"58'114","reported_seq":191,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978227+0000","last_change":"2023-11-25T04:25:49.920320+0000","last_active":"2023-11-25T04:25:59.978227+0000","last_peered":"2023-11-25T04:25:59.978227+0000","last_clean":"2023-11-25T04:25:59.978227+0000","last_became_active":"2023-11-25T04:25:49.920010+0000","last_became_peered":"2023-11-25T04:25:49.920010+0000","last_unstale":"2023-11-25T04:25:59.978227+0000","last_undegraded":"2023-11-25T04:25:59.978227+0000","last_fullsized":"2023-11-25T04:25:59.978227+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:48:58.746789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2816216,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":113,"num_write_kb":2816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":2816182,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"54'145","reported_seq":213,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978995+0000","last_change":"2023-11-25T04:22:47.037888+0000","last_active":"2023-11-25T04:25:59.978995+0000","last_peered":"2023-11-25T04:25:59.978995+0000","last_clean":"2023-11-25T04:25:59.978995+0000","last_became_active":"2023-11-25T04:22:47.037158+0000","last_became_peered":"2023-11-25T04:22:47.037158+0000","last_unstale":"2023-11-25T04:25:59.978995+0000","last_undegraded":"2023-11-25T04:25:59.978995+0000","last_fullsized":"2023-11-25T04:25:59.978995+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:56:59.689578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":205976,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.12","version":"49'2","reported_seq":82,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978301+0000","last_change":"2023-11-25T04:25:47.000916+0000","last_active":"2023-11-25T04:25:59.978301+0000","last_peered":"2023-11-25T04:25:59.978301+0000","last_clean":"2023-11-25T04:25:59.978301+0000","last_became_active":"2023-11-25T04:25:47.000516+0000","last_became_peered":"2023-11-25T04:25:47.000516+0000","last_unstale":"2023-11-25T04:25:59.978301+0000","last_undegraded":"2023-11-25T04:25:59.978301+0000","last_fullsized":"2023-11-25T04:25:59.978301+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:53:58.873447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"54'175","reported_seq":489,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978855+0000","last_change":"2023-11-25T04:25:52.923694+0000","last_active":"2023-11-25T04:25:59.978855+0000","last_peered":"2023-11-25T04:25:59.978855+0000","last_clean":"2023-11-25T04:25:59.978855+0000","last_became_active":"2023-11-25T04:25:52.923045+0000","last_became_peered":"2023-11-25T04:25:52.923045+0000","last_unstale":"2023-11-25T04:25:59.978855+0000","last_undegraded":"2023-11-25T04:25:59.978855+0000","last_fullsized":"2023-11-25T04:25:59.978855+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:46:12.914433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":171823,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":185,"num_write_kb":340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":166387,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":80,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978355+0000","last_change":"2023-11-25T04:25:46.994324+0000","last_active":"2023-11-25T04:25:59.978355+0000","last_peered":"2023-11-25T04:25:59.978355+0000","last_clean":"2023-11-25T04:25:59.978355+0000","last_became_active":"2023-11-25T04:25:46.993870+0000","last_became_peered":"2023-11-25T04:25:46.993870+0000","last_unstale":"2023-11-25T04:25:59.978355+0000","last_undegraded":"2023-11-25T04:25:59.978355+0000","last_fullsized":"2023-11-25T04:25:59.978355+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:51:09.827080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"54'152","reported_seq":435,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978827+0000","last_change":"2023-11-25T04:25:50.922922+0000","last_active":"2023-11-25T04:25:59.978827+0000","last_peered":"2023-11-25T04:25:59.978827+0000","last_clean":"2023-11-25T04:25:59.978827+0000","last_became_active":"2023-11-25T04:25:50.922511+0000","last_became_peered":"2023-11-25T04:25:50.922511+0000","last_unstale":"2023-11-25T04:25:59.978827+0000","last_undegraded":"2023-11-25T04:25:59.978827+0000","last_fullsized":"2023-11-25T04:25:59.978827+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:37:14.620534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":579704,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":97,"num_bytes_recovered":570044,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.10","version":"18'8","reported_seq":39,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.977878+0000","last_change":"2023-11-25T04:25:47.000499+0000","last_active":"2023-11-25T04:25:59.977878+0000","last_peered":"2023-11-25T04:25:59.977878+0000","last_clean":"2023-11-25T04:25:59.977878+0000","last_became_active":"2023-11-25T04:25:46.998778+0000","last_became_peered":"2023-11-25T04:25:46.998778+0000","last_unstale":"2023-11-25T04:25:59.977878+0000","last_undegraded":"2023-11-25T04:25:59.977878+0000","last_fullsized":"2023-11-25T04:25:59.977878+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:33:09.169689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"49'2","reported_seq":37,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979103+0000","last_change":"2023-11-25T04:25:46.998392+0000","last_active":"2023-11-25T04:25:59.979103+0000","last_peered":"2023-11-25T04:25:59.979103+0000","last_clean":"2023-11-25T04:25:59.979103+0000","last_became_active":"2023-11-25T04:25:46.997108+0000","last_became_peered":"2023-11-25T04:25:46.997108+0000","last_unstale":"2023-11-25T04:25:59.979103+0000","last_undegraded":"2023-11-25T04:25:59.979103+0000","last_fullsized":"2023-11-25T04:25:59.979103+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:02:00.073630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"54'159","reported_seq":443,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978233+0000","last_change":"2023-11-25T04:25:50.922282+0000","last_active":"2023-11-25T04:25:59.978233+0000","last_peered":"2023-11-25T04:25:59.978233+0000","last_clean":"2023-11-25T04:25:59.978233+0000","last_became_active":"2023-11-25T04:25:50.921772+0000","last_became_peered":"2023-11-25T04:25:50.921772+0000","last_unstale":"2023-11-25T04:25:59.978233+0000","last_undegraded":"2023-11-25T04:25:59.978233+0000","last_fullsized":"2023-11-25T04:25:59.978233+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:36:33.451689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112103,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":164,"num_write_kb":240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":99,"num_bytes_recovered":103472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"18'2","reported_seq":82,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979281+0000","last_change":"2023-11-25T04:25:46.994943+0000","last_active":"2023-11-25T04:25:59.979281+0000","last_peered":"2023-11-25T04:25:59.979281+0000","last_clean":"2023-11-25T04:25:59.979281+0000","last_became_active":"2023-11-25T04:25:46.994640+0000","last_became_peered":"2023-11-25T04:25:46.994640+0000","last_unstale":"2023-11-25T04:25:59.979281+0000","last_undegraded":"2023-11-25T04:25:59.979281+0000","last_fullsized":"2023-11-25T04:25:59.979281+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:08:12.601941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"54'173","reported_seq":241,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978244+0000","last_change":"2023-11-25T04:22:47.039685+0000","last_active":"2023-11-25T04:25:59.978244+0000","last_peered":"2023-11-25T04:25:59.978244+0000","last_clean":"2023-11-25T04:25:59.978244+0000","last_became_active":"2023-11-25T04:22:47.039296+0000","last_became_peered":"2023-11-25T04:22:47.039296+0000","last_unstale":"2023-11-25T04:25:59.978244+0000","last_undegraded":"2023-11-25T04:25:59.978244+0000","last_fullsized":"2023-11-25T04:25:59.978244+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:11:40.532140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154533,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":182,"num_write_kb":299,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"18'1","reported_seq":37,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978725+0000","last_change":"2023-11-25T04:25:47.000059+0000","last_active":"2023-11-25T04:25:59.978725+0000","last_peered":"2023-11-25T04:25:59.978725+0000","last_clean":"2023-11-25T04:25:59.978725+0000","last_became_active":"2023-11-25T04:25:46.999604+0000","last_became_peered":"2023-11-25T04:25:46.999604+0000","last_unstale":"2023-11-25T04:25:59.978725+0000","last_undegraded":"2023-11-25T04:25:59.978725+0000","last_fullsized":"2023-11-25T04:25:59.978725+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:14:43.431272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"54'147","reported_seq":211,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978982+0000","last_change":"2023-11-25T04:25:54.939274+0000","last_active":"2023-11-25T04:25:59.978982+0000","last_peered":"2023-11-25T04:25:59.978982+0000","last_clean":"2023-11-25T04:25:59.978982+0000","last_became_active":"2023-11-25T04:25:54.938236+0000","last_became_peered":"2023-11-25T04:25:54.938236+0000","last_unstale":"2023-11-25T04:25:59.978982+0000","last_undegraded":"2023-11-25T04:25:59.978982+0000","last_fullsized":"2023-11-25T04:25:59.978982+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":147,"log_dups_size":0,"ondisk_log_size":147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:11:43.785294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":87524,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":155,"num_write_kb":235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":87524,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978546+0000","last_change":"2023-11-25T04:25:46.997080+0000","last_active":"2023-11-25T04:25:59.978546+0000","last_peered":"2023-11-25T04:25:59.978546+0000","last_clean":"2023-11-25T04:25:59.978546+0000","last_became_active":"2023-11-25T04:25:46.995712+0000","last_became_peered":"2023-11-25T04:25:46.995712+0000","last_unstale":"2023-11-25T04:25:59.978546+0000","last_undegraded":"2023-11-25T04:25:59.978546+0000","last_fullsized":"2023-11-25T04:25:59.978546+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T16:01:00.922843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"54'155","reported_seq":441,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978274+0000","last_change":"2023-11-25T04:25:52.923096+0000","last_active":"2023-11-25T04:25:59.978274+0000","last_peered":"2023-11-25T04:25:59.978274+0000","last_clean":"2023-11-25T04:25:59.978274+0000","last_became_active":"2023-11-25T04:25:52.922549+0000","last_became_peered":"2023-11-25T04:25:52.922549+0000","last_unstale":"2023-11-25T04:25:59.978274+0000","last_undegraded":"2023-11-25T04:25:59.978274+0000","last_fullsized":"2023-11-25T04:25:59.978274+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:52:38.053808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":266919,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":264258,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"45'11","reported_seq":96,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978532+0000","last_change":"2023-11-25T04:25:47.092497+0000","last_active":"2023-11-25T04:25:59.978532+0000","last_peered":"2023-11-25T04:25:59.978532+0000","last_clean":"2023-11-25T04:25:59.978532+0000","last_became_active":"2023-11-25T04:25:47.000687+0000","last_became_peered":"2023-11-25T04:25:47.000687+0000","last_unstale":"2023-11-25T04:25:59.978532+0000","last_undegraded":"2023-11-25T04:25:59.978532+0000","last_fullsized":"2023-11-25T04:25:59.978532+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:47:09.798238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":4101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4194304,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"54'173","reported_seq":241,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978732+0000","last_change":"2023-11-25T04:25:50.923240+0000","last_active":"2023-11-25T04:25:59.978732+0000","last_peered":"2023-11-25T04:25:59.978732+0000","last_clean":"2023-11-25T04:25:59.978732+0000","last_became_active":"2023-11-25T04:25:50.922409+0000","last_became_peered":"2023-11-25T04:25:50.922409+0000","last_unstale":"2023-11-25T04:25:59.978732+0000","last_undegraded":"2023-11-25T04:25:59.978732+0000","last_fullsized":"2023-11-25T04:25:59.978732+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:53:30.920627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":553893,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":169,"num_write_kb":770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":553893,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978079+0000","last_change":"2023-11-25T04:25:46.998818+0000","last_active":"2023-11-25T04:25:59.978079+0000","last_peered":"2023-11-25T04:25:59.978079+0000","last_clean":"2023-11-25T04:25:59.978079+0000","last_became_active":"2023-11-25T04:25:46.997019+0000","last_became_peered":"2023-11-25T04:25:46.997019+0000","last_unstale":"2023-11-25T04:25:59.978079+0000","last_undegraded":"2023-11-25T04:25:59.978079+0000","last_fullsized":"2023-11-25T04:25:59.978079+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:51:47.882117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"54'163","reported_seq":228,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978685+0000","last_change":"2023-11-25T04:25:53.934183+0000","last_active":"2023-11-25T04:25:59.978685+0000","last_peered":"2023-11-25T04:25:59.978685+0000","last_clean":"2023-11-25T04:25:59.978685+0000","last_became_active":"2023-11-25T04:25:53.933471+0000","last_became_peered":"2023-11-25T04:25:53.933471+0000","last_unstale":"2023-11-25T04:25:59.978685+0000","last_undegraded":"2023-11-25T04:25:59.978685+0000","last_fullsized":"2023-11-25T04:25:59.978685+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:57:07.515414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":320867,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":190,"num_write_kb":577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":108,"num_bytes_recovered":311640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":80,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978412+0000","last_change":"2023-11-25T04:25:46.994480+0000","last_active":"2023-11-25T04:25:59.978412+0000","last_peered":"2023-11-25T04:25:59.978412+0000","last_clean":"2023-11-25T04:25:59.978412+0000","last_became_active":"2023-11-25T04:25:46.994101+0000","last_became_peered":"2023-11-25T04:25:46.994101+0000","last_unstale":"2023-11-25T04:25:59.978412+0000","last_undegraded":"2023-11-25T04:25:59.978412+0000","last_fullsized":"2023-11-25T04:25:59.978412+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:53:58.873447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"54'141","reported_seq":210,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979040+0000","last_change":"2023-11-25T04:25:49.921950+0000","last_active":"2023-11-25T04:25:59.979040+0000","last_peered":"2023-11-25T04:25:59.979040+0000","last_clean":"2023-11-25T04:25:59.979040+0000","last_became_active":"2023-11-25T04:25:49.920971+0000","last_became_peered":"2023-11-25T04:25:49.920971+0000","last_unstale":"2023-11-25T04:25:59.979040+0000","last_undegraded":"2023-11-25T04:25:59.979040+0000","last_fullsized":"2023-11-25T04:25:59.979040+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:22:38.004934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3502371,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":3718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":89,"num_bytes_recovered":3497669,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"49'1","reported_seq":70,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978352+0000","last_change":"2023-11-25T04:22:46.034509+0000","last_active":"2023-11-25T04:25:59.978352+0000","last_peered":"2023-11-25T04:25:59.978352+0000","last_clean":"2023-11-25T04:25:59.978352+0000","last_became_active":"2023-11-25T04:22:46.034158+0000","last_became_peered":"2023-11-25T04:22:46.034158+0000","last_unstale":"2023-11-25T04:25:59.978352+0000","last_undegraded":"2023-11-25T04:25:59.978352+0000","last_fullsized":"2023-11-25T04:25:59.978352+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:26:25.338117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"54'141","reported_seq":209,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978912+0000","last_change":"2023-11-25T04:22:47.038800+0000","last_active":"2023-11-25T04:25:59.978912+0000","last_peered":"2023-11-25T04:25:59.978912+0000","last_clean":"2023-11-25T04:25:59.978912+0000","last_became_active":"2023-11-25T04:22:47.038081+0000","last_became_peered":"2023-11-25T04:22:47.038081+0000","last_unstale":"2023-11-25T04:25:59.978912+0000","last_undegraded":"2023-11-25T04:25:59.978912+0000","last_fullsized":"2023-11-25T04:25:59.978912+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:45:40.065146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":174088,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":146,"num_write_kb":385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"12'157","reported_seq":232,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.977771+0000","last_change":"2023-11-25T04:25:47.000504+0000","last_active":"2023-11-25T04:25:59.977771+0000","last_peered":"2023-11-25T04:25:59.977771+0000","last_clean":"2023-11-25T04:25:59.977771+0000","last_became_active":"2023-11-25T04:25:46.999721+0000","last_became_peered":"2023-11-25T04:25:46.999721+0000","last_unstale":"2023-11-25T04:25:59.977771+0000","last_undegraded":"2023-11-25T04:25:59.977771+0000","last_fullsized":"2023-11-25T04:25:59.977771+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:35.919850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:35.919850+0000","last_clean_scrub_stamp":"2023-11-25T04:22:35.919850+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:26:33.607474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978082+0000","last_change":"2023-11-25T04:25:46.998960+0000","last_active":"2023-11-25T04:25:59.978082+0000","last_peered":"2023-11-25T04:25:59.978082+0000","last_clean":"2023-11-25T04:25:59.978082+0000","last_became_active":"2023-11-25T04:25:46.997693+0000","last_became_peered":"2023-11-25T04:25:46.997693+0000","last_unstale":"2023-11-25T04:25:59.978082+0000","last_undegraded":"2023-11-25T04:25:59.978082+0000","last_fullsized":"2023-11-25T04:25:59.978082+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:51:45.180957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"46'1","reported_seq":70,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978331+0000","last_change":"2023-11-25T04:22:46.037044+0000","last_active":"2023-11-25T04:25:59.978331+0000","last_peered":"2023-11-25T04:25:59.978331+0000","last_clean":"2023-11-25T04:25:59.978331+0000","last_became_active":"2023-11-25T04:22:46.035916+0000","last_became_peered":"2023-11-25T04:22:46.035916+0000","last_unstale":"2023-11-25T04:25:59.978331+0000","last_undegraded":"2023-11-25T04:25:59.978331+0000","last_fullsized":"2023-11-25T04:25:59.978331+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:38:20.501102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"54'146","reported_seq":213,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978804+0000","last_change":"2023-11-25T04:25:51.923452+0000","last_active":"2023-11-25T04:25:59.978804+0000","last_peered":"2023-11-25T04:25:59.978804+0000","last_clean":"2023-11-25T04:25:59.978804+0000","last_became_active":"2023-11-25T04:25:51.922490+0000","last_became_peered":"2023-11-25T04:25:51.922490+0000","last_unstale":"2023-11-25T04:25:59.978804+0000","last_undegraded":"2023-11-25T04:25:59.978804+0000","last_fullsized":"2023-11-25T04:25:59.978804+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:40:11.001062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":502154,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":170,"num_write_kb":687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":99,"num_bytes_recovered":502101,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978713+0000","last_change":"2023-11-25T04:25:47.000763+0000","last_active":"2023-11-25T04:25:59.978713+0000","last_peered":"2023-11-25T04:25:59.978713+0000","last_clean":"2023-11-25T04:25:59.978713+0000","last_became_active":"2023-11-25T04:25:47.000033+0000","last_became_peered":"2023-11-25T04:25:47.000033+0000","last_unstale":"2023-11-25T04:25:59.978713+0000","last_undegraded":"2023-11-25T04:25:59.978713+0000","last_fullsized":"2023-11-25T04:25:59.978713+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:53:59.131478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"50'37","reported_seq":156,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978116+0000","last_change":"2023-11-25T04:25:47.002009+0000","last_active":"2023-11-25T04:25:59.978116+0000","last_peered":"2023-11-25T04:25:59.978116+0000","last_clean":"2023-11-25T04:25:59.978116+0000","last_became_active":"2023-11-25T04:25:46.995637+0000","last_became_peered":"2023-11-25T04:25:46.995637+0000","last_unstale":"2023-11-25T04:25:59.978116+0000","last_undegraded":"2023-11-25T04:25:59.978116+0000","last_fullsized":"2023-11-25T04:25:59.978116+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:55:01.351319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":319160,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":35,"num_read_kb":334,"num_write":35,"num_write_kb":334,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":319160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"54'158","reported_seq":226,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979092+0000","last_change":"2023-11-25T04:22:47.039583+0000","last_active":"2023-11-25T04:25:59.979092+0000","last_peered":"2023-11-25T04:25:59.979092+0000","last_clean":"2023-11-25T04:25:59.979092+0000","last_became_active":"2023-11-25T04:22:47.039197+0000","last_became_peered":"2023-11-25T04:22:47.039197+0000","last_unstale":"2023-11-25T04:25:59.979092+0000","last_undegraded":"2023-11-25T04:25:59.979092+0000","last_fullsized":"2023-11-25T04:25:59.979092+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:37:30.959147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":169303,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":189,"num_write_kb":333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":72,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978222+0000","last_change":"2023-11-25T04:22:42.000382+0000","last_active":"2023-11-25T04:25:59.978222+0000","last_peered":"2023-11-25T04:25:59.978222+0000","last_clean":"2023-11-25T04:25:59.978222+0000","last_became_active":"2023-11-25T04:22:42.000041+0000","last_became_peered":"2023-11-25T04:22:42.000041+0000","last_unstale":"2023-11-25T04:25:59.978222+0000","last_undegraded":"2023-11-25T04:25:59.978222+0000","last_fullsized":"2023-11-25T04:25:59.978222+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:08:19.655864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"49'71","reported_seq":155,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978346+0000","last_change":"2023-11-25T04:25:47.109628+0000","last_active":"2023-11-25T04:25:59.978346+0000","last_peered":"2023-11-25T04:25:59.978346+0000","last_clean":"2023-11-25T04:25:59.978346+0000","last_became_active":"2023-11-25T04:25:47.001162+0000","last_became_peered":"2023-11-25T04:25:47.001162+0000","last_unstale":"2023-11-25T04:25:59.978346+0000","last_undegraded":"2023-11-25T04:25:59.978346+0000","last_fullsized":"2023-11-25T04:25:59.978346+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:03:33.658392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":69,"num_write_kb":4130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4194304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"54'158","reported_seq":226,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978945+0000","last_change":"2023-11-25T04:22:47.038454+0000","last_active":"2023-11-25T04:25:59.978945+0000","last_peered":"2023-11-25T04:25:59.978945+0000","last_clean":"2023-11-25T04:25:59.978945+0000","last_became_active":"2023-11-25T04:22:47.038046+0000","last_became_peered":"2023-11-25T04:22:47.038046+0000","last_unstale":"2023-11-25T04:25:59.978945+0000","last_undegraded":"2023-11-25T04:25:59.978945+0000","last_fullsized":"2023-11-25T04:25:59.978945+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:38:38.839941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":115501,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":163,"num_write_kb":243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.977941+0000","last_change":"2023-11-25T04:25:46.998739+0000","last_active":"2023-11-25T04:25:59.977941+0000","last_peered":"2023-11-25T04:25:59.977941+0000","last_clean":"2023-11-25T04:25:59.977941+0000","last_became_active":"2023-11-25T04:25:46.997391+0000","last_became_peered":"2023-11-25T04:25:46.997391+0000","last_unstale":"2023-11-25T04:25:59.977941+0000","last_undegraded":"2023-11-25T04:25:59.977941+0000","last_fullsized":"2023-11-25T04:25:59.977941+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:47:46.300596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"50'1","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978166+0000","last_change":"2023-11-25T04:25:46.999446+0000","last_active":"2023-11-25T04:25:59.978166+0000","last_peered":"2023-11-25T04:25:59.978166+0000","last_clean":"2023-11-25T04:25:59.978166+0000","last_became_active":"2023-11-25T04:25:46.998132+0000","last_became_peered":"2023-11-25T04:25:46.998132+0000","last_unstale":"2023-11-25T04:25:59.978166+0000","last_undegraded":"2023-11-25T04:25:59.978166+0000","last_fullsized":"2023-11-25T04:25:59.978166+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:42:51.166678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"54'157","reported_seq":225,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978638+0000","last_change":"2023-11-25T04:22:47.037355+0000","last_active":"2023-11-25T04:25:59.978638+0000","last_peered":"2023-11-25T04:25:59.978638+0000","last_clean":"2023-11-25T04:25:59.978638+0000","last_became_active":"2023-11-25T04:22:47.036924+0000","last_became_peered":"2023-11-25T04:22:47.036924+0000","last_unstale":"2023-11-25T04:25:59.978638+0000","last_undegraded":"2023-11-25T04:25:59.978638+0000","last_fullsized":"2023-11-25T04:25:59.978638+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:15:39.511043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":93321,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":148,"num_write_kb":360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978520+0000","last_change":"2023-11-25T04:25:46.999738+0000","last_active":"2023-11-25T04:25:59.978520+0000","last_peered":"2023-11-25T04:25:59.978520+0000","last_clean":"2023-11-25T04:25:59.978520+0000","last_became_active":"2023-11-25T04:25:46.999403+0000","last_became_peered":"2023-11-25T04:25:46.999403+0000","last_unstale":"2023-11-25T04:25:59.978520+0000","last_undegraded":"2023-11-25T04:25:59.978520+0000","last_fullsized":"2023-11-25T04:25:59.978520+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:45:53.304063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":72,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978705+0000","last_change":"2023-11-25T04:22:42.000204+0000","last_active":"2023-11-25T04:25:59.978705+0000","last_peered":"2023-11-25T04:25:59.978705+0000","last_clean":"2023-11-25T04:25:59.978705+0000","last_became_active":"2023-11-25T04:22:41.999915+0000","last_became_peered":"2023-11-25T04:22:41.999915+0000","last_unstale":"2023-11-25T04:25:59.978705+0000","last_undegraded":"2023-11-25T04:25:59.978705+0000","last_fullsized":"2023-11-25T04:25:59.978705+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:07:32.457905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"54'152","reported_seq":432,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978379+0000","last_change":"2023-11-25T04:25:53.933239+0000","last_active":"2023-11-25T04:25:59.978379+0000","last_peered":"2023-11-25T04:25:59.978379+0000","last_clean":"2023-11-25T04:25:59.978379+0000","last_became_active":"2023-11-25T04:25:53.932586+0000","last_became_peered":"2023-11-25T04:25:53.932586+0000","last_unstale":"2023-11-25T04:25:59.978379+0000","last_undegraded":"2023-11-25T04:25:59.978379+0000","last_fullsized":"2023-11-25T04:25:59.978379+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:54:51.339025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":329327,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":151,"num_write_kb":471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":97,"num_bytes_recovered":329278,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":69,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978632+0000","last_change":"2023-11-25T04:22:46.035885+0000","last_active":"2023-11-25T04:25:59.978632+0000","last_peered":"2023-11-25T04:25:59.978632+0000","last_clean":"2023-11-25T04:25:59.978632+0000","last_became_active":"2023-11-25T04:22:46.035405+0000","last_became_peered":"2023-11-25T04:22:46.035405+0000","last_unstale":"2023-11-25T04:25:59.978632+0000","last_undegraded":"2023-11-25T04:25:59.978632+0000","last_fullsized":"2023-11-25T04:25:59.978632+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:53:47.398746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":72,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979221+0000","last_change":"2023-11-25T04:22:41.999067+0000","last_active":"2023-11-25T04:25:59.979221+0000","last_peered":"2023-11-25T04:25:59.979221+0000","last_clean":"2023-11-25T04:25:59.979221+0000","last_became_active":"2023-11-25T04:22:41.998798+0000","last_became_peered":"2023-11-25T04:22:41.998798+0000","last_unstale":"2023-11-25T04:25:59.979221+0000","last_undegraded":"2023-11-25T04:25:59.979221+0000","last_fullsized":"2023-11-25T04:25:59.979221+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:00:04.266051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"54'172","reported_seq":470,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978595+0000","last_change":"2023-11-25T04:25:51.922403+0000","last_active":"2023-11-25T04:25:59.978595+0000","last_peered":"2023-11-25T04:25:59.978595+0000","last_clean":"2023-11-25T04:25:59.978595+0000","last_became_active":"2023-11-25T04:25:51.921858+0000","last_became_peered":"2023-11-25T04:25:51.921858+0000","last_unstale":"2023-11-25T04:25:59.978595+0000","last_undegraded":"2023-11-25T04:25:59.978595+0000","last_fullsized":"2023-11-25T04:25:59.978595+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:04:58.273982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":129965,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":166,"num_write_kb":290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":129965,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"18'2","reported_seq":71,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979088+0000","last_change":"2023-11-25T04:22:46.034606+0000","last_active":"2023-11-25T04:25:59.979088+0000","last_peered":"2023-11-25T04:25:59.979088+0000","last_clean":"2023-11-25T04:25:59.979088+0000","last_became_active":"2023-11-25T04:22:46.034229+0000","last_became_peered":"2023-11-25T04:22:46.034229+0000","last_unstale":"2023-11-25T04:25:59.979088+0000","last_undegraded":"2023-11-25T04:25:59.979088+0000","last_fullsized":"2023-11-25T04:25:59.979088+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:21:42.146264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":72,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978952+0000","last_change":"2023-11-25T04:22:42.000258+0000","last_active":"2023-11-25T04:25:59.978952+0000","last_peered":"2023-11-25T04:25:59.978952+0000","last_clean":"2023-11-25T04:25:59.978952+0000","last_became_active":"2023-11-25T04:22:41.999992+0000","last_became_peered":"2023-11-25T04:22:41.999992+0000","last_unstale":"2023-11-25T04:25:59.978952+0000","last_undegraded":"2023-11-25T04:25:59.978952+0000","last_fullsized":"2023-11-25T04:25:59.978952+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:31:13.715992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"54'150","reported_seq":436,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.977926+0000","last_change":"2023-11-25T04:25:54.938479+0000","last_active":"2023-11-25T04:25:59.977926+0000","last_peered":"2023-11-25T04:25:59.977926+0000","last_clean":"2023-11-25T04:25:59.977926+0000","last_became_active":"2023-11-25T04:25:54.937838+0000","last_became_peered":"2023-11-25T04:25:54.937838+0000","last_unstale":"2023-11-25T04:25:59.977926+0000","last_undegraded":"2023-11-25T04:25:59.977926+0000","last_fullsized":"2023-11-25T04:25:59.977926+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:51:14.394190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":159585,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":154224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"54'14","reported_seq":99,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978916+0000","last_change":"2023-11-25T04:25:47.026749+0000","last_active":"2023-11-25T04:25:59.978916+0000","last_peered":"2023-11-25T04:25:59.978916+0000","last_clean":"2023-11-25T04:25:59.978916+0000","last_became_active":"2023-11-25T04:25:46.997404+0000","last_became_peered":"2023-11-25T04:25:46.997404+0000","last_unstale":"2023-11-25T04:25:59.978916+0000","last_undegraded":"2023-11-25T04:25:59.978916+0000","last_fullsized":"2023-11-25T04:25:59.978916+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:20:31.719529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978930+0000","last_change":"2023-11-25T04:25:46.995200+0000","last_active":"2023-11-25T04:25:59.978930+0000","last_peered":"2023-11-25T04:25:59.978930+0000","last_clean":"2023-11-25T04:25:59.978930+0000","last_became_active":"2023-11-25T04:25:46.994834+0000","last_became_peered":"2023-11-25T04:25:46.994834+0000","last_unstale":"2023-11-25T04:25:59.978930+0000","last_undegraded":"2023-11-25T04:25:59.978930+0000","last_fullsized":"2023-11-25T04:25:59.978930+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:33:07.913911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"54'135","reported_seq":407,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978276+0000","last_change":"2023-11-25T04:25:49.922046+0000","last_active":"2023-11-25T04:25:59.978276+0000","last_peered":"2023-11-25T04:25:59.978276+0000","last_clean":"2023-11-25T04:25:59.978276+0000","last_became_active":"2023-11-25T04:25:49.921465+0000","last_became_peered":"2023-11-25T04:25:49.921465+0000","last_unstale":"2023-11-25T04:25:59.978276+0000","last_undegraded":"2023-11-25T04:25:59.978276+0000","last_fullsized":"2023-11-25T04:25:59.978276+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:12:54.077901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":99503,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":148,"num_write_kb":250,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":93,"num_bytes_recovered":99503,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"54'166","reported_seq":233,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978407+0000","last_change":"2023-11-25T04:25:51.923141+0000","last_active":"2023-11-25T04:25:59.978407+0000","last_peered":"2023-11-25T04:25:59.978407+0000","last_clean":"2023-11-25T04:25:59.978407+0000","last_became_active":"2023-11-25T04:25:51.922070+0000","last_became_peered":"2023-11-25T04:25:51.922070+0000","last_unstale":"2023-11-25T04:25:59.978407+0000","last_undegraded":"2023-11-25T04:25:59.978407+0000","last_fullsized":"2023-11-25T04:25:59.978407+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:34:53.021081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":185321,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":172,"num_write_kb":312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":164176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"58'50","reported_seq":53,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978227+0000","last_change":"2023-11-25T04:25:47.021705+0000","last_active":"2023-11-25T04:25:59.978227+0000","last_peered":"2023-11-25T04:25:59.978227+0000","last_clean":"2023-11-25T04:25:59.978227+0000","last_became_active":"2023-11-25T04:25:46.994997+0000","last_became_peered":"2023-11-25T04:25:46.994997+0000","last_unstale":"2023-11-25T04:25:59.978227+0000","last_undegraded":"2023-11-25T04:25:59.978227+0000","last_fullsized":"2023-11-25T04:25:59.978227+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:41:04.147372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":4119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4190183,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"54'153","reported_seq":221,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978107+0000","last_change":"2023-11-25T04:22:47.037181+0000","last_active":"2023-11-25T04:25:59.978107+0000","last_peered":"2023-11-25T04:25:59.978107+0000","last_clean":"2023-11-25T04:25:59.978107+0000","last_became_active":"2023-11-25T04:22:47.036821+0000","last_became_peered":"2023-11-25T04:22:47.036821+0000","last_unstale":"2023-11-25T04:25:59.978107+0000","last_undegraded":"2023-11-25T04:25:59.978107+0000","last_fullsized":"2023-11-25T04:25:59.978107+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:12:51.990448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":101117,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":161,"num_write_kb":242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"50'22","reported_seq":107,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979124+0000","last_change":"2023-11-25T04:25:47.028636+0000","last_active":"2023-11-25T04:25:59.979124+0000","last_peered":"2023-11-25T04:25:59.979124+0000","last_clean":"2023-11-25T04:25:59.979124+0000","last_became_active":"2023-11-25T04:25:47.000478+0000","last_became_peered":"2023-11-25T04:25:47.000478+0000","last_unstale":"2023-11-25T04:25:59.979124+0000","last_undegraded":"2023-11-25T04:25:59.979124+0000","last_fullsized":"2023-11-25T04:25:59.979124+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:50:13.896985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":90,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978907+0000","last_change":"2023-11-25T04:25:46.996182+0000","last_active":"2023-11-25T04:25:59.978907+0000","last_peered":"2023-11-25T04:25:59.978907+0000","last_clean":"2023-11-25T04:25:59.978907+0000","last_became_active":"2023-11-25T04:25:46.994525+0000","last_became_peered":"2023-11-25T04:25:46.994525+0000","last_unstale":"2023-11-25T04:25:59.978907+0000","last_undegraded":"2023-11-25T04:25:59.978907+0000","last_fullsized":"2023-11-25T04:25:59.978907+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:26:24.469530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"54'177","reported_seq":485,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.977915+0000","last_change":"2023-11-25T04:25:51.922466+0000","last_active":"2023-11-25T04:25:59.977915+0000","last_peered":"2023-11-25T04:25:59.977915+0000","last_clean":"2023-11-25T04:25:59.977915+0000","last_became_active":"2023-11-25T04:25:51.921938+0000","last_became_peered":"2023-11-25T04:25:51.921938+0000","last_unstale":"2023-11-25T04:25:59.977915+0000","last_undegraded":"2023-11-25T04:25:59.977915+0000","last_fullsized":"2023-11-25T04:25:59.977915+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:38:59.016525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":522494,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":168,"num_write_kb":749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":111,"num_bytes_recovered":522494,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"46'2","reported_seq":82,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979031+0000","last_change":"2023-11-25T04:25:46.998867+0000","last_active":"2023-11-25T04:25:59.979031+0000","last_peered":"2023-11-25T04:25:59.979031+0000","last_clean":"2023-11-25T04:25:59.979031+0000","last_became_active":"2023-11-25T04:25:46.998508+0000","last_became_peered":"2023-11-25T04:25:46.998508+0000","last_unstale":"2023-11-25T04:25:59.979031+0000","last_undegraded":"2023-11-25T04:25:59.979031+0000","last_fullsized":"2023-11-25T04:25:59.979031+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:50:13.896985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.10","version":"54'153","reported_seq":221,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.977904+0000","last_change":"2023-11-25T04:22:47.037790+0000","last_active":"2023-11-25T04:25:59.977904+0000","last_peered":"2023-11-25T04:25:59.977904+0000","last_clean":"2023-11-25T04:25:59.977904+0000","last_became_active":"2023-11-25T04:22:47.037149+0000","last_became_peered":"2023-11-25T04:22:47.037149+0000","last_unstale":"2023-11-25T04:25:59.977904+0000","last_undegraded":"2023-11-25T04:25:59.977904+0000","last_fullsized":"2023-11-25T04:25:59.977904+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:31:54.857270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":218073,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1f","version":"54'156","reported_seq":448,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979259+0000","last_change":"2023-11-25T04:25:49.921904+0000","last_active":"2023-11-25T04:25:59.979259+0000","last_peered":"2023-11-25T04:25:59.979259+0000","last_clean":"2023-11-25T04:25:59.979259+0000","last_became_active":"2023-11-25T04:25:49.921458+0000","last_became_peered":"2023-11-25T04:25:49.921458+0000","last_unstale":"2023-11-25T04:25:59.979259+0000","last_undegraded":"2023-11-25T04:25:59.979259+0000","last_fullsized":"2023-11-25T04:25:59.979259+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:59:27.602060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":107712,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":103,"num_bytes_recovered":107703,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"48'1","reported_seq":70,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978057+0000","last_change":"2023-11-25T04:22:46.036085+0000","last_active":"2023-11-25T04:25:59.978057+0000","last_peered":"2023-11-25T04:25:59.978057+0000","last_clean":"2023-11-25T04:25:59.978057+0000","last_became_active":"2023-11-25T04:22:46.035495+0000","last_became_peered":"2023-11-25T04:22:46.035495+0000","last_unstale":"2023-11-25T04:25:59.978057+0000","last_undegraded":"2023-11-25T04:25:59.978057+0000","last_fullsized":"2023-11-25T04:25:59.978057+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:14:56.735365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"54'152","reported_seq":430,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978757+0000","last_change":"2023-11-25T04:25:50.923882+0000","last_active":"2023-11-25T04:25:59.978757+0000","last_peered":"2023-11-25T04:25:59.978757+0000","last_clean":"2023-11-25T04:25:59.978757+0000","last_became_active":"2023-11-25T04:25:50.923375+0000","last_became_peered":"2023-11-25T04:25:50.923375+0000","last_unstale":"2023-11-25T04:25:59.978757+0000","last_undegraded":"2023-11-25T04:25:59.978757+0000","last_fullsized":"2023-11-25T04:25:59.978757+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:54:51.339025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":76435,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":76435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":80,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978069+0000","last_change":"2023-11-25T04:25:46.995228+0000","last_active":"2023-11-25T04:25:59.978069+0000","last_peered":"2023-11-25T04:25:59.978069+0000","last_clean":"2023-11-25T04:25:59.978069+0000","last_became_active":"2023-11-25T04:25:46.994871+0000","last_became_peered":"2023-11-25T04:25:46.994871+0000","last_unstale":"2023-11-25T04:25:59.978069+0000","last_undegraded":"2023-11-25T04:25:59.978069+0000","last_fullsized":"2023-11-25T04:25:59.978069+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:53:34.807075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"54'158","reported_seq":432,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978899+0000","last_change":"2023-11-25T04:25:52.925523+0000","last_active":"2023-11-25T04:25:59.978899+0000","last_peered":"2023-11-25T04:25:59.978899+0000","last_clean":"2023-11-25T04:25:59.978899+0000","last_became_active":"2023-11-25T04:25:52.925038+0000","last_became_peered":"2023-11-25T04:25:52.925038+0000","last_unstale":"2023-11-25T04:25:59.978899+0000","last_undegraded":"2023-11-25T04:25:59.978899+0000","last_fullsized":"2023-11-25T04:25:59.978899+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:37:14.620534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":209500,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":139,"num_write_kb":351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":209500,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1a","version":"48'4","reported_seq":73,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978665+0000","last_change":"2023-11-25T04:22:46.034340+0000","last_active":"2023-11-25T04:25:59.978665+0000","last_peered":"2023-11-25T04:25:59.978665+0000","last_clean":"2023-11-25T04:25:59.978665+0000","last_became_active":"2023-11-25T04:22:46.033944+0000","last_became_peered":"2023-11-25T04:22:46.033944+0000","last_unstale":"2023-11-25T04:25:59.978665+0000","last_undegraded":"2023-11-25T04:25:59.978665+0000","last_fullsized":"2023-11-25T04:25:59.978665+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:59:30.291004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"54'163","reported_seq":231,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978741+0000","last_change":"2023-11-25T04:22:47.039547+0000","last_active":"2023-11-25T04:25:59.978741+0000","last_peered":"2023-11-25T04:25:59.978741+0000","last_clean":"2023-11-25T04:25:59.978741+0000","last_became_active":"2023-11-25T04:22:47.039218+0000","last_became_peered":"2023-11-25T04:22:47.039218+0000","last_unstale":"2023-11-25T04:25:59.978741+0000","last_undegraded":"2023-11-25T04:25:59.978741+0000","last_fullsized":"2023-11-25T04:25:59.978741+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:44:48.553972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":165883,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"45'9","reported_seq":94,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978700+0000","last_change":"2023-11-25T04:25:47.023365+0000","last_active":"2023-11-25T04:25:59.978700+0000","last_peered":"2023-11-25T04:25:59.978700+0000","last_clean":"2023-11-25T04:25:59.978700+0000","last_became_active":"2023-11-25T04:25:46.999998+0000","last_became_peered":"2023-11-25T04:25:46.999998+0000","last_unstale":"2023-11-25T04:25:59.978700+0000","last_undegraded":"2023-11-25T04:25:59.978700+0000","last_fullsized":"2023-11-25T04:25:59.978700+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:53:34.807075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":4101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4194304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"54'149","reported_seq":217,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978573+0000","last_change":"2023-11-25T04:25:50.923041+0000","last_active":"2023-11-25T04:25:59.978573+0000","last_peered":"2023-11-25T04:25:59.978573+0000","last_clean":"2023-11-25T04:25:59.978573+0000","last_became_active":"2023-11-25T04:25:50.922164+0000","last_became_peered":"2023-11-25T04:25:50.922164+0000","last_unstale":"2023-11-25T04:25:59.978573+0000","last_undegraded":"2023-11-25T04:25:59.978573+0000","last_fullsized":"2023-11-25T04:25:59.978573+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:31:10.744814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":179506,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":155,"num_write_kb":317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":179506,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"18'2","reported_seq":37,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978409+0000","last_change":"2023-11-25T04:25:46.996675+0000","last_active":"2023-11-25T04:25:59.978409+0000","last_peered":"2023-11-25T04:25:59.978409+0000","last_clean":"2023-11-25T04:25:59.978409+0000","last_became_active":"2023-11-25T04:25:46.995160+0000","last_became_peered":"2023-11-25T04:25:46.995160+0000","last_unstale":"2023-11-25T04:25:59.978409+0000","last_undegraded":"2023-11-25T04:25:59.978409+0000","last_fullsized":"2023-11-25T04:25:59.978409+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:59:39.131605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1a","version":"54'162","reported_seq":226,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978332+0000","last_change":"2023-11-25T04:25:54.939182+0000","last_active":"2023-11-25T04:25:59.978332+0000","last_peered":"2023-11-25T04:25:59.978332+0000","last_clean":"2023-11-25T04:25:59.978332+0000","last_became_active":"2023-11-25T04:25:54.937821+0000","last_became_peered":"2023-11-25T04:25:54.937821+0000","last_unstale":"2023-11-25T04:25:59.978332+0000","last_undegraded":"2023-11-25T04:25:59.978332+0000","last_fullsized":"2023-11-25T04:25:59.978332+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:05:34.575966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":134672,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":169,"num_write_kb":311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":126291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"48'59","reported_seq":146,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979001+0000","last_change":"2023-11-25T04:25:47.077194+0000","last_active":"2023-11-25T04:25:59.979001+0000","last_peered":"2023-11-25T04:25:59.979001+0000","last_clean":"2023-11-25T04:25:59.979001+0000","last_became_active":"2023-11-25T04:25:47.000722+0000","last_became_peered":"2023-11-25T04:25:47.000722+0000","last_unstale":"2023-11-25T04:25:59.979001+0000","last_undegraded":"2023-11-25T04:25:59.979001+0000","last_fullsized":"2023-11-25T04:25:59.979001+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:55:01.351319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8388608,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"54'163","reported_seq":231,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978500+0000","last_change":"2023-11-25T04:22:47.036764+0000","last_active":"2023-11-25T04:25:59.978500+0000","last_peered":"2023-11-25T04:25:59.978500+0000","last_clean":"2023-11-25T04:25:59.978500+0000","last_became_active":"2023-11-25T04:22:47.036406+0000","last_became_peered":"2023-11-25T04:22:47.036406+0000","last_unstale":"2023-11-25T04:25:59.978500+0000","last_undegraded":"2023-11-25T04:25:59.978500+0000","last_fullsized":"2023-11-25T04:25:59.978500+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:09:55.357642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":527891,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":163,"num_write_kb":687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1e","version":"46'2","reported_seq":71,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978868+0000","last_change":"2023-11-25T04:22:46.035659+0000","last_active":"2023-11-25T04:25:59.978868+0000","last_peered":"2023-11-25T04:25:59.978868+0000","last_clean":"2023-11-25T04:25:59.978868+0000","last_became_active":"2023-11-25T04:22:46.035113+0000","last_became_peered":"2023-11-25T04:22:46.035113+0000","last_unstale":"2023-11-25T04:25:59.978868+0000","last_undegraded":"2023-11-25T04:25:59.978868+0000","last_fullsized":"2023-11-25T04:25:59.978868+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:26:25.338117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.18","version":"54'195","reported_seq":259,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978263+0000","last_change":"2023-11-25T04:25:54.938573+0000","last_active":"2023-11-25T04:25:59.978263+0000","last_peered":"2023-11-25T04:25:59.978263+0000","last_clean":"2023-11-25T04:25:59.978263+0000","last_became_active":"2023-11-25T04:25:54.937444+0000","last_became_peered":"2023-11-25T04:25:54.937444+0000","last_unstale":"2023-11-25T04:25:59.978263+0000","last_undegraded":"2023-11-25T04:25:59.978263+0000","last_fullsized":"2023-11-25T04:25:59.978263+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:20:15.814691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":128641,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":185,"num_write_kb":408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":122,"num_bytes_recovered":128641,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"50'17","reported_seq":102,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978969+0000","last_change":"2023-11-25T04:25:47.044228+0000","last_active":"2023-11-25T04:25:59.978969+0000","last_peered":"2023-11-25T04:25:59.978969+0000","last_clean":"2023-11-25T04:25:59.978969+0000","last_became_active":"2023-11-25T04:25:46.998870+0000","last_became_peered":"2023-11-25T04:25:46.998870+0000","last_unstale":"2023-11-25T04:25:59.978969+0000","last_undegraded":"2023-11-25T04:25:59.978969+0000","last_fullsized":"2023-11-25T04:25:59.978969+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:08:12.601941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":4103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4194304,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":26330855,"num_objects":3447,"num_object_clones":0,"num_object_copies":6894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10423,"num_write_kb":38550,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4189,"num_bytes_recovered":12749181,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":72376320,"data_stored":52324046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10246,"ondisk_log_size":10246,"up":128,"acting":128,"num_store_stats":3},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":40885548,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":35,"num_read_kb":334,"num_write":435,"num_write_kb":40462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":32491439,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":81833984,"data_stored":81771096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5548,"internal_metadata":0},"log_size":435,"ondisk_log_size":435,"up":64,"acting":64,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738415,"num_pgs":72,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73208,"kb_used_data":46188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94298632,"statfs":{"total":96636764160,"available":96561799168,"internally_reserved":0,"allocated":47296512,"data_stored":39787071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":1,"last update":"Sat Nov 25 04:25:40 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.433,"15min":0.44500000000000001},"min":{"1min":0.28899999999999998,"5min":0.216,"15min":0.216},"max":{"1min":0.60499999999999998,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.439,"15min":0.45100000000000001},"min":{"1min":0.25600000000000001,"5min":0.184,"15min":0.184},"max":{"1min":0.63400000000000001,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.63}]}]},{"osd":1,"up_from":8,"seq":34359738415,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86880,"kb_used_data":55700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94284960,"statfs":{"total":96636764160,"available":96547799040,"internally_reserved":0,"allocated":57036800,"data_stored":50261554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":2,"last update":"Sat Nov 25 04:25:42 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.40600000000000003,"15min":0.41099999999999998},"min":{"1min":0.29099999999999998,"5min":0.182,"15min":0.182},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.40200000000000002,"15min":0.40600000000000003},"min":{"1min":0.312,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.39000000000000001}]}]},{"osd":0,"up_from":60,"seq":257698037766,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87624,"kb_used_data":51764,"kb_used_omap":6,"kb_used_meta":35833,"kb_avail":94284216,"statfs":{"total":96636764160,"available":96547037184,"internally_reserved":0,"allocated":53006336,"data_stored":46141107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7143,"internal_metadata":36693017},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32530432,"data_stored":32496940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5548,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":29380608,"data_stored":29360982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":19922944,"data_stored":19913174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":19234816,"data_stored":12749181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":26415104,"data_stored":20005586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":26726400,"data_stored":19569279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-25T04:26:06.196 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-25T04:26:06.532 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:26:06.534 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2023-11-25T04:26:06.544 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":179,"stamp":"2023-11-25T04:26:05.550158+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":67806771,"num_objects":3481,"num_object_clones":0,"num_object_copies":6962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3481,"num_whiteouts":0,"num_read":171,"num_read_kb":452,"num_write":11061,"num_write_kb":82472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4201,"num_bytes_recovered":45240620,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10838,"ondisk_log_size":10838,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":247712,"kb_used_data":153652,"kb_used_omap":10,"kb_used_meta":94005,"kb_avail":282867808,"statfs":{"total":289910292480,"available":289656635392,"internally_reserved":0,"allocated":157339648,"data_stored":136189732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10333,"internal_metadata":96262051},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.586462"},"pg_stats":[{"pgid":"4.3f","version":"54'166","reported_seq":234,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979173+0000","last_change":"2023-11-25T04:22:47.039847+0000","last_active":"2023-11-25T04:25:59.979173+0000","last_peered":"2023-11-25T04:25:59.979173+0000","last_clean":"2023-11-25T04:25:59.979173+0000","last_became_active":"2023-11-25T04:22:47.039490+0000","last_became_peered":"2023-11-25T04:22:47.039490+0000","last_unstale":"2023-11-25T04:25:59.979173+0000","last_undegraded":"2023-11-25T04:25:59.979173+0000","last_fullsized":"2023-11-25T04:25:59.979173+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:25:06.132517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":577866,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":174,"num_write_kb":756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3e","version":"54'212","reported_seq":280,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978984+0000","last_change":"2023-11-25T04:22:47.040836+0000","last_active":"2023-11-25T04:25:59.978984+0000","last_peered":"2023-11-25T04:25:59.978984+0000","last_clean":"2023-11-25T04:25:59.978984+0000","last_became_active":"2023-11-25T04:22:47.040511+0000","last_became_peered":"2023-11-25T04:22:47.040511+0000","last_unstale":"2023-11-25T04:25:59.978984+0000","last_undegraded":"2023-11-25T04:25:59.978984+0000","last_fullsized":"2023-11-25T04:25:59.978984+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:15:39.511043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":461579,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":236,"num_write_kb":841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3d","version":"54'167","reported_seq":233,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978931+0000","last_change":"2023-11-25T04:25:52.924678+0000","last_active":"2023-11-25T04:25:59.978931+0000","last_peered":"2023-11-25T04:25:59.978931+0000","last_clean":"2023-11-25T04:25:59.978931+0000","last_became_active":"2023-11-25T04:25:52.923459+0000","last_became_peered":"2023-11-25T04:25:52.923459+0000","last_unstale":"2023-11-25T04:25:59.978931+0000","last_undegraded":"2023-11-25T04:25:59.978931+0000","last_fullsized":"2023-11-25T04:25:59.978931+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:24:06.950438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":145844,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":175,"num_write_kb":312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":138765,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"54'124","reported_seq":191,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978386+0000","last_change":"2023-11-25T04:25:51.922843+0000","last_active":"2023-11-25T04:25:59.978386+0000","last_peered":"2023-11-25T04:25:59.978386+0000","last_clean":"2023-11-25T04:25:59.978386+0000","last_became_active":"2023-11-25T04:25:51.921633+0000","last_became_peered":"2023-11-25T04:25:51.921633+0000","last_unstale":"2023-11-25T04:25:59.978386+0000","last_undegraded":"2023-11-25T04:25:59.978386+0000","last_fullsized":"2023-11-25T04:25:59.978386+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:05:31.332951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61303,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119,"num_write_kb":161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":61303,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"54'169","reported_seq":237,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978640+0000","last_change":"2023-11-25T04:22:47.039499+0000","last_active":"2023-11-25T04:25:59.978640+0000","last_peered":"2023-11-25T04:25:59.978640+0000","last_clean":"2023-11-25T04:25:59.978640+0000","last_became_active":"2023-11-25T04:22:47.039165+0000","last_became_peered":"2023-11-25T04:22:47.039165+0000","last_unstale":"2023-11-25T04:25:59.978640+0000","last_undegraded":"2023-11-25T04:25:59.978640+0000","last_fullsized":"2023-11-25T04:25:59.978640+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:57:28.855151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":118227,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":185,"num_write_kb":351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3a","version":"54'169","reported_seq":234,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978863+0000","last_change":"2023-11-25T04:25:53.933198+0000","last_active":"2023-11-25T04:25:59.978863+0000","last_peered":"2023-11-25T04:25:59.978863+0000","last_clean":"2023-11-25T04:25:59.978863+0000","last_became_active":"2023-11-25T04:25:53.931982+0000","last_became_peered":"2023-11-25T04:25:53.931982+0000","last_unstale":"2023-11-25T04:25:59.978863+0000","last_undegraded":"2023-11-25T04:25:59.978863+0000","last_fullsized":"2023-11-25T04:25:59.978863+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:28:40.449148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100304,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":161,"num_write_kb":424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":100304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"54'168","reported_seq":237,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978380+0000","last_change":"2023-11-25T04:25:49.921033+0000","last_active":"2023-11-25T04:25:59.978380+0000","last_peered":"2023-11-25T04:25:59.978380+0000","last_clean":"2023-11-25T04:25:59.978380+0000","last_became_active":"2023-11-25T04:25:49.919354+0000","last_became_peered":"2023-11-25T04:25:49.919354+0000","last_unstale":"2023-11-25T04:25:59.978380+0000","last_undegraded":"2023-11-25T04:25:59.978380+0000","last_fullsized":"2023-11-25T04:25:59.978380+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:36:13.489397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":124558,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":167,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":111,"num_bytes_recovered":124533,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.38","version":"54'152","reported_seq":221,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978153+0000","last_change":"2023-11-25T04:25:49.920421+0000","last_active":"2023-11-25T04:25:59.978153+0000","last_peered":"2023-11-25T04:25:59.978153+0000","last_clean":"2023-11-25T04:25:59.978153+0000","last_became_active":"2023-11-25T04:25:49.919236+0000","last_became_peered":"2023-11-25T04:25:49.919236+0000","last_unstale":"2023-11-25T04:25:59.978153+0000","last_undegraded":"2023-11-25T04:25:59.978153+0000","last_fullsized":"2023-11-25T04:25:59.978153+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:32:41.956579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":120147,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":120121,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.37","version":"54'167","reported_seq":235,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978818+0000","last_change":"2023-11-25T04:22:47.038397+0000","last_active":"2023-11-25T04:25:59.978818+0000","last_peered":"2023-11-25T04:25:59.978818+0000","last_clean":"2023-11-25T04:25:59.978818+0000","last_became_active":"2023-11-25T04:22:47.037328+0000","last_became_peered":"2023-11-25T04:22:47.037328+0000","last_unstale":"2023-11-25T04:25:59.978818+0000","last_undegraded":"2023-11-25T04:25:59.978818+0000","last_fullsized":"2023-11-25T04:25:59.978818+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:56:59.689578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":174147,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":184,"num_write_kb":353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.36","version":"54'149","reported_seq":217,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979047+0000","last_change":"2023-11-25T04:22:47.040182+0000","last_active":"2023-11-25T04:25:59.979047+0000","last_peered":"2023-11-25T04:25:59.979047+0000","last_clean":"2023-11-25T04:25:59.979047+0000","last_became_active":"2023-11-25T04:22:47.039892+0000","last_became_peered":"2023-11-25T04:22:47.039892+0000","last_unstale":"2023-11-25T04:25:59.979047+0000","last_undegraded":"2023-11-25T04:25:59.979047+0000","last_fullsized":"2023-11-25T04:25:59.979047+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:38:09.176582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":233330,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"54'156","reported_seq":223,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979021+0000","last_change":"2023-11-25T04:25:51.923087+0000","last_active":"2023-11-25T04:25:59.979021+0000","last_peered":"2023-11-25T04:25:59.979021+0000","last_clean":"2023-11-25T04:25:59.979021+0000","last_became_active":"2023-11-25T04:25:51.921800+0000","last_became_peered":"2023-11-25T04:25:51.921800+0000","last_unstale":"2023-11-25T04:25:59.979021+0000","last_undegraded":"2023-11-25T04:25:59.979021+0000","last_fullsized":"2023-11-25T04:25:59.979021+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:29:47.766517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":302550,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":163,"num_write_kb":431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":300323,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.34","version":"54'136","reported_seq":400,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978706+0000","last_change":"2023-11-25T04:25:50.922843+0000","last_active":"2023-11-25T04:25:59.978706+0000","last_peered":"2023-11-25T04:25:59.978706+0000","last_clean":"2023-11-25T04:25:59.978706+0000","last_became_active":"2023-11-25T04:25:50.922309+0000","last_became_peered":"2023-11-25T04:25:50.922309+0000","last_unstale":"2023-11-25T04:25:59.978706+0000","last_undegraded":"2023-11-25T04:25:59.978706+0000","last_fullsized":"2023-11-25T04:25:59.978706+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:38:33.939562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":93053,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":131,"num_write_kb":248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":89,"num_bytes_recovered":93053,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"54'134","reported_seq":202,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978529+0000","last_change":"2023-11-25T04:22:47.040388+0000","last_active":"2023-11-25T04:25:59.978529+0000","last_peered":"2023-11-25T04:25:59.978529+0000","last_clean":"2023-11-25T04:25:59.978529+0000","last_became_active":"2023-11-25T04:22:47.040129+0000","last_became_peered":"2023-11-25T04:22:47.040129+0000","last_unstale":"2023-11-25T04:25:59.978529+0000","last_undegraded":"2023-11-25T04:25:59.978529+0000","last_fullsized":"2023-11-25T04:25:59.978529+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:31:54.857270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":149524,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":139,"num_write_kb":252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.32","version":"54'174","reported_seq":239,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978762+0000","last_change":"2023-11-25T04:25:53.934526+0000","last_active":"2023-11-25T04:25:59.978762+0000","last_peered":"2023-11-25T04:25:59.978762+0000","last_clean":"2023-11-25T04:25:59.978762+0000","last_became_active":"2023-11-25T04:25:53.933694+0000","last_became_peered":"2023-11-25T04:25:53.933694+0000","last_unstale":"2023-11-25T04:25:59.978762+0000","last_undegraded":"2023-11-25T04:25:59.978762+0000","last_fullsized":"2023-11-25T04:25:59.978762+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:01:16.858976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":95797,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":181,"num_write_kb":547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":103,"num_bytes_recovered":95797,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.31","version":"54'156","reported_seq":225,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978209+0000","last_change":"2023-11-25T04:25:49.922396+0000","last_active":"2023-11-25T04:25:59.978209+0000","last_peered":"2023-11-25T04:25:59.978209+0000","last_clean":"2023-11-25T04:25:59.978209+0000","last_became_active":"2023-11-25T04:25:49.921129+0000","last_became_peered":"2023-11-25T04:25:49.921129+0000","last_unstale":"2023-11-25T04:25:59.978209+0000","last_undegraded":"2023-11-25T04:25:59.978209+0000","last_fullsized":"2023-11-25T04:25:59.978209+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:57:02.387207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":163190,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":169,"num_write_kb":336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":154576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"54'174","reported_seq":239,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978986+0000","last_change":"2023-11-25T04:25:53.934614+0000","last_active":"2023-11-25T04:25:59.978986+0000","last_peered":"2023-11-25T04:25:59.978986+0000","last_clean":"2023-11-25T04:25:59.978986+0000","last_became_active":"2023-11-25T04:25:53.933395+0000","last_became_peered":"2023-11-25T04:25:53.933395+0000","last_unstale":"2023-11-25T04:25:59.978986+0000","last_undegraded":"2023-11-25T04:25:59.978986+0000","last_fullsized":"2023-11-25T04:25:59.978986+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:53:09.827809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1423587,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":182,"num_write_kb":1626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":1414508,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2f","version":"54'152","reported_seq":220,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979084+0000","last_change":"2023-11-25T04:22:47.039389+0000","last_active":"2023-11-25T04:25:59.979084+0000","last_peered":"2023-11-25T04:25:59.979084+0000","last_clean":"2023-11-25T04:25:59.979084+0000","last_became_active":"2023-11-25T04:22:47.039123+0000","last_became_peered":"2023-11-25T04:22:47.039123+0000","last_unstale":"2023-11-25T04:25:59.979084+0000","last_undegraded":"2023-11-25T04:25:59.979084+0000","last_fullsized":"2023-11-25T04:25:59.979084+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:09:11.570561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":122752,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":143,"num_write_kb":435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"54'148","reported_seq":216,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979169+0000","last_change":"2023-11-25T04:22:47.036946+0000","last_active":"2023-11-25T04:25:59.979169+0000","last_peered":"2023-11-25T04:25:59.979169+0000","last_clean":"2023-11-25T04:25:59.979169+0000","last_became_active":"2023-11-25T04:22:47.036411+0000","last_became_peered":"2023-11-25T04:22:47.036411+0000","last_unstale":"2023-11-25T04:25:59.979169+0000","last_undegraded":"2023-11-25T04:25:59.979169+0000","last_fullsized":"2023-11-25T04:25:59.979169+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:16:29.085050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4770899,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":140,"num_write_kb":4870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2d","version":"54'143","reported_seq":211,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978472+0000","last_change":"2023-11-25T04:22:47.038983+0000","last_active":"2023-11-25T04:25:59.978472+0000","last_peered":"2023-11-25T04:25:59.978472+0000","last_clean":"2023-11-25T04:25:59.978472+0000","last_became_active":"2023-11-25T04:22:47.038477+0000","last_became_peered":"2023-11-25T04:22:47.038477+0000","last_unstale":"2023-11-25T04:25:59.978472+0000","last_undegraded":"2023-11-25T04:25:59.978472+0000","last_fullsized":"2023-11-25T04:25:59.978472+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:30:04.568790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":115975,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"54'167","reported_seq":233,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978615+0000","last_change":"2023-11-25T04:25:52.923640+0000","last_active":"2023-11-25T04:25:59.978615+0000","last_peered":"2023-11-25T04:25:59.978615+0000","last_clean":"2023-11-25T04:25:59.978615+0000","last_became_active":"2023-11-25T04:25:52.922367+0000","last_became_peered":"2023-11-25T04:25:52.922367+0000","last_unstale":"2023-11-25T04:25:59.978615+0000","last_undegraded":"2023-11-25T04:25:59.978615+0000","last_fullsized":"2023-11-25T04:25:59.978615+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:08:24.794280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":747090,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":211,"num_write_kb":985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":732276,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"54'156","reported_seq":222,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978632+0000","last_change":"2023-11-25T04:25:52.924457+0000","last_active":"2023-11-25T04:25:59.978632+0000","last_peered":"2023-11-25T04:25:59.978632+0000","last_clean":"2023-11-25T04:25:59.978632+0000","last_became_active":"2023-11-25T04:25:52.923378+0000","last_became_peered":"2023-11-25T04:25:52.923378+0000","last_unstale":"2023-11-25T04:25:59.978632+0000","last_undegraded":"2023-11-25T04:25:59.978632+0000","last_fullsized":"2023-11-25T04:25:59.978632+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:34:30.942779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69580,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":142,"num_write_kb":315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":98,"num_bytes_recovered":69580,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2a","version":"54'171","reported_seq":467,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978347+0000","last_change":"2023-11-25T04:25:52.925532+0000","last_active":"2023-11-25T04:25:59.978347+0000","last_peered":"2023-11-25T04:25:59.978347+0000","last_clean":"2023-11-25T04:25:59.978347+0000","last_became_active":"2023-11-25T04:25:52.924844+0000","last_became_peered":"2023-11-25T04:25:52.924844+0000","last_unstale":"2023-11-25T04:25:59.978347+0000","last_undegraded":"2023-11-25T04:25:59.978347+0000","last_fullsized":"2023-11-25T04:25:59.978347+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:11:30.124326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":115997,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":162,"num_write_kb":261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":115997,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.29","version":"54'152","reported_seq":438,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.977803+0000","last_change":"2023-11-25T04:25:49.920367+0000","last_active":"2023-11-25T04:25:59.977803+0000","last_peered":"2023-11-25T04:25:59.977803+0000","last_clean":"2023-11-25T04:25:59.977803+0000","last_became_active":"2023-11-25T04:25:49.919886+0000","last_became_peered":"2023-11-25T04:25:49.919886+0000","last_unstale":"2023-11-25T04:25:59.977803+0000","last_undegraded":"2023-11-25T04:25:59.977803+0000","last_fullsized":"2023-11-25T04:25:59.977803+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:04:58.273982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":212878,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":212878,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"54'169","reported_seq":237,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978222+0000","last_change":"2023-11-25T04:22:47.036805+0000","last_active":"2023-11-25T04:25:59.978222+0000","last_peered":"2023-11-25T04:25:59.978222+0000","last_clean":"2023-11-25T04:25:59.978222+0000","last_became_active":"2023-11-25T04:22:47.036135+0000","last_became_peered":"2023-11-25T04:22:47.036135+0000","last_unstale":"2023-11-25T04:25:59.978222+0000","last_undegraded":"2023-11-25T04:25:59.978222+0000","last_fullsized":"2023-11-25T04:25:59.978222+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:25:21.730992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":102957,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":172,"num_write_kb":610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"54'167","reported_seq":465,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979220+0000","last_change":"2023-11-25T04:25:51.922925+0000","last_active":"2023-11-25T04:25:59.979220+0000","last_peered":"2023-11-25T04:25:59.979220+0000","last_clean":"2023-11-25T04:25:59.979220+0000","last_became_active":"2023-11-25T04:25:51.922364+0000","last_became_peered":"2023-11-25T04:25:51.922364+0000","last_unstale":"2023-11-25T04:25:59.979220+0000","last_undegraded":"2023-11-25T04:25:59.979220+0000","last_fullsized":"2023-11-25T04:25:59.979220+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:09:23.741871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":182683,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":171,"num_write_kb":791,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":182683,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"54'187","reported_seq":505,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978723+0000","last_change":"2023-11-25T04:25:51.922529+0000","last_active":"2023-11-25T04:25:59.978723+0000","last_peered":"2023-11-25T04:25:59.978723+0000","last_clean":"2023-11-25T04:25:59.978723+0000","last_became_active":"2023-11-25T04:25:51.922064+0000","last_became_peered":"2023-11-25T04:25:51.922064+0000","last_unstale":"2023-11-25T04:25:59.978723+0000","last_undegraded":"2023-11-25T04:25:59.978723+0000","last_fullsized":"2023-11-25T04:25:59.978723+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":187,"log_dups_size":0,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:01:39.413986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":86311,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":171,"num_write_kb":274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":116,"num_bytes_recovered":86311,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.25","version":"54'149","reported_seq":215,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978829+0000","last_change":"2023-11-25T04:25:52.925505+0000","last_active":"2023-11-25T04:25:59.978829+0000","last_peered":"2023-11-25T04:25:59.978829+0000","last_clean":"2023-11-25T04:25:59.978829+0000","last_became_active":"2023-11-25T04:25:52.924795+0000","last_became_peered":"2023-11-25T04:25:52.924795+0000","last_unstale":"2023-11-25T04:25:59.978829+0000","last_undegraded":"2023-11-25T04:25:59.978829+0000","last_fullsized":"2023-11-25T04:25:59.978829+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:30:46.532141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":128288,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":167,"num_write_kb":267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":120265,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.24","version":"54'151","reported_seq":427,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978903+0000","last_change":"2023-11-25T04:25:50.922751+0000","last_active":"2023-11-25T04:25:59.978903+0000","last_peered":"2023-11-25T04:25:59.978903+0000","last_clean":"2023-11-25T04:25:59.978903+0000","last_became_active":"2023-11-25T04:25:50.922216+0000","last_became_peered":"2023-11-25T04:25:50.922216+0000","last_unstale":"2023-11-25T04:25:59.978903+0000","last_undegraded":"2023-11-25T04:25:59.978903+0000","last_fullsized":"2023-11-25T04:25:59.978903+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:40:16.255665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":107507,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":148,"num_write_kb":288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":107494,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.23","version":"54'155","reported_seq":223,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978207+0000","last_change":"2023-11-25T04:22:47.037668+0000","last_active":"2023-11-25T04:25:59.978207+0000","last_peered":"2023-11-25T04:25:59.978207+0000","last_clean":"2023-11-25T04:25:59.978207+0000","last_became_active":"2023-11-25T04:22:47.036823+0000","last_became_peered":"2023-11-25T04:22:47.036823+0000","last_unstale":"2023-11-25T04:25:59.978207+0000","last_undegraded":"2023-11-25T04:25:59.978207+0000","last_fullsized":"2023-11-25T04:25:59.978207+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:02:07.088241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4304582,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":152,"num_write_kb":4331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.22","version":"54'189","reported_seq":517,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978625+0000","last_change":"2023-11-25T04:25:54.940559+0000","last_active":"2023-11-25T04:25:59.978625+0000","last_peered":"2023-11-25T04:25:59.978625+0000","last_clean":"2023-11-25T04:25:59.978625+0000","last_became_active":"2023-11-25T04:25:54.939584+0000","last_became_peered":"2023-11-25T04:25:54.939584+0000","last_unstale":"2023-11-25T04:25:59.978625+0000","last_undegraded":"2023-11-25T04:25:59.978625+0000","last_fullsized":"2023-11-25T04:25:59.978625+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:01:39.413986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":129145,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":185,"num_write_kb":432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":121,"num_bytes_recovered":129107,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"54'201","reported_seq":269,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.977881+0000","last_change":"2023-11-25T04:22:47.037786+0000","last_active":"2023-11-25T04:25:59.977881+0000","last_peered":"2023-11-25T04:25:59.977881+0000","last_clean":"2023-11-25T04:25:59.977881+0000","last_became_active":"2023-11-25T04:22:47.037206+0000","last_became_peered":"2023-11-25T04:22:47.037206+0000","last_unstale":"2023-11-25T04:25:59.977881+0000","last_undegraded":"2023-11-25T04:25:59.977881+0000","last_fullsized":"2023-11-25T04:25:59.977881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:11:40.532140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":215311,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":194,"num_write_kb":381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"54'180","reported_seq":245,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.977884+0000","last_change":"2023-11-25T04:25:53.933176+0000","last_active":"2023-11-25T04:25:59.977884+0000","last_peered":"2023-11-25T04:25:59.977884+0000","last_clean":"2023-11-25T04:25:59.977884+0000","last_became_active":"2023-11-25T04:25:53.931937+0000","last_became_peered":"2023-11-25T04:25:53.931937+0000","last_unstale":"2023-11-25T04:25:59.977884+0000","last_undegraded":"2023-11-25T04:25:59.977884+0000","last_fullsized":"2023-11-25T04:25:59.977884+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:41:10.697064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104603,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":172,"num_write_kb":267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":104603,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"54'137","reported_seq":205,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978811+0000","last_change":"2023-11-25T04:22:47.036689+0000","last_active":"2023-11-25T04:25:59.978811+0000","last_peered":"2023-11-25T04:25:59.978811+0000","last_clean":"2023-11-25T04:25:59.978811+0000","last_became_active":"2023-11-25T04:22:47.036107+0000","last_became_peered":"2023-11-25T04:22:47.036107+0000","last_unstale":"2023-11-25T04:25:59.978811+0000","last_undegraded":"2023-11-25T04:25:59.978811+0000","last_fullsized":"2023-11-25T04:25:59.978811+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:02:07.088241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":183586,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":127,"num_write_kb":284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"58'114","reported_seq":191,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978227+0000","last_change":"2023-11-25T04:25:49.920320+0000","last_active":"2023-11-25T04:25:59.978227+0000","last_peered":"2023-11-25T04:25:59.978227+0000","last_clean":"2023-11-25T04:25:59.978227+0000","last_became_active":"2023-11-25T04:25:49.920010+0000","last_became_peered":"2023-11-25T04:25:49.920010+0000","last_unstale":"2023-11-25T04:25:59.978227+0000","last_undegraded":"2023-11-25T04:25:59.978227+0000","last_fullsized":"2023-11-25T04:25:59.978227+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:48:58.746789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2816216,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":113,"num_write_kb":2816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":2816182,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"54'145","reported_seq":213,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978995+0000","last_change":"2023-11-25T04:22:47.037888+0000","last_active":"2023-11-25T04:25:59.978995+0000","last_peered":"2023-11-25T04:25:59.978995+0000","last_clean":"2023-11-25T04:25:59.978995+0000","last_became_active":"2023-11-25T04:22:47.037158+0000","last_became_peered":"2023-11-25T04:22:47.037158+0000","last_unstale":"2023-11-25T04:25:59.978995+0000","last_undegraded":"2023-11-25T04:25:59.978995+0000","last_fullsized":"2023-11-25T04:25:59.978995+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:56:59.689578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":205976,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.12","version":"49'2","reported_seq":82,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978301+0000","last_change":"2023-11-25T04:25:47.000916+0000","last_active":"2023-11-25T04:25:59.978301+0000","last_peered":"2023-11-25T04:25:59.978301+0000","last_clean":"2023-11-25T04:25:59.978301+0000","last_became_active":"2023-11-25T04:25:47.000516+0000","last_became_peered":"2023-11-25T04:25:47.000516+0000","last_unstale":"2023-11-25T04:25:59.978301+0000","last_undegraded":"2023-11-25T04:25:59.978301+0000","last_fullsized":"2023-11-25T04:25:59.978301+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:53:58.873447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"54'175","reported_seq":489,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978855+0000","last_change":"2023-11-25T04:25:52.923694+0000","last_active":"2023-11-25T04:25:59.978855+0000","last_peered":"2023-11-25T04:25:59.978855+0000","last_clean":"2023-11-25T04:25:59.978855+0000","last_became_active":"2023-11-25T04:25:52.923045+0000","last_became_peered":"2023-11-25T04:25:52.923045+0000","last_unstale":"2023-11-25T04:25:59.978855+0000","last_undegraded":"2023-11-25T04:25:59.978855+0000","last_fullsized":"2023-11-25T04:25:59.978855+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:46:12.914433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":171823,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":185,"num_write_kb":340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":166387,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":80,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978355+0000","last_change":"2023-11-25T04:25:46.994324+0000","last_active":"2023-11-25T04:25:59.978355+0000","last_peered":"2023-11-25T04:25:59.978355+0000","last_clean":"2023-11-25T04:25:59.978355+0000","last_became_active":"2023-11-25T04:25:46.993870+0000","last_became_peered":"2023-11-25T04:25:46.993870+0000","last_unstale":"2023-11-25T04:25:59.978355+0000","last_undegraded":"2023-11-25T04:25:59.978355+0000","last_fullsized":"2023-11-25T04:25:59.978355+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:51:09.827080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"54'152","reported_seq":435,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978827+0000","last_change":"2023-11-25T04:25:50.922922+0000","last_active":"2023-11-25T04:25:59.978827+0000","last_peered":"2023-11-25T04:25:59.978827+0000","last_clean":"2023-11-25T04:25:59.978827+0000","last_became_active":"2023-11-25T04:25:50.922511+0000","last_became_peered":"2023-11-25T04:25:50.922511+0000","last_unstale":"2023-11-25T04:25:59.978827+0000","last_undegraded":"2023-11-25T04:25:59.978827+0000","last_fullsized":"2023-11-25T04:25:59.978827+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:37:14.620534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":579704,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":97,"num_bytes_recovered":570044,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.10","version":"18'8","reported_seq":39,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.977878+0000","last_change":"2023-11-25T04:25:47.000499+0000","last_active":"2023-11-25T04:25:59.977878+0000","last_peered":"2023-11-25T04:25:59.977878+0000","last_clean":"2023-11-25T04:25:59.977878+0000","last_became_active":"2023-11-25T04:25:46.998778+0000","last_became_peered":"2023-11-25T04:25:46.998778+0000","last_unstale":"2023-11-25T04:25:59.977878+0000","last_undegraded":"2023-11-25T04:25:59.977878+0000","last_fullsized":"2023-11-25T04:25:59.977878+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:33:09.169689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"49'2","reported_seq":37,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979103+0000","last_change":"2023-11-25T04:25:46.998392+0000","last_active":"2023-11-25T04:25:59.979103+0000","last_peered":"2023-11-25T04:25:59.979103+0000","last_clean":"2023-11-25T04:25:59.979103+0000","last_became_active":"2023-11-25T04:25:46.997108+0000","last_became_peered":"2023-11-25T04:25:46.997108+0000","last_unstale":"2023-11-25T04:25:59.979103+0000","last_undegraded":"2023-11-25T04:25:59.979103+0000","last_fullsized":"2023-11-25T04:25:59.979103+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:02:00.073630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"54'159","reported_seq":443,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978233+0000","last_change":"2023-11-25T04:25:50.922282+0000","last_active":"2023-11-25T04:25:59.978233+0000","last_peered":"2023-11-25T04:25:59.978233+0000","last_clean":"2023-11-25T04:25:59.978233+0000","last_became_active":"2023-11-25T04:25:50.921772+0000","last_became_peered":"2023-11-25T04:25:50.921772+0000","last_unstale":"2023-11-25T04:25:59.978233+0000","last_undegraded":"2023-11-25T04:25:59.978233+0000","last_fullsized":"2023-11-25T04:25:59.978233+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:36:33.451689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112103,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":164,"num_write_kb":240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":99,"num_bytes_recovered":103472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"18'2","reported_seq":82,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979281+0000","last_change":"2023-11-25T04:25:46.994943+0000","last_active":"2023-11-25T04:25:59.979281+0000","last_peered":"2023-11-25T04:25:59.979281+0000","last_clean":"2023-11-25T04:25:59.979281+0000","last_became_active":"2023-11-25T04:25:46.994640+0000","last_became_peered":"2023-11-25T04:25:46.994640+0000","last_unstale":"2023-11-25T04:25:59.979281+0000","last_undegraded":"2023-11-25T04:25:59.979281+0000","last_fullsized":"2023-11-25T04:25:59.979281+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:08:12.601941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"54'173","reported_seq":241,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978244+0000","last_change":"2023-11-25T04:22:47.039685+0000","last_active":"2023-11-25T04:25:59.978244+0000","last_peered":"2023-11-25T04:25:59.978244+0000","last_clean":"2023-11-25T04:25:59.978244+0000","last_became_active":"2023-11-25T04:22:47.039296+0000","last_became_peered":"2023-11-25T04:22:47.039296+0000","last_unstale":"2023-11-25T04:25:59.978244+0000","last_undegraded":"2023-11-25T04:25:59.978244+0000","last_fullsized":"2023-11-25T04:25:59.978244+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:11:40.532140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154533,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":182,"num_write_kb":299,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"18'1","reported_seq":37,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978725+0000","last_change":"2023-11-25T04:25:47.000059+0000","last_active":"2023-11-25T04:25:59.978725+0000","last_peered":"2023-11-25T04:25:59.978725+0000","last_clean":"2023-11-25T04:25:59.978725+0000","last_became_active":"2023-11-25T04:25:46.999604+0000","last_became_peered":"2023-11-25T04:25:46.999604+0000","last_unstale":"2023-11-25T04:25:59.978725+0000","last_undegraded":"2023-11-25T04:25:59.978725+0000","last_fullsized":"2023-11-25T04:25:59.978725+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:14:43.431272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"54'147","reported_seq":211,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978982+0000","last_change":"2023-11-25T04:25:54.939274+0000","last_active":"2023-11-25T04:25:59.978982+0000","last_peered":"2023-11-25T04:25:59.978982+0000","last_clean":"2023-11-25T04:25:59.978982+0000","last_became_active":"2023-11-25T04:25:54.938236+0000","last_became_peered":"2023-11-25T04:25:54.938236+0000","last_unstale":"2023-11-25T04:25:59.978982+0000","last_undegraded":"2023-11-25T04:25:59.978982+0000","last_fullsized":"2023-11-25T04:25:59.978982+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":147,"log_dups_size":0,"ondisk_log_size":147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:11:43.785294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":87524,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":155,"num_write_kb":235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":87524,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978546+0000","last_change":"2023-11-25T04:25:46.997080+0000","last_active":"2023-11-25T04:25:59.978546+0000","last_peered":"2023-11-25T04:25:59.978546+0000","last_clean":"2023-11-25T04:25:59.978546+0000","last_became_active":"2023-11-25T04:25:46.995712+0000","last_became_peered":"2023-11-25T04:25:46.995712+0000","last_unstale":"2023-11-25T04:25:59.978546+0000","last_undegraded":"2023-11-25T04:25:59.978546+0000","last_fullsized":"2023-11-25T04:25:59.978546+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T16:01:00.922843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"54'155","reported_seq":441,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978274+0000","last_change":"2023-11-25T04:25:52.923096+0000","last_active":"2023-11-25T04:25:59.978274+0000","last_peered":"2023-11-25T04:25:59.978274+0000","last_clean":"2023-11-25T04:25:59.978274+0000","last_became_active":"2023-11-25T04:25:52.922549+0000","last_became_peered":"2023-11-25T04:25:52.922549+0000","last_unstale":"2023-11-25T04:25:59.978274+0000","last_undegraded":"2023-11-25T04:25:59.978274+0000","last_fullsized":"2023-11-25T04:25:59.978274+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:52:38.053808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":266919,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":264258,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"45'11","reported_seq":96,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978532+0000","last_change":"2023-11-25T04:25:47.092497+0000","last_active":"2023-11-25T04:25:59.978532+0000","last_peered":"2023-11-25T04:25:59.978532+0000","last_clean":"2023-11-25T04:25:59.978532+0000","last_became_active":"2023-11-25T04:25:47.000687+0000","last_became_peered":"2023-11-25T04:25:47.000687+0000","last_unstale":"2023-11-25T04:25:59.978532+0000","last_undegraded":"2023-11-25T04:25:59.978532+0000","last_fullsized":"2023-11-25T04:25:59.978532+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:47:09.798238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":4101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4194304,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"54'173","reported_seq":241,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978732+0000","last_change":"2023-11-25T04:25:50.923240+0000","last_active":"2023-11-25T04:25:59.978732+0000","last_peered":"2023-11-25T04:25:59.978732+0000","last_clean":"2023-11-25T04:25:59.978732+0000","last_became_active":"2023-11-25T04:25:50.922409+0000","last_became_peered":"2023-11-25T04:25:50.922409+0000","last_unstale":"2023-11-25T04:25:59.978732+0000","last_undegraded":"2023-11-25T04:25:59.978732+0000","last_fullsized":"2023-11-25T04:25:59.978732+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:53:30.920627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":553893,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":169,"num_write_kb":770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":553893,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978079+0000","last_change":"2023-11-25T04:25:46.998818+0000","last_active":"2023-11-25T04:25:59.978079+0000","last_peered":"2023-11-25T04:25:59.978079+0000","last_clean":"2023-11-25T04:25:59.978079+0000","last_became_active":"2023-11-25T04:25:46.997019+0000","last_became_peered":"2023-11-25T04:25:46.997019+0000","last_unstale":"2023-11-25T04:25:59.978079+0000","last_undegraded":"2023-11-25T04:25:59.978079+0000","last_fullsized":"2023-11-25T04:25:59.978079+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:51:47.882117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"54'163","reported_seq":228,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978685+0000","last_change":"2023-11-25T04:25:53.934183+0000","last_active":"2023-11-25T04:25:59.978685+0000","last_peered":"2023-11-25T04:25:59.978685+0000","last_clean":"2023-11-25T04:25:59.978685+0000","last_became_active":"2023-11-25T04:25:53.933471+0000","last_became_peered":"2023-11-25T04:25:53.933471+0000","last_unstale":"2023-11-25T04:25:59.978685+0000","last_undegraded":"2023-11-25T04:25:59.978685+0000","last_fullsized":"2023-11-25T04:25:59.978685+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:57:07.515414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":320867,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":190,"num_write_kb":577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":108,"num_bytes_recovered":311640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":80,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978412+0000","last_change":"2023-11-25T04:25:46.994480+0000","last_active":"2023-11-25T04:25:59.978412+0000","last_peered":"2023-11-25T04:25:59.978412+0000","last_clean":"2023-11-25T04:25:59.978412+0000","last_became_active":"2023-11-25T04:25:46.994101+0000","last_became_peered":"2023-11-25T04:25:46.994101+0000","last_unstale":"2023-11-25T04:25:59.978412+0000","last_undegraded":"2023-11-25T04:25:59.978412+0000","last_fullsized":"2023-11-25T04:25:59.978412+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:53:58.873447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"54'141","reported_seq":210,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979040+0000","last_change":"2023-11-25T04:25:49.921950+0000","last_active":"2023-11-25T04:25:59.979040+0000","last_peered":"2023-11-25T04:25:59.979040+0000","last_clean":"2023-11-25T04:25:59.979040+0000","last_became_active":"2023-11-25T04:25:49.920971+0000","last_became_peered":"2023-11-25T04:25:49.920971+0000","last_unstale":"2023-11-25T04:25:59.979040+0000","last_undegraded":"2023-11-25T04:25:59.979040+0000","last_fullsized":"2023-11-25T04:25:59.979040+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:22:38.004934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3502371,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":3718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":89,"num_bytes_recovered":3497669,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"49'1","reported_seq":70,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978352+0000","last_change":"2023-11-25T04:22:46.034509+0000","last_active":"2023-11-25T04:25:59.978352+0000","last_peered":"2023-11-25T04:25:59.978352+0000","last_clean":"2023-11-25T04:25:59.978352+0000","last_became_active":"2023-11-25T04:22:46.034158+0000","last_became_peered":"2023-11-25T04:22:46.034158+0000","last_unstale":"2023-11-25T04:25:59.978352+0000","last_undegraded":"2023-11-25T04:25:59.978352+0000","last_fullsized":"2023-11-25T04:25:59.978352+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:26:25.338117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"54'141","reported_seq":209,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978912+0000","last_change":"2023-11-25T04:22:47.038800+0000","last_active":"2023-11-25T04:25:59.978912+0000","last_peered":"2023-11-25T04:25:59.978912+0000","last_clean":"2023-11-25T04:25:59.978912+0000","last_became_active":"2023-11-25T04:22:47.038081+0000","last_became_peered":"2023-11-25T04:22:47.038081+0000","last_unstale":"2023-11-25T04:25:59.978912+0000","last_undegraded":"2023-11-25T04:25:59.978912+0000","last_fullsized":"2023-11-25T04:25:59.978912+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:45:40.065146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":174088,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":146,"num_write_kb":385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"12'157","reported_seq":232,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.977771+0000","last_change":"2023-11-25T04:25:47.000504+0000","last_active":"2023-11-25T04:25:59.977771+0000","last_peered":"2023-11-25T04:25:59.977771+0000","last_clean":"2023-11-25T04:25:59.977771+0000","last_became_active":"2023-11-25T04:25:46.999721+0000","last_became_peered":"2023-11-25T04:25:46.999721+0000","last_unstale":"2023-11-25T04:25:59.977771+0000","last_undegraded":"2023-11-25T04:25:59.977771+0000","last_fullsized":"2023-11-25T04:25:59.977771+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:35.919850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:35.919850+0000","last_clean_scrub_stamp":"2023-11-25T04:22:35.919850+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:26:33.607474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978082+0000","last_change":"2023-11-25T04:25:46.998960+0000","last_active":"2023-11-25T04:25:59.978082+0000","last_peered":"2023-11-25T04:25:59.978082+0000","last_clean":"2023-11-25T04:25:59.978082+0000","last_became_active":"2023-11-25T04:25:46.997693+0000","last_became_peered":"2023-11-25T04:25:46.997693+0000","last_unstale":"2023-11-25T04:25:59.978082+0000","last_undegraded":"2023-11-25T04:25:59.978082+0000","last_fullsized":"2023-11-25T04:25:59.978082+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:51:45.180957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"46'1","reported_seq":70,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978331+0000","last_change":"2023-11-25T04:22:46.037044+0000","last_active":"2023-11-25T04:25:59.978331+0000","last_peered":"2023-11-25T04:25:59.978331+0000","last_clean":"2023-11-25T04:25:59.978331+0000","last_became_active":"2023-11-25T04:22:46.035916+0000","last_became_peered":"2023-11-25T04:22:46.035916+0000","last_unstale":"2023-11-25T04:25:59.978331+0000","last_undegraded":"2023-11-25T04:25:59.978331+0000","last_fullsized":"2023-11-25T04:25:59.978331+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:38:20.501102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"54'146","reported_seq":213,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978804+0000","last_change":"2023-11-25T04:25:51.923452+0000","last_active":"2023-11-25T04:25:59.978804+0000","last_peered":"2023-11-25T04:25:59.978804+0000","last_clean":"2023-11-25T04:25:59.978804+0000","last_became_active":"2023-11-25T04:25:51.922490+0000","last_became_peered":"2023-11-25T04:25:51.922490+0000","last_unstale":"2023-11-25T04:25:59.978804+0000","last_undegraded":"2023-11-25T04:25:59.978804+0000","last_fullsized":"2023-11-25T04:25:59.978804+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:40:11.001062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":502154,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":170,"num_write_kb":687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":99,"num_bytes_recovered":502101,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978713+0000","last_change":"2023-11-25T04:25:47.000763+0000","last_active":"2023-11-25T04:25:59.978713+0000","last_peered":"2023-11-25T04:25:59.978713+0000","last_clean":"2023-11-25T04:25:59.978713+0000","last_became_active":"2023-11-25T04:25:47.000033+0000","last_became_peered":"2023-11-25T04:25:47.000033+0000","last_unstale":"2023-11-25T04:25:59.978713+0000","last_undegraded":"2023-11-25T04:25:59.978713+0000","last_fullsized":"2023-11-25T04:25:59.978713+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:53:59.131478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"50'37","reported_seq":156,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978116+0000","last_change":"2023-11-25T04:25:47.002009+0000","last_active":"2023-11-25T04:25:59.978116+0000","last_peered":"2023-11-25T04:25:59.978116+0000","last_clean":"2023-11-25T04:25:59.978116+0000","last_became_active":"2023-11-25T04:25:46.995637+0000","last_became_peered":"2023-11-25T04:25:46.995637+0000","last_unstale":"2023-11-25T04:25:59.978116+0000","last_undegraded":"2023-11-25T04:25:59.978116+0000","last_fullsized":"2023-11-25T04:25:59.978116+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:55:01.351319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":319160,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":35,"num_read_kb":334,"num_write":35,"num_write_kb":334,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":319160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"54'158","reported_seq":226,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979092+0000","last_change":"2023-11-25T04:22:47.039583+0000","last_active":"2023-11-25T04:25:59.979092+0000","last_peered":"2023-11-25T04:25:59.979092+0000","last_clean":"2023-11-25T04:25:59.979092+0000","last_became_active":"2023-11-25T04:22:47.039197+0000","last_became_peered":"2023-11-25T04:22:47.039197+0000","last_unstale":"2023-11-25T04:25:59.979092+0000","last_undegraded":"2023-11-25T04:25:59.979092+0000","last_fullsized":"2023-11-25T04:25:59.979092+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:37:30.959147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":169303,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":189,"num_write_kb":333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":72,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978222+0000","last_change":"2023-11-25T04:22:42.000382+0000","last_active":"2023-11-25T04:25:59.978222+0000","last_peered":"2023-11-25T04:25:59.978222+0000","last_clean":"2023-11-25T04:25:59.978222+0000","last_became_active":"2023-11-25T04:22:42.000041+0000","last_became_peered":"2023-11-25T04:22:42.000041+0000","last_unstale":"2023-11-25T04:25:59.978222+0000","last_undegraded":"2023-11-25T04:25:59.978222+0000","last_fullsized":"2023-11-25T04:25:59.978222+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:08:19.655864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"49'71","reported_seq":155,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978346+0000","last_change":"2023-11-25T04:25:47.109628+0000","last_active":"2023-11-25T04:25:59.978346+0000","last_peered":"2023-11-25T04:25:59.978346+0000","last_clean":"2023-11-25T04:25:59.978346+0000","last_became_active":"2023-11-25T04:25:47.001162+0000","last_became_peered":"2023-11-25T04:25:47.001162+0000","last_unstale":"2023-11-25T04:25:59.978346+0000","last_undegraded":"2023-11-25T04:25:59.978346+0000","last_fullsized":"2023-11-25T04:25:59.978346+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:03:33.658392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":69,"num_write_kb":4130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4194304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"54'158","reported_seq":226,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978945+0000","last_change":"2023-11-25T04:22:47.038454+0000","last_active":"2023-11-25T04:25:59.978945+0000","last_peered":"2023-11-25T04:25:59.978945+0000","last_clean":"2023-11-25T04:25:59.978945+0000","last_became_active":"2023-11-25T04:22:47.038046+0000","last_became_peered":"2023-11-25T04:22:47.038046+0000","last_unstale":"2023-11-25T04:25:59.978945+0000","last_undegraded":"2023-11-25T04:25:59.978945+0000","last_fullsized":"2023-11-25T04:25:59.978945+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:38:38.839941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":115501,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":163,"num_write_kb":243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.977941+0000","last_change":"2023-11-25T04:25:46.998739+0000","last_active":"2023-11-25T04:25:59.977941+0000","last_peered":"2023-11-25T04:25:59.977941+0000","last_clean":"2023-11-25T04:25:59.977941+0000","last_became_active":"2023-11-25T04:25:46.997391+0000","last_became_peered":"2023-11-25T04:25:46.997391+0000","last_unstale":"2023-11-25T04:25:59.977941+0000","last_undegraded":"2023-11-25T04:25:59.977941+0000","last_fullsized":"2023-11-25T04:25:59.977941+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:47:46.300596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"50'1","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978166+0000","last_change":"2023-11-25T04:25:46.999446+0000","last_active":"2023-11-25T04:25:59.978166+0000","last_peered":"2023-11-25T04:25:59.978166+0000","last_clean":"2023-11-25T04:25:59.978166+0000","last_became_active":"2023-11-25T04:25:46.998132+0000","last_became_peered":"2023-11-25T04:25:46.998132+0000","last_unstale":"2023-11-25T04:25:59.978166+0000","last_undegraded":"2023-11-25T04:25:59.978166+0000","last_fullsized":"2023-11-25T04:25:59.978166+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:42:51.166678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"54'157","reported_seq":225,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978638+0000","last_change":"2023-11-25T04:22:47.037355+0000","last_active":"2023-11-25T04:25:59.978638+0000","last_peered":"2023-11-25T04:25:59.978638+0000","last_clean":"2023-11-25T04:25:59.978638+0000","last_became_active":"2023-11-25T04:22:47.036924+0000","last_became_peered":"2023-11-25T04:22:47.036924+0000","last_unstale":"2023-11-25T04:25:59.978638+0000","last_undegraded":"2023-11-25T04:25:59.978638+0000","last_fullsized":"2023-11-25T04:25:59.978638+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:15:39.511043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":93321,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":148,"num_write_kb":360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978520+0000","last_change":"2023-11-25T04:25:46.999738+0000","last_active":"2023-11-25T04:25:59.978520+0000","last_peered":"2023-11-25T04:25:59.978520+0000","last_clean":"2023-11-25T04:25:59.978520+0000","last_became_active":"2023-11-25T04:25:46.999403+0000","last_became_peered":"2023-11-25T04:25:46.999403+0000","last_unstale":"2023-11-25T04:25:59.978520+0000","last_undegraded":"2023-11-25T04:25:59.978520+0000","last_fullsized":"2023-11-25T04:25:59.978520+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:45:53.304063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":72,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978705+0000","last_change":"2023-11-25T04:22:42.000204+0000","last_active":"2023-11-25T04:25:59.978705+0000","last_peered":"2023-11-25T04:25:59.978705+0000","last_clean":"2023-11-25T04:25:59.978705+0000","last_became_active":"2023-11-25T04:22:41.999915+0000","last_became_peered":"2023-11-25T04:22:41.999915+0000","last_unstale":"2023-11-25T04:25:59.978705+0000","last_undegraded":"2023-11-25T04:25:59.978705+0000","last_fullsized":"2023-11-25T04:25:59.978705+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:07:32.457905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"54'152","reported_seq":432,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978379+0000","last_change":"2023-11-25T04:25:53.933239+0000","last_active":"2023-11-25T04:25:59.978379+0000","last_peered":"2023-11-25T04:25:59.978379+0000","last_clean":"2023-11-25T04:25:59.978379+0000","last_became_active":"2023-11-25T04:25:53.932586+0000","last_became_peered":"2023-11-25T04:25:53.932586+0000","last_unstale":"2023-11-25T04:25:59.978379+0000","last_undegraded":"2023-11-25T04:25:59.978379+0000","last_fullsized":"2023-11-25T04:25:59.978379+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:54:51.339025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":329327,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":151,"num_write_kb":471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":97,"num_bytes_recovered":329278,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":69,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978632+0000","last_change":"2023-11-25T04:22:46.035885+0000","last_active":"2023-11-25T04:25:59.978632+0000","last_peered":"2023-11-25T04:25:59.978632+0000","last_clean":"2023-11-25T04:25:59.978632+0000","last_became_active":"2023-11-25T04:22:46.035405+0000","last_became_peered":"2023-11-25T04:22:46.035405+0000","last_unstale":"2023-11-25T04:25:59.978632+0000","last_undegraded":"2023-11-25T04:25:59.978632+0000","last_fullsized":"2023-11-25T04:25:59.978632+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:53:47.398746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":72,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979221+0000","last_change":"2023-11-25T04:22:41.999067+0000","last_active":"2023-11-25T04:25:59.979221+0000","last_peered":"2023-11-25T04:25:59.979221+0000","last_clean":"2023-11-25T04:25:59.979221+0000","last_became_active":"2023-11-25T04:22:41.998798+0000","last_became_peered":"2023-11-25T04:22:41.998798+0000","last_unstale":"2023-11-25T04:25:59.979221+0000","last_undegraded":"2023-11-25T04:25:59.979221+0000","last_fullsized":"2023-11-25T04:25:59.979221+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:00:04.266051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"54'172","reported_seq":470,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978595+0000","last_change":"2023-11-25T04:25:51.922403+0000","last_active":"2023-11-25T04:25:59.978595+0000","last_peered":"2023-11-25T04:25:59.978595+0000","last_clean":"2023-11-25T04:25:59.978595+0000","last_became_active":"2023-11-25T04:25:51.921858+0000","last_became_peered":"2023-11-25T04:25:51.921858+0000","last_unstale":"2023-11-25T04:25:59.978595+0000","last_undegraded":"2023-11-25T04:25:59.978595+0000","last_fullsized":"2023-11-25T04:25:59.978595+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:04:58.273982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":129965,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":166,"num_write_kb":290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":129965,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"18'2","reported_seq":71,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979088+0000","last_change":"2023-11-25T04:22:46.034606+0000","last_active":"2023-11-25T04:25:59.979088+0000","last_peered":"2023-11-25T04:25:59.979088+0000","last_clean":"2023-11-25T04:25:59.979088+0000","last_became_active":"2023-11-25T04:22:46.034229+0000","last_became_peered":"2023-11-25T04:22:46.034229+0000","last_unstale":"2023-11-25T04:25:59.979088+0000","last_undegraded":"2023-11-25T04:25:59.979088+0000","last_fullsized":"2023-11-25T04:25:59.979088+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:21:42.146264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":72,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978952+0000","last_change":"2023-11-25T04:22:42.000258+0000","last_active":"2023-11-25T04:25:59.978952+0000","last_peered":"2023-11-25T04:25:59.978952+0000","last_clean":"2023-11-25T04:25:59.978952+0000","last_became_active":"2023-11-25T04:22:41.999992+0000","last_became_peered":"2023-11-25T04:22:41.999992+0000","last_unstale":"2023-11-25T04:25:59.978952+0000","last_undegraded":"2023-11-25T04:25:59.978952+0000","last_fullsized":"2023-11-25T04:25:59.978952+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:31:13.715992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"54'150","reported_seq":436,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.977926+0000","last_change":"2023-11-25T04:25:54.938479+0000","last_active":"2023-11-25T04:25:59.977926+0000","last_peered":"2023-11-25T04:25:59.977926+0000","last_clean":"2023-11-25T04:25:59.977926+0000","last_became_active":"2023-11-25T04:25:54.937838+0000","last_became_peered":"2023-11-25T04:25:54.937838+0000","last_unstale":"2023-11-25T04:25:59.977926+0000","last_undegraded":"2023-11-25T04:25:59.977926+0000","last_fullsized":"2023-11-25T04:25:59.977926+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:51:14.394190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":159585,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":154224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"54'14","reported_seq":99,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978916+0000","last_change":"2023-11-25T04:25:47.026749+0000","last_active":"2023-11-25T04:25:59.978916+0000","last_peered":"2023-11-25T04:25:59.978916+0000","last_clean":"2023-11-25T04:25:59.978916+0000","last_became_active":"2023-11-25T04:25:46.997404+0000","last_became_peered":"2023-11-25T04:25:46.997404+0000","last_unstale":"2023-11-25T04:25:59.978916+0000","last_undegraded":"2023-11-25T04:25:59.978916+0000","last_fullsized":"2023-11-25T04:25:59.978916+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:20:31.719529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978930+0000","last_change":"2023-11-25T04:25:46.995200+0000","last_active":"2023-11-25T04:25:59.978930+0000","last_peered":"2023-11-25T04:25:59.978930+0000","last_clean":"2023-11-25T04:25:59.978930+0000","last_became_active":"2023-11-25T04:25:46.994834+0000","last_became_peered":"2023-11-25T04:25:46.994834+0000","last_unstale":"2023-11-25T04:25:59.978930+0000","last_undegraded":"2023-11-25T04:25:59.978930+0000","last_fullsized":"2023-11-25T04:25:59.978930+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:33:07.913911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"54'135","reported_seq":407,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978276+0000","last_change":"2023-11-25T04:25:49.922046+0000","last_active":"2023-11-25T04:25:59.978276+0000","last_peered":"2023-11-25T04:25:59.978276+0000","last_clean":"2023-11-25T04:25:59.978276+0000","last_became_active":"2023-11-25T04:25:49.921465+0000","last_became_peered":"2023-11-25T04:25:49.921465+0000","last_unstale":"2023-11-25T04:25:59.978276+0000","last_undegraded":"2023-11-25T04:25:59.978276+0000","last_fullsized":"2023-11-25T04:25:59.978276+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:12:54.077901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":99503,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":148,"num_write_kb":250,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":93,"num_bytes_recovered":99503,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"54'166","reported_seq":233,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978407+0000","last_change":"2023-11-25T04:25:51.923141+0000","last_active":"2023-11-25T04:25:59.978407+0000","last_peered":"2023-11-25T04:25:59.978407+0000","last_clean":"2023-11-25T04:25:59.978407+0000","last_became_active":"2023-11-25T04:25:51.922070+0000","last_became_peered":"2023-11-25T04:25:51.922070+0000","last_unstale":"2023-11-25T04:25:59.978407+0000","last_undegraded":"2023-11-25T04:25:59.978407+0000","last_fullsized":"2023-11-25T04:25:59.978407+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:34:53.021081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":185321,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":172,"num_write_kb":312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":164176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"58'50","reported_seq":53,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978227+0000","last_change":"2023-11-25T04:25:47.021705+0000","last_active":"2023-11-25T04:25:59.978227+0000","last_peered":"2023-11-25T04:25:59.978227+0000","last_clean":"2023-11-25T04:25:59.978227+0000","last_became_active":"2023-11-25T04:25:46.994997+0000","last_became_peered":"2023-11-25T04:25:46.994997+0000","last_unstale":"2023-11-25T04:25:59.978227+0000","last_undegraded":"2023-11-25T04:25:59.978227+0000","last_fullsized":"2023-11-25T04:25:59.978227+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:41:04.147372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":4119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4190183,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"54'153","reported_seq":221,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978107+0000","last_change":"2023-11-25T04:22:47.037181+0000","last_active":"2023-11-25T04:25:59.978107+0000","last_peered":"2023-11-25T04:25:59.978107+0000","last_clean":"2023-11-25T04:25:59.978107+0000","last_became_active":"2023-11-25T04:22:47.036821+0000","last_became_peered":"2023-11-25T04:22:47.036821+0000","last_unstale":"2023-11-25T04:25:59.978107+0000","last_undegraded":"2023-11-25T04:25:59.978107+0000","last_fullsized":"2023-11-25T04:25:59.978107+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:12:51.990448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":101117,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":161,"num_write_kb":242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"50'22","reported_seq":107,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979124+0000","last_change":"2023-11-25T04:25:47.028636+0000","last_active":"2023-11-25T04:25:59.979124+0000","last_peered":"2023-11-25T04:25:59.979124+0000","last_clean":"2023-11-25T04:25:59.979124+0000","last_became_active":"2023-11-25T04:25:47.000478+0000","last_became_peered":"2023-11-25T04:25:47.000478+0000","last_unstale":"2023-11-25T04:25:59.979124+0000","last_undegraded":"2023-11-25T04:25:59.979124+0000","last_fullsized":"2023-11-25T04:25:59.979124+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:50:13.896985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":90,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978907+0000","last_change":"2023-11-25T04:25:46.996182+0000","last_active":"2023-11-25T04:25:59.978907+0000","last_peered":"2023-11-25T04:25:59.978907+0000","last_clean":"2023-11-25T04:25:59.978907+0000","last_became_active":"2023-11-25T04:25:46.994525+0000","last_became_peered":"2023-11-25T04:25:46.994525+0000","last_unstale":"2023-11-25T04:25:59.978907+0000","last_undegraded":"2023-11-25T04:25:59.978907+0000","last_fullsized":"2023-11-25T04:25:59.978907+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:26:24.469530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"54'177","reported_seq":485,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.977915+0000","last_change":"2023-11-25T04:25:51.922466+0000","last_active":"2023-11-25T04:25:59.977915+0000","last_peered":"2023-11-25T04:25:59.977915+0000","last_clean":"2023-11-25T04:25:59.977915+0000","last_became_active":"2023-11-25T04:25:51.921938+0000","last_became_peered":"2023-11-25T04:25:51.921938+0000","last_unstale":"2023-11-25T04:25:59.977915+0000","last_undegraded":"2023-11-25T04:25:59.977915+0000","last_fullsized":"2023-11-25T04:25:59.977915+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:38:59.016525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":522494,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":168,"num_write_kb":749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":111,"num_bytes_recovered":522494,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"46'2","reported_seq":82,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979031+0000","last_change":"2023-11-25T04:25:46.998867+0000","last_active":"2023-11-25T04:25:59.979031+0000","last_peered":"2023-11-25T04:25:59.979031+0000","last_clean":"2023-11-25T04:25:59.979031+0000","last_became_active":"2023-11-25T04:25:46.998508+0000","last_became_peered":"2023-11-25T04:25:46.998508+0000","last_unstale":"2023-11-25T04:25:59.979031+0000","last_undegraded":"2023-11-25T04:25:59.979031+0000","last_fullsized":"2023-11-25T04:25:59.979031+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:50:13.896985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.10","version":"54'153","reported_seq":221,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.977904+0000","last_change":"2023-11-25T04:22:47.037790+0000","last_active":"2023-11-25T04:25:59.977904+0000","last_peered":"2023-11-25T04:25:59.977904+0000","last_clean":"2023-11-25T04:25:59.977904+0000","last_became_active":"2023-11-25T04:22:47.037149+0000","last_became_peered":"2023-11-25T04:22:47.037149+0000","last_unstale":"2023-11-25T04:25:59.977904+0000","last_undegraded":"2023-11-25T04:25:59.977904+0000","last_fullsized":"2023-11-25T04:25:59.977904+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:31:54.857270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":218073,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1f","version":"54'156","reported_seq":448,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979259+0000","last_change":"2023-11-25T04:25:49.921904+0000","last_active":"2023-11-25T04:25:59.979259+0000","last_peered":"2023-11-25T04:25:59.979259+0000","last_clean":"2023-11-25T04:25:59.979259+0000","last_became_active":"2023-11-25T04:25:49.921458+0000","last_became_peered":"2023-11-25T04:25:49.921458+0000","last_unstale":"2023-11-25T04:25:59.979259+0000","last_undegraded":"2023-11-25T04:25:59.979259+0000","last_fullsized":"2023-11-25T04:25:59.979259+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:59:27.602060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":107712,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":103,"num_bytes_recovered":107703,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"48'1","reported_seq":70,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978057+0000","last_change":"2023-11-25T04:22:46.036085+0000","last_active":"2023-11-25T04:25:59.978057+0000","last_peered":"2023-11-25T04:25:59.978057+0000","last_clean":"2023-11-25T04:25:59.978057+0000","last_became_active":"2023-11-25T04:22:46.035495+0000","last_became_peered":"2023-11-25T04:22:46.035495+0000","last_unstale":"2023-11-25T04:25:59.978057+0000","last_undegraded":"2023-11-25T04:25:59.978057+0000","last_fullsized":"2023-11-25T04:25:59.978057+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:14:56.735365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"54'152","reported_seq":430,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978757+0000","last_change":"2023-11-25T04:25:50.923882+0000","last_active":"2023-11-25T04:25:59.978757+0000","last_peered":"2023-11-25T04:25:59.978757+0000","last_clean":"2023-11-25T04:25:59.978757+0000","last_became_active":"2023-11-25T04:25:50.923375+0000","last_became_peered":"2023-11-25T04:25:50.923375+0000","last_unstale":"2023-11-25T04:25:59.978757+0000","last_undegraded":"2023-11-25T04:25:59.978757+0000","last_fullsized":"2023-11-25T04:25:59.978757+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:54:51.339025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":76435,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":76435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":80,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978069+0000","last_change":"2023-11-25T04:25:46.995228+0000","last_active":"2023-11-25T04:25:59.978069+0000","last_peered":"2023-11-25T04:25:59.978069+0000","last_clean":"2023-11-25T04:25:59.978069+0000","last_became_active":"2023-11-25T04:25:46.994871+0000","last_became_peered":"2023-11-25T04:25:46.994871+0000","last_unstale":"2023-11-25T04:25:59.978069+0000","last_undegraded":"2023-11-25T04:25:59.978069+0000","last_fullsized":"2023-11-25T04:25:59.978069+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:53:34.807075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"54'158","reported_seq":432,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978899+0000","last_change":"2023-11-25T04:25:52.925523+0000","last_active":"2023-11-25T04:25:59.978899+0000","last_peered":"2023-11-25T04:25:59.978899+0000","last_clean":"2023-11-25T04:25:59.978899+0000","last_became_active":"2023-11-25T04:25:52.925038+0000","last_became_peered":"2023-11-25T04:25:52.925038+0000","last_unstale":"2023-11-25T04:25:59.978899+0000","last_undegraded":"2023-11-25T04:25:59.978899+0000","last_fullsized":"2023-11-25T04:25:59.978899+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:37:14.620534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":209500,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":139,"num_write_kb":351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":209500,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1a","version":"48'4","reported_seq":73,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978665+0000","last_change":"2023-11-25T04:22:46.034340+0000","last_active":"2023-11-25T04:25:59.978665+0000","last_peered":"2023-11-25T04:25:59.978665+0000","last_clean":"2023-11-25T04:25:59.978665+0000","last_became_active":"2023-11-25T04:22:46.033944+0000","last_became_peered":"2023-11-25T04:22:46.033944+0000","last_unstale":"2023-11-25T04:25:59.978665+0000","last_undegraded":"2023-11-25T04:25:59.978665+0000","last_fullsized":"2023-11-25T04:25:59.978665+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:59:30.291004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"54'163","reported_seq":231,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978741+0000","last_change":"2023-11-25T04:22:47.039547+0000","last_active":"2023-11-25T04:25:59.978741+0000","last_peered":"2023-11-25T04:25:59.978741+0000","last_clean":"2023-11-25T04:25:59.978741+0000","last_became_active":"2023-11-25T04:22:47.039218+0000","last_became_peered":"2023-11-25T04:22:47.039218+0000","last_unstale":"2023-11-25T04:25:59.978741+0000","last_undegraded":"2023-11-25T04:25:59.978741+0000","last_fullsized":"2023-11-25T04:25:59.978741+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:44:48.553972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":165883,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"45'9","reported_seq":94,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978700+0000","last_change":"2023-11-25T04:25:47.023365+0000","last_active":"2023-11-25T04:25:59.978700+0000","last_peered":"2023-11-25T04:25:59.978700+0000","last_clean":"2023-11-25T04:25:59.978700+0000","last_became_active":"2023-11-25T04:25:46.999998+0000","last_became_peered":"2023-11-25T04:25:46.999998+0000","last_unstale":"2023-11-25T04:25:59.978700+0000","last_undegraded":"2023-11-25T04:25:59.978700+0000","last_fullsized":"2023-11-25T04:25:59.978700+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:53:34.807075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":4101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4194304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"54'149","reported_seq":217,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978573+0000","last_change":"2023-11-25T04:25:50.923041+0000","last_active":"2023-11-25T04:25:59.978573+0000","last_peered":"2023-11-25T04:25:59.978573+0000","last_clean":"2023-11-25T04:25:59.978573+0000","last_became_active":"2023-11-25T04:25:50.922164+0000","last_became_peered":"2023-11-25T04:25:50.922164+0000","last_unstale":"2023-11-25T04:25:59.978573+0000","last_undegraded":"2023-11-25T04:25:59.978573+0000","last_fullsized":"2023-11-25T04:25:59.978573+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:31:10.744814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":179506,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":155,"num_write_kb":317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":179506,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"18'2","reported_seq":37,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978409+0000","last_change":"2023-11-25T04:25:46.996675+0000","last_active":"2023-11-25T04:25:59.978409+0000","last_peered":"2023-11-25T04:25:59.978409+0000","last_clean":"2023-11-25T04:25:59.978409+0000","last_became_active":"2023-11-25T04:25:46.995160+0000","last_became_peered":"2023-11-25T04:25:46.995160+0000","last_unstale":"2023-11-25T04:25:59.978409+0000","last_undegraded":"2023-11-25T04:25:59.978409+0000","last_fullsized":"2023-11-25T04:25:59.978409+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:59:39.131605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1a","version":"54'162","reported_seq":226,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978332+0000","last_change":"2023-11-25T04:25:54.939182+0000","last_active":"2023-11-25T04:25:59.978332+0000","last_peered":"2023-11-25T04:25:59.978332+0000","last_clean":"2023-11-25T04:25:59.978332+0000","last_became_active":"2023-11-25T04:25:54.937821+0000","last_became_peered":"2023-11-25T04:25:54.937821+0000","last_unstale":"2023-11-25T04:25:59.978332+0000","last_undegraded":"2023-11-25T04:25:59.978332+0000","last_fullsized":"2023-11-25T04:25:59.978332+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:05:34.575966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":134672,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":169,"num_write_kb":311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":126291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"48'59","reported_seq":146,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979001+0000","last_change":"2023-11-25T04:25:47.077194+0000","last_active":"2023-11-25T04:25:59.979001+0000","last_peered":"2023-11-25T04:25:59.979001+0000","last_clean":"2023-11-25T04:25:59.979001+0000","last_became_active":"2023-11-25T04:25:47.000722+0000","last_became_peered":"2023-11-25T04:25:47.000722+0000","last_unstale":"2023-11-25T04:25:59.979001+0000","last_undegraded":"2023-11-25T04:25:59.979001+0000","last_fullsized":"2023-11-25T04:25:59.979001+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:55:01.351319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8388608,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"54'163","reported_seq":231,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978500+0000","last_change":"2023-11-25T04:22:47.036764+0000","last_active":"2023-11-25T04:25:59.978500+0000","last_peered":"2023-11-25T04:25:59.978500+0000","last_clean":"2023-11-25T04:25:59.978500+0000","last_became_active":"2023-11-25T04:22:47.036406+0000","last_became_peered":"2023-11-25T04:22:47.036406+0000","last_unstale":"2023-11-25T04:25:59.978500+0000","last_undegraded":"2023-11-25T04:25:59.978500+0000","last_fullsized":"2023-11-25T04:25:59.978500+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:09:55.357642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":527891,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":163,"num_write_kb":687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1e","version":"46'2","reported_seq":71,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978868+0000","last_change":"2023-11-25T04:22:46.035659+0000","last_active":"2023-11-25T04:25:59.978868+0000","last_peered":"2023-11-25T04:25:59.978868+0000","last_clean":"2023-11-25T04:25:59.978868+0000","last_became_active":"2023-11-25T04:22:46.035113+0000","last_became_peered":"2023-11-25T04:22:46.035113+0000","last_unstale":"2023-11-25T04:25:59.978868+0000","last_undegraded":"2023-11-25T04:25:59.978868+0000","last_fullsized":"2023-11-25T04:25:59.978868+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:26:25.338117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.18","version":"54'195","reported_seq":259,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978263+0000","last_change":"2023-11-25T04:25:54.938573+0000","last_active":"2023-11-25T04:25:59.978263+0000","last_peered":"2023-11-25T04:25:59.978263+0000","last_clean":"2023-11-25T04:25:59.978263+0000","last_became_active":"2023-11-25T04:25:54.937444+0000","last_became_peered":"2023-11-25T04:25:54.937444+0000","last_unstale":"2023-11-25T04:25:59.978263+0000","last_undegraded":"2023-11-25T04:25:59.978263+0000","last_fullsized":"2023-11-25T04:25:59.978263+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:20:15.814691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":128641,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":185,"num_write_kb":408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":122,"num_bytes_recovered":128641,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"50'17","reported_seq":102,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978969+0000","last_change":"2023-11-25T04:25:47.044228+0000","last_active":"2023-11-25T04:25:59.978969+0000","last_peered":"2023-11-25T04:25:59.978969+0000","last_clean":"2023-11-25T04:25:59.978969+0000","last_became_active":"2023-11-25T04:25:46.998870+0000","last_became_peered":"2023-11-25T04:25:46.998870+0000","last_unstale":"2023-11-25T04:25:59.978969+0000","last_undegraded":"2023-11-25T04:25:59.978969+0000","last_fullsized":"2023-11-25T04:25:59.978969+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:08:12.601941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":4103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4194304,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":26330855,"num_objects":3447,"num_object_clones":0,"num_object_copies":6894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10423,"num_write_kb":38550,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4189,"num_bytes_recovered":12749181,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":72376320,"data_stored":52324046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10246,"ondisk_log_size":10246,"up":128,"acting":128,"num_store_stats":3},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":40885548,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":35,"num_read_kb":334,"num_write":435,"num_write_kb":40462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":32491439,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":81833984,"data_stored":81771096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5548,"internal_metadata":0},"log_size":435,"ondisk_log_size":435,"up":64,"acting":64,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738415,"num_pgs":72,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73208,"kb_used_data":46188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94298632,"statfs":{"total":96636764160,"available":96561799168,"internally_reserved":0,"allocated":47296512,"data_stored":39787071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":1,"last update":"Sat Nov 25 04:25:40 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.433,"15min":0.44500000000000001},"min":{"1min":0.28899999999999998,"5min":0.216,"15min":0.216},"max":{"1min":0.60499999999999998,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.439,"15min":0.45100000000000001},"min":{"1min":0.25600000000000001,"5min":0.184,"15min":0.184},"max":{"1min":0.63400000000000001,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.63}]}]},{"osd":1,"up_from":8,"seq":34359738415,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86880,"kb_used_data":55700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94284960,"statfs":{"total":96636764160,"available":96547799040,"internally_reserved":0,"allocated":57036800,"data_stored":50261554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":2,"last update":"Sat Nov 25 04:25:42 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.40600000000000003,"15min":0.41099999999999998},"min":{"1min":0.29099999999999998,"5min":0.182,"15min":0.182},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.40200000000000002,"15min":0.40600000000000003},"min":{"1min":0.312,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.39000000000000001}]}]},{"osd":0,"up_from":60,"seq":257698037766,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87624,"kb_used_data":51764,"kb_used_omap":6,"kb_used_meta":35833,"kb_avail":94284216,"statfs":{"total":96636764160,"available":96547037184,"internally_reserved":0,"allocated":53006336,"data_stored":46141107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7143,"internal_metadata":36693017},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32530432,"data_stored":32496940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5548,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":29380608,"data_stored":29360982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":19922944,"data_stored":19913174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":19234816,"data_stored":12749181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":26415104,"data_stored":20005586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":26726400,"data_stored":19569279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-25T04:26:06.548 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-25T04:26:06.549 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-25T04:26:06.885 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:26:06.887 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2023-11-25T04:26:06.896 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":179,"stamp":"2023-11-25T04:26:05.550158+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":67806771,"num_objects":3481,"num_object_clones":0,"num_object_copies":6962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3481,"num_whiteouts":0,"num_read":171,"num_read_kb":452,"num_write":11061,"num_write_kb":82472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4201,"num_bytes_recovered":45240620,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10838,"ondisk_log_size":10838,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":247712,"kb_used_data":153652,"kb_used_omap":10,"kb_used_meta":94005,"kb_avail":282867808,"statfs":{"total":289910292480,"available":289656635392,"internally_reserved":0,"allocated":157339648,"data_stored":136189732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10333,"internal_metadata":96262051},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.586462"},"pg_stats":[{"pgid":"4.3f","version":"54'166","reported_seq":234,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979173+0000","last_change":"2023-11-25T04:22:47.039847+0000","last_active":"2023-11-25T04:25:59.979173+0000","last_peered":"2023-11-25T04:25:59.979173+0000","last_clean":"2023-11-25T04:25:59.979173+0000","last_became_active":"2023-11-25T04:22:47.039490+0000","last_became_peered":"2023-11-25T04:22:47.039490+0000","last_unstale":"2023-11-25T04:25:59.979173+0000","last_undegraded":"2023-11-25T04:25:59.979173+0000","last_fullsized":"2023-11-25T04:25:59.979173+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:25:06.132517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":577866,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":174,"num_write_kb":756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3e","version":"54'212","reported_seq":280,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978984+0000","last_change":"2023-11-25T04:22:47.040836+0000","last_active":"2023-11-25T04:25:59.978984+0000","last_peered":"2023-11-25T04:25:59.978984+0000","last_clean":"2023-11-25T04:25:59.978984+0000","last_became_active":"2023-11-25T04:22:47.040511+0000","last_became_peered":"2023-11-25T04:22:47.040511+0000","last_unstale":"2023-11-25T04:25:59.978984+0000","last_undegraded":"2023-11-25T04:25:59.978984+0000","last_fullsized":"2023-11-25T04:25:59.978984+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:15:39.511043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":461579,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":236,"num_write_kb":841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3d","version":"54'167","reported_seq":233,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978931+0000","last_change":"2023-11-25T04:25:52.924678+0000","last_active":"2023-11-25T04:25:59.978931+0000","last_peered":"2023-11-25T04:25:59.978931+0000","last_clean":"2023-11-25T04:25:59.978931+0000","last_became_active":"2023-11-25T04:25:52.923459+0000","last_became_peered":"2023-11-25T04:25:52.923459+0000","last_unstale":"2023-11-25T04:25:59.978931+0000","last_undegraded":"2023-11-25T04:25:59.978931+0000","last_fullsized":"2023-11-25T04:25:59.978931+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:24:06.950438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":145844,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":175,"num_write_kb":312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":138765,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"54'124","reported_seq":191,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978386+0000","last_change":"2023-11-25T04:25:51.922843+0000","last_active":"2023-11-25T04:25:59.978386+0000","last_peered":"2023-11-25T04:25:59.978386+0000","last_clean":"2023-11-25T04:25:59.978386+0000","last_became_active":"2023-11-25T04:25:51.921633+0000","last_became_peered":"2023-11-25T04:25:51.921633+0000","last_unstale":"2023-11-25T04:25:59.978386+0000","last_undegraded":"2023-11-25T04:25:59.978386+0000","last_fullsized":"2023-11-25T04:25:59.978386+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:05:31.332951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61303,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119,"num_write_kb":161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":61303,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"54'169","reported_seq":237,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978640+0000","last_change":"2023-11-25T04:22:47.039499+0000","last_active":"2023-11-25T04:25:59.978640+0000","last_peered":"2023-11-25T04:25:59.978640+0000","last_clean":"2023-11-25T04:25:59.978640+0000","last_became_active":"2023-11-25T04:22:47.039165+0000","last_became_peered":"2023-11-25T04:22:47.039165+0000","last_unstale":"2023-11-25T04:25:59.978640+0000","last_undegraded":"2023-11-25T04:25:59.978640+0000","last_fullsized":"2023-11-25T04:25:59.978640+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:57:28.855151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":118227,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":185,"num_write_kb":351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3a","version":"54'169","reported_seq":234,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978863+0000","last_change":"2023-11-25T04:25:53.933198+0000","last_active":"2023-11-25T04:25:59.978863+0000","last_peered":"2023-11-25T04:25:59.978863+0000","last_clean":"2023-11-25T04:25:59.978863+0000","last_became_active":"2023-11-25T04:25:53.931982+0000","last_became_peered":"2023-11-25T04:25:53.931982+0000","last_unstale":"2023-11-25T04:25:59.978863+0000","last_undegraded":"2023-11-25T04:25:59.978863+0000","last_fullsized":"2023-11-25T04:25:59.978863+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:28:40.449148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100304,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":161,"num_write_kb":424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":100304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"54'168","reported_seq":237,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978380+0000","last_change":"2023-11-25T04:25:49.921033+0000","last_active":"2023-11-25T04:25:59.978380+0000","last_peered":"2023-11-25T04:25:59.978380+0000","last_clean":"2023-11-25T04:25:59.978380+0000","last_became_active":"2023-11-25T04:25:49.919354+0000","last_became_peered":"2023-11-25T04:25:49.919354+0000","last_unstale":"2023-11-25T04:25:59.978380+0000","last_undegraded":"2023-11-25T04:25:59.978380+0000","last_fullsized":"2023-11-25T04:25:59.978380+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:36:13.489397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":124558,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":167,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":111,"num_bytes_recovered":124533,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.38","version":"54'152","reported_seq":221,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978153+0000","last_change":"2023-11-25T04:25:49.920421+0000","last_active":"2023-11-25T04:25:59.978153+0000","last_peered":"2023-11-25T04:25:59.978153+0000","last_clean":"2023-11-25T04:25:59.978153+0000","last_became_active":"2023-11-25T04:25:49.919236+0000","last_became_peered":"2023-11-25T04:25:49.919236+0000","last_unstale":"2023-11-25T04:25:59.978153+0000","last_undegraded":"2023-11-25T04:25:59.978153+0000","last_fullsized":"2023-11-25T04:25:59.978153+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:32:41.956579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":120147,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":120121,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.37","version":"54'167","reported_seq":235,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978818+0000","last_change":"2023-11-25T04:22:47.038397+0000","last_active":"2023-11-25T04:25:59.978818+0000","last_peered":"2023-11-25T04:25:59.978818+0000","last_clean":"2023-11-25T04:25:59.978818+0000","last_became_active":"2023-11-25T04:22:47.037328+0000","last_became_peered":"2023-11-25T04:22:47.037328+0000","last_unstale":"2023-11-25T04:25:59.978818+0000","last_undegraded":"2023-11-25T04:25:59.978818+0000","last_fullsized":"2023-11-25T04:25:59.978818+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:56:59.689578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":174147,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":184,"num_write_kb":353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.36","version":"54'149","reported_seq":217,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979047+0000","last_change":"2023-11-25T04:22:47.040182+0000","last_active":"2023-11-25T04:25:59.979047+0000","last_peered":"2023-11-25T04:25:59.979047+0000","last_clean":"2023-11-25T04:25:59.979047+0000","last_became_active":"2023-11-25T04:22:47.039892+0000","last_became_peered":"2023-11-25T04:22:47.039892+0000","last_unstale":"2023-11-25T04:25:59.979047+0000","last_undegraded":"2023-11-25T04:25:59.979047+0000","last_fullsized":"2023-11-25T04:25:59.979047+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:38:09.176582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":233330,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"54'156","reported_seq":223,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979021+0000","last_change":"2023-11-25T04:25:51.923087+0000","last_active":"2023-11-25T04:25:59.979021+0000","last_peered":"2023-11-25T04:25:59.979021+0000","last_clean":"2023-11-25T04:25:59.979021+0000","last_became_active":"2023-11-25T04:25:51.921800+0000","last_became_peered":"2023-11-25T04:25:51.921800+0000","last_unstale":"2023-11-25T04:25:59.979021+0000","last_undegraded":"2023-11-25T04:25:59.979021+0000","last_fullsized":"2023-11-25T04:25:59.979021+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:29:47.766517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":302550,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":163,"num_write_kb":431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":300323,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.34","version":"54'136","reported_seq":400,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978706+0000","last_change":"2023-11-25T04:25:50.922843+0000","last_active":"2023-11-25T04:25:59.978706+0000","last_peered":"2023-11-25T04:25:59.978706+0000","last_clean":"2023-11-25T04:25:59.978706+0000","last_became_active":"2023-11-25T04:25:50.922309+0000","last_became_peered":"2023-11-25T04:25:50.922309+0000","last_unstale":"2023-11-25T04:25:59.978706+0000","last_undegraded":"2023-11-25T04:25:59.978706+0000","last_fullsized":"2023-11-25T04:25:59.978706+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:38:33.939562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":93053,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":131,"num_write_kb":248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":89,"num_bytes_recovered":93053,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"54'134","reported_seq":202,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978529+0000","last_change":"2023-11-25T04:22:47.040388+0000","last_active":"2023-11-25T04:25:59.978529+0000","last_peered":"2023-11-25T04:25:59.978529+0000","last_clean":"2023-11-25T04:25:59.978529+0000","last_became_active":"2023-11-25T04:22:47.040129+0000","last_became_peered":"2023-11-25T04:22:47.040129+0000","last_unstale":"2023-11-25T04:25:59.978529+0000","last_undegraded":"2023-11-25T04:25:59.978529+0000","last_fullsized":"2023-11-25T04:25:59.978529+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:31:54.857270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":149524,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":139,"num_write_kb":252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.32","version":"54'174","reported_seq":239,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978762+0000","last_change":"2023-11-25T04:25:53.934526+0000","last_active":"2023-11-25T04:25:59.978762+0000","last_peered":"2023-11-25T04:25:59.978762+0000","last_clean":"2023-11-25T04:25:59.978762+0000","last_became_active":"2023-11-25T04:25:53.933694+0000","last_became_peered":"2023-11-25T04:25:53.933694+0000","last_unstale":"2023-11-25T04:25:59.978762+0000","last_undegraded":"2023-11-25T04:25:59.978762+0000","last_fullsized":"2023-11-25T04:25:59.978762+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:01:16.858976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":95797,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":181,"num_write_kb":547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":103,"num_bytes_recovered":95797,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.31","version":"54'156","reported_seq":225,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978209+0000","last_change":"2023-11-25T04:25:49.922396+0000","last_active":"2023-11-25T04:25:59.978209+0000","last_peered":"2023-11-25T04:25:59.978209+0000","last_clean":"2023-11-25T04:25:59.978209+0000","last_became_active":"2023-11-25T04:25:49.921129+0000","last_became_peered":"2023-11-25T04:25:49.921129+0000","last_unstale":"2023-11-25T04:25:59.978209+0000","last_undegraded":"2023-11-25T04:25:59.978209+0000","last_fullsized":"2023-11-25T04:25:59.978209+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:57:02.387207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":163190,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":169,"num_write_kb":336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":154576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"54'174","reported_seq":239,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978986+0000","last_change":"2023-11-25T04:25:53.934614+0000","last_active":"2023-11-25T04:25:59.978986+0000","last_peered":"2023-11-25T04:25:59.978986+0000","last_clean":"2023-11-25T04:25:59.978986+0000","last_became_active":"2023-11-25T04:25:53.933395+0000","last_became_peered":"2023-11-25T04:25:53.933395+0000","last_unstale":"2023-11-25T04:25:59.978986+0000","last_undegraded":"2023-11-25T04:25:59.978986+0000","last_fullsized":"2023-11-25T04:25:59.978986+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:53:09.827809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1423587,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":182,"num_write_kb":1626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":1414508,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2f","version":"54'152","reported_seq":220,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979084+0000","last_change":"2023-11-25T04:22:47.039389+0000","last_active":"2023-11-25T04:25:59.979084+0000","last_peered":"2023-11-25T04:25:59.979084+0000","last_clean":"2023-11-25T04:25:59.979084+0000","last_became_active":"2023-11-25T04:22:47.039123+0000","last_became_peered":"2023-11-25T04:22:47.039123+0000","last_unstale":"2023-11-25T04:25:59.979084+0000","last_undegraded":"2023-11-25T04:25:59.979084+0000","last_fullsized":"2023-11-25T04:25:59.979084+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:09:11.570561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":122752,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":143,"num_write_kb":435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"54'148","reported_seq":216,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979169+0000","last_change":"2023-11-25T04:22:47.036946+0000","last_active":"2023-11-25T04:25:59.979169+0000","last_peered":"2023-11-25T04:25:59.979169+0000","last_clean":"2023-11-25T04:25:59.979169+0000","last_became_active":"2023-11-25T04:22:47.036411+0000","last_became_peered":"2023-11-25T04:22:47.036411+0000","last_unstale":"2023-11-25T04:25:59.979169+0000","last_undegraded":"2023-11-25T04:25:59.979169+0000","last_fullsized":"2023-11-25T04:25:59.979169+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:16:29.085050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4770899,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":140,"num_write_kb":4870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2d","version":"54'143","reported_seq":211,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978472+0000","last_change":"2023-11-25T04:22:47.038983+0000","last_active":"2023-11-25T04:25:59.978472+0000","last_peered":"2023-11-25T04:25:59.978472+0000","last_clean":"2023-11-25T04:25:59.978472+0000","last_became_active":"2023-11-25T04:22:47.038477+0000","last_became_peered":"2023-11-25T04:22:47.038477+0000","last_unstale":"2023-11-25T04:25:59.978472+0000","last_undegraded":"2023-11-25T04:25:59.978472+0000","last_fullsized":"2023-11-25T04:25:59.978472+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:30:04.568790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":115975,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"54'167","reported_seq":233,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978615+0000","last_change":"2023-11-25T04:25:52.923640+0000","last_active":"2023-11-25T04:25:59.978615+0000","last_peered":"2023-11-25T04:25:59.978615+0000","last_clean":"2023-11-25T04:25:59.978615+0000","last_became_active":"2023-11-25T04:25:52.922367+0000","last_became_peered":"2023-11-25T04:25:52.922367+0000","last_unstale":"2023-11-25T04:25:59.978615+0000","last_undegraded":"2023-11-25T04:25:59.978615+0000","last_fullsized":"2023-11-25T04:25:59.978615+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:08:24.794280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":747090,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":211,"num_write_kb":985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":732276,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"54'156","reported_seq":222,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978632+0000","last_change":"2023-11-25T04:25:52.924457+0000","last_active":"2023-11-25T04:25:59.978632+0000","last_peered":"2023-11-25T04:25:59.978632+0000","last_clean":"2023-11-25T04:25:59.978632+0000","last_became_active":"2023-11-25T04:25:52.923378+0000","last_became_peered":"2023-11-25T04:25:52.923378+0000","last_unstale":"2023-11-25T04:25:59.978632+0000","last_undegraded":"2023-11-25T04:25:59.978632+0000","last_fullsized":"2023-11-25T04:25:59.978632+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:34:30.942779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69580,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":142,"num_write_kb":315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":98,"num_bytes_recovered":69580,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2a","version":"54'171","reported_seq":467,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978347+0000","last_change":"2023-11-25T04:25:52.925532+0000","last_active":"2023-11-25T04:25:59.978347+0000","last_peered":"2023-11-25T04:25:59.978347+0000","last_clean":"2023-11-25T04:25:59.978347+0000","last_became_active":"2023-11-25T04:25:52.924844+0000","last_became_peered":"2023-11-25T04:25:52.924844+0000","last_unstale":"2023-11-25T04:25:59.978347+0000","last_undegraded":"2023-11-25T04:25:59.978347+0000","last_fullsized":"2023-11-25T04:25:59.978347+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:11:30.124326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":115997,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":162,"num_write_kb":261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":115997,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.29","version":"54'152","reported_seq":438,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.977803+0000","last_change":"2023-11-25T04:25:49.920367+0000","last_active":"2023-11-25T04:25:59.977803+0000","last_peered":"2023-11-25T04:25:59.977803+0000","last_clean":"2023-11-25T04:25:59.977803+0000","last_became_active":"2023-11-25T04:25:49.919886+0000","last_became_peered":"2023-11-25T04:25:49.919886+0000","last_unstale":"2023-11-25T04:25:59.977803+0000","last_undegraded":"2023-11-25T04:25:59.977803+0000","last_fullsized":"2023-11-25T04:25:59.977803+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:04:58.273982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":212878,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":212878,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"54'169","reported_seq":237,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978222+0000","last_change":"2023-11-25T04:22:47.036805+0000","last_active":"2023-11-25T04:25:59.978222+0000","last_peered":"2023-11-25T04:25:59.978222+0000","last_clean":"2023-11-25T04:25:59.978222+0000","last_became_active":"2023-11-25T04:22:47.036135+0000","last_became_peered":"2023-11-25T04:22:47.036135+0000","last_unstale":"2023-11-25T04:25:59.978222+0000","last_undegraded":"2023-11-25T04:25:59.978222+0000","last_fullsized":"2023-11-25T04:25:59.978222+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:25:21.730992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":102957,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":172,"num_write_kb":610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"54'167","reported_seq":465,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979220+0000","last_change":"2023-11-25T04:25:51.922925+0000","last_active":"2023-11-25T04:25:59.979220+0000","last_peered":"2023-11-25T04:25:59.979220+0000","last_clean":"2023-11-25T04:25:59.979220+0000","last_became_active":"2023-11-25T04:25:51.922364+0000","last_became_peered":"2023-11-25T04:25:51.922364+0000","last_unstale":"2023-11-25T04:25:59.979220+0000","last_undegraded":"2023-11-25T04:25:59.979220+0000","last_fullsized":"2023-11-25T04:25:59.979220+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:09:23.741871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":182683,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":171,"num_write_kb":791,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":182683,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"54'187","reported_seq":505,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978723+0000","last_change":"2023-11-25T04:25:51.922529+0000","last_active":"2023-11-25T04:25:59.978723+0000","last_peered":"2023-11-25T04:25:59.978723+0000","last_clean":"2023-11-25T04:25:59.978723+0000","last_became_active":"2023-11-25T04:25:51.922064+0000","last_became_peered":"2023-11-25T04:25:51.922064+0000","last_unstale":"2023-11-25T04:25:59.978723+0000","last_undegraded":"2023-11-25T04:25:59.978723+0000","last_fullsized":"2023-11-25T04:25:59.978723+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":187,"log_dups_size":0,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:01:39.413986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":86311,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":171,"num_write_kb":274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":116,"num_bytes_recovered":86311,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.25","version":"54'149","reported_seq":215,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978829+0000","last_change":"2023-11-25T04:25:52.925505+0000","last_active":"2023-11-25T04:25:59.978829+0000","last_peered":"2023-11-25T04:25:59.978829+0000","last_clean":"2023-11-25T04:25:59.978829+0000","last_became_active":"2023-11-25T04:25:52.924795+0000","last_became_peered":"2023-11-25T04:25:52.924795+0000","last_unstale":"2023-11-25T04:25:59.978829+0000","last_undegraded":"2023-11-25T04:25:59.978829+0000","last_fullsized":"2023-11-25T04:25:59.978829+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:30:46.532141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":128288,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":167,"num_write_kb":267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":120265,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.24","version":"54'151","reported_seq":427,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978903+0000","last_change":"2023-11-25T04:25:50.922751+0000","last_active":"2023-11-25T04:25:59.978903+0000","last_peered":"2023-11-25T04:25:59.978903+0000","last_clean":"2023-11-25T04:25:59.978903+0000","last_became_active":"2023-11-25T04:25:50.922216+0000","last_became_peered":"2023-11-25T04:25:50.922216+0000","last_unstale":"2023-11-25T04:25:59.978903+0000","last_undegraded":"2023-11-25T04:25:59.978903+0000","last_fullsized":"2023-11-25T04:25:59.978903+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:40:16.255665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":107507,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":148,"num_write_kb":288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":107494,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.23","version":"54'155","reported_seq":223,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978207+0000","last_change":"2023-11-25T04:22:47.037668+0000","last_active":"2023-11-25T04:25:59.978207+0000","last_peered":"2023-11-25T04:25:59.978207+0000","last_clean":"2023-11-25T04:25:59.978207+0000","last_became_active":"2023-11-25T04:22:47.036823+0000","last_became_peered":"2023-11-25T04:22:47.036823+0000","last_unstale":"2023-11-25T04:25:59.978207+0000","last_undegraded":"2023-11-25T04:25:59.978207+0000","last_fullsized":"2023-11-25T04:25:59.978207+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:02:07.088241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4304582,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":152,"num_write_kb":4331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.22","version":"54'189","reported_seq":517,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978625+0000","last_change":"2023-11-25T04:25:54.940559+0000","last_active":"2023-11-25T04:25:59.978625+0000","last_peered":"2023-11-25T04:25:59.978625+0000","last_clean":"2023-11-25T04:25:59.978625+0000","last_became_active":"2023-11-25T04:25:54.939584+0000","last_became_peered":"2023-11-25T04:25:54.939584+0000","last_unstale":"2023-11-25T04:25:59.978625+0000","last_undegraded":"2023-11-25T04:25:59.978625+0000","last_fullsized":"2023-11-25T04:25:59.978625+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:01:39.413986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":129145,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":185,"num_write_kb":432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":121,"num_bytes_recovered":129107,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"54'201","reported_seq":269,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.977881+0000","last_change":"2023-11-25T04:22:47.037786+0000","last_active":"2023-11-25T04:25:59.977881+0000","last_peered":"2023-11-25T04:25:59.977881+0000","last_clean":"2023-11-25T04:25:59.977881+0000","last_became_active":"2023-11-25T04:22:47.037206+0000","last_became_peered":"2023-11-25T04:22:47.037206+0000","last_unstale":"2023-11-25T04:25:59.977881+0000","last_undegraded":"2023-11-25T04:25:59.977881+0000","last_fullsized":"2023-11-25T04:25:59.977881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:11:40.532140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":215311,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":194,"num_write_kb":381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"54'180","reported_seq":245,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.977884+0000","last_change":"2023-11-25T04:25:53.933176+0000","last_active":"2023-11-25T04:25:59.977884+0000","last_peered":"2023-11-25T04:25:59.977884+0000","last_clean":"2023-11-25T04:25:59.977884+0000","last_became_active":"2023-11-25T04:25:53.931937+0000","last_became_peered":"2023-11-25T04:25:53.931937+0000","last_unstale":"2023-11-25T04:25:59.977884+0000","last_undegraded":"2023-11-25T04:25:59.977884+0000","last_fullsized":"2023-11-25T04:25:59.977884+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:41:10.697064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104603,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":172,"num_write_kb":267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":104603,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"54'137","reported_seq":205,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978811+0000","last_change":"2023-11-25T04:22:47.036689+0000","last_active":"2023-11-25T04:25:59.978811+0000","last_peered":"2023-11-25T04:25:59.978811+0000","last_clean":"2023-11-25T04:25:59.978811+0000","last_became_active":"2023-11-25T04:22:47.036107+0000","last_became_peered":"2023-11-25T04:22:47.036107+0000","last_unstale":"2023-11-25T04:25:59.978811+0000","last_undegraded":"2023-11-25T04:25:59.978811+0000","last_fullsized":"2023-11-25T04:25:59.978811+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:02:07.088241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":183586,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":127,"num_write_kb":284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"58'114","reported_seq":191,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978227+0000","last_change":"2023-11-25T04:25:49.920320+0000","last_active":"2023-11-25T04:25:59.978227+0000","last_peered":"2023-11-25T04:25:59.978227+0000","last_clean":"2023-11-25T04:25:59.978227+0000","last_became_active":"2023-11-25T04:25:49.920010+0000","last_became_peered":"2023-11-25T04:25:49.920010+0000","last_unstale":"2023-11-25T04:25:59.978227+0000","last_undegraded":"2023-11-25T04:25:59.978227+0000","last_fullsized":"2023-11-25T04:25:59.978227+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:48:58.746789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2816216,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":113,"num_write_kb":2816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":2816182,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"54'145","reported_seq":213,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978995+0000","last_change":"2023-11-25T04:22:47.037888+0000","last_active":"2023-11-25T04:25:59.978995+0000","last_peered":"2023-11-25T04:25:59.978995+0000","last_clean":"2023-11-25T04:25:59.978995+0000","last_became_active":"2023-11-25T04:22:47.037158+0000","last_became_peered":"2023-11-25T04:22:47.037158+0000","last_unstale":"2023-11-25T04:25:59.978995+0000","last_undegraded":"2023-11-25T04:25:59.978995+0000","last_fullsized":"2023-11-25T04:25:59.978995+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:56:59.689578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":205976,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.12","version":"49'2","reported_seq":82,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978301+0000","last_change":"2023-11-25T04:25:47.000916+0000","last_active":"2023-11-25T04:25:59.978301+0000","last_peered":"2023-11-25T04:25:59.978301+0000","last_clean":"2023-11-25T04:25:59.978301+0000","last_became_active":"2023-11-25T04:25:47.000516+0000","last_became_peered":"2023-11-25T04:25:47.000516+0000","last_unstale":"2023-11-25T04:25:59.978301+0000","last_undegraded":"2023-11-25T04:25:59.978301+0000","last_fullsized":"2023-11-25T04:25:59.978301+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:53:58.873447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"54'175","reported_seq":489,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978855+0000","last_change":"2023-11-25T04:25:52.923694+0000","last_active":"2023-11-25T04:25:59.978855+0000","last_peered":"2023-11-25T04:25:59.978855+0000","last_clean":"2023-11-25T04:25:59.978855+0000","last_became_active":"2023-11-25T04:25:52.923045+0000","last_became_peered":"2023-11-25T04:25:52.923045+0000","last_unstale":"2023-11-25T04:25:59.978855+0000","last_undegraded":"2023-11-25T04:25:59.978855+0000","last_fullsized":"2023-11-25T04:25:59.978855+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:46:12.914433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":171823,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":185,"num_write_kb":340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":166387,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":80,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978355+0000","last_change":"2023-11-25T04:25:46.994324+0000","last_active":"2023-11-25T04:25:59.978355+0000","last_peered":"2023-11-25T04:25:59.978355+0000","last_clean":"2023-11-25T04:25:59.978355+0000","last_became_active":"2023-11-25T04:25:46.993870+0000","last_became_peered":"2023-11-25T04:25:46.993870+0000","last_unstale":"2023-11-25T04:25:59.978355+0000","last_undegraded":"2023-11-25T04:25:59.978355+0000","last_fullsized":"2023-11-25T04:25:59.978355+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:51:09.827080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"54'152","reported_seq":435,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978827+0000","last_change":"2023-11-25T04:25:50.922922+0000","last_active":"2023-11-25T04:25:59.978827+0000","last_peered":"2023-11-25T04:25:59.978827+0000","last_clean":"2023-11-25T04:25:59.978827+0000","last_became_active":"2023-11-25T04:25:50.922511+0000","last_became_peered":"2023-11-25T04:25:50.922511+0000","last_unstale":"2023-11-25T04:25:59.978827+0000","last_undegraded":"2023-11-25T04:25:59.978827+0000","last_fullsized":"2023-11-25T04:25:59.978827+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:37:14.620534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":579704,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":97,"num_bytes_recovered":570044,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.10","version":"18'8","reported_seq":39,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.977878+0000","last_change":"2023-11-25T04:25:47.000499+0000","last_active":"2023-11-25T04:25:59.977878+0000","last_peered":"2023-11-25T04:25:59.977878+0000","last_clean":"2023-11-25T04:25:59.977878+0000","last_became_active":"2023-11-25T04:25:46.998778+0000","last_became_peered":"2023-11-25T04:25:46.998778+0000","last_unstale":"2023-11-25T04:25:59.977878+0000","last_undegraded":"2023-11-25T04:25:59.977878+0000","last_fullsized":"2023-11-25T04:25:59.977878+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:33:09.169689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"49'2","reported_seq":37,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979103+0000","last_change":"2023-11-25T04:25:46.998392+0000","last_active":"2023-11-25T04:25:59.979103+0000","last_peered":"2023-11-25T04:25:59.979103+0000","last_clean":"2023-11-25T04:25:59.979103+0000","last_became_active":"2023-11-25T04:25:46.997108+0000","last_became_peered":"2023-11-25T04:25:46.997108+0000","last_unstale":"2023-11-25T04:25:59.979103+0000","last_undegraded":"2023-11-25T04:25:59.979103+0000","last_fullsized":"2023-11-25T04:25:59.979103+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:02:00.073630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"54'159","reported_seq":443,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978233+0000","last_change":"2023-11-25T04:25:50.922282+0000","last_active":"2023-11-25T04:25:59.978233+0000","last_peered":"2023-11-25T04:25:59.978233+0000","last_clean":"2023-11-25T04:25:59.978233+0000","last_became_active":"2023-11-25T04:25:50.921772+0000","last_became_peered":"2023-11-25T04:25:50.921772+0000","last_unstale":"2023-11-25T04:25:59.978233+0000","last_undegraded":"2023-11-25T04:25:59.978233+0000","last_fullsized":"2023-11-25T04:25:59.978233+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:36:33.451689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112103,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":164,"num_write_kb":240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":99,"num_bytes_recovered":103472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"18'2","reported_seq":82,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979281+0000","last_change":"2023-11-25T04:25:46.994943+0000","last_active":"2023-11-25T04:25:59.979281+0000","last_peered":"2023-11-25T04:25:59.979281+0000","last_clean":"2023-11-25T04:25:59.979281+0000","last_became_active":"2023-11-25T04:25:46.994640+0000","last_became_peered":"2023-11-25T04:25:46.994640+0000","last_unstale":"2023-11-25T04:25:59.979281+0000","last_undegraded":"2023-11-25T04:25:59.979281+0000","last_fullsized":"2023-11-25T04:25:59.979281+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:08:12.601941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"54'173","reported_seq":241,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978244+0000","last_change":"2023-11-25T04:22:47.039685+0000","last_active":"2023-11-25T04:25:59.978244+0000","last_peered":"2023-11-25T04:25:59.978244+0000","last_clean":"2023-11-25T04:25:59.978244+0000","last_became_active":"2023-11-25T04:22:47.039296+0000","last_became_peered":"2023-11-25T04:22:47.039296+0000","last_unstale":"2023-11-25T04:25:59.978244+0000","last_undegraded":"2023-11-25T04:25:59.978244+0000","last_fullsized":"2023-11-25T04:25:59.978244+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:11:40.532140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154533,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":182,"num_write_kb":299,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"18'1","reported_seq":37,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978725+0000","last_change":"2023-11-25T04:25:47.000059+0000","last_active":"2023-11-25T04:25:59.978725+0000","last_peered":"2023-11-25T04:25:59.978725+0000","last_clean":"2023-11-25T04:25:59.978725+0000","last_became_active":"2023-11-25T04:25:46.999604+0000","last_became_peered":"2023-11-25T04:25:46.999604+0000","last_unstale":"2023-11-25T04:25:59.978725+0000","last_undegraded":"2023-11-25T04:25:59.978725+0000","last_fullsized":"2023-11-25T04:25:59.978725+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:14:43.431272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"54'147","reported_seq":211,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978982+0000","last_change":"2023-11-25T04:25:54.939274+0000","last_active":"2023-11-25T04:25:59.978982+0000","last_peered":"2023-11-25T04:25:59.978982+0000","last_clean":"2023-11-25T04:25:59.978982+0000","last_became_active":"2023-11-25T04:25:54.938236+0000","last_became_peered":"2023-11-25T04:25:54.938236+0000","last_unstale":"2023-11-25T04:25:59.978982+0000","last_undegraded":"2023-11-25T04:25:59.978982+0000","last_fullsized":"2023-11-25T04:25:59.978982+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":147,"log_dups_size":0,"ondisk_log_size":147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:11:43.785294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":87524,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":155,"num_write_kb":235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":87524,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978546+0000","last_change":"2023-11-25T04:25:46.997080+0000","last_active":"2023-11-25T04:25:59.978546+0000","last_peered":"2023-11-25T04:25:59.978546+0000","last_clean":"2023-11-25T04:25:59.978546+0000","last_became_active":"2023-11-25T04:25:46.995712+0000","last_became_peered":"2023-11-25T04:25:46.995712+0000","last_unstale":"2023-11-25T04:25:59.978546+0000","last_undegraded":"2023-11-25T04:25:59.978546+0000","last_fullsized":"2023-11-25T04:25:59.978546+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T16:01:00.922843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"54'155","reported_seq":441,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978274+0000","last_change":"2023-11-25T04:25:52.923096+0000","last_active":"2023-11-25T04:25:59.978274+0000","last_peered":"2023-11-25T04:25:59.978274+0000","last_clean":"2023-11-25T04:25:59.978274+0000","last_became_active":"2023-11-25T04:25:52.922549+0000","last_became_peered":"2023-11-25T04:25:52.922549+0000","last_unstale":"2023-11-25T04:25:59.978274+0000","last_undegraded":"2023-11-25T04:25:59.978274+0000","last_fullsized":"2023-11-25T04:25:59.978274+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:52:38.053808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":266919,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":264258,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"45'11","reported_seq":96,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978532+0000","last_change":"2023-11-25T04:25:47.092497+0000","last_active":"2023-11-25T04:25:59.978532+0000","last_peered":"2023-11-25T04:25:59.978532+0000","last_clean":"2023-11-25T04:25:59.978532+0000","last_became_active":"2023-11-25T04:25:47.000687+0000","last_became_peered":"2023-11-25T04:25:47.000687+0000","last_unstale":"2023-11-25T04:25:59.978532+0000","last_undegraded":"2023-11-25T04:25:59.978532+0000","last_fullsized":"2023-11-25T04:25:59.978532+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:47:09.798238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":4101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4194304,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"54'173","reported_seq":241,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978732+0000","last_change":"2023-11-25T04:25:50.923240+0000","last_active":"2023-11-25T04:25:59.978732+0000","last_peered":"2023-11-25T04:25:59.978732+0000","last_clean":"2023-11-25T04:25:59.978732+0000","last_became_active":"2023-11-25T04:25:50.922409+0000","last_became_peered":"2023-11-25T04:25:50.922409+0000","last_unstale":"2023-11-25T04:25:59.978732+0000","last_undegraded":"2023-11-25T04:25:59.978732+0000","last_fullsized":"2023-11-25T04:25:59.978732+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:53:30.920627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":553893,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":169,"num_write_kb":770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":553893,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978079+0000","last_change":"2023-11-25T04:25:46.998818+0000","last_active":"2023-11-25T04:25:59.978079+0000","last_peered":"2023-11-25T04:25:59.978079+0000","last_clean":"2023-11-25T04:25:59.978079+0000","last_became_active":"2023-11-25T04:25:46.997019+0000","last_became_peered":"2023-11-25T04:25:46.997019+0000","last_unstale":"2023-11-25T04:25:59.978079+0000","last_undegraded":"2023-11-25T04:25:59.978079+0000","last_fullsized":"2023-11-25T04:25:59.978079+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:51:47.882117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"54'163","reported_seq":228,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978685+0000","last_change":"2023-11-25T04:25:53.934183+0000","last_active":"2023-11-25T04:25:59.978685+0000","last_peered":"2023-11-25T04:25:59.978685+0000","last_clean":"2023-11-25T04:25:59.978685+0000","last_became_active":"2023-11-25T04:25:53.933471+0000","last_became_peered":"2023-11-25T04:25:53.933471+0000","last_unstale":"2023-11-25T04:25:59.978685+0000","last_undegraded":"2023-11-25T04:25:59.978685+0000","last_fullsized":"2023-11-25T04:25:59.978685+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:57:07.515414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":320867,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":190,"num_write_kb":577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":108,"num_bytes_recovered":311640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":80,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978412+0000","last_change":"2023-11-25T04:25:46.994480+0000","last_active":"2023-11-25T04:25:59.978412+0000","last_peered":"2023-11-25T04:25:59.978412+0000","last_clean":"2023-11-25T04:25:59.978412+0000","last_became_active":"2023-11-25T04:25:46.994101+0000","last_became_peered":"2023-11-25T04:25:46.994101+0000","last_unstale":"2023-11-25T04:25:59.978412+0000","last_undegraded":"2023-11-25T04:25:59.978412+0000","last_fullsized":"2023-11-25T04:25:59.978412+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:53:58.873447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"54'141","reported_seq":210,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979040+0000","last_change":"2023-11-25T04:25:49.921950+0000","last_active":"2023-11-25T04:25:59.979040+0000","last_peered":"2023-11-25T04:25:59.979040+0000","last_clean":"2023-11-25T04:25:59.979040+0000","last_became_active":"2023-11-25T04:25:49.920971+0000","last_became_peered":"2023-11-25T04:25:49.920971+0000","last_unstale":"2023-11-25T04:25:59.979040+0000","last_undegraded":"2023-11-25T04:25:59.979040+0000","last_fullsized":"2023-11-25T04:25:59.979040+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:22:38.004934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3502371,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":3718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":89,"num_bytes_recovered":3497669,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"49'1","reported_seq":70,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978352+0000","last_change":"2023-11-25T04:22:46.034509+0000","last_active":"2023-11-25T04:25:59.978352+0000","last_peered":"2023-11-25T04:25:59.978352+0000","last_clean":"2023-11-25T04:25:59.978352+0000","last_became_active":"2023-11-25T04:22:46.034158+0000","last_became_peered":"2023-11-25T04:22:46.034158+0000","last_unstale":"2023-11-25T04:25:59.978352+0000","last_undegraded":"2023-11-25T04:25:59.978352+0000","last_fullsized":"2023-11-25T04:25:59.978352+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:26:25.338117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"54'141","reported_seq":209,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978912+0000","last_change":"2023-11-25T04:22:47.038800+0000","last_active":"2023-11-25T04:25:59.978912+0000","last_peered":"2023-11-25T04:25:59.978912+0000","last_clean":"2023-11-25T04:25:59.978912+0000","last_became_active":"2023-11-25T04:22:47.038081+0000","last_became_peered":"2023-11-25T04:22:47.038081+0000","last_unstale":"2023-11-25T04:25:59.978912+0000","last_undegraded":"2023-11-25T04:25:59.978912+0000","last_fullsized":"2023-11-25T04:25:59.978912+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:45:40.065146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":174088,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":146,"num_write_kb":385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"12'157","reported_seq":232,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.977771+0000","last_change":"2023-11-25T04:25:47.000504+0000","last_active":"2023-11-25T04:25:59.977771+0000","last_peered":"2023-11-25T04:25:59.977771+0000","last_clean":"2023-11-25T04:25:59.977771+0000","last_became_active":"2023-11-25T04:25:46.999721+0000","last_became_peered":"2023-11-25T04:25:46.999721+0000","last_unstale":"2023-11-25T04:25:59.977771+0000","last_undegraded":"2023-11-25T04:25:59.977771+0000","last_fullsized":"2023-11-25T04:25:59.977771+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:35.919850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:35.919850+0000","last_clean_scrub_stamp":"2023-11-25T04:22:35.919850+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:26:33.607474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978082+0000","last_change":"2023-11-25T04:25:46.998960+0000","last_active":"2023-11-25T04:25:59.978082+0000","last_peered":"2023-11-25T04:25:59.978082+0000","last_clean":"2023-11-25T04:25:59.978082+0000","last_became_active":"2023-11-25T04:25:46.997693+0000","last_became_peered":"2023-11-25T04:25:46.997693+0000","last_unstale":"2023-11-25T04:25:59.978082+0000","last_undegraded":"2023-11-25T04:25:59.978082+0000","last_fullsized":"2023-11-25T04:25:59.978082+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:51:45.180957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"46'1","reported_seq":70,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978331+0000","last_change":"2023-11-25T04:22:46.037044+0000","last_active":"2023-11-25T04:25:59.978331+0000","last_peered":"2023-11-25T04:25:59.978331+0000","last_clean":"2023-11-25T04:25:59.978331+0000","last_became_active":"2023-11-25T04:22:46.035916+0000","last_became_peered":"2023-11-25T04:22:46.035916+0000","last_unstale":"2023-11-25T04:25:59.978331+0000","last_undegraded":"2023-11-25T04:25:59.978331+0000","last_fullsized":"2023-11-25T04:25:59.978331+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:38:20.501102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"54'146","reported_seq":213,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978804+0000","last_change":"2023-11-25T04:25:51.923452+0000","last_active":"2023-11-25T04:25:59.978804+0000","last_peered":"2023-11-25T04:25:59.978804+0000","last_clean":"2023-11-25T04:25:59.978804+0000","last_became_active":"2023-11-25T04:25:51.922490+0000","last_became_peered":"2023-11-25T04:25:51.922490+0000","last_unstale":"2023-11-25T04:25:59.978804+0000","last_undegraded":"2023-11-25T04:25:59.978804+0000","last_fullsized":"2023-11-25T04:25:59.978804+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:40:11.001062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":502154,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":170,"num_write_kb":687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":99,"num_bytes_recovered":502101,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978713+0000","last_change":"2023-11-25T04:25:47.000763+0000","last_active":"2023-11-25T04:25:59.978713+0000","last_peered":"2023-11-25T04:25:59.978713+0000","last_clean":"2023-11-25T04:25:59.978713+0000","last_became_active":"2023-11-25T04:25:47.000033+0000","last_became_peered":"2023-11-25T04:25:47.000033+0000","last_unstale":"2023-11-25T04:25:59.978713+0000","last_undegraded":"2023-11-25T04:25:59.978713+0000","last_fullsized":"2023-11-25T04:25:59.978713+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:53:59.131478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"50'37","reported_seq":156,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978116+0000","last_change":"2023-11-25T04:25:47.002009+0000","last_active":"2023-11-25T04:25:59.978116+0000","last_peered":"2023-11-25T04:25:59.978116+0000","last_clean":"2023-11-25T04:25:59.978116+0000","last_became_active":"2023-11-25T04:25:46.995637+0000","last_became_peered":"2023-11-25T04:25:46.995637+0000","last_unstale":"2023-11-25T04:25:59.978116+0000","last_undegraded":"2023-11-25T04:25:59.978116+0000","last_fullsized":"2023-11-25T04:25:59.978116+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:55:01.351319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":319160,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":35,"num_read_kb":334,"num_write":35,"num_write_kb":334,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":319160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"54'158","reported_seq":226,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979092+0000","last_change":"2023-11-25T04:22:47.039583+0000","last_active":"2023-11-25T04:25:59.979092+0000","last_peered":"2023-11-25T04:25:59.979092+0000","last_clean":"2023-11-25T04:25:59.979092+0000","last_became_active":"2023-11-25T04:22:47.039197+0000","last_became_peered":"2023-11-25T04:22:47.039197+0000","last_unstale":"2023-11-25T04:25:59.979092+0000","last_undegraded":"2023-11-25T04:25:59.979092+0000","last_fullsized":"2023-11-25T04:25:59.979092+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:37:30.959147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":169303,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":189,"num_write_kb":333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":72,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978222+0000","last_change":"2023-11-25T04:22:42.000382+0000","last_active":"2023-11-25T04:25:59.978222+0000","last_peered":"2023-11-25T04:25:59.978222+0000","last_clean":"2023-11-25T04:25:59.978222+0000","last_became_active":"2023-11-25T04:22:42.000041+0000","last_became_peered":"2023-11-25T04:22:42.000041+0000","last_unstale":"2023-11-25T04:25:59.978222+0000","last_undegraded":"2023-11-25T04:25:59.978222+0000","last_fullsized":"2023-11-25T04:25:59.978222+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:08:19.655864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"49'71","reported_seq":155,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978346+0000","last_change":"2023-11-25T04:25:47.109628+0000","last_active":"2023-11-25T04:25:59.978346+0000","last_peered":"2023-11-25T04:25:59.978346+0000","last_clean":"2023-11-25T04:25:59.978346+0000","last_became_active":"2023-11-25T04:25:47.001162+0000","last_became_peered":"2023-11-25T04:25:47.001162+0000","last_unstale":"2023-11-25T04:25:59.978346+0000","last_undegraded":"2023-11-25T04:25:59.978346+0000","last_fullsized":"2023-11-25T04:25:59.978346+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:03:33.658392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":69,"num_write_kb":4130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4194304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"54'158","reported_seq":226,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978945+0000","last_change":"2023-11-25T04:22:47.038454+0000","last_active":"2023-11-25T04:25:59.978945+0000","last_peered":"2023-11-25T04:25:59.978945+0000","last_clean":"2023-11-25T04:25:59.978945+0000","last_became_active":"2023-11-25T04:22:47.038046+0000","last_became_peered":"2023-11-25T04:22:47.038046+0000","last_unstale":"2023-11-25T04:25:59.978945+0000","last_undegraded":"2023-11-25T04:25:59.978945+0000","last_fullsized":"2023-11-25T04:25:59.978945+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:38:38.839941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":115501,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":163,"num_write_kb":243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.977941+0000","last_change":"2023-11-25T04:25:46.998739+0000","last_active":"2023-11-25T04:25:59.977941+0000","last_peered":"2023-11-25T04:25:59.977941+0000","last_clean":"2023-11-25T04:25:59.977941+0000","last_became_active":"2023-11-25T04:25:46.997391+0000","last_became_peered":"2023-11-25T04:25:46.997391+0000","last_unstale":"2023-11-25T04:25:59.977941+0000","last_undegraded":"2023-11-25T04:25:59.977941+0000","last_fullsized":"2023-11-25T04:25:59.977941+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:47:46.300596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"50'1","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978166+0000","last_change":"2023-11-25T04:25:46.999446+0000","last_active":"2023-11-25T04:25:59.978166+0000","last_peered":"2023-11-25T04:25:59.978166+0000","last_clean":"2023-11-25T04:25:59.978166+0000","last_became_active":"2023-11-25T04:25:46.998132+0000","last_became_peered":"2023-11-25T04:25:46.998132+0000","last_unstale":"2023-11-25T04:25:59.978166+0000","last_undegraded":"2023-11-25T04:25:59.978166+0000","last_fullsized":"2023-11-25T04:25:59.978166+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:42:51.166678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"54'157","reported_seq":225,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978638+0000","last_change":"2023-11-25T04:22:47.037355+0000","last_active":"2023-11-25T04:25:59.978638+0000","last_peered":"2023-11-25T04:25:59.978638+0000","last_clean":"2023-11-25T04:25:59.978638+0000","last_became_active":"2023-11-25T04:22:47.036924+0000","last_became_peered":"2023-11-25T04:22:47.036924+0000","last_unstale":"2023-11-25T04:25:59.978638+0000","last_undegraded":"2023-11-25T04:25:59.978638+0000","last_fullsized":"2023-11-25T04:25:59.978638+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:15:39.511043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":93321,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":148,"num_write_kb":360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978520+0000","last_change":"2023-11-25T04:25:46.999738+0000","last_active":"2023-11-25T04:25:59.978520+0000","last_peered":"2023-11-25T04:25:59.978520+0000","last_clean":"2023-11-25T04:25:59.978520+0000","last_became_active":"2023-11-25T04:25:46.999403+0000","last_became_peered":"2023-11-25T04:25:46.999403+0000","last_unstale":"2023-11-25T04:25:59.978520+0000","last_undegraded":"2023-11-25T04:25:59.978520+0000","last_fullsized":"2023-11-25T04:25:59.978520+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:45:53.304063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":72,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978705+0000","last_change":"2023-11-25T04:22:42.000204+0000","last_active":"2023-11-25T04:25:59.978705+0000","last_peered":"2023-11-25T04:25:59.978705+0000","last_clean":"2023-11-25T04:25:59.978705+0000","last_became_active":"2023-11-25T04:22:41.999915+0000","last_became_peered":"2023-11-25T04:22:41.999915+0000","last_unstale":"2023-11-25T04:25:59.978705+0000","last_undegraded":"2023-11-25T04:25:59.978705+0000","last_fullsized":"2023-11-25T04:25:59.978705+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:07:32.457905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"54'152","reported_seq":432,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978379+0000","last_change":"2023-11-25T04:25:53.933239+0000","last_active":"2023-11-25T04:25:59.978379+0000","last_peered":"2023-11-25T04:25:59.978379+0000","last_clean":"2023-11-25T04:25:59.978379+0000","last_became_active":"2023-11-25T04:25:53.932586+0000","last_became_peered":"2023-11-25T04:25:53.932586+0000","last_unstale":"2023-11-25T04:25:59.978379+0000","last_undegraded":"2023-11-25T04:25:59.978379+0000","last_fullsized":"2023-11-25T04:25:59.978379+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:54:51.339025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":329327,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":151,"num_write_kb":471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":97,"num_bytes_recovered":329278,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":69,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978632+0000","last_change":"2023-11-25T04:22:46.035885+0000","last_active":"2023-11-25T04:25:59.978632+0000","last_peered":"2023-11-25T04:25:59.978632+0000","last_clean":"2023-11-25T04:25:59.978632+0000","last_became_active":"2023-11-25T04:22:46.035405+0000","last_became_peered":"2023-11-25T04:22:46.035405+0000","last_unstale":"2023-11-25T04:25:59.978632+0000","last_undegraded":"2023-11-25T04:25:59.978632+0000","last_fullsized":"2023-11-25T04:25:59.978632+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:53:47.398746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":72,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979221+0000","last_change":"2023-11-25T04:22:41.999067+0000","last_active":"2023-11-25T04:25:59.979221+0000","last_peered":"2023-11-25T04:25:59.979221+0000","last_clean":"2023-11-25T04:25:59.979221+0000","last_became_active":"2023-11-25T04:22:41.998798+0000","last_became_peered":"2023-11-25T04:22:41.998798+0000","last_unstale":"2023-11-25T04:25:59.979221+0000","last_undegraded":"2023-11-25T04:25:59.979221+0000","last_fullsized":"2023-11-25T04:25:59.979221+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:00:04.266051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"54'172","reported_seq":470,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978595+0000","last_change":"2023-11-25T04:25:51.922403+0000","last_active":"2023-11-25T04:25:59.978595+0000","last_peered":"2023-11-25T04:25:59.978595+0000","last_clean":"2023-11-25T04:25:59.978595+0000","last_became_active":"2023-11-25T04:25:51.921858+0000","last_became_peered":"2023-11-25T04:25:51.921858+0000","last_unstale":"2023-11-25T04:25:59.978595+0000","last_undegraded":"2023-11-25T04:25:59.978595+0000","last_fullsized":"2023-11-25T04:25:59.978595+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:04:58.273982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":129965,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":166,"num_write_kb":290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":129965,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"18'2","reported_seq":71,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979088+0000","last_change":"2023-11-25T04:22:46.034606+0000","last_active":"2023-11-25T04:25:59.979088+0000","last_peered":"2023-11-25T04:25:59.979088+0000","last_clean":"2023-11-25T04:25:59.979088+0000","last_became_active":"2023-11-25T04:22:46.034229+0000","last_became_peered":"2023-11-25T04:22:46.034229+0000","last_unstale":"2023-11-25T04:25:59.979088+0000","last_undegraded":"2023-11-25T04:25:59.979088+0000","last_fullsized":"2023-11-25T04:25:59.979088+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:21:42.146264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":72,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978952+0000","last_change":"2023-11-25T04:22:42.000258+0000","last_active":"2023-11-25T04:25:59.978952+0000","last_peered":"2023-11-25T04:25:59.978952+0000","last_clean":"2023-11-25T04:25:59.978952+0000","last_became_active":"2023-11-25T04:22:41.999992+0000","last_became_peered":"2023-11-25T04:22:41.999992+0000","last_unstale":"2023-11-25T04:25:59.978952+0000","last_undegraded":"2023-11-25T04:25:59.978952+0000","last_fullsized":"2023-11-25T04:25:59.978952+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:31:13.715992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"54'150","reported_seq":436,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.977926+0000","last_change":"2023-11-25T04:25:54.938479+0000","last_active":"2023-11-25T04:25:59.977926+0000","last_peered":"2023-11-25T04:25:59.977926+0000","last_clean":"2023-11-25T04:25:59.977926+0000","last_became_active":"2023-11-25T04:25:54.937838+0000","last_became_peered":"2023-11-25T04:25:54.937838+0000","last_unstale":"2023-11-25T04:25:59.977926+0000","last_undegraded":"2023-11-25T04:25:59.977926+0000","last_fullsized":"2023-11-25T04:25:59.977926+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:51:14.394190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":159585,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":154224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"54'14","reported_seq":99,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978916+0000","last_change":"2023-11-25T04:25:47.026749+0000","last_active":"2023-11-25T04:25:59.978916+0000","last_peered":"2023-11-25T04:25:59.978916+0000","last_clean":"2023-11-25T04:25:59.978916+0000","last_became_active":"2023-11-25T04:25:46.997404+0000","last_became_peered":"2023-11-25T04:25:46.997404+0000","last_unstale":"2023-11-25T04:25:59.978916+0000","last_undegraded":"2023-11-25T04:25:59.978916+0000","last_fullsized":"2023-11-25T04:25:59.978916+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:20:31.719529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978930+0000","last_change":"2023-11-25T04:25:46.995200+0000","last_active":"2023-11-25T04:25:59.978930+0000","last_peered":"2023-11-25T04:25:59.978930+0000","last_clean":"2023-11-25T04:25:59.978930+0000","last_became_active":"2023-11-25T04:25:46.994834+0000","last_became_peered":"2023-11-25T04:25:46.994834+0000","last_unstale":"2023-11-25T04:25:59.978930+0000","last_undegraded":"2023-11-25T04:25:59.978930+0000","last_fullsized":"2023-11-25T04:25:59.978930+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:33:07.913911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"54'135","reported_seq":407,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978276+0000","last_change":"2023-11-25T04:25:49.922046+0000","last_active":"2023-11-25T04:25:59.978276+0000","last_peered":"2023-11-25T04:25:59.978276+0000","last_clean":"2023-11-25T04:25:59.978276+0000","last_became_active":"2023-11-25T04:25:49.921465+0000","last_became_peered":"2023-11-25T04:25:49.921465+0000","last_unstale":"2023-11-25T04:25:59.978276+0000","last_undegraded":"2023-11-25T04:25:59.978276+0000","last_fullsized":"2023-11-25T04:25:59.978276+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:12:54.077901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":99503,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":148,"num_write_kb":250,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":93,"num_bytes_recovered":99503,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"54'166","reported_seq":233,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978407+0000","last_change":"2023-11-25T04:25:51.923141+0000","last_active":"2023-11-25T04:25:59.978407+0000","last_peered":"2023-11-25T04:25:59.978407+0000","last_clean":"2023-11-25T04:25:59.978407+0000","last_became_active":"2023-11-25T04:25:51.922070+0000","last_became_peered":"2023-11-25T04:25:51.922070+0000","last_unstale":"2023-11-25T04:25:59.978407+0000","last_undegraded":"2023-11-25T04:25:59.978407+0000","last_fullsized":"2023-11-25T04:25:59.978407+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:34:53.021081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":185321,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":172,"num_write_kb":312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":164176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"58'50","reported_seq":53,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978227+0000","last_change":"2023-11-25T04:25:47.021705+0000","last_active":"2023-11-25T04:25:59.978227+0000","last_peered":"2023-11-25T04:25:59.978227+0000","last_clean":"2023-11-25T04:25:59.978227+0000","last_became_active":"2023-11-25T04:25:46.994997+0000","last_became_peered":"2023-11-25T04:25:46.994997+0000","last_unstale":"2023-11-25T04:25:59.978227+0000","last_undegraded":"2023-11-25T04:25:59.978227+0000","last_fullsized":"2023-11-25T04:25:59.978227+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:41:04.147372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":4119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4190183,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"54'153","reported_seq":221,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978107+0000","last_change":"2023-11-25T04:22:47.037181+0000","last_active":"2023-11-25T04:25:59.978107+0000","last_peered":"2023-11-25T04:25:59.978107+0000","last_clean":"2023-11-25T04:25:59.978107+0000","last_became_active":"2023-11-25T04:22:47.036821+0000","last_became_peered":"2023-11-25T04:22:47.036821+0000","last_unstale":"2023-11-25T04:25:59.978107+0000","last_undegraded":"2023-11-25T04:25:59.978107+0000","last_fullsized":"2023-11-25T04:25:59.978107+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:12:51.990448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":101117,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":161,"num_write_kb":242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"50'22","reported_seq":107,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979124+0000","last_change":"2023-11-25T04:25:47.028636+0000","last_active":"2023-11-25T04:25:59.979124+0000","last_peered":"2023-11-25T04:25:59.979124+0000","last_clean":"2023-11-25T04:25:59.979124+0000","last_became_active":"2023-11-25T04:25:47.000478+0000","last_became_peered":"2023-11-25T04:25:47.000478+0000","last_unstale":"2023-11-25T04:25:59.979124+0000","last_undegraded":"2023-11-25T04:25:59.979124+0000","last_fullsized":"2023-11-25T04:25:59.979124+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:50:13.896985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":90,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978907+0000","last_change":"2023-11-25T04:25:46.996182+0000","last_active":"2023-11-25T04:25:59.978907+0000","last_peered":"2023-11-25T04:25:59.978907+0000","last_clean":"2023-11-25T04:25:59.978907+0000","last_became_active":"2023-11-25T04:25:46.994525+0000","last_became_peered":"2023-11-25T04:25:46.994525+0000","last_unstale":"2023-11-25T04:25:59.978907+0000","last_undegraded":"2023-11-25T04:25:59.978907+0000","last_fullsized":"2023-11-25T04:25:59.978907+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:26:24.469530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"54'177","reported_seq":485,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.977915+0000","last_change":"2023-11-25T04:25:51.922466+0000","last_active":"2023-11-25T04:25:59.977915+0000","last_peered":"2023-11-25T04:25:59.977915+0000","last_clean":"2023-11-25T04:25:59.977915+0000","last_became_active":"2023-11-25T04:25:51.921938+0000","last_became_peered":"2023-11-25T04:25:51.921938+0000","last_unstale":"2023-11-25T04:25:59.977915+0000","last_undegraded":"2023-11-25T04:25:59.977915+0000","last_fullsized":"2023-11-25T04:25:59.977915+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:38:59.016525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":522494,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":168,"num_write_kb":749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":111,"num_bytes_recovered":522494,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"46'2","reported_seq":82,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979031+0000","last_change":"2023-11-25T04:25:46.998867+0000","last_active":"2023-11-25T04:25:59.979031+0000","last_peered":"2023-11-25T04:25:59.979031+0000","last_clean":"2023-11-25T04:25:59.979031+0000","last_became_active":"2023-11-25T04:25:46.998508+0000","last_became_peered":"2023-11-25T04:25:46.998508+0000","last_unstale":"2023-11-25T04:25:59.979031+0000","last_undegraded":"2023-11-25T04:25:59.979031+0000","last_fullsized":"2023-11-25T04:25:59.979031+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:50:13.896985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.10","version":"54'153","reported_seq":221,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.977904+0000","last_change":"2023-11-25T04:22:47.037790+0000","last_active":"2023-11-25T04:25:59.977904+0000","last_peered":"2023-11-25T04:25:59.977904+0000","last_clean":"2023-11-25T04:25:59.977904+0000","last_became_active":"2023-11-25T04:22:47.037149+0000","last_became_peered":"2023-11-25T04:22:47.037149+0000","last_unstale":"2023-11-25T04:25:59.977904+0000","last_undegraded":"2023-11-25T04:25:59.977904+0000","last_fullsized":"2023-11-25T04:25:59.977904+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:31:54.857270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":218073,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1f","version":"54'156","reported_seq":448,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979259+0000","last_change":"2023-11-25T04:25:49.921904+0000","last_active":"2023-11-25T04:25:59.979259+0000","last_peered":"2023-11-25T04:25:59.979259+0000","last_clean":"2023-11-25T04:25:59.979259+0000","last_became_active":"2023-11-25T04:25:49.921458+0000","last_became_peered":"2023-11-25T04:25:49.921458+0000","last_unstale":"2023-11-25T04:25:59.979259+0000","last_undegraded":"2023-11-25T04:25:59.979259+0000","last_fullsized":"2023-11-25T04:25:59.979259+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:59:27.602060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":107712,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":103,"num_bytes_recovered":107703,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"48'1","reported_seq":70,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978057+0000","last_change":"2023-11-25T04:22:46.036085+0000","last_active":"2023-11-25T04:25:59.978057+0000","last_peered":"2023-11-25T04:25:59.978057+0000","last_clean":"2023-11-25T04:25:59.978057+0000","last_became_active":"2023-11-25T04:22:46.035495+0000","last_became_peered":"2023-11-25T04:22:46.035495+0000","last_unstale":"2023-11-25T04:25:59.978057+0000","last_undegraded":"2023-11-25T04:25:59.978057+0000","last_fullsized":"2023-11-25T04:25:59.978057+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:14:56.735365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"54'152","reported_seq":430,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978757+0000","last_change":"2023-11-25T04:25:50.923882+0000","last_active":"2023-11-25T04:25:59.978757+0000","last_peered":"2023-11-25T04:25:59.978757+0000","last_clean":"2023-11-25T04:25:59.978757+0000","last_became_active":"2023-11-25T04:25:50.923375+0000","last_became_peered":"2023-11-25T04:25:50.923375+0000","last_unstale":"2023-11-25T04:25:59.978757+0000","last_undegraded":"2023-11-25T04:25:59.978757+0000","last_fullsized":"2023-11-25T04:25:59.978757+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:54:51.339025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":76435,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":76435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":80,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978069+0000","last_change":"2023-11-25T04:25:46.995228+0000","last_active":"2023-11-25T04:25:59.978069+0000","last_peered":"2023-11-25T04:25:59.978069+0000","last_clean":"2023-11-25T04:25:59.978069+0000","last_became_active":"2023-11-25T04:25:46.994871+0000","last_became_peered":"2023-11-25T04:25:46.994871+0000","last_unstale":"2023-11-25T04:25:59.978069+0000","last_undegraded":"2023-11-25T04:25:59.978069+0000","last_fullsized":"2023-11-25T04:25:59.978069+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:53:34.807075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"54'158","reported_seq":432,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978899+0000","last_change":"2023-11-25T04:25:52.925523+0000","last_active":"2023-11-25T04:25:59.978899+0000","last_peered":"2023-11-25T04:25:59.978899+0000","last_clean":"2023-11-25T04:25:59.978899+0000","last_became_active":"2023-11-25T04:25:52.925038+0000","last_became_peered":"2023-11-25T04:25:52.925038+0000","last_unstale":"2023-11-25T04:25:59.978899+0000","last_undegraded":"2023-11-25T04:25:59.978899+0000","last_fullsized":"2023-11-25T04:25:59.978899+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:37:14.620534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":209500,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":139,"num_write_kb":351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":209500,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1a","version":"48'4","reported_seq":73,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978665+0000","last_change":"2023-11-25T04:22:46.034340+0000","last_active":"2023-11-25T04:25:59.978665+0000","last_peered":"2023-11-25T04:25:59.978665+0000","last_clean":"2023-11-25T04:25:59.978665+0000","last_became_active":"2023-11-25T04:22:46.033944+0000","last_became_peered":"2023-11-25T04:22:46.033944+0000","last_unstale":"2023-11-25T04:25:59.978665+0000","last_undegraded":"2023-11-25T04:25:59.978665+0000","last_fullsized":"2023-11-25T04:25:59.978665+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:59:30.291004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"54'163","reported_seq":231,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978741+0000","last_change":"2023-11-25T04:22:47.039547+0000","last_active":"2023-11-25T04:25:59.978741+0000","last_peered":"2023-11-25T04:25:59.978741+0000","last_clean":"2023-11-25T04:25:59.978741+0000","last_became_active":"2023-11-25T04:22:47.039218+0000","last_became_peered":"2023-11-25T04:22:47.039218+0000","last_unstale":"2023-11-25T04:25:59.978741+0000","last_undegraded":"2023-11-25T04:25:59.978741+0000","last_fullsized":"2023-11-25T04:25:59.978741+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:44:48.553972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":165883,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"45'9","reported_seq":94,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978700+0000","last_change":"2023-11-25T04:25:47.023365+0000","last_active":"2023-11-25T04:25:59.978700+0000","last_peered":"2023-11-25T04:25:59.978700+0000","last_clean":"2023-11-25T04:25:59.978700+0000","last_became_active":"2023-11-25T04:25:46.999998+0000","last_became_peered":"2023-11-25T04:25:46.999998+0000","last_unstale":"2023-11-25T04:25:59.978700+0000","last_undegraded":"2023-11-25T04:25:59.978700+0000","last_fullsized":"2023-11-25T04:25:59.978700+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:53:34.807075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":4101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4194304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"54'149","reported_seq":217,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978573+0000","last_change":"2023-11-25T04:25:50.923041+0000","last_active":"2023-11-25T04:25:59.978573+0000","last_peered":"2023-11-25T04:25:59.978573+0000","last_clean":"2023-11-25T04:25:59.978573+0000","last_became_active":"2023-11-25T04:25:50.922164+0000","last_became_peered":"2023-11-25T04:25:50.922164+0000","last_unstale":"2023-11-25T04:25:59.978573+0000","last_undegraded":"2023-11-25T04:25:59.978573+0000","last_fullsized":"2023-11-25T04:25:59.978573+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:31:10.744814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":179506,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":155,"num_write_kb":317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":179506,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"18'2","reported_seq":37,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978409+0000","last_change":"2023-11-25T04:25:46.996675+0000","last_active":"2023-11-25T04:25:59.978409+0000","last_peered":"2023-11-25T04:25:59.978409+0000","last_clean":"2023-11-25T04:25:59.978409+0000","last_became_active":"2023-11-25T04:25:46.995160+0000","last_became_peered":"2023-11-25T04:25:46.995160+0000","last_unstale":"2023-11-25T04:25:59.978409+0000","last_undegraded":"2023-11-25T04:25:59.978409+0000","last_fullsized":"2023-11-25T04:25:59.978409+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:59:39.131605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1a","version":"54'162","reported_seq":226,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978332+0000","last_change":"2023-11-25T04:25:54.939182+0000","last_active":"2023-11-25T04:25:59.978332+0000","last_peered":"2023-11-25T04:25:59.978332+0000","last_clean":"2023-11-25T04:25:59.978332+0000","last_became_active":"2023-11-25T04:25:54.937821+0000","last_became_peered":"2023-11-25T04:25:54.937821+0000","last_unstale":"2023-11-25T04:25:59.978332+0000","last_undegraded":"2023-11-25T04:25:59.978332+0000","last_fullsized":"2023-11-25T04:25:59.978332+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:05:34.575966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":134672,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":169,"num_write_kb":311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":126291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"48'59","reported_seq":146,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979001+0000","last_change":"2023-11-25T04:25:47.077194+0000","last_active":"2023-11-25T04:25:59.979001+0000","last_peered":"2023-11-25T04:25:59.979001+0000","last_clean":"2023-11-25T04:25:59.979001+0000","last_became_active":"2023-11-25T04:25:47.000722+0000","last_became_peered":"2023-11-25T04:25:47.000722+0000","last_unstale":"2023-11-25T04:25:59.979001+0000","last_undegraded":"2023-11-25T04:25:59.979001+0000","last_fullsized":"2023-11-25T04:25:59.979001+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:55:01.351319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8388608,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"54'163","reported_seq":231,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978500+0000","last_change":"2023-11-25T04:22:47.036764+0000","last_active":"2023-11-25T04:25:59.978500+0000","last_peered":"2023-11-25T04:25:59.978500+0000","last_clean":"2023-11-25T04:25:59.978500+0000","last_became_active":"2023-11-25T04:22:47.036406+0000","last_became_peered":"2023-11-25T04:22:47.036406+0000","last_unstale":"2023-11-25T04:25:59.978500+0000","last_undegraded":"2023-11-25T04:25:59.978500+0000","last_fullsized":"2023-11-25T04:25:59.978500+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:09:55.357642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":527891,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":163,"num_write_kb":687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1e","version":"46'2","reported_seq":71,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978868+0000","last_change":"2023-11-25T04:22:46.035659+0000","last_active":"2023-11-25T04:25:59.978868+0000","last_peered":"2023-11-25T04:25:59.978868+0000","last_clean":"2023-11-25T04:25:59.978868+0000","last_became_active":"2023-11-25T04:22:46.035113+0000","last_became_peered":"2023-11-25T04:22:46.035113+0000","last_unstale":"2023-11-25T04:25:59.978868+0000","last_undegraded":"2023-11-25T04:25:59.978868+0000","last_fullsized":"2023-11-25T04:25:59.978868+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:26:25.338117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.18","version":"54'195","reported_seq":259,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978263+0000","last_change":"2023-11-25T04:25:54.938573+0000","last_active":"2023-11-25T04:25:59.978263+0000","last_peered":"2023-11-25T04:25:59.978263+0000","last_clean":"2023-11-25T04:25:59.978263+0000","last_became_active":"2023-11-25T04:25:54.937444+0000","last_became_peered":"2023-11-25T04:25:54.937444+0000","last_unstale":"2023-11-25T04:25:59.978263+0000","last_undegraded":"2023-11-25T04:25:59.978263+0000","last_fullsized":"2023-11-25T04:25:59.978263+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:20:15.814691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":128641,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":185,"num_write_kb":408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":122,"num_bytes_recovered":128641,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"50'17","reported_seq":102,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978969+0000","last_change":"2023-11-25T04:25:47.044228+0000","last_active":"2023-11-25T04:25:59.978969+0000","last_peered":"2023-11-25T04:25:59.978969+0000","last_clean":"2023-11-25T04:25:59.978969+0000","last_became_active":"2023-11-25T04:25:46.998870+0000","last_became_peered":"2023-11-25T04:25:46.998870+0000","last_unstale":"2023-11-25T04:25:59.978969+0000","last_undegraded":"2023-11-25T04:25:59.978969+0000","last_fullsized":"2023-11-25T04:25:59.978969+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:08:12.601941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":4103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4194304,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":26330855,"num_objects":3447,"num_object_clones":0,"num_object_copies":6894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10423,"num_write_kb":38550,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4189,"num_bytes_recovered":12749181,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":72376320,"data_stored":52324046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10246,"ondisk_log_size":10246,"up":128,"acting":128,"num_store_stats":3},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":40885548,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":35,"num_read_kb":334,"num_write":435,"num_write_kb":40462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":32491439,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":81833984,"data_stored":81771096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5548,"internal_metadata":0},"log_size":435,"ondisk_log_size":435,"up":64,"acting":64,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738415,"num_pgs":72,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73208,"kb_used_data":46188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94298632,"statfs":{"total":96636764160,"available":96561799168,"internally_reserved":0,"allocated":47296512,"data_stored":39787071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":1,"last update":"Sat Nov 25 04:25:40 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.433,"15min":0.44500000000000001},"min":{"1min":0.28899999999999998,"5min":0.216,"15min":0.216},"max":{"1min":0.60499999999999998,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.439,"15min":0.45100000000000001},"min":{"1min":0.25600000000000001,"5min":0.184,"15min":0.184},"max":{"1min":0.63400000000000001,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.63}]}]},{"osd":1,"up_from":8,"seq":34359738415,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86880,"kb_used_data":55700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94284960,"statfs":{"total":96636764160,"available":96547799040,"internally_reserved":0,"allocated":57036800,"data_stored":50261554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":2,"last update":"Sat Nov 25 04:25:42 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.40600000000000003,"15min":0.41099999999999998},"min":{"1min":0.29099999999999998,"5min":0.182,"15min":0.182},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.40200000000000002,"15min":0.40600000000000003},"min":{"1min":0.312,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.39000000000000001}]}]},{"osd":0,"up_from":60,"seq":257698037766,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87624,"kb_used_data":51764,"kb_used_omap":6,"kb_used_meta":35833,"kb_avail":94284216,"statfs":{"total":96636764160,"available":96547037184,"internally_reserved":0,"allocated":53006336,"data_stored":46141107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7143,"internal_metadata":36693017},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32530432,"data_stored":32496940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5548,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":29380608,"data_stored":29360982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":19922944,"data_stored":19913174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":19234816,"data_stored":12749181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":26415104,"data_stored":20005586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":26726400,"data_stored":19569279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-25T04:26:06.902 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-25T04:26:07.234 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:26:07.234 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":75,"fsid":"a677f511-1143-4e76-8da6-5ddfadaff077","created":"2023-11-25T04:22:25.140474+0000","modified":"2023-11-25T04:26:00.979788+0000","last_up_change":"2023-11-25T04:25:45.970689+0000","last_in_change":"2023-11-25T04:22:25.985372+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-25T04:22:35.548359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-25T04:22:40.218527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-25T04:22:44.844906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-25T04:22:45.365536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bc397d35-53ea-430c-a96a-63a4160e64c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":20,"up_from":60,"up_thru":68,"down_at":21,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1420426974},{"type":"v1","addr":"172.21.15.76:6801","nonce":1420426974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1420426974},{"type":"v1","addr":"172.21.15.76:6803","nonce":1420426974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1420426974},{"type":"v1","addr":"172.21.15.76:6807","nonce":1420426974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1420426974},{"type":"v1","addr":"172.21.15.76:6805","nonce":1420426974}]},"public_addr":"172.21.15.76:6801/1420426974","cluster_addr":"172.21.15.76:6803/1420426974","heartbeat_back_addr":"172.21.15.76:6807/1420426974","heartbeat_front_addr":"172.21.15.76:6805/1420426974","state":["exists","up"]},{"osd":1,"uuid":"d4f1b88a-bbd4-4e58-9519-569489ced7b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6801","nonce":3130474108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6803","nonce":3130474108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6807","nonce":3130474108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6805","nonce":3130474108}]},"public_addr":"172.21.15.92:6801/3130474108","cluster_addr":"172.21.15.92:6803/3130474108","heartbeat_back_addr":"172.21.15.92:6807/3130474108","heartbeat_front_addr":"172.21.15.92:6805/3130474108","state":["exists","up"]},{"osd":2,"uuid":"55c82801-e090-4c10-91e5-5fa9dbfba36d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6801","nonce":4099978417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6803","nonce":4099978417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6807","nonce":4099978417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6805","nonce":4099978417}]},"public_addr":"172.21.15.195:6801/4099978417","cluster_addr":"172.21.15.195:6803/4099978417","heartbeat_back_addr":"172.21.15.195:6807/4099978417","heartbeat_front_addr":"172.21.15.195:6805/4099978417","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-25T04:22:31.039867+0000","dead_epoch":20},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-25T04:22:31.021863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-25T04:22:31.009821+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-25T04:26:08.248 INFO:tasks.ceph:Scrubbing osd.0 2023-11-25T04:26:08.249 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-11-25T04:26:08.394 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2023-11-25T04:26:08.394 INFO:teuthology.orchestra.run.smithi067.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_pg_delete_cost = '' (not observed, change may require restart) osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2023-11-25T04:26:08.394 INFO:teuthology.orchestra.run.smithi067.stdout:} 2023-11-25T04:26:08.405 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-11-25T04:26:08.700 INFO:teuthology.orchestra.run.smithi067.stderr:instructed osd(s) 0 to deep-scrub 2023-11-25T04:26:08.713 INFO:tasks.ceph:Scrubbing osd.1 2023-11-25T04:26:08.713 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-11-25T04:26:08.845 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2023-11-25T04:26:08.846 INFO:teuthology.orchestra.run.smithi067.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-25T04:26:08.846 INFO:teuthology.orchestra.run.smithi067.stdout:} 2023-11-25T04:26:08.856 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-11-25T04:26:09.152 INFO:teuthology.orchestra.run.smithi067.stderr:instructed osd(s) 1 to deep-scrub 2023-11-25T04:26:09.229 INFO:tasks.ceph:Scrubbing osd.2 2023-11-25T04:26:09.230 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-11-25T04:26:09.363 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2023-11-25T04:26:09.363 INFO:teuthology.orchestra.run.smithi067.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-25T04:26:09.363 INFO:teuthology.orchestra.run.smithi067.stdout:} 2023-11-25T04:26:09.374 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-11-25T04:26:09.671 INFO:teuthology.orchestra.run.smithi067.stderr:instructed osd(s) 2 to deep-scrub 2023-11-25T04:26:09.684 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-25T04:26:09.983 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:26:10.188 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2023-11-25T04:26:10.191 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":181,"stamp":"2023-11-25T04:26:09.551239+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":67806771,"num_objects":3481,"num_object_clones":0,"num_object_copies":6962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3481,"num_whiteouts":0,"num_read":171,"num_read_kb":452,"num_write":11061,"num_write_kb":82472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4201,"num_bytes_recovered":45240620,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10838,"ondisk_log_size":10838,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":247712,"kb_used_data":153652,"kb_used_omap":10,"kb_used_meta":94005,"kb_avail":282867808,"statfs":{"total":289910292480,"available":289656635392,"internally_reserved":0,"allocated":157339648,"data_stored":136189732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10333,"internal_metadata":96262051},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.574683"},"pg_stats":[{"pgid":"4.3f","version":"54'166","reported_seq":234,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979173+0000","last_change":"2023-11-25T04:22:47.039847+0000","last_active":"2023-11-25T04:25:59.979173+0000","last_peered":"2023-11-25T04:25:59.979173+0000","last_clean":"2023-11-25T04:25:59.979173+0000","last_became_active":"2023-11-25T04:22:47.039490+0000","last_became_peered":"2023-11-25T04:22:47.039490+0000","last_unstale":"2023-11-25T04:25:59.979173+0000","last_undegraded":"2023-11-25T04:25:59.979173+0000","last_fullsized":"2023-11-25T04:25:59.979173+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:25:06.132517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":577866,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":174,"num_write_kb":756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3e","version":"54'212","reported_seq":280,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978984+0000","last_change":"2023-11-25T04:22:47.040836+0000","last_active":"2023-11-25T04:25:59.978984+0000","last_peered":"2023-11-25T04:25:59.978984+0000","last_clean":"2023-11-25T04:25:59.978984+0000","last_became_active":"2023-11-25T04:22:47.040511+0000","last_became_peered":"2023-11-25T04:22:47.040511+0000","last_unstale":"2023-11-25T04:25:59.978984+0000","last_undegraded":"2023-11-25T04:25:59.978984+0000","last_fullsized":"2023-11-25T04:25:59.978984+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:15:39.511043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":461579,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":236,"num_write_kb":841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3d","version":"54'167","reported_seq":233,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978931+0000","last_change":"2023-11-25T04:25:52.924678+0000","last_active":"2023-11-25T04:25:59.978931+0000","last_peered":"2023-11-25T04:25:59.978931+0000","last_clean":"2023-11-25T04:25:59.978931+0000","last_became_active":"2023-11-25T04:25:52.923459+0000","last_became_peered":"2023-11-25T04:25:52.923459+0000","last_unstale":"2023-11-25T04:25:59.978931+0000","last_undegraded":"2023-11-25T04:25:59.978931+0000","last_fullsized":"2023-11-25T04:25:59.978931+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:24:06.950438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":145844,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":175,"num_write_kb":312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":138765,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"54'124","reported_seq":191,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978386+0000","last_change":"2023-11-25T04:25:51.922843+0000","last_active":"2023-11-25T04:25:59.978386+0000","last_peered":"2023-11-25T04:25:59.978386+0000","last_clean":"2023-11-25T04:25:59.978386+0000","last_became_active":"2023-11-25T04:25:51.921633+0000","last_became_peered":"2023-11-25T04:25:51.921633+0000","last_unstale":"2023-11-25T04:25:59.978386+0000","last_undegraded":"2023-11-25T04:25:59.978386+0000","last_fullsized":"2023-11-25T04:25:59.978386+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:05:31.332951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61303,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119,"num_write_kb":161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":61303,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"54'169","reported_seq":237,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978640+0000","last_change":"2023-11-25T04:22:47.039499+0000","last_active":"2023-11-25T04:25:59.978640+0000","last_peered":"2023-11-25T04:25:59.978640+0000","last_clean":"2023-11-25T04:25:59.978640+0000","last_became_active":"2023-11-25T04:22:47.039165+0000","last_became_peered":"2023-11-25T04:22:47.039165+0000","last_unstale":"2023-11-25T04:25:59.978640+0000","last_undegraded":"2023-11-25T04:25:59.978640+0000","last_fullsized":"2023-11-25T04:25:59.978640+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:57:28.855151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":118227,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":185,"num_write_kb":351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3a","version":"54'169","reported_seq":234,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978863+0000","last_change":"2023-11-25T04:25:53.933198+0000","last_active":"2023-11-25T04:25:59.978863+0000","last_peered":"2023-11-25T04:25:59.978863+0000","last_clean":"2023-11-25T04:25:59.978863+0000","last_became_active":"2023-11-25T04:25:53.931982+0000","last_became_peered":"2023-11-25T04:25:53.931982+0000","last_unstale":"2023-11-25T04:25:59.978863+0000","last_undegraded":"2023-11-25T04:25:59.978863+0000","last_fullsized":"2023-11-25T04:25:59.978863+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:28:40.449148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100304,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":161,"num_write_kb":424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":100304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"54'168","reported_seq":237,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978380+0000","last_change":"2023-11-25T04:25:49.921033+0000","last_active":"2023-11-25T04:25:59.978380+0000","last_peered":"2023-11-25T04:25:59.978380+0000","last_clean":"2023-11-25T04:25:59.978380+0000","last_became_active":"2023-11-25T04:25:49.919354+0000","last_became_peered":"2023-11-25T04:25:49.919354+0000","last_unstale":"2023-11-25T04:25:59.978380+0000","last_undegraded":"2023-11-25T04:25:59.978380+0000","last_fullsized":"2023-11-25T04:25:59.978380+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:36:13.489397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":124558,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":167,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":111,"num_bytes_recovered":124533,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.38","version":"54'152","reported_seq":221,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978153+0000","last_change":"2023-11-25T04:25:49.920421+0000","last_active":"2023-11-25T04:25:59.978153+0000","last_peered":"2023-11-25T04:25:59.978153+0000","last_clean":"2023-11-25T04:25:59.978153+0000","last_became_active":"2023-11-25T04:25:49.919236+0000","last_became_peered":"2023-11-25T04:25:49.919236+0000","last_unstale":"2023-11-25T04:25:59.978153+0000","last_undegraded":"2023-11-25T04:25:59.978153+0000","last_fullsized":"2023-11-25T04:25:59.978153+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:32:41.956579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":120147,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":120121,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.37","version":"54'167","reported_seq":235,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978818+0000","last_change":"2023-11-25T04:22:47.038397+0000","last_active":"2023-11-25T04:25:59.978818+0000","last_peered":"2023-11-25T04:25:59.978818+0000","last_clean":"2023-11-25T04:25:59.978818+0000","last_became_active":"2023-11-25T04:22:47.037328+0000","last_became_peered":"2023-11-25T04:22:47.037328+0000","last_unstale":"2023-11-25T04:25:59.978818+0000","last_undegraded":"2023-11-25T04:25:59.978818+0000","last_fullsized":"2023-11-25T04:25:59.978818+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:56:59.689578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":174147,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":184,"num_write_kb":353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.36","version":"54'149","reported_seq":217,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979047+0000","last_change":"2023-11-25T04:22:47.040182+0000","last_active":"2023-11-25T04:25:59.979047+0000","last_peered":"2023-11-25T04:25:59.979047+0000","last_clean":"2023-11-25T04:25:59.979047+0000","last_became_active":"2023-11-25T04:22:47.039892+0000","last_became_peered":"2023-11-25T04:22:47.039892+0000","last_unstale":"2023-11-25T04:25:59.979047+0000","last_undegraded":"2023-11-25T04:25:59.979047+0000","last_fullsized":"2023-11-25T04:25:59.979047+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:38:09.176582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":233330,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"54'156","reported_seq":223,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979021+0000","last_change":"2023-11-25T04:25:51.923087+0000","last_active":"2023-11-25T04:25:59.979021+0000","last_peered":"2023-11-25T04:25:59.979021+0000","last_clean":"2023-11-25T04:25:59.979021+0000","last_became_active":"2023-11-25T04:25:51.921800+0000","last_became_peered":"2023-11-25T04:25:51.921800+0000","last_unstale":"2023-11-25T04:25:59.979021+0000","last_undegraded":"2023-11-25T04:25:59.979021+0000","last_fullsized":"2023-11-25T04:25:59.979021+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:29:47.766517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":302550,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":163,"num_write_kb":431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":300323,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.34","version":"54'136","reported_seq":400,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978706+0000","last_change":"2023-11-25T04:25:50.922843+0000","last_active":"2023-11-25T04:25:59.978706+0000","last_peered":"2023-11-25T04:25:59.978706+0000","last_clean":"2023-11-25T04:25:59.978706+0000","last_became_active":"2023-11-25T04:25:50.922309+0000","last_became_peered":"2023-11-25T04:25:50.922309+0000","last_unstale":"2023-11-25T04:25:59.978706+0000","last_undegraded":"2023-11-25T04:25:59.978706+0000","last_fullsized":"2023-11-25T04:25:59.978706+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:38:33.939562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":93053,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":131,"num_write_kb":248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":89,"num_bytes_recovered":93053,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"54'134","reported_seq":202,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978529+0000","last_change":"2023-11-25T04:22:47.040388+0000","last_active":"2023-11-25T04:25:59.978529+0000","last_peered":"2023-11-25T04:25:59.978529+0000","last_clean":"2023-11-25T04:25:59.978529+0000","last_became_active":"2023-11-25T04:22:47.040129+0000","last_became_peered":"2023-11-25T04:22:47.040129+0000","last_unstale":"2023-11-25T04:25:59.978529+0000","last_undegraded":"2023-11-25T04:25:59.978529+0000","last_fullsized":"2023-11-25T04:25:59.978529+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:31:54.857270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":149524,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":139,"num_write_kb":252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.32","version":"54'174","reported_seq":239,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978762+0000","last_change":"2023-11-25T04:25:53.934526+0000","last_active":"2023-11-25T04:25:59.978762+0000","last_peered":"2023-11-25T04:25:59.978762+0000","last_clean":"2023-11-25T04:25:59.978762+0000","last_became_active":"2023-11-25T04:25:53.933694+0000","last_became_peered":"2023-11-25T04:25:53.933694+0000","last_unstale":"2023-11-25T04:25:59.978762+0000","last_undegraded":"2023-11-25T04:25:59.978762+0000","last_fullsized":"2023-11-25T04:25:59.978762+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:01:16.858976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":95797,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":181,"num_write_kb":547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":103,"num_bytes_recovered":95797,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.31","version":"54'156","reported_seq":225,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978209+0000","last_change":"2023-11-25T04:25:49.922396+0000","last_active":"2023-11-25T04:25:59.978209+0000","last_peered":"2023-11-25T04:25:59.978209+0000","last_clean":"2023-11-25T04:25:59.978209+0000","last_became_active":"2023-11-25T04:25:49.921129+0000","last_became_peered":"2023-11-25T04:25:49.921129+0000","last_unstale":"2023-11-25T04:25:59.978209+0000","last_undegraded":"2023-11-25T04:25:59.978209+0000","last_fullsized":"2023-11-25T04:25:59.978209+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:57:02.387207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":163190,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":169,"num_write_kb":336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":154576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"54'174","reported_seq":239,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978986+0000","last_change":"2023-11-25T04:25:53.934614+0000","last_active":"2023-11-25T04:25:59.978986+0000","last_peered":"2023-11-25T04:25:59.978986+0000","last_clean":"2023-11-25T04:25:59.978986+0000","last_became_active":"2023-11-25T04:25:53.933395+0000","last_became_peered":"2023-11-25T04:25:53.933395+0000","last_unstale":"2023-11-25T04:25:59.978986+0000","last_undegraded":"2023-11-25T04:25:59.978986+0000","last_fullsized":"2023-11-25T04:25:59.978986+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:53:09.827809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1423587,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":182,"num_write_kb":1626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":1414508,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2f","version":"54'152","reported_seq":220,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979084+0000","last_change":"2023-11-25T04:22:47.039389+0000","last_active":"2023-11-25T04:25:59.979084+0000","last_peered":"2023-11-25T04:25:59.979084+0000","last_clean":"2023-11-25T04:25:59.979084+0000","last_became_active":"2023-11-25T04:22:47.039123+0000","last_became_peered":"2023-11-25T04:22:47.039123+0000","last_unstale":"2023-11-25T04:25:59.979084+0000","last_undegraded":"2023-11-25T04:25:59.979084+0000","last_fullsized":"2023-11-25T04:25:59.979084+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:09:11.570561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":122752,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":143,"num_write_kb":435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"54'148","reported_seq":216,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979169+0000","last_change":"2023-11-25T04:22:47.036946+0000","last_active":"2023-11-25T04:25:59.979169+0000","last_peered":"2023-11-25T04:25:59.979169+0000","last_clean":"2023-11-25T04:25:59.979169+0000","last_became_active":"2023-11-25T04:22:47.036411+0000","last_became_peered":"2023-11-25T04:22:47.036411+0000","last_unstale":"2023-11-25T04:25:59.979169+0000","last_undegraded":"2023-11-25T04:25:59.979169+0000","last_fullsized":"2023-11-25T04:25:59.979169+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:16:29.085050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4770899,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":140,"num_write_kb":4870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2d","version":"54'143","reported_seq":211,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978472+0000","last_change":"2023-11-25T04:22:47.038983+0000","last_active":"2023-11-25T04:25:59.978472+0000","last_peered":"2023-11-25T04:25:59.978472+0000","last_clean":"2023-11-25T04:25:59.978472+0000","last_became_active":"2023-11-25T04:22:47.038477+0000","last_became_peered":"2023-11-25T04:22:47.038477+0000","last_unstale":"2023-11-25T04:25:59.978472+0000","last_undegraded":"2023-11-25T04:25:59.978472+0000","last_fullsized":"2023-11-25T04:25:59.978472+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:30:04.568790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":115975,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"54'167","reported_seq":233,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978615+0000","last_change":"2023-11-25T04:25:52.923640+0000","last_active":"2023-11-25T04:25:59.978615+0000","last_peered":"2023-11-25T04:25:59.978615+0000","last_clean":"2023-11-25T04:25:59.978615+0000","last_became_active":"2023-11-25T04:25:52.922367+0000","last_became_peered":"2023-11-25T04:25:52.922367+0000","last_unstale":"2023-11-25T04:25:59.978615+0000","last_undegraded":"2023-11-25T04:25:59.978615+0000","last_fullsized":"2023-11-25T04:25:59.978615+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:08:24.794280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":747090,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":211,"num_write_kb":985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":732276,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"54'156","reported_seq":222,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978632+0000","last_change":"2023-11-25T04:25:52.924457+0000","last_active":"2023-11-25T04:25:59.978632+0000","last_peered":"2023-11-25T04:25:59.978632+0000","last_clean":"2023-11-25T04:25:59.978632+0000","last_became_active":"2023-11-25T04:25:52.923378+0000","last_became_peered":"2023-11-25T04:25:52.923378+0000","last_unstale":"2023-11-25T04:25:59.978632+0000","last_undegraded":"2023-11-25T04:25:59.978632+0000","last_fullsized":"2023-11-25T04:25:59.978632+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:34:30.942779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69580,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":142,"num_write_kb":315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":98,"num_bytes_recovered":69580,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2a","version":"54'171","reported_seq":467,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978347+0000","last_change":"2023-11-25T04:25:52.925532+0000","last_active":"2023-11-25T04:25:59.978347+0000","last_peered":"2023-11-25T04:25:59.978347+0000","last_clean":"2023-11-25T04:25:59.978347+0000","last_became_active":"2023-11-25T04:25:52.924844+0000","last_became_peered":"2023-11-25T04:25:52.924844+0000","last_unstale":"2023-11-25T04:25:59.978347+0000","last_undegraded":"2023-11-25T04:25:59.978347+0000","last_fullsized":"2023-11-25T04:25:59.978347+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:11:30.124326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":115997,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":162,"num_write_kb":261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":115997,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.29","version":"54'152","reported_seq":438,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.977803+0000","last_change":"2023-11-25T04:25:49.920367+0000","last_active":"2023-11-25T04:25:59.977803+0000","last_peered":"2023-11-25T04:25:59.977803+0000","last_clean":"2023-11-25T04:25:59.977803+0000","last_became_active":"2023-11-25T04:25:49.919886+0000","last_became_peered":"2023-11-25T04:25:49.919886+0000","last_unstale":"2023-11-25T04:25:59.977803+0000","last_undegraded":"2023-11-25T04:25:59.977803+0000","last_fullsized":"2023-11-25T04:25:59.977803+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:04:58.273982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":212878,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":212878,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"54'169","reported_seq":237,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978222+0000","last_change":"2023-11-25T04:22:47.036805+0000","last_active":"2023-11-25T04:25:59.978222+0000","last_peered":"2023-11-25T04:25:59.978222+0000","last_clean":"2023-11-25T04:25:59.978222+0000","last_became_active":"2023-11-25T04:22:47.036135+0000","last_became_peered":"2023-11-25T04:22:47.036135+0000","last_unstale":"2023-11-25T04:25:59.978222+0000","last_undegraded":"2023-11-25T04:25:59.978222+0000","last_fullsized":"2023-11-25T04:25:59.978222+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:25:21.730992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":102957,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":172,"num_write_kb":610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"54'167","reported_seq":465,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979220+0000","last_change":"2023-11-25T04:25:51.922925+0000","last_active":"2023-11-25T04:25:59.979220+0000","last_peered":"2023-11-25T04:25:59.979220+0000","last_clean":"2023-11-25T04:25:59.979220+0000","last_became_active":"2023-11-25T04:25:51.922364+0000","last_became_peered":"2023-11-25T04:25:51.922364+0000","last_unstale":"2023-11-25T04:25:59.979220+0000","last_undegraded":"2023-11-25T04:25:59.979220+0000","last_fullsized":"2023-11-25T04:25:59.979220+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:09:23.741871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":182683,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":171,"num_write_kb":791,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":182683,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"54'187","reported_seq":505,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978723+0000","last_change":"2023-11-25T04:25:51.922529+0000","last_active":"2023-11-25T04:25:59.978723+0000","last_peered":"2023-11-25T04:25:59.978723+0000","last_clean":"2023-11-25T04:25:59.978723+0000","last_became_active":"2023-11-25T04:25:51.922064+0000","last_became_peered":"2023-11-25T04:25:51.922064+0000","last_unstale":"2023-11-25T04:25:59.978723+0000","last_undegraded":"2023-11-25T04:25:59.978723+0000","last_fullsized":"2023-11-25T04:25:59.978723+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":187,"log_dups_size":0,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:01:39.413986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":86311,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":171,"num_write_kb":274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":116,"num_bytes_recovered":86311,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.25","version":"54'149","reported_seq":215,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978829+0000","last_change":"2023-11-25T04:25:52.925505+0000","last_active":"2023-11-25T04:25:59.978829+0000","last_peered":"2023-11-25T04:25:59.978829+0000","last_clean":"2023-11-25T04:25:59.978829+0000","last_became_active":"2023-11-25T04:25:52.924795+0000","last_became_peered":"2023-11-25T04:25:52.924795+0000","last_unstale":"2023-11-25T04:25:59.978829+0000","last_undegraded":"2023-11-25T04:25:59.978829+0000","last_fullsized":"2023-11-25T04:25:59.978829+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:30:46.532141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":128288,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":167,"num_write_kb":267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":120265,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.24","version":"54'151","reported_seq":427,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978903+0000","last_change":"2023-11-25T04:25:50.922751+0000","last_active":"2023-11-25T04:25:59.978903+0000","last_peered":"2023-11-25T04:25:59.978903+0000","last_clean":"2023-11-25T04:25:59.978903+0000","last_became_active":"2023-11-25T04:25:50.922216+0000","last_became_peered":"2023-11-25T04:25:50.922216+0000","last_unstale":"2023-11-25T04:25:59.978903+0000","last_undegraded":"2023-11-25T04:25:59.978903+0000","last_fullsized":"2023-11-25T04:25:59.978903+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:40:16.255665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":107507,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":148,"num_write_kb":288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":107494,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.23","version":"54'155","reported_seq":223,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978207+0000","last_change":"2023-11-25T04:22:47.037668+0000","last_active":"2023-11-25T04:25:59.978207+0000","last_peered":"2023-11-25T04:25:59.978207+0000","last_clean":"2023-11-25T04:25:59.978207+0000","last_became_active":"2023-11-25T04:22:47.036823+0000","last_became_peered":"2023-11-25T04:22:47.036823+0000","last_unstale":"2023-11-25T04:25:59.978207+0000","last_undegraded":"2023-11-25T04:25:59.978207+0000","last_fullsized":"2023-11-25T04:25:59.978207+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:02:07.088241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4304582,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":152,"num_write_kb":4331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.22","version":"54'189","reported_seq":517,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978625+0000","last_change":"2023-11-25T04:25:54.940559+0000","last_active":"2023-11-25T04:25:59.978625+0000","last_peered":"2023-11-25T04:25:59.978625+0000","last_clean":"2023-11-25T04:25:59.978625+0000","last_became_active":"2023-11-25T04:25:54.939584+0000","last_became_peered":"2023-11-25T04:25:54.939584+0000","last_unstale":"2023-11-25T04:25:59.978625+0000","last_undegraded":"2023-11-25T04:25:59.978625+0000","last_fullsized":"2023-11-25T04:25:59.978625+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:01:39.413986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":129145,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":185,"num_write_kb":432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":121,"num_bytes_recovered":129107,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"54'201","reported_seq":269,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.977881+0000","last_change":"2023-11-25T04:22:47.037786+0000","last_active":"2023-11-25T04:25:59.977881+0000","last_peered":"2023-11-25T04:25:59.977881+0000","last_clean":"2023-11-25T04:25:59.977881+0000","last_became_active":"2023-11-25T04:22:47.037206+0000","last_became_peered":"2023-11-25T04:22:47.037206+0000","last_unstale":"2023-11-25T04:25:59.977881+0000","last_undegraded":"2023-11-25T04:25:59.977881+0000","last_fullsized":"2023-11-25T04:25:59.977881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:11:40.532140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":215311,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":194,"num_write_kb":381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"54'180","reported_seq":245,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.977884+0000","last_change":"2023-11-25T04:25:53.933176+0000","last_active":"2023-11-25T04:25:59.977884+0000","last_peered":"2023-11-25T04:25:59.977884+0000","last_clean":"2023-11-25T04:25:59.977884+0000","last_became_active":"2023-11-25T04:25:53.931937+0000","last_became_peered":"2023-11-25T04:25:53.931937+0000","last_unstale":"2023-11-25T04:25:59.977884+0000","last_undegraded":"2023-11-25T04:25:59.977884+0000","last_fullsized":"2023-11-25T04:25:59.977884+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:41:10.697064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104603,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":172,"num_write_kb":267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":104603,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"54'137","reported_seq":205,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978811+0000","last_change":"2023-11-25T04:22:47.036689+0000","last_active":"2023-11-25T04:25:59.978811+0000","last_peered":"2023-11-25T04:25:59.978811+0000","last_clean":"2023-11-25T04:25:59.978811+0000","last_became_active":"2023-11-25T04:22:47.036107+0000","last_became_peered":"2023-11-25T04:22:47.036107+0000","last_unstale":"2023-11-25T04:25:59.978811+0000","last_undegraded":"2023-11-25T04:25:59.978811+0000","last_fullsized":"2023-11-25T04:25:59.978811+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:02:07.088241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":183586,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":127,"num_write_kb":284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"58'114","reported_seq":191,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978227+0000","last_change":"2023-11-25T04:25:49.920320+0000","last_active":"2023-11-25T04:25:59.978227+0000","last_peered":"2023-11-25T04:25:59.978227+0000","last_clean":"2023-11-25T04:25:59.978227+0000","last_became_active":"2023-11-25T04:25:49.920010+0000","last_became_peered":"2023-11-25T04:25:49.920010+0000","last_unstale":"2023-11-25T04:25:59.978227+0000","last_undegraded":"2023-11-25T04:25:59.978227+0000","last_fullsized":"2023-11-25T04:25:59.978227+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:48:58.746789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2816216,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":113,"num_write_kb":2816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":2816182,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"54'145","reported_seq":213,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978995+0000","last_change":"2023-11-25T04:22:47.037888+0000","last_active":"2023-11-25T04:25:59.978995+0000","last_peered":"2023-11-25T04:25:59.978995+0000","last_clean":"2023-11-25T04:25:59.978995+0000","last_became_active":"2023-11-25T04:22:47.037158+0000","last_became_peered":"2023-11-25T04:22:47.037158+0000","last_unstale":"2023-11-25T04:25:59.978995+0000","last_undegraded":"2023-11-25T04:25:59.978995+0000","last_fullsized":"2023-11-25T04:25:59.978995+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:56:59.689578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":205976,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.12","version":"49'2","reported_seq":82,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978301+0000","last_change":"2023-11-25T04:25:47.000916+0000","last_active":"2023-11-25T04:25:59.978301+0000","last_peered":"2023-11-25T04:25:59.978301+0000","last_clean":"2023-11-25T04:25:59.978301+0000","last_became_active":"2023-11-25T04:25:47.000516+0000","last_became_peered":"2023-11-25T04:25:47.000516+0000","last_unstale":"2023-11-25T04:25:59.978301+0000","last_undegraded":"2023-11-25T04:25:59.978301+0000","last_fullsized":"2023-11-25T04:25:59.978301+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:53:58.873447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"54'175","reported_seq":489,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978855+0000","last_change":"2023-11-25T04:25:52.923694+0000","last_active":"2023-11-25T04:25:59.978855+0000","last_peered":"2023-11-25T04:25:59.978855+0000","last_clean":"2023-11-25T04:25:59.978855+0000","last_became_active":"2023-11-25T04:25:52.923045+0000","last_became_peered":"2023-11-25T04:25:52.923045+0000","last_unstale":"2023-11-25T04:25:59.978855+0000","last_undegraded":"2023-11-25T04:25:59.978855+0000","last_fullsized":"2023-11-25T04:25:59.978855+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:46:12.914433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":171823,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":185,"num_write_kb":340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":166387,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":80,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978355+0000","last_change":"2023-11-25T04:25:46.994324+0000","last_active":"2023-11-25T04:25:59.978355+0000","last_peered":"2023-11-25T04:25:59.978355+0000","last_clean":"2023-11-25T04:25:59.978355+0000","last_became_active":"2023-11-25T04:25:46.993870+0000","last_became_peered":"2023-11-25T04:25:46.993870+0000","last_unstale":"2023-11-25T04:25:59.978355+0000","last_undegraded":"2023-11-25T04:25:59.978355+0000","last_fullsized":"2023-11-25T04:25:59.978355+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:51:09.827080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"54'152","reported_seq":435,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978827+0000","last_change":"2023-11-25T04:25:50.922922+0000","last_active":"2023-11-25T04:25:59.978827+0000","last_peered":"2023-11-25T04:25:59.978827+0000","last_clean":"2023-11-25T04:25:59.978827+0000","last_became_active":"2023-11-25T04:25:50.922511+0000","last_became_peered":"2023-11-25T04:25:50.922511+0000","last_unstale":"2023-11-25T04:25:59.978827+0000","last_undegraded":"2023-11-25T04:25:59.978827+0000","last_fullsized":"2023-11-25T04:25:59.978827+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:37:14.620534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":579704,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":97,"num_bytes_recovered":570044,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.10","version":"18'8","reported_seq":39,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.977878+0000","last_change":"2023-11-25T04:25:47.000499+0000","last_active":"2023-11-25T04:25:59.977878+0000","last_peered":"2023-11-25T04:25:59.977878+0000","last_clean":"2023-11-25T04:25:59.977878+0000","last_became_active":"2023-11-25T04:25:46.998778+0000","last_became_peered":"2023-11-25T04:25:46.998778+0000","last_unstale":"2023-11-25T04:25:59.977878+0000","last_undegraded":"2023-11-25T04:25:59.977878+0000","last_fullsized":"2023-11-25T04:25:59.977878+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:33:09.169689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"49'2","reported_seq":37,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979103+0000","last_change":"2023-11-25T04:25:46.998392+0000","last_active":"2023-11-25T04:25:59.979103+0000","last_peered":"2023-11-25T04:25:59.979103+0000","last_clean":"2023-11-25T04:25:59.979103+0000","last_became_active":"2023-11-25T04:25:46.997108+0000","last_became_peered":"2023-11-25T04:25:46.997108+0000","last_unstale":"2023-11-25T04:25:59.979103+0000","last_undegraded":"2023-11-25T04:25:59.979103+0000","last_fullsized":"2023-11-25T04:25:59.979103+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:02:00.073630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"54'159","reported_seq":443,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978233+0000","last_change":"2023-11-25T04:25:50.922282+0000","last_active":"2023-11-25T04:25:59.978233+0000","last_peered":"2023-11-25T04:25:59.978233+0000","last_clean":"2023-11-25T04:25:59.978233+0000","last_became_active":"2023-11-25T04:25:50.921772+0000","last_became_peered":"2023-11-25T04:25:50.921772+0000","last_unstale":"2023-11-25T04:25:59.978233+0000","last_undegraded":"2023-11-25T04:25:59.978233+0000","last_fullsized":"2023-11-25T04:25:59.978233+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:36:33.451689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112103,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":164,"num_write_kb":240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":99,"num_bytes_recovered":103472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"18'2","reported_seq":82,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979281+0000","last_change":"2023-11-25T04:25:46.994943+0000","last_active":"2023-11-25T04:25:59.979281+0000","last_peered":"2023-11-25T04:25:59.979281+0000","last_clean":"2023-11-25T04:25:59.979281+0000","last_became_active":"2023-11-25T04:25:46.994640+0000","last_became_peered":"2023-11-25T04:25:46.994640+0000","last_unstale":"2023-11-25T04:25:59.979281+0000","last_undegraded":"2023-11-25T04:25:59.979281+0000","last_fullsized":"2023-11-25T04:25:59.979281+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:08:12.601941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"54'173","reported_seq":241,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978244+0000","last_change":"2023-11-25T04:22:47.039685+0000","last_active":"2023-11-25T04:25:59.978244+0000","last_peered":"2023-11-25T04:25:59.978244+0000","last_clean":"2023-11-25T04:25:59.978244+0000","last_became_active":"2023-11-25T04:22:47.039296+0000","last_became_peered":"2023-11-25T04:22:47.039296+0000","last_unstale":"2023-11-25T04:25:59.978244+0000","last_undegraded":"2023-11-25T04:25:59.978244+0000","last_fullsized":"2023-11-25T04:25:59.978244+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:11:40.532140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154533,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":182,"num_write_kb":299,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"18'1","reported_seq":37,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978725+0000","last_change":"2023-11-25T04:25:47.000059+0000","last_active":"2023-11-25T04:25:59.978725+0000","last_peered":"2023-11-25T04:25:59.978725+0000","last_clean":"2023-11-25T04:25:59.978725+0000","last_became_active":"2023-11-25T04:25:46.999604+0000","last_became_peered":"2023-11-25T04:25:46.999604+0000","last_unstale":"2023-11-25T04:25:59.978725+0000","last_undegraded":"2023-11-25T04:25:59.978725+0000","last_fullsized":"2023-11-25T04:25:59.978725+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:14:43.431272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"54'147","reported_seq":211,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978982+0000","last_change":"2023-11-25T04:25:54.939274+0000","last_active":"2023-11-25T04:25:59.978982+0000","last_peered":"2023-11-25T04:25:59.978982+0000","last_clean":"2023-11-25T04:25:59.978982+0000","last_became_active":"2023-11-25T04:25:54.938236+0000","last_became_peered":"2023-11-25T04:25:54.938236+0000","last_unstale":"2023-11-25T04:25:59.978982+0000","last_undegraded":"2023-11-25T04:25:59.978982+0000","last_fullsized":"2023-11-25T04:25:59.978982+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":147,"log_dups_size":0,"ondisk_log_size":147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:11:43.785294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":87524,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":155,"num_write_kb":235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":87524,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978546+0000","last_change":"2023-11-25T04:25:46.997080+0000","last_active":"2023-11-25T04:25:59.978546+0000","last_peered":"2023-11-25T04:25:59.978546+0000","last_clean":"2023-11-25T04:25:59.978546+0000","last_became_active":"2023-11-25T04:25:46.995712+0000","last_became_peered":"2023-11-25T04:25:46.995712+0000","last_unstale":"2023-11-25T04:25:59.978546+0000","last_undegraded":"2023-11-25T04:25:59.978546+0000","last_fullsized":"2023-11-25T04:25:59.978546+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T16:01:00.922843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"54'155","reported_seq":441,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978274+0000","last_change":"2023-11-25T04:25:52.923096+0000","last_active":"2023-11-25T04:25:59.978274+0000","last_peered":"2023-11-25T04:25:59.978274+0000","last_clean":"2023-11-25T04:25:59.978274+0000","last_became_active":"2023-11-25T04:25:52.922549+0000","last_became_peered":"2023-11-25T04:25:52.922549+0000","last_unstale":"2023-11-25T04:25:59.978274+0000","last_undegraded":"2023-11-25T04:25:59.978274+0000","last_fullsized":"2023-11-25T04:25:59.978274+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:52:38.053808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":266919,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":264258,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"45'11","reported_seq":96,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978532+0000","last_change":"2023-11-25T04:25:47.092497+0000","last_active":"2023-11-25T04:25:59.978532+0000","last_peered":"2023-11-25T04:25:59.978532+0000","last_clean":"2023-11-25T04:25:59.978532+0000","last_became_active":"2023-11-25T04:25:47.000687+0000","last_became_peered":"2023-11-25T04:25:47.000687+0000","last_unstale":"2023-11-25T04:25:59.978532+0000","last_undegraded":"2023-11-25T04:25:59.978532+0000","last_fullsized":"2023-11-25T04:25:59.978532+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:47:09.798238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":4101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4194304,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"54'173","reported_seq":241,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978732+0000","last_change":"2023-11-25T04:25:50.923240+0000","last_active":"2023-11-25T04:25:59.978732+0000","last_peered":"2023-11-25T04:25:59.978732+0000","last_clean":"2023-11-25T04:25:59.978732+0000","last_became_active":"2023-11-25T04:25:50.922409+0000","last_became_peered":"2023-11-25T04:25:50.922409+0000","last_unstale":"2023-11-25T04:25:59.978732+0000","last_undegraded":"2023-11-25T04:25:59.978732+0000","last_fullsized":"2023-11-25T04:25:59.978732+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:53:30.920627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":553893,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":169,"num_write_kb":770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":553893,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978079+0000","last_change":"2023-11-25T04:25:46.998818+0000","last_active":"2023-11-25T04:25:59.978079+0000","last_peered":"2023-11-25T04:25:59.978079+0000","last_clean":"2023-11-25T04:25:59.978079+0000","last_became_active":"2023-11-25T04:25:46.997019+0000","last_became_peered":"2023-11-25T04:25:46.997019+0000","last_unstale":"2023-11-25T04:25:59.978079+0000","last_undegraded":"2023-11-25T04:25:59.978079+0000","last_fullsized":"2023-11-25T04:25:59.978079+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:51:47.882117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"54'163","reported_seq":228,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978685+0000","last_change":"2023-11-25T04:25:53.934183+0000","last_active":"2023-11-25T04:25:59.978685+0000","last_peered":"2023-11-25T04:25:59.978685+0000","last_clean":"2023-11-25T04:25:59.978685+0000","last_became_active":"2023-11-25T04:25:53.933471+0000","last_became_peered":"2023-11-25T04:25:53.933471+0000","last_unstale":"2023-11-25T04:25:59.978685+0000","last_undegraded":"2023-11-25T04:25:59.978685+0000","last_fullsized":"2023-11-25T04:25:59.978685+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:57:07.515414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":320867,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":190,"num_write_kb":577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":108,"num_bytes_recovered":311640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":80,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978412+0000","last_change":"2023-11-25T04:25:46.994480+0000","last_active":"2023-11-25T04:25:59.978412+0000","last_peered":"2023-11-25T04:25:59.978412+0000","last_clean":"2023-11-25T04:25:59.978412+0000","last_became_active":"2023-11-25T04:25:46.994101+0000","last_became_peered":"2023-11-25T04:25:46.994101+0000","last_unstale":"2023-11-25T04:25:59.978412+0000","last_undegraded":"2023-11-25T04:25:59.978412+0000","last_fullsized":"2023-11-25T04:25:59.978412+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:53:58.873447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"54'141","reported_seq":210,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979040+0000","last_change":"2023-11-25T04:25:49.921950+0000","last_active":"2023-11-25T04:25:59.979040+0000","last_peered":"2023-11-25T04:25:59.979040+0000","last_clean":"2023-11-25T04:25:59.979040+0000","last_became_active":"2023-11-25T04:25:49.920971+0000","last_became_peered":"2023-11-25T04:25:49.920971+0000","last_unstale":"2023-11-25T04:25:59.979040+0000","last_undegraded":"2023-11-25T04:25:59.979040+0000","last_fullsized":"2023-11-25T04:25:59.979040+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:22:38.004934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3502371,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":3718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":89,"num_bytes_recovered":3497669,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"49'1","reported_seq":70,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978352+0000","last_change":"2023-11-25T04:22:46.034509+0000","last_active":"2023-11-25T04:25:59.978352+0000","last_peered":"2023-11-25T04:25:59.978352+0000","last_clean":"2023-11-25T04:25:59.978352+0000","last_became_active":"2023-11-25T04:22:46.034158+0000","last_became_peered":"2023-11-25T04:22:46.034158+0000","last_unstale":"2023-11-25T04:25:59.978352+0000","last_undegraded":"2023-11-25T04:25:59.978352+0000","last_fullsized":"2023-11-25T04:25:59.978352+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:26:25.338117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"54'141","reported_seq":209,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978912+0000","last_change":"2023-11-25T04:22:47.038800+0000","last_active":"2023-11-25T04:25:59.978912+0000","last_peered":"2023-11-25T04:25:59.978912+0000","last_clean":"2023-11-25T04:25:59.978912+0000","last_became_active":"2023-11-25T04:22:47.038081+0000","last_became_peered":"2023-11-25T04:22:47.038081+0000","last_unstale":"2023-11-25T04:25:59.978912+0000","last_undegraded":"2023-11-25T04:25:59.978912+0000","last_fullsized":"2023-11-25T04:25:59.978912+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:45:40.065146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":174088,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":146,"num_write_kb":385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"12'157","reported_seq":232,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.977771+0000","last_change":"2023-11-25T04:25:47.000504+0000","last_active":"2023-11-25T04:25:59.977771+0000","last_peered":"2023-11-25T04:25:59.977771+0000","last_clean":"2023-11-25T04:25:59.977771+0000","last_became_active":"2023-11-25T04:25:46.999721+0000","last_became_peered":"2023-11-25T04:25:46.999721+0000","last_unstale":"2023-11-25T04:25:59.977771+0000","last_undegraded":"2023-11-25T04:25:59.977771+0000","last_fullsized":"2023-11-25T04:25:59.977771+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:35.919850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:35.919850+0000","last_clean_scrub_stamp":"2023-11-25T04:22:35.919850+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:26:33.607474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978082+0000","last_change":"2023-11-25T04:25:46.998960+0000","last_active":"2023-11-25T04:25:59.978082+0000","last_peered":"2023-11-25T04:25:59.978082+0000","last_clean":"2023-11-25T04:25:59.978082+0000","last_became_active":"2023-11-25T04:25:46.997693+0000","last_became_peered":"2023-11-25T04:25:46.997693+0000","last_unstale":"2023-11-25T04:25:59.978082+0000","last_undegraded":"2023-11-25T04:25:59.978082+0000","last_fullsized":"2023-11-25T04:25:59.978082+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:51:45.180957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"46'1","reported_seq":70,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978331+0000","last_change":"2023-11-25T04:22:46.037044+0000","last_active":"2023-11-25T04:25:59.978331+0000","last_peered":"2023-11-25T04:25:59.978331+0000","last_clean":"2023-11-25T04:25:59.978331+0000","last_became_active":"2023-11-25T04:22:46.035916+0000","last_became_peered":"2023-11-25T04:22:46.035916+0000","last_unstale":"2023-11-25T04:25:59.978331+0000","last_undegraded":"2023-11-25T04:25:59.978331+0000","last_fullsized":"2023-11-25T04:25:59.978331+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:38:20.501102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"54'146","reported_seq":213,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978804+0000","last_change":"2023-11-25T04:25:51.923452+0000","last_active":"2023-11-25T04:25:59.978804+0000","last_peered":"2023-11-25T04:25:59.978804+0000","last_clean":"2023-11-25T04:25:59.978804+0000","last_became_active":"2023-11-25T04:25:51.922490+0000","last_became_peered":"2023-11-25T04:25:51.922490+0000","last_unstale":"2023-11-25T04:25:59.978804+0000","last_undegraded":"2023-11-25T04:25:59.978804+0000","last_fullsized":"2023-11-25T04:25:59.978804+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:40:11.001062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":502154,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":170,"num_write_kb":687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":99,"num_bytes_recovered":502101,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978713+0000","last_change":"2023-11-25T04:25:47.000763+0000","last_active":"2023-11-25T04:25:59.978713+0000","last_peered":"2023-11-25T04:25:59.978713+0000","last_clean":"2023-11-25T04:25:59.978713+0000","last_became_active":"2023-11-25T04:25:47.000033+0000","last_became_peered":"2023-11-25T04:25:47.000033+0000","last_unstale":"2023-11-25T04:25:59.978713+0000","last_undegraded":"2023-11-25T04:25:59.978713+0000","last_fullsized":"2023-11-25T04:25:59.978713+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:53:59.131478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"50'37","reported_seq":156,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978116+0000","last_change":"2023-11-25T04:25:47.002009+0000","last_active":"2023-11-25T04:25:59.978116+0000","last_peered":"2023-11-25T04:25:59.978116+0000","last_clean":"2023-11-25T04:25:59.978116+0000","last_became_active":"2023-11-25T04:25:46.995637+0000","last_became_peered":"2023-11-25T04:25:46.995637+0000","last_unstale":"2023-11-25T04:25:59.978116+0000","last_undegraded":"2023-11-25T04:25:59.978116+0000","last_fullsized":"2023-11-25T04:25:59.978116+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:55:01.351319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":319160,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":35,"num_read_kb":334,"num_write":35,"num_write_kb":334,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":319160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"54'158","reported_seq":226,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979092+0000","last_change":"2023-11-25T04:22:47.039583+0000","last_active":"2023-11-25T04:25:59.979092+0000","last_peered":"2023-11-25T04:25:59.979092+0000","last_clean":"2023-11-25T04:25:59.979092+0000","last_became_active":"2023-11-25T04:22:47.039197+0000","last_became_peered":"2023-11-25T04:22:47.039197+0000","last_unstale":"2023-11-25T04:25:59.979092+0000","last_undegraded":"2023-11-25T04:25:59.979092+0000","last_fullsized":"2023-11-25T04:25:59.979092+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:37:30.959147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":169303,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":189,"num_write_kb":333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":72,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978222+0000","last_change":"2023-11-25T04:22:42.000382+0000","last_active":"2023-11-25T04:25:59.978222+0000","last_peered":"2023-11-25T04:25:59.978222+0000","last_clean":"2023-11-25T04:25:59.978222+0000","last_became_active":"2023-11-25T04:22:42.000041+0000","last_became_peered":"2023-11-25T04:22:42.000041+0000","last_unstale":"2023-11-25T04:25:59.978222+0000","last_undegraded":"2023-11-25T04:25:59.978222+0000","last_fullsized":"2023-11-25T04:25:59.978222+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:08:19.655864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"49'71","reported_seq":155,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978346+0000","last_change":"2023-11-25T04:25:47.109628+0000","last_active":"2023-11-25T04:25:59.978346+0000","last_peered":"2023-11-25T04:25:59.978346+0000","last_clean":"2023-11-25T04:25:59.978346+0000","last_became_active":"2023-11-25T04:25:47.001162+0000","last_became_peered":"2023-11-25T04:25:47.001162+0000","last_unstale":"2023-11-25T04:25:59.978346+0000","last_undegraded":"2023-11-25T04:25:59.978346+0000","last_fullsized":"2023-11-25T04:25:59.978346+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:03:33.658392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":69,"num_write_kb":4130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4194304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"54'158","reported_seq":226,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978945+0000","last_change":"2023-11-25T04:22:47.038454+0000","last_active":"2023-11-25T04:25:59.978945+0000","last_peered":"2023-11-25T04:25:59.978945+0000","last_clean":"2023-11-25T04:25:59.978945+0000","last_became_active":"2023-11-25T04:22:47.038046+0000","last_became_peered":"2023-11-25T04:22:47.038046+0000","last_unstale":"2023-11-25T04:25:59.978945+0000","last_undegraded":"2023-11-25T04:25:59.978945+0000","last_fullsized":"2023-11-25T04:25:59.978945+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:38:38.839941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":115501,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":163,"num_write_kb":243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.977941+0000","last_change":"2023-11-25T04:25:46.998739+0000","last_active":"2023-11-25T04:25:59.977941+0000","last_peered":"2023-11-25T04:25:59.977941+0000","last_clean":"2023-11-25T04:25:59.977941+0000","last_became_active":"2023-11-25T04:25:46.997391+0000","last_became_peered":"2023-11-25T04:25:46.997391+0000","last_unstale":"2023-11-25T04:25:59.977941+0000","last_undegraded":"2023-11-25T04:25:59.977941+0000","last_fullsized":"2023-11-25T04:25:59.977941+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:47:46.300596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"50'1","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978166+0000","last_change":"2023-11-25T04:25:46.999446+0000","last_active":"2023-11-25T04:25:59.978166+0000","last_peered":"2023-11-25T04:25:59.978166+0000","last_clean":"2023-11-25T04:25:59.978166+0000","last_became_active":"2023-11-25T04:25:46.998132+0000","last_became_peered":"2023-11-25T04:25:46.998132+0000","last_unstale":"2023-11-25T04:25:59.978166+0000","last_undegraded":"2023-11-25T04:25:59.978166+0000","last_fullsized":"2023-11-25T04:25:59.978166+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:42:51.166678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"54'157","reported_seq":225,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978638+0000","last_change":"2023-11-25T04:22:47.037355+0000","last_active":"2023-11-25T04:25:59.978638+0000","last_peered":"2023-11-25T04:25:59.978638+0000","last_clean":"2023-11-25T04:25:59.978638+0000","last_became_active":"2023-11-25T04:22:47.036924+0000","last_became_peered":"2023-11-25T04:22:47.036924+0000","last_unstale":"2023-11-25T04:25:59.978638+0000","last_undegraded":"2023-11-25T04:25:59.978638+0000","last_fullsized":"2023-11-25T04:25:59.978638+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:15:39.511043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":93321,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":148,"num_write_kb":360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978520+0000","last_change":"2023-11-25T04:25:46.999738+0000","last_active":"2023-11-25T04:25:59.978520+0000","last_peered":"2023-11-25T04:25:59.978520+0000","last_clean":"2023-11-25T04:25:59.978520+0000","last_became_active":"2023-11-25T04:25:46.999403+0000","last_became_peered":"2023-11-25T04:25:46.999403+0000","last_unstale":"2023-11-25T04:25:59.978520+0000","last_undegraded":"2023-11-25T04:25:59.978520+0000","last_fullsized":"2023-11-25T04:25:59.978520+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:45:53.304063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":72,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978705+0000","last_change":"2023-11-25T04:22:42.000204+0000","last_active":"2023-11-25T04:25:59.978705+0000","last_peered":"2023-11-25T04:25:59.978705+0000","last_clean":"2023-11-25T04:25:59.978705+0000","last_became_active":"2023-11-25T04:22:41.999915+0000","last_became_peered":"2023-11-25T04:22:41.999915+0000","last_unstale":"2023-11-25T04:25:59.978705+0000","last_undegraded":"2023-11-25T04:25:59.978705+0000","last_fullsized":"2023-11-25T04:25:59.978705+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:07:32.457905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"54'152","reported_seq":432,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978379+0000","last_change":"2023-11-25T04:25:53.933239+0000","last_active":"2023-11-25T04:25:59.978379+0000","last_peered":"2023-11-25T04:25:59.978379+0000","last_clean":"2023-11-25T04:25:59.978379+0000","last_became_active":"2023-11-25T04:25:53.932586+0000","last_became_peered":"2023-11-25T04:25:53.932586+0000","last_unstale":"2023-11-25T04:25:59.978379+0000","last_undegraded":"2023-11-25T04:25:59.978379+0000","last_fullsized":"2023-11-25T04:25:59.978379+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:54:51.339025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":329327,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":151,"num_write_kb":471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":97,"num_bytes_recovered":329278,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":69,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978632+0000","last_change":"2023-11-25T04:22:46.035885+0000","last_active":"2023-11-25T04:25:59.978632+0000","last_peered":"2023-11-25T04:25:59.978632+0000","last_clean":"2023-11-25T04:25:59.978632+0000","last_became_active":"2023-11-25T04:22:46.035405+0000","last_became_peered":"2023-11-25T04:22:46.035405+0000","last_unstale":"2023-11-25T04:25:59.978632+0000","last_undegraded":"2023-11-25T04:25:59.978632+0000","last_fullsized":"2023-11-25T04:25:59.978632+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:53:47.398746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":72,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979221+0000","last_change":"2023-11-25T04:22:41.999067+0000","last_active":"2023-11-25T04:25:59.979221+0000","last_peered":"2023-11-25T04:25:59.979221+0000","last_clean":"2023-11-25T04:25:59.979221+0000","last_became_active":"2023-11-25T04:22:41.998798+0000","last_became_peered":"2023-11-25T04:22:41.998798+0000","last_unstale":"2023-11-25T04:25:59.979221+0000","last_undegraded":"2023-11-25T04:25:59.979221+0000","last_fullsized":"2023-11-25T04:25:59.979221+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:00:04.266051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"54'172","reported_seq":470,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978595+0000","last_change":"2023-11-25T04:25:51.922403+0000","last_active":"2023-11-25T04:25:59.978595+0000","last_peered":"2023-11-25T04:25:59.978595+0000","last_clean":"2023-11-25T04:25:59.978595+0000","last_became_active":"2023-11-25T04:25:51.921858+0000","last_became_peered":"2023-11-25T04:25:51.921858+0000","last_unstale":"2023-11-25T04:25:59.978595+0000","last_undegraded":"2023-11-25T04:25:59.978595+0000","last_fullsized":"2023-11-25T04:25:59.978595+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:04:58.273982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":129965,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":166,"num_write_kb":290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":129965,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"18'2","reported_seq":71,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979088+0000","last_change":"2023-11-25T04:22:46.034606+0000","last_active":"2023-11-25T04:25:59.979088+0000","last_peered":"2023-11-25T04:25:59.979088+0000","last_clean":"2023-11-25T04:25:59.979088+0000","last_became_active":"2023-11-25T04:22:46.034229+0000","last_became_peered":"2023-11-25T04:22:46.034229+0000","last_unstale":"2023-11-25T04:25:59.979088+0000","last_undegraded":"2023-11-25T04:25:59.979088+0000","last_fullsized":"2023-11-25T04:25:59.979088+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:21:42.146264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":72,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978952+0000","last_change":"2023-11-25T04:22:42.000258+0000","last_active":"2023-11-25T04:25:59.978952+0000","last_peered":"2023-11-25T04:25:59.978952+0000","last_clean":"2023-11-25T04:25:59.978952+0000","last_became_active":"2023-11-25T04:22:41.999992+0000","last_became_peered":"2023-11-25T04:22:41.999992+0000","last_unstale":"2023-11-25T04:25:59.978952+0000","last_undegraded":"2023-11-25T04:25:59.978952+0000","last_fullsized":"2023-11-25T04:25:59.978952+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:31:13.715992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"54'150","reported_seq":436,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.977926+0000","last_change":"2023-11-25T04:25:54.938479+0000","last_active":"2023-11-25T04:25:59.977926+0000","last_peered":"2023-11-25T04:25:59.977926+0000","last_clean":"2023-11-25T04:25:59.977926+0000","last_became_active":"2023-11-25T04:25:54.937838+0000","last_became_peered":"2023-11-25T04:25:54.937838+0000","last_unstale":"2023-11-25T04:25:59.977926+0000","last_undegraded":"2023-11-25T04:25:59.977926+0000","last_fullsized":"2023-11-25T04:25:59.977926+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:51:14.394190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":159585,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":154224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"54'14","reported_seq":99,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978916+0000","last_change":"2023-11-25T04:25:47.026749+0000","last_active":"2023-11-25T04:25:59.978916+0000","last_peered":"2023-11-25T04:25:59.978916+0000","last_clean":"2023-11-25T04:25:59.978916+0000","last_became_active":"2023-11-25T04:25:46.997404+0000","last_became_peered":"2023-11-25T04:25:46.997404+0000","last_unstale":"2023-11-25T04:25:59.978916+0000","last_undegraded":"2023-11-25T04:25:59.978916+0000","last_fullsized":"2023-11-25T04:25:59.978916+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:20:31.719529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978930+0000","last_change":"2023-11-25T04:25:46.995200+0000","last_active":"2023-11-25T04:25:59.978930+0000","last_peered":"2023-11-25T04:25:59.978930+0000","last_clean":"2023-11-25T04:25:59.978930+0000","last_became_active":"2023-11-25T04:25:46.994834+0000","last_became_peered":"2023-11-25T04:25:46.994834+0000","last_unstale":"2023-11-25T04:25:59.978930+0000","last_undegraded":"2023-11-25T04:25:59.978930+0000","last_fullsized":"2023-11-25T04:25:59.978930+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:33:07.913911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"54'135","reported_seq":407,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978276+0000","last_change":"2023-11-25T04:25:49.922046+0000","last_active":"2023-11-25T04:25:59.978276+0000","last_peered":"2023-11-25T04:25:59.978276+0000","last_clean":"2023-11-25T04:25:59.978276+0000","last_became_active":"2023-11-25T04:25:49.921465+0000","last_became_peered":"2023-11-25T04:25:49.921465+0000","last_unstale":"2023-11-25T04:25:59.978276+0000","last_undegraded":"2023-11-25T04:25:59.978276+0000","last_fullsized":"2023-11-25T04:25:59.978276+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:12:54.077901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":99503,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":148,"num_write_kb":250,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":93,"num_bytes_recovered":99503,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"54'166","reported_seq":233,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978407+0000","last_change":"2023-11-25T04:25:51.923141+0000","last_active":"2023-11-25T04:25:59.978407+0000","last_peered":"2023-11-25T04:25:59.978407+0000","last_clean":"2023-11-25T04:25:59.978407+0000","last_became_active":"2023-11-25T04:25:51.922070+0000","last_became_peered":"2023-11-25T04:25:51.922070+0000","last_unstale":"2023-11-25T04:25:59.978407+0000","last_undegraded":"2023-11-25T04:25:59.978407+0000","last_fullsized":"2023-11-25T04:25:59.978407+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:34:53.021081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":185321,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":172,"num_write_kb":312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":164176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"58'50","reported_seq":53,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978227+0000","last_change":"2023-11-25T04:25:47.021705+0000","last_active":"2023-11-25T04:25:59.978227+0000","last_peered":"2023-11-25T04:25:59.978227+0000","last_clean":"2023-11-25T04:25:59.978227+0000","last_became_active":"2023-11-25T04:25:46.994997+0000","last_became_peered":"2023-11-25T04:25:46.994997+0000","last_unstale":"2023-11-25T04:25:59.978227+0000","last_undegraded":"2023-11-25T04:25:59.978227+0000","last_fullsized":"2023-11-25T04:25:59.978227+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:41:04.147372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":4119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4190183,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"54'153","reported_seq":221,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978107+0000","last_change":"2023-11-25T04:22:47.037181+0000","last_active":"2023-11-25T04:25:59.978107+0000","last_peered":"2023-11-25T04:25:59.978107+0000","last_clean":"2023-11-25T04:25:59.978107+0000","last_became_active":"2023-11-25T04:22:47.036821+0000","last_became_peered":"2023-11-25T04:22:47.036821+0000","last_unstale":"2023-11-25T04:25:59.978107+0000","last_undegraded":"2023-11-25T04:25:59.978107+0000","last_fullsized":"2023-11-25T04:25:59.978107+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:12:51.990448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":101117,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":161,"num_write_kb":242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"50'22","reported_seq":107,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979124+0000","last_change":"2023-11-25T04:25:47.028636+0000","last_active":"2023-11-25T04:25:59.979124+0000","last_peered":"2023-11-25T04:25:59.979124+0000","last_clean":"2023-11-25T04:25:59.979124+0000","last_became_active":"2023-11-25T04:25:47.000478+0000","last_became_peered":"2023-11-25T04:25:47.000478+0000","last_unstale":"2023-11-25T04:25:59.979124+0000","last_undegraded":"2023-11-25T04:25:59.979124+0000","last_fullsized":"2023-11-25T04:25:59.979124+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:50:13.896985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":90,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978907+0000","last_change":"2023-11-25T04:25:46.996182+0000","last_active":"2023-11-25T04:25:59.978907+0000","last_peered":"2023-11-25T04:25:59.978907+0000","last_clean":"2023-11-25T04:25:59.978907+0000","last_became_active":"2023-11-25T04:25:46.994525+0000","last_became_peered":"2023-11-25T04:25:46.994525+0000","last_unstale":"2023-11-25T04:25:59.978907+0000","last_undegraded":"2023-11-25T04:25:59.978907+0000","last_fullsized":"2023-11-25T04:25:59.978907+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:26:24.469530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"54'177","reported_seq":485,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.977915+0000","last_change":"2023-11-25T04:25:51.922466+0000","last_active":"2023-11-25T04:25:59.977915+0000","last_peered":"2023-11-25T04:25:59.977915+0000","last_clean":"2023-11-25T04:25:59.977915+0000","last_became_active":"2023-11-25T04:25:51.921938+0000","last_became_peered":"2023-11-25T04:25:51.921938+0000","last_unstale":"2023-11-25T04:25:59.977915+0000","last_undegraded":"2023-11-25T04:25:59.977915+0000","last_fullsized":"2023-11-25T04:25:59.977915+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:38:59.016525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":522494,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":168,"num_write_kb":749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":111,"num_bytes_recovered":522494,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"46'2","reported_seq":82,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979031+0000","last_change":"2023-11-25T04:25:46.998867+0000","last_active":"2023-11-25T04:25:59.979031+0000","last_peered":"2023-11-25T04:25:59.979031+0000","last_clean":"2023-11-25T04:25:59.979031+0000","last_became_active":"2023-11-25T04:25:46.998508+0000","last_became_peered":"2023-11-25T04:25:46.998508+0000","last_unstale":"2023-11-25T04:25:59.979031+0000","last_undegraded":"2023-11-25T04:25:59.979031+0000","last_fullsized":"2023-11-25T04:25:59.979031+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:50:13.896985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.10","version":"54'153","reported_seq":221,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.977904+0000","last_change":"2023-11-25T04:22:47.037790+0000","last_active":"2023-11-25T04:25:59.977904+0000","last_peered":"2023-11-25T04:25:59.977904+0000","last_clean":"2023-11-25T04:25:59.977904+0000","last_became_active":"2023-11-25T04:22:47.037149+0000","last_became_peered":"2023-11-25T04:22:47.037149+0000","last_unstale":"2023-11-25T04:25:59.977904+0000","last_undegraded":"2023-11-25T04:25:59.977904+0000","last_fullsized":"2023-11-25T04:25:59.977904+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:31:54.857270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":218073,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1f","version":"54'156","reported_seq":448,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979259+0000","last_change":"2023-11-25T04:25:49.921904+0000","last_active":"2023-11-25T04:25:59.979259+0000","last_peered":"2023-11-25T04:25:59.979259+0000","last_clean":"2023-11-25T04:25:59.979259+0000","last_became_active":"2023-11-25T04:25:49.921458+0000","last_became_peered":"2023-11-25T04:25:49.921458+0000","last_unstale":"2023-11-25T04:25:59.979259+0000","last_undegraded":"2023-11-25T04:25:59.979259+0000","last_fullsized":"2023-11-25T04:25:59.979259+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:59:27.602060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":107712,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":103,"num_bytes_recovered":107703,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"48'1","reported_seq":70,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978057+0000","last_change":"2023-11-25T04:22:46.036085+0000","last_active":"2023-11-25T04:25:59.978057+0000","last_peered":"2023-11-25T04:25:59.978057+0000","last_clean":"2023-11-25T04:25:59.978057+0000","last_became_active":"2023-11-25T04:22:46.035495+0000","last_became_peered":"2023-11-25T04:22:46.035495+0000","last_unstale":"2023-11-25T04:25:59.978057+0000","last_undegraded":"2023-11-25T04:25:59.978057+0000","last_fullsized":"2023-11-25T04:25:59.978057+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:14:56.735365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"54'152","reported_seq":430,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978757+0000","last_change":"2023-11-25T04:25:50.923882+0000","last_active":"2023-11-25T04:25:59.978757+0000","last_peered":"2023-11-25T04:25:59.978757+0000","last_clean":"2023-11-25T04:25:59.978757+0000","last_became_active":"2023-11-25T04:25:50.923375+0000","last_became_peered":"2023-11-25T04:25:50.923375+0000","last_unstale":"2023-11-25T04:25:59.978757+0000","last_undegraded":"2023-11-25T04:25:59.978757+0000","last_fullsized":"2023-11-25T04:25:59.978757+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:54:51.339025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":76435,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":76435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":80,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978069+0000","last_change":"2023-11-25T04:25:46.995228+0000","last_active":"2023-11-25T04:25:59.978069+0000","last_peered":"2023-11-25T04:25:59.978069+0000","last_clean":"2023-11-25T04:25:59.978069+0000","last_became_active":"2023-11-25T04:25:46.994871+0000","last_became_peered":"2023-11-25T04:25:46.994871+0000","last_unstale":"2023-11-25T04:25:59.978069+0000","last_undegraded":"2023-11-25T04:25:59.978069+0000","last_fullsized":"2023-11-25T04:25:59.978069+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:53:34.807075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"54'158","reported_seq":432,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978899+0000","last_change":"2023-11-25T04:25:52.925523+0000","last_active":"2023-11-25T04:25:59.978899+0000","last_peered":"2023-11-25T04:25:59.978899+0000","last_clean":"2023-11-25T04:25:59.978899+0000","last_became_active":"2023-11-25T04:25:52.925038+0000","last_became_peered":"2023-11-25T04:25:52.925038+0000","last_unstale":"2023-11-25T04:25:59.978899+0000","last_undegraded":"2023-11-25T04:25:59.978899+0000","last_fullsized":"2023-11-25T04:25:59.978899+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:37:14.620534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":209500,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":139,"num_write_kb":351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":209500,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1a","version":"48'4","reported_seq":73,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978665+0000","last_change":"2023-11-25T04:22:46.034340+0000","last_active":"2023-11-25T04:25:59.978665+0000","last_peered":"2023-11-25T04:25:59.978665+0000","last_clean":"2023-11-25T04:25:59.978665+0000","last_became_active":"2023-11-25T04:22:46.033944+0000","last_became_peered":"2023-11-25T04:22:46.033944+0000","last_unstale":"2023-11-25T04:25:59.978665+0000","last_undegraded":"2023-11-25T04:25:59.978665+0000","last_fullsized":"2023-11-25T04:25:59.978665+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:59:30.291004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"54'163","reported_seq":231,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978741+0000","last_change":"2023-11-25T04:22:47.039547+0000","last_active":"2023-11-25T04:25:59.978741+0000","last_peered":"2023-11-25T04:25:59.978741+0000","last_clean":"2023-11-25T04:25:59.978741+0000","last_became_active":"2023-11-25T04:22:47.039218+0000","last_became_peered":"2023-11-25T04:22:47.039218+0000","last_unstale":"2023-11-25T04:25:59.978741+0000","last_undegraded":"2023-11-25T04:25:59.978741+0000","last_fullsized":"2023-11-25T04:25:59.978741+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:44:48.553972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":165883,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"45'9","reported_seq":94,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978700+0000","last_change":"2023-11-25T04:25:47.023365+0000","last_active":"2023-11-25T04:25:59.978700+0000","last_peered":"2023-11-25T04:25:59.978700+0000","last_clean":"2023-11-25T04:25:59.978700+0000","last_became_active":"2023-11-25T04:25:46.999998+0000","last_became_peered":"2023-11-25T04:25:46.999998+0000","last_unstale":"2023-11-25T04:25:59.978700+0000","last_undegraded":"2023-11-25T04:25:59.978700+0000","last_fullsized":"2023-11-25T04:25:59.978700+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:53:34.807075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":4101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4194304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"54'149","reported_seq":217,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978573+0000","last_change":"2023-11-25T04:25:50.923041+0000","last_active":"2023-11-25T04:25:59.978573+0000","last_peered":"2023-11-25T04:25:59.978573+0000","last_clean":"2023-11-25T04:25:59.978573+0000","last_became_active":"2023-11-25T04:25:50.922164+0000","last_became_peered":"2023-11-25T04:25:50.922164+0000","last_unstale":"2023-11-25T04:25:59.978573+0000","last_undegraded":"2023-11-25T04:25:59.978573+0000","last_fullsized":"2023-11-25T04:25:59.978573+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:31:10.744814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":179506,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":155,"num_write_kb":317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":179506,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"18'2","reported_seq":37,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978409+0000","last_change":"2023-11-25T04:25:46.996675+0000","last_active":"2023-11-25T04:25:59.978409+0000","last_peered":"2023-11-25T04:25:59.978409+0000","last_clean":"2023-11-25T04:25:59.978409+0000","last_became_active":"2023-11-25T04:25:46.995160+0000","last_became_peered":"2023-11-25T04:25:46.995160+0000","last_unstale":"2023-11-25T04:25:59.978409+0000","last_undegraded":"2023-11-25T04:25:59.978409+0000","last_fullsized":"2023-11-25T04:25:59.978409+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:59:39.131605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1a","version":"54'162","reported_seq":226,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978332+0000","last_change":"2023-11-25T04:25:54.939182+0000","last_active":"2023-11-25T04:25:59.978332+0000","last_peered":"2023-11-25T04:25:59.978332+0000","last_clean":"2023-11-25T04:25:59.978332+0000","last_became_active":"2023-11-25T04:25:54.937821+0000","last_became_peered":"2023-11-25T04:25:54.937821+0000","last_unstale":"2023-11-25T04:25:59.978332+0000","last_undegraded":"2023-11-25T04:25:59.978332+0000","last_fullsized":"2023-11-25T04:25:59.978332+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:05:34.575966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":134672,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":169,"num_write_kb":311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":126291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"48'59","reported_seq":146,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.979001+0000","last_change":"2023-11-25T04:25:47.077194+0000","last_active":"2023-11-25T04:25:59.979001+0000","last_peered":"2023-11-25T04:25:59.979001+0000","last_clean":"2023-11-25T04:25:59.979001+0000","last_became_active":"2023-11-25T04:25:47.000722+0000","last_became_peered":"2023-11-25T04:25:47.000722+0000","last_unstale":"2023-11-25T04:25:59.979001+0000","last_undegraded":"2023-11-25T04:25:59.979001+0000","last_fullsized":"2023-11-25T04:25:59.979001+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:55:01.351319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8388608,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"54'163","reported_seq":231,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978500+0000","last_change":"2023-11-25T04:22:47.036764+0000","last_active":"2023-11-25T04:25:59.978500+0000","last_peered":"2023-11-25T04:25:59.978500+0000","last_clean":"2023-11-25T04:25:59.978500+0000","last_became_active":"2023-11-25T04:22:47.036406+0000","last_became_peered":"2023-11-25T04:22:47.036406+0000","last_unstale":"2023-11-25T04:25:59.978500+0000","last_undegraded":"2023-11-25T04:25:59.978500+0000","last_fullsized":"2023-11-25T04:25:59.978500+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:09:55.357642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":527891,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":163,"num_write_kb":687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1e","version":"46'2","reported_seq":71,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978868+0000","last_change":"2023-11-25T04:22:46.035659+0000","last_active":"2023-11-25T04:25:59.978868+0000","last_peered":"2023-11-25T04:25:59.978868+0000","last_clean":"2023-11-25T04:25:59.978868+0000","last_became_active":"2023-11-25T04:22:46.035113+0000","last_became_peered":"2023-11-25T04:22:46.035113+0000","last_unstale":"2023-11-25T04:25:59.978868+0000","last_undegraded":"2023-11-25T04:25:59.978868+0000","last_fullsized":"2023-11-25T04:25:59.978868+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:26:25.338117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.18","version":"54'195","reported_seq":259,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978263+0000","last_change":"2023-11-25T04:25:54.938573+0000","last_active":"2023-11-25T04:25:59.978263+0000","last_peered":"2023-11-25T04:25:59.978263+0000","last_clean":"2023-11-25T04:25:59.978263+0000","last_became_active":"2023-11-25T04:25:54.937444+0000","last_became_peered":"2023-11-25T04:25:54.937444+0000","last_unstale":"2023-11-25T04:25:59.978263+0000","last_undegraded":"2023-11-25T04:25:59.978263+0000","last_fullsized":"2023-11-25T04:25:59.978263+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:20:15.814691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":128641,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":185,"num_write_kb":408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":122,"num_bytes_recovered":128641,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"50'17","reported_seq":102,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-25T04:25:59.978969+0000","last_change":"2023-11-25T04:25:47.044228+0000","last_active":"2023-11-25T04:25:59.978969+0000","last_peered":"2023-11-25T04:25:59.978969+0000","last_clean":"2023-11-25T04:25:59.978969+0000","last_became_active":"2023-11-25T04:25:46.998870+0000","last_became_peered":"2023-11-25T04:25:46.998870+0000","last_unstale":"2023-11-25T04:25:59.978969+0000","last_undegraded":"2023-11-25T04:25:59.978969+0000","last_fullsized":"2023-11-25T04:25:59.978969+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:08:12.601941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":4103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4194304,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":26330855,"num_objects":3447,"num_object_clones":0,"num_object_copies":6894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10423,"num_write_kb":38550,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4189,"num_bytes_recovered":12749181,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":72376320,"data_stored":52324046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10246,"ondisk_log_size":10246,"up":128,"acting":128,"num_store_stats":3},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":40885548,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":35,"num_read_kb":334,"num_write":435,"num_write_kb":40462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":32491439,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":81833984,"data_stored":81771096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5548,"internal_metadata":0},"log_size":435,"ondisk_log_size":435,"up":64,"acting":64,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738416,"num_pgs":72,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73208,"kb_used_data":46188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94298632,"statfs":{"total":96636764160,"available":96561799168,"internally_reserved":0,"allocated":47296512,"data_stored":39787071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":1,"last update":"Sat Nov 25 04:25:40 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.433,"15min":0.44500000000000001},"min":{"1min":0.28899999999999998,"5min":0.216,"15min":0.216},"max":{"1min":0.60499999999999998,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.439,"15min":0.45100000000000001},"min":{"1min":0.25600000000000001,"5min":0.184,"15min":0.184},"max":{"1min":0.63400000000000001,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.42699999999999999}]}]},{"osd":1,"up_from":8,"seq":34359738416,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86880,"kb_used_data":55700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94284960,"statfs":{"total":96636764160,"available":96547799040,"internally_reserved":0,"allocated":57036800,"data_stored":50261554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":2,"last update":"Sat Nov 25 04:25:42 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.40600000000000003,"15min":0.41099999999999998},"min":{"1min":0.29099999999999998,"5min":0.182,"15min":0.182},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.40200000000000002,"15min":0.40600000000000003},"min":{"1min":0.312,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.39000000000000001}]}]},{"osd":0,"up_from":60,"seq":257698037766,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87624,"kb_used_data":51764,"kb_used_omap":6,"kb_used_meta":35833,"kb_avail":94284216,"statfs":{"total":96636764160,"available":96547037184,"internally_reserved":0,"allocated":53006336,"data_stored":46141107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7143,"internal_metadata":36693017},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32530432,"data_stored":32496940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5548,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":29380608,"data_stored":29360982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":19922944,"data_stored":19913174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":19234816,"data_stored":12749181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":26415104,"data_stored":20005586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":26726400,"data_stored":19569279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-25T04:26:10.223 INFO:tasks.ceph:pgid 4.3f last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.223 INFO:tasks.ceph:pgid 4.3e last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.224 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.224 INFO:tasks.ceph:pgid 4.3c last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.224 INFO:tasks.ceph:pgid 4.3b last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.224 INFO:tasks.ceph:pgid 4.3a last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.225 INFO:tasks.ceph:pgid 4.39 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.225 INFO:tasks.ceph:pgid 4.38 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.225 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.225 INFO:tasks.ceph:pgid 4.36 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.226 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.226 INFO:tasks.ceph:pgid 4.34 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.226 INFO:tasks.ceph:pgid 4.33 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.226 INFO:tasks.ceph:pgid 4.32 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.227 INFO:tasks.ceph:pgid 4.31 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.227 INFO:tasks.ceph:pgid 4.30 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.227 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.228 INFO:tasks.ceph:pgid 4.2e last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.228 INFO:tasks.ceph:pgid 4.2d last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.228 INFO:tasks.ceph:pgid 4.2c last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.228 INFO:tasks.ceph:pgid 4.2b last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.229 INFO:tasks.ceph:pgid 4.2a last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.229 INFO:tasks.ceph:pgid 4.29 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.229 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.230 INFO:tasks.ceph:pgid 4.27 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.230 INFO:tasks.ceph:pgid 4.26 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.230 INFO:tasks.ceph:pgid 4.25 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.230 INFO:tasks.ceph:pgid 4.24 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.231 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.231 INFO:tasks.ceph:pgid 4.22 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.231 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.231 INFO:tasks.ceph:pgid 4.20 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.232 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.232 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.232 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.232 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.233 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.233 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.233 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.233 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.233 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.233 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.234 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.234 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.234 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.234 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.234 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.234 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.235 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.235 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.235 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.235 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.235 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.236 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.236 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.236 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.236 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-11-25T04:22:35.919850+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=35, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.237 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-11-25T04:22:40.987592+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=40, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.237 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.237 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.237 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-11-25T04:22:40.987592+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=40, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.237 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.237 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.238 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-11-25T04:22:40.987592+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=40, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.238 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.238 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.238 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-11-25T04:22:40.987592+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=40, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.238 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.239 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.239 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.239 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-11-25T04:22:40.987592+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=40, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.239 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.239 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.240 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-11-25T04:22:40.987592+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=40, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.240 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.240 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.240 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-11-25T04:22:40.987592+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=40, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.240 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.240 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.241 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-11-25T04:22:40.987592+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=40, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.241 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.241 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.241 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.241 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.242 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.242 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.242 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.242 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.242 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.243 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.243 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.243 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.243 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.243 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.243 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.244 INFO:tasks.ceph:pgid 4.1c last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.244 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.244 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.244 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.244 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.244 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.244 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.245 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.245 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.245 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:10.245 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-11-25T04:26:30.246 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-25T04:26:30.559 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:26:30.560 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2023-11-25T04:26:30.570 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":191,"stamp":"2023-11-25T04:26:29.556564+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":67806771,"num_objects":3481,"num_object_clones":0,"num_object_copies":6962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3481,"num_whiteouts":0,"num_read":171,"num_read_kb":452,"num_write":11061,"num_write_kb":82472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4201,"num_bytes_recovered":45240620,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11739,"ondisk_log_size":11739,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":256012,"kb_used_data":153652,"kb_used_omap":10,"kb_used_meta":102325,"kb_avail":282859508,"statfs":{"total":289910292480,"available":289648136192,"internally_reserved":0,"allocated":157339648,"data_stored":136189732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10333,"internal_metadata":104781731},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003142"},"pg_stats":[{"pgid":"4.3f","version":"54'166","reported_seq":235,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986740+0000","last_change":"2023-11-25T04:22:47.039847+0000","last_active":"2023-11-25T04:26:00.986740+0000","last_peered":"2023-11-25T04:26:00.986740+0000","last_clean":"2023-11-25T04:26:00.986740+0000","last_became_active":"2023-11-25T04:22:47.039490+0000","last_became_peered":"2023-11-25T04:22:47.039490+0000","last_unstale":"2023-11-25T04:26:00.986740+0000","last_undegraded":"2023-11-25T04:26:00.986740+0000","last_fullsized":"2023-11-25T04:26:00.986740+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":577866,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":174,"num_write_kb":756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3e","version":"75'257","reported_seq":335,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:16.435239+0000","last_change":"2023-11-25T04:26:16.435276+0000","last_active":"2023-11-25T04:26:16.435239+0000","last_peered":"2023-11-25T04:26:16.435239+0000","last_clean":"2023-11-25T04:26:16.435239+0000","last_became_active":"2023-11-25T04:22:47.040511+0000","last_became_peered":"2023-11-25T04:22:47.040511+0000","last_unstale":"2023-11-25T04:26:16.435239+0000","last_undegraded":"2023-11-25T04:26:16.435239+0000","last_fullsized":"2023-11-25T04:26:16.435239+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'257","last_scrub_stamp":"2023-11-25T04:26:16.435161+0000","last_deep_scrub":"75'257","last_deep_scrub_stamp":"2023-11-25T04:26:16.435161+0000","last_clean_scrub_stamp":"2023-11-25T04:26:16.435161+0000","objects_scrubbed":45,"log_size":257,"log_dups_size":0,"ondisk_log_size":257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:13:07.641904+0000","scrub_duration":0.036488224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":461579,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":236,"num_write_kb":841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3d","version":"75'209","reported_seq":285,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:12.330753+0000","last_change":"2023-11-25T04:26:12.330801+0000","last_active":"2023-11-25T04:26:12.330753+0000","last_peered":"2023-11-25T04:26:12.330753+0000","last_clean":"2023-11-25T04:26:12.330753+0000","last_became_active":"2023-11-25T04:25:52.923459+0000","last_became_peered":"2023-11-25T04:25:52.923459+0000","last_unstale":"2023-11-25T04:26:12.330753+0000","last_undegraded":"2023-11-25T04:26:12.330753+0000","last_fullsized":"2023-11-25T04:26:12.330753+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'209","last_scrub_stamp":"2023-11-25T04:26:12.330672+0000","last_deep_scrub":"75'209","last_deep_scrub_stamp":"2023-11-25T04:26:12.330672+0000","last_clean_scrub_stamp":"2023-11-25T04:26:12.330672+0000","objects_scrubbed":42,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:49:33.381569+0000","scrub_duration":0.027938688999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":145844,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":175,"num_write_kb":312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":138765,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"54'124","reported_seq":192,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986326+0000","last_change":"2023-11-25T04:25:51.922843+0000","last_active":"2023-11-25T04:26:00.986326+0000","last_peered":"2023-11-25T04:26:00.986326+0000","last_clean":"2023-11-25T04:26:00.986326+0000","last_became_active":"2023-11-25T04:25:51.921633+0000","last_became_peered":"2023-11-25T04:25:51.921633+0000","last_unstale":"2023-11-25T04:26:00.986326+0000","last_undegraded":"2023-11-25T04:26:00.986326+0000","last_fullsized":"2023-11-25T04:26:00.986326+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61303,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119,"num_write_kb":161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":61303,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"54'169","reported_seq":238,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986191+0000","last_change":"2023-11-25T04:22:47.039499+0000","last_active":"2023-11-25T04:26:00.986191+0000","last_peered":"2023-11-25T04:26:00.986191+0000","last_clean":"2023-11-25T04:26:00.986191+0000","last_became_active":"2023-11-25T04:22:47.039165+0000","last_became_peered":"2023-11-25T04:22:47.039165+0000","last_unstale":"2023-11-25T04:26:00.986191+0000","last_undegraded":"2023-11-25T04:26:00.986191+0000","last_fullsized":"2023-11-25T04:26:00.986191+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":118227,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":185,"num_write_kb":351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3a","version":"54'169","reported_seq":235,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986335+0000","last_change":"2023-11-25T04:25:53.933198+0000","last_active":"2023-11-25T04:26:00.986335+0000","last_peered":"2023-11-25T04:26:00.986335+0000","last_clean":"2023-11-25T04:26:00.986335+0000","last_became_active":"2023-11-25T04:25:53.931982+0000","last_became_peered":"2023-11-25T04:25:53.931982+0000","last_unstale":"2023-11-25T04:26:00.986335+0000","last_undegraded":"2023-11-25T04:26:00.986335+0000","last_fullsized":"2023-11-25T04:26:00.986335+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100304,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":161,"num_write_kb":424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":100304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"54'168","reported_seq":238,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986042+0000","last_change":"2023-11-25T04:25:49.921033+0000","last_active":"2023-11-25T04:26:00.986042+0000","last_peered":"2023-11-25T04:26:00.986042+0000","last_clean":"2023-11-25T04:26:00.986042+0000","last_became_active":"2023-11-25T04:25:49.919354+0000","last_became_peered":"2023-11-25T04:25:49.919354+0000","last_unstale":"2023-11-25T04:26:00.986042+0000","last_undegraded":"2023-11-25T04:26:00.986042+0000","last_fullsized":"2023-11-25T04:26:00.986042+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":124558,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":167,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":111,"num_bytes_recovered":124533,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.38","version":"54'152","reported_seq":222,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.985923+0000","last_change":"2023-11-25T04:25:49.920421+0000","last_active":"2023-11-25T04:26:00.985923+0000","last_peered":"2023-11-25T04:26:00.985923+0000","last_clean":"2023-11-25T04:26:00.985923+0000","last_became_active":"2023-11-25T04:25:49.919236+0000","last_became_peered":"2023-11-25T04:25:49.919236+0000","last_unstale":"2023-11-25T04:26:00.985923+0000","last_undegraded":"2023-11-25T04:26:00.985923+0000","last_fullsized":"2023-11-25T04:26:00.985923+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":120147,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":120121,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.37","version":"75'220","reported_seq":299,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:28.772505+0000","last_change":"2023-11-25T04:26:28.772532+0000","last_active":"2023-11-25T04:26:28.772505+0000","last_peered":"2023-11-25T04:26:28.772505+0000","last_clean":"2023-11-25T04:26:28.772505+0000","last_became_active":"2023-11-25T04:22:47.037328+0000","last_became_peered":"2023-11-25T04:22:47.037328+0000","last_unstale":"2023-11-25T04:26:28.772505+0000","last_undegraded":"2023-11-25T04:26:28.772505+0000","last_fullsized":"2023-11-25T04:26:28.772505+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'220","last_scrub_stamp":"2023-11-25T04:26:28.772426+0000","last_deep_scrub":"75'220","last_deep_scrub_stamp":"2023-11-25T04:26:28.772426+0000","last_clean_scrub_stamp":"2023-11-25T04:26:28.772426+0000","objects_scrubbed":53,"log_size":220,"log_dups_size":0,"ondisk_log_size":220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:08:21.318788+0000","scrub_duration":0.036194551999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":174147,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":184,"num_write_kb":353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.36","version":"75'200","reported_seq":279,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:24.608814+0000","last_change":"2023-11-25T04:26:24.608840+0000","last_active":"2023-11-25T04:26:24.608814+0000","last_peered":"2023-11-25T04:26:24.608814+0000","last_clean":"2023-11-25T04:26:24.608814+0000","last_became_active":"2023-11-25T04:22:47.039892+0000","last_became_peered":"2023-11-25T04:22:47.039892+0000","last_unstale":"2023-11-25T04:26:24.608814+0000","last_undegraded":"2023-11-25T04:26:24.608814+0000","last_fullsized":"2023-11-25T04:26:24.608814+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'200","last_scrub_stamp":"2023-11-25T04:26:24.608737+0000","last_deep_scrub":"75'200","last_deep_scrub_stamp":"2023-11-25T04:26:24.608737+0000","last_clean_scrub_stamp":"2023-11-25T04:26:24.608737+0000","objects_scrubbed":51,"log_size":200,"log_dups_size":0,"ondisk_log_size":200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:49:43.116414+0000","scrub_duration":0.03530929,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":233330,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"54'156","reported_seq":224,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986831+0000","last_change":"2023-11-25T04:25:51.923087+0000","last_active":"2023-11-25T04:26:00.986831+0000","last_peered":"2023-11-25T04:26:00.986831+0000","last_clean":"2023-11-25T04:26:00.986831+0000","last_became_active":"2023-11-25T04:25:51.921800+0000","last_became_peered":"2023-11-25T04:25:51.921800+0000","last_unstale":"2023-11-25T04:26:00.986831+0000","last_undegraded":"2023-11-25T04:26:00.986831+0000","last_fullsized":"2023-11-25T04:26:00.986831+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":302550,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":163,"num_write_kb":431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":300323,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.34","version":"54'136","reported_seq":401,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986260+0000","last_change":"2023-11-25T04:25:50.922843+0000","last_active":"2023-11-25T04:26:00.986260+0000","last_peered":"2023-11-25T04:26:00.986260+0000","last_clean":"2023-11-25T04:26:00.986260+0000","last_became_active":"2023-11-25T04:25:50.922309+0000","last_became_peered":"2023-11-25T04:25:50.922309+0000","last_unstale":"2023-11-25T04:26:00.986260+0000","last_undegraded":"2023-11-25T04:26:00.986260+0000","last_fullsized":"2023-11-25T04:26:00.986260+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":93053,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":131,"num_write_kb":248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":89,"num_bytes_recovered":93053,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"75'183","reported_seq":261,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:19.516152+0000","last_change":"2023-11-25T04:26:19.516218+0000","last_active":"2023-11-25T04:26:19.516152+0000","last_peered":"2023-11-25T04:26:19.516152+0000","last_clean":"2023-11-25T04:26:19.516152+0000","last_became_active":"2023-11-25T04:22:47.040129+0000","last_became_peered":"2023-11-25T04:22:47.040129+0000","last_unstale":"2023-11-25T04:26:19.516152+0000","last_undegraded":"2023-11-25T04:26:19.516152+0000","last_fullsized":"2023-11-25T04:26:19.516152+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'183","last_scrub_stamp":"2023-11-25T04:26:19.516065+0000","last_deep_scrub":"75'183","last_deep_scrub_stamp":"2023-11-25T04:26:19.516065+0000","last_clean_scrub_stamp":"2023-11-25T04:26:19.516065+0000","objects_scrubbed":49,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:43:33.021008+0000","scrub_duration":0.038467327000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":149524,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":139,"num_write_kb":252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.32","version":"54'174","reported_seq":240,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986344+0000","last_change":"2023-11-25T04:25:53.934526+0000","last_active":"2023-11-25T04:26:00.986344+0000","last_peered":"2023-11-25T04:26:00.986344+0000","last_clean":"2023-11-25T04:26:00.986344+0000","last_became_active":"2023-11-25T04:25:53.933694+0000","last_became_peered":"2023-11-25T04:25:53.933694+0000","last_unstale":"2023-11-25T04:26:00.986344+0000","last_undegraded":"2023-11-25T04:26:00.986344+0000","last_fullsized":"2023-11-25T04:26:00.986344+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":95797,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":181,"num_write_kb":547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":103,"num_bytes_recovered":95797,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.31","version":"54'156","reported_seq":226,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.985942+0000","last_change":"2023-11-25T04:25:49.922396+0000","last_active":"2023-11-25T04:26:00.985942+0000","last_peered":"2023-11-25T04:26:00.985942+0000","last_clean":"2023-11-25T04:26:00.985942+0000","last_became_active":"2023-11-25T04:25:49.921129+0000","last_became_peered":"2023-11-25T04:25:49.921129+0000","last_unstale":"2023-11-25T04:26:00.985942+0000","last_undegraded":"2023-11-25T04:26:00.985942+0000","last_fullsized":"2023-11-25T04:26:00.985942+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":163190,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":169,"num_write_kb":336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":154576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"75'229","reported_seq":305,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:20.470401+0000","last_change":"2023-11-25T04:26:20.470429+0000","last_active":"2023-11-25T04:26:20.470401+0000","last_peered":"2023-11-25T04:26:20.470401+0000","last_clean":"2023-11-25T04:26:20.470401+0000","last_became_active":"2023-11-25T04:25:53.933395+0000","last_became_peered":"2023-11-25T04:25:53.933395+0000","last_unstale":"2023-11-25T04:26:20.470401+0000","last_undegraded":"2023-11-25T04:26:20.470401+0000","last_fullsized":"2023-11-25T04:26:20.470401+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'229","last_scrub_stamp":"2023-11-25T04:26:20.470325+0000","last_deep_scrub":"75'229","last_deep_scrub_stamp":"2023-11-25T04:26:20.470325+0000","last_clean_scrub_stamp":"2023-11-25T04:26:20.470325+0000","objects_scrubbed":55,"log_size":229,"log_dups_size":0,"ondisk_log_size":229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:55:52.038708+0000","scrub_duration":0.036798694,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1423587,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":182,"num_write_kb":1626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":1414508,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2f","version":"75'203","reported_seq":282,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:18.718015+0000","last_change":"2023-11-25T04:26:18.718065+0000","last_active":"2023-11-25T04:26:18.718015+0000","last_peered":"2023-11-25T04:26:18.718015+0000","last_clean":"2023-11-25T04:26:18.718015+0000","last_became_active":"2023-11-25T04:22:47.039123+0000","last_became_peered":"2023-11-25T04:22:47.039123+0000","last_unstale":"2023-11-25T04:26:18.718015+0000","last_undegraded":"2023-11-25T04:26:18.718015+0000","last_fullsized":"2023-11-25T04:26:18.718015+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'203","last_scrub_stamp":"2023-11-25T04:26:18.717922+0000","last_deep_scrub":"75'203","last_deep_scrub_stamp":"2023-11-25T04:26:18.717922+0000","last_clean_scrub_stamp":"2023-11-25T04:26:18.717922+0000","objects_scrubbed":51,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:33:40.286516+0000","scrub_duration":0.039125398999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":122752,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":143,"num_write_kb":435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"54'148","reported_seq":217,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986605+0000","last_change":"2023-11-25T04:22:47.036946+0000","last_active":"2023-11-25T04:26:00.986605+0000","last_peered":"2023-11-25T04:26:00.986605+0000","last_clean":"2023-11-25T04:26:00.986605+0000","last_became_active":"2023-11-25T04:22:47.036411+0000","last_became_peered":"2023-11-25T04:22:47.036411+0000","last_unstale":"2023-11-25T04:26:00.986605+0000","last_undegraded":"2023-11-25T04:26:00.986605+0000","last_fullsized":"2023-11-25T04:26:00.986605+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4770899,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":140,"num_write_kb":4870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2d","version":"54'143","reported_seq":212,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986566+0000","last_change":"2023-11-25T04:22:47.038983+0000","last_active":"2023-11-25T04:26:00.986566+0000","last_peered":"2023-11-25T04:26:00.986566+0000","last_clean":"2023-11-25T04:26:00.986566+0000","last_became_active":"2023-11-25T04:22:47.038477+0000","last_became_peered":"2023-11-25T04:22:47.038477+0000","last_unstale":"2023-11-25T04:26:00.986566+0000","last_undegraded":"2023-11-25T04:26:00.986566+0000","last_fullsized":"2023-11-25T04:26:00.986566+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":115975,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"75'230","reported_seq":307,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:19.428426+0000","last_change":"2023-11-25T04:26:19.428460+0000","last_active":"2023-11-25T04:26:19.428426+0000","last_peered":"2023-11-25T04:26:19.428426+0000","last_clean":"2023-11-25T04:26:19.428426+0000","last_became_active":"2023-11-25T04:25:52.922367+0000","last_became_peered":"2023-11-25T04:25:52.922367+0000","last_unstale":"2023-11-25T04:26:19.428426+0000","last_undegraded":"2023-11-25T04:26:19.428426+0000","last_fullsized":"2023-11-25T04:26:19.428426+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'230","last_scrub_stamp":"2023-11-25T04:26:19.428350+0000","last_deep_scrub":"75'230","last_deep_scrub_stamp":"2023-11-25T04:26:19.428350+0000","last_clean_scrub_stamp":"2023-11-25T04:26:19.428350+0000","objects_scrubbed":63,"log_size":230,"log_dups_size":0,"ondisk_log_size":230,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:10:12.216682+0000","scrub_duration":0.042625061999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":747090,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":211,"num_write_kb":985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":732276,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"75'211","reported_seq":288,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:17.381634+0000","last_change":"2023-11-25T04:26:17.381662+0000","last_active":"2023-11-25T04:26:17.381634+0000","last_peered":"2023-11-25T04:26:17.381634+0000","last_clean":"2023-11-25T04:26:17.381634+0000","last_became_active":"2023-11-25T04:25:52.923378+0000","last_became_peered":"2023-11-25T04:25:52.923378+0000","last_unstale":"2023-11-25T04:26:17.381634+0000","last_undegraded":"2023-11-25T04:26:17.381634+0000","last_fullsized":"2023-11-25T04:26:17.381634+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'211","last_scrub_stamp":"2023-11-25T04:26:17.381560+0000","last_deep_scrub":"75'211","last_deep_scrub_stamp":"2023-11-25T04:26:17.381560+0000","last_clean_scrub_stamp":"2023-11-25T04:26:17.381560+0000","objects_scrubbed":55,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:26:19.664059+0000","scrub_duration":0.036605545000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69580,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":142,"num_write_kb":315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":98,"num_bytes_recovered":69580,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2a","version":"54'171","reported_seq":468,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.985986+0000","last_change":"2023-11-25T04:25:52.925532+0000","last_active":"2023-11-25T04:26:00.985986+0000","last_peered":"2023-11-25T04:26:00.985986+0000","last_clean":"2023-11-25T04:26:00.985986+0000","last_became_active":"2023-11-25T04:25:52.924844+0000","last_became_peered":"2023-11-25T04:25:52.924844+0000","last_unstale":"2023-11-25T04:26:00.985986+0000","last_undegraded":"2023-11-25T04:26:00.985986+0000","last_fullsized":"2023-11-25T04:26:00.985986+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":115997,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":162,"num_write_kb":261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":115997,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.29","version":"75'211","reported_seq":508,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:12.692578+0000","last_change":"2023-11-25T04:26:12.692632+0000","last_active":"2023-11-25T04:26:12.692578+0000","last_peered":"2023-11-25T04:26:12.692578+0000","last_clean":"2023-11-25T04:26:12.692578+0000","last_became_active":"2023-11-25T04:25:49.919886+0000","last_became_peered":"2023-11-25T04:25:49.919886+0000","last_unstale":"2023-11-25T04:26:12.692578+0000","last_undegraded":"2023-11-25T04:26:12.692578+0000","last_fullsized":"2023-11-25T04:26:12.692578+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'211","last_scrub_stamp":"2023-11-25T04:26:12.692467+0000","last_deep_scrub":"75'211","last_deep_scrub_stamp":"2023-11-25T04:26:12.692467+0000","last_clean_scrub_stamp":"2023-11-25T04:26:12.692467+0000","objects_scrubbed":59,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:13:03.899209+0000","scrub_duration":0.043132342999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":212878,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":212878,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"54'169","reported_seq":238,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.985651+0000","last_change":"2023-11-25T04:22:47.036805+0000","last_active":"2023-11-25T04:26:00.985651+0000","last_peered":"2023-11-25T04:26:00.985651+0000","last_clean":"2023-11-25T04:26:00.985651+0000","last_became_active":"2023-11-25T04:22:47.036135+0000","last_became_peered":"2023-11-25T04:22:47.036135+0000","last_unstale":"2023-11-25T04:26:00.985651+0000","last_undegraded":"2023-11-25T04:26:00.985651+0000","last_fullsized":"2023-11-25T04:26:00.985651+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":102957,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":172,"num_write_kb":610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"54'167","reported_seq":466,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986816+0000","last_change":"2023-11-25T04:25:51.922925+0000","last_active":"2023-11-25T04:26:00.986816+0000","last_peered":"2023-11-25T04:26:00.986816+0000","last_clean":"2023-11-25T04:26:00.986816+0000","last_became_active":"2023-11-25T04:25:51.922364+0000","last_became_peered":"2023-11-25T04:25:51.922364+0000","last_unstale":"2023-11-25T04:26:00.986816+0000","last_undegraded":"2023-11-25T04:26:00.986816+0000","last_fullsized":"2023-11-25T04:26:00.986816+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":182683,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":171,"num_write_kb":791,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":182683,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"54'187","reported_seq":506,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986695+0000","last_change":"2023-11-25T04:25:51.922529+0000","last_active":"2023-11-25T04:26:00.986695+0000","last_peered":"2023-11-25T04:26:00.986695+0000","last_clean":"2023-11-25T04:26:00.986695+0000","last_became_active":"2023-11-25T04:25:51.922064+0000","last_became_peered":"2023-11-25T04:25:51.922064+0000","last_unstale":"2023-11-25T04:26:00.986695+0000","last_undegraded":"2023-11-25T04:26:00.986695+0000","last_fullsized":"2023-11-25T04:26:00.986695+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":187,"log_dups_size":0,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":86311,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":171,"num_write_kb":274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":116,"num_bytes_recovered":86311,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.25","version":"75'201","reported_seq":278,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:10.399641+0000","last_change":"2023-11-25T04:26:10.399668+0000","last_active":"2023-11-25T04:26:10.399641+0000","last_peered":"2023-11-25T04:26:10.399641+0000","last_clean":"2023-11-25T04:26:10.399641+0000","last_became_active":"2023-11-25T04:25:52.924795+0000","last_became_peered":"2023-11-25T04:25:52.924795+0000","last_unstale":"2023-11-25T04:26:10.399641+0000","last_undegraded":"2023-11-25T04:26:10.399641+0000","last_fullsized":"2023-11-25T04:26:10.399641+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'201","last_scrub_stamp":"2023-11-25T04:26:10.399561+0000","last_deep_scrub":"75'201","last_deep_scrub_stamp":"2023-11-25T04:26:10.399561+0000","last_clean_scrub_stamp":"2023-11-25T04:26:10.399561+0000","objects_scrubbed":52,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:15:15.077501+0000","scrub_duration":0.038027564,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":128288,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":167,"num_write_kb":267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":120265,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.24","version":"54'151","reported_seq":428,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986458+0000","last_change":"2023-11-25T04:25:50.922751+0000","last_active":"2023-11-25T04:26:00.986458+0000","last_peered":"2023-11-25T04:26:00.986458+0000","last_clean":"2023-11-25T04:26:00.986458+0000","last_became_active":"2023-11-25T04:25:50.922216+0000","last_became_peered":"2023-11-25T04:25:50.922216+0000","last_unstale":"2023-11-25T04:26:00.986458+0000","last_undegraded":"2023-11-25T04:26:00.986458+0000","last_fullsized":"2023-11-25T04:26:00.986458+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":107507,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":148,"num_write_kb":288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":107494,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.23","version":"54'155","reported_seq":224,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986172+0000","last_change":"2023-11-25T04:22:47.037668+0000","last_active":"2023-11-25T04:26:00.986172+0000","last_peered":"2023-11-25T04:26:00.986172+0000","last_clean":"2023-11-25T04:26:00.986172+0000","last_became_active":"2023-11-25T04:22:47.036823+0000","last_became_peered":"2023-11-25T04:22:47.036823+0000","last_unstale":"2023-11-25T04:26:00.986172+0000","last_undegraded":"2023-11-25T04:26:00.986172+0000","last_fullsized":"2023-11-25T04:26:00.986172+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4304582,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":152,"num_write_kb":4331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.22","version":"54'189","reported_seq":518,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986099+0000","last_change":"2023-11-25T04:25:54.940559+0000","last_active":"2023-11-25T04:26:00.986099+0000","last_peered":"2023-11-25T04:26:00.986099+0000","last_clean":"2023-11-25T04:26:00.986099+0000","last_became_active":"2023-11-25T04:25:54.939584+0000","last_became_peered":"2023-11-25T04:25:54.939584+0000","last_unstale":"2023-11-25T04:26:00.986099+0000","last_undegraded":"2023-11-25T04:26:00.986099+0000","last_fullsized":"2023-11-25T04:26:00.986099+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":129145,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":185,"num_write_kb":432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":121,"num_bytes_recovered":129107,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"54'201","reported_seq":270,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.985795+0000","last_change":"2023-11-25T04:22:47.037786+0000","last_active":"2023-11-25T04:26:00.985795+0000","last_peered":"2023-11-25T04:26:00.985795+0000","last_clean":"2023-11-25T04:26:00.985795+0000","last_became_active":"2023-11-25T04:22:47.037206+0000","last_became_peered":"2023-11-25T04:22:47.037206+0000","last_unstale":"2023-11-25T04:26:00.985795+0000","last_undegraded":"2023-11-25T04:26:00.985795+0000","last_fullsized":"2023-11-25T04:26:00.985795+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":215311,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":194,"num_write_kb":381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"54'180","reported_seq":246,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.985760+0000","last_change":"2023-11-25T04:25:53.933176+0000","last_active":"2023-11-25T04:26:00.985760+0000","last_peered":"2023-11-25T04:26:00.985760+0000","last_clean":"2023-11-25T04:26:00.985760+0000","last_became_active":"2023-11-25T04:25:53.931937+0000","last_became_peered":"2023-11-25T04:25:53.931937+0000","last_unstale":"2023-11-25T04:26:00.985760+0000","last_undegraded":"2023-11-25T04:26:00.985760+0000","last_fullsized":"2023-11-25T04:26:00.985760+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104603,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":172,"num_write_kb":267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":104603,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"54'137","reported_seq":206,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986374+0000","last_change":"2023-11-25T04:22:47.036689+0000","last_active":"2023-11-25T04:26:00.986374+0000","last_peered":"2023-11-25T04:26:00.986374+0000","last_clean":"2023-11-25T04:26:00.986374+0000","last_became_active":"2023-11-25T04:22:47.036107+0000","last_became_peered":"2023-11-25T04:22:47.036107+0000","last_unstale":"2023-11-25T04:26:00.986374+0000","last_undegraded":"2023-11-25T04:26:00.986374+0000","last_fullsized":"2023-11-25T04:26:00.986374+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":183586,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":127,"num_write_kb":284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"58'114","reported_seq":192,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986128+0000","last_change":"2023-11-25T04:25:49.920320+0000","last_active":"2023-11-25T04:26:00.986128+0000","last_peered":"2023-11-25T04:26:00.986128+0000","last_clean":"2023-11-25T04:26:00.986128+0000","last_became_active":"2023-11-25T04:25:49.920010+0000","last_became_peered":"2023-11-25T04:25:49.920010+0000","last_unstale":"2023-11-25T04:26:00.986128+0000","last_undegraded":"2023-11-25T04:26:00.986128+0000","last_fullsized":"2023-11-25T04:26:00.986128+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2816216,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":113,"num_write_kb":2816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":2816182,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"54'145","reported_seq":214,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986504+0000","last_change":"2023-11-25T04:22:47.037888+0000","last_active":"2023-11-25T04:26:00.986504+0000","last_peered":"2023-11-25T04:26:00.986504+0000","last_clean":"2023-11-25T04:26:00.986504+0000","last_became_active":"2023-11-25T04:22:47.037158+0000","last_became_peered":"2023-11-25T04:22:47.037158+0000","last_unstale":"2023-11-25T04:26:00.986504+0000","last_undegraded":"2023-11-25T04:26:00.986504+0000","last_fullsized":"2023-11-25T04:26:00.986504+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":205976,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.12","version":"49'2","reported_seq":83,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986188+0000","last_change":"2023-11-25T04:25:47.000916+0000","last_active":"2023-11-25T04:26:00.986188+0000","last_peered":"2023-11-25T04:26:00.986188+0000","last_clean":"2023-11-25T04:26:00.986188+0000","last_became_active":"2023-11-25T04:25:47.000516+0000","last_became_peered":"2023-11-25T04:25:47.000516+0000","last_unstale":"2023-11-25T04:26:00.986188+0000","last_undegraded":"2023-11-25T04:26:00.986188+0000","last_fullsized":"2023-11-25T04:26:00.986188+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"54'175","reported_seq":490,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986834+0000","last_change":"2023-11-25T04:25:52.923694+0000","last_active":"2023-11-25T04:26:00.986834+0000","last_peered":"2023-11-25T04:26:00.986834+0000","last_clean":"2023-11-25T04:26:00.986834+0000","last_became_active":"2023-11-25T04:25:52.923045+0000","last_became_peered":"2023-11-25T04:25:52.923045+0000","last_unstale":"2023-11-25T04:26:00.986834+0000","last_undegraded":"2023-11-25T04:26:00.986834+0000","last_fullsized":"2023-11-25T04:26:00.986834+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":171823,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":185,"num_write_kb":340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":166387,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":89,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:18.488426+0000","last_change":"2023-11-25T04:26:18.488445+0000","last_active":"2023-11-25T04:26:18.488426+0000","last_peered":"2023-11-25T04:26:18.488426+0000","last_clean":"2023-11-25T04:26:18.488426+0000","last_became_active":"2023-11-25T04:25:46.993870+0000","last_became_peered":"2023-11-25T04:25:46.993870+0000","last_unstale":"2023-11-25T04:26:18.488426+0000","last_undegraded":"2023-11-25T04:26:18.488426+0000","last_fullsized":"2023-11-25T04:26:18.488426+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:26:18.488345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:26:18.488345+0000","last_clean_scrub_stamp":"2023-11-25T04:26:18.488345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:43:21.558596+0000","scrub_duration":0.002108399,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"54'152","reported_seq":436,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986747+0000","last_change":"2023-11-25T04:25:50.922922+0000","last_active":"2023-11-25T04:26:00.986747+0000","last_peered":"2023-11-25T04:26:00.986747+0000","last_clean":"2023-11-25T04:26:00.986747+0000","last_became_active":"2023-11-25T04:25:50.922511+0000","last_became_peered":"2023-11-25T04:25:50.922511+0000","last_unstale":"2023-11-25T04:26:00.986747+0000","last_undegraded":"2023-11-25T04:26:00.986747+0000","last_fullsized":"2023-11-25T04:26:00.986747+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":579704,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":97,"num_bytes_recovered":570044,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.10","version":"18'8","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.985749+0000","last_change":"2023-11-25T04:25:47.000499+0000","last_active":"2023-11-25T04:26:00.985749+0000","last_peered":"2023-11-25T04:26:00.985749+0000","last_clean":"2023-11-25T04:26:00.985749+0000","last_became_active":"2023-11-25T04:25:46.998778+0000","last_became_peered":"2023-11-25T04:25:46.998778+0000","last_unstale":"2023-11-25T04:26:00.985749+0000","last_undegraded":"2023-11-25T04:26:00.985749+0000","last_fullsized":"2023-11-25T04:26:00.985749+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"49'2","reported_seq":38,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986895+0000","last_change":"2023-11-25T04:25:46.998392+0000","last_active":"2023-11-25T04:26:00.986895+0000","last_peered":"2023-11-25T04:26:00.986895+0000","last_clean":"2023-11-25T04:26:00.986895+0000","last_became_active":"2023-11-25T04:25:46.997108+0000","last_became_peered":"2023-11-25T04:25:46.997108+0000","last_unstale":"2023-11-25T04:26:00.986895+0000","last_undegraded":"2023-11-25T04:26:00.986895+0000","last_fullsized":"2023-11-25T04:26:00.986895+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"54'159","reported_seq":444,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.985722+0000","last_change":"2023-11-25T04:25:50.922282+0000","last_active":"2023-11-25T04:26:00.985722+0000","last_peered":"2023-11-25T04:26:00.985722+0000","last_clean":"2023-11-25T04:26:00.985722+0000","last_became_active":"2023-11-25T04:25:50.921772+0000","last_became_peered":"2023-11-25T04:25:50.921772+0000","last_unstale":"2023-11-25T04:26:00.985722+0000","last_undegraded":"2023-11-25T04:26:00.985722+0000","last_fullsized":"2023-11-25T04:26:00.985722+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112103,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":164,"num_write_kb":240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":99,"num_bytes_recovered":103472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"18'2","reported_seq":83,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986710+0000","last_change":"2023-11-25T04:25:46.994943+0000","last_active":"2023-11-25T04:26:00.986710+0000","last_peered":"2023-11-25T04:26:00.986710+0000","last_clean":"2023-11-25T04:26:00.986710+0000","last_became_active":"2023-11-25T04:25:46.994640+0000","last_became_peered":"2023-11-25T04:25:46.994640+0000","last_unstale":"2023-11-25T04:26:00.986710+0000","last_undegraded":"2023-11-25T04:26:00.986710+0000","last_fullsized":"2023-11-25T04:26:00.986710+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"54'173","reported_seq":242,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.985870+0000","last_change":"2023-11-25T04:22:47.039685+0000","last_active":"2023-11-25T04:26:00.985870+0000","last_peered":"2023-11-25T04:26:00.985870+0000","last_clean":"2023-11-25T04:26:00.985870+0000","last_became_active":"2023-11-25T04:22:47.039296+0000","last_became_peered":"2023-11-25T04:22:47.039296+0000","last_unstale":"2023-11-25T04:26:00.985870+0000","last_undegraded":"2023-11-25T04:26:00.985870+0000","last_fullsized":"2023-11-25T04:26:00.985870+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154533,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":182,"num_write_kb":299,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"18'1","reported_seq":38,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986574+0000","last_change":"2023-11-25T04:25:47.000059+0000","last_active":"2023-11-25T04:26:00.986574+0000","last_peered":"2023-11-25T04:26:00.986574+0000","last_clean":"2023-11-25T04:26:00.986574+0000","last_became_active":"2023-11-25T04:25:46.999604+0000","last_became_peered":"2023-11-25T04:25:46.999604+0000","last_unstale":"2023-11-25T04:26:00.986574+0000","last_undegraded":"2023-11-25T04:26:00.986574+0000","last_fullsized":"2023-11-25T04:26:00.986574+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"54'147","reported_seq":212,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986571+0000","last_change":"2023-11-25T04:25:54.939274+0000","last_active":"2023-11-25T04:26:00.986571+0000","last_peered":"2023-11-25T04:26:00.986571+0000","last_clean":"2023-11-25T04:26:00.986571+0000","last_became_active":"2023-11-25T04:25:54.938236+0000","last_became_peered":"2023-11-25T04:25:54.938236+0000","last_unstale":"2023-11-25T04:26:00.986571+0000","last_undegraded":"2023-11-25T04:26:00.986571+0000","last_fullsized":"2023-11-25T04:26:00.986571+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":147,"log_dups_size":0,"ondisk_log_size":147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":87524,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":155,"num_write_kb":235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":87524,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":35,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986425+0000","last_change":"2023-11-25T04:25:46.997080+0000","last_active":"2023-11-25T04:26:00.986425+0000","last_peered":"2023-11-25T04:26:00.986425+0000","last_clean":"2023-11-25T04:26:00.986425+0000","last_became_active":"2023-11-25T04:25:46.995712+0000","last_became_peered":"2023-11-25T04:25:46.995712+0000","last_unstale":"2023-11-25T04:26:00.986425+0000","last_undegraded":"2023-11-25T04:26:00.986425+0000","last_fullsized":"2023-11-25T04:26:00.986425+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"54'155","reported_seq":442,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986276+0000","last_change":"2023-11-25T04:25:52.923096+0000","last_active":"2023-11-25T04:26:00.986276+0000","last_peered":"2023-11-25T04:26:00.986276+0000","last_clean":"2023-11-25T04:26:00.986276+0000","last_became_active":"2023-11-25T04:25:52.922549+0000","last_became_peered":"2023-11-25T04:25:52.922549+0000","last_unstale":"2023-11-25T04:26:00.986276+0000","last_undegraded":"2023-11-25T04:26:00.986276+0000","last_fullsized":"2023-11-25T04:26:00.986276+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":266919,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":264258,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"45'11","reported_seq":97,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986446+0000","last_change":"2023-11-25T04:25:47.092497+0000","last_active":"2023-11-25T04:26:00.986446+0000","last_peered":"2023-11-25T04:26:00.986446+0000","last_clean":"2023-11-25T04:26:00.986446+0000","last_became_active":"2023-11-25T04:25:47.000687+0000","last_became_peered":"2023-11-25T04:25:47.000687+0000","last_unstale":"2023-11-25T04:26:00.986446+0000","last_undegraded":"2023-11-25T04:26:00.986446+0000","last_fullsized":"2023-11-25T04:26:00.986446+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":4101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4194304,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"54'173","reported_seq":242,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986729+0000","last_change":"2023-11-25T04:25:50.923240+0000","last_active":"2023-11-25T04:26:00.986729+0000","last_peered":"2023-11-25T04:26:00.986729+0000","last_clean":"2023-11-25T04:26:00.986729+0000","last_became_active":"2023-11-25T04:25:50.922409+0000","last_became_peered":"2023-11-25T04:25:50.922409+0000","last_unstale":"2023-11-25T04:26:00.986729+0000","last_undegraded":"2023-11-25T04:26:00.986729+0000","last_fullsized":"2023-11-25T04:26:00.986729+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":553893,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":169,"num_write_kb":770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":553893,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":35,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986132+0000","last_change":"2023-11-25T04:25:46.998818+0000","last_active":"2023-11-25T04:26:00.986132+0000","last_peered":"2023-11-25T04:26:00.986132+0000","last_clean":"2023-11-25T04:26:00.986132+0000","last_became_active":"2023-11-25T04:25:46.997019+0000","last_became_peered":"2023-11-25T04:25:46.997019+0000","last_unstale":"2023-11-25T04:26:00.986132+0000","last_undegraded":"2023-11-25T04:26:00.986132+0000","last_fullsized":"2023-11-25T04:26:00.986132+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"54'163","reported_seq":229,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986523+0000","last_change":"2023-11-25T04:25:53.934183+0000","last_active":"2023-11-25T04:26:00.986523+0000","last_peered":"2023-11-25T04:26:00.986523+0000","last_clean":"2023-11-25T04:26:00.986523+0000","last_became_active":"2023-11-25T04:25:53.933471+0000","last_became_peered":"2023-11-25T04:25:53.933471+0000","last_unstale":"2023-11-25T04:26:00.986523+0000","last_undegraded":"2023-11-25T04:26:00.986523+0000","last_fullsized":"2023-11-25T04:26:00.986523+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":320867,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":190,"num_write_kb":577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":108,"num_bytes_recovered":311640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":89,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:26.554438+0000","last_change":"2023-11-25T04:26:26.554458+0000","last_active":"2023-11-25T04:26:26.554438+0000","last_peered":"2023-11-25T04:26:26.554438+0000","last_clean":"2023-11-25T04:26:26.554438+0000","last_became_active":"2023-11-25T04:25:46.994101+0000","last_became_peered":"2023-11-25T04:25:46.994101+0000","last_unstale":"2023-11-25T04:26:26.554438+0000","last_undegraded":"2023-11-25T04:26:26.554438+0000","last_fullsized":"2023-11-25T04:26:26.554438+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:26:26.554355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:26:26.554355+0000","last_clean_scrub_stamp":"2023-11-25T04:26:26.554355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:20:20.095629+0000","scrub_duration":0.0016349400000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"54'141","reported_seq":211,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986843+0000","last_change":"2023-11-25T04:25:49.921950+0000","last_active":"2023-11-25T04:26:00.986843+0000","last_peered":"2023-11-25T04:26:00.986843+0000","last_clean":"2023-11-25T04:26:00.986843+0000","last_became_active":"2023-11-25T04:25:49.920971+0000","last_became_peered":"2023-11-25T04:25:49.920971+0000","last_unstale":"2023-11-25T04:26:00.986843+0000","last_undegraded":"2023-11-25T04:26:00.986843+0000","last_fullsized":"2023-11-25T04:26:00.986843+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3502371,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":3718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":89,"num_bytes_recovered":3497669,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"49'1","reported_seq":71,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.985742+0000","last_change":"2023-11-25T04:22:46.034509+0000","last_active":"2023-11-25T04:26:00.985742+0000","last_peered":"2023-11-25T04:26:00.985742+0000","last_clean":"2023-11-25T04:26:00.985742+0000","last_became_active":"2023-11-25T04:22:46.034158+0000","last_became_peered":"2023-11-25T04:22:46.034158+0000","last_unstale":"2023-11-25T04:26:00.985742+0000","last_undegraded":"2023-11-25T04:26:00.985742+0000","last_fullsized":"2023-11-25T04:26:00.985742+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"75'194","reported_seq":273,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:15.719582+0000","last_change":"2023-11-25T04:26:15.719633+0000","last_active":"2023-11-25T04:26:15.719582+0000","last_peered":"2023-11-25T04:26:15.719582+0000","last_clean":"2023-11-25T04:26:15.719582+0000","last_became_active":"2023-11-25T04:22:47.038081+0000","last_became_peered":"2023-11-25T04:22:47.038081+0000","last_unstale":"2023-11-25T04:26:15.719582+0000","last_undegraded":"2023-11-25T04:26:15.719582+0000","last_fullsized":"2023-11-25T04:26:15.719582+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'194","last_scrub_stamp":"2023-11-25T04:26:15.719490+0000","last_deep_scrub":"75'194","last_deep_scrub_stamp":"2023-11-25T04:26:15.719490+0000","last_clean_scrub_stamp":"2023-11-25T04:26:15.719490+0000","objects_scrubbed":53,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:43:29.224433+0000","scrub_duration":0.037765356,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":174088,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":146,"num_write_kb":385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"12'157","reported_seq":233,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.985653+0000","last_change":"2023-11-25T04:25:47.000504+0000","last_active":"2023-11-25T04:26:00.985653+0000","last_peered":"2023-11-25T04:26:00.985653+0000","last_clean":"2023-11-25T04:26:00.985653+0000","last_became_active":"2023-11-25T04:25:46.999721+0000","last_became_peered":"2023-11-25T04:25:46.999721+0000","last_unstale":"2023-11-25T04:26:00.985653+0000","last_undegraded":"2023-11-25T04:26:00.985653+0000","last_fullsized":"2023-11-25T04:26:00.985653+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:35.919850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:35.919850+0000","last_clean_scrub_stamp":"2023-11-25T04:22:35.919850+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":35,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.985910+0000","last_change":"2023-11-25T04:25:46.998960+0000","last_active":"2023-11-25T04:26:00.985910+0000","last_peered":"2023-11-25T04:26:00.985910+0000","last_clean":"2023-11-25T04:26:00.985910+0000","last_became_active":"2023-11-25T04:25:46.997693+0000","last_became_peered":"2023-11-25T04:25:46.997693+0000","last_unstale":"2023-11-25T04:26:00.985910+0000","last_undegraded":"2023-11-25T04:26:00.985910+0000","last_fullsized":"2023-11-25T04:26:00.985910+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"46'1","reported_seq":71,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986287+0000","last_change":"2023-11-25T04:22:46.037044+0000","last_active":"2023-11-25T04:26:00.986287+0000","last_peered":"2023-11-25T04:26:00.986287+0000","last_clean":"2023-11-25T04:26:00.986287+0000","last_became_active":"2023-11-25T04:22:46.035916+0000","last_became_peered":"2023-11-25T04:22:46.035916+0000","last_unstale":"2023-11-25T04:26:00.986287+0000","last_undegraded":"2023-11-25T04:26:00.986287+0000","last_fullsized":"2023-11-25T04:26:00.986287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"54'146","reported_seq":214,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986701+0000","last_change":"2023-11-25T04:25:51.923452+0000","last_active":"2023-11-25T04:26:00.986701+0000","last_peered":"2023-11-25T04:26:00.986701+0000","last_clean":"2023-11-25T04:26:00.986701+0000","last_became_active":"2023-11-25T04:25:51.922490+0000","last_became_peered":"2023-11-25T04:25:51.922490+0000","last_unstale":"2023-11-25T04:26:00.986701+0000","last_undegraded":"2023-11-25T04:26:00.986701+0000","last_fullsized":"2023-11-25T04:26:00.986701+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":502154,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":170,"num_write_kb":687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":99,"num_bytes_recovered":502101,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":35,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986008+0000","last_change":"2023-11-25T04:25:47.000763+0000","last_active":"2023-11-25T04:26:00.986008+0000","last_peered":"2023-11-25T04:26:00.986008+0000","last_clean":"2023-11-25T04:26:00.986008+0000","last_became_active":"2023-11-25T04:25:47.000033+0000","last_became_peered":"2023-11-25T04:25:47.000033+0000","last_unstale":"2023-11-25T04:26:00.986008+0000","last_undegraded":"2023-11-25T04:26:00.986008+0000","last_fullsized":"2023-11-25T04:26:00.986008+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"50'37","reported_seq":157,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986069+0000","last_change":"2023-11-25T04:25:47.002009+0000","last_active":"2023-11-25T04:26:00.986069+0000","last_peered":"2023-11-25T04:26:00.986069+0000","last_clean":"2023-11-25T04:26:00.986069+0000","last_became_active":"2023-11-25T04:25:46.995637+0000","last_became_peered":"2023-11-25T04:25:46.995637+0000","last_unstale":"2023-11-25T04:26:00.986069+0000","last_undegraded":"2023-11-25T04:26:00.986069+0000","last_fullsized":"2023-11-25T04:26:00.986069+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":319160,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":35,"num_read_kb":334,"num_write":35,"num_write_kb":334,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":319160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"54'158","reported_seq":227,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986599+0000","last_change":"2023-11-25T04:22:47.039583+0000","last_active":"2023-11-25T04:26:00.986599+0000","last_peered":"2023-11-25T04:26:00.986599+0000","last_clean":"2023-11-25T04:26:00.986599+0000","last_became_active":"2023-11-25T04:22:47.039197+0000","last_became_peered":"2023-11-25T04:22:47.039197+0000","last_unstale":"2023-11-25T04:26:00.986599+0000","last_undegraded":"2023-11-25T04:26:00.986599+0000","last_fullsized":"2023-11-25T04:26:00.986599+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":169303,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":189,"num_write_kb":333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":73,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986177+0000","last_change":"2023-11-25T04:22:42.000382+0000","last_active":"2023-11-25T04:26:00.986177+0000","last_peered":"2023-11-25T04:26:00.986177+0000","last_clean":"2023-11-25T04:26:00.986177+0000","last_became_active":"2023-11-25T04:22:42.000041+0000","last_became_peered":"2023-11-25T04:22:42.000041+0000","last_unstale":"2023-11-25T04:26:00.986177+0000","last_undegraded":"2023-11-25T04:26:00.986177+0000","last_fullsized":"2023-11-25T04:26:00.986177+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"75'72","reported_seq":165,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:15.436240+0000","last_change":"2023-11-25T04:26:15.436284+0000","last_active":"2023-11-25T04:26:15.436240+0000","last_peered":"2023-11-25T04:26:15.436240+0000","last_clean":"2023-11-25T04:26:15.436240+0000","last_became_active":"2023-11-25T04:25:47.001162+0000","last_became_peered":"2023-11-25T04:25:47.001162+0000","last_unstale":"2023-11-25T04:26:15.436240+0000","last_undegraded":"2023-11-25T04:26:15.436240+0000","last_fullsized":"2023-11-25T04:26:15.436240+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'72","last_scrub_stamp":"2023-11-25T04:26:15.436148+0000","last_deep_scrub":"75'72","last_deep_scrub_stamp":"2023-11-25T04:26:15.436148+0000","last_clean_scrub_stamp":"2023-11-25T04:26:15.436148+0000","objects_scrubbed":1,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:37:38.643270+0000","scrub_duration":0.01087811,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":69,"num_write_kb":4130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4194304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"75'220","reported_seq":299,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:14.717988+0000","last_change":"2023-11-25T04:26:14.718055+0000","last_active":"2023-11-25T04:26:14.717988+0000","last_peered":"2023-11-25T04:26:14.717988+0000","last_clean":"2023-11-25T04:26:14.717988+0000","last_became_active":"2023-11-25T04:22:47.038046+0000","last_became_peered":"2023-11-25T04:22:47.038046+0000","last_unstale":"2023-11-25T04:26:14.717988+0000","last_undegraded":"2023-11-25T04:26:14.717988+0000","last_fullsized":"2023-11-25T04:26:14.717988+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'220","last_scrub_stamp":"2023-11-25T04:26:14.717897+0000","last_deep_scrub":"75'220","last_deep_scrub_stamp":"2023-11-25T04:26:14.717897+0000","last_clean_scrub_stamp":"2023-11-25T04:26:14.717897+0000","objects_scrubbed":62,"log_size":220,"log_dups_size":0,"ondisk_log_size":220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:43:17.788148+0000","scrub_duration":0.044136959000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":115501,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":163,"num_write_kb":243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":35,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.985805+0000","last_change":"2023-11-25T04:25:46.998739+0000","last_active":"2023-11-25T04:26:00.985805+0000","last_peered":"2023-11-25T04:26:00.985805+0000","last_clean":"2023-11-25T04:26:00.985805+0000","last_became_active":"2023-11-25T04:25:46.997391+0000","last_became_peered":"2023-11-25T04:25:46.997391+0000","last_unstale":"2023-11-25T04:26:00.985805+0000","last_undegraded":"2023-11-25T04:26:00.985805+0000","last_fullsized":"2023-11-25T04:26:00.985805+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"50'1","reported_seq":35,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986046+0000","last_change":"2023-11-25T04:25:46.999446+0000","last_active":"2023-11-25T04:26:00.986046+0000","last_peered":"2023-11-25T04:26:00.986046+0000","last_clean":"2023-11-25T04:26:00.986046+0000","last_became_active":"2023-11-25T04:25:46.998132+0000","last_became_peered":"2023-11-25T04:25:46.998132+0000","last_unstale":"2023-11-25T04:26:00.986046+0000","last_undegraded":"2023-11-25T04:26:00.986046+0000","last_fullsized":"2023-11-25T04:26:00.986046+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"75'206","reported_seq":284,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:19.694413+0000","last_change":"2023-11-25T04:26:19.694459+0000","last_active":"2023-11-25T04:26:19.694413+0000","last_peered":"2023-11-25T04:26:19.694413+0000","last_clean":"2023-11-25T04:26:19.694413+0000","last_became_active":"2023-11-25T04:22:47.036924+0000","last_became_peered":"2023-11-25T04:22:47.036924+0000","last_unstale":"2023-11-25T04:26:19.694413+0000","last_undegraded":"2023-11-25T04:26:19.694413+0000","last_fullsized":"2023-11-25T04:26:19.694413+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'206","last_scrub_stamp":"2023-11-25T04:26:19.694341+0000","last_deep_scrub":"75'206","last_deep_scrub_stamp":"2023-11-25T04:26:19.694341+0000","last_clean_scrub_stamp":"2023-11-25T04:26:19.694341+0000","objects_scrubbed":49,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:49:38.202017+0000","scrub_duration":0.031428719000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":93321,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":148,"num_write_kb":360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":35,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986393+0000","last_change":"2023-11-25T04:25:46.999738+0000","last_active":"2023-11-25T04:26:00.986393+0000","last_peered":"2023-11-25T04:26:00.986393+0000","last_clean":"2023-11-25T04:26:00.986393+0000","last_became_active":"2023-11-25T04:25:46.999403+0000","last_became_peered":"2023-11-25T04:25:46.999403+0000","last_unstale":"2023-11-25T04:26:00.986393+0000","last_undegraded":"2023-11-25T04:26:00.986393+0000","last_fullsized":"2023-11-25T04:26:00.986393+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":73,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986717+0000","last_change":"2023-11-25T04:22:42.000204+0000","last_active":"2023-11-25T04:26:00.986717+0000","last_peered":"2023-11-25T04:26:00.986717+0000","last_clean":"2023-11-25T04:26:00.986717+0000","last_became_active":"2023-11-25T04:22:41.999915+0000","last_became_peered":"2023-11-25T04:22:41.999915+0000","last_unstale":"2023-11-25T04:26:00.986717+0000","last_undegraded":"2023-11-25T04:26:00.986717+0000","last_fullsized":"2023-11-25T04:26:00.986717+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"54'152","reported_seq":433,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.985964+0000","last_change":"2023-11-25T04:25:53.933239+0000","last_active":"2023-11-25T04:26:00.985964+0000","last_peered":"2023-11-25T04:26:00.985964+0000","last_clean":"2023-11-25T04:26:00.985964+0000","last_became_active":"2023-11-25T04:25:53.932586+0000","last_became_peered":"2023-11-25T04:25:53.932586+0000","last_unstale":"2023-11-25T04:26:00.985964+0000","last_undegraded":"2023-11-25T04:26:00.985964+0000","last_fullsized":"2023-11-25T04:26:00.985964+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":329327,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":151,"num_write_kb":471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":97,"num_bytes_recovered":329278,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":70,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986629+0000","last_change":"2023-11-25T04:22:46.035885+0000","last_active":"2023-11-25T04:26:00.986629+0000","last_peered":"2023-11-25T04:26:00.986629+0000","last_clean":"2023-11-25T04:26:00.986629+0000","last_became_active":"2023-11-25T04:22:46.035405+0000","last_became_peered":"2023-11-25T04:22:46.035405+0000","last_unstale":"2023-11-25T04:26:00.986629+0000","last_undegraded":"2023-11-25T04:26:00.986629+0000","last_fullsized":"2023-11-25T04:26:00.986629+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":73,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986810+0000","last_change":"2023-11-25T04:22:41.999067+0000","last_active":"2023-11-25T04:26:00.986810+0000","last_peered":"2023-11-25T04:26:00.986810+0000","last_clean":"2023-11-25T04:26:00.986810+0000","last_became_active":"2023-11-25T04:22:41.998798+0000","last_became_peered":"2023-11-25T04:22:41.998798+0000","last_unstale":"2023-11-25T04:26:00.986810+0000","last_undegraded":"2023-11-25T04:26:00.986810+0000","last_fullsized":"2023-11-25T04:26:00.986810+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"75'221","reported_seq":529,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:13.479628+0000","last_change":"2023-11-25T04:26:13.479673+0000","last_active":"2023-11-25T04:26:13.479628+0000","last_peered":"2023-11-25T04:26:13.479628+0000","last_clean":"2023-11-25T04:26:13.479628+0000","last_became_active":"2023-11-25T04:25:51.921858+0000","last_became_peered":"2023-11-25T04:25:51.921858+0000","last_unstale":"2023-11-25T04:26:13.479628+0000","last_undegraded":"2023-11-25T04:26:13.479628+0000","last_fullsized":"2023-11-25T04:26:13.479628+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'221","last_scrub_stamp":"2023-11-25T04:26:13.479549+0000","last_deep_scrub":"75'221","last_deep_scrub_stamp":"2023-11-25T04:26:13.479549+0000","last_clean_scrub_stamp":"2023-11-25T04:26:13.479549+0000","objects_scrubbed":49,"log_size":221,"log_dups_size":0,"ondisk_log_size":221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:44:56.142187+0000","scrub_duration":0.035520731999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":129965,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":166,"num_write_kb":290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":129965,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"18'2","reported_seq":72,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.987053+0000","last_change":"2023-11-25T04:22:46.034606+0000","last_active":"2023-11-25T04:26:00.987053+0000","last_peered":"2023-11-25T04:26:00.987053+0000","last_clean":"2023-11-25T04:26:00.987053+0000","last_became_active":"2023-11-25T04:22:46.034229+0000","last_became_peered":"2023-11-25T04:22:46.034229+0000","last_unstale":"2023-11-25T04:26:00.987053+0000","last_undegraded":"2023-11-25T04:26:00.987053+0000","last_fullsized":"2023-11-25T04:26:00.987053+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":73,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986933+0000","last_change":"2023-11-25T04:22:42.000258+0000","last_active":"2023-11-25T04:26:00.986933+0000","last_peered":"2023-11-25T04:26:00.986933+0000","last_clean":"2023-11-25T04:26:00.986933+0000","last_became_active":"2023-11-25T04:22:41.999992+0000","last_became_peered":"2023-11-25T04:22:41.999992+0000","last_unstale":"2023-11-25T04:26:00.986933+0000","last_undegraded":"2023-11-25T04:26:00.986933+0000","last_fullsized":"2023-11-25T04:26:00.986933+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"54'150","reported_seq":437,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.985896+0000","last_change":"2023-11-25T04:25:54.938479+0000","last_active":"2023-11-25T04:26:00.985896+0000","last_peered":"2023-11-25T04:26:00.985896+0000","last_clean":"2023-11-25T04:26:00.985896+0000","last_became_active":"2023-11-25T04:25:54.937838+0000","last_became_peered":"2023-11-25T04:25:54.937838+0000","last_unstale":"2023-11-25T04:26:00.985896+0000","last_undegraded":"2023-11-25T04:26:00.985896+0000","last_fullsized":"2023-11-25T04:26:00.985896+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":159585,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":154224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"54'14","reported_seq":100,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986916+0000","last_change":"2023-11-25T04:25:47.026749+0000","last_active":"2023-11-25T04:26:00.986916+0000","last_peered":"2023-11-25T04:26:00.986916+0000","last_clean":"2023-11-25T04:26:00.986916+0000","last_became_active":"2023-11-25T04:25:46.997404+0000","last_became_peered":"2023-11-25T04:25:46.997404+0000","last_unstale":"2023-11-25T04:26:00.986916+0000","last_undegraded":"2023-11-25T04:26:00.986916+0000","last_fullsized":"2023-11-25T04:26:00.986916+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":35,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986659+0000","last_change":"2023-11-25T04:25:46.995200+0000","last_active":"2023-11-25T04:26:00.986659+0000","last_peered":"2023-11-25T04:26:00.986659+0000","last_clean":"2023-11-25T04:26:00.986659+0000","last_became_active":"2023-11-25T04:25:46.994834+0000","last_became_peered":"2023-11-25T04:25:46.994834+0000","last_unstale":"2023-11-25T04:26:00.986659+0000","last_undegraded":"2023-11-25T04:26:00.986659+0000","last_fullsized":"2023-11-25T04:26:00.986659+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"75'191","reported_seq":474,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:14.466588+0000","last_change":"2023-11-25T04:26:14.466619+0000","last_active":"2023-11-25T04:26:14.466588+0000","last_peered":"2023-11-25T04:26:14.466588+0000","last_clean":"2023-11-25T04:26:14.466588+0000","last_became_active":"2023-11-25T04:25:49.921465+0000","last_became_peered":"2023-11-25T04:25:49.921465+0000","last_unstale":"2023-11-25T04:26:14.466588+0000","last_undegraded":"2023-11-25T04:26:14.466588+0000","last_fullsized":"2023-11-25T04:26:14.466588+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'191","last_scrub_stamp":"2023-11-25T04:26:14.466515+0000","last_deep_scrub":"75'191","last_deep_scrub_stamp":"2023-11-25T04:26:14.466515+0000","last_clean_scrub_stamp":"2023-11-25T04:26:14.466515+0000","objects_scrubbed":56,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:04:34.752431+0000","scrub_duration":0.037056879000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":99503,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":148,"num_write_kb":250,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":93,"num_bytes_recovered":99503,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"54'166","reported_seq":234,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986367+0000","last_change":"2023-11-25T04:25:51.923141+0000","last_active":"2023-11-25T04:26:00.986367+0000","last_peered":"2023-11-25T04:26:00.986367+0000","last_clean":"2023-11-25T04:26:00.986367+0000","last_became_active":"2023-11-25T04:25:51.922070+0000","last_became_peered":"2023-11-25T04:25:51.922070+0000","last_unstale":"2023-11-25T04:26:00.986367+0000","last_undegraded":"2023-11-25T04:26:00.986367+0000","last_fullsized":"2023-11-25T04:26:00.986367+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":185321,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":172,"num_write_kb":312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":164176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"58'50","reported_seq":54,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986198+0000","last_change":"2023-11-25T04:25:47.021705+0000","last_active":"2023-11-25T04:26:00.986198+0000","last_peered":"2023-11-25T04:26:00.986198+0000","last_clean":"2023-11-25T04:26:00.986198+0000","last_became_active":"2023-11-25T04:25:46.994997+0000","last_became_peered":"2023-11-25T04:25:46.994997+0000","last_unstale":"2023-11-25T04:26:00.986198+0000","last_undegraded":"2023-11-25T04:26:00.986198+0000","last_fullsized":"2023-11-25T04:26:00.986198+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":4119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4190183,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"54'153","reported_seq":222,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986103+0000","last_change":"2023-11-25T04:22:47.037181+0000","last_active":"2023-11-25T04:26:00.986103+0000","last_peered":"2023-11-25T04:26:00.986103+0000","last_clean":"2023-11-25T04:26:00.986103+0000","last_became_active":"2023-11-25T04:22:47.036821+0000","last_became_peered":"2023-11-25T04:22:47.036821+0000","last_unstale":"2023-11-25T04:26:00.986103+0000","last_undegraded":"2023-11-25T04:26:00.986103+0000","last_fullsized":"2023-11-25T04:26:00.986103+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":101117,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":161,"num_write_kb":242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"50'22","reported_seq":108,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986756+0000","last_change":"2023-11-25T04:25:47.028636+0000","last_active":"2023-11-25T04:26:00.986756+0000","last_peered":"2023-11-25T04:26:00.986756+0000","last_clean":"2023-11-25T04:26:00.986756+0000","last_became_active":"2023-11-25T04:25:47.000478+0000","last_became_peered":"2023-11-25T04:25:47.000478+0000","last_unstale":"2023-11-25T04:26:00.986756+0000","last_undegraded":"2023-11-25T04:26:00.986756+0000","last_fullsized":"2023-11-25T04:26:00.986756+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":90,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":35,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986686+0000","last_change":"2023-11-25T04:25:46.996182+0000","last_active":"2023-11-25T04:26:00.986686+0000","last_peered":"2023-11-25T04:26:00.986686+0000","last_clean":"2023-11-25T04:26:00.986686+0000","last_became_active":"2023-11-25T04:25:46.994525+0000","last_became_peered":"2023-11-25T04:25:46.994525+0000","last_unstale":"2023-11-25T04:26:00.986686+0000","last_undegraded":"2023-11-25T04:26:00.986686+0000","last_fullsized":"2023-11-25T04:26:00.986686+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"54'177","reported_seq":486,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.985858+0000","last_change":"2023-11-25T04:25:51.922466+0000","last_active":"2023-11-25T04:26:00.985858+0000","last_peered":"2023-11-25T04:26:00.985858+0000","last_clean":"2023-11-25T04:26:00.985858+0000","last_became_active":"2023-11-25T04:25:51.921938+0000","last_became_peered":"2023-11-25T04:25:51.921938+0000","last_unstale":"2023-11-25T04:26:00.985858+0000","last_undegraded":"2023-11-25T04:26:00.985858+0000","last_fullsized":"2023-11-25T04:26:00.985858+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":522494,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":168,"num_write_kb":749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":111,"num_bytes_recovered":522494,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"46'2","reported_seq":83,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986974+0000","last_change":"2023-11-25T04:25:46.998867+0000","last_active":"2023-11-25T04:26:00.986974+0000","last_peered":"2023-11-25T04:26:00.986974+0000","last_clean":"2023-11-25T04:26:00.986974+0000","last_became_active":"2023-11-25T04:25:46.998508+0000","last_became_peered":"2023-11-25T04:25:46.998508+0000","last_unstale":"2023-11-25T04:26:00.986974+0000","last_undegraded":"2023-11-25T04:26:00.986974+0000","last_fullsized":"2023-11-25T04:26:00.986974+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.10","version":"54'153","reported_seq":222,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.985832+0000","last_change":"2023-11-25T04:22:47.037790+0000","last_active":"2023-11-25T04:26:00.985832+0000","last_peered":"2023-11-25T04:26:00.985832+0000","last_clean":"2023-11-25T04:26:00.985832+0000","last_became_active":"2023-11-25T04:22:47.037149+0000","last_became_peered":"2023-11-25T04:22:47.037149+0000","last_unstale":"2023-11-25T04:26:00.985832+0000","last_undegraded":"2023-11-25T04:26:00.985832+0000","last_fullsized":"2023-11-25T04:26:00.985832+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":218073,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1f","version":"54'156","reported_seq":449,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986814+0000","last_change":"2023-11-25T04:25:49.921904+0000","last_active":"2023-11-25T04:26:00.986814+0000","last_peered":"2023-11-25T04:26:00.986814+0000","last_clean":"2023-11-25T04:26:00.986814+0000","last_became_active":"2023-11-25T04:25:49.921458+0000","last_became_peered":"2023-11-25T04:25:49.921458+0000","last_unstale":"2023-11-25T04:26:00.986814+0000","last_undegraded":"2023-11-25T04:26:00.986814+0000","last_fullsized":"2023-11-25T04:26:00.986814+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":107712,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":103,"num_bytes_recovered":107703,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"48'1","reported_seq":71,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986022+0000","last_change":"2023-11-25T04:22:46.036085+0000","last_active":"2023-11-25T04:26:00.986022+0000","last_peered":"2023-11-25T04:26:00.986022+0000","last_clean":"2023-11-25T04:26:00.986022+0000","last_became_active":"2023-11-25T04:22:46.035495+0000","last_became_peered":"2023-11-25T04:22:46.035495+0000","last_unstale":"2023-11-25T04:26:00.986022+0000","last_undegraded":"2023-11-25T04:26:00.986022+0000","last_fullsized":"2023-11-25T04:26:00.986022+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"54'152","reported_seq":431,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986735+0000","last_change":"2023-11-25T04:25:50.923882+0000","last_active":"2023-11-25T04:26:00.986735+0000","last_peered":"2023-11-25T04:26:00.986735+0000","last_clean":"2023-11-25T04:26:00.986735+0000","last_became_active":"2023-11-25T04:25:50.923375+0000","last_became_peered":"2023-11-25T04:25:50.923375+0000","last_unstale":"2023-11-25T04:26:00.986735+0000","last_undegraded":"2023-11-25T04:26:00.986735+0000","last_fullsized":"2023-11-25T04:26:00.986735+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":76435,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":76435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":89,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:23.683261+0000","last_change":"2023-11-25T04:26:23.683288+0000","last_active":"2023-11-25T04:26:23.683261+0000","last_peered":"2023-11-25T04:26:23.683261+0000","last_clean":"2023-11-25T04:26:23.683261+0000","last_became_active":"2023-11-25T04:25:46.994871+0000","last_became_peered":"2023-11-25T04:25:46.994871+0000","last_unstale":"2023-11-25T04:26:23.683261+0000","last_undegraded":"2023-11-25T04:26:23.683261+0000","last_fullsized":"2023-11-25T04:26:23.683261+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:26:23.683176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:26:23.683176+0000","last_clean_scrub_stamp":"2023-11-25T04:26:23.683176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:31:54.450606+0000","scrub_duration":0.00162587,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"54'158","reported_seq":433,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986512+0000","last_change":"2023-11-25T04:25:52.925523+0000","last_active":"2023-11-25T04:26:00.986512+0000","last_peered":"2023-11-25T04:26:00.986512+0000","last_clean":"2023-11-25T04:26:00.986512+0000","last_became_active":"2023-11-25T04:25:52.925038+0000","last_became_peered":"2023-11-25T04:25:52.925038+0000","last_unstale":"2023-11-25T04:26:00.986512+0000","last_undegraded":"2023-11-25T04:26:00.986512+0000","last_fullsized":"2023-11-25T04:26:00.986512+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":209500,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":139,"num_write_kb":351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":209500,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1a","version":"48'4","reported_seq":74,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986289+0000","last_change":"2023-11-25T04:22:46.034340+0000","last_active":"2023-11-25T04:26:00.986289+0000","last_peered":"2023-11-25T04:26:00.986289+0000","last_clean":"2023-11-25T04:26:00.986289+0000","last_became_active":"2023-11-25T04:22:46.033944+0000","last_became_peered":"2023-11-25T04:22:46.033944+0000","last_unstale":"2023-11-25T04:26:00.986289+0000","last_undegraded":"2023-11-25T04:26:00.986289+0000","last_fullsized":"2023-11-25T04:26:00.986289+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"75'218","reported_seq":297,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:28.540048+0000","last_change":"2023-11-25T04:26:28.540082+0000","last_active":"2023-11-25T04:26:28.540048+0000","last_peered":"2023-11-25T04:26:28.540048+0000","last_clean":"2023-11-25T04:26:28.540048+0000","last_became_active":"2023-11-25T04:22:47.039218+0000","last_became_peered":"2023-11-25T04:22:47.039218+0000","last_unstale":"2023-11-25T04:26:28.540048+0000","last_undegraded":"2023-11-25T04:26:28.540048+0000","last_fullsized":"2023-11-25T04:26:28.540048+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'218","last_scrub_stamp":"2023-11-25T04:26:28.539970+0000","last_deep_scrub":"75'218","last_deep_scrub_stamp":"2023-11-25T04:26:28.539970+0000","last_clean_scrub_stamp":"2023-11-25T04:26:28.539970+0000","objects_scrubbed":55,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:46:27.913704+0000","scrub_duration":0.040559401000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":165883,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"75'10","reported_seq":104,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:20.499200+0000","last_change":"2023-11-25T04:26:20.499236+0000","last_active":"2023-11-25T04:26:20.499200+0000","last_peered":"2023-11-25T04:26:20.499200+0000","last_clean":"2023-11-25T04:26:20.499200+0000","last_became_active":"2023-11-25T04:25:46.999998+0000","last_became_peered":"2023-11-25T04:25:46.999998+0000","last_unstale":"2023-11-25T04:26:20.499200+0000","last_undegraded":"2023-11-25T04:26:20.499200+0000","last_fullsized":"2023-11-25T04:26:20.499200+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'10","last_scrub_stamp":"2023-11-25T04:26:20.499093+0000","last_deep_scrub":"75'10","last_deep_scrub_stamp":"2023-11-25T04:26:20.499093+0000","last_clean_scrub_stamp":"2023-11-25T04:26:20.499093+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:35:51.569332+0000","scrub_duration":0.0094161829999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":4101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4194304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"54'149","reported_seq":218,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986245+0000","last_change":"2023-11-25T04:25:50.923041+0000","last_active":"2023-11-25T04:26:00.986245+0000","last_peered":"2023-11-25T04:26:00.986245+0000","last_clean":"2023-11-25T04:26:00.986245+0000","last_became_active":"2023-11-25T04:25:50.922164+0000","last_became_peered":"2023-11-25T04:25:50.922164+0000","last_unstale":"2023-11-25T04:26:00.986245+0000","last_undegraded":"2023-11-25T04:26:00.986245+0000","last_fullsized":"2023-11-25T04:26:00.986245+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":179506,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":155,"num_write_kb":317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":179506,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"18'2","reported_seq":38,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986217+0000","last_change":"2023-11-25T04:25:46.996675+0000","last_active":"2023-11-25T04:26:00.986217+0000","last_peered":"2023-11-25T04:26:00.986217+0000","last_clean":"2023-11-25T04:26:00.986217+0000","last_became_active":"2023-11-25T04:25:46.995160+0000","last_became_peered":"2023-11-25T04:25:46.995160+0000","last_unstale":"2023-11-25T04:26:00.986217+0000","last_undegraded":"2023-11-25T04:26:00.986217+0000","last_fullsized":"2023-11-25T04:26:00.986217+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1a","version":"54'162","reported_seq":227,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986038+0000","last_change":"2023-11-25T04:25:54.939182+0000","last_active":"2023-11-25T04:26:00.986038+0000","last_peered":"2023-11-25T04:26:00.986038+0000","last_clean":"2023-11-25T04:26:00.986038+0000","last_became_active":"2023-11-25T04:25:54.937821+0000","last_became_peered":"2023-11-25T04:25:54.937821+0000","last_unstale":"2023-11-25T04:26:00.986038+0000","last_undegraded":"2023-11-25T04:26:00.986038+0000","last_fullsized":"2023-11-25T04:26:00.986038+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":134672,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":169,"num_write_kb":311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":126291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"48'59","reported_seq":147,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986634+0000","last_change":"2023-11-25T04:25:47.077194+0000","last_active":"2023-11-25T04:26:00.986634+0000","last_peered":"2023-11-25T04:26:00.986634+0000","last_clean":"2023-11-25T04:26:00.986634+0000","last_became_active":"2023-11-25T04:25:47.000722+0000","last_became_peered":"2023-11-25T04:25:47.000722+0000","last_unstale":"2023-11-25T04:26:00.986634+0000","last_undegraded":"2023-11-25T04:26:00.986634+0000","last_fullsized":"2023-11-25T04:26:00.986634+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8388608,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"54'163","reported_seq":232,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986171+0000","last_change":"2023-11-25T04:22:47.036764+0000","last_active":"2023-11-25T04:26:00.986171+0000","last_peered":"2023-11-25T04:26:00.986171+0000","last_clean":"2023-11-25T04:26:00.986171+0000","last_became_active":"2023-11-25T04:22:47.036406+0000","last_became_peered":"2023-11-25T04:22:47.036406+0000","last_unstale":"2023-11-25T04:26:00.986171+0000","last_undegraded":"2023-11-25T04:26:00.986171+0000","last_fullsized":"2023-11-25T04:26:00.986171+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":527891,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":163,"num_write_kb":687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1e","version":"46'2","reported_seq":72,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986826+0000","last_change":"2023-11-25T04:22:46.035659+0000","last_active":"2023-11-25T04:26:00.986826+0000","last_peered":"2023-11-25T04:26:00.986826+0000","last_clean":"2023-11-25T04:26:00.986826+0000","last_became_active":"2023-11-25T04:22:46.035113+0000","last_became_peered":"2023-11-25T04:22:46.035113+0000","last_unstale":"2023-11-25T04:26:00.986826+0000","last_undegraded":"2023-11-25T04:26:00.986826+0000","last_fullsized":"2023-11-25T04:26:00.986826+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.18","version":"54'195","reported_seq":260,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986052+0000","last_change":"2023-11-25T04:25:54.938573+0000","last_active":"2023-11-25T04:26:00.986052+0000","last_peered":"2023-11-25T04:26:00.986052+0000","last_clean":"2023-11-25T04:26:00.986052+0000","last_became_active":"2023-11-25T04:25:54.937444+0000","last_became_peered":"2023-11-25T04:25:54.937444+0000","last_unstale":"2023-11-25T04:26:00.986052+0000","last_undegraded":"2023-11-25T04:26:00.986052+0000","last_fullsized":"2023-11-25T04:26:00.986052+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":128641,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":185,"num_write_kb":408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":122,"num_bytes_recovered":128641,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"50'17","reported_seq":103,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986905+0000","last_change":"2023-11-25T04:25:47.044228+0000","last_active":"2023-11-25T04:26:00.986905+0000","last_peered":"2023-11-25T04:26:00.986905+0000","last_clean":"2023-11-25T04:26:00.986905+0000","last_became_active":"2023-11-25T04:25:46.998870+0000","last_became_peered":"2023-11-25T04:25:46.998870+0000","last_unstale":"2023-11-25T04:26:00.986905+0000","last_undegraded":"2023-11-25T04:26:00.986905+0000","last_fullsized":"2023-11-25T04:26:00.986905+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":4103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4194304,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":26330855,"num_objects":3447,"num_object_clones":0,"num_object_copies":6894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10423,"num_write_kb":38550,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4189,"num_bytes_recovered":12749181,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":72376320,"data_stored":52324046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11145,"ondisk_log_size":11145,"up":128,"acting":128,"num_store_stats":3},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":40885548,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":35,"num_read_kb":334,"num_write":435,"num_write_kb":40462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":32491439,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":81833984,"data_stored":81771096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5548,"internal_metadata":0},"log_size":437,"ondisk_log_size":437,"up":64,"acting":64,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738420,"num_pgs":72,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77364,"kb_used_data":46188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94294476,"statfs":{"total":96636764160,"available":96557543424,"internally_reserved":0,"allocated":47296512,"data_stored":39787071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":1,"last update":"Sat Nov 25 04:25:40 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.433,"15min":0.44500000000000001},"min":{"1min":0.28899999999999998,"5min":0.216,"15min":0.216},"max":{"1min":0.60499999999999998,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.439,"15min":0.45100000000000001},"min":{"1min":0.25600000000000001,"5min":0.184,"15min":0.184},"max":{"1min":0.63400000000000001,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.40500000000000003}]}]},{"osd":1,"up_from":8,"seq":34359738420,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":91036,"kb_used_data":55700,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94280804,"statfs":{"total":96636764160,"available":96543543296,"internally_reserved":0,"allocated":57036800,"data_stored":50261554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":2,"last update":"Sat Nov 25 04:25:42 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.40600000000000003,"15min":0.41099999999999998},"min":{"1min":0.29099999999999998,"5min":0.182,"15min":0.182},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.40200000000000002,"15min":0.40600000000000003},"min":{"1min":0.312,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.53300000000000003}]}]},{"osd":0,"up_from":60,"seq":257698037770,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87612,"kb_used_data":51764,"kb_used_omap":6,"kb_used_meta":35833,"kb_avail":94284228,"statfs":{"total":96636764160,"available":96547049472,"internally_reserved":0,"allocated":53006336,"data_stored":46141107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7143,"internal_metadata":36693017},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32530432,"data_stored":32496940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5548,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":29380608,"data_stored":29360982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":19922944,"data_stored":19913174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":19234816,"data_stored":12749181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":26415104,"data_stored":20005586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":26726400,"data_stored":19569279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-25T04:26:30.577 INFO:tasks.ceph:pgid 4.3f last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.577 INFO:tasks.ceph:pgid 4.3c last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.577 INFO:tasks.ceph:pgid 4.3b last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.578 INFO:tasks.ceph:pgid 4.3a last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.578 INFO:tasks.ceph:pgid 4.39 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.578 INFO:tasks.ceph:pgid 4.38 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.578 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.579 INFO:tasks.ceph:pgid 4.34 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.579 INFO:tasks.ceph:pgid 4.32 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.579 INFO:tasks.ceph:pgid 4.31 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.580 INFO:tasks.ceph:pgid 4.2e last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.580 INFO:tasks.ceph:pgid 4.2d last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.580 INFO:tasks.ceph:pgid 4.2a last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.581 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.581 INFO:tasks.ceph:pgid 4.27 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.581 INFO:tasks.ceph:pgid 4.26 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.581 INFO:tasks.ceph:pgid 4.24 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.582 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.582 INFO:tasks.ceph:pgid 4.22 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.582 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.583 INFO:tasks.ceph:pgid 4.20 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.583 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.583 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.583 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.584 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.584 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.584 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.584 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.585 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.585 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.585 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.585 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.585 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.586 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.586 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.586 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.586 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.586 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.586 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.587 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.587 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.587 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.587 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-11-25T04:22:35.919850+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=35, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.587 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-11-25T04:22:40.987592+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=40, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.588 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.588 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.588 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-11-25T04:22:40.987592+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=40, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.588 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.588 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.588 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-11-25T04:22:40.987592+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=40, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.589 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-11-25T04:22:40.987592+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=40, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.589 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.589 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.589 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-11-25T04:22:40.987592+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=40, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.589 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.589 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.589 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-11-25T04:22:40.987592+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=40, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.590 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.590 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-11-25T04:22:40.987592+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=40, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.590 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.591 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.591 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-11-25T04:22:40.987592+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=40, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.591 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.591 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.591 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.591 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.592 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.592 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.592 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.592 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.592 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.592 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.593 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.593 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.593 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.593 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.593 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.593 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.594 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.594 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.594 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.594 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.594 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:30.594 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-11-25T04:26:50.596 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-25T04:26:50.913 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:26:50.914 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2023-11-25T04:26:50.924 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":201,"stamp":"2023-11-25T04:26:49.561464+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":67806771,"num_objects":3481,"num_object_clones":0,"num_object_copies":6962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3481,"num_whiteouts":0,"num_read":171,"num_read_kb":452,"num_write":11061,"num_write_kb":82472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4201,"num_bytes_recovered":45240620,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12417,"ondisk_log_size":12417,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":260172,"kb_used_data":153652,"kb_used_omap":10,"kb_used_meta":106485,"kb_avail":282855348,"statfs":{"total":289910292480,"available":289643876352,"internally_reserved":0,"allocated":157339648,"data_stored":136189732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10333,"internal_metadata":109041571},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002949"},"pg_stats":[{"pgid":"4.3f","version":"75'212","reported_seq":290,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:48.487585+0000","last_change":"2023-11-25T04:26:48.487625+0000","last_active":"2023-11-25T04:26:48.487585+0000","last_peered":"2023-11-25T04:26:48.487585+0000","last_clean":"2023-11-25T04:26:48.487585+0000","last_became_active":"2023-11-25T04:22:47.039490+0000","last_became_peered":"2023-11-25T04:22:47.039490+0000","last_unstale":"2023-11-25T04:26:48.487585+0000","last_undegraded":"2023-11-25T04:26:48.487585+0000","last_fullsized":"2023-11-25T04:26:48.487585+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'212","last_scrub_stamp":"2023-11-25T04:26:48.487485+0000","last_deep_scrub":"75'212","last_deep_scrub_stamp":"2023-11-25T04:26:48.487485+0000","last_clean_scrub_stamp":"2023-11-25T04:26:48.487485+0000","objects_scrubbed":46,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:02:55.160351+0000","scrub_duration":0.031121083000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":577866,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":174,"num_write_kb":756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3e","version":"75'257","reported_seq":335,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:16.435239+0000","last_change":"2023-11-25T04:26:16.435276+0000","last_active":"2023-11-25T04:26:16.435239+0000","last_peered":"2023-11-25T04:26:16.435239+0000","last_clean":"2023-11-25T04:26:16.435239+0000","last_became_active":"2023-11-25T04:22:47.040511+0000","last_became_peered":"2023-11-25T04:22:47.040511+0000","last_unstale":"2023-11-25T04:26:16.435239+0000","last_undegraded":"2023-11-25T04:26:16.435239+0000","last_fullsized":"2023-11-25T04:26:16.435239+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'257","last_scrub_stamp":"2023-11-25T04:26:16.435161+0000","last_deep_scrub":"75'257","last_deep_scrub_stamp":"2023-11-25T04:26:16.435161+0000","last_clean_scrub_stamp":"2023-11-25T04:26:16.435161+0000","objects_scrubbed":45,"log_size":257,"log_dups_size":0,"ondisk_log_size":257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:13:07.641904+0000","scrub_duration":0.036488224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":461579,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":236,"num_write_kb":841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3d","version":"75'209","reported_seq":285,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:12.330753+0000","last_change":"2023-11-25T04:26:12.330801+0000","last_active":"2023-11-25T04:26:12.330753+0000","last_peered":"2023-11-25T04:26:12.330753+0000","last_clean":"2023-11-25T04:26:12.330753+0000","last_became_active":"2023-11-25T04:25:52.923459+0000","last_became_peered":"2023-11-25T04:25:52.923459+0000","last_unstale":"2023-11-25T04:26:12.330753+0000","last_undegraded":"2023-11-25T04:26:12.330753+0000","last_fullsized":"2023-11-25T04:26:12.330753+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'209","last_scrub_stamp":"2023-11-25T04:26:12.330672+0000","last_deep_scrub":"75'209","last_deep_scrub_stamp":"2023-11-25T04:26:12.330672+0000","last_clean_scrub_stamp":"2023-11-25T04:26:12.330672+0000","objects_scrubbed":42,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:49:33.381569+0000","scrub_duration":0.027938688999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":145844,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":175,"num_write_kb":312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":138765,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"54'124","reported_seq":192,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986326+0000","last_change":"2023-11-25T04:25:51.922843+0000","last_active":"2023-11-25T04:26:00.986326+0000","last_peered":"2023-11-25T04:26:00.986326+0000","last_clean":"2023-11-25T04:26:00.986326+0000","last_became_active":"2023-11-25T04:25:51.921633+0000","last_became_peered":"2023-11-25T04:25:51.921633+0000","last_unstale":"2023-11-25T04:26:00.986326+0000","last_undegraded":"2023-11-25T04:26:00.986326+0000","last_fullsized":"2023-11-25T04:26:00.986326+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61303,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119,"num_write_kb":161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":61303,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"54'169","reported_seq":238,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986191+0000","last_change":"2023-11-25T04:22:47.039499+0000","last_active":"2023-11-25T04:26:00.986191+0000","last_peered":"2023-11-25T04:26:00.986191+0000","last_clean":"2023-11-25T04:26:00.986191+0000","last_became_active":"2023-11-25T04:22:47.039165+0000","last_became_peered":"2023-11-25T04:22:47.039165+0000","last_unstale":"2023-11-25T04:26:00.986191+0000","last_undegraded":"2023-11-25T04:26:00.986191+0000","last_fullsized":"2023-11-25T04:26:00.986191+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":118227,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":185,"num_write_kb":351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3a","version":"54'169","reported_seq":235,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986335+0000","last_change":"2023-11-25T04:25:53.933198+0000","last_active":"2023-11-25T04:26:00.986335+0000","last_peered":"2023-11-25T04:26:00.986335+0000","last_clean":"2023-11-25T04:26:00.986335+0000","last_became_active":"2023-11-25T04:25:53.931982+0000","last_became_peered":"2023-11-25T04:25:53.931982+0000","last_unstale":"2023-11-25T04:26:00.986335+0000","last_undegraded":"2023-11-25T04:26:00.986335+0000","last_fullsized":"2023-11-25T04:26:00.986335+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100304,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":161,"num_write_kb":424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":100304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"75'228","reported_seq":308,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:27.488226+0000","last_change":"2023-11-25T04:26:27.488269+0000","last_active":"2023-11-25T04:26:27.488226+0000","last_peered":"2023-11-25T04:26:27.488226+0000","last_clean":"2023-11-25T04:26:27.488226+0000","last_became_active":"2023-11-25T04:25:49.919354+0000","last_became_peered":"2023-11-25T04:25:49.919354+0000","last_unstale":"2023-11-25T04:26:27.488226+0000","last_undegraded":"2023-11-25T04:26:27.488226+0000","last_fullsized":"2023-11-25T04:26:27.488226+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'228","last_scrub_stamp":"2023-11-25T04:26:27.488145+0000","last_deep_scrub":"75'228","last_deep_scrub_stamp":"2023-11-25T04:26:27.488145+0000","last_clean_scrub_stamp":"2023-11-25T04:26:27.488145+0000","objects_scrubbed":60,"log_size":228,"log_dups_size":0,"ondisk_log_size":228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:22:24.257272+0000","scrub_duration":0.042766632999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":124558,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":167,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":111,"num_bytes_recovered":124533,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.38","version":"75'199","reported_seq":278,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:28.501531+0000","last_change":"2023-11-25T04:26:28.501582+0000","last_active":"2023-11-25T04:26:28.501531+0000","last_peered":"2023-11-25T04:26:28.501531+0000","last_clean":"2023-11-25T04:26:28.501531+0000","last_became_active":"2023-11-25T04:25:49.919236+0000","last_became_peered":"2023-11-25T04:25:49.919236+0000","last_unstale":"2023-11-25T04:26:28.501531+0000","last_undegraded":"2023-11-25T04:26:28.501531+0000","last_fullsized":"2023-11-25T04:26:28.501531+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'199","last_scrub_stamp":"2023-11-25T04:26:28.501441+0000","last_deep_scrub":"75'199","last_deep_scrub_stamp":"2023-11-25T04:26:28.501441+0000","last_clean_scrub_stamp":"2023-11-25T04:26:28.501441+0000","objects_scrubbed":47,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:21:13.937490+0000","scrub_duration":0.030866581000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":120147,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":120121,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.37","version":"75'220","reported_seq":299,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:28.772505+0000","last_change":"2023-11-25T04:26:28.772532+0000","last_active":"2023-11-25T04:26:28.772505+0000","last_peered":"2023-11-25T04:26:28.772505+0000","last_clean":"2023-11-25T04:26:28.772505+0000","last_became_active":"2023-11-25T04:22:47.037328+0000","last_became_peered":"2023-11-25T04:22:47.037328+0000","last_unstale":"2023-11-25T04:26:28.772505+0000","last_undegraded":"2023-11-25T04:26:28.772505+0000","last_fullsized":"2023-11-25T04:26:28.772505+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'220","last_scrub_stamp":"2023-11-25T04:26:28.772426+0000","last_deep_scrub":"75'220","last_deep_scrub_stamp":"2023-11-25T04:26:28.772426+0000","last_clean_scrub_stamp":"2023-11-25T04:26:28.772426+0000","objects_scrubbed":53,"log_size":220,"log_dups_size":0,"ondisk_log_size":220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:08:21.318788+0000","scrub_duration":0.036194551999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":174147,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":184,"num_write_kb":353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.36","version":"75'200","reported_seq":279,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:24.608814+0000","last_change":"2023-11-25T04:26:24.608840+0000","last_active":"2023-11-25T04:26:24.608814+0000","last_peered":"2023-11-25T04:26:24.608814+0000","last_clean":"2023-11-25T04:26:24.608814+0000","last_became_active":"2023-11-25T04:22:47.039892+0000","last_became_peered":"2023-11-25T04:22:47.039892+0000","last_unstale":"2023-11-25T04:26:24.608814+0000","last_undegraded":"2023-11-25T04:26:24.608814+0000","last_fullsized":"2023-11-25T04:26:24.608814+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'200","last_scrub_stamp":"2023-11-25T04:26:24.608737+0000","last_deep_scrub":"75'200","last_deep_scrub_stamp":"2023-11-25T04:26:24.608737+0000","last_clean_scrub_stamp":"2023-11-25T04:26:24.608737+0000","objects_scrubbed":51,"log_size":200,"log_dups_size":0,"ondisk_log_size":200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:49:43.116414+0000","scrub_duration":0.03530929,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":233330,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"75'211","reported_seq":289,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:25.542014+0000","last_change":"2023-11-25T04:26:25.542044+0000","last_active":"2023-11-25T04:26:25.542014+0000","last_peered":"2023-11-25T04:26:25.542014+0000","last_clean":"2023-11-25T04:26:25.542014+0000","last_became_active":"2023-11-25T04:25:51.921800+0000","last_became_peered":"2023-11-25T04:25:51.921800+0000","last_unstale":"2023-11-25T04:26:25.542014+0000","last_undegraded":"2023-11-25T04:26:25.542014+0000","last_fullsized":"2023-11-25T04:26:25.542014+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'211","last_scrub_stamp":"2023-11-25T04:26:25.541938+0000","last_deep_scrub":"75'211","last_deep_scrub_stamp":"2023-11-25T04:26:25.541938+0000","last_clean_scrub_stamp":"2023-11-25T04:26:25.541938+0000","objects_scrubbed":55,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:32:27.981332+0000","scrub_duration":0.043489338000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":302550,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":163,"num_write_kb":431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":300323,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.34","version":"54'136","reported_seq":401,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986260+0000","last_change":"2023-11-25T04:25:50.922843+0000","last_active":"2023-11-25T04:26:00.986260+0000","last_peered":"2023-11-25T04:26:00.986260+0000","last_clean":"2023-11-25T04:26:00.986260+0000","last_became_active":"2023-11-25T04:25:50.922309+0000","last_became_peered":"2023-11-25T04:25:50.922309+0000","last_unstale":"2023-11-25T04:26:00.986260+0000","last_undegraded":"2023-11-25T04:26:00.986260+0000","last_fullsized":"2023-11-25T04:26:00.986260+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":93053,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":131,"num_write_kb":248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":89,"num_bytes_recovered":93053,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"75'183","reported_seq":261,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:19.516152+0000","last_change":"2023-11-25T04:26:19.516218+0000","last_active":"2023-11-25T04:26:19.516152+0000","last_peered":"2023-11-25T04:26:19.516152+0000","last_clean":"2023-11-25T04:26:19.516152+0000","last_became_active":"2023-11-25T04:22:47.040129+0000","last_became_peered":"2023-11-25T04:22:47.040129+0000","last_unstale":"2023-11-25T04:26:19.516152+0000","last_undegraded":"2023-11-25T04:26:19.516152+0000","last_fullsized":"2023-11-25T04:26:19.516152+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'183","last_scrub_stamp":"2023-11-25T04:26:19.516065+0000","last_deep_scrub":"75'183","last_deep_scrub_stamp":"2023-11-25T04:26:19.516065+0000","last_clean_scrub_stamp":"2023-11-25T04:26:19.516065+0000","objects_scrubbed":49,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:43:33.021008+0000","scrub_duration":0.038467327000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":149524,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":139,"num_write_kb":252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.32","version":"54'174","reported_seq":240,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986344+0000","last_change":"2023-11-25T04:25:53.934526+0000","last_active":"2023-11-25T04:26:00.986344+0000","last_peered":"2023-11-25T04:26:00.986344+0000","last_clean":"2023-11-25T04:26:00.986344+0000","last_became_active":"2023-11-25T04:25:53.933694+0000","last_became_peered":"2023-11-25T04:25:53.933694+0000","last_unstale":"2023-11-25T04:26:00.986344+0000","last_undegraded":"2023-11-25T04:26:00.986344+0000","last_fullsized":"2023-11-25T04:26:00.986344+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":95797,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":181,"num_write_kb":547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":103,"num_bytes_recovered":95797,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.31","version":"75'212","reported_seq":292,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:29.518025+0000","last_change":"2023-11-25T04:26:29.518088+0000","last_active":"2023-11-25T04:26:29.518025+0000","last_peered":"2023-11-25T04:26:29.518025+0000","last_clean":"2023-11-25T04:26:29.518025+0000","last_became_active":"2023-11-25T04:25:49.921129+0000","last_became_peered":"2023-11-25T04:25:49.921129+0000","last_unstale":"2023-11-25T04:26:29.518025+0000","last_undegraded":"2023-11-25T04:26:29.518025+0000","last_fullsized":"2023-11-25T04:26:29.518025+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'212","last_scrub_stamp":"2023-11-25T04:26:29.517931+0000","last_deep_scrub":"75'212","last_deep_scrub_stamp":"2023-11-25T04:26:29.517931+0000","last_clean_scrub_stamp":"2023-11-25T04:26:29.517931+0000","objects_scrubbed":56,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:49:54.902849+0000","scrub_duration":0.039475015000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":163190,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":169,"num_write_kb":336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":154576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"75'229","reported_seq":305,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:20.470401+0000","last_change":"2023-11-25T04:26:20.470429+0000","last_active":"2023-11-25T04:26:20.470401+0000","last_peered":"2023-11-25T04:26:20.470401+0000","last_clean":"2023-11-25T04:26:20.470401+0000","last_became_active":"2023-11-25T04:25:53.933395+0000","last_became_peered":"2023-11-25T04:25:53.933395+0000","last_unstale":"2023-11-25T04:26:20.470401+0000","last_undegraded":"2023-11-25T04:26:20.470401+0000","last_fullsized":"2023-11-25T04:26:20.470401+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'229","last_scrub_stamp":"2023-11-25T04:26:20.470325+0000","last_deep_scrub":"75'229","last_deep_scrub_stamp":"2023-11-25T04:26:20.470325+0000","last_clean_scrub_stamp":"2023-11-25T04:26:20.470325+0000","objects_scrubbed":55,"log_size":229,"log_dups_size":0,"ondisk_log_size":229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:55:52.038708+0000","scrub_duration":0.036798694,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1423587,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":182,"num_write_kb":1626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":1414508,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2f","version":"75'203","reported_seq":282,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:18.718015+0000","last_change":"2023-11-25T04:26:18.718065+0000","last_active":"2023-11-25T04:26:18.718015+0000","last_peered":"2023-11-25T04:26:18.718015+0000","last_clean":"2023-11-25T04:26:18.718015+0000","last_became_active":"2023-11-25T04:22:47.039123+0000","last_became_peered":"2023-11-25T04:22:47.039123+0000","last_unstale":"2023-11-25T04:26:18.718015+0000","last_undegraded":"2023-11-25T04:26:18.718015+0000","last_fullsized":"2023-11-25T04:26:18.718015+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'203","last_scrub_stamp":"2023-11-25T04:26:18.717922+0000","last_deep_scrub":"75'203","last_deep_scrub_stamp":"2023-11-25T04:26:18.717922+0000","last_clean_scrub_stamp":"2023-11-25T04:26:18.717922+0000","objects_scrubbed":51,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:33:40.286516+0000","scrub_duration":0.039125398999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":122752,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":143,"num_write_kb":435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"54'148","reported_seq":217,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986605+0000","last_change":"2023-11-25T04:22:47.036946+0000","last_active":"2023-11-25T04:26:00.986605+0000","last_peered":"2023-11-25T04:26:00.986605+0000","last_clean":"2023-11-25T04:26:00.986605+0000","last_became_active":"2023-11-25T04:22:47.036411+0000","last_became_peered":"2023-11-25T04:22:47.036411+0000","last_unstale":"2023-11-25T04:26:00.986605+0000","last_undegraded":"2023-11-25T04:26:00.986605+0000","last_fullsized":"2023-11-25T04:26:00.986605+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4770899,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":140,"num_write_kb":4870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2d","version":"75'194","reported_seq":273,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:47.814115+0000","last_change":"2023-11-25T04:26:47.814157+0000","last_active":"2023-11-25T04:26:47.814115+0000","last_peered":"2023-11-25T04:26:47.814115+0000","last_clean":"2023-11-25T04:26:47.814115+0000","last_became_active":"2023-11-25T04:22:47.038477+0000","last_became_peered":"2023-11-25T04:22:47.038477+0000","last_unstale":"2023-11-25T04:26:47.814115+0000","last_undegraded":"2023-11-25T04:26:47.814115+0000","last_fullsized":"2023-11-25T04:26:47.814115+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'194","last_scrub_stamp":"2023-11-25T04:26:47.814018+0000","last_deep_scrub":"75'194","last_deep_scrub_stamp":"2023-11-25T04:26:47.814018+0000","last_clean_scrub_stamp":"2023-11-25T04:26:47.814018+0000","objects_scrubbed":51,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:20:21.601431+0000","scrub_duration":0.038348491999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":115975,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"75'230","reported_seq":307,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:19.428426+0000","last_change":"2023-11-25T04:26:19.428460+0000","last_active":"2023-11-25T04:26:19.428426+0000","last_peered":"2023-11-25T04:26:19.428426+0000","last_clean":"2023-11-25T04:26:19.428426+0000","last_became_active":"2023-11-25T04:25:52.922367+0000","last_became_peered":"2023-11-25T04:25:52.922367+0000","last_unstale":"2023-11-25T04:26:19.428426+0000","last_undegraded":"2023-11-25T04:26:19.428426+0000","last_fullsized":"2023-11-25T04:26:19.428426+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'230","last_scrub_stamp":"2023-11-25T04:26:19.428350+0000","last_deep_scrub":"75'230","last_deep_scrub_stamp":"2023-11-25T04:26:19.428350+0000","last_clean_scrub_stamp":"2023-11-25T04:26:19.428350+0000","objects_scrubbed":63,"log_size":230,"log_dups_size":0,"ondisk_log_size":230,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:10:12.216682+0000","scrub_duration":0.042625061999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":747090,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":211,"num_write_kb":985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":732276,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"75'211","reported_seq":288,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:17.381634+0000","last_change":"2023-11-25T04:26:17.381662+0000","last_active":"2023-11-25T04:26:17.381634+0000","last_peered":"2023-11-25T04:26:17.381634+0000","last_clean":"2023-11-25T04:26:17.381634+0000","last_became_active":"2023-11-25T04:25:52.923378+0000","last_became_peered":"2023-11-25T04:25:52.923378+0000","last_unstale":"2023-11-25T04:26:17.381634+0000","last_undegraded":"2023-11-25T04:26:17.381634+0000","last_fullsized":"2023-11-25T04:26:17.381634+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'211","last_scrub_stamp":"2023-11-25T04:26:17.381560+0000","last_deep_scrub":"75'211","last_deep_scrub_stamp":"2023-11-25T04:26:17.381560+0000","last_clean_scrub_stamp":"2023-11-25T04:26:17.381560+0000","objects_scrubbed":55,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:26:19.664059+0000","scrub_duration":0.036605545000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69580,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":142,"num_write_kb":315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":98,"num_bytes_recovered":69580,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2a","version":"54'171","reported_seq":468,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.985986+0000","last_change":"2023-11-25T04:25:52.925532+0000","last_active":"2023-11-25T04:26:00.985986+0000","last_peered":"2023-11-25T04:26:00.985986+0000","last_clean":"2023-11-25T04:26:00.985986+0000","last_became_active":"2023-11-25T04:25:52.924844+0000","last_became_peered":"2023-11-25T04:25:52.924844+0000","last_unstale":"2023-11-25T04:26:00.985986+0000","last_undegraded":"2023-11-25T04:26:00.985986+0000","last_fullsized":"2023-11-25T04:26:00.985986+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":115997,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":162,"num_write_kb":261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":115997,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.29","version":"75'211","reported_seq":508,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:12.692578+0000","last_change":"2023-11-25T04:26:12.692632+0000","last_active":"2023-11-25T04:26:12.692578+0000","last_peered":"2023-11-25T04:26:12.692578+0000","last_clean":"2023-11-25T04:26:12.692578+0000","last_became_active":"2023-11-25T04:25:49.919886+0000","last_became_peered":"2023-11-25T04:25:49.919886+0000","last_unstale":"2023-11-25T04:26:12.692578+0000","last_undegraded":"2023-11-25T04:26:12.692578+0000","last_fullsized":"2023-11-25T04:26:12.692578+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'211","last_scrub_stamp":"2023-11-25T04:26:12.692467+0000","last_deep_scrub":"75'211","last_deep_scrub_stamp":"2023-11-25T04:26:12.692467+0000","last_clean_scrub_stamp":"2023-11-25T04:26:12.692467+0000","objects_scrubbed":59,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:13:03.899209+0000","scrub_duration":0.043132342999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":212878,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":212878,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"54'169","reported_seq":238,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.985651+0000","last_change":"2023-11-25T04:22:47.036805+0000","last_active":"2023-11-25T04:26:00.985651+0000","last_peered":"2023-11-25T04:26:00.985651+0000","last_clean":"2023-11-25T04:26:00.985651+0000","last_became_active":"2023-11-25T04:22:47.036135+0000","last_became_peered":"2023-11-25T04:22:47.036135+0000","last_unstale":"2023-11-25T04:26:00.985651+0000","last_undegraded":"2023-11-25T04:26:00.985651+0000","last_fullsized":"2023-11-25T04:26:00.985651+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":102957,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":172,"num_write_kb":610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"54'167","reported_seq":466,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986816+0000","last_change":"2023-11-25T04:25:51.922925+0000","last_active":"2023-11-25T04:26:00.986816+0000","last_peered":"2023-11-25T04:26:00.986816+0000","last_clean":"2023-11-25T04:26:00.986816+0000","last_became_active":"2023-11-25T04:25:51.922364+0000","last_became_peered":"2023-11-25T04:25:51.922364+0000","last_unstale":"2023-11-25T04:26:00.986816+0000","last_undegraded":"2023-11-25T04:26:00.986816+0000","last_fullsized":"2023-11-25T04:26:00.986816+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":182683,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":171,"num_write_kb":791,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":182683,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"54'187","reported_seq":506,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986695+0000","last_change":"2023-11-25T04:25:51.922529+0000","last_active":"2023-11-25T04:26:00.986695+0000","last_peered":"2023-11-25T04:26:00.986695+0000","last_clean":"2023-11-25T04:26:00.986695+0000","last_became_active":"2023-11-25T04:25:51.922064+0000","last_became_peered":"2023-11-25T04:25:51.922064+0000","last_unstale":"2023-11-25T04:26:00.986695+0000","last_undegraded":"2023-11-25T04:26:00.986695+0000","last_fullsized":"2023-11-25T04:26:00.986695+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":187,"log_dups_size":0,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":86311,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":171,"num_write_kb":274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":116,"num_bytes_recovered":86311,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.25","version":"75'201","reported_seq":278,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:10.399641+0000","last_change":"2023-11-25T04:26:10.399668+0000","last_active":"2023-11-25T04:26:10.399641+0000","last_peered":"2023-11-25T04:26:10.399641+0000","last_clean":"2023-11-25T04:26:10.399641+0000","last_became_active":"2023-11-25T04:25:52.924795+0000","last_became_peered":"2023-11-25T04:25:52.924795+0000","last_unstale":"2023-11-25T04:26:10.399641+0000","last_undegraded":"2023-11-25T04:26:10.399641+0000","last_fullsized":"2023-11-25T04:26:10.399641+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'201","last_scrub_stamp":"2023-11-25T04:26:10.399561+0000","last_deep_scrub":"75'201","last_deep_scrub_stamp":"2023-11-25T04:26:10.399561+0000","last_clean_scrub_stamp":"2023-11-25T04:26:10.399561+0000","objects_scrubbed":52,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:15:15.077501+0000","scrub_duration":0.038027564,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":128288,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":167,"num_write_kb":267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":120265,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.24","version":"75'195","reported_seq":481,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:43.546662+0000","last_change":"2023-11-25T04:26:43.546743+0000","last_active":"2023-11-25T04:26:43.546662+0000","last_peered":"2023-11-25T04:26:43.546662+0000","last_clean":"2023-11-25T04:26:43.546662+0000","last_became_active":"2023-11-25T04:25:50.922216+0000","last_became_peered":"2023-11-25T04:25:50.922216+0000","last_unstale":"2023-11-25T04:26:43.546662+0000","last_undegraded":"2023-11-25T04:26:43.546662+0000","last_fullsized":"2023-11-25T04:26:43.546662+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'195","last_scrub_stamp":"2023-11-25T04:26:43.546552+0000","last_deep_scrub":"75'195","last_deep_scrub_stamp":"2023-11-25T04:26:43.546552+0000","last_clean_scrub_stamp":"2023-11-25T04:26:43.546552+0000","objects_scrubbed":44,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:16:44.328689+0000","scrub_duration":0.034140957,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":107507,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":148,"num_write_kb":288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":107494,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.23","version":"54'155","reported_seq":224,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986172+0000","last_change":"2023-11-25T04:22:47.037668+0000","last_active":"2023-11-25T04:26:00.986172+0000","last_peered":"2023-11-25T04:26:00.986172+0000","last_clean":"2023-11-25T04:26:00.986172+0000","last_became_active":"2023-11-25T04:22:47.036823+0000","last_became_peered":"2023-11-25T04:22:47.036823+0000","last_unstale":"2023-11-25T04:26:00.986172+0000","last_undegraded":"2023-11-25T04:26:00.986172+0000","last_fullsized":"2023-11-25T04:26:00.986172+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4304582,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":152,"num_write_kb":4331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.22","version":"54'189","reported_seq":518,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986099+0000","last_change":"2023-11-25T04:25:54.940559+0000","last_active":"2023-11-25T04:26:00.986099+0000","last_peered":"2023-11-25T04:26:00.986099+0000","last_clean":"2023-11-25T04:26:00.986099+0000","last_became_active":"2023-11-25T04:25:54.939584+0000","last_became_peered":"2023-11-25T04:25:54.939584+0000","last_unstale":"2023-11-25T04:26:00.986099+0000","last_undegraded":"2023-11-25T04:26:00.986099+0000","last_fullsized":"2023-11-25T04:26:00.986099+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":129145,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":185,"num_write_kb":432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":121,"num_bytes_recovered":129107,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"54'201","reported_seq":270,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.985795+0000","last_change":"2023-11-25T04:22:47.037786+0000","last_active":"2023-11-25T04:26:00.985795+0000","last_peered":"2023-11-25T04:26:00.985795+0000","last_clean":"2023-11-25T04:26:00.985795+0000","last_became_active":"2023-11-25T04:22:47.037206+0000","last_became_peered":"2023-11-25T04:22:47.037206+0000","last_unstale":"2023-11-25T04:26:00.985795+0000","last_undegraded":"2023-11-25T04:26:00.985795+0000","last_fullsized":"2023-11-25T04:26:00.985795+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":215311,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":194,"num_write_kb":381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"54'180","reported_seq":246,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.985760+0000","last_change":"2023-11-25T04:25:53.933176+0000","last_active":"2023-11-25T04:26:00.985760+0000","last_peered":"2023-11-25T04:26:00.985760+0000","last_clean":"2023-11-25T04:26:00.985760+0000","last_became_active":"2023-11-25T04:25:53.931937+0000","last_became_peered":"2023-11-25T04:25:53.931937+0000","last_unstale":"2023-11-25T04:26:00.985760+0000","last_undegraded":"2023-11-25T04:26:00.985760+0000","last_fullsized":"2023-11-25T04:26:00.985760+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104603,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":172,"num_write_kb":267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":104603,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"75'193","reported_seq":272,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:32.532707+0000","last_change":"2023-11-25T04:26:32.532758+0000","last_active":"2023-11-25T04:26:32.532707+0000","last_peered":"2023-11-25T04:26:32.532707+0000","last_clean":"2023-11-25T04:26:32.532707+0000","last_became_active":"2023-11-25T04:22:47.036107+0000","last_became_peered":"2023-11-25T04:22:47.036107+0000","last_unstale":"2023-11-25T04:26:32.532707+0000","last_undegraded":"2023-11-25T04:26:32.532707+0000","last_fullsized":"2023-11-25T04:26:32.532707+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'193","last_scrub_stamp":"2023-11-25T04:26:32.532624+0000","last_deep_scrub":"75'193","last_deep_scrub_stamp":"2023-11-25T04:26:32.532624+0000","last_clean_scrub_stamp":"2023-11-25T04:26:32.532624+0000","objects_scrubbed":56,"log_size":193,"log_dups_size":0,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:09:38.769552+0000","scrub_duration":0.038075093999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":183586,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":127,"num_write_kb":284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"75'117","reported_seq":203,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:31.496279+0000","last_change":"2023-11-25T04:26:31.496307+0000","last_active":"2023-11-25T04:26:31.496279+0000","last_peered":"2023-11-25T04:26:31.496279+0000","last_clean":"2023-11-25T04:26:31.496279+0000","last_became_active":"2023-11-25T04:25:49.920010+0000","last_became_peered":"2023-11-25T04:25:49.920010+0000","last_unstale":"2023-11-25T04:26:31.496279+0000","last_undegraded":"2023-11-25T04:26:31.496279+0000","last_fullsized":"2023-11-25T04:26:31.496279+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'117","last_scrub_stamp":"2023-11-25T04:26:31.496196+0000","last_deep_scrub":"75'117","last_deep_scrub_stamp":"2023-11-25T04:26:31.496196+0000","last_clean_scrub_stamp":"2023-11-25T04:26:31.496196+0000","objects_scrubbed":3,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:26:02.874431+0000","scrub_duration":0.0076030029999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2816216,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":113,"num_write_kb":2816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":2816182,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"75'189","reported_seq":267,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:40.589192+0000","last_change":"2023-11-25T04:26:40.589251+0000","last_active":"2023-11-25T04:26:40.589192+0000","last_peered":"2023-11-25T04:26:40.589192+0000","last_clean":"2023-11-25T04:26:40.589192+0000","last_became_active":"2023-11-25T04:22:47.037158+0000","last_became_peered":"2023-11-25T04:22:47.037158+0000","last_unstale":"2023-11-25T04:26:40.589192+0000","last_undegraded":"2023-11-25T04:26:40.589192+0000","last_fullsized":"2023-11-25T04:26:40.589192+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'189","last_scrub_stamp":"2023-11-25T04:26:40.589106+0000","last_deep_scrub":"75'189","last_deep_scrub_stamp":"2023-11-25T04:26:40.589106+0000","last_clean_scrub_stamp":"2023-11-25T04:26:40.589106+0000","objects_scrubbed":44,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:32:56.267980+0000","scrub_duration":0.032171351000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":205976,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.12","version":"49'2","reported_seq":91,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:30.784729+0000","last_change":"2023-11-25T04:26:30.784752+0000","last_active":"2023-11-25T04:26:30.784729+0000","last_peered":"2023-11-25T04:26:30.784729+0000","last_clean":"2023-11-25T04:26:30.784729+0000","last_became_active":"2023-11-25T04:25:47.000516+0000","last_became_peered":"2023-11-25T04:25:47.000516+0000","last_unstale":"2023-11-25T04:26:30.784729+0000","last_undegraded":"2023-11-25T04:26:30.784729+0000","last_fullsized":"2023-11-25T04:26:30.784729+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'2","last_scrub_stamp":"2023-11-25T04:26:30.784639+0000","last_deep_scrub":"49'2","last_deep_scrub_stamp":"2023-11-25T04:26:30.784639+0000","last_clean_scrub_stamp":"2023-11-25T04:26:30.784639+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:38:15.370559+0000","scrub_duration":0.001803528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"54'175","reported_seq":490,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986834+0000","last_change":"2023-11-25T04:25:52.923694+0000","last_active":"2023-11-25T04:26:00.986834+0000","last_peered":"2023-11-25T04:26:00.986834+0000","last_clean":"2023-11-25T04:26:00.986834+0000","last_became_active":"2023-11-25T04:25:52.923045+0000","last_became_peered":"2023-11-25T04:25:52.923045+0000","last_unstale":"2023-11-25T04:26:00.986834+0000","last_undegraded":"2023-11-25T04:26:00.986834+0000","last_fullsized":"2023-11-25T04:26:00.986834+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":171823,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":185,"num_write_kb":340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":166387,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":89,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:18.488426+0000","last_change":"2023-11-25T04:26:18.488445+0000","last_active":"2023-11-25T04:26:18.488426+0000","last_peered":"2023-11-25T04:26:18.488426+0000","last_clean":"2023-11-25T04:26:18.488426+0000","last_became_active":"2023-11-25T04:25:46.993870+0000","last_became_peered":"2023-11-25T04:25:46.993870+0000","last_unstale":"2023-11-25T04:26:18.488426+0000","last_undegraded":"2023-11-25T04:26:18.488426+0000","last_fullsized":"2023-11-25T04:26:18.488426+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:26:18.488345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:26:18.488345+0000","last_clean_scrub_stamp":"2023-11-25T04:26:18.488345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:43:21.558596+0000","scrub_duration":0.002108399,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"54'152","reported_seq":436,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986747+0000","last_change":"2023-11-25T04:25:50.922922+0000","last_active":"2023-11-25T04:26:00.986747+0000","last_peered":"2023-11-25T04:26:00.986747+0000","last_clean":"2023-11-25T04:26:00.986747+0000","last_became_active":"2023-11-25T04:25:50.922511+0000","last_became_peered":"2023-11-25T04:25:50.922511+0000","last_unstale":"2023-11-25T04:26:00.986747+0000","last_undegraded":"2023-11-25T04:26:00.986747+0000","last_fullsized":"2023-11-25T04:26:00.986747+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":579704,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":97,"num_bytes_recovered":570044,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.10","version":"18'8","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.985749+0000","last_change":"2023-11-25T04:25:47.000499+0000","last_active":"2023-11-25T04:26:00.985749+0000","last_peered":"2023-11-25T04:26:00.985749+0000","last_clean":"2023-11-25T04:26:00.985749+0000","last_became_active":"2023-11-25T04:25:46.998778+0000","last_became_peered":"2023-11-25T04:25:46.998778+0000","last_unstale":"2023-11-25T04:26:00.985749+0000","last_undegraded":"2023-11-25T04:26:00.985749+0000","last_fullsized":"2023-11-25T04:26:00.985749+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"75'3","reported_seq":47,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:36.494324+0000","last_change":"2023-11-25T04:26:36.494358+0000","last_active":"2023-11-25T04:26:36.494324+0000","last_peered":"2023-11-25T04:26:36.494324+0000","last_clean":"2023-11-25T04:26:36.494324+0000","last_became_active":"2023-11-25T04:25:46.997108+0000","last_became_peered":"2023-11-25T04:25:46.997108+0000","last_unstale":"2023-11-25T04:26:36.494324+0000","last_undegraded":"2023-11-25T04:26:36.494324+0000","last_fullsized":"2023-11-25T04:26:36.494324+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'3","last_scrub_stamp":"2023-11-25T04:26:36.494205+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2023-11-25T04:26:36.494205+0000","last_clean_scrub_stamp":"2023-11-25T04:26:36.494205+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:47:03.918120+0000","scrub_duration":0.0026531879999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"54'159","reported_seq":444,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.985722+0000","last_change":"2023-11-25T04:25:50.922282+0000","last_active":"2023-11-25T04:26:00.985722+0000","last_peered":"2023-11-25T04:26:00.985722+0000","last_clean":"2023-11-25T04:26:00.985722+0000","last_became_active":"2023-11-25T04:25:50.921772+0000","last_became_peered":"2023-11-25T04:25:50.921772+0000","last_unstale":"2023-11-25T04:26:00.985722+0000","last_undegraded":"2023-11-25T04:26:00.985722+0000","last_fullsized":"2023-11-25T04:26:00.985722+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112103,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":164,"num_write_kb":240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":99,"num_bytes_recovered":103472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"75'3","reported_seq":92,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:36.492691+0000","last_change":"2023-11-25T04:26:36.492740+0000","last_active":"2023-11-25T04:26:36.492691+0000","last_peered":"2023-11-25T04:26:36.492691+0000","last_clean":"2023-11-25T04:26:36.492691+0000","last_became_active":"2023-11-25T04:25:46.994640+0000","last_became_peered":"2023-11-25T04:25:46.994640+0000","last_unstale":"2023-11-25T04:26:36.492691+0000","last_undegraded":"2023-11-25T04:26:36.492691+0000","last_fullsized":"2023-11-25T04:26:36.492691+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'3","last_scrub_stamp":"2023-11-25T04:26:36.492569+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2023-11-25T04:26:36.492569+0000","last_clean_scrub_stamp":"2023-11-25T04:26:36.492569+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:28:25.392135+0000","scrub_duration":0.0033623030000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"54'173","reported_seq":242,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.985870+0000","last_change":"2023-11-25T04:22:47.039685+0000","last_active":"2023-11-25T04:26:00.985870+0000","last_peered":"2023-11-25T04:26:00.985870+0000","last_clean":"2023-11-25T04:26:00.985870+0000","last_became_active":"2023-11-25T04:22:47.039296+0000","last_became_peered":"2023-11-25T04:22:47.039296+0000","last_unstale":"2023-11-25T04:26:00.985870+0000","last_undegraded":"2023-11-25T04:26:00.985870+0000","last_fullsized":"2023-11-25T04:26:00.985870+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154533,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":182,"num_write_kb":299,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"75'2","reported_seq":47,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:33.490361+0000","last_change":"2023-11-25T04:26:33.490414+0000","last_active":"2023-11-25T04:26:33.490361+0000","last_peered":"2023-11-25T04:26:33.490361+0000","last_clean":"2023-11-25T04:26:33.490361+0000","last_became_active":"2023-11-25T04:25:46.999604+0000","last_became_peered":"2023-11-25T04:25:46.999604+0000","last_unstale":"2023-11-25T04:26:33.490361+0000","last_undegraded":"2023-11-25T04:26:33.490361+0000","last_fullsized":"2023-11-25T04:26:33.490361+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'2","last_scrub_stamp":"2023-11-25T04:26:33.490271+0000","last_deep_scrub":"75'2","last_deep_scrub_stamp":"2023-11-25T04:26:33.490271+0000","last_clean_scrub_stamp":"2023-11-25T04:26:33.490271+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:29:08.502565+0000","scrub_duration":0.0030545450000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"54'147","reported_seq":212,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986571+0000","last_change":"2023-11-25T04:25:54.939274+0000","last_active":"2023-11-25T04:26:00.986571+0000","last_peered":"2023-11-25T04:26:00.986571+0000","last_clean":"2023-11-25T04:26:00.986571+0000","last_became_active":"2023-11-25T04:25:54.938236+0000","last_became_peered":"2023-11-25T04:25:54.938236+0000","last_unstale":"2023-11-25T04:26:00.986571+0000","last_undegraded":"2023-11-25T04:26:00.986571+0000","last_fullsized":"2023-11-25T04:26:00.986571+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":147,"log_dups_size":0,"ondisk_log_size":147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":87524,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":155,"num_write_kb":235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":87524,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":43,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:41.504866+0000","last_change":"2023-11-25T04:26:41.504892+0000","last_active":"2023-11-25T04:26:41.504866+0000","last_peered":"2023-11-25T04:26:41.504866+0000","last_clean":"2023-11-25T04:26:41.504866+0000","last_became_active":"2023-11-25T04:25:46.995712+0000","last_became_peered":"2023-11-25T04:25:46.995712+0000","last_unstale":"2023-11-25T04:26:41.504866+0000","last_undegraded":"2023-11-25T04:26:41.504866+0000","last_fullsized":"2023-11-25T04:26:41.504866+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:26:41.504775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:26:41.504775+0000","last_clean_scrub_stamp":"2023-11-25T04:26:41.504775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:35:58.006258+0000","scrub_duration":0.001777907,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"54'155","reported_seq":442,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986276+0000","last_change":"2023-11-25T04:25:52.923096+0000","last_active":"2023-11-25T04:26:00.986276+0000","last_peered":"2023-11-25T04:26:00.986276+0000","last_clean":"2023-11-25T04:26:00.986276+0000","last_became_active":"2023-11-25T04:25:52.922549+0000","last_became_peered":"2023-11-25T04:25:52.922549+0000","last_unstale":"2023-11-25T04:26:00.986276+0000","last_undegraded":"2023-11-25T04:26:00.986276+0000","last_fullsized":"2023-11-25T04:26:00.986276+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":266919,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":264258,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"45'11","reported_seq":97,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986446+0000","last_change":"2023-11-25T04:25:47.092497+0000","last_active":"2023-11-25T04:26:00.986446+0000","last_peered":"2023-11-25T04:26:00.986446+0000","last_clean":"2023-11-25T04:26:00.986446+0000","last_became_active":"2023-11-25T04:25:47.000687+0000","last_became_peered":"2023-11-25T04:25:47.000687+0000","last_unstale":"2023-11-25T04:26:00.986446+0000","last_undegraded":"2023-11-25T04:26:00.986446+0000","last_fullsized":"2023-11-25T04:26:00.986446+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":4101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4194304,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"54'173","reported_seq":242,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986729+0000","last_change":"2023-11-25T04:25:50.923240+0000","last_active":"2023-11-25T04:26:00.986729+0000","last_peered":"2023-11-25T04:26:00.986729+0000","last_clean":"2023-11-25T04:26:00.986729+0000","last_became_active":"2023-11-25T04:25:50.922409+0000","last_became_peered":"2023-11-25T04:25:50.922409+0000","last_unstale":"2023-11-25T04:26:00.986729+0000","last_undegraded":"2023-11-25T04:26:00.986729+0000","last_fullsized":"2023-11-25T04:26:00.986729+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":553893,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":169,"num_write_kb":770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":553893,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":35,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986132+0000","last_change":"2023-11-25T04:25:46.998818+0000","last_active":"2023-11-25T04:26:00.986132+0000","last_peered":"2023-11-25T04:26:00.986132+0000","last_clean":"2023-11-25T04:26:00.986132+0000","last_became_active":"2023-11-25T04:25:46.997019+0000","last_became_peered":"2023-11-25T04:25:46.997019+0000","last_unstale":"2023-11-25T04:26:00.986132+0000","last_undegraded":"2023-11-25T04:26:00.986132+0000","last_fullsized":"2023-11-25T04:26:00.986132+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"54'163","reported_seq":229,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986523+0000","last_change":"2023-11-25T04:25:53.934183+0000","last_active":"2023-11-25T04:26:00.986523+0000","last_peered":"2023-11-25T04:26:00.986523+0000","last_clean":"2023-11-25T04:26:00.986523+0000","last_became_active":"2023-11-25T04:25:53.933471+0000","last_became_peered":"2023-11-25T04:25:53.933471+0000","last_unstale":"2023-11-25T04:26:00.986523+0000","last_undegraded":"2023-11-25T04:26:00.986523+0000","last_fullsized":"2023-11-25T04:26:00.986523+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":320867,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":190,"num_write_kb":577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":108,"num_bytes_recovered":311640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":89,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:26.554438+0000","last_change":"2023-11-25T04:26:26.554458+0000","last_active":"2023-11-25T04:26:26.554438+0000","last_peered":"2023-11-25T04:26:26.554438+0000","last_clean":"2023-11-25T04:26:26.554438+0000","last_became_active":"2023-11-25T04:25:46.994101+0000","last_became_peered":"2023-11-25T04:25:46.994101+0000","last_unstale":"2023-11-25T04:26:26.554438+0000","last_undegraded":"2023-11-25T04:26:26.554438+0000","last_fullsized":"2023-11-25T04:26:26.554438+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:26:26.554355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:26:26.554355+0000","last_clean_scrub_stamp":"2023-11-25T04:26:26.554355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:20:20.095629+0000","scrub_duration":0.0016349400000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"75'187","reported_seq":266,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:35.557378+0000","last_change":"2023-11-25T04:26:35.557414+0000","last_active":"2023-11-25T04:26:35.557378+0000","last_peered":"2023-11-25T04:26:35.557378+0000","last_clean":"2023-11-25T04:26:35.557378+0000","last_became_active":"2023-11-25T04:25:49.920971+0000","last_became_peered":"2023-11-25T04:25:49.920971+0000","last_unstale":"2023-11-25T04:26:35.557378+0000","last_undegraded":"2023-11-25T04:26:35.557378+0000","last_fullsized":"2023-11-25T04:26:35.557378+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'187","last_scrub_stamp":"2023-11-25T04:26:35.557305+0000","last_deep_scrub":"75'187","last_deep_scrub_stamp":"2023-11-25T04:26:35.557305+0000","last_clean_scrub_stamp":"2023-11-25T04:26:35.557305+0000","objects_scrubbed":46,"log_size":187,"log_dups_size":0,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:15:17.199719+0000","scrub_duration":0.039856336999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3502371,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":3718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":89,"num_bytes_recovered":3497669,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"49'1","reported_seq":71,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.985742+0000","last_change":"2023-11-25T04:22:46.034509+0000","last_active":"2023-11-25T04:26:00.985742+0000","last_peered":"2023-11-25T04:26:00.985742+0000","last_clean":"2023-11-25T04:26:00.985742+0000","last_became_active":"2023-11-25T04:22:46.034158+0000","last_became_peered":"2023-11-25T04:22:46.034158+0000","last_unstale":"2023-11-25T04:26:00.985742+0000","last_undegraded":"2023-11-25T04:26:00.985742+0000","last_fullsized":"2023-11-25T04:26:00.985742+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"75'194","reported_seq":273,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:15.719582+0000","last_change":"2023-11-25T04:26:15.719633+0000","last_active":"2023-11-25T04:26:15.719582+0000","last_peered":"2023-11-25T04:26:15.719582+0000","last_clean":"2023-11-25T04:26:15.719582+0000","last_became_active":"2023-11-25T04:22:47.038081+0000","last_became_peered":"2023-11-25T04:22:47.038081+0000","last_unstale":"2023-11-25T04:26:15.719582+0000","last_undegraded":"2023-11-25T04:26:15.719582+0000","last_fullsized":"2023-11-25T04:26:15.719582+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'194","last_scrub_stamp":"2023-11-25T04:26:15.719490+0000","last_deep_scrub":"75'194","last_deep_scrub_stamp":"2023-11-25T04:26:15.719490+0000","last_clean_scrub_stamp":"2023-11-25T04:26:15.719490+0000","objects_scrubbed":53,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:43:29.224433+0000","scrub_duration":0.037765356,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":174088,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":146,"num_write_kb":385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"12'157","reported_seq":233,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.985653+0000","last_change":"2023-11-25T04:25:47.000504+0000","last_active":"2023-11-25T04:26:00.985653+0000","last_peered":"2023-11-25T04:26:00.985653+0000","last_clean":"2023-11-25T04:26:00.985653+0000","last_became_active":"2023-11-25T04:25:46.999721+0000","last_became_peered":"2023-11-25T04:25:46.999721+0000","last_unstale":"2023-11-25T04:26:00.985653+0000","last_undegraded":"2023-11-25T04:26:00.985653+0000","last_fullsized":"2023-11-25T04:26:00.985653+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:35.919850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:35.919850+0000","last_clean_scrub_stamp":"2023-11-25T04:22:35.919850+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":35,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.985910+0000","last_change":"2023-11-25T04:25:46.998960+0000","last_active":"2023-11-25T04:26:00.985910+0000","last_peered":"2023-11-25T04:26:00.985910+0000","last_clean":"2023-11-25T04:26:00.985910+0000","last_became_active":"2023-11-25T04:25:46.997693+0000","last_became_peered":"2023-11-25T04:25:46.997693+0000","last_unstale":"2023-11-25T04:26:00.985910+0000","last_undegraded":"2023-11-25T04:26:00.985910+0000","last_fullsized":"2023-11-25T04:26:00.985910+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"46'1","reported_seq":71,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986287+0000","last_change":"2023-11-25T04:22:46.037044+0000","last_active":"2023-11-25T04:26:00.986287+0000","last_peered":"2023-11-25T04:26:00.986287+0000","last_clean":"2023-11-25T04:26:00.986287+0000","last_became_active":"2023-11-25T04:22:46.035916+0000","last_became_peered":"2023-11-25T04:22:46.035916+0000","last_unstale":"2023-11-25T04:26:00.986287+0000","last_undegraded":"2023-11-25T04:26:00.986287+0000","last_fullsized":"2023-11-25T04:26:00.986287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"54'146","reported_seq":214,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986701+0000","last_change":"2023-11-25T04:25:51.923452+0000","last_active":"2023-11-25T04:26:00.986701+0000","last_peered":"2023-11-25T04:26:00.986701+0000","last_clean":"2023-11-25T04:26:00.986701+0000","last_became_active":"2023-11-25T04:25:51.922490+0000","last_became_peered":"2023-11-25T04:25:51.922490+0000","last_unstale":"2023-11-25T04:26:00.986701+0000","last_undegraded":"2023-11-25T04:26:00.986701+0000","last_fullsized":"2023-11-25T04:26:00.986701+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":502154,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":170,"num_write_kb":687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":99,"num_bytes_recovered":502101,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":43,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:30.509601+0000","last_change":"2023-11-25T04:26:30.509621+0000","last_active":"2023-11-25T04:26:30.509601+0000","last_peered":"2023-11-25T04:26:30.509601+0000","last_clean":"2023-11-25T04:26:30.509601+0000","last_became_active":"2023-11-25T04:25:47.000033+0000","last_became_peered":"2023-11-25T04:25:47.000033+0000","last_unstale":"2023-11-25T04:26:30.509601+0000","last_undegraded":"2023-11-25T04:26:30.509601+0000","last_fullsized":"2023-11-25T04:26:30.509601+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:26:30.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:26:30.509522+0000","last_clean_scrub_stamp":"2023-11-25T04:26:30.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:17:09.583541+0000","scrub_duration":0.001771145,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"75'38","reported_seq":166,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:44.819110+0000","last_change":"2023-11-25T04:26:44.819150+0000","last_active":"2023-11-25T04:26:44.819110+0000","last_peered":"2023-11-25T04:26:44.819110+0000","last_clean":"2023-11-25T04:26:44.819110+0000","last_became_active":"2023-11-25T04:25:46.995637+0000","last_became_peered":"2023-11-25T04:25:46.995637+0000","last_unstale":"2023-11-25T04:26:44.819110+0000","last_undegraded":"2023-11-25T04:26:44.819110+0000","last_fullsized":"2023-11-25T04:26:44.819110+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'38","last_scrub_stamp":"2023-11-25T04:26:44.819010+0000","last_deep_scrub":"75'38","last_deep_scrub_stamp":"2023-11-25T04:26:44.819010+0000","last_clean_scrub_stamp":"2023-11-25T04:26:44.819010+0000","objects_scrubbed":1,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:17:29.276433+0000","scrub_duration":0.003812251,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":319160,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":35,"num_read_kb":334,"num_write":35,"num_write_kb":334,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":319160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"75'210","reported_seq":289,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:38.568563+0000","last_change":"2023-11-25T04:26:38.568589+0000","last_active":"2023-11-25T04:26:38.568563+0000","last_peered":"2023-11-25T04:26:38.568563+0000","last_clean":"2023-11-25T04:26:38.568563+0000","last_became_active":"2023-11-25T04:22:47.039197+0000","last_became_peered":"2023-11-25T04:22:47.039197+0000","last_unstale":"2023-11-25T04:26:38.568563+0000","last_undegraded":"2023-11-25T04:26:38.568563+0000","last_fullsized":"2023-11-25T04:26:38.568563+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'210","last_scrub_stamp":"2023-11-25T04:26:38.568488+0000","last_deep_scrub":"75'210","last_deep_scrub_stamp":"2023-11-25T04:26:38.568488+0000","last_clean_scrub_stamp":"2023-11-25T04:26:38.568488+0000","objects_scrubbed":52,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:17:23.025912+0000","scrub_duration":0.034345986000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":169303,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":189,"num_write_kb":333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":73,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986177+0000","last_change":"2023-11-25T04:22:42.000382+0000","last_active":"2023-11-25T04:26:00.986177+0000","last_peered":"2023-11-25T04:26:00.986177+0000","last_clean":"2023-11-25T04:26:00.986177+0000","last_became_active":"2023-11-25T04:22:42.000041+0000","last_became_peered":"2023-11-25T04:22:42.000041+0000","last_unstale":"2023-11-25T04:26:00.986177+0000","last_undegraded":"2023-11-25T04:26:00.986177+0000","last_fullsized":"2023-11-25T04:26:00.986177+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"75'72","reported_seq":166,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:15.436284+0000","last_change":"2023-11-25T04:26:15.436284+0000","last_active":"2023-11-25T04:26:15.436284+0000","last_peered":"2023-11-25T04:26:15.436284+0000","last_clean":"2023-11-25T04:26:15.436284+0000","last_became_active":"2023-11-25T04:25:47.001162+0000","last_became_peered":"2023-11-25T04:25:47.001162+0000","last_unstale":"2023-11-25T04:26:15.436284+0000","last_undegraded":"2023-11-25T04:26:15.436284+0000","last_fullsized":"2023-11-25T04:26:15.436284+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'72","last_scrub_stamp":"2023-11-25T04:26:15.436148+0000","last_deep_scrub":"75'72","last_deep_scrub_stamp":"2023-11-25T04:26:15.436148+0000","last_clean_scrub_stamp":"2023-11-25T04:26:15.436148+0000","objects_scrubbed":1,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:37:38.643270+0000","scrub_duration":0.01087811,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":69,"num_write_kb":4130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4194304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"75'220","reported_seq":299,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:14.717988+0000","last_change":"2023-11-25T04:26:14.718055+0000","last_active":"2023-11-25T04:26:14.717988+0000","last_peered":"2023-11-25T04:26:14.717988+0000","last_clean":"2023-11-25T04:26:14.717988+0000","last_became_active":"2023-11-25T04:22:47.038046+0000","last_became_peered":"2023-11-25T04:22:47.038046+0000","last_unstale":"2023-11-25T04:26:14.717988+0000","last_undegraded":"2023-11-25T04:26:14.717988+0000","last_fullsized":"2023-11-25T04:26:14.717988+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'220","last_scrub_stamp":"2023-11-25T04:26:14.717897+0000","last_deep_scrub":"75'220","last_deep_scrub_stamp":"2023-11-25T04:26:14.717897+0000","last_clean_scrub_stamp":"2023-11-25T04:26:14.717897+0000","objects_scrubbed":62,"log_size":220,"log_dups_size":0,"ondisk_log_size":220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:43:17.788148+0000","scrub_duration":0.044136959000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":115501,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":163,"num_write_kb":243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":35,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.985805+0000","last_change":"2023-11-25T04:25:46.998739+0000","last_active":"2023-11-25T04:26:00.985805+0000","last_peered":"2023-11-25T04:26:00.985805+0000","last_clean":"2023-11-25T04:26:00.985805+0000","last_became_active":"2023-11-25T04:25:46.997391+0000","last_became_peered":"2023-11-25T04:25:46.997391+0000","last_unstale":"2023-11-25T04:26:00.985805+0000","last_undegraded":"2023-11-25T04:26:00.985805+0000","last_fullsized":"2023-11-25T04:26:00.985805+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"50'1","reported_seq":35,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986046+0000","last_change":"2023-11-25T04:25:46.999446+0000","last_active":"2023-11-25T04:26:00.986046+0000","last_peered":"2023-11-25T04:26:00.986046+0000","last_clean":"2023-11-25T04:26:00.986046+0000","last_became_active":"2023-11-25T04:25:46.998132+0000","last_became_peered":"2023-11-25T04:25:46.998132+0000","last_unstale":"2023-11-25T04:26:00.986046+0000","last_undegraded":"2023-11-25T04:26:00.986046+0000","last_fullsized":"2023-11-25T04:26:00.986046+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"75'206","reported_seq":284,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:19.694413+0000","last_change":"2023-11-25T04:26:19.694459+0000","last_active":"2023-11-25T04:26:19.694413+0000","last_peered":"2023-11-25T04:26:19.694413+0000","last_clean":"2023-11-25T04:26:19.694413+0000","last_became_active":"2023-11-25T04:22:47.036924+0000","last_became_peered":"2023-11-25T04:22:47.036924+0000","last_unstale":"2023-11-25T04:26:19.694413+0000","last_undegraded":"2023-11-25T04:26:19.694413+0000","last_fullsized":"2023-11-25T04:26:19.694413+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'206","last_scrub_stamp":"2023-11-25T04:26:19.694341+0000","last_deep_scrub":"75'206","last_deep_scrub_stamp":"2023-11-25T04:26:19.694341+0000","last_clean_scrub_stamp":"2023-11-25T04:26:19.694341+0000","objects_scrubbed":49,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:49:38.202017+0000","scrub_duration":0.031428719000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":93321,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":148,"num_write_kb":360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":35,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986393+0000","last_change":"2023-11-25T04:25:46.999738+0000","last_active":"2023-11-25T04:26:00.986393+0000","last_peered":"2023-11-25T04:26:00.986393+0000","last_clean":"2023-11-25T04:26:00.986393+0000","last_became_active":"2023-11-25T04:25:46.999403+0000","last_became_peered":"2023-11-25T04:25:46.999403+0000","last_unstale":"2023-11-25T04:26:00.986393+0000","last_undegraded":"2023-11-25T04:26:00.986393+0000","last_fullsized":"2023-11-25T04:26:00.986393+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":73,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986717+0000","last_change":"2023-11-25T04:22:42.000204+0000","last_active":"2023-11-25T04:26:00.986717+0000","last_peered":"2023-11-25T04:26:00.986717+0000","last_clean":"2023-11-25T04:26:00.986717+0000","last_became_active":"2023-11-25T04:22:41.999915+0000","last_became_peered":"2023-11-25T04:22:41.999915+0000","last_unstale":"2023-11-25T04:26:00.986717+0000","last_undegraded":"2023-11-25T04:26:00.986717+0000","last_fullsized":"2023-11-25T04:26:00.986717+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"54'152","reported_seq":433,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.985964+0000","last_change":"2023-11-25T04:25:53.933239+0000","last_active":"2023-11-25T04:26:00.985964+0000","last_peered":"2023-11-25T04:26:00.985964+0000","last_clean":"2023-11-25T04:26:00.985964+0000","last_became_active":"2023-11-25T04:25:53.932586+0000","last_became_peered":"2023-11-25T04:25:53.932586+0000","last_unstale":"2023-11-25T04:26:00.985964+0000","last_undegraded":"2023-11-25T04:26:00.985964+0000","last_fullsized":"2023-11-25T04:26:00.985964+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":329327,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":151,"num_write_kb":471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":97,"num_bytes_recovered":329278,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":70,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986629+0000","last_change":"2023-11-25T04:22:46.035885+0000","last_active":"2023-11-25T04:26:00.986629+0000","last_peered":"2023-11-25T04:26:00.986629+0000","last_clean":"2023-11-25T04:26:00.986629+0000","last_became_active":"2023-11-25T04:22:46.035405+0000","last_became_peered":"2023-11-25T04:22:46.035405+0000","last_unstale":"2023-11-25T04:26:00.986629+0000","last_undegraded":"2023-11-25T04:26:00.986629+0000","last_fullsized":"2023-11-25T04:26:00.986629+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":73,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986810+0000","last_change":"2023-11-25T04:22:41.999067+0000","last_active":"2023-11-25T04:26:00.986810+0000","last_peered":"2023-11-25T04:26:00.986810+0000","last_clean":"2023-11-25T04:26:00.986810+0000","last_became_active":"2023-11-25T04:22:41.998798+0000","last_became_peered":"2023-11-25T04:22:41.998798+0000","last_unstale":"2023-11-25T04:26:00.986810+0000","last_undegraded":"2023-11-25T04:26:00.986810+0000","last_fullsized":"2023-11-25T04:26:00.986810+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"75'221","reported_seq":529,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:13.479628+0000","last_change":"2023-11-25T04:26:13.479673+0000","last_active":"2023-11-25T04:26:13.479628+0000","last_peered":"2023-11-25T04:26:13.479628+0000","last_clean":"2023-11-25T04:26:13.479628+0000","last_became_active":"2023-11-25T04:25:51.921858+0000","last_became_peered":"2023-11-25T04:25:51.921858+0000","last_unstale":"2023-11-25T04:26:13.479628+0000","last_undegraded":"2023-11-25T04:26:13.479628+0000","last_fullsized":"2023-11-25T04:26:13.479628+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'221","last_scrub_stamp":"2023-11-25T04:26:13.479549+0000","last_deep_scrub":"75'221","last_deep_scrub_stamp":"2023-11-25T04:26:13.479549+0000","last_clean_scrub_stamp":"2023-11-25T04:26:13.479549+0000","objects_scrubbed":49,"log_size":221,"log_dups_size":0,"ondisk_log_size":221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:44:56.142187+0000","scrub_duration":0.035520731999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":129965,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":166,"num_write_kb":290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":129965,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"18'2","reported_seq":72,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.987053+0000","last_change":"2023-11-25T04:22:46.034606+0000","last_active":"2023-11-25T04:26:00.987053+0000","last_peered":"2023-11-25T04:26:00.987053+0000","last_clean":"2023-11-25T04:26:00.987053+0000","last_became_active":"2023-11-25T04:22:46.034229+0000","last_became_peered":"2023-11-25T04:22:46.034229+0000","last_unstale":"2023-11-25T04:26:00.987053+0000","last_undegraded":"2023-11-25T04:26:00.987053+0000","last_fullsized":"2023-11-25T04:26:00.987053+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":73,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986933+0000","last_change":"2023-11-25T04:22:42.000258+0000","last_active":"2023-11-25T04:26:00.986933+0000","last_peered":"2023-11-25T04:26:00.986933+0000","last_clean":"2023-11-25T04:26:00.986933+0000","last_became_active":"2023-11-25T04:22:41.999992+0000","last_became_peered":"2023-11-25T04:22:41.999992+0000","last_unstale":"2023-11-25T04:26:00.986933+0000","last_undegraded":"2023-11-25T04:26:00.986933+0000","last_fullsized":"2023-11-25T04:26:00.986933+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"54'150","reported_seq":438,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:09.671351+0000","last_change":"2023-11-25T04:25:54.938479+0000","last_active":"2023-11-25T04:26:09.671351+0000","last_peered":"2023-11-25T04:26:09.671351+0000","last_clean":"2023-11-25T04:26:09.671351+0000","last_became_active":"2023-11-25T04:25:54.937838+0000","last_became_peered":"2023-11-25T04:25:54.937838+0000","last_unstale":"2023-11-25T04:26:09.671351+0000","last_undegraded":"2023-11-25T04:26:09.671351+0000","last_fullsized":"2023-11-25T04:26:09.671351+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":159585,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":154224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"54'14","reported_seq":100,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986916+0000","last_change":"2023-11-25T04:25:47.026749+0000","last_active":"2023-11-25T04:26:00.986916+0000","last_peered":"2023-11-25T04:26:00.986916+0000","last_clean":"2023-11-25T04:26:00.986916+0000","last_became_active":"2023-11-25T04:25:46.997404+0000","last_became_peered":"2023-11-25T04:25:46.997404+0000","last_unstale":"2023-11-25T04:26:00.986916+0000","last_undegraded":"2023-11-25T04:26:00.986916+0000","last_fullsized":"2023-11-25T04:26:00.986916+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":35,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986659+0000","last_change":"2023-11-25T04:25:46.995200+0000","last_active":"2023-11-25T04:26:00.986659+0000","last_peered":"2023-11-25T04:26:00.986659+0000","last_clean":"2023-11-25T04:26:00.986659+0000","last_became_active":"2023-11-25T04:25:46.994834+0000","last_became_peered":"2023-11-25T04:25:46.994834+0000","last_unstale":"2023-11-25T04:26:00.986659+0000","last_undegraded":"2023-11-25T04:26:00.986659+0000","last_fullsized":"2023-11-25T04:26:00.986659+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"75'191","reported_seq":474,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:14.466588+0000","last_change":"2023-11-25T04:26:14.466619+0000","last_active":"2023-11-25T04:26:14.466588+0000","last_peered":"2023-11-25T04:26:14.466588+0000","last_clean":"2023-11-25T04:26:14.466588+0000","last_became_active":"2023-11-25T04:25:49.921465+0000","last_became_peered":"2023-11-25T04:25:49.921465+0000","last_unstale":"2023-11-25T04:26:14.466588+0000","last_undegraded":"2023-11-25T04:26:14.466588+0000","last_fullsized":"2023-11-25T04:26:14.466588+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'191","last_scrub_stamp":"2023-11-25T04:26:14.466515+0000","last_deep_scrub":"75'191","last_deep_scrub_stamp":"2023-11-25T04:26:14.466515+0000","last_clean_scrub_stamp":"2023-11-25T04:26:14.466515+0000","objects_scrubbed":56,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:04:34.752431+0000","scrub_duration":0.037056879000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":99503,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":148,"num_write_kb":250,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":93,"num_bytes_recovered":99503,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"54'166","reported_seq":234,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986367+0000","last_change":"2023-11-25T04:25:51.923141+0000","last_active":"2023-11-25T04:26:00.986367+0000","last_peered":"2023-11-25T04:26:00.986367+0000","last_clean":"2023-11-25T04:26:00.986367+0000","last_became_active":"2023-11-25T04:25:51.922070+0000","last_became_peered":"2023-11-25T04:25:51.922070+0000","last_unstale":"2023-11-25T04:26:00.986367+0000","last_undegraded":"2023-11-25T04:26:00.986367+0000","last_fullsized":"2023-11-25T04:26:00.986367+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":185321,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":172,"num_write_kb":312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":164176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"58'50","reported_seq":54,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986198+0000","last_change":"2023-11-25T04:25:47.021705+0000","last_active":"2023-11-25T04:26:00.986198+0000","last_peered":"2023-11-25T04:26:00.986198+0000","last_clean":"2023-11-25T04:26:00.986198+0000","last_became_active":"2023-11-25T04:25:46.994997+0000","last_became_peered":"2023-11-25T04:25:46.994997+0000","last_unstale":"2023-11-25T04:26:00.986198+0000","last_undegraded":"2023-11-25T04:26:00.986198+0000","last_fullsized":"2023-11-25T04:26:00.986198+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":4119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4190183,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"75'199","reported_seq":277,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:43.858563+0000","last_change":"2023-11-25T04:26:43.858609+0000","last_active":"2023-11-25T04:26:43.858563+0000","last_peered":"2023-11-25T04:26:43.858563+0000","last_clean":"2023-11-25T04:26:43.858563+0000","last_became_active":"2023-11-25T04:22:47.036821+0000","last_became_peered":"2023-11-25T04:22:47.036821+0000","last_unstale":"2023-11-25T04:26:43.858563+0000","last_undegraded":"2023-11-25T04:26:43.858563+0000","last_fullsized":"2023-11-25T04:26:43.858563+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'199","last_scrub_stamp":"2023-11-25T04:26:43.858473+0000","last_deep_scrub":"75'199","last_deep_scrub_stamp":"2023-11-25T04:26:43.858473+0000","last_clean_scrub_stamp":"2023-11-25T04:26:43.858473+0000","objects_scrubbed":46,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:28:32.758038+0000","scrub_duration":0.029966083000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":101117,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":161,"num_write_kb":242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"75'24","reported_seq":118,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:39.535607+0000","last_change":"2023-11-25T04:26:39.535658+0000","last_active":"2023-11-25T04:26:39.535607+0000","last_peered":"2023-11-25T04:26:39.535607+0000","last_clean":"2023-11-25T04:26:39.535607+0000","last_became_active":"2023-11-25T04:25:47.000478+0000","last_became_peered":"2023-11-25T04:25:47.000478+0000","last_unstale":"2023-11-25T04:26:39.535607+0000","last_undegraded":"2023-11-25T04:26:39.535607+0000","last_fullsized":"2023-11-25T04:26:39.535607+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'24","last_scrub_stamp":"2023-11-25T04:26:39.535504+0000","last_deep_scrub":"75'24","last_deep_scrub_stamp":"2023-11-25T04:26:39.535504+0000","last_clean_scrub_stamp":"2023-11-25T04:26:39.535504+0000","objects_scrubbed":2,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:50:41.041415+0000","scrub_duration":0.0040188020000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":90,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":35,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986686+0000","last_change":"2023-11-25T04:25:46.996182+0000","last_active":"2023-11-25T04:26:00.986686+0000","last_peered":"2023-11-25T04:26:00.986686+0000","last_clean":"2023-11-25T04:26:00.986686+0000","last_became_active":"2023-11-25T04:25:46.994525+0000","last_became_peered":"2023-11-25T04:25:46.994525+0000","last_unstale":"2023-11-25T04:26:00.986686+0000","last_undegraded":"2023-11-25T04:26:00.986686+0000","last_fullsized":"2023-11-25T04:26:00.986686+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"54'177","reported_seq":486,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.985858+0000","last_change":"2023-11-25T04:25:51.922466+0000","last_active":"2023-11-25T04:26:00.985858+0000","last_peered":"2023-11-25T04:26:00.985858+0000","last_clean":"2023-11-25T04:26:00.985858+0000","last_became_active":"2023-11-25T04:25:51.921938+0000","last_became_peered":"2023-11-25T04:25:51.921938+0000","last_unstale":"2023-11-25T04:26:00.985858+0000","last_undegraded":"2023-11-25T04:26:00.985858+0000","last_fullsized":"2023-11-25T04:26:00.985858+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":522494,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":168,"num_write_kb":749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":111,"num_bytes_recovered":522494,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"46'2","reported_seq":83,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986974+0000","last_change":"2023-11-25T04:25:46.998867+0000","last_active":"2023-11-25T04:26:00.986974+0000","last_peered":"2023-11-25T04:26:00.986974+0000","last_clean":"2023-11-25T04:26:00.986974+0000","last_became_active":"2023-11-25T04:25:46.998508+0000","last_became_peered":"2023-11-25T04:25:46.998508+0000","last_unstale":"2023-11-25T04:26:00.986974+0000","last_undegraded":"2023-11-25T04:26:00.986974+0000","last_fullsized":"2023-11-25T04:26:00.986974+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.10","version":"54'153","reported_seq":222,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.985832+0000","last_change":"2023-11-25T04:22:47.037790+0000","last_active":"2023-11-25T04:26:00.985832+0000","last_peered":"2023-11-25T04:26:00.985832+0000","last_clean":"2023-11-25T04:26:00.985832+0000","last_became_active":"2023-11-25T04:22:47.037149+0000","last_became_peered":"2023-11-25T04:22:47.037149+0000","last_unstale":"2023-11-25T04:26:00.985832+0000","last_undegraded":"2023-11-25T04:26:00.985832+0000","last_fullsized":"2023-11-25T04:26:00.985832+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":218073,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1f","version":"54'156","reported_seq":449,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986814+0000","last_change":"2023-11-25T04:25:49.921904+0000","last_active":"2023-11-25T04:26:00.986814+0000","last_peered":"2023-11-25T04:26:00.986814+0000","last_clean":"2023-11-25T04:26:00.986814+0000","last_became_active":"2023-11-25T04:25:49.921458+0000","last_became_peered":"2023-11-25T04:25:49.921458+0000","last_unstale":"2023-11-25T04:26:00.986814+0000","last_undegraded":"2023-11-25T04:26:00.986814+0000","last_fullsized":"2023-11-25T04:26:00.986814+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":107712,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":103,"num_bytes_recovered":107703,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"48'1","reported_seq":71,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986022+0000","last_change":"2023-11-25T04:22:46.036085+0000","last_active":"2023-11-25T04:26:00.986022+0000","last_peered":"2023-11-25T04:26:00.986022+0000","last_clean":"2023-11-25T04:26:00.986022+0000","last_became_active":"2023-11-25T04:22:46.035495+0000","last_became_peered":"2023-11-25T04:22:46.035495+0000","last_unstale":"2023-11-25T04:26:00.986022+0000","last_undegraded":"2023-11-25T04:26:00.986022+0000","last_fullsized":"2023-11-25T04:26:00.986022+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"54'152","reported_seq":432,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:09.671534+0000","last_change":"2023-11-25T04:25:50.923882+0000","last_active":"2023-11-25T04:26:09.671534+0000","last_peered":"2023-11-25T04:26:09.671534+0000","last_clean":"2023-11-25T04:26:09.671534+0000","last_became_active":"2023-11-25T04:25:50.923375+0000","last_became_peered":"2023-11-25T04:25:50.923375+0000","last_unstale":"2023-11-25T04:26:09.671534+0000","last_undegraded":"2023-11-25T04:26:09.671534+0000","last_fullsized":"2023-11-25T04:26:09.671534+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":76435,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":76435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":89,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:23.683261+0000","last_change":"2023-11-25T04:26:23.683288+0000","last_active":"2023-11-25T04:26:23.683261+0000","last_peered":"2023-11-25T04:26:23.683261+0000","last_clean":"2023-11-25T04:26:23.683261+0000","last_became_active":"2023-11-25T04:25:46.994871+0000","last_became_peered":"2023-11-25T04:25:46.994871+0000","last_unstale":"2023-11-25T04:26:23.683261+0000","last_undegraded":"2023-11-25T04:26:23.683261+0000","last_fullsized":"2023-11-25T04:26:23.683261+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:26:23.683176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:26:23.683176+0000","last_clean_scrub_stamp":"2023-11-25T04:26:23.683176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:31:54.450606+0000","scrub_duration":0.00162587,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"54'158","reported_seq":433,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986512+0000","last_change":"2023-11-25T04:25:52.925523+0000","last_active":"2023-11-25T04:26:00.986512+0000","last_peered":"2023-11-25T04:26:00.986512+0000","last_clean":"2023-11-25T04:26:00.986512+0000","last_became_active":"2023-11-25T04:25:52.925038+0000","last_became_peered":"2023-11-25T04:25:52.925038+0000","last_unstale":"2023-11-25T04:26:00.986512+0000","last_undegraded":"2023-11-25T04:26:00.986512+0000","last_fullsized":"2023-11-25T04:26:00.986512+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":209500,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":139,"num_write_kb":351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":209500,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1a","version":"48'4","reported_seq":74,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986289+0000","last_change":"2023-11-25T04:22:46.034340+0000","last_active":"2023-11-25T04:26:00.986289+0000","last_peered":"2023-11-25T04:26:00.986289+0000","last_clean":"2023-11-25T04:26:00.986289+0000","last_became_active":"2023-11-25T04:22:46.033944+0000","last_became_peered":"2023-11-25T04:22:46.033944+0000","last_unstale":"2023-11-25T04:26:00.986289+0000","last_undegraded":"2023-11-25T04:26:00.986289+0000","last_fullsized":"2023-11-25T04:26:00.986289+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"75'218","reported_seq":297,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:28.540048+0000","last_change":"2023-11-25T04:26:28.540082+0000","last_active":"2023-11-25T04:26:28.540048+0000","last_peered":"2023-11-25T04:26:28.540048+0000","last_clean":"2023-11-25T04:26:28.540048+0000","last_became_active":"2023-11-25T04:22:47.039218+0000","last_became_peered":"2023-11-25T04:22:47.039218+0000","last_unstale":"2023-11-25T04:26:28.540048+0000","last_undegraded":"2023-11-25T04:26:28.540048+0000","last_fullsized":"2023-11-25T04:26:28.540048+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'218","last_scrub_stamp":"2023-11-25T04:26:28.539970+0000","last_deep_scrub":"75'218","last_deep_scrub_stamp":"2023-11-25T04:26:28.539970+0000","last_clean_scrub_stamp":"2023-11-25T04:26:28.539970+0000","objects_scrubbed":55,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:46:27.913704+0000","scrub_duration":0.040559401000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":165883,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"75'10","reported_seq":104,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:20.499200+0000","last_change":"2023-11-25T04:26:20.499236+0000","last_active":"2023-11-25T04:26:20.499200+0000","last_peered":"2023-11-25T04:26:20.499200+0000","last_clean":"2023-11-25T04:26:20.499200+0000","last_became_active":"2023-11-25T04:25:46.999998+0000","last_became_peered":"2023-11-25T04:25:46.999998+0000","last_unstale":"2023-11-25T04:26:20.499200+0000","last_undegraded":"2023-11-25T04:26:20.499200+0000","last_fullsized":"2023-11-25T04:26:20.499200+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'10","last_scrub_stamp":"2023-11-25T04:26:20.499093+0000","last_deep_scrub":"75'10","last_deep_scrub_stamp":"2023-11-25T04:26:20.499093+0000","last_clean_scrub_stamp":"2023-11-25T04:26:20.499093+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:35:51.569332+0000","scrub_duration":0.0094161829999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":4101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4194304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"75'212","reported_seq":291,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:39.477807+0000","last_change":"2023-11-25T04:26:39.477840+0000","last_active":"2023-11-25T04:26:39.477807+0000","last_peered":"2023-11-25T04:26:39.477807+0000","last_clean":"2023-11-25T04:26:39.477807+0000","last_became_active":"2023-11-25T04:25:50.922164+0000","last_became_peered":"2023-11-25T04:25:50.922164+0000","last_unstale":"2023-11-25T04:26:39.477807+0000","last_undegraded":"2023-11-25T04:26:39.477807+0000","last_fullsized":"2023-11-25T04:26:39.477807+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'212","last_scrub_stamp":"2023-11-25T04:26:39.477723+0000","last_deep_scrub":"75'212","last_deep_scrub_stamp":"2023-11-25T04:26:39.477723+0000","last_clean_scrub_stamp":"2023-11-25T04:26:39.477723+0000","objects_scrubbed":63,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:34:17.609339+0000","scrub_duration":0.044092971000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":179506,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":155,"num_write_kb":317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":179506,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"18'2","reported_seq":38,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986217+0000","last_change":"2023-11-25T04:25:46.996675+0000","last_active":"2023-11-25T04:26:00.986217+0000","last_peered":"2023-11-25T04:26:00.986217+0000","last_clean":"2023-11-25T04:26:00.986217+0000","last_became_active":"2023-11-25T04:25:46.995160+0000","last_became_peered":"2023-11-25T04:25:46.995160+0000","last_unstale":"2023-11-25T04:26:00.986217+0000","last_undegraded":"2023-11-25T04:26:00.986217+0000","last_fullsized":"2023-11-25T04:26:00.986217+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1a","version":"54'162","reported_seq":227,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986038+0000","last_change":"2023-11-25T04:25:54.939182+0000","last_active":"2023-11-25T04:26:00.986038+0000","last_peered":"2023-11-25T04:26:00.986038+0000","last_clean":"2023-11-25T04:26:00.986038+0000","last_became_active":"2023-11-25T04:25:54.937821+0000","last_became_peered":"2023-11-25T04:25:54.937821+0000","last_unstale":"2023-11-25T04:26:00.986038+0000","last_undegraded":"2023-11-25T04:26:00.986038+0000","last_fullsized":"2023-11-25T04:26:00.986038+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":134672,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":169,"num_write_kb":311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":126291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"75'62","reported_seq":158,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:30.554569+0000","last_change":"2023-11-25T04:26:30.554596+0000","last_active":"2023-11-25T04:26:30.554569+0000","last_peered":"2023-11-25T04:26:30.554569+0000","last_clean":"2023-11-25T04:26:30.554569+0000","last_became_active":"2023-11-25T04:25:47.000722+0000","last_became_peered":"2023-11-25T04:25:47.000722+0000","last_unstale":"2023-11-25T04:26:30.554569+0000","last_undegraded":"2023-11-25T04:26:30.554569+0000","last_fullsized":"2023-11-25T04:26:30.554569+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'62","last_scrub_stamp":"2023-11-25T04:26:30.554482+0000","last_deep_scrub":"75'62","last_deep_scrub_stamp":"2023-11-25T04:26:30.554482+0000","last_clean_scrub_stamp":"2023-11-25T04:26:30.554482+0000","objects_scrubbed":3,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:38:15.140403+0000","scrub_duration":0.015921703999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8388608,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"54'163","reported_seq":232,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986171+0000","last_change":"2023-11-25T04:22:47.036764+0000","last_active":"2023-11-25T04:26:00.986171+0000","last_peered":"2023-11-25T04:26:00.986171+0000","last_clean":"2023-11-25T04:26:00.986171+0000","last_became_active":"2023-11-25T04:22:47.036406+0000","last_became_peered":"2023-11-25T04:22:47.036406+0000","last_unstale":"2023-11-25T04:26:00.986171+0000","last_undegraded":"2023-11-25T04:26:00.986171+0000","last_fullsized":"2023-11-25T04:26:00.986171+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":527891,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":163,"num_write_kb":687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1e","version":"46'2","reported_seq":73,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:09.671303+0000","last_change":"2023-11-25T04:22:46.035659+0000","last_active":"2023-11-25T04:26:09.671303+0000","last_peered":"2023-11-25T04:26:09.671303+0000","last_clean":"2023-11-25T04:26:09.671303+0000","last_became_active":"2023-11-25T04:22:46.035113+0000","last_became_peered":"2023-11-25T04:22:46.035113+0000","last_unstale":"2023-11-25T04:26:09.671303+0000","last_undegraded":"2023-11-25T04:26:09.671303+0000","last_fullsized":"2023-11-25T04:26:09.671303+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.18","version":"54'195","reported_seq":260,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986052+0000","last_change":"2023-11-25T04:25:54.938573+0000","last_active":"2023-11-25T04:26:00.986052+0000","last_peered":"2023-11-25T04:26:00.986052+0000","last_clean":"2023-11-25T04:26:00.986052+0000","last_became_active":"2023-11-25T04:25:54.937444+0000","last_became_peered":"2023-11-25T04:25:54.937444+0000","last_unstale":"2023-11-25T04:26:00.986052+0000","last_undegraded":"2023-11-25T04:26:00.986052+0000","last_fullsized":"2023-11-25T04:26:00.986052+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":128641,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":185,"num_write_kb":408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":122,"num_bytes_recovered":128641,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"50'17","reported_seq":103,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986905+0000","last_change":"2023-11-25T04:25:47.044228+0000","last_active":"2023-11-25T04:26:00.986905+0000","last_peered":"2023-11-25T04:26:00.986905+0000","last_clean":"2023-11-25T04:26:00.986905+0000","last_became_active":"2023-11-25T04:25:46.998870+0000","last_became_peered":"2023-11-25T04:25:46.998870+0000","last_unstale":"2023-11-25T04:26:00.986905+0000","last_undegraded":"2023-11-25T04:26:00.986905+0000","last_fullsized":"2023-11-25T04:26:00.986905+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":4103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4194304,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":26330855,"num_objects":3447,"num_object_clones":0,"num_object_copies":6894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10423,"num_write_kb":38550,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4189,"num_bytes_recovered":12749181,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":72376320,"data_stored":52324046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11811,"ondisk_log_size":11811,"up":128,"acting":128,"num_store_stats":3},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":40885548,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":35,"num_read_kb":334,"num_write":435,"num_write_kb":40462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":32491439,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":81833984,"data_stored":81771096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5548,"internal_metadata":0},"log_size":449,"ondisk_log_size":449,"up":64,"acting":64,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738424,"num_pgs":72,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77364,"kb_used_data":46188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94294476,"statfs":{"total":96636764160,"available":96557543424,"internally_reserved":0,"allocated":47296512,"data_stored":39787071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.373}]},{"osd":1,"last update":"Sat Nov 25 04:26:45 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.436,"15min":0.44600000000000001},"min":{"1min":0.23300000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.61299999999999999,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.437,"15min":0.45100000000000001},"min":{"1min":0.249,"5min":0.184,"15min":0.184},"max":{"1min":0.63,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.46600000000000003}]}]},{"osd":1,"up_from":8,"seq":34359738424,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":91036,"kb_used_data":55700,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94280804,"statfs":{"total":96636764160,"available":96543543296,"internally_reserved":0,"allocated":57036800,"data_stored":50261554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 25 04:26:46 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.495},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.54100000000000004}]},{"osd":2,"last update":"Sat Nov 25 04:26:42 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.41999999999999998,"15min":0.41599999999999998},"min":{"1min":0.28299999999999997,"5min":0.182,"15min":0.182},"max":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.41899999999999998,"15min":0.41099999999999998},"min":{"1min":0.27000000000000002,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.58799999999999997}]}]},{"osd":0,"up_from":60,"seq":257698037774,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":91772,"kb_used_data":51764,"kb_used_omap":6,"kb_used_meta":39993,"kb_avail":94280068,"statfs":{"total":96636764160,"available":96542789632,"internally_reserved":0,"allocated":53006336,"data_stored":46141107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7143,"internal_metadata":40952857},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32530432,"data_stored":32496940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5548,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":29380608,"data_stored":29360982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":19922944,"data_stored":19913174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":19234816,"data_stored":12749181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":26415104,"data_stored":20005586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":26726400,"data_stored":19569279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-25T04:26:50.928 INFO:tasks.ceph:pgid 4.3c last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:50.929 INFO:tasks.ceph:pgid 4.3b last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:50.929 INFO:tasks.ceph:pgid 4.3a last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:50.929 INFO:tasks.ceph:pgid 4.34 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:50.929 INFO:tasks.ceph:pgid 4.32 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:50.930 INFO:tasks.ceph:pgid 4.2e last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:50.930 INFO:tasks.ceph:pgid 4.2a last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:50.930 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:50.930 INFO:tasks.ceph:pgid 4.27 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:50.930 INFO:tasks.ceph:pgid 4.26 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:50.931 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:50.931 INFO:tasks.ceph:pgid 4.22 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:50.931 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:50.931 INFO:tasks.ceph:pgid 4.20 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:50.931 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:50.931 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:50.932 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:50.932 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:50.932 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:50.932 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:50.933 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:50.933 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:50.933 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:50.933 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:50.933 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:50.933 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:50.934 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-11-25T04:22:35.919850+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=35, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:50.934 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-11-25T04:22:40.987592+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=40, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:50.934 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:50.934 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:50.934 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-11-25T04:22:40.987592+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=40, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:50.935 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-11-25T04:22:40.987592+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=40, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:50.935 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:50.935 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:50.935 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-11-25T04:22:40.987592+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=40, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:50.935 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:50.935 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:50.936 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-11-25T04:22:40.987592+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=40, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:50.936 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:50.936 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-11-25T04:22:40.987592+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=40, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:50.936 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:50.936 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:50.936 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-11-25T04:22:40.987592+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=40, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:50.937 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:50.937 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:50.937 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:50.937 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:50.937 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:50.937 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:50.938 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:50.938 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:50.938 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:50.938 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:50.938 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:50.939 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:50.939 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:50.939 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:50.939 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:50.939 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:50.940 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:26:50.940 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-11-25T04:27:10.941 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-25T04:27:11.256 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:27:11.258 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2023-11-25T04:27:11.268 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":211,"stamp":"2023-11-25T04:27:09.566972+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":67806771,"num_objects":3481,"num_object_clones":0,"num_object_copies":6962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3481,"num_whiteouts":0,"num_read":171,"num_read_kb":452,"num_write":11061,"num_write_kb":82472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4201,"num_bytes_recovered":45240620,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13101,"ondisk_log_size":13101,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":268492,"kb_used_data":153652,"kb_used_omap":10,"kb_used_meta":114805,"kb_avail":282847028,"statfs":{"total":289910292480,"available":289635356672,"internally_reserved":0,"allocated":157339648,"data_stored":136189732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10333,"internal_metadata":117561251},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003162"},"pg_stats":[{"pgid":"4.3f","version":"75'212","reported_seq":291,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:48.487625+0000","last_change":"2023-11-25T04:26:48.487625+0000","last_active":"2023-11-25T04:26:48.487625+0000","last_peered":"2023-11-25T04:26:48.487625+0000","last_clean":"2023-11-25T04:26:48.487625+0000","last_became_active":"2023-11-25T04:22:47.039490+0000","last_became_peered":"2023-11-25T04:22:47.039490+0000","last_unstale":"2023-11-25T04:26:48.487625+0000","last_undegraded":"2023-11-25T04:26:48.487625+0000","last_fullsized":"2023-11-25T04:26:48.487625+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'212","last_scrub_stamp":"2023-11-25T04:26:48.487485+0000","last_deep_scrub":"75'212","last_deep_scrub_stamp":"2023-11-25T04:26:48.487485+0000","last_clean_scrub_stamp":"2023-11-25T04:26:48.487485+0000","objects_scrubbed":46,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:02:55.160351+0000","scrub_duration":0.031121083000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":577866,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":174,"num_write_kb":756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3e","version":"75'257","reported_seq":336,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:16.435276+0000","last_change":"2023-11-25T04:26:16.435276+0000","last_active":"2023-11-25T04:26:16.435276+0000","last_peered":"2023-11-25T04:26:16.435276+0000","last_clean":"2023-11-25T04:26:16.435276+0000","last_became_active":"2023-11-25T04:22:47.040511+0000","last_became_peered":"2023-11-25T04:22:47.040511+0000","last_unstale":"2023-11-25T04:26:16.435276+0000","last_undegraded":"2023-11-25T04:26:16.435276+0000","last_fullsized":"2023-11-25T04:26:16.435276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'257","last_scrub_stamp":"2023-11-25T04:26:16.435161+0000","last_deep_scrub":"75'257","last_deep_scrub_stamp":"2023-11-25T04:26:16.435161+0000","last_clean_scrub_stamp":"2023-11-25T04:26:16.435161+0000","objects_scrubbed":45,"log_size":257,"log_dups_size":0,"ondisk_log_size":257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:13:07.641904+0000","scrub_duration":0.036488224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":461579,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":236,"num_write_kb":841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3d","version":"75'209","reported_seq":285,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:12.330753+0000","last_change":"2023-11-25T04:26:12.330801+0000","last_active":"2023-11-25T04:26:12.330753+0000","last_peered":"2023-11-25T04:26:12.330753+0000","last_clean":"2023-11-25T04:26:12.330753+0000","last_became_active":"2023-11-25T04:25:52.923459+0000","last_became_peered":"2023-11-25T04:25:52.923459+0000","last_unstale":"2023-11-25T04:26:12.330753+0000","last_undegraded":"2023-11-25T04:26:12.330753+0000","last_fullsized":"2023-11-25T04:26:12.330753+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'209","last_scrub_stamp":"2023-11-25T04:26:12.330672+0000","last_deep_scrub":"75'209","last_deep_scrub_stamp":"2023-11-25T04:26:12.330672+0000","last_clean_scrub_stamp":"2023-11-25T04:26:12.330672+0000","objects_scrubbed":42,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:49:33.381569+0000","scrub_duration":0.027938688999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":145844,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":175,"num_write_kb":312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":138765,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"75'172","reported_seq":249,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:57.448967+0000","last_change":"2023-11-25T04:26:57.449017+0000","last_active":"2023-11-25T04:26:57.448967+0000","last_peered":"2023-11-25T04:26:57.448967+0000","last_clean":"2023-11-25T04:26:57.448967+0000","last_became_active":"2023-11-25T04:25:51.921633+0000","last_became_peered":"2023-11-25T04:25:51.921633+0000","last_unstale":"2023-11-25T04:26:57.448967+0000","last_undegraded":"2023-11-25T04:26:57.448967+0000","last_fullsized":"2023-11-25T04:26:57.448967+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'172","last_scrub_stamp":"2023-11-25T04:26:57.448886+0000","last_deep_scrub":"75'172","last_deep_scrub_stamp":"2023-11-25T04:26:57.448886+0000","last_clean_scrub_stamp":"2023-11-25T04:26:57.448886+0000","objects_scrubbed":48,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:58:16.336649+0000","scrub_duration":0.030012414000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61303,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119,"num_write_kb":161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":61303,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"75'236","reported_seq":315,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:03.442942+0000","last_change":"2023-11-25T04:27:03.442984+0000","last_active":"2023-11-25T04:27:03.442942+0000","last_peered":"2023-11-25T04:27:03.442942+0000","last_clean":"2023-11-25T04:27:03.442942+0000","last_became_active":"2023-11-25T04:22:47.039165+0000","last_became_peered":"2023-11-25T04:22:47.039165+0000","last_unstale":"2023-11-25T04:27:03.442942+0000","last_undegraded":"2023-11-25T04:27:03.442942+0000","last_fullsized":"2023-11-25T04:27:03.442942+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'236","last_scrub_stamp":"2023-11-25T04:27:03.442858+0000","last_deep_scrub":"75'236","last_deep_scrub_stamp":"2023-11-25T04:27:03.442858+0000","last_clean_scrub_stamp":"2023-11-25T04:27:03.442858+0000","objects_scrubbed":67,"log_size":236,"log_dups_size":0,"ondisk_log_size":236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:08:36.083041+0000","scrub_duration":0.043065032000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":118227,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":185,"num_write_kb":351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3a","version":"54'169","reported_seq":235,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986335+0000","last_change":"2023-11-25T04:25:53.933198+0000","last_active":"2023-11-25T04:26:00.986335+0000","last_peered":"2023-11-25T04:26:00.986335+0000","last_clean":"2023-11-25T04:26:00.986335+0000","last_became_active":"2023-11-25T04:25:53.931982+0000","last_became_peered":"2023-11-25T04:25:53.931982+0000","last_unstale":"2023-11-25T04:26:00.986335+0000","last_undegraded":"2023-11-25T04:26:00.986335+0000","last_fullsized":"2023-11-25T04:26:00.986335+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100304,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":161,"num_write_kb":424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":100304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"75'228","reported_seq":308,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:27.488226+0000","last_change":"2023-11-25T04:26:27.488269+0000","last_active":"2023-11-25T04:26:27.488226+0000","last_peered":"2023-11-25T04:26:27.488226+0000","last_clean":"2023-11-25T04:26:27.488226+0000","last_became_active":"2023-11-25T04:25:49.919354+0000","last_became_peered":"2023-11-25T04:25:49.919354+0000","last_unstale":"2023-11-25T04:26:27.488226+0000","last_undegraded":"2023-11-25T04:26:27.488226+0000","last_fullsized":"2023-11-25T04:26:27.488226+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'228","last_scrub_stamp":"2023-11-25T04:26:27.488145+0000","last_deep_scrub":"75'228","last_deep_scrub_stamp":"2023-11-25T04:26:27.488145+0000","last_clean_scrub_stamp":"2023-11-25T04:26:27.488145+0000","objects_scrubbed":60,"log_size":228,"log_dups_size":0,"ondisk_log_size":228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:22:24.257272+0000","scrub_duration":0.042766632999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":124558,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":167,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":111,"num_bytes_recovered":124533,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.38","version":"75'199","reported_seq":278,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:28.501531+0000","last_change":"2023-11-25T04:26:28.501582+0000","last_active":"2023-11-25T04:26:28.501531+0000","last_peered":"2023-11-25T04:26:28.501531+0000","last_clean":"2023-11-25T04:26:28.501531+0000","last_became_active":"2023-11-25T04:25:49.919236+0000","last_became_peered":"2023-11-25T04:25:49.919236+0000","last_unstale":"2023-11-25T04:26:28.501531+0000","last_undegraded":"2023-11-25T04:26:28.501531+0000","last_fullsized":"2023-11-25T04:26:28.501531+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'199","last_scrub_stamp":"2023-11-25T04:26:28.501441+0000","last_deep_scrub":"75'199","last_deep_scrub_stamp":"2023-11-25T04:26:28.501441+0000","last_clean_scrub_stamp":"2023-11-25T04:26:28.501441+0000","objects_scrubbed":47,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:21:13.937490+0000","scrub_duration":0.030866581000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":120147,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":120121,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.37","version":"75'220","reported_seq":299,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:28.772505+0000","last_change":"2023-11-25T04:26:28.772532+0000","last_active":"2023-11-25T04:26:28.772505+0000","last_peered":"2023-11-25T04:26:28.772505+0000","last_clean":"2023-11-25T04:26:28.772505+0000","last_became_active":"2023-11-25T04:22:47.037328+0000","last_became_peered":"2023-11-25T04:22:47.037328+0000","last_unstale":"2023-11-25T04:26:28.772505+0000","last_undegraded":"2023-11-25T04:26:28.772505+0000","last_fullsized":"2023-11-25T04:26:28.772505+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'220","last_scrub_stamp":"2023-11-25T04:26:28.772426+0000","last_deep_scrub":"75'220","last_deep_scrub_stamp":"2023-11-25T04:26:28.772426+0000","last_clean_scrub_stamp":"2023-11-25T04:26:28.772426+0000","objects_scrubbed":53,"log_size":220,"log_dups_size":0,"ondisk_log_size":220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:08:21.318788+0000","scrub_duration":0.036194551999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":174147,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":184,"num_write_kb":353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.36","version":"75'200","reported_seq":279,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:24.608814+0000","last_change":"2023-11-25T04:26:24.608840+0000","last_active":"2023-11-25T04:26:24.608814+0000","last_peered":"2023-11-25T04:26:24.608814+0000","last_clean":"2023-11-25T04:26:24.608814+0000","last_became_active":"2023-11-25T04:22:47.039892+0000","last_became_peered":"2023-11-25T04:22:47.039892+0000","last_unstale":"2023-11-25T04:26:24.608814+0000","last_undegraded":"2023-11-25T04:26:24.608814+0000","last_fullsized":"2023-11-25T04:26:24.608814+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'200","last_scrub_stamp":"2023-11-25T04:26:24.608737+0000","last_deep_scrub":"75'200","last_deep_scrub_stamp":"2023-11-25T04:26:24.608737+0000","last_clean_scrub_stamp":"2023-11-25T04:26:24.608737+0000","objects_scrubbed":51,"log_size":200,"log_dups_size":0,"ondisk_log_size":200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:49:43.116414+0000","scrub_duration":0.03530929,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":233330,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"75'211","reported_seq":289,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:25.542014+0000","last_change":"2023-11-25T04:26:25.542044+0000","last_active":"2023-11-25T04:26:25.542014+0000","last_peered":"2023-11-25T04:26:25.542014+0000","last_clean":"2023-11-25T04:26:25.542014+0000","last_became_active":"2023-11-25T04:25:51.921800+0000","last_became_peered":"2023-11-25T04:25:51.921800+0000","last_unstale":"2023-11-25T04:26:25.542014+0000","last_undegraded":"2023-11-25T04:26:25.542014+0000","last_fullsized":"2023-11-25T04:26:25.542014+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'211","last_scrub_stamp":"2023-11-25T04:26:25.541938+0000","last_deep_scrub":"75'211","last_deep_scrub_stamp":"2023-11-25T04:26:25.541938+0000","last_clean_scrub_stamp":"2023-11-25T04:26:25.541938+0000","objects_scrubbed":55,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:32:27.981332+0000","scrub_duration":0.043489338000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":302550,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":163,"num_write_kb":431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":300323,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.34","version":"54'136","reported_seq":401,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986260+0000","last_change":"2023-11-25T04:25:50.922843+0000","last_active":"2023-11-25T04:26:00.986260+0000","last_peered":"2023-11-25T04:26:00.986260+0000","last_clean":"2023-11-25T04:26:00.986260+0000","last_became_active":"2023-11-25T04:25:50.922309+0000","last_became_peered":"2023-11-25T04:25:50.922309+0000","last_unstale":"2023-11-25T04:26:00.986260+0000","last_undegraded":"2023-11-25T04:26:00.986260+0000","last_fullsized":"2023-11-25T04:26:00.986260+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":93053,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":131,"num_write_kb":248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":89,"num_bytes_recovered":93053,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"75'183","reported_seq":261,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:19.516152+0000","last_change":"2023-11-25T04:26:19.516218+0000","last_active":"2023-11-25T04:26:19.516152+0000","last_peered":"2023-11-25T04:26:19.516152+0000","last_clean":"2023-11-25T04:26:19.516152+0000","last_became_active":"2023-11-25T04:22:47.040129+0000","last_became_peered":"2023-11-25T04:22:47.040129+0000","last_unstale":"2023-11-25T04:26:19.516152+0000","last_undegraded":"2023-11-25T04:26:19.516152+0000","last_fullsized":"2023-11-25T04:26:19.516152+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'183","last_scrub_stamp":"2023-11-25T04:26:19.516065+0000","last_deep_scrub":"75'183","last_deep_scrub_stamp":"2023-11-25T04:26:19.516065+0000","last_clean_scrub_stamp":"2023-11-25T04:26:19.516065+0000","objects_scrubbed":49,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:43:33.021008+0000","scrub_duration":0.038467327000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":149524,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":139,"num_write_kb":252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.32","version":"54'174","reported_seq":240,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986344+0000","last_change":"2023-11-25T04:25:53.934526+0000","last_active":"2023-11-25T04:26:00.986344+0000","last_peered":"2023-11-25T04:26:00.986344+0000","last_clean":"2023-11-25T04:26:00.986344+0000","last_became_active":"2023-11-25T04:25:53.933694+0000","last_became_peered":"2023-11-25T04:25:53.933694+0000","last_unstale":"2023-11-25T04:26:00.986344+0000","last_undegraded":"2023-11-25T04:26:00.986344+0000","last_fullsized":"2023-11-25T04:26:00.986344+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":95797,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":181,"num_write_kb":547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":103,"num_bytes_recovered":95797,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.31","version":"75'212","reported_seq":292,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:29.518025+0000","last_change":"2023-11-25T04:26:29.518088+0000","last_active":"2023-11-25T04:26:29.518025+0000","last_peered":"2023-11-25T04:26:29.518025+0000","last_clean":"2023-11-25T04:26:29.518025+0000","last_became_active":"2023-11-25T04:25:49.921129+0000","last_became_peered":"2023-11-25T04:25:49.921129+0000","last_unstale":"2023-11-25T04:26:29.518025+0000","last_undegraded":"2023-11-25T04:26:29.518025+0000","last_fullsized":"2023-11-25T04:26:29.518025+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'212","last_scrub_stamp":"2023-11-25T04:26:29.517931+0000","last_deep_scrub":"75'212","last_deep_scrub_stamp":"2023-11-25T04:26:29.517931+0000","last_clean_scrub_stamp":"2023-11-25T04:26:29.517931+0000","objects_scrubbed":56,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:49:54.902849+0000","scrub_duration":0.039475015000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":163190,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":169,"num_write_kb":336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":154576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"75'229","reported_seq":305,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:20.470401+0000","last_change":"2023-11-25T04:26:20.470429+0000","last_active":"2023-11-25T04:26:20.470401+0000","last_peered":"2023-11-25T04:26:20.470401+0000","last_clean":"2023-11-25T04:26:20.470401+0000","last_became_active":"2023-11-25T04:25:53.933395+0000","last_became_peered":"2023-11-25T04:25:53.933395+0000","last_unstale":"2023-11-25T04:26:20.470401+0000","last_undegraded":"2023-11-25T04:26:20.470401+0000","last_fullsized":"2023-11-25T04:26:20.470401+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'229","last_scrub_stamp":"2023-11-25T04:26:20.470325+0000","last_deep_scrub":"75'229","last_deep_scrub_stamp":"2023-11-25T04:26:20.470325+0000","last_clean_scrub_stamp":"2023-11-25T04:26:20.470325+0000","objects_scrubbed":55,"log_size":229,"log_dups_size":0,"ondisk_log_size":229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:55:52.038708+0000","scrub_duration":0.036798694,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1423587,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":182,"num_write_kb":1626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":1414508,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2f","version":"75'203","reported_seq":282,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:18.718015+0000","last_change":"2023-11-25T04:26:18.718065+0000","last_active":"2023-11-25T04:26:18.718015+0000","last_peered":"2023-11-25T04:26:18.718015+0000","last_clean":"2023-11-25T04:26:18.718015+0000","last_became_active":"2023-11-25T04:22:47.039123+0000","last_became_peered":"2023-11-25T04:22:47.039123+0000","last_unstale":"2023-11-25T04:26:18.718015+0000","last_undegraded":"2023-11-25T04:26:18.718015+0000","last_fullsized":"2023-11-25T04:26:18.718015+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'203","last_scrub_stamp":"2023-11-25T04:26:18.717922+0000","last_deep_scrub":"75'203","last_deep_scrub_stamp":"2023-11-25T04:26:18.717922+0000","last_clean_scrub_stamp":"2023-11-25T04:26:18.717922+0000","objects_scrubbed":51,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:33:40.286516+0000","scrub_duration":0.039125398999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":122752,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":143,"num_write_kb":435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"75'199","reported_seq":278,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:58.496090+0000","last_change":"2023-11-25T04:26:58.496118+0000","last_active":"2023-11-25T04:26:58.496090+0000","last_peered":"2023-11-25T04:26:58.496090+0000","last_clean":"2023-11-25T04:26:58.496090+0000","last_became_active":"2023-11-25T04:22:47.036411+0000","last_became_peered":"2023-11-25T04:22:47.036411+0000","last_unstale":"2023-11-25T04:26:58.496090+0000","last_undegraded":"2023-11-25T04:26:58.496090+0000","last_fullsized":"2023-11-25T04:26:58.496090+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'199","last_scrub_stamp":"2023-11-25T04:26:58.496005+0000","last_deep_scrub":"75'199","last_deep_scrub_stamp":"2023-11-25T04:26:58.496005+0000","last_clean_scrub_stamp":"2023-11-25T04:26:58.496005+0000","objects_scrubbed":51,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:57:18.007638+0000","scrub_duration":0.041894438999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4770899,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":140,"num_write_kb":4870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2d","version":"75'194","reported_seq":273,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:47.814115+0000","last_change":"2023-11-25T04:26:47.814157+0000","last_active":"2023-11-25T04:26:47.814115+0000","last_peered":"2023-11-25T04:26:47.814115+0000","last_clean":"2023-11-25T04:26:47.814115+0000","last_became_active":"2023-11-25T04:22:47.038477+0000","last_became_peered":"2023-11-25T04:22:47.038477+0000","last_unstale":"2023-11-25T04:26:47.814115+0000","last_undegraded":"2023-11-25T04:26:47.814115+0000","last_fullsized":"2023-11-25T04:26:47.814115+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'194","last_scrub_stamp":"2023-11-25T04:26:47.814018+0000","last_deep_scrub":"75'194","last_deep_scrub_stamp":"2023-11-25T04:26:47.814018+0000","last_clean_scrub_stamp":"2023-11-25T04:26:47.814018+0000","objects_scrubbed":51,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:20:21.601431+0000","scrub_duration":0.038348491999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":115975,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"75'230","reported_seq":307,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:19.428426+0000","last_change":"2023-11-25T04:26:19.428460+0000","last_active":"2023-11-25T04:26:19.428426+0000","last_peered":"2023-11-25T04:26:19.428426+0000","last_clean":"2023-11-25T04:26:19.428426+0000","last_became_active":"2023-11-25T04:25:52.922367+0000","last_became_peered":"2023-11-25T04:25:52.922367+0000","last_unstale":"2023-11-25T04:26:19.428426+0000","last_undegraded":"2023-11-25T04:26:19.428426+0000","last_fullsized":"2023-11-25T04:26:19.428426+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'230","last_scrub_stamp":"2023-11-25T04:26:19.428350+0000","last_deep_scrub":"75'230","last_deep_scrub_stamp":"2023-11-25T04:26:19.428350+0000","last_clean_scrub_stamp":"2023-11-25T04:26:19.428350+0000","objects_scrubbed":63,"log_size":230,"log_dups_size":0,"ondisk_log_size":230,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:10:12.216682+0000","scrub_duration":0.042625061999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":747090,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":211,"num_write_kb":985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":732276,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"75'211","reported_seq":288,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:17.381634+0000","last_change":"2023-11-25T04:26:17.381662+0000","last_active":"2023-11-25T04:26:17.381634+0000","last_peered":"2023-11-25T04:26:17.381634+0000","last_clean":"2023-11-25T04:26:17.381634+0000","last_became_active":"2023-11-25T04:25:52.923378+0000","last_became_peered":"2023-11-25T04:25:52.923378+0000","last_unstale":"2023-11-25T04:26:17.381634+0000","last_undegraded":"2023-11-25T04:26:17.381634+0000","last_fullsized":"2023-11-25T04:26:17.381634+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'211","last_scrub_stamp":"2023-11-25T04:26:17.381560+0000","last_deep_scrub":"75'211","last_deep_scrub_stamp":"2023-11-25T04:26:17.381560+0000","last_clean_scrub_stamp":"2023-11-25T04:26:17.381560+0000","objects_scrubbed":55,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:26:19.664059+0000","scrub_duration":0.036605545000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69580,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":142,"num_write_kb":315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":98,"num_bytes_recovered":69580,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2a","version":"54'171","reported_seq":468,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.985986+0000","last_change":"2023-11-25T04:25:52.925532+0000","last_active":"2023-11-25T04:26:00.985986+0000","last_peered":"2023-11-25T04:26:00.985986+0000","last_clean":"2023-11-25T04:26:00.985986+0000","last_became_active":"2023-11-25T04:25:52.924844+0000","last_became_peered":"2023-11-25T04:25:52.924844+0000","last_unstale":"2023-11-25T04:26:00.985986+0000","last_undegraded":"2023-11-25T04:26:00.985986+0000","last_fullsized":"2023-11-25T04:26:00.985986+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":115997,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":162,"num_write_kb":261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":115997,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.29","version":"75'211","reported_seq":508,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:12.692578+0000","last_change":"2023-11-25T04:26:12.692632+0000","last_active":"2023-11-25T04:26:12.692578+0000","last_peered":"2023-11-25T04:26:12.692578+0000","last_clean":"2023-11-25T04:26:12.692578+0000","last_became_active":"2023-11-25T04:25:49.919886+0000","last_became_peered":"2023-11-25T04:25:49.919886+0000","last_unstale":"2023-11-25T04:26:12.692578+0000","last_undegraded":"2023-11-25T04:26:12.692578+0000","last_fullsized":"2023-11-25T04:26:12.692578+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'211","last_scrub_stamp":"2023-11-25T04:26:12.692467+0000","last_deep_scrub":"75'211","last_deep_scrub_stamp":"2023-11-25T04:26:12.692467+0000","last_clean_scrub_stamp":"2023-11-25T04:26:12.692467+0000","objects_scrubbed":59,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:13:03.899209+0000","scrub_duration":0.043132342999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":212878,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":212878,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"75'218","reported_seq":297,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:00.495508+0000","last_change":"2023-11-25T04:27:00.495508+0000","last_active":"2023-11-25T04:27:00.495508+0000","last_peered":"2023-11-25T04:27:00.495508+0000","last_clean":"2023-11-25T04:27:00.495508+0000","last_became_active":"2023-11-25T04:22:47.036135+0000","last_became_peered":"2023-11-25T04:22:47.036135+0000","last_unstale":"2023-11-25T04:27:00.495508+0000","last_undegraded":"2023-11-25T04:27:00.495508+0000","last_fullsized":"2023-11-25T04:27:00.495508+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'218","last_scrub_stamp":"2023-11-25T04:27:00.495361+0000","last_deep_scrub":"75'218","last_deep_scrub_stamp":"2023-11-25T04:27:00.495361+0000","last_clean_scrub_stamp":"2023-11-25T04:27:00.495361+0000","objects_scrubbed":49,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:17:48.678120+0000","scrub_duration":0.034866902999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":102957,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":172,"num_write_kb":610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"54'167","reported_seq":467,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:09.152384+0000","last_change":"2023-11-25T04:25:51.922925+0000","last_active":"2023-11-25T04:26:09.152384+0000","last_peered":"2023-11-25T04:26:09.152384+0000","last_clean":"2023-11-25T04:26:09.152384+0000","last_became_active":"2023-11-25T04:25:51.922364+0000","last_became_peered":"2023-11-25T04:25:51.922364+0000","last_unstale":"2023-11-25T04:26:09.152384+0000","last_undegraded":"2023-11-25T04:26:09.152384+0000","last_fullsized":"2023-11-25T04:26:09.152384+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":182683,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":171,"num_write_kb":791,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":182683,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"54'187","reported_seq":507,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:09.671589+0000","last_change":"2023-11-25T04:25:51.922529+0000","last_active":"2023-11-25T04:26:09.671589+0000","last_peered":"2023-11-25T04:26:09.671589+0000","last_clean":"2023-11-25T04:26:09.671589+0000","last_became_active":"2023-11-25T04:25:51.922064+0000","last_became_peered":"2023-11-25T04:25:51.922064+0000","last_unstale":"2023-11-25T04:26:09.671589+0000","last_undegraded":"2023-11-25T04:26:09.671589+0000","last_fullsized":"2023-11-25T04:26:09.671589+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":187,"log_dups_size":0,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":86311,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":171,"num_write_kb":274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":116,"num_bytes_recovered":86311,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.25","version":"75'201","reported_seq":278,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:10.399641+0000","last_change":"2023-11-25T04:26:10.399668+0000","last_active":"2023-11-25T04:26:10.399641+0000","last_peered":"2023-11-25T04:26:10.399641+0000","last_clean":"2023-11-25T04:26:10.399641+0000","last_became_active":"2023-11-25T04:25:52.924795+0000","last_became_peered":"2023-11-25T04:25:52.924795+0000","last_unstale":"2023-11-25T04:26:10.399641+0000","last_undegraded":"2023-11-25T04:26:10.399641+0000","last_fullsized":"2023-11-25T04:26:10.399641+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'201","last_scrub_stamp":"2023-11-25T04:26:10.399561+0000","last_deep_scrub":"75'201","last_deep_scrub_stamp":"2023-11-25T04:26:10.399561+0000","last_clean_scrub_stamp":"2023-11-25T04:26:10.399561+0000","objects_scrubbed":52,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:15:15.077501+0000","scrub_duration":0.038027564,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":128288,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":167,"num_write_kb":267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":120265,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.24","version":"75'195","reported_seq":482,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:43.546743+0000","last_change":"2023-11-25T04:26:43.546743+0000","last_active":"2023-11-25T04:26:43.546743+0000","last_peered":"2023-11-25T04:26:43.546743+0000","last_clean":"2023-11-25T04:26:43.546743+0000","last_became_active":"2023-11-25T04:25:50.922216+0000","last_became_peered":"2023-11-25T04:25:50.922216+0000","last_unstale":"2023-11-25T04:26:43.546743+0000","last_undegraded":"2023-11-25T04:26:43.546743+0000","last_fullsized":"2023-11-25T04:26:43.546743+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'195","last_scrub_stamp":"2023-11-25T04:26:43.546552+0000","last_deep_scrub":"75'195","last_deep_scrub_stamp":"2023-11-25T04:26:43.546552+0000","last_clean_scrub_stamp":"2023-11-25T04:26:43.546552+0000","objects_scrubbed":44,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:16:44.328689+0000","scrub_duration":0.034140957,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":107507,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":148,"num_write_kb":288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":107494,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.23","version":"75'222","reported_seq":302,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:50.770920+0000","last_change":"2023-11-25T04:26:50.770920+0000","last_active":"2023-11-25T04:26:50.770920+0000","last_peered":"2023-11-25T04:26:50.770920+0000","last_clean":"2023-11-25T04:26:50.770920+0000","last_became_active":"2023-11-25T04:22:47.036823+0000","last_became_peered":"2023-11-25T04:22:47.036823+0000","last_unstale":"2023-11-25T04:26:50.770920+0000","last_undegraded":"2023-11-25T04:26:50.770920+0000","last_fullsized":"2023-11-25T04:26:50.770920+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'222","last_scrub_stamp":"2023-11-25T04:26:50.770805+0000","last_deep_scrub":"75'222","last_deep_scrub_stamp":"2023-11-25T04:26:50.770805+0000","last_clean_scrub_stamp":"2023-11-25T04:26:50.770805+0000","objects_scrubbed":67,"log_size":222,"log_dups_size":0,"ondisk_log_size":222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:16:51.552942+0000","scrub_duration":0.048356029000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4304582,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":152,"num_write_kb":4331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.22","version":"54'189","reported_seq":519,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:09.152355+0000","last_change":"2023-11-25T04:25:54.940559+0000","last_active":"2023-11-25T04:26:09.152355+0000","last_peered":"2023-11-25T04:26:09.152355+0000","last_clean":"2023-11-25T04:26:09.152355+0000","last_became_active":"2023-11-25T04:25:54.939584+0000","last_became_peered":"2023-11-25T04:25:54.939584+0000","last_unstale":"2023-11-25T04:26:09.152355+0000","last_undegraded":"2023-11-25T04:26:09.152355+0000","last_fullsized":"2023-11-25T04:26:09.152355+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":129145,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":185,"num_write_kb":432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":121,"num_bytes_recovered":129107,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"75'260","reported_seq":340,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:06.501506+0000","last_change":"2023-11-25T04:27:06.501536+0000","last_active":"2023-11-25T04:27:06.501506+0000","last_peered":"2023-11-25T04:27:06.501506+0000","last_clean":"2023-11-25T04:27:06.501506+0000","last_became_active":"2023-11-25T04:22:47.037206+0000","last_became_peered":"2023-11-25T04:22:47.037206+0000","last_unstale":"2023-11-25T04:27:06.501506+0000","last_undegraded":"2023-11-25T04:27:06.501506+0000","last_fullsized":"2023-11-25T04:27:06.501506+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'260","last_scrub_stamp":"2023-11-25T04:27:06.501413+0000","last_deep_scrub":"75'260","last_deep_scrub_stamp":"2023-11-25T04:27:06.501413+0000","last_clean_scrub_stamp":"2023-11-25T04:27:06.501413+0000","objects_scrubbed":59,"log_size":260,"log_dups_size":0,"ondisk_log_size":260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:18:02.436135+0000","scrub_duration":0.044736607999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":215311,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":194,"num_write_kb":381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"54'180","reported_seq":246,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.985760+0000","last_change":"2023-11-25T04:25:53.933176+0000","last_active":"2023-11-25T04:26:00.985760+0000","last_peered":"2023-11-25T04:26:00.985760+0000","last_clean":"2023-11-25T04:26:00.985760+0000","last_became_active":"2023-11-25T04:25:53.931937+0000","last_became_peered":"2023-11-25T04:25:53.931937+0000","last_unstale":"2023-11-25T04:26:00.985760+0000","last_undegraded":"2023-11-25T04:26:00.985760+0000","last_fullsized":"2023-11-25T04:26:00.985760+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104603,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":172,"num_write_kb":267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":104603,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"75'193","reported_seq":272,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:32.532707+0000","last_change":"2023-11-25T04:26:32.532758+0000","last_active":"2023-11-25T04:26:32.532707+0000","last_peered":"2023-11-25T04:26:32.532707+0000","last_clean":"2023-11-25T04:26:32.532707+0000","last_became_active":"2023-11-25T04:22:47.036107+0000","last_became_peered":"2023-11-25T04:22:47.036107+0000","last_unstale":"2023-11-25T04:26:32.532707+0000","last_undegraded":"2023-11-25T04:26:32.532707+0000","last_fullsized":"2023-11-25T04:26:32.532707+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'193","last_scrub_stamp":"2023-11-25T04:26:32.532624+0000","last_deep_scrub":"75'193","last_deep_scrub_stamp":"2023-11-25T04:26:32.532624+0000","last_clean_scrub_stamp":"2023-11-25T04:26:32.532624+0000","objects_scrubbed":56,"log_size":193,"log_dups_size":0,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:09:38.769552+0000","scrub_duration":0.038075093999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":183586,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":127,"num_write_kb":284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"75'117","reported_seq":203,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:31.496279+0000","last_change":"2023-11-25T04:26:31.496307+0000","last_active":"2023-11-25T04:26:31.496279+0000","last_peered":"2023-11-25T04:26:31.496279+0000","last_clean":"2023-11-25T04:26:31.496279+0000","last_became_active":"2023-11-25T04:25:49.920010+0000","last_became_peered":"2023-11-25T04:25:49.920010+0000","last_unstale":"2023-11-25T04:26:31.496279+0000","last_undegraded":"2023-11-25T04:26:31.496279+0000","last_fullsized":"2023-11-25T04:26:31.496279+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'117","last_scrub_stamp":"2023-11-25T04:26:31.496196+0000","last_deep_scrub":"75'117","last_deep_scrub_stamp":"2023-11-25T04:26:31.496196+0000","last_clean_scrub_stamp":"2023-11-25T04:26:31.496196+0000","objects_scrubbed":3,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:26:02.874431+0000","scrub_duration":0.0076030029999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2816216,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":113,"num_write_kb":2816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":2816182,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"75'189","reported_seq":267,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:40.589192+0000","last_change":"2023-11-25T04:26:40.589251+0000","last_active":"2023-11-25T04:26:40.589192+0000","last_peered":"2023-11-25T04:26:40.589192+0000","last_clean":"2023-11-25T04:26:40.589192+0000","last_became_active":"2023-11-25T04:22:47.037158+0000","last_became_peered":"2023-11-25T04:22:47.037158+0000","last_unstale":"2023-11-25T04:26:40.589192+0000","last_undegraded":"2023-11-25T04:26:40.589192+0000","last_fullsized":"2023-11-25T04:26:40.589192+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'189","last_scrub_stamp":"2023-11-25T04:26:40.589106+0000","last_deep_scrub":"75'189","last_deep_scrub_stamp":"2023-11-25T04:26:40.589106+0000","last_clean_scrub_stamp":"2023-11-25T04:26:40.589106+0000","objects_scrubbed":44,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:32:56.267980+0000","scrub_duration":0.032171351000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":205976,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.12","version":"49'2","reported_seq":91,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:30.784729+0000","last_change":"2023-11-25T04:26:30.784752+0000","last_active":"2023-11-25T04:26:30.784729+0000","last_peered":"2023-11-25T04:26:30.784729+0000","last_clean":"2023-11-25T04:26:30.784729+0000","last_became_active":"2023-11-25T04:25:47.000516+0000","last_became_peered":"2023-11-25T04:25:47.000516+0000","last_unstale":"2023-11-25T04:26:30.784729+0000","last_undegraded":"2023-11-25T04:26:30.784729+0000","last_fullsized":"2023-11-25T04:26:30.784729+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'2","last_scrub_stamp":"2023-11-25T04:26:30.784639+0000","last_deep_scrub":"49'2","last_deep_scrub_stamp":"2023-11-25T04:26:30.784639+0000","last_clean_scrub_stamp":"2023-11-25T04:26:30.784639+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:38:15.370559+0000","scrub_duration":0.001803528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"54'175","reported_seq":490,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986834+0000","last_change":"2023-11-25T04:25:52.923694+0000","last_active":"2023-11-25T04:26:00.986834+0000","last_peered":"2023-11-25T04:26:00.986834+0000","last_clean":"2023-11-25T04:26:00.986834+0000","last_became_active":"2023-11-25T04:25:52.923045+0000","last_became_peered":"2023-11-25T04:25:52.923045+0000","last_unstale":"2023-11-25T04:26:00.986834+0000","last_undegraded":"2023-11-25T04:26:00.986834+0000","last_fullsized":"2023-11-25T04:26:00.986834+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":171823,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":185,"num_write_kb":340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":166387,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":89,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:18.488426+0000","last_change":"2023-11-25T04:26:18.488445+0000","last_active":"2023-11-25T04:26:18.488426+0000","last_peered":"2023-11-25T04:26:18.488426+0000","last_clean":"2023-11-25T04:26:18.488426+0000","last_became_active":"2023-11-25T04:25:46.993870+0000","last_became_peered":"2023-11-25T04:25:46.993870+0000","last_unstale":"2023-11-25T04:26:18.488426+0000","last_undegraded":"2023-11-25T04:26:18.488426+0000","last_fullsized":"2023-11-25T04:26:18.488426+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:26:18.488345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:26:18.488345+0000","last_clean_scrub_stamp":"2023-11-25T04:26:18.488345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:43:21.558596+0000","scrub_duration":0.002108399,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"54'152","reported_seq":436,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986747+0000","last_change":"2023-11-25T04:25:50.922922+0000","last_active":"2023-11-25T04:26:00.986747+0000","last_peered":"2023-11-25T04:26:00.986747+0000","last_clean":"2023-11-25T04:26:00.986747+0000","last_became_active":"2023-11-25T04:25:50.922511+0000","last_became_peered":"2023-11-25T04:25:50.922511+0000","last_unstale":"2023-11-25T04:26:00.986747+0000","last_undegraded":"2023-11-25T04:26:00.986747+0000","last_fullsized":"2023-11-25T04:26:00.986747+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":579704,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":97,"num_bytes_recovered":570044,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.10","version":"18'8","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.985749+0000","last_change":"2023-11-25T04:25:47.000499+0000","last_active":"2023-11-25T04:26:00.985749+0000","last_peered":"2023-11-25T04:26:00.985749+0000","last_clean":"2023-11-25T04:26:00.985749+0000","last_became_active":"2023-11-25T04:25:46.998778+0000","last_became_peered":"2023-11-25T04:25:46.998778+0000","last_unstale":"2023-11-25T04:26:00.985749+0000","last_undegraded":"2023-11-25T04:26:00.985749+0000","last_fullsized":"2023-11-25T04:26:00.985749+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"75'3","reported_seq":47,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:36.494324+0000","last_change":"2023-11-25T04:26:36.494358+0000","last_active":"2023-11-25T04:26:36.494324+0000","last_peered":"2023-11-25T04:26:36.494324+0000","last_clean":"2023-11-25T04:26:36.494324+0000","last_became_active":"2023-11-25T04:25:46.997108+0000","last_became_peered":"2023-11-25T04:25:46.997108+0000","last_unstale":"2023-11-25T04:26:36.494324+0000","last_undegraded":"2023-11-25T04:26:36.494324+0000","last_fullsized":"2023-11-25T04:26:36.494324+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'3","last_scrub_stamp":"2023-11-25T04:26:36.494205+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2023-11-25T04:26:36.494205+0000","last_clean_scrub_stamp":"2023-11-25T04:26:36.494205+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:47:03.918120+0000","scrub_duration":0.0026531879999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"75'208","reported_seq":503,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:07.384457+0000","last_change":"2023-11-25T04:27:07.384457+0000","last_active":"2023-11-25T04:27:07.384457+0000","last_peered":"2023-11-25T04:27:07.384457+0000","last_clean":"2023-11-25T04:27:07.384457+0000","last_became_active":"2023-11-25T04:25:50.921772+0000","last_became_peered":"2023-11-25T04:25:50.921772+0000","last_unstale":"2023-11-25T04:27:07.384457+0000","last_undegraded":"2023-11-25T04:27:07.384457+0000","last_fullsized":"2023-11-25T04:27:07.384457+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'208","last_scrub_stamp":"2023-11-25T04:27:07.384323+0000","last_deep_scrub":"75'208","last_deep_scrub_stamp":"2023-11-25T04:27:07.384323+0000","last_clean_scrub_stamp":"2023-11-25T04:27:07.384323+0000","objects_scrubbed":49,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:52:56.354420+0000","scrub_duration":0.034118876999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112103,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":164,"num_write_kb":240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":99,"num_bytes_recovered":103472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"75'3","reported_seq":92,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:36.492691+0000","last_change":"2023-11-25T04:26:36.492740+0000","last_active":"2023-11-25T04:26:36.492691+0000","last_peered":"2023-11-25T04:26:36.492691+0000","last_clean":"2023-11-25T04:26:36.492691+0000","last_became_active":"2023-11-25T04:25:46.994640+0000","last_became_peered":"2023-11-25T04:25:46.994640+0000","last_unstale":"2023-11-25T04:26:36.492691+0000","last_undegraded":"2023-11-25T04:26:36.492691+0000","last_fullsized":"2023-11-25T04:26:36.492691+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'3","last_scrub_stamp":"2023-11-25T04:26:36.492569+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2023-11-25T04:26:36.492569+0000","last_clean_scrub_stamp":"2023-11-25T04:26:36.492569+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:28:25.392135+0000","scrub_duration":0.0033623030000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"75'229","reported_seq":309,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:05.415841+0000","last_change":"2023-11-25T04:27:05.415841+0000","last_active":"2023-11-25T04:27:05.415841+0000","last_peered":"2023-11-25T04:27:05.415841+0000","last_clean":"2023-11-25T04:27:05.415841+0000","last_became_active":"2023-11-25T04:22:47.039296+0000","last_became_peered":"2023-11-25T04:22:47.039296+0000","last_unstale":"2023-11-25T04:27:05.415841+0000","last_undegraded":"2023-11-25T04:27:05.415841+0000","last_fullsized":"2023-11-25T04:27:05.415841+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'229","last_scrub_stamp":"2023-11-25T04:27:05.415739+0000","last_deep_scrub":"75'229","last_deep_scrub_stamp":"2023-11-25T04:27:05.415739+0000","last_clean_scrub_stamp":"2023-11-25T04:27:05.415739+0000","objects_scrubbed":56,"log_size":229,"log_dups_size":0,"ondisk_log_size":229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:46:38.706963+0000","scrub_duration":0.037100175999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154533,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":182,"num_write_kb":299,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"75'2","reported_seq":47,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:33.490361+0000","last_change":"2023-11-25T04:26:33.490414+0000","last_active":"2023-11-25T04:26:33.490361+0000","last_peered":"2023-11-25T04:26:33.490361+0000","last_clean":"2023-11-25T04:26:33.490361+0000","last_became_active":"2023-11-25T04:25:46.999604+0000","last_became_peered":"2023-11-25T04:25:46.999604+0000","last_unstale":"2023-11-25T04:26:33.490361+0000","last_undegraded":"2023-11-25T04:26:33.490361+0000","last_fullsized":"2023-11-25T04:26:33.490361+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'2","last_scrub_stamp":"2023-11-25T04:26:33.490271+0000","last_deep_scrub":"75'2","last_deep_scrub_stamp":"2023-11-25T04:26:33.490271+0000","last_clean_scrub_stamp":"2023-11-25T04:26:33.490271+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:29:08.502565+0000","scrub_duration":0.0030545450000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"54'147","reported_seq":212,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986571+0000","last_change":"2023-11-25T04:25:54.939274+0000","last_active":"2023-11-25T04:26:00.986571+0000","last_peered":"2023-11-25T04:26:00.986571+0000","last_clean":"2023-11-25T04:26:00.986571+0000","last_became_active":"2023-11-25T04:25:54.938236+0000","last_became_peered":"2023-11-25T04:25:54.938236+0000","last_unstale":"2023-11-25T04:26:00.986571+0000","last_undegraded":"2023-11-25T04:26:00.986571+0000","last_fullsized":"2023-11-25T04:26:00.986571+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":147,"log_dups_size":0,"ondisk_log_size":147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":87524,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":155,"num_write_kb":235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":87524,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":43,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:41.504866+0000","last_change":"2023-11-25T04:26:41.504892+0000","last_active":"2023-11-25T04:26:41.504866+0000","last_peered":"2023-11-25T04:26:41.504866+0000","last_clean":"2023-11-25T04:26:41.504866+0000","last_became_active":"2023-11-25T04:25:46.995712+0000","last_became_peered":"2023-11-25T04:25:46.995712+0000","last_unstale":"2023-11-25T04:26:41.504866+0000","last_undegraded":"2023-11-25T04:26:41.504866+0000","last_fullsized":"2023-11-25T04:26:41.504866+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:26:41.504775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:26:41.504775+0000","last_clean_scrub_stamp":"2023-11-25T04:26:41.504775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:35:58.006258+0000","scrub_duration":0.001777907,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"54'155","reported_seq":442,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986276+0000","last_change":"2023-11-25T04:25:52.923096+0000","last_active":"2023-11-25T04:26:00.986276+0000","last_peered":"2023-11-25T04:26:00.986276+0000","last_clean":"2023-11-25T04:26:00.986276+0000","last_became_active":"2023-11-25T04:25:52.922549+0000","last_became_peered":"2023-11-25T04:25:52.922549+0000","last_unstale":"2023-11-25T04:26:00.986276+0000","last_undegraded":"2023-11-25T04:26:00.986276+0000","last_fullsized":"2023-11-25T04:26:00.986276+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":266919,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":264258,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"75'13","reported_seq":107,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:05.498676+0000","last_change":"2023-11-25T04:27:05.498699+0000","last_active":"2023-11-25T04:27:05.498676+0000","last_peered":"2023-11-25T04:27:05.498676+0000","last_clean":"2023-11-25T04:27:05.498676+0000","last_became_active":"2023-11-25T04:25:47.000687+0000","last_became_peered":"2023-11-25T04:25:47.000687+0000","last_unstale":"2023-11-25T04:27:05.498676+0000","last_undegraded":"2023-11-25T04:27:05.498676+0000","last_fullsized":"2023-11-25T04:27:05.498676+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'13","last_scrub_stamp":"2023-11-25T04:27:05.498602+0000","last_deep_scrub":"75'13","last_deep_scrub_stamp":"2023-11-25T04:27:05.498602+0000","last_clean_scrub_stamp":"2023-11-25T04:27:05.498602+0000","objects_scrubbed":2,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:44:21.302320+0000","scrub_duration":0.0090048890000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":4101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4194304,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"75'235","reported_seq":314,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:49.411583+0000","last_change":"2023-11-25T04:26:49.411614+0000","last_active":"2023-11-25T04:26:49.411583+0000","last_peered":"2023-11-25T04:26:49.411583+0000","last_clean":"2023-11-25T04:26:49.411583+0000","last_became_active":"2023-11-25T04:25:50.922409+0000","last_became_peered":"2023-11-25T04:25:50.922409+0000","last_unstale":"2023-11-25T04:26:49.411583+0000","last_undegraded":"2023-11-25T04:26:49.411583+0000","last_fullsized":"2023-11-25T04:26:49.411583+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'235","last_scrub_stamp":"2023-11-25T04:26:49.411506+0000","last_deep_scrub":"75'235","last_deep_scrub_stamp":"2023-11-25T04:26:49.411506+0000","last_clean_scrub_stamp":"2023-11-25T04:26:49.411506+0000","objects_scrubbed":62,"log_size":235,"log_dups_size":0,"ondisk_log_size":235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:38:13.278410+0000","scrub_duration":0.044042721,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":553893,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":169,"num_write_kb":770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":553893,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":35,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986132+0000","last_change":"2023-11-25T04:25:46.998818+0000","last_active":"2023-11-25T04:26:00.986132+0000","last_peered":"2023-11-25T04:26:00.986132+0000","last_clean":"2023-11-25T04:26:00.986132+0000","last_became_active":"2023-11-25T04:25:46.997019+0000","last_became_peered":"2023-11-25T04:25:46.997019+0000","last_unstale":"2023-11-25T04:26:00.986132+0000","last_undegraded":"2023-11-25T04:26:00.986132+0000","last_fullsized":"2023-11-25T04:26:00.986132+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"54'163","reported_seq":229,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986523+0000","last_change":"2023-11-25T04:25:53.934183+0000","last_active":"2023-11-25T04:26:00.986523+0000","last_peered":"2023-11-25T04:26:00.986523+0000","last_clean":"2023-11-25T04:26:00.986523+0000","last_became_active":"2023-11-25T04:25:53.933471+0000","last_became_peered":"2023-11-25T04:25:53.933471+0000","last_unstale":"2023-11-25T04:26:00.986523+0000","last_undegraded":"2023-11-25T04:26:00.986523+0000","last_fullsized":"2023-11-25T04:26:00.986523+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":320867,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":190,"num_write_kb":577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":108,"num_bytes_recovered":311640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":90,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:26.554458+0000","last_change":"2023-11-25T04:26:26.554458+0000","last_active":"2023-11-25T04:26:26.554458+0000","last_peered":"2023-11-25T04:26:26.554458+0000","last_clean":"2023-11-25T04:26:26.554458+0000","last_became_active":"2023-11-25T04:25:46.994101+0000","last_became_peered":"2023-11-25T04:25:46.994101+0000","last_unstale":"2023-11-25T04:26:26.554458+0000","last_undegraded":"2023-11-25T04:26:26.554458+0000","last_fullsized":"2023-11-25T04:26:26.554458+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:26:26.554355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:26:26.554355+0000","last_clean_scrub_stamp":"2023-11-25T04:26:26.554355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:20:20.095629+0000","scrub_duration":0.0016349400000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"75'187","reported_seq":266,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:35.557378+0000","last_change":"2023-11-25T04:26:35.557414+0000","last_active":"2023-11-25T04:26:35.557378+0000","last_peered":"2023-11-25T04:26:35.557378+0000","last_clean":"2023-11-25T04:26:35.557378+0000","last_became_active":"2023-11-25T04:25:49.920971+0000","last_became_peered":"2023-11-25T04:25:49.920971+0000","last_unstale":"2023-11-25T04:26:35.557378+0000","last_undegraded":"2023-11-25T04:26:35.557378+0000","last_fullsized":"2023-11-25T04:26:35.557378+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'187","last_scrub_stamp":"2023-11-25T04:26:35.557305+0000","last_deep_scrub":"75'187","last_deep_scrub_stamp":"2023-11-25T04:26:35.557305+0000","last_clean_scrub_stamp":"2023-11-25T04:26:35.557305+0000","objects_scrubbed":46,"log_size":187,"log_dups_size":0,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:15:17.199719+0000","scrub_duration":0.039856336999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3502371,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":3718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":89,"num_bytes_recovered":3497669,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"49'1","reported_seq":80,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:52.518138+0000","last_change":"2023-11-25T04:26:52.518138+0000","last_active":"2023-11-25T04:26:52.518138+0000","last_peered":"2023-11-25T04:26:52.518138+0000","last_clean":"2023-11-25T04:26:52.518138+0000","last_became_active":"2023-11-25T04:22:46.034158+0000","last_became_peered":"2023-11-25T04:22:46.034158+0000","last_unstale":"2023-11-25T04:26:52.518138+0000","last_undegraded":"2023-11-25T04:26:52.518138+0000","last_fullsized":"2023-11-25T04:26:52.518138+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'1","last_scrub_stamp":"2023-11-25T04:26:52.518023+0000","last_deep_scrub":"49'1","last_deep_scrub_stamp":"2023-11-25T04:26:52.518023+0000","last_clean_scrub_stamp":"2023-11-25T04:26:52.518023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:44:08.321741+0000","scrub_duration":0.001721757,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"75'194","reported_seq":273,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:15.719582+0000","last_change":"2023-11-25T04:26:15.719633+0000","last_active":"2023-11-25T04:26:15.719582+0000","last_peered":"2023-11-25T04:26:15.719582+0000","last_clean":"2023-11-25T04:26:15.719582+0000","last_became_active":"2023-11-25T04:22:47.038081+0000","last_became_peered":"2023-11-25T04:22:47.038081+0000","last_unstale":"2023-11-25T04:26:15.719582+0000","last_undegraded":"2023-11-25T04:26:15.719582+0000","last_fullsized":"2023-11-25T04:26:15.719582+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'194","last_scrub_stamp":"2023-11-25T04:26:15.719490+0000","last_deep_scrub":"75'194","last_deep_scrub_stamp":"2023-11-25T04:26:15.719490+0000","last_clean_scrub_stamp":"2023-11-25T04:26:15.719490+0000","objects_scrubbed":53,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:43:29.224433+0000","scrub_duration":0.037765356,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":174088,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":146,"num_write_kb":385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"12'157","reported_seq":233,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.985653+0000","last_change":"2023-11-25T04:25:47.000504+0000","last_active":"2023-11-25T04:26:00.985653+0000","last_peered":"2023-11-25T04:26:00.985653+0000","last_clean":"2023-11-25T04:26:00.985653+0000","last_became_active":"2023-11-25T04:25:46.999721+0000","last_became_peered":"2023-11-25T04:25:46.999721+0000","last_unstale":"2023-11-25T04:26:00.985653+0000","last_undegraded":"2023-11-25T04:26:00.985653+0000","last_fullsized":"2023-11-25T04:26:00.985653+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:35.919850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:35.919850+0000","last_clean_scrub_stamp":"2023-11-25T04:22:35.919850+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":35,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.985910+0000","last_change":"2023-11-25T04:25:46.998960+0000","last_active":"2023-11-25T04:26:00.985910+0000","last_peered":"2023-11-25T04:26:00.985910+0000","last_clean":"2023-11-25T04:26:00.985910+0000","last_became_active":"2023-11-25T04:25:46.997693+0000","last_became_peered":"2023-11-25T04:25:46.997693+0000","last_unstale":"2023-11-25T04:26:00.985910+0000","last_undegraded":"2023-11-25T04:26:00.985910+0000","last_fullsized":"2023-11-25T04:26:00.985910+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"46'1","reported_seq":72,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:09.671195+0000","last_change":"2023-11-25T04:22:46.037044+0000","last_active":"2023-11-25T04:26:09.671195+0000","last_peered":"2023-11-25T04:26:09.671195+0000","last_clean":"2023-11-25T04:26:09.671195+0000","last_became_active":"2023-11-25T04:22:46.035916+0000","last_became_peered":"2023-11-25T04:22:46.035916+0000","last_unstale":"2023-11-25T04:26:09.671195+0000","last_undegraded":"2023-11-25T04:26:09.671195+0000","last_fullsized":"2023-11-25T04:26:09.671195+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"54'146","reported_seq":214,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986701+0000","last_change":"2023-11-25T04:25:51.923452+0000","last_active":"2023-11-25T04:26:00.986701+0000","last_peered":"2023-11-25T04:26:00.986701+0000","last_clean":"2023-11-25T04:26:00.986701+0000","last_became_active":"2023-11-25T04:25:51.922490+0000","last_became_peered":"2023-11-25T04:25:51.922490+0000","last_unstale":"2023-11-25T04:26:00.986701+0000","last_undegraded":"2023-11-25T04:26:00.986701+0000","last_fullsized":"2023-11-25T04:26:00.986701+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":502154,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":170,"num_write_kb":687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":99,"num_bytes_recovered":502101,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":43,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:30.509601+0000","last_change":"2023-11-25T04:26:30.509621+0000","last_active":"2023-11-25T04:26:30.509601+0000","last_peered":"2023-11-25T04:26:30.509601+0000","last_clean":"2023-11-25T04:26:30.509601+0000","last_became_active":"2023-11-25T04:25:47.000033+0000","last_became_peered":"2023-11-25T04:25:47.000033+0000","last_unstale":"2023-11-25T04:26:30.509601+0000","last_undegraded":"2023-11-25T04:26:30.509601+0000","last_fullsized":"2023-11-25T04:26:30.509601+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:26:30.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:26:30.509522+0000","last_clean_scrub_stamp":"2023-11-25T04:26:30.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:17:09.583541+0000","scrub_duration":0.001771145,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"75'38","reported_seq":167,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:44.819150+0000","last_change":"2023-11-25T04:26:44.819150+0000","last_active":"2023-11-25T04:26:44.819150+0000","last_peered":"2023-11-25T04:26:44.819150+0000","last_clean":"2023-11-25T04:26:44.819150+0000","last_became_active":"2023-11-25T04:25:46.995637+0000","last_became_peered":"2023-11-25T04:25:46.995637+0000","last_unstale":"2023-11-25T04:26:44.819150+0000","last_undegraded":"2023-11-25T04:26:44.819150+0000","last_fullsized":"2023-11-25T04:26:44.819150+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'38","last_scrub_stamp":"2023-11-25T04:26:44.819010+0000","last_deep_scrub":"75'38","last_deep_scrub_stamp":"2023-11-25T04:26:44.819010+0000","last_clean_scrub_stamp":"2023-11-25T04:26:44.819010+0000","objects_scrubbed":1,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:17:29.276433+0000","scrub_duration":0.003812251,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":319160,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":35,"num_read_kb":334,"num_write":35,"num_write_kb":334,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":319160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"75'210","reported_seq":290,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:38.568589+0000","last_change":"2023-11-25T04:26:38.568589+0000","last_active":"2023-11-25T04:26:38.568589+0000","last_peered":"2023-11-25T04:26:38.568589+0000","last_clean":"2023-11-25T04:26:38.568589+0000","last_became_active":"2023-11-25T04:22:47.039197+0000","last_became_peered":"2023-11-25T04:22:47.039197+0000","last_unstale":"2023-11-25T04:26:38.568589+0000","last_undegraded":"2023-11-25T04:26:38.568589+0000","last_fullsized":"2023-11-25T04:26:38.568589+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'210","last_scrub_stamp":"2023-11-25T04:26:38.568488+0000","last_deep_scrub":"75'210","last_deep_scrub_stamp":"2023-11-25T04:26:38.568488+0000","last_clean_scrub_stamp":"2023-11-25T04:26:38.568488+0000","objects_scrubbed":52,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:17:23.025912+0000","scrub_duration":0.034345986000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":169303,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":189,"num_write_kb":333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":74,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:09.671105+0000","last_change":"2023-11-25T04:22:42.000382+0000","last_active":"2023-11-25T04:26:09.671105+0000","last_peered":"2023-11-25T04:26:09.671105+0000","last_clean":"2023-11-25T04:26:09.671105+0000","last_became_active":"2023-11-25T04:22:42.000041+0000","last_became_peered":"2023-11-25T04:22:42.000041+0000","last_unstale":"2023-11-25T04:26:09.671105+0000","last_undegraded":"2023-11-25T04:26:09.671105+0000","last_fullsized":"2023-11-25T04:26:09.671105+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"75'72","reported_seq":166,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:15.436284+0000","last_change":"2023-11-25T04:26:15.436284+0000","last_active":"2023-11-25T04:26:15.436284+0000","last_peered":"2023-11-25T04:26:15.436284+0000","last_clean":"2023-11-25T04:26:15.436284+0000","last_became_active":"2023-11-25T04:25:47.001162+0000","last_became_peered":"2023-11-25T04:25:47.001162+0000","last_unstale":"2023-11-25T04:26:15.436284+0000","last_undegraded":"2023-11-25T04:26:15.436284+0000","last_fullsized":"2023-11-25T04:26:15.436284+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'72","last_scrub_stamp":"2023-11-25T04:26:15.436148+0000","last_deep_scrub":"75'72","last_deep_scrub_stamp":"2023-11-25T04:26:15.436148+0000","last_clean_scrub_stamp":"2023-11-25T04:26:15.436148+0000","objects_scrubbed":1,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:37:38.643270+0000","scrub_duration":0.01087811,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":69,"num_write_kb":4130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4194304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"75'220","reported_seq":300,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:14.718055+0000","last_change":"2023-11-25T04:26:14.718055+0000","last_active":"2023-11-25T04:26:14.718055+0000","last_peered":"2023-11-25T04:26:14.718055+0000","last_clean":"2023-11-25T04:26:14.718055+0000","last_became_active":"2023-11-25T04:22:47.038046+0000","last_became_peered":"2023-11-25T04:22:47.038046+0000","last_unstale":"2023-11-25T04:26:14.718055+0000","last_undegraded":"2023-11-25T04:26:14.718055+0000","last_fullsized":"2023-11-25T04:26:14.718055+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'220","last_scrub_stamp":"2023-11-25T04:26:14.717897+0000","last_deep_scrub":"75'220","last_deep_scrub_stamp":"2023-11-25T04:26:14.717897+0000","last_clean_scrub_stamp":"2023-11-25T04:26:14.717897+0000","objects_scrubbed":62,"log_size":220,"log_dups_size":0,"ondisk_log_size":220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:43:17.788148+0000","scrub_duration":0.044136959000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":115501,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":163,"num_write_kb":243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":43,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:04.449342+0000","last_change":"2023-11-25T04:27:04.449359+0000","last_active":"2023-11-25T04:27:04.449342+0000","last_peered":"2023-11-25T04:27:04.449342+0000","last_clean":"2023-11-25T04:27:04.449342+0000","last_became_active":"2023-11-25T04:25:46.997391+0000","last_became_peered":"2023-11-25T04:25:46.997391+0000","last_unstale":"2023-11-25T04:27:04.449342+0000","last_undegraded":"2023-11-25T04:27:04.449342+0000","last_fullsized":"2023-11-25T04:27:04.449342+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:27:04.449261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:27:04.449261+0000","last_clean_scrub_stamp":"2023-11-25T04:27:04.449261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:55:52.982507+0000","scrub_duration":0.00176385,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"50'1","reported_seq":35,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986046+0000","last_change":"2023-11-25T04:25:46.999446+0000","last_active":"2023-11-25T04:26:00.986046+0000","last_peered":"2023-11-25T04:26:00.986046+0000","last_clean":"2023-11-25T04:26:00.986046+0000","last_became_active":"2023-11-25T04:25:46.998132+0000","last_became_peered":"2023-11-25T04:25:46.998132+0000","last_unstale":"2023-11-25T04:26:00.986046+0000","last_undegraded":"2023-11-25T04:26:00.986046+0000","last_fullsized":"2023-11-25T04:26:00.986046+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"75'206","reported_seq":285,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:19.694459+0000","last_change":"2023-11-25T04:26:19.694459+0000","last_active":"2023-11-25T04:26:19.694459+0000","last_peered":"2023-11-25T04:26:19.694459+0000","last_clean":"2023-11-25T04:26:19.694459+0000","last_became_active":"2023-11-25T04:22:47.036924+0000","last_became_peered":"2023-11-25T04:22:47.036924+0000","last_unstale":"2023-11-25T04:26:19.694459+0000","last_undegraded":"2023-11-25T04:26:19.694459+0000","last_fullsized":"2023-11-25T04:26:19.694459+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'206","last_scrub_stamp":"2023-11-25T04:26:19.694341+0000","last_deep_scrub":"75'206","last_deep_scrub_stamp":"2023-11-25T04:26:19.694341+0000","last_clean_scrub_stamp":"2023-11-25T04:26:19.694341+0000","objects_scrubbed":49,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:49:38.202017+0000","scrub_duration":0.031428719000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":93321,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":148,"num_write_kb":360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":35,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986393+0000","last_change":"2023-11-25T04:25:46.999738+0000","last_active":"2023-11-25T04:26:00.986393+0000","last_peered":"2023-11-25T04:26:00.986393+0000","last_clean":"2023-11-25T04:26:00.986393+0000","last_became_active":"2023-11-25T04:25:46.999403+0000","last_became_peered":"2023-11-25T04:25:46.999403+0000","last_unstale":"2023-11-25T04:26:00.986393+0000","last_undegraded":"2023-11-25T04:26:00.986393+0000","last_fullsized":"2023-11-25T04:26:00.986393+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":74,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:09.671106+0000","last_change":"2023-11-25T04:22:42.000204+0000","last_active":"2023-11-25T04:26:09.671106+0000","last_peered":"2023-11-25T04:26:09.671106+0000","last_clean":"2023-11-25T04:26:09.671106+0000","last_became_active":"2023-11-25T04:22:41.999915+0000","last_became_peered":"2023-11-25T04:22:41.999915+0000","last_unstale":"2023-11-25T04:26:09.671106+0000","last_undegraded":"2023-11-25T04:26:09.671106+0000","last_fullsized":"2023-11-25T04:26:09.671106+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"54'152","reported_seq":434,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:09.152189+0000","last_change":"2023-11-25T04:25:53.933239+0000","last_active":"2023-11-25T04:26:09.152189+0000","last_peered":"2023-11-25T04:26:09.152189+0000","last_clean":"2023-11-25T04:26:09.152189+0000","last_became_active":"2023-11-25T04:25:53.932586+0000","last_became_peered":"2023-11-25T04:25:53.932586+0000","last_unstale":"2023-11-25T04:26:09.152189+0000","last_undegraded":"2023-11-25T04:26:09.152189+0000","last_fullsized":"2023-11-25T04:26:09.152189+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":329327,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":151,"num_write_kb":471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":97,"num_bytes_recovered":329278,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":71,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:09.671147+0000","last_change":"2023-11-25T04:22:46.035885+0000","last_active":"2023-11-25T04:26:09.671147+0000","last_peered":"2023-11-25T04:26:09.671147+0000","last_clean":"2023-11-25T04:26:09.671147+0000","last_became_active":"2023-11-25T04:22:46.035405+0000","last_became_peered":"2023-11-25T04:22:46.035405+0000","last_unstale":"2023-11-25T04:26:09.671147+0000","last_undegraded":"2023-11-25T04:26:09.671147+0000","last_fullsized":"2023-11-25T04:26:09.671147+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":82,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:49.454240+0000","last_change":"2023-11-25T04:26:49.454240+0000","last_active":"2023-11-25T04:26:49.454240+0000","last_peered":"2023-11-25T04:26:49.454240+0000","last_clean":"2023-11-25T04:26:49.454240+0000","last_became_active":"2023-11-25T04:22:41.998798+0000","last_became_peered":"2023-11-25T04:22:41.998798+0000","last_unstale":"2023-11-25T04:26:49.454240+0000","last_undegraded":"2023-11-25T04:26:49.454240+0000","last_fullsized":"2023-11-25T04:26:49.454240+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:26:49.454102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:26:49.454102+0000","last_clean_scrub_stamp":"2023-11-25T04:26:49.454102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:49:39.814155+0000","scrub_duration":0.0015686599999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"75'221","reported_seq":530,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:13.479673+0000","last_change":"2023-11-25T04:26:13.479673+0000","last_active":"2023-11-25T04:26:13.479673+0000","last_peered":"2023-11-25T04:26:13.479673+0000","last_clean":"2023-11-25T04:26:13.479673+0000","last_became_active":"2023-11-25T04:25:51.921858+0000","last_became_peered":"2023-11-25T04:25:51.921858+0000","last_unstale":"2023-11-25T04:26:13.479673+0000","last_undegraded":"2023-11-25T04:26:13.479673+0000","last_fullsized":"2023-11-25T04:26:13.479673+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'221","last_scrub_stamp":"2023-11-25T04:26:13.479549+0000","last_deep_scrub":"75'221","last_deep_scrub_stamp":"2023-11-25T04:26:13.479549+0000","last_clean_scrub_stamp":"2023-11-25T04:26:13.479549+0000","objects_scrubbed":49,"log_size":221,"log_dups_size":0,"ondisk_log_size":221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:44:56.142187+0000","scrub_duration":0.035520731999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":129965,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":166,"num_write_kb":290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":129965,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"18'2","reported_seq":73,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:09.671254+0000","last_change":"2023-11-25T04:22:46.034606+0000","last_active":"2023-11-25T04:26:09.671254+0000","last_peered":"2023-11-25T04:26:09.671254+0000","last_clean":"2023-11-25T04:26:09.671254+0000","last_became_active":"2023-11-25T04:22:46.034229+0000","last_became_peered":"2023-11-25T04:22:46.034229+0000","last_unstale":"2023-11-25T04:26:09.671254+0000","last_undegraded":"2023-11-25T04:26:09.671254+0000","last_fullsized":"2023-11-25T04:26:09.671254+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":74,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:09.671112+0000","last_change":"2023-11-25T04:22:42.000258+0000","last_active":"2023-11-25T04:26:09.671112+0000","last_peered":"2023-11-25T04:26:09.671112+0000","last_clean":"2023-11-25T04:26:09.671112+0000","last_became_active":"2023-11-25T04:22:41.999992+0000","last_became_peered":"2023-11-25T04:22:41.999992+0000","last_unstale":"2023-11-25T04:26:09.671112+0000","last_undegraded":"2023-11-25T04:26:09.671112+0000","last_fullsized":"2023-11-25T04:26:09.671112+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:40.987592+0000","last_clean_scrub_stamp":"2023-11-25T04:22:40.987592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"54'150","reported_seq":438,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:09.671351+0000","last_change":"2023-11-25T04:25:54.938479+0000","last_active":"2023-11-25T04:26:09.671351+0000","last_peered":"2023-11-25T04:26:09.671351+0000","last_clean":"2023-11-25T04:26:09.671351+0000","last_became_active":"2023-11-25T04:25:54.937838+0000","last_became_peered":"2023-11-25T04:25:54.937838+0000","last_unstale":"2023-11-25T04:26:09.671351+0000","last_undegraded":"2023-11-25T04:26:09.671351+0000","last_fullsized":"2023-11-25T04:26:09.671351+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":159585,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":154224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"54'14","reported_seq":101,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:09.671169+0000","last_change":"2023-11-25T04:25:47.026749+0000","last_active":"2023-11-25T04:26:09.671169+0000","last_peered":"2023-11-25T04:26:09.671169+0000","last_clean":"2023-11-25T04:26:09.671169+0000","last_became_active":"2023-11-25T04:25:46.997404+0000","last_became_peered":"2023-11-25T04:25:46.997404+0000","last_unstale":"2023-11-25T04:26:09.671169+0000","last_undegraded":"2023-11-25T04:26:09.671169+0000","last_fullsized":"2023-11-25T04:26:09.671169+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":43,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:00.412619+0000","last_change":"2023-11-25T04:27:00.412639+0000","last_active":"2023-11-25T04:27:00.412619+0000","last_peered":"2023-11-25T04:27:00.412619+0000","last_clean":"2023-11-25T04:27:00.412619+0000","last_became_active":"2023-11-25T04:25:46.994834+0000","last_became_peered":"2023-11-25T04:25:46.994834+0000","last_unstale":"2023-11-25T04:27:00.412619+0000","last_undegraded":"2023-11-25T04:27:00.412619+0000","last_fullsized":"2023-11-25T04:27:00.412619+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:27:00.412529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:27:00.412529+0000","last_clean_scrub_stamp":"2023-11-25T04:27:00.412529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:30:28.354258+0000","scrub_duration":0.0016774050000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"75'191","reported_seq":475,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:14.466619+0000","last_change":"2023-11-25T04:26:14.466619+0000","last_active":"2023-11-25T04:26:14.466619+0000","last_peered":"2023-11-25T04:26:14.466619+0000","last_clean":"2023-11-25T04:26:14.466619+0000","last_became_active":"2023-11-25T04:25:49.921465+0000","last_became_peered":"2023-11-25T04:25:49.921465+0000","last_unstale":"2023-11-25T04:26:14.466619+0000","last_undegraded":"2023-11-25T04:26:14.466619+0000","last_fullsized":"2023-11-25T04:26:14.466619+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'191","last_scrub_stamp":"2023-11-25T04:26:14.466515+0000","last_deep_scrub":"75'191","last_deep_scrub_stamp":"2023-11-25T04:26:14.466515+0000","last_clean_scrub_stamp":"2023-11-25T04:26:14.466515+0000","objects_scrubbed":56,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:04:34.752431+0000","scrub_duration":0.037056879000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":99503,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":148,"num_write_kb":250,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":93,"num_bytes_recovered":99503,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"75'218","reported_seq":296,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:59.486464+0000","last_change":"2023-11-25T04:26:59.486489+0000","last_active":"2023-11-25T04:26:59.486464+0000","last_peered":"2023-11-25T04:26:59.486464+0000","last_clean":"2023-11-25T04:26:59.486464+0000","last_became_active":"2023-11-25T04:25:51.922070+0000","last_became_peered":"2023-11-25T04:25:51.922070+0000","last_unstale":"2023-11-25T04:26:59.486464+0000","last_undegraded":"2023-11-25T04:26:59.486464+0000","last_fullsized":"2023-11-25T04:26:59.486464+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'218","last_scrub_stamp":"2023-11-25T04:26:59.486387+0000","last_deep_scrub":"75'218","last_deep_scrub_stamp":"2023-11-25T04:26:59.486387+0000","last_clean_scrub_stamp":"2023-11-25T04:26:59.486387+0000","objects_scrubbed":52,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:32:01.142142+0000","scrub_duration":0.040859969000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":185321,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":172,"num_write_kb":312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":164176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"75'54","reported_seq":66,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:58.474448+0000","last_change":"2023-11-25T04:26:58.474489+0000","last_active":"2023-11-25T04:26:58.474448+0000","last_peered":"2023-11-25T04:26:58.474448+0000","last_clean":"2023-11-25T04:26:58.474448+0000","last_became_active":"2023-11-25T04:25:46.994997+0000","last_became_peered":"2023-11-25T04:25:46.994997+0000","last_unstale":"2023-11-25T04:26:58.474448+0000","last_undegraded":"2023-11-25T04:26:58.474448+0000","last_fullsized":"2023-11-25T04:26:58.474448+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'54","last_scrub_stamp":"2023-11-25T04:26:58.474367+0000","last_deep_scrub":"75'54","last_deep_scrub_stamp":"2023-11-25T04:26:58.474367+0000","last_clean_scrub_stamp":"2023-11-25T04:26:58.474367+0000","objects_scrubbed":4,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:32:12.109149+0000","scrub_duration":0.0152154,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":4119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4190183,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"75'199","reported_seq":277,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:43.858563+0000","last_change":"2023-11-25T04:26:43.858609+0000","last_active":"2023-11-25T04:26:43.858563+0000","last_peered":"2023-11-25T04:26:43.858563+0000","last_clean":"2023-11-25T04:26:43.858563+0000","last_became_active":"2023-11-25T04:22:47.036821+0000","last_became_peered":"2023-11-25T04:22:47.036821+0000","last_unstale":"2023-11-25T04:26:43.858563+0000","last_undegraded":"2023-11-25T04:26:43.858563+0000","last_fullsized":"2023-11-25T04:26:43.858563+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'199","last_scrub_stamp":"2023-11-25T04:26:43.858473+0000","last_deep_scrub":"75'199","last_deep_scrub_stamp":"2023-11-25T04:26:43.858473+0000","last_clean_scrub_stamp":"2023-11-25T04:26:43.858473+0000","objects_scrubbed":46,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:28:32.758038+0000","scrub_duration":0.029966083000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":101117,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":161,"num_write_kb":242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"75'24","reported_seq":118,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:39.535607+0000","last_change":"2023-11-25T04:26:39.535658+0000","last_active":"2023-11-25T04:26:39.535607+0000","last_peered":"2023-11-25T04:26:39.535607+0000","last_clean":"2023-11-25T04:26:39.535607+0000","last_became_active":"2023-11-25T04:25:47.000478+0000","last_became_peered":"2023-11-25T04:25:47.000478+0000","last_unstale":"2023-11-25T04:26:39.535607+0000","last_undegraded":"2023-11-25T04:26:39.535607+0000","last_fullsized":"2023-11-25T04:26:39.535607+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'24","last_scrub_stamp":"2023-11-25T04:26:39.535504+0000","last_deep_scrub":"75'24","last_deep_scrub_stamp":"2023-11-25T04:26:39.535504+0000","last_clean_scrub_stamp":"2023-11-25T04:26:39.535504+0000","objects_scrubbed":2,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:50:41.041415+0000","scrub_duration":0.0040188020000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":90,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":43,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:54.349677+0000","last_change":"2023-11-25T04:26:54.349711+0000","last_active":"2023-11-25T04:26:54.349677+0000","last_peered":"2023-11-25T04:26:54.349677+0000","last_clean":"2023-11-25T04:26:54.349677+0000","last_became_active":"2023-11-25T04:25:46.994525+0000","last_became_peered":"2023-11-25T04:25:46.994525+0000","last_unstale":"2023-11-25T04:26:54.349677+0000","last_undegraded":"2023-11-25T04:26:54.349677+0000","last_fullsized":"2023-11-25T04:26:54.349677+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:26:54.349592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:26:54.349592+0000","last_clean_scrub_stamp":"2023-11-25T04:26:54.349592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:34:14.985604+0000","scrub_duration":0.0018546229999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"54'177","reported_seq":486,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.985858+0000","last_change":"2023-11-25T04:25:51.922466+0000","last_active":"2023-11-25T04:26:00.985858+0000","last_peered":"2023-11-25T04:26:00.985858+0000","last_clean":"2023-11-25T04:26:00.985858+0000","last_became_active":"2023-11-25T04:25:51.921938+0000","last_became_peered":"2023-11-25T04:25:51.921938+0000","last_unstale":"2023-11-25T04:26:00.985858+0000","last_undegraded":"2023-11-25T04:26:00.985858+0000","last_fullsized":"2023-11-25T04:26:00.985858+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":522494,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":168,"num_write_kb":749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":111,"num_bytes_recovered":522494,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"46'2","reported_seq":91,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:51.691510+0000","last_change":"2023-11-25T04:26:51.691533+0000","last_active":"2023-11-25T04:26:51.691510+0000","last_peered":"2023-11-25T04:26:51.691510+0000","last_clean":"2023-11-25T04:26:51.691510+0000","last_became_active":"2023-11-25T04:25:46.998508+0000","last_became_peered":"2023-11-25T04:25:46.998508+0000","last_unstale":"2023-11-25T04:26:51.691510+0000","last_undegraded":"2023-11-25T04:26:51.691510+0000","last_fullsized":"2023-11-25T04:26:51.691510+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"46'2","last_scrub_stamp":"2023-11-25T04:26:51.691391+0000","last_deep_scrub":"46'2","last_deep_scrub_stamp":"2023-11-25T04:26:51.691391+0000","last_clean_scrub_stamp":"2023-11-25T04:26:51.691391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:31:13.002647+0000","scrub_duration":0.0021417300000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.10","version":"75'209","reported_seq":288,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:02.586465+0000","last_change":"2023-11-25T04:27:02.586496+0000","last_active":"2023-11-25T04:27:02.586465+0000","last_peered":"2023-11-25T04:27:02.586465+0000","last_clean":"2023-11-25T04:27:02.586465+0000","last_became_active":"2023-11-25T04:22:47.037149+0000","last_became_peered":"2023-11-25T04:22:47.037149+0000","last_unstale":"2023-11-25T04:27:02.586465+0000","last_undegraded":"2023-11-25T04:27:02.586465+0000","last_fullsized":"2023-11-25T04:27:02.586465+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'209","last_scrub_stamp":"2023-11-25T04:27:02.586379+0000","last_deep_scrub":"75'209","last_deep_scrub_stamp":"2023-11-25T04:27:02.586379+0000","last_clean_scrub_stamp":"2023-11-25T04:27:02.586379+0000","objects_scrubbed":56,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:55:23.713881+0000","scrub_duration":0.037826797000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":218073,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1f","version":"54'156","reported_seq":450,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:09.152342+0000","last_change":"2023-11-25T04:25:49.921904+0000","last_active":"2023-11-25T04:26:09.152342+0000","last_peered":"2023-11-25T04:26:09.152342+0000","last_clean":"2023-11-25T04:26:09.152342+0000","last_became_active":"2023-11-25T04:25:49.921458+0000","last_became_peered":"2023-11-25T04:25:49.921458+0000","last_unstale":"2023-11-25T04:26:09.152342+0000","last_undegraded":"2023-11-25T04:26:09.152342+0000","last_fullsized":"2023-11-25T04:26:09.152342+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":107712,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":103,"num_bytes_recovered":107703,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"48'1","reported_seq":71,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986022+0000","last_change":"2023-11-25T04:22:46.036085+0000","last_active":"2023-11-25T04:26:00.986022+0000","last_peered":"2023-11-25T04:26:00.986022+0000","last_clean":"2023-11-25T04:26:00.986022+0000","last_became_active":"2023-11-25T04:22:46.035495+0000","last_became_peered":"2023-11-25T04:22:46.035495+0000","last_unstale":"2023-11-25T04:26:00.986022+0000","last_undegraded":"2023-11-25T04:26:00.986022+0000","last_fullsized":"2023-11-25T04:26:00.986022+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"54'152","reported_seq":432,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:09.671534+0000","last_change":"2023-11-25T04:25:50.923882+0000","last_active":"2023-11-25T04:26:09.671534+0000","last_peered":"2023-11-25T04:26:09.671534+0000","last_clean":"2023-11-25T04:26:09.671534+0000","last_became_active":"2023-11-25T04:25:50.923375+0000","last_became_peered":"2023-11-25T04:25:50.923375+0000","last_unstale":"2023-11-25T04:26:09.671534+0000","last_undegraded":"2023-11-25T04:26:09.671534+0000","last_fullsized":"2023-11-25T04:26:09.671534+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":76435,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":76435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":89,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:23.683261+0000","last_change":"2023-11-25T04:26:23.683288+0000","last_active":"2023-11-25T04:26:23.683261+0000","last_peered":"2023-11-25T04:26:23.683261+0000","last_clean":"2023-11-25T04:26:23.683261+0000","last_became_active":"2023-11-25T04:25:46.994871+0000","last_became_peered":"2023-11-25T04:25:46.994871+0000","last_unstale":"2023-11-25T04:26:23.683261+0000","last_undegraded":"2023-11-25T04:26:23.683261+0000","last_fullsized":"2023-11-25T04:26:23.683261+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:26:23.683176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:26:23.683176+0000","last_clean_scrub_stamp":"2023-11-25T04:26:23.683176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:31:54.450606+0000","scrub_duration":0.00162587,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"54'158","reported_seq":433,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986512+0000","last_change":"2023-11-25T04:25:52.925523+0000","last_active":"2023-11-25T04:26:00.986512+0000","last_peered":"2023-11-25T04:26:00.986512+0000","last_clean":"2023-11-25T04:26:00.986512+0000","last_became_active":"2023-11-25T04:25:52.925038+0000","last_became_peered":"2023-11-25T04:25:52.925038+0000","last_unstale":"2023-11-25T04:26:00.986512+0000","last_undegraded":"2023-11-25T04:26:00.986512+0000","last_fullsized":"2023-11-25T04:26:00.986512+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":209500,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":139,"num_write_kb":351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":209500,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1a","version":"75'5","reported_seq":83,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:51.508201+0000","last_change":"2023-11-25T04:26:51.508225+0000","last_active":"2023-11-25T04:26:51.508201+0000","last_peered":"2023-11-25T04:26:51.508201+0000","last_clean":"2023-11-25T04:26:51.508201+0000","last_became_active":"2023-11-25T04:22:46.033944+0000","last_became_peered":"2023-11-25T04:22:46.033944+0000","last_unstale":"2023-11-25T04:26:51.508201+0000","last_undegraded":"2023-11-25T04:26:51.508201+0000","last_fullsized":"2023-11-25T04:26:51.508201+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'5","last_scrub_stamp":"2023-11-25T04:26:51.508119+0000","last_deep_scrub":"75'5","last_deep_scrub_stamp":"2023-11-25T04:26:51.508119+0000","last_clean_scrub_stamp":"2023-11-25T04:26:51.508119+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:55:12.635621+0000","scrub_duration":0.0083650619999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"75'218","reported_seq":297,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:28.540048+0000","last_change":"2023-11-25T04:26:28.540082+0000","last_active":"2023-11-25T04:26:28.540048+0000","last_peered":"2023-11-25T04:26:28.540048+0000","last_clean":"2023-11-25T04:26:28.540048+0000","last_became_active":"2023-11-25T04:22:47.039218+0000","last_became_peered":"2023-11-25T04:22:47.039218+0000","last_unstale":"2023-11-25T04:26:28.540048+0000","last_undegraded":"2023-11-25T04:26:28.540048+0000","last_fullsized":"2023-11-25T04:26:28.540048+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'218","last_scrub_stamp":"2023-11-25T04:26:28.539970+0000","last_deep_scrub":"75'218","last_deep_scrub_stamp":"2023-11-25T04:26:28.539970+0000","last_clean_scrub_stamp":"2023-11-25T04:26:28.539970+0000","objects_scrubbed":55,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:46:27.913704+0000","scrub_duration":0.040559401000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":165883,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"75'10","reported_seq":104,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:20.499200+0000","last_change":"2023-11-25T04:26:20.499236+0000","last_active":"2023-11-25T04:26:20.499200+0000","last_peered":"2023-11-25T04:26:20.499200+0000","last_clean":"2023-11-25T04:26:20.499200+0000","last_became_active":"2023-11-25T04:25:46.999998+0000","last_became_peered":"2023-11-25T04:25:46.999998+0000","last_unstale":"2023-11-25T04:26:20.499200+0000","last_undegraded":"2023-11-25T04:26:20.499200+0000","last_fullsized":"2023-11-25T04:26:20.499200+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'10","last_scrub_stamp":"2023-11-25T04:26:20.499093+0000","last_deep_scrub":"75'10","last_deep_scrub_stamp":"2023-11-25T04:26:20.499093+0000","last_clean_scrub_stamp":"2023-11-25T04:26:20.499093+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:35:51.569332+0000","scrub_duration":0.0094161829999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":4101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4194304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"75'212","reported_seq":291,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:39.477807+0000","last_change":"2023-11-25T04:26:39.477840+0000","last_active":"2023-11-25T04:26:39.477807+0000","last_peered":"2023-11-25T04:26:39.477807+0000","last_clean":"2023-11-25T04:26:39.477807+0000","last_became_active":"2023-11-25T04:25:50.922164+0000","last_became_peered":"2023-11-25T04:25:50.922164+0000","last_unstale":"2023-11-25T04:26:39.477807+0000","last_undegraded":"2023-11-25T04:26:39.477807+0000","last_fullsized":"2023-11-25T04:26:39.477807+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'212","last_scrub_stamp":"2023-11-25T04:26:39.477723+0000","last_deep_scrub":"75'212","last_deep_scrub_stamp":"2023-11-25T04:26:39.477723+0000","last_clean_scrub_stamp":"2023-11-25T04:26:39.477723+0000","objects_scrubbed":63,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:34:17.609339+0000","scrub_duration":0.044092971000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":179506,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":155,"num_write_kb":317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":179506,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"18'2","reported_seq":38,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986217+0000","last_change":"2023-11-25T04:25:46.996675+0000","last_active":"2023-11-25T04:26:00.986217+0000","last_peered":"2023-11-25T04:26:00.986217+0000","last_clean":"2023-11-25T04:26:00.986217+0000","last_became_active":"2023-11-25T04:25:46.995160+0000","last_became_peered":"2023-11-25T04:25:46.995160+0000","last_unstale":"2023-11-25T04:26:00.986217+0000","last_undegraded":"2023-11-25T04:26:00.986217+0000","last_fullsized":"2023-11-25T04:26:00.986217+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1a","version":"54'162","reported_seq":227,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986038+0000","last_change":"2023-11-25T04:25:54.939182+0000","last_active":"2023-11-25T04:26:00.986038+0000","last_peered":"2023-11-25T04:26:00.986038+0000","last_clean":"2023-11-25T04:26:00.986038+0000","last_became_active":"2023-11-25T04:25:54.937821+0000","last_became_peered":"2023-11-25T04:25:54.937821+0000","last_unstale":"2023-11-25T04:26:00.986038+0000","last_undegraded":"2023-11-25T04:26:00.986038+0000","last_fullsized":"2023-11-25T04:26:00.986038+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":134672,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":169,"num_write_kb":311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":126291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"75'62","reported_seq":158,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:30.554569+0000","last_change":"2023-11-25T04:26:30.554596+0000","last_active":"2023-11-25T04:26:30.554569+0000","last_peered":"2023-11-25T04:26:30.554569+0000","last_clean":"2023-11-25T04:26:30.554569+0000","last_became_active":"2023-11-25T04:25:47.000722+0000","last_became_peered":"2023-11-25T04:25:47.000722+0000","last_unstale":"2023-11-25T04:26:30.554569+0000","last_undegraded":"2023-11-25T04:26:30.554569+0000","last_fullsized":"2023-11-25T04:26:30.554569+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'62","last_scrub_stamp":"2023-11-25T04:26:30.554482+0000","last_deep_scrub":"75'62","last_deep_scrub_stamp":"2023-11-25T04:26:30.554482+0000","last_clean_scrub_stamp":"2023-11-25T04:26:30.554482+0000","objects_scrubbed":3,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:38:15.140403+0000","scrub_duration":0.015921703999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8388608,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"75'221","reported_seq":300,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:55.521285+0000","last_change":"2023-11-25T04:26:55.521345+0000","last_active":"2023-11-25T04:26:55.521285+0000","last_peered":"2023-11-25T04:26:55.521285+0000","last_clean":"2023-11-25T04:26:55.521285+0000","last_became_active":"2023-11-25T04:22:47.036406+0000","last_became_peered":"2023-11-25T04:22:47.036406+0000","last_unstale":"2023-11-25T04:26:55.521285+0000","last_undegraded":"2023-11-25T04:26:55.521285+0000","last_fullsized":"2023-11-25T04:26:55.521285+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'221","last_scrub_stamp":"2023-11-25T04:26:55.521198+0000","last_deep_scrub":"75'221","last_deep_scrub_stamp":"2023-11-25T04:26:55.521198+0000","last_clean_scrub_stamp":"2023-11-25T04:26:55.521198+0000","objects_scrubbed":58,"log_size":221,"log_dups_size":0,"ondisk_log_size":221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:17:51.455919+0000","scrub_duration":0.041323743000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":527891,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":163,"num_write_kb":687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1e","version":"46'2","reported_seq":73,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:09.671303+0000","last_change":"2023-11-25T04:22:46.035659+0000","last_active":"2023-11-25T04:26:09.671303+0000","last_peered":"2023-11-25T04:26:09.671303+0000","last_clean":"2023-11-25T04:26:09.671303+0000","last_became_active":"2023-11-25T04:22:46.035113+0000","last_became_peered":"2023-11-25T04:22:46.035113+0000","last_unstale":"2023-11-25T04:26:09.671303+0000","last_undegraded":"2023-11-25T04:26:09.671303+0000","last_fullsized":"2023-11-25T04:26:09.671303+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.18","version":"54'195","reported_seq":260,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986052+0000","last_change":"2023-11-25T04:25:54.938573+0000","last_active":"2023-11-25T04:26:00.986052+0000","last_peered":"2023-11-25T04:26:00.986052+0000","last_clean":"2023-11-25T04:26:00.986052+0000","last_became_active":"2023-11-25T04:25:54.937444+0000","last_became_peered":"2023-11-25T04:25:54.937444+0000","last_unstale":"2023-11-25T04:26:00.986052+0000","last_undegraded":"2023-11-25T04:26:00.986052+0000","last_fullsized":"2023-11-25T04:26:00.986052+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":128641,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":185,"num_write_kb":408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":122,"num_bytes_recovered":128641,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"75'20","reported_seq":115,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:49.762359+0000","last_change":"2023-11-25T04:26:49.762359+0000","last_active":"2023-11-25T04:26:49.762359+0000","last_peered":"2023-11-25T04:26:49.762359+0000","last_clean":"2023-11-25T04:26:49.762359+0000","last_became_active":"2023-11-25T04:25:46.998870+0000","last_became_peered":"2023-11-25T04:25:46.998870+0000","last_unstale":"2023-11-25T04:26:49.762359+0000","last_undegraded":"2023-11-25T04:26:49.762359+0000","last_fullsized":"2023-11-25T04:26:49.762359+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'20","last_scrub_stamp":"2023-11-25T04:26:49.762253+0000","last_deep_scrub":"75'20","last_deep_scrub_stamp":"2023-11-25T04:26:49.762253+0000","last_clean_scrub_stamp":"2023-11-25T04:26:49.762253+0000","objects_scrubbed":3,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:38:21.361584+0000","scrub_duration":0.0095441299999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":4103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4194304,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":26330855,"num_objects":3447,"num_object_clones":0,"num_object_copies":6894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10423,"num_write_kb":38550,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4189,"num_bytes_recovered":12749181,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":72376320,"data_stored":52324046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12485,"ondisk_log_size":12485,"up":128,"acting":128,"num_store_stats":3},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":40885548,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":35,"num_read_kb":334,"num_write":435,"num_write_kb":40462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":32491439,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":81833984,"data_stored":81771096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5548,"internal_metadata":0},"log_size":459,"ondisk_log_size":459,"up":64,"acting":64,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738428,"num_pgs":72,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81524,"kb_used_data":46188,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94290316,"statfs":{"total":96636764160,"available":96553283584,"internally_reserved":0,"allocated":47296512,"data_stored":39787071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 25 04:26:49 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.435},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.495}]},{"osd":1,"last update":"Sat Nov 25 04:26:45 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.436,"15min":0.44600000000000001},"min":{"1min":0.23300000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.61299999999999999,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.437,"15min":0.45100000000000001},"min":{"1min":0.249,"5min":0.184,"15min":0.184},"max":{"1min":0.63,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.45300000000000001}]}]},{"osd":1,"up_from":8,"seq":34359738428,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":95196,"kb_used_data":55700,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94276644,"statfs":{"total":96636764160,"available":96539283456,"internally_reserved":0,"allocated":57036800,"data_stored":50261554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 25 04:26:46 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.372},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.38300000000000001}]},{"osd":2,"last update":"Sat Nov 25 04:26:42 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.41999999999999998,"15min":0.41599999999999998},"min":{"1min":0.28299999999999997,"5min":0.182,"15min":0.182},"max":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.41899999999999998,"15min":0.41099999999999998},"min":{"1min":0.27000000000000002,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.54700000000000004}]}]},{"osd":0,"up_from":60,"seq":257698037778,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":91772,"kb_used_data":51764,"kb_used_omap":6,"kb_used_meta":39993,"kb_avail":94280068,"statfs":{"total":96636764160,"available":96542789632,"internally_reserved":0,"allocated":53006336,"data_stored":46141107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7143,"internal_metadata":40952857},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Nov 25 04:26:51 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.503},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.47499999999999998}]},{"osd":2,"last update":"Sat Nov 25 04:26:51 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.40300000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32530432,"data_stored":32496940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5548,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":29380608,"data_stored":29360982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":19922944,"data_stored":19913174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":19234816,"data_stored":12749181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":26415104,"data_stored":20005586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":26726400,"data_stored":19569279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-25T04:27:11.273 INFO:tasks.ceph:pgid 4.3a last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:27:11.273 INFO:tasks.ceph:pgid 4.34 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:27:11.273 INFO:tasks.ceph:pgid 4.32 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:27:11.273 INFO:tasks.ceph:pgid 4.2a last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:27:11.274 INFO:tasks.ceph:pgid 4.27 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:27:11.274 INFO:tasks.ceph:pgid 4.26 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:27:11.274 INFO:tasks.ceph:pgid 4.22 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:27:11.274 INFO:tasks.ceph:pgid 4.20 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:27:11.274 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:27:11.275 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:27:11.275 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:27:11.275 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:27:11.275 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:27:11.275 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:27:11.276 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:27:11.276 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-11-25T04:22:35.919850+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=35, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:27:11.276 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-11-25T04:22:40.987592+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=40, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:27:11.276 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:27:11.276 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:27:11.277 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-11-25T04:22:40.987592+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=40, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:27:11.277 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:27:11.277 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:27:11.277 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-11-25T04:22:40.987592+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=40, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:27:11.277 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:27:11.277 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:27:11.278 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:27:11.278 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-11-25T04:22:40.987592+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=40, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:27:11.278 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:27:11.278 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:27:11.278 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:27:11.279 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:27:11.279 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:27:11.279 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:27:11.279 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:27:11.279 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:27:11.280 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:27:11.280 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:27:11.280 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:27:11.280 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-11-25T04:27:31.281 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-25T04:27:31.593 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:27:31.594 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2023-11-25T04:27:31.604 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":222,"stamp":"2023-11-25T04:27:31.572753+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":67806771,"num_objects":3481,"num_object_clones":0,"num_object_copies":6962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3481,"num_whiteouts":0,"num_read":171,"num_read_kb":452,"num_write":11061,"num_write_kb":82472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4201,"num_bytes_recovered":45240620,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13539,"ondisk_log_size":13539,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":268492,"kb_used_data":153652,"kb_used_omap":10,"kb_used_meta":114805,"kb_avail":282847028,"statfs":{"total":289910292480,"available":289635356672,"internally_reserved":0,"allocated":157339648,"data_stored":136189732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10333,"internal_metadata":117561251},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003225"},"pg_stats":[{"pgid":"4.3f","version":"75'212","reported_seq":291,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:48.487625+0000","last_change":"2023-11-25T04:26:48.487625+0000","last_active":"2023-11-25T04:26:48.487625+0000","last_peered":"2023-11-25T04:26:48.487625+0000","last_clean":"2023-11-25T04:26:48.487625+0000","last_became_active":"2023-11-25T04:22:47.039490+0000","last_became_peered":"2023-11-25T04:22:47.039490+0000","last_unstale":"2023-11-25T04:26:48.487625+0000","last_undegraded":"2023-11-25T04:26:48.487625+0000","last_fullsized":"2023-11-25T04:26:48.487625+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'212","last_scrub_stamp":"2023-11-25T04:26:48.487485+0000","last_deep_scrub":"75'212","last_deep_scrub_stamp":"2023-11-25T04:26:48.487485+0000","last_clean_scrub_stamp":"2023-11-25T04:26:48.487485+0000","objects_scrubbed":46,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:02:55.160351+0000","scrub_duration":0.031121083000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":577866,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":174,"num_write_kb":756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3e","version":"75'257","reported_seq":336,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:16.435276+0000","last_change":"2023-11-25T04:26:16.435276+0000","last_active":"2023-11-25T04:26:16.435276+0000","last_peered":"2023-11-25T04:26:16.435276+0000","last_clean":"2023-11-25T04:26:16.435276+0000","last_became_active":"2023-11-25T04:22:47.040511+0000","last_became_peered":"2023-11-25T04:22:47.040511+0000","last_unstale":"2023-11-25T04:26:16.435276+0000","last_undegraded":"2023-11-25T04:26:16.435276+0000","last_fullsized":"2023-11-25T04:26:16.435276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'257","last_scrub_stamp":"2023-11-25T04:26:16.435161+0000","last_deep_scrub":"75'257","last_deep_scrub_stamp":"2023-11-25T04:26:16.435161+0000","last_clean_scrub_stamp":"2023-11-25T04:26:16.435161+0000","objects_scrubbed":45,"log_size":257,"log_dups_size":0,"ondisk_log_size":257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:13:07.641904+0000","scrub_duration":0.036488224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":461579,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":236,"num_write_kb":841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3d","version":"75'209","reported_seq":285,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:12.330753+0000","last_change":"2023-11-25T04:26:12.330801+0000","last_active":"2023-11-25T04:26:12.330753+0000","last_peered":"2023-11-25T04:26:12.330753+0000","last_clean":"2023-11-25T04:26:12.330753+0000","last_became_active":"2023-11-25T04:25:52.923459+0000","last_became_peered":"2023-11-25T04:25:52.923459+0000","last_unstale":"2023-11-25T04:26:12.330753+0000","last_undegraded":"2023-11-25T04:26:12.330753+0000","last_fullsized":"2023-11-25T04:26:12.330753+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'209","last_scrub_stamp":"2023-11-25T04:26:12.330672+0000","last_deep_scrub":"75'209","last_deep_scrub_stamp":"2023-11-25T04:26:12.330672+0000","last_clean_scrub_stamp":"2023-11-25T04:26:12.330672+0000","objects_scrubbed":42,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:49:33.381569+0000","scrub_duration":0.027938688999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":145844,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":175,"num_write_kb":312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":138765,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"75'172","reported_seq":249,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:57.448967+0000","last_change":"2023-11-25T04:26:57.449017+0000","last_active":"2023-11-25T04:26:57.448967+0000","last_peered":"2023-11-25T04:26:57.448967+0000","last_clean":"2023-11-25T04:26:57.448967+0000","last_became_active":"2023-11-25T04:25:51.921633+0000","last_became_peered":"2023-11-25T04:25:51.921633+0000","last_unstale":"2023-11-25T04:26:57.448967+0000","last_undegraded":"2023-11-25T04:26:57.448967+0000","last_fullsized":"2023-11-25T04:26:57.448967+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'172","last_scrub_stamp":"2023-11-25T04:26:57.448886+0000","last_deep_scrub":"75'172","last_deep_scrub_stamp":"2023-11-25T04:26:57.448886+0000","last_clean_scrub_stamp":"2023-11-25T04:26:57.448886+0000","objects_scrubbed":48,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:58:16.336649+0000","scrub_duration":0.030012414000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61303,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119,"num_write_kb":161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":61303,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"75'236","reported_seq":315,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:03.442942+0000","last_change":"2023-11-25T04:27:03.442984+0000","last_active":"2023-11-25T04:27:03.442942+0000","last_peered":"2023-11-25T04:27:03.442942+0000","last_clean":"2023-11-25T04:27:03.442942+0000","last_became_active":"2023-11-25T04:22:47.039165+0000","last_became_peered":"2023-11-25T04:22:47.039165+0000","last_unstale":"2023-11-25T04:27:03.442942+0000","last_undegraded":"2023-11-25T04:27:03.442942+0000","last_fullsized":"2023-11-25T04:27:03.442942+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'236","last_scrub_stamp":"2023-11-25T04:27:03.442858+0000","last_deep_scrub":"75'236","last_deep_scrub_stamp":"2023-11-25T04:27:03.442858+0000","last_clean_scrub_stamp":"2023-11-25T04:27:03.442858+0000","objects_scrubbed":67,"log_size":236,"log_dups_size":0,"ondisk_log_size":236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:08:36.083041+0000","scrub_duration":0.043065032000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":118227,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":185,"num_write_kb":351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3a","version":"54'169","reported_seq":235,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986335+0000","last_change":"2023-11-25T04:25:53.933198+0000","last_active":"2023-11-25T04:26:00.986335+0000","last_peered":"2023-11-25T04:26:00.986335+0000","last_clean":"2023-11-25T04:26:00.986335+0000","last_became_active":"2023-11-25T04:25:53.931982+0000","last_became_peered":"2023-11-25T04:25:53.931982+0000","last_unstale":"2023-11-25T04:26:00.986335+0000","last_undegraded":"2023-11-25T04:26:00.986335+0000","last_fullsized":"2023-11-25T04:26:00.986335+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100304,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":161,"num_write_kb":424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":100304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"75'228","reported_seq":308,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:27.488226+0000","last_change":"2023-11-25T04:26:27.488269+0000","last_active":"2023-11-25T04:26:27.488226+0000","last_peered":"2023-11-25T04:26:27.488226+0000","last_clean":"2023-11-25T04:26:27.488226+0000","last_became_active":"2023-11-25T04:25:49.919354+0000","last_became_peered":"2023-11-25T04:25:49.919354+0000","last_unstale":"2023-11-25T04:26:27.488226+0000","last_undegraded":"2023-11-25T04:26:27.488226+0000","last_fullsized":"2023-11-25T04:26:27.488226+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'228","last_scrub_stamp":"2023-11-25T04:26:27.488145+0000","last_deep_scrub":"75'228","last_deep_scrub_stamp":"2023-11-25T04:26:27.488145+0000","last_clean_scrub_stamp":"2023-11-25T04:26:27.488145+0000","objects_scrubbed":60,"log_size":228,"log_dups_size":0,"ondisk_log_size":228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:22:24.257272+0000","scrub_duration":0.042766632999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":124558,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":167,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":111,"num_bytes_recovered":124533,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.38","version":"75'199","reported_seq":278,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:28.501531+0000","last_change":"2023-11-25T04:26:28.501582+0000","last_active":"2023-11-25T04:26:28.501531+0000","last_peered":"2023-11-25T04:26:28.501531+0000","last_clean":"2023-11-25T04:26:28.501531+0000","last_became_active":"2023-11-25T04:25:49.919236+0000","last_became_peered":"2023-11-25T04:25:49.919236+0000","last_unstale":"2023-11-25T04:26:28.501531+0000","last_undegraded":"2023-11-25T04:26:28.501531+0000","last_fullsized":"2023-11-25T04:26:28.501531+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'199","last_scrub_stamp":"2023-11-25T04:26:28.501441+0000","last_deep_scrub":"75'199","last_deep_scrub_stamp":"2023-11-25T04:26:28.501441+0000","last_clean_scrub_stamp":"2023-11-25T04:26:28.501441+0000","objects_scrubbed":47,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:21:13.937490+0000","scrub_duration":0.030866581000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":120147,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":120121,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.37","version":"75'220","reported_seq":299,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:28.772505+0000","last_change":"2023-11-25T04:26:28.772532+0000","last_active":"2023-11-25T04:26:28.772505+0000","last_peered":"2023-11-25T04:26:28.772505+0000","last_clean":"2023-11-25T04:26:28.772505+0000","last_became_active":"2023-11-25T04:22:47.037328+0000","last_became_peered":"2023-11-25T04:22:47.037328+0000","last_unstale":"2023-11-25T04:26:28.772505+0000","last_undegraded":"2023-11-25T04:26:28.772505+0000","last_fullsized":"2023-11-25T04:26:28.772505+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'220","last_scrub_stamp":"2023-11-25T04:26:28.772426+0000","last_deep_scrub":"75'220","last_deep_scrub_stamp":"2023-11-25T04:26:28.772426+0000","last_clean_scrub_stamp":"2023-11-25T04:26:28.772426+0000","objects_scrubbed":53,"log_size":220,"log_dups_size":0,"ondisk_log_size":220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:08:21.318788+0000","scrub_duration":0.036194551999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":174147,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":184,"num_write_kb":353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.36","version":"75'200","reported_seq":279,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:24.608814+0000","last_change":"2023-11-25T04:26:24.608840+0000","last_active":"2023-11-25T04:26:24.608814+0000","last_peered":"2023-11-25T04:26:24.608814+0000","last_clean":"2023-11-25T04:26:24.608814+0000","last_became_active":"2023-11-25T04:22:47.039892+0000","last_became_peered":"2023-11-25T04:22:47.039892+0000","last_unstale":"2023-11-25T04:26:24.608814+0000","last_undegraded":"2023-11-25T04:26:24.608814+0000","last_fullsized":"2023-11-25T04:26:24.608814+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'200","last_scrub_stamp":"2023-11-25T04:26:24.608737+0000","last_deep_scrub":"75'200","last_deep_scrub_stamp":"2023-11-25T04:26:24.608737+0000","last_clean_scrub_stamp":"2023-11-25T04:26:24.608737+0000","objects_scrubbed":51,"log_size":200,"log_dups_size":0,"ondisk_log_size":200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:49:43.116414+0000","scrub_duration":0.03530929,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":233330,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"75'211","reported_seq":289,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:25.542014+0000","last_change":"2023-11-25T04:26:25.542044+0000","last_active":"2023-11-25T04:26:25.542014+0000","last_peered":"2023-11-25T04:26:25.542014+0000","last_clean":"2023-11-25T04:26:25.542014+0000","last_became_active":"2023-11-25T04:25:51.921800+0000","last_became_peered":"2023-11-25T04:25:51.921800+0000","last_unstale":"2023-11-25T04:26:25.542014+0000","last_undegraded":"2023-11-25T04:26:25.542014+0000","last_fullsized":"2023-11-25T04:26:25.542014+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'211","last_scrub_stamp":"2023-11-25T04:26:25.541938+0000","last_deep_scrub":"75'211","last_deep_scrub_stamp":"2023-11-25T04:26:25.541938+0000","last_clean_scrub_stamp":"2023-11-25T04:26:25.541938+0000","objects_scrubbed":55,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:32:27.981332+0000","scrub_duration":0.043489338000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":302550,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":163,"num_write_kb":431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":300323,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.34","version":"75'186","reported_seq":461,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:12.416174+0000","last_change":"2023-11-25T04:27:12.416198+0000","last_active":"2023-11-25T04:27:12.416174+0000","last_peered":"2023-11-25T04:27:12.416174+0000","last_clean":"2023-11-25T04:27:12.416174+0000","last_became_active":"2023-11-25T04:25:50.922309+0000","last_became_peered":"2023-11-25T04:25:50.922309+0000","last_unstale":"2023-11-25T04:27:12.416174+0000","last_undegraded":"2023-11-25T04:27:12.416174+0000","last_fullsized":"2023-11-25T04:27:12.416174+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'186","last_scrub_stamp":"2023-11-25T04:27:12.416101+0000","last_deep_scrub":"75'186","last_deep_scrub_stamp":"2023-11-25T04:27:12.416101+0000","last_clean_scrub_stamp":"2023-11-25T04:27:12.416101+0000","objects_scrubbed":50,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:42:35.547259+0000","scrub_duration":0.040358303999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":93053,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":131,"num_write_kb":248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":89,"num_bytes_recovered":93053,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"75'183","reported_seq":261,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:19.516152+0000","last_change":"2023-11-25T04:26:19.516218+0000","last_active":"2023-11-25T04:26:19.516152+0000","last_peered":"2023-11-25T04:26:19.516152+0000","last_clean":"2023-11-25T04:26:19.516152+0000","last_became_active":"2023-11-25T04:22:47.040129+0000","last_became_peered":"2023-11-25T04:22:47.040129+0000","last_unstale":"2023-11-25T04:26:19.516152+0000","last_undegraded":"2023-11-25T04:26:19.516152+0000","last_fullsized":"2023-11-25T04:26:19.516152+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'183","last_scrub_stamp":"2023-11-25T04:26:19.516065+0000","last_deep_scrub":"75'183","last_deep_scrub_stamp":"2023-11-25T04:26:19.516065+0000","last_clean_scrub_stamp":"2023-11-25T04:26:19.516065+0000","objects_scrubbed":49,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:43:33.021008+0000","scrub_duration":0.038467327000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":149524,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":139,"num_write_kb":252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.32","version":"54'174","reported_seq":240,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986344+0000","last_change":"2023-11-25T04:25:53.934526+0000","last_active":"2023-11-25T04:26:00.986344+0000","last_peered":"2023-11-25T04:26:00.986344+0000","last_clean":"2023-11-25T04:26:00.986344+0000","last_became_active":"2023-11-25T04:25:53.933694+0000","last_became_peered":"2023-11-25T04:25:53.933694+0000","last_unstale":"2023-11-25T04:26:00.986344+0000","last_undegraded":"2023-11-25T04:26:00.986344+0000","last_fullsized":"2023-11-25T04:26:00.986344+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":95797,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":181,"num_write_kb":547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":103,"num_bytes_recovered":95797,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.31","version":"75'212","reported_seq":292,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:29.518025+0000","last_change":"2023-11-25T04:26:29.518088+0000","last_active":"2023-11-25T04:26:29.518025+0000","last_peered":"2023-11-25T04:26:29.518025+0000","last_clean":"2023-11-25T04:26:29.518025+0000","last_became_active":"2023-11-25T04:25:49.921129+0000","last_became_peered":"2023-11-25T04:25:49.921129+0000","last_unstale":"2023-11-25T04:26:29.518025+0000","last_undegraded":"2023-11-25T04:26:29.518025+0000","last_fullsized":"2023-11-25T04:26:29.518025+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'212","last_scrub_stamp":"2023-11-25T04:26:29.517931+0000","last_deep_scrub":"75'212","last_deep_scrub_stamp":"2023-11-25T04:26:29.517931+0000","last_clean_scrub_stamp":"2023-11-25T04:26:29.517931+0000","objects_scrubbed":56,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:49:54.902849+0000","scrub_duration":0.039475015000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":163190,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":169,"num_write_kb":336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":154576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"75'229","reported_seq":305,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:20.470401+0000","last_change":"2023-11-25T04:26:20.470429+0000","last_active":"2023-11-25T04:26:20.470401+0000","last_peered":"2023-11-25T04:26:20.470401+0000","last_clean":"2023-11-25T04:26:20.470401+0000","last_became_active":"2023-11-25T04:25:53.933395+0000","last_became_peered":"2023-11-25T04:25:53.933395+0000","last_unstale":"2023-11-25T04:26:20.470401+0000","last_undegraded":"2023-11-25T04:26:20.470401+0000","last_fullsized":"2023-11-25T04:26:20.470401+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'229","last_scrub_stamp":"2023-11-25T04:26:20.470325+0000","last_deep_scrub":"75'229","last_deep_scrub_stamp":"2023-11-25T04:26:20.470325+0000","last_clean_scrub_stamp":"2023-11-25T04:26:20.470325+0000","objects_scrubbed":55,"log_size":229,"log_dups_size":0,"ondisk_log_size":229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:55:52.038708+0000","scrub_duration":0.036798694,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1423587,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":182,"num_write_kb":1626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":1414508,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2f","version":"75'203","reported_seq":283,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:18.718065+0000","last_change":"2023-11-25T04:26:18.718065+0000","last_active":"2023-11-25T04:26:18.718065+0000","last_peered":"2023-11-25T04:26:18.718065+0000","last_clean":"2023-11-25T04:26:18.718065+0000","last_became_active":"2023-11-25T04:22:47.039123+0000","last_became_peered":"2023-11-25T04:22:47.039123+0000","last_unstale":"2023-11-25T04:26:18.718065+0000","last_undegraded":"2023-11-25T04:26:18.718065+0000","last_fullsized":"2023-11-25T04:26:18.718065+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'203","last_scrub_stamp":"2023-11-25T04:26:18.717922+0000","last_deep_scrub":"75'203","last_deep_scrub_stamp":"2023-11-25T04:26:18.717922+0000","last_clean_scrub_stamp":"2023-11-25T04:26:18.717922+0000","objects_scrubbed":51,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:33:40.286516+0000","scrub_duration":0.039125398999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":122752,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":143,"num_write_kb":435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"75'199","reported_seq":279,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:58.496118+0000","last_change":"2023-11-25T04:26:58.496118+0000","last_active":"2023-11-25T04:26:58.496118+0000","last_peered":"2023-11-25T04:26:58.496118+0000","last_clean":"2023-11-25T04:26:58.496118+0000","last_became_active":"2023-11-25T04:22:47.036411+0000","last_became_peered":"2023-11-25T04:22:47.036411+0000","last_unstale":"2023-11-25T04:26:58.496118+0000","last_undegraded":"2023-11-25T04:26:58.496118+0000","last_fullsized":"2023-11-25T04:26:58.496118+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'199","last_scrub_stamp":"2023-11-25T04:26:58.496005+0000","last_deep_scrub":"75'199","last_deep_scrub_stamp":"2023-11-25T04:26:58.496005+0000","last_clean_scrub_stamp":"2023-11-25T04:26:58.496005+0000","objects_scrubbed":51,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:57:18.007638+0000","scrub_duration":0.041894438999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4770899,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":140,"num_write_kb":4870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2d","version":"75'194","reported_seq":274,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:47.814157+0000","last_change":"2023-11-25T04:26:47.814157+0000","last_active":"2023-11-25T04:26:47.814157+0000","last_peered":"2023-11-25T04:26:47.814157+0000","last_clean":"2023-11-25T04:26:47.814157+0000","last_became_active":"2023-11-25T04:22:47.038477+0000","last_became_peered":"2023-11-25T04:22:47.038477+0000","last_unstale":"2023-11-25T04:26:47.814157+0000","last_undegraded":"2023-11-25T04:26:47.814157+0000","last_fullsized":"2023-11-25T04:26:47.814157+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'194","last_scrub_stamp":"2023-11-25T04:26:47.814018+0000","last_deep_scrub":"75'194","last_deep_scrub_stamp":"2023-11-25T04:26:47.814018+0000","last_clean_scrub_stamp":"2023-11-25T04:26:47.814018+0000","objects_scrubbed":51,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:20:21.601431+0000","scrub_duration":0.038348491999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":115975,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"75'230","reported_seq":307,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:19.428426+0000","last_change":"2023-11-25T04:26:19.428460+0000","last_active":"2023-11-25T04:26:19.428426+0000","last_peered":"2023-11-25T04:26:19.428426+0000","last_clean":"2023-11-25T04:26:19.428426+0000","last_became_active":"2023-11-25T04:25:52.922367+0000","last_became_peered":"2023-11-25T04:25:52.922367+0000","last_unstale":"2023-11-25T04:26:19.428426+0000","last_undegraded":"2023-11-25T04:26:19.428426+0000","last_fullsized":"2023-11-25T04:26:19.428426+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'230","last_scrub_stamp":"2023-11-25T04:26:19.428350+0000","last_deep_scrub":"75'230","last_deep_scrub_stamp":"2023-11-25T04:26:19.428350+0000","last_clean_scrub_stamp":"2023-11-25T04:26:19.428350+0000","objects_scrubbed":63,"log_size":230,"log_dups_size":0,"ondisk_log_size":230,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:10:12.216682+0000","scrub_duration":0.042625061999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":747090,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":211,"num_write_kb":985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":732276,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"75'211","reported_seq":288,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:17.381634+0000","last_change":"2023-11-25T04:26:17.381662+0000","last_active":"2023-11-25T04:26:17.381634+0000","last_peered":"2023-11-25T04:26:17.381634+0000","last_clean":"2023-11-25T04:26:17.381634+0000","last_became_active":"2023-11-25T04:25:52.923378+0000","last_became_peered":"2023-11-25T04:25:52.923378+0000","last_unstale":"2023-11-25T04:26:17.381634+0000","last_undegraded":"2023-11-25T04:26:17.381634+0000","last_fullsized":"2023-11-25T04:26:17.381634+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'211","last_scrub_stamp":"2023-11-25T04:26:17.381560+0000","last_deep_scrub":"75'211","last_deep_scrub_stamp":"2023-11-25T04:26:17.381560+0000","last_clean_scrub_stamp":"2023-11-25T04:26:17.381560+0000","objects_scrubbed":55,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:26:19.664059+0000","scrub_duration":0.036605545000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69580,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":142,"num_write_kb":315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":98,"num_bytes_recovered":69580,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2a","version":"75'222","reported_seq":530,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:14.438320+0000","last_change":"2023-11-25T04:27:14.438320+0000","last_active":"2023-11-25T04:27:14.438320+0000","last_peered":"2023-11-25T04:27:14.438320+0000","last_clean":"2023-11-25T04:27:14.438320+0000","last_became_active":"2023-11-25T04:25:52.924844+0000","last_became_peered":"2023-11-25T04:25:52.924844+0000","last_unstale":"2023-11-25T04:27:14.438320+0000","last_undegraded":"2023-11-25T04:27:14.438320+0000","last_fullsized":"2023-11-25T04:27:14.438320+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'222","last_scrub_stamp":"2023-11-25T04:27:14.438222+0000","last_deep_scrub":"75'222","last_deep_scrub_stamp":"2023-11-25T04:27:14.438222+0000","last_clean_scrub_stamp":"2023-11-25T04:27:14.438222+0000","objects_scrubbed":51,"log_size":222,"log_dups_size":0,"ondisk_log_size":222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:18:44.242226+0000","scrub_duration":0.036903918000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":115997,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":162,"num_write_kb":261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":115997,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.29","version":"75'211","reported_seq":509,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:12.692632+0000","last_change":"2023-11-25T04:26:12.692632+0000","last_active":"2023-11-25T04:26:12.692632+0000","last_peered":"2023-11-25T04:26:12.692632+0000","last_clean":"2023-11-25T04:26:12.692632+0000","last_became_active":"2023-11-25T04:25:49.919886+0000","last_became_peered":"2023-11-25T04:25:49.919886+0000","last_unstale":"2023-11-25T04:26:12.692632+0000","last_undegraded":"2023-11-25T04:26:12.692632+0000","last_fullsized":"2023-11-25T04:26:12.692632+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'211","last_scrub_stamp":"2023-11-25T04:26:12.692467+0000","last_deep_scrub":"75'211","last_deep_scrub_stamp":"2023-11-25T04:26:12.692467+0000","last_clean_scrub_stamp":"2023-11-25T04:26:12.692467+0000","objects_scrubbed":59,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:13:03.899209+0000","scrub_duration":0.043132342999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":212878,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":212878,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"75'218","reported_seq":297,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:00.495508+0000","last_change":"2023-11-25T04:27:00.495508+0000","last_active":"2023-11-25T04:27:00.495508+0000","last_peered":"2023-11-25T04:27:00.495508+0000","last_clean":"2023-11-25T04:27:00.495508+0000","last_became_active":"2023-11-25T04:22:47.036135+0000","last_became_peered":"2023-11-25T04:22:47.036135+0000","last_unstale":"2023-11-25T04:27:00.495508+0000","last_undegraded":"2023-11-25T04:27:00.495508+0000","last_fullsized":"2023-11-25T04:27:00.495508+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'218","last_scrub_stamp":"2023-11-25T04:27:00.495361+0000","last_deep_scrub":"75'218","last_deep_scrub_stamp":"2023-11-25T04:27:00.495361+0000","last_clean_scrub_stamp":"2023-11-25T04:27:00.495361+0000","objects_scrubbed":49,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:17:48.678120+0000","scrub_duration":0.034866902999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":102957,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":172,"num_write_kb":610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"75'217","reported_seq":527,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:18.423856+0000","last_change":"2023-11-25T04:27:18.423894+0000","last_active":"2023-11-25T04:27:18.423856+0000","last_peered":"2023-11-25T04:27:18.423856+0000","last_clean":"2023-11-25T04:27:18.423856+0000","last_became_active":"2023-11-25T04:25:51.922364+0000","last_became_peered":"2023-11-25T04:25:51.922364+0000","last_unstale":"2023-11-25T04:27:18.423856+0000","last_undegraded":"2023-11-25T04:27:18.423856+0000","last_fullsized":"2023-11-25T04:27:18.423856+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'217","last_scrub_stamp":"2023-11-25T04:27:18.423755+0000","last_deep_scrub":"75'217","last_deep_scrub_stamp":"2023-11-25T04:27:18.423755+0000","last_clean_scrub_stamp":"2023-11-25T04:27:18.423755+0000","objects_scrubbed":50,"log_size":217,"log_dups_size":0,"ondisk_log_size":217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T16:12:12.360172+0000","scrub_duration":0.042117390999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":182683,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":171,"num_write_kb":791,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":182683,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"54'187","reported_seq":507,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:09.671589+0000","last_change":"2023-11-25T04:25:51.922529+0000","last_active":"2023-11-25T04:26:09.671589+0000","last_peered":"2023-11-25T04:26:09.671589+0000","last_clean":"2023-11-25T04:26:09.671589+0000","last_became_active":"2023-11-25T04:25:51.922064+0000","last_became_peered":"2023-11-25T04:25:51.922064+0000","last_unstale":"2023-11-25T04:26:09.671589+0000","last_undegraded":"2023-11-25T04:26:09.671589+0000","last_fullsized":"2023-11-25T04:26:09.671589+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":187,"log_dups_size":0,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":86311,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":171,"num_write_kb":274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":116,"num_bytes_recovered":86311,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.25","version":"75'201","reported_seq":278,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:10.399641+0000","last_change":"2023-11-25T04:26:10.399668+0000","last_active":"2023-11-25T04:26:10.399641+0000","last_peered":"2023-11-25T04:26:10.399641+0000","last_clean":"2023-11-25T04:26:10.399641+0000","last_became_active":"2023-11-25T04:25:52.924795+0000","last_became_peered":"2023-11-25T04:25:52.924795+0000","last_unstale":"2023-11-25T04:26:10.399641+0000","last_undegraded":"2023-11-25T04:26:10.399641+0000","last_fullsized":"2023-11-25T04:26:10.399641+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'201","last_scrub_stamp":"2023-11-25T04:26:10.399561+0000","last_deep_scrub":"75'201","last_deep_scrub_stamp":"2023-11-25T04:26:10.399561+0000","last_clean_scrub_stamp":"2023-11-25T04:26:10.399561+0000","objects_scrubbed":52,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:15:15.077501+0000","scrub_duration":0.038027564,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":128288,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":167,"num_write_kb":267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":120265,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.24","version":"75'195","reported_seq":482,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:43.546743+0000","last_change":"2023-11-25T04:26:43.546743+0000","last_active":"2023-11-25T04:26:43.546743+0000","last_peered":"2023-11-25T04:26:43.546743+0000","last_clean":"2023-11-25T04:26:43.546743+0000","last_became_active":"2023-11-25T04:25:50.922216+0000","last_became_peered":"2023-11-25T04:25:50.922216+0000","last_unstale":"2023-11-25T04:26:43.546743+0000","last_undegraded":"2023-11-25T04:26:43.546743+0000","last_fullsized":"2023-11-25T04:26:43.546743+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'195","last_scrub_stamp":"2023-11-25T04:26:43.546552+0000","last_deep_scrub":"75'195","last_deep_scrub_stamp":"2023-11-25T04:26:43.546552+0000","last_clean_scrub_stamp":"2023-11-25T04:26:43.546552+0000","objects_scrubbed":44,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:16:44.328689+0000","scrub_duration":0.034140957,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":107507,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":148,"num_write_kb":288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":107494,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.23","version":"75'222","reported_seq":302,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:50.770920+0000","last_change":"2023-11-25T04:26:50.770920+0000","last_active":"2023-11-25T04:26:50.770920+0000","last_peered":"2023-11-25T04:26:50.770920+0000","last_clean":"2023-11-25T04:26:50.770920+0000","last_became_active":"2023-11-25T04:22:47.036823+0000","last_became_peered":"2023-11-25T04:22:47.036823+0000","last_unstale":"2023-11-25T04:26:50.770920+0000","last_undegraded":"2023-11-25T04:26:50.770920+0000","last_fullsized":"2023-11-25T04:26:50.770920+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'222","last_scrub_stamp":"2023-11-25T04:26:50.770805+0000","last_deep_scrub":"75'222","last_deep_scrub_stamp":"2023-11-25T04:26:50.770805+0000","last_clean_scrub_stamp":"2023-11-25T04:26:50.770805+0000","objects_scrubbed":67,"log_size":222,"log_dups_size":0,"ondisk_log_size":222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:16:51.552942+0000","scrub_duration":0.048356029000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4304582,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":152,"num_write_kb":4331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.22","version":"75'253","reported_seq":593,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:27.393520+0000","last_change":"2023-11-25T04:27:27.393555+0000","last_active":"2023-11-25T04:27:27.393520+0000","last_peered":"2023-11-25T04:27:27.393520+0000","last_clean":"2023-11-25T04:27:27.393520+0000","last_became_active":"2023-11-25T04:25:54.939584+0000","last_became_peered":"2023-11-25T04:25:54.939584+0000","last_unstale":"2023-11-25T04:27:27.393520+0000","last_undegraded":"2023-11-25T04:27:27.393520+0000","last_fullsized":"2023-11-25T04:27:27.393520+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'253","last_scrub_stamp":"2023-11-25T04:27:27.393439+0000","last_deep_scrub":"75'253","last_deep_scrub_stamp":"2023-11-25T04:27:27.393439+0000","last_clean_scrub_stamp":"2023-11-25T04:27:27.393439+0000","objects_scrubbed":64,"log_size":253,"log_dups_size":0,"ondisk_log_size":253,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:03:46.694422+0000","scrub_duration":0.044005476000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":129145,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":185,"num_write_kb":432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":121,"num_bytes_recovered":129107,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"75'260","reported_seq":340,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:06.501506+0000","last_change":"2023-11-25T04:27:06.501536+0000","last_active":"2023-11-25T04:27:06.501506+0000","last_peered":"2023-11-25T04:27:06.501506+0000","last_clean":"2023-11-25T04:27:06.501506+0000","last_became_active":"2023-11-25T04:22:47.037206+0000","last_became_peered":"2023-11-25T04:22:47.037206+0000","last_unstale":"2023-11-25T04:27:06.501506+0000","last_undegraded":"2023-11-25T04:27:06.501506+0000","last_fullsized":"2023-11-25T04:27:06.501506+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'260","last_scrub_stamp":"2023-11-25T04:27:06.501413+0000","last_deep_scrub":"75'260","last_deep_scrub_stamp":"2023-11-25T04:27:06.501413+0000","last_clean_scrub_stamp":"2023-11-25T04:27:06.501413+0000","objects_scrubbed":59,"log_size":260,"log_dups_size":0,"ondisk_log_size":260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:18:02.436135+0000","scrub_duration":0.044736607999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":215311,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":194,"num_write_kb":381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"54'180","reported_seq":246,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.985760+0000","last_change":"2023-11-25T04:25:53.933176+0000","last_active":"2023-11-25T04:26:00.985760+0000","last_peered":"2023-11-25T04:26:00.985760+0000","last_clean":"2023-11-25T04:26:00.985760+0000","last_became_active":"2023-11-25T04:25:53.931937+0000","last_became_peered":"2023-11-25T04:25:53.931937+0000","last_unstale":"2023-11-25T04:26:00.985760+0000","last_undegraded":"2023-11-25T04:26:00.985760+0000","last_fullsized":"2023-11-25T04:26:00.985760+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104603,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":172,"num_write_kb":267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":104603,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"75'193","reported_seq":273,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:32.532758+0000","last_change":"2023-11-25T04:26:32.532758+0000","last_active":"2023-11-25T04:26:32.532758+0000","last_peered":"2023-11-25T04:26:32.532758+0000","last_clean":"2023-11-25T04:26:32.532758+0000","last_became_active":"2023-11-25T04:22:47.036107+0000","last_became_peered":"2023-11-25T04:22:47.036107+0000","last_unstale":"2023-11-25T04:26:32.532758+0000","last_undegraded":"2023-11-25T04:26:32.532758+0000","last_fullsized":"2023-11-25T04:26:32.532758+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'193","last_scrub_stamp":"2023-11-25T04:26:32.532624+0000","last_deep_scrub":"75'193","last_deep_scrub_stamp":"2023-11-25T04:26:32.532624+0000","last_clean_scrub_stamp":"2023-11-25T04:26:32.532624+0000","objects_scrubbed":56,"log_size":193,"log_dups_size":0,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:09:38.769552+0000","scrub_duration":0.038075093999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":183586,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":127,"num_write_kb":284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"75'117","reported_seq":203,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:31.496279+0000","last_change":"2023-11-25T04:26:31.496307+0000","last_active":"2023-11-25T04:26:31.496279+0000","last_peered":"2023-11-25T04:26:31.496279+0000","last_clean":"2023-11-25T04:26:31.496279+0000","last_became_active":"2023-11-25T04:25:49.920010+0000","last_became_peered":"2023-11-25T04:25:49.920010+0000","last_unstale":"2023-11-25T04:26:31.496279+0000","last_undegraded":"2023-11-25T04:26:31.496279+0000","last_fullsized":"2023-11-25T04:26:31.496279+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'117","last_scrub_stamp":"2023-11-25T04:26:31.496196+0000","last_deep_scrub":"75'117","last_deep_scrub_stamp":"2023-11-25T04:26:31.496196+0000","last_clean_scrub_stamp":"2023-11-25T04:26:31.496196+0000","objects_scrubbed":3,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:26:02.874431+0000","scrub_duration":0.0076030029999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2816216,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":113,"num_write_kb":2816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":2816182,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"75'189","reported_seq":267,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:40.589192+0000","last_change":"2023-11-25T04:26:40.589251+0000","last_active":"2023-11-25T04:26:40.589192+0000","last_peered":"2023-11-25T04:26:40.589192+0000","last_clean":"2023-11-25T04:26:40.589192+0000","last_became_active":"2023-11-25T04:22:47.037158+0000","last_became_peered":"2023-11-25T04:22:47.037158+0000","last_unstale":"2023-11-25T04:26:40.589192+0000","last_undegraded":"2023-11-25T04:26:40.589192+0000","last_fullsized":"2023-11-25T04:26:40.589192+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'189","last_scrub_stamp":"2023-11-25T04:26:40.589106+0000","last_deep_scrub":"75'189","last_deep_scrub_stamp":"2023-11-25T04:26:40.589106+0000","last_clean_scrub_stamp":"2023-11-25T04:26:40.589106+0000","objects_scrubbed":44,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:32:56.267980+0000","scrub_duration":0.032171351000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":205976,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.12","version":"49'2","reported_seq":92,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:30.784752+0000","last_change":"2023-11-25T04:26:30.784752+0000","last_active":"2023-11-25T04:26:30.784752+0000","last_peered":"2023-11-25T04:26:30.784752+0000","last_clean":"2023-11-25T04:26:30.784752+0000","last_became_active":"2023-11-25T04:25:47.000516+0000","last_became_peered":"2023-11-25T04:25:47.000516+0000","last_unstale":"2023-11-25T04:26:30.784752+0000","last_undegraded":"2023-11-25T04:26:30.784752+0000","last_fullsized":"2023-11-25T04:26:30.784752+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'2","last_scrub_stamp":"2023-11-25T04:26:30.784639+0000","last_deep_scrub":"49'2","last_deep_scrub_stamp":"2023-11-25T04:26:30.784639+0000","last_clean_scrub_stamp":"2023-11-25T04:26:30.784639+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:38:15.370559+0000","scrub_duration":0.001803528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"54'175","reported_seq":490,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986834+0000","last_change":"2023-11-25T04:25:52.923694+0000","last_active":"2023-11-25T04:26:00.986834+0000","last_peered":"2023-11-25T04:26:00.986834+0000","last_clean":"2023-11-25T04:26:00.986834+0000","last_became_active":"2023-11-25T04:25:52.923045+0000","last_became_peered":"2023-11-25T04:25:52.923045+0000","last_unstale":"2023-11-25T04:26:00.986834+0000","last_undegraded":"2023-11-25T04:26:00.986834+0000","last_fullsized":"2023-11-25T04:26:00.986834+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":171823,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":185,"num_write_kb":340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":166387,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":90,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:18.488445+0000","last_change":"2023-11-25T04:26:18.488445+0000","last_active":"2023-11-25T04:26:18.488445+0000","last_peered":"2023-11-25T04:26:18.488445+0000","last_clean":"2023-11-25T04:26:18.488445+0000","last_became_active":"2023-11-25T04:25:46.993870+0000","last_became_peered":"2023-11-25T04:25:46.993870+0000","last_unstale":"2023-11-25T04:26:18.488445+0000","last_undegraded":"2023-11-25T04:26:18.488445+0000","last_fullsized":"2023-11-25T04:26:18.488445+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:26:18.488345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:26:18.488345+0000","last_clean_scrub_stamp":"2023-11-25T04:26:18.488345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:43:21.558596+0000","scrub_duration":0.002108399,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"54'152","reported_seq":436,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986747+0000","last_change":"2023-11-25T04:25:50.922922+0000","last_active":"2023-11-25T04:26:00.986747+0000","last_peered":"2023-11-25T04:26:00.986747+0000","last_clean":"2023-11-25T04:26:00.986747+0000","last_became_active":"2023-11-25T04:25:50.922511+0000","last_became_peered":"2023-11-25T04:25:50.922511+0000","last_unstale":"2023-11-25T04:26:00.986747+0000","last_undegraded":"2023-11-25T04:26:00.986747+0000","last_fullsized":"2023-11-25T04:26:00.986747+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":579704,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":97,"num_bytes_recovered":570044,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.10","version":"75'12","reported_seq":52,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:06.483215+0000","last_change":"2023-11-25T04:27:06.483272+0000","last_active":"2023-11-25T04:27:06.483215+0000","last_peered":"2023-11-25T04:27:06.483215+0000","last_clean":"2023-11-25T04:27:06.483215+0000","last_became_active":"2023-11-25T04:25:46.998778+0000","last_became_peered":"2023-11-25T04:25:46.998778+0000","last_unstale":"2023-11-25T04:27:06.483215+0000","last_undegraded":"2023-11-25T04:27:06.483215+0000","last_fullsized":"2023-11-25T04:27:06.483215+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'12","last_scrub_stamp":"2023-11-25T04:27:06.483113+0000","last_deep_scrub":"75'12","last_deep_scrub_stamp":"2023-11-25T04:27:06.483113+0000","last_clean_scrub_stamp":"2023-11-25T04:27:06.483113+0000","objects_scrubbed":4,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:26:21.828117+0000","scrub_duration":0.0040538359999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"75'3","reported_seq":47,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:36.494324+0000","last_change":"2023-11-25T04:26:36.494358+0000","last_active":"2023-11-25T04:26:36.494324+0000","last_peered":"2023-11-25T04:26:36.494324+0000","last_clean":"2023-11-25T04:26:36.494324+0000","last_became_active":"2023-11-25T04:25:46.997108+0000","last_became_peered":"2023-11-25T04:25:46.997108+0000","last_unstale":"2023-11-25T04:26:36.494324+0000","last_undegraded":"2023-11-25T04:26:36.494324+0000","last_fullsized":"2023-11-25T04:26:36.494324+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'3","last_scrub_stamp":"2023-11-25T04:26:36.494205+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2023-11-25T04:26:36.494205+0000","last_clean_scrub_stamp":"2023-11-25T04:26:36.494205+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:47:03.918120+0000","scrub_duration":0.0026531879999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"75'208","reported_seq":503,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:07.384457+0000","last_change":"2023-11-25T04:27:07.384457+0000","last_active":"2023-11-25T04:27:07.384457+0000","last_peered":"2023-11-25T04:27:07.384457+0000","last_clean":"2023-11-25T04:27:07.384457+0000","last_became_active":"2023-11-25T04:25:50.921772+0000","last_became_peered":"2023-11-25T04:25:50.921772+0000","last_unstale":"2023-11-25T04:27:07.384457+0000","last_undegraded":"2023-11-25T04:27:07.384457+0000","last_fullsized":"2023-11-25T04:27:07.384457+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'208","last_scrub_stamp":"2023-11-25T04:27:07.384323+0000","last_deep_scrub":"75'208","last_deep_scrub_stamp":"2023-11-25T04:27:07.384323+0000","last_clean_scrub_stamp":"2023-11-25T04:27:07.384323+0000","objects_scrubbed":49,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:52:56.354420+0000","scrub_duration":0.034118876999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112103,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":164,"num_write_kb":240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":99,"num_bytes_recovered":103472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"75'3","reported_seq":93,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:36.492740+0000","last_change":"2023-11-25T04:26:36.492740+0000","last_active":"2023-11-25T04:26:36.492740+0000","last_peered":"2023-11-25T04:26:36.492740+0000","last_clean":"2023-11-25T04:26:36.492740+0000","last_became_active":"2023-11-25T04:25:46.994640+0000","last_became_peered":"2023-11-25T04:25:46.994640+0000","last_unstale":"2023-11-25T04:26:36.492740+0000","last_undegraded":"2023-11-25T04:26:36.492740+0000","last_fullsized":"2023-11-25T04:26:36.492740+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'3","last_scrub_stamp":"2023-11-25T04:26:36.492569+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2023-11-25T04:26:36.492569+0000","last_clean_scrub_stamp":"2023-11-25T04:26:36.492569+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:28:25.392135+0000","scrub_duration":0.0033623030000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"75'229","reported_seq":309,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:05.415841+0000","last_change":"2023-11-25T04:27:05.415841+0000","last_active":"2023-11-25T04:27:05.415841+0000","last_peered":"2023-11-25T04:27:05.415841+0000","last_clean":"2023-11-25T04:27:05.415841+0000","last_became_active":"2023-11-25T04:22:47.039296+0000","last_became_peered":"2023-11-25T04:22:47.039296+0000","last_unstale":"2023-11-25T04:27:05.415841+0000","last_undegraded":"2023-11-25T04:27:05.415841+0000","last_fullsized":"2023-11-25T04:27:05.415841+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'229","last_scrub_stamp":"2023-11-25T04:27:05.415739+0000","last_deep_scrub":"75'229","last_deep_scrub_stamp":"2023-11-25T04:27:05.415739+0000","last_clean_scrub_stamp":"2023-11-25T04:27:05.415739+0000","objects_scrubbed":56,"log_size":229,"log_dups_size":0,"ondisk_log_size":229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:46:38.706963+0000","scrub_duration":0.037100175999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154533,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":182,"num_write_kb":299,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"75'2","reported_seq":47,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:33.490361+0000","last_change":"2023-11-25T04:26:33.490414+0000","last_active":"2023-11-25T04:26:33.490361+0000","last_peered":"2023-11-25T04:26:33.490361+0000","last_clean":"2023-11-25T04:26:33.490361+0000","last_became_active":"2023-11-25T04:25:46.999604+0000","last_became_peered":"2023-11-25T04:25:46.999604+0000","last_unstale":"2023-11-25T04:26:33.490361+0000","last_undegraded":"2023-11-25T04:26:33.490361+0000","last_fullsized":"2023-11-25T04:26:33.490361+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'2","last_scrub_stamp":"2023-11-25T04:26:33.490271+0000","last_deep_scrub":"75'2","last_deep_scrub_stamp":"2023-11-25T04:26:33.490271+0000","last_clean_scrub_stamp":"2023-11-25T04:26:33.490271+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:29:08.502565+0000","scrub_duration":0.0030545450000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"54'147","reported_seq":212,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986571+0000","last_change":"2023-11-25T04:25:54.939274+0000","last_active":"2023-11-25T04:26:00.986571+0000","last_peered":"2023-11-25T04:26:00.986571+0000","last_clean":"2023-11-25T04:26:00.986571+0000","last_became_active":"2023-11-25T04:25:54.938236+0000","last_became_peered":"2023-11-25T04:25:54.938236+0000","last_unstale":"2023-11-25T04:26:00.986571+0000","last_undegraded":"2023-11-25T04:26:00.986571+0000","last_fullsized":"2023-11-25T04:26:00.986571+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":147,"log_dups_size":0,"ondisk_log_size":147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":87524,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":155,"num_write_kb":235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":87524,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":43,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:41.504866+0000","last_change":"2023-11-25T04:26:41.504892+0000","last_active":"2023-11-25T04:26:41.504866+0000","last_peered":"2023-11-25T04:26:41.504866+0000","last_clean":"2023-11-25T04:26:41.504866+0000","last_became_active":"2023-11-25T04:25:46.995712+0000","last_became_peered":"2023-11-25T04:25:46.995712+0000","last_unstale":"2023-11-25T04:26:41.504866+0000","last_undegraded":"2023-11-25T04:26:41.504866+0000","last_fullsized":"2023-11-25T04:26:41.504866+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:26:41.504775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:26:41.504775+0000","last_clean_scrub_stamp":"2023-11-25T04:26:41.504775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:35:58.006258+0000","scrub_duration":0.001777907,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"54'155","reported_seq":443,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:09.671401+0000","last_change":"2023-11-25T04:25:52.923096+0000","last_active":"2023-11-25T04:26:09.671401+0000","last_peered":"2023-11-25T04:26:09.671401+0000","last_clean":"2023-11-25T04:26:09.671401+0000","last_became_active":"2023-11-25T04:25:52.922549+0000","last_became_peered":"2023-11-25T04:25:52.922549+0000","last_unstale":"2023-11-25T04:26:09.671401+0000","last_undegraded":"2023-11-25T04:26:09.671401+0000","last_fullsized":"2023-11-25T04:26:09.671401+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":266919,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":264258,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"75'13","reported_seq":108,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:05.498699+0000","last_change":"2023-11-25T04:27:05.498699+0000","last_active":"2023-11-25T04:27:05.498699+0000","last_peered":"2023-11-25T04:27:05.498699+0000","last_clean":"2023-11-25T04:27:05.498699+0000","last_became_active":"2023-11-25T04:25:47.000687+0000","last_became_peered":"2023-11-25T04:25:47.000687+0000","last_unstale":"2023-11-25T04:27:05.498699+0000","last_undegraded":"2023-11-25T04:27:05.498699+0000","last_fullsized":"2023-11-25T04:27:05.498699+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'13","last_scrub_stamp":"2023-11-25T04:27:05.498602+0000","last_deep_scrub":"75'13","last_deep_scrub_stamp":"2023-11-25T04:27:05.498602+0000","last_clean_scrub_stamp":"2023-11-25T04:27:05.498602+0000","objects_scrubbed":2,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:44:21.302320+0000","scrub_duration":0.0090048890000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":4101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4194304,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"75'235","reported_seq":314,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:49.411583+0000","last_change":"2023-11-25T04:26:49.411614+0000","last_active":"2023-11-25T04:26:49.411583+0000","last_peered":"2023-11-25T04:26:49.411583+0000","last_clean":"2023-11-25T04:26:49.411583+0000","last_became_active":"2023-11-25T04:25:50.922409+0000","last_became_peered":"2023-11-25T04:25:50.922409+0000","last_unstale":"2023-11-25T04:26:49.411583+0000","last_undegraded":"2023-11-25T04:26:49.411583+0000","last_fullsized":"2023-11-25T04:26:49.411583+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'235","last_scrub_stamp":"2023-11-25T04:26:49.411506+0000","last_deep_scrub":"75'235","last_deep_scrub_stamp":"2023-11-25T04:26:49.411506+0000","last_clean_scrub_stamp":"2023-11-25T04:26:49.411506+0000","objects_scrubbed":62,"log_size":235,"log_dups_size":0,"ondisk_log_size":235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:38:13.278410+0000","scrub_duration":0.044042721,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":553893,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":169,"num_write_kb":770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":553893,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":43,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:05.477955+0000","last_change":"2023-11-25T04:27:05.477987+0000","last_active":"2023-11-25T04:27:05.477955+0000","last_peered":"2023-11-25T04:27:05.477955+0000","last_clean":"2023-11-25T04:27:05.477955+0000","last_became_active":"2023-11-25T04:25:46.997019+0000","last_became_peered":"2023-11-25T04:25:46.997019+0000","last_unstale":"2023-11-25T04:27:05.477955+0000","last_undegraded":"2023-11-25T04:27:05.477955+0000","last_fullsized":"2023-11-25T04:27:05.477955+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:27:05.477862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:27:05.477862+0000","last_clean_scrub_stamp":"2023-11-25T04:27:05.477862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:44:35.653542+0000","scrub_duration":0.0018186669999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"54'163","reported_seq":229,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986523+0000","last_change":"2023-11-25T04:25:53.934183+0000","last_active":"2023-11-25T04:26:00.986523+0000","last_peered":"2023-11-25T04:26:00.986523+0000","last_clean":"2023-11-25T04:26:00.986523+0000","last_became_active":"2023-11-25T04:25:53.933471+0000","last_became_peered":"2023-11-25T04:25:53.933471+0000","last_unstale":"2023-11-25T04:26:00.986523+0000","last_undegraded":"2023-11-25T04:26:00.986523+0000","last_fullsized":"2023-11-25T04:26:00.986523+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":320867,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":190,"num_write_kb":577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":108,"num_bytes_recovered":311640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":90,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:26.554458+0000","last_change":"2023-11-25T04:26:26.554458+0000","last_active":"2023-11-25T04:26:26.554458+0000","last_peered":"2023-11-25T04:26:26.554458+0000","last_clean":"2023-11-25T04:26:26.554458+0000","last_became_active":"2023-11-25T04:25:46.994101+0000","last_became_peered":"2023-11-25T04:25:46.994101+0000","last_unstale":"2023-11-25T04:26:26.554458+0000","last_undegraded":"2023-11-25T04:26:26.554458+0000","last_fullsized":"2023-11-25T04:26:26.554458+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:26:26.554355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:26:26.554355+0000","last_clean_scrub_stamp":"2023-11-25T04:26:26.554355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:20:20.095629+0000","scrub_duration":0.0016349400000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"75'187","reported_seq":266,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:35.557378+0000","last_change":"2023-11-25T04:26:35.557414+0000","last_active":"2023-11-25T04:26:35.557378+0000","last_peered":"2023-11-25T04:26:35.557378+0000","last_clean":"2023-11-25T04:26:35.557378+0000","last_became_active":"2023-11-25T04:25:49.920971+0000","last_became_peered":"2023-11-25T04:25:49.920971+0000","last_unstale":"2023-11-25T04:26:35.557378+0000","last_undegraded":"2023-11-25T04:26:35.557378+0000","last_fullsized":"2023-11-25T04:26:35.557378+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'187","last_scrub_stamp":"2023-11-25T04:26:35.557305+0000","last_deep_scrub":"75'187","last_deep_scrub_stamp":"2023-11-25T04:26:35.557305+0000","last_clean_scrub_stamp":"2023-11-25T04:26:35.557305+0000","objects_scrubbed":46,"log_size":187,"log_dups_size":0,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:15:17.199719+0000","scrub_duration":0.039856336999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3502371,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":3718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":89,"num_bytes_recovered":3497669,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"49'1","reported_seq":80,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:52.518138+0000","last_change":"2023-11-25T04:26:52.518138+0000","last_active":"2023-11-25T04:26:52.518138+0000","last_peered":"2023-11-25T04:26:52.518138+0000","last_clean":"2023-11-25T04:26:52.518138+0000","last_became_active":"2023-11-25T04:22:46.034158+0000","last_became_peered":"2023-11-25T04:22:46.034158+0000","last_unstale":"2023-11-25T04:26:52.518138+0000","last_undegraded":"2023-11-25T04:26:52.518138+0000","last_fullsized":"2023-11-25T04:26:52.518138+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'1","last_scrub_stamp":"2023-11-25T04:26:52.518023+0000","last_deep_scrub":"49'1","last_deep_scrub_stamp":"2023-11-25T04:26:52.518023+0000","last_clean_scrub_stamp":"2023-11-25T04:26:52.518023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:44:08.321741+0000","scrub_duration":0.001721757,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"75'194","reported_seq":274,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:15.719633+0000","last_change":"2023-11-25T04:26:15.719633+0000","last_active":"2023-11-25T04:26:15.719633+0000","last_peered":"2023-11-25T04:26:15.719633+0000","last_clean":"2023-11-25T04:26:15.719633+0000","last_became_active":"2023-11-25T04:22:47.038081+0000","last_became_peered":"2023-11-25T04:22:47.038081+0000","last_unstale":"2023-11-25T04:26:15.719633+0000","last_undegraded":"2023-11-25T04:26:15.719633+0000","last_fullsized":"2023-11-25T04:26:15.719633+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'194","last_scrub_stamp":"2023-11-25T04:26:15.719490+0000","last_deep_scrub":"75'194","last_deep_scrub_stamp":"2023-11-25T04:26:15.719490+0000","last_clean_scrub_stamp":"2023-11-25T04:26:15.719490+0000","objects_scrubbed":53,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:43:29.224433+0000","scrub_duration":0.037765356,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":174088,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":146,"num_write_kb":385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"75'159","reported_seq":243,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:21.467675+0000","last_change":"2023-11-25T04:27:21.467705+0000","last_active":"2023-11-25T04:27:21.467675+0000","last_peered":"2023-11-25T04:27:21.467675+0000","last_clean":"2023-11-25T04:27:21.467675+0000","last_became_active":"2023-11-25T04:25:46.999721+0000","last_became_peered":"2023-11-25T04:25:46.999721+0000","last_unstale":"2023-11-25T04:27:21.467675+0000","last_undegraded":"2023-11-25T04:27:21.467675+0000","last_fullsized":"2023-11-25T04:27:21.467675+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'159","last_scrub_stamp":"2023-11-25T04:27:21.467591+0000","last_deep_scrub":"75'159","last_deep_scrub_stamp":"2023-11-25T04:27:21.467591+0000","last_clean_scrub_stamp":"2023-11-25T04:27:21.467591+0000","objects_scrubbed":2,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:55:47.253212+0000","scrub_duration":0.005680058,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":43,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:19.454504+0000","last_change":"2023-11-25T04:27:19.454533+0000","last_active":"2023-11-25T04:27:19.454504+0000","last_peered":"2023-11-25T04:27:19.454504+0000","last_clean":"2023-11-25T04:27:19.454504+0000","last_became_active":"2023-11-25T04:25:46.997693+0000","last_became_peered":"2023-11-25T04:25:46.997693+0000","last_unstale":"2023-11-25T04:27:19.454504+0000","last_undegraded":"2023-11-25T04:27:19.454504+0000","last_fullsized":"2023-11-25T04:27:19.454504+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:27:19.454370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:27:19.454370+0000","last_clean_scrub_stamp":"2023-11-25T04:27:19.454370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:43:29.432453+0000","scrub_duration":0.001737955,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"46'1","reported_seq":80,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:25.228633+0000","last_change":"2023-11-25T04:27:25.228654+0000","last_active":"2023-11-25T04:27:25.228633+0000","last_peered":"2023-11-25T04:27:25.228633+0000","last_clean":"2023-11-25T04:27:25.228633+0000","last_became_active":"2023-11-25T04:22:46.035916+0000","last_became_peered":"2023-11-25T04:22:46.035916+0000","last_unstale":"2023-11-25T04:27:25.228633+0000","last_undegraded":"2023-11-25T04:27:25.228633+0000","last_fullsized":"2023-11-25T04:27:25.228633+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"46'1","last_scrub_stamp":"2023-11-25T04:27:25.228531+0000","last_deep_scrub":"46'1","last_deep_scrub_stamp":"2023-11-25T04:27:25.228531+0000","last_clean_scrub_stamp":"2023-11-25T04:27:25.228531+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:47:55.490961+0000","scrub_duration":0.0016495870000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"75'204","reported_seq":282,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:28.483923+0000","last_change":"2023-11-25T04:27:28.483954+0000","last_active":"2023-11-25T04:27:28.483923+0000","last_peered":"2023-11-25T04:27:28.483923+0000","last_clean":"2023-11-25T04:27:28.483923+0000","last_became_active":"2023-11-25T04:25:51.922490+0000","last_became_peered":"2023-11-25T04:25:51.922490+0000","last_unstale":"2023-11-25T04:27:28.483923+0000","last_undegraded":"2023-11-25T04:27:28.483923+0000","last_fullsized":"2023-11-25T04:27:28.483923+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'204","last_scrub_stamp":"2023-11-25T04:27:28.483839+0000","last_deep_scrub":"75'204","last_deep_scrub_stamp":"2023-11-25T04:27:28.483839+0000","last_clean_scrub_stamp":"2023-11-25T04:27:28.483839+0000","objects_scrubbed":58,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:49:24.463794+0000","scrub_duration":0.041785592000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":502154,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":170,"num_write_kb":687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":99,"num_bytes_recovered":502101,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":43,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:30.509601+0000","last_change":"2023-11-25T04:26:30.509621+0000","last_active":"2023-11-25T04:26:30.509601+0000","last_peered":"2023-11-25T04:26:30.509601+0000","last_clean":"2023-11-25T04:26:30.509601+0000","last_became_active":"2023-11-25T04:25:47.000033+0000","last_became_peered":"2023-11-25T04:25:47.000033+0000","last_unstale":"2023-11-25T04:26:30.509601+0000","last_undegraded":"2023-11-25T04:26:30.509601+0000","last_fullsized":"2023-11-25T04:26:30.509601+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:26:30.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:26:30.509522+0000","last_clean_scrub_stamp":"2023-11-25T04:26:30.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:17:09.583541+0000","scrub_duration":0.001771145,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"75'38","reported_seq":167,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:44.819150+0000","last_change":"2023-11-25T04:26:44.819150+0000","last_active":"2023-11-25T04:26:44.819150+0000","last_peered":"2023-11-25T04:26:44.819150+0000","last_clean":"2023-11-25T04:26:44.819150+0000","last_became_active":"2023-11-25T04:25:46.995637+0000","last_became_peered":"2023-11-25T04:25:46.995637+0000","last_unstale":"2023-11-25T04:26:44.819150+0000","last_undegraded":"2023-11-25T04:26:44.819150+0000","last_fullsized":"2023-11-25T04:26:44.819150+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'38","last_scrub_stamp":"2023-11-25T04:26:44.819010+0000","last_deep_scrub":"75'38","last_deep_scrub_stamp":"2023-11-25T04:26:44.819010+0000","last_clean_scrub_stamp":"2023-11-25T04:26:44.819010+0000","objects_scrubbed":1,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:17:29.276433+0000","scrub_duration":0.003812251,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":319160,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":35,"num_read_kb":334,"num_write":35,"num_write_kb":334,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":319160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"75'210","reported_seq":290,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:38.568589+0000","last_change":"2023-11-25T04:26:38.568589+0000","last_active":"2023-11-25T04:26:38.568589+0000","last_peered":"2023-11-25T04:26:38.568589+0000","last_clean":"2023-11-25T04:26:38.568589+0000","last_became_active":"2023-11-25T04:22:47.039197+0000","last_became_peered":"2023-11-25T04:22:47.039197+0000","last_unstale":"2023-11-25T04:26:38.568589+0000","last_undegraded":"2023-11-25T04:26:38.568589+0000","last_fullsized":"2023-11-25T04:26:38.568589+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'210","last_scrub_stamp":"2023-11-25T04:26:38.568488+0000","last_deep_scrub":"75'210","last_deep_scrub_stamp":"2023-11-25T04:26:38.568488+0000","last_clean_scrub_stamp":"2023-11-25T04:26:38.568488+0000","objects_scrubbed":52,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:17:23.025912+0000","scrub_duration":0.034345986000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":169303,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":189,"num_write_kb":333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":82,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:12.445153+0000","last_change":"2023-11-25T04:27:12.445171+0000","last_active":"2023-11-25T04:27:12.445153+0000","last_peered":"2023-11-25T04:27:12.445153+0000","last_clean":"2023-11-25T04:27:12.445153+0000","last_became_active":"2023-11-25T04:22:42.000041+0000","last_became_peered":"2023-11-25T04:22:42.000041+0000","last_unstale":"2023-11-25T04:27:12.445153+0000","last_undegraded":"2023-11-25T04:27:12.445153+0000","last_fullsized":"2023-11-25T04:27:12.445153+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:27:12.445067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:27:12.445067+0000","last_clean_scrub_stamp":"2023-11-25T04:27:12.445067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:18:00.627826+0000","scrub_duration":0.001522886,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"75'72","reported_seq":166,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:15.436284+0000","last_change":"2023-11-25T04:26:15.436284+0000","last_active":"2023-11-25T04:26:15.436284+0000","last_peered":"2023-11-25T04:26:15.436284+0000","last_clean":"2023-11-25T04:26:15.436284+0000","last_became_active":"2023-11-25T04:25:47.001162+0000","last_became_peered":"2023-11-25T04:25:47.001162+0000","last_unstale":"2023-11-25T04:26:15.436284+0000","last_undegraded":"2023-11-25T04:26:15.436284+0000","last_fullsized":"2023-11-25T04:26:15.436284+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'72","last_scrub_stamp":"2023-11-25T04:26:15.436148+0000","last_deep_scrub":"75'72","last_deep_scrub_stamp":"2023-11-25T04:26:15.436148+0000","last_clean_scrub_stamp":"2023-11-25T04:26:15.436148+0000","objects_scrubbed":1,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:37:38.643270+0000","scrub_duration":0.01087811,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":69,"num_write_kb":4130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4194304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"75'220","reported_seq":300,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:14.718055+0000","last_change":"2023-11-25T04:26:14.718055+0000","last_active":"2023-11-25T04:26:14.718055+0000","last_peered":"2023-11-25T04:26:14.718055+0000","last_clean":"2023-11-25T04:26:14.718055+0000","last_became_active":"2023-11-25T04:22:47.038046+0000","last_became_peered":"2023-11-25T04:22:47.038046+0000","last_unstale":"2023-11-25T04:26:14.718055+0000","last_undegraded":"2023-11-25T04:26:14.718055+0000","last_fullsized":"2023-11-25T04:26:14.718055+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'220","last_scrub_stamp":"2023-11-25T04:26:14.717897+0000","last_deep_scrub":"75'220","last_deep_scrub_stamp":"2023-11-25T04:26:14.717897+0000","last_clean_scrub_stamp":"2023-11-25T04:26:14.717897+0000","objects_scrubbed":62,"log_size":220,"log_dups_size":0,"ondisk_log_size":220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:43:17.788148+0000","scrub_duration":0.044136959000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":115501,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":163,"num_write_kb":243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":43,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:04.449342+0000","last_change":"2023-11-25T04:27:04.449359+0000","last_active":"2023-11-25T04:27:04.449342+0000","last_peered":"2023-11-25T04:27:04.449342+0000","last_clean":"2023-11-25T04:27:04.449342+0000","last_became_active":"2023-11-25T04:25:46.997391+0000","last_became_peered":"2023-11-25T04:25:46.997391+0000","last_unstale":"2023-11-25T04:27:04.449342+0000","last_undegraded":"2023-11-25T04:27:04.449342+0000","last_fullsized":"2023-11-25T04:27:04.449342+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:27:04.449261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:27:04.449261+0000","last_clean_scrub_stamp":"2023-11-25T04:27:04.449261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:55:52.982507+0000","scrub_duration":0.00176385,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"50'1","reported_seq":43,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:22.429640+0000","last_change":"2023-11-25T04:27:22.429663+0000","last_active":"2023-11-25T04:27:22.429640+0000","last_peered":"2023-11-25T04:27:22.429640+0000","last_clean":"2023-11-25T04:27:22.429640+0000","last_became_active":"2023-11-25T04:25:46.998132+0000","last_became_peered":"2023-11-25T04:25:46.998132+0000","last_unstale":"2023-11-25T04:27:22.429640+0000","last_undegraded":"2023-11-25T04:27:22.429640+0000","last_fullsized":"2023-11-25T04:27:22.429640+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'1","last_scrub_stamp":"2023-11-25T04:27:22.429551+0000","last_deep_scrub":"50'1","last_deep_scrub_stamp":"2023-11-25T04:27:22.429551+0000","last_clean_scrub_stamp":"2023-11-25T04:27:22.429551+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:41:27.142323+0000","scrub_duration":0.0015038580000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"75'206","reported_seq":285,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:19.694459+0000","last_change":"2023-11-25T04:26:19.694459+0000","last_active":"2023-11-25T04:26:19.694459+0000","last_peered":"2023-11-25T04:26:19.694459+0000","last_clean":"2023-11-25T04:26:19.694459+0000","last_became_active":"2023-11-25T04:22:47.036924+0000","last_became_peered":"2023-11-25T04:22:47.036924+0000","last_unstale":"2023-11-25T04:26:19.694459+0000","last_undegraded":"2023-11-25T04:26:19.694459+0000","last_fullsized":"2023-11-25T04:26:19.694459+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'206","last_scrub_stamp":"2023-11-25T04:26:19.694341+0000","last_deep_scrub":"75'206","last_deep_scrub_stamp":"2023-11-25T04:26:19.694341+0000","last_clean_scrub_stamp":"2023-11-25T04:26:19.694341+0000","objects_scrubbed":49,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:49:38.202017+0000","scrub_duration":0.031428719000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":93321,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":148,"num_write_kb":360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":43,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:26.443189+0000","last_change":"2023-11-25T04:27:26.443241+0000","last_active":"2023-11-25T04:27:26.443189+0000","last_peered":"2023-11-25T04:27:26.443189+0000","last_clean":"2023-11-25T04:27:26.443189+0000","last_became_active":"2023-11-25T04:25:46.999403+0000","last_became_peered":"2023-11-25T04:25:46.999403+0000","last_unstale":"2023-11-25T04:27:26.443189+0000","last_undegraded":"2023-11-25T04:27:26.443189+0000","last_fullsized":"2023-11-25T04:27:26.443189+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:27:26.443086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:27:26.443086+0000","last_clean_scrub_stamp":"2023-11-25T04:27:26.443086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:44:08.788259+0000","scrub_duration":0.001551729,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":82,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:23.259065+0000","last_change":"2023-11-25T04:27:23.259088+0000","last_active":"2023-11-25T04:27:23.259065+0000","last_peered":"2023-11-25T04:27:23.259065+0000","last_clean":"2023-11-25T04:27:23.259065+0000","last_became_active":"2023-11-25T04:22:41.999915+0000","last_became_peered":"2023-11-25T04:22:41.999915+0000","last_unstale":"2023-11-25T04:27:23.259065+0000","last_undegraded":"2023-11-25T04:27:23.259065+0000","last_fullsized":"2023-11-25T04:27:23.259065+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:27:23.258941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:27:23.258941+0000","last_clean_scrub_stamp":"2023-11-25T04:27:23.258941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:46:56.550165+0000","scrub_duration":0.0020338119999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"75'203","reported_seq":495,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:21.405576+0000","last_change":"2023-11-25T04:27:21.405602+0000","last_active":"2023-11-25T04:27:21.405576+0000","last_peered":"2023-11-25T04:27:21.405576+0000","last_clean":"2023-11-25T04:27:21.405576+0000","last_became_active":"2023-11-25T04:25:53.932586+0000","last_became_peered":"2023-11-25T04:25:53.932586+0000","last_unstale":"2023-11-25T04:27:21.405576+0000","last_undegraded":"2023-11-25T04:27:21.405576+0000","last_fullsized":"2023-11-25T04:27:21.405576+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'203","last_scrub_stamp":"2023-11-25T04:27:21.405481+0000","last_deep_scrub":"75'203","last_deep_scrub_stamp":"2023-11-25T04:27:21.405481+0000","last_clean_scrub_stamp":"2023-11-25T04:27:21.405481+0000","objects_scrubbed":51,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:22:14.998841+0000","scrub_duration":0.038507947000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":329327,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":151,"num_write_kb":471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":97,"num_bytes_recovered":329278,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":79,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:28.272266+0000","last_change":"2023-11-25T04:27:28.272286+0000","last_active":"2023-11-25T04:27:28.272266+0000","last_peered":"2023-11-25T04:27:28.272266+0000","last_clean":"2023-11-25T04:27:28.272266+0000","last_became_active":"2023-11-25T04:22:46.035405+0000","last_became_peered":"2023-11-25T04:22:46.035405+0000","last_unstale":"2023-11-25T04:27:28.272266+0000","last_undegraded":"2023-11-25T04:27:28.272266+0000","last_fullsized":"2023-11-25T04:27:28.272266+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:27:28.272187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:27:28.272187+0000","last_clean_scrub_stamp":"2023-11-25T04:27:28.272187+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:42:51.403345+0000","scrub_duration":0.001573315,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":82,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:49.454240+0000","last_change":"2023-11-25T04:26:49.454240+0000","last_active":"2023-11-25T04:26:49.454240+0000","last_peered":"2023-11-25T04:26:49.454240+0000","last_clean":"2023-11-25T04:26:49.454240+0000","last_became_active":"2023-11-25T04:22:41.998798+0000","last_became_peered":"2023-11-25T04:22:41.998798+0000","last_unstale":"2023-11-25T04:26:49.454240+0000","last_undegraded":"2023-11-25T04:26:49.454240+0000","last_fullsized":"2023-11-25T04:26:49.454240+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:26:49.454102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:26:49.454102+0000","last_clean_scrub_stamp":"2023-11-25T04:26:49.454102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:49:39.814155+0000","scrub_duration":0.0015686599999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"75'221","reported_seq":530,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:13.479673+0000","last_change":"2023-11-25T04:26:13.479673+0000","last_active":"2023-11-25T04:26:13.479673+0000","last_peered":"2023-11-25T04:26:13.479673+0000","last_clean":"2023-11-25T04:26:13.479673+0000","last_became_active":"2023-11-25T04:25:51.921858+0000","last_became_peered":"2023-11-25T04:25:51.921858+0000","last_unstale":"2023-11-25T04:26:13.479673+0000","last_undegraded":"2023-11-25T04:26:13.479673+0000","last_fullsized":"2023-11-25T04:26:13.479673+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'221","last_scrub_stamp":"2023-11-25T04:26:13.479549+0000","last_deep_scrub":"75'221","last_deep_scrub_stamp":"2023-11-25T04:26:13.479549+0000","last_clean_scrub_stamp":"2023-11-25T04:26:13.479549+0000","objects_scrubbed":49,"log_size":221,"log_dups_size":0,"ondisk_log_size":221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:44:56.142187+0000","scrub_duration":0.035520731999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":129965,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":166,"num_write_kb":290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":129965,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"18'2","reported_seq":73,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:09.671254+0000","last_change":"2023-11-25T04:22:46.034606+0000","last_active":"2023-11-25T04:26:09.671254+0000","last_peered":"2023-11-25T04:26:09.671254+0000","last_clean":"2023-11-25T04:26:09.671254+0000","last_became_active":"2023-11-25T04:22:46.034229+0000","last_became_peered":"2023-11-25T04:22:46.034229+0000","last_unstale":"2023-11-25T04:26:09.671254+0000","last_undegraded":"2023-11-25T04:26:09.671254+0000","last_fullsized":"2023-11-25T04:26:09.671254+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":82,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:10.500127+0000","last_change":"2023-11-25T04:27:10.500147+0000","last_active":"2023-11-25T04:27:10.500127+0000","last_peered":"2023-11-25T04:27:10.500127+0000","last_clean":"2023-11-25T04:27:10.500127+0000","last_became_active":"2023-11-25T04:22:41.999992+0000","last_became_peered":"2023-11-25T04:22:41.999992+0000","last_unstale":"2023-11-25T04:27:10.500127+0000","last_undegraded":"2023-11-25T04:27:10.500127+0000","last_fullsized":"2023-11-25T04:27:10.500127+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:27:10.500045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:27:10.500045+0000","last_clean_scrub_stamp":"2023-11-25T04:27:10.500045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:57:30.011678+0000","scrub_duration":0.0018675779999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"54'150","reported_seq":438,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:09.671351+0000","last_change":"2023-11-25T04:25:54.938479+0000","last_active":"2023-11-25T04:26:09.671351+0000","last_peered":"2023-11-25T04:26:09.671351+0000","last_clean":"2023-11-25T04:26:09.671351+0000","last_became_active":"2023-11-25T04:25:54.937838+0000","last_became_peered":"2023-11-25T04:25:54.937838+0000","last_unstale":"2023-11-25T04:26:09.671351+0000","last_undegraded":"2023-11-25T04:26:09.671351+0000","last_fullsized":"2023-11-25T04:26:09.671351+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":159585,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":154224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"54'14","reported_seq":101,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:09.671169+0000","last_change":"2023-11-25T04:25:47.026749+0000","last_active":"2023-11-25T04:26:09.671169+0000","last_peered":"2023-11-25T04:26:09.671169+0000","last_clean":"2023-11-25T04:26:09.671169+0000","last_became_active":"2023-11-25T04:25:46.997404+0000","last_became_peered":"2023-11-25T04:25:46.997404+0000","last_unstale":"2023-11-25T04:26:09.671169+0000","last_undegraded":"2023-11-25T04:26:09.671169+0000","last_fullsized":"2023-11-25T04:26:09.671169+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":43,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:00.412619+0000","last_change":"2023-11-25T04:27:00.412639+0000","last_active":"2023-11-25T04:27:00.412619+0000","last_peered":"2023-11-25T04:27:00.412619+0000","last_clean":"2023-11-25T04:27:00.412619+0000","last_became_active":"2023-11-25T04:25:46.994834+0000","last_became_peered":"2023-11-25T04:25:46.994834+0000","last_unstale":"2023-11-25T04:27:00.412619+0000","last_undegraded":"2023-11-25T04:27:00.412619+0000","last_fullsized":"2023-11-25T04:27:00.412619+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:27:00.412529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:27:00.412529+0000","last_clean_scrub_stamp":"2023-11-25T04:27:00.412529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:30:28.354258+0000","scrub_duration":0.0016774050000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"75'191","reported_seq":475,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:14.466619+0000","last_change":"2023-11-25T04:26:14.466619+0000","last_active":"2023-11-25T04:26:14.466619+0000","last_peered":"2023-11-25T04:26:14.466619+0000","last_clean":"2023-11-25T04:26:14.466619+0000","last_became_active":"2023-11-25T04:25:49.921465+0000","last_became_peered":"2023-11-25T04:25:49.921465+0000","last_unstale":"2023-11-25T04:26:14.466619+0000","last_undegraded":"2023-11-25T04:26:14.466619+0000","last_fullsized":"2023-11-25T04:26:14.466619+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'191","last_scrub_stamp":"2023-11-25T04:26:14.466515+0000","last_deep_scrub":"75'191","last_deep_scrub_stamp":"2023-11-25T04:26:14.466515+0000","last_clean_scrub_stamp":"2023-11-25T04:26:14.466515+0000","objects_scrubbed":56,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:04:34.752431+0000","scrub_duration":0.037056879000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":99503,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":148,"num_write_kb":250,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":93,"num_bytes_recovered":99503,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"75'218","reported_seq":296,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:59.486464+0000","last_change":"2023-11-25T04:26:59.486489+0000","last_active":"2023-11-25T04:26:59.486464+0000","last_peered":"2023-11-25T04:26:59.486464+0000","last_clean":"2023-11-25T04:26:59.486464+0000","last_became_active":"2023-11-25T04:25:51.922070+0000","last_became_peered":"2023-11-25T04:25:51.922070+0000","last_unstale":"2023-11-25T04:26:59.486464+0000","last_undegraded":"2023-11-25T04:26:59.486464+0000","last_fullsized":"2023-11-25T04:26:59.486464+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'218","last_scrub_stamp":"2023-11-25T04:26:59.486387+0000","last_deep_scrub":"75'218","last_deep_scrub_stamp":"2023-11-25T04:26:59.486387+0000","last_clean_scrub_stamp":"2023-11-25T04:26:59.486387+0000","objects_scrubbed":52,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:32:01.142142+0000","scrub_duration":0.040859969000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":185321,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":172,"num_write_kb":312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":164176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"75'54","reported_seq":66,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:58.474448+0000","last_change":"2023-11-25T04:26:58.474489+0000","last_active":"2023-11-25T04:26:58.474448+0000","last_peered":"2023-11-25T04:26:58.474448+0000","last_clean":"2023-11-25T04:26:58.474448+0000","last_became_active":"2023-11-25T04:25:46.994997+0000","last_became_peered":"2023-11-25T04:25:46.994997+0000","last_unstale":"2023-11-25T04:26:58.474448+0000","last_undegraded":"2023-11-25T04:26:58.474448+0000","last_fullsized":"2023-11-25T04:26:58.474448+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'54","last_scrub_stamp":"2023-11-25T04:26:58.474367+0000","last_deep_scrub":"75'54","last_deep_scrub_stamp":"2023-11-25T04:26:58.474367+0000","last_clean_scrub_stamp":"2023-11-25T04:26:58.474367+0000","objects_scrubbed":4,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:32:12.109149+0000","scrub_duration":0.0152154,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":4119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4190183,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"75'199","reported_seq":278,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:43.858609+0000","last_change":"2023-11-25T04:26:43.858609+0000","last_active":"2023-11-25T04:26:43.858609+0000","last_peered":"2023-11-25T04:26:43.858609+0000","last_clean":"2023-11-25T04:26:43.858609+0000","last_became_active":"2023-11-25T04:22:47.036821+0000","last_became_peered":"2023-11-25T04:22:47.036821+0000","last_unstale":"2023-11-25T04:26:43.858609+0000","last_undegraded":"2023-11-25T04:26:43.858609+0000","last_fullsized":"2023-11-25T04:26:43.858609+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'199","last_scrub_stamp":"2023-11-25T04:26:43.858473+0000","last_deep_scrub":"75'199","last_deep_scrub_stamp":"2023-11-25T04:26:43.858473+0000","last_clean_scrub_stamp":"2023-11-25T04:26:43.858473+0000","objects_scrubbed":46,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:28:32.758038+0000","scrub_duration":0.029966083000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":101117,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":161,"num_write_kb":242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"75'24","reported_seq":118,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:39.535607+0000","last_change":"2023-11-25T04:26:39.535658+0000","last_active":"2023-11-25T04:26:39.535607+0000","last_peered":"2023-11-25T04:26:39.535607+0000","last_clean":"2023-11-25T04:26:39.535607+0000","last_became_active":"2023-11-25T04:25:47.000478+0000","last_became_peered":"2023-11-25T04:25:47.000478+0000","last_unstale":"2023-11-25T04:26:39.535607+0000","last_undegraded":"2023-11-25T04:26:39.535607+0000","last_fullsized":"2023-11-25T04:26:39.535607+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'24","last_scrub_stamp":"2023-11-25T04:26:39.535504+0000","last_deep_scrub":"75'24","last_deep_scrub_stamp":"2023-11-25T04:26:39.535504+0000","last_clean_scrub_stamp":"2023-11-25T04:26:39.535504+0000","objects_scrubbed":2,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:50:41.041415+0000","scrub_duration":0.0040188020000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":90,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":43,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:54.349677+0000","last_change":"2023-11-25T04:26:54.349711+0000","last_active":"2023-11-25T04:26:54.349677+0000","last_peered":"2023-11-25T04:26:54.349677+0000","last_clean":"2023-11-25T04:26:54.349677+0000","last_became_active":"2023-11-25T04:25:46.994525+0000","last_became_peered":"2023-11-25T04:25:46.994525+0000","last_unstale":"2023-11-25T04:26:54.349677+0000","last_undegraded":"2023-11-25T04:26:54.349677+0000","last_fullsized":"2023-11-25T04:26:54.349677+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:26:54.349592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:26:54.349592+0000","last_clean_scrub_stamp":"2023-11-25T04:26:54.349592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:34:14.985604+0000","scrub_duration":0.0018546229999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"54'177","reported_seq":487,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:09.671445+0000","last_change":"2023-11-25T04:25:51.922466+0000","last_active":"2023-11-25T04:26:09.671445+0000","last_peered":"2023-11-25T04:26:09.671445+0000","last_clean":"2023-11-25T04:26:09.671445+0000","last_became_active":"2023-11-25T04:25:51.921938+0000","last_became_peered":"2023-11-25T04:25:51.921938+0000","last_unstale":"2023-11-25T04:26:09.671445+0000","last_undegraded":"2023-11-25T04:26:09.671445+0000","last_fullsized":"2023-11-25T04:26:09.671445+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":522494,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":168,"num_write_kb":749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":111,"num_bytes_recovered":522494,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"46'2","reported_seq":91,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:51.691510+0000","last_change":"2023-11-25T04:26:51.691533+0000","last_active":"2023-11-25T04:26:51.691510+0000","last_peered":"2023-11-25T04:26:51.691510+0000","last_clean":"2023-11-25T04:26:51.691510+0000","last_became_active":"2023-11-25T04:25:46.998508+0000","last_became_peered":"2023-11-25T04:25:46.998508+0000","last_unstale":"2023-11-25T04:26:51.691510+0000","last_undegraded":"2023-11-25T04:26:51.691510+0000","last_fullsized":"2023-11-25T04:26:51.691510+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"46'2","last_scrub_stamp":"2023-11-25T04:26:51.691391+0000","last_deep_scrub":"46'2","last_deep_scrub_stamp":"2023-11-25T04:26:51.691391+0000","last_clean_scrub_stamp":"2023-11-25T04:26:51.691391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:31:13.002647+0000","scrub_duration":0.0021417300000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.10","version":"75'209","reported_seq":289,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:02.586496+0000","last_change":"2023-11-25T04:27:02.586496+0000","last_active":"2023-11-25T04:27:02.586496+0000","last_peered":"2023-11-25T04:27:02.586496+0000","last_clean":"2023-11-25T04:27:02.586496+0000","last_became_active":"2023-11-25T04:22:47.037149+0000","last_became_peered":"2023-11-25T04:22:47.037149+0000","last_unstale":"2023-11-25T04:27:02.586496+0000","last_undegraded":"2023-11-25T04:27:02.586496+0000","last_fullsized":"2023-11-25T04:27:02.586496+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'209","last_scrub_stamp":"2023-11-25T04:27:02.586379+0000","last_deep_scrub":"75'209","last_deep_scrub_stamp":"2023-11-25T04:27:02.586379+0000","last_clean_scrub_stamp":"2023-11-25T04:27:02.586379+0000","objects_scrubbed":56,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:55:23.713881+0000","scrub_duration":0.037826797000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":218073,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1f","version":"75'214","reported_seq":518,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:16.412889+0000","last_change":"2023-11-25T04:27:16.412922+0000","last_active":"2023-11-25T04:27:16.412889+0000","last_peered":"2023-11-25T04:27:16.412889+0000","last_clean":"2023-11-25T04:27:16.412889+0000","last_became_active":"2023-11-25T04:25:49.921458+0000","last_became_peered":"2023-11-25T04:25:49.921458+0000","last_unstale":"2023-11-25T04:27:16.412889+0000","last_undegraded":"2023-11-25T04:27:16.412889+0000","last_fullsized":"2023-11-25T04:27:16.412889+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'214","last_scrub_stamp":"2023-11-25T04:27:16.412805+0000","last_deep_scrub":"75'214","last_deep_scrub_stamp":"2023-11-25T04:27:16.412805+0000","last_clean_scrub_stamp":"2023-11-25T04:27:16.412805+0000","objects_scrubbed":58,"log_size":214,"log_dups_size":0,"ondisk_log_size":214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:05:28.377468+0000","scrub_duration":0.044986529999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":107712,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":103,"num_bytes_recovered":107703,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"48'1","reported_seq":79,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:27.250407+0000","last_change":"2023-11-25T04:27:27.250447+0000","last_active":"2023-11-25T04:27:27.250407+0000","last_peered":"2023-11-25T04:27:27.250407+0000","last_clean":"2023-11-25T04:27:27.250407+0000","last_became_active":"2023-11-25T04:22:46.035495+0000","last_became_peered":"2023-11-25T04:22:46.035495+0000","last_unstale":"2023-11-25T04:27:27.250407+0000","last_undegraded":"2023-11-25T04:27:27.250407+0000","last_fullsized":"2023-11-25T04:27:27.250407+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"48'1","last_scrub_stamp":"2023-11-25T04:27:27.250306+0000","last_deep_scrub":"48'1","last_deep_scrub_stamp":"2023-11-25T04:27:27.250306+0000","last_clean_scrub_stamp":"2023-11-25T04:27:27.250306+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:52:37.111984+0000","scrub_duration":0.001682764,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"54'152","reported_seq":432,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:09.671534+0000","last_change":"2023-11-25T04:25:50.923882+0000","last_active":"2023-11-25T04:26:09.671534+0000","last_peered":"2023-11-25T04:26:09.671534+0000","last_clean":"2023-11-25T04:26:09.671534+0000","last_became_active":"2023-11-25T04:25:50.923375+0000","last_became_peered":"2023-11-25T04:25:50.923375+0000","last_unstale":"2023-11-25T04:26:09.671534+0000","last_undegraded":"2023-11-25T04:26:09.671534+0000","last_fullsized":"2023-11-25T04:26:09.671534+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":76435,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":76435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":89,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:23.683261+0000","last_change":"2023-11-25T04:26:23.683288+0000","last_active":"2023-11-25T04:26:23.683261+0000","last_peered":"2023-11-25T04:26:23.683261+0000","last_clean":"2023-11-25T04:26:23.683261+0000","last_became_active":"2023-11-25T04:25:46.994871+0000","last_became_peered":"2023-11-25T04:25:46.994871+0000","last_unstale":"2023-11-25T04:26:23.683261+0000","last_undegraded":"2023-11-25T04:26:23.683261+0000","last_fullsized":"2023-11-25T04:26:23.683261+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:26:23.683176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:26:23.683176+0000","last_clean_scrub_stamp":"2023-11-25T04:26:23.683176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:31:54.450606+0000","scrub_duration":0.00162587,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"75'207","reported_seq":491,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:23.401047+0000","last_change":"2023-11-25T04:27:23.401113+0000","last_active":"2023-11-25T04:27:23.401047+0000","last_peered":"2023-11-25T04:27:23.401047+0000","last_clean":"2023-11-25T04:27:23.401047+0000","last_became_active":"2023-11-25T04:25:52.925038+0000","last_became_peered":"2023-11-25T04:25:52.925038+0000","last_unstale":"2023-11-25T04:27:23.401047+0000","last_undegraded":"2023-11-25T04:27:23.401047+0000","last_fullsized":"2023-11-25T04:27:23.401047+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'207","last_scrub_stamp":"2023-11-25T04:27:23.400949+0000","last_deep_scrub":"75'207","last_deep_scrub_stamp":"2023-11-25T04:27:23.400949+0000","last_clean_scrub_stamp":"2023-11-25T04:27:23.400949+0000","objects_scrubbed":49,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:20:49.215409+0000","scrub_duration":0.040233620999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":209500,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":139,"num_write_kb":351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":209500,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1a","version":"75'5","reported_seq":83,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:51.508201+0000","last_change":"2023-11-25T04:26:51.508225+0000","last_active":"2023-11-25T04:26:51.508201+0000","last_peered":"2023-11-25T04:26:51.508201+0000","last_clean":"2023-11-25T04:26:51.508201+0000","last_became_active":"2023-11-25T04:22:46.033944+0000","last_became_peered":"2023-11-25T04:22:46.033944+0000","last_unstale":"2023-11-25T04:26:51.508201+0000","last_undegraded":"2023-11-25T04:26:51.508201+0000","last_fullsized":"2023-11-25T04:26:51.508201+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'5","last_scrub_stamp":"2023-11-25T04:26:51.508119+0000","last_deep_scrub":"75'5","last_deep_scrub_stamp":"2023-11-25T04:26:51.508119+0000","last_clean_scrub_stamp":"2023-11-25T04:26:51.508119+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:55:12.635621+0000","scrub_duration":0.0083650619999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"75'218","reported_seq":297,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:28.540048+0000","last_change":"2023-11-25T04:26:28.540082+0000","last_active":"2023-11-25T04:26:28.540048+0000","last_peered":"2023-11-25T04:26:28.540048+0000","last_clean":"2023-11-25T04:26:28.540048+0000","last_became_active":"2023-11-25T04:22:47.039218+0000","last_became_peered":"2023-11-25T04:22:47.039218+0000","last_unstale":"2023-11-25T04:26:28.540048+0000","last_undegraded":"2023-11-25T04:26:28.540048+0000","last_fullsized":"2023-11-25T04:26:28.540048+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'218","last_scrub_stamp":"2023-11-25T04:26:28.539970+0000","last_deep_scrub":"75'218","last_deep_scrub_stamp":"2023-11-25T04:26:28.539970+0000","last_clean_scrub_stamp":"2023-11-25T04:26:28.539970+0000","objects_scrubbed":55,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:46:27.913704+0000","scrub_duration":0.040559401000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":165883,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"75'10","reported_seq":104,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:20.499200+0000","last_change":"2023-11-25T04:26:20.499236+0000","last_active":"2023-11-25T04:26:20.499200+0000","last_peered":"2023-11-25T04:26:20.499200+0000","last_clean":"2023-11-25T04:26:20.499200+0000","last_became_active":"2023-11-25T04:25:46.999998+0000","last_became_peered":"2023-11-25T04:25:46.999998+0000","last_unstale":"2023-11-25T04:26:20.499200+0000","last_undegraded":"2023-11-25T04:26:20.499200+0000","last_fullsized":"2023-11-25T04:26:20.499200+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'10","last_scrub_stamp":"2023-11-25T04:26:20.499093+0000","last_deep_scrub":"75'10","last_deep_scrub_stamp":"2023-11-25T04:26:20.499093+0000","last_clean_scrub_stamp":"2023-11-25T04:26:20.499093+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:35:51.569332+0000","scrub_duration":0.0094161829999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":4101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4194304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"75'212","reported_seq":291,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:39.477807+0000","last_change":"2023-11-25T04:26:39.477840+0000","last_active":"2023-11-25T04:26:39.477807+0000","last_peered":"2023-11-25T04:26:39.477807+0000","last_clean":"2023-11-25T04:26:39.477807+0000","last_became_active":"2023-11-25T04:25:50.922164+0000","last_became_peered":"2023-11-25T04:25:50.922164+0000","last_unstale":"2023-11-25T04:26:39.477807+0000","last_undegraded":"2023-11-25T04:26:39.477807+0000","last_fullsized":"2023-11-25T04:26:39.477807+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'212","last_scrub_stamp":"2023-11-25T04:26:39.477723+0000","last_deep_scrub":"75'212","last_deep_scrub_stamp":"2023-11-25T04:26:39.477723+0000","last_clean_scrub_stamp":"2023-11-25T04:26:39.477723+0000","objects_scrubbed":63,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:34:17.609339+0000","scrub_duration":0.044092971000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":179506,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":155,"num_write_kb":317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":179506,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"75'3","reported_seq":47,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:25.424173+0000","last_change":"2023-11-25T04:27:25.424197+0000","last_active":"2023-11-25T04:27:25.424173+0000","last_peered":"2023-11-25T04:27:25.424173+0000","last_clean":"2023-11-25T04:27:25.424173+0000","last_became_active":"2023-11-25T04:25:46.995160+0000","last_became_peered":"2023-11-25T04:25:46.995160+0000","last_unstale":"2023-11-25T04:27:25.424173+0000","last_undegraded":"2023-11-25T04:27:25.424173+0000","last_fullsized":"2023-11-25T04:27:25.424173+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'3","last_scrub_stamp":"2023-11-25T04:27:25.424088+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2023-11-25T04:27:25.424088+0000","last_clean_scrub_stamp":"2023-11-25T04:27:25.424088+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:47:26.636974+0000","scrub_duration":0.0032298869999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1a","version":"54'162","reported_seq":227,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986038+0000","last_change":"2023-11-25T04:25:54.939182+0000","last_active":"2023-11-25T04:26:00.986038+0000","last_peered":"2023-11-25T04:26:00.986038+0000","last_clean":"2023-11-25T04:26:00.986038+0000","last_became_active":"2023-11-25T04:25:54.937821+0000","last_became_peered":"2023-11-25T04:25:54.937821+0000","last_unstale":"2023-11-25T04:26:00.986038+0000","last_undegraded":"2023-11-25T04:26:00.986038+0000","last_fullsized":"2023-11-25T04:26:00.986038+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":134672,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":169,"num_write_kb":311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":126291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"75'62","reported_seq":158,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:30.554569+0000","last_change":"2023-11-25T04:26:30.554596+0000","last_active":"2023-11-25T04:26:30.554569+0000","last_peered":"2023-11-25T04:26:30.554569+0000","last_clean":"2023-11-25T04:26:30.554569+0000","last_became_active":"2023-11-25T04:25:47.000722+0000","last_became_peered":"2023-11-25T04:25:47.000722+0000","last_unstale":"2023-11-25T04:26:30.554569+0000","last_undegraded":"2023-11-25T04:26:30.554569+0000","last_fullsized":"2023-11-25T04:26:30.554569+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'62","last_scrub_stamp":"2023-11-25T04:26:30.554482+0000","last_deep_scrub":"75'62","last_deep_scrub_stamp":"2023-11-25T04:26:30.554482+0000","last_clean_scrub_stamp":"2023-11-25T04:26:30.554482+0000","objects_scrubbed":3,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:38:15.140403+0000","scrub_duration":0.015921703999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8388608,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"75'221","reported_seq":300,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:55.521285+0000","last_change":"2023-11-25T04:26:55.521345+0000","last_active":"2023-11-25T04:26:55.521285+0000","last_peered":"2023-11-25T04:26:55.521285+0000","last_clean":"2023-11-25T04:26:55.521285+0000","last_became_active":"2023-11-25T04:22:47.036406+0000","last_became_peered":"2023-11-25T04:22:47.036406+0000","last_unstale":"2023-11-25T04:26:55.521285+0000","last_undegraded":"2023-11-25T04:26:55.521285+0000","last_fullsized":"2023-11-25T04:26:55.521285+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'221","last_scrub_stamp":"2023-11-25T04:26:55.521198+0000","last_deep_scrub":"75'221","last_deep_scrub_stamp":"2023-11-25T04:26:55.521198+0000","last_clean_scrub_stamp":"2023-11-25T04:26:55.521198+0000","objects_scrubbed":58,"log_size":221,"log_dups_size":0,"ondisk_log_size":221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:17:51.455919+0000","scrub_duration":0.041323743000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":527891,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":163,"num_write_kb":687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1e","version":"46'2","reported_seq":73,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:09.671303+0000","last_change":"2023-11-25T04:22:46.035659+0000","last_active":"2023-11-25T04:26:09.671303+0000","last_peered":"2023-11-25T04:26:09.671303+0000","last_clean":"2023-11-25T04:26:09.671303+0000","last_became_active":"2023-11-25T04:22:46.035113+0000","last_became_peered":"2023-11-25T04:22:46.035113+0000","last_unstale":"2023-11-25T04:26:09.671303+0000","last_undegraded":"2023-11-25T04:26:09.671303+0000","last_fullsized":"2023-11-25T04:26:09.671303+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:45.004766+0000","last_clean_scrub_stamp":"2023-11-25T04:22:45.004766+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.18","version":"54'195","reported_seq":260,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986052+0000","last_change":"2023-11-25T04:25:54.938573+0000","last_active":"2023-11-25T04:26:00.986052+0000","last_peered":"2023-11-25T04:26:00.986052+0000","last_clean":"2023-11-25T04:26:00.986052+0000","last_became_active":"2023-11-25T04:25:54.937444+0000","last_became_peered":"2023-11-25T04:25:54.937444+0000","last_unstale":"2023-11-25T04:26:00.986052+0000","last_undegraded":"2023-11-25T04:26:00.986052+0000","last_fullsized":"2023-11-25T04:26:00.986052+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":128641,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":185,"num_write_kb":408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":122,"num_bytes_recovered":128641,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"75'20","reported_seq":115,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:49.762359+0000","last_change":"2023-11-25T04:26:49.762359+0000","last_active":"2023-11-25T04:26:49.762359+0000","last_peered":"2023-11-25T04:26:49.762359+0000","last_clean":"2023-11-25T04:26:49.762359+0000","last_became_active":"2023-11-25T04:25:46.998870+0000","last_became_peered":"2023-11-25T04:25:46.998870+0000","last_unstale":"2023-11-25T04:26:49.762359+0000","last_undegraded":"2023-11-25T04:26:49.762359+0000","last_fullsized":"2023-11-25T04:26:49.762359+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'20","last_scrub_stamp":"2023-11-25T04:26:49.762253+0000","last_deep_scrub":"75'20","last_deep_scrub_stamp":"2023-11-25T04:26:49.762253+0000","last_clean_scrub_stamp":"2023-11-25T04:26:49.762253+0000","objects_scrubbed":3,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:38:21.361584+0000","scrub_duration":0.0095441299999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":4103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4194304,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":26330855,"num_objects":3447,"num_object_clones":0,"num_object_copies":6894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10423,"num_write_kb":38550,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4189,"num_bytes_recovered":12749181,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":72376320,"data_stored":52324046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12916,"ondisk_log_size":12916,"up":128,"acting":128,"num_store_stats":3},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":40885548,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":35,"num_read_kb":334,"num_write":435,"num_write_kb":40462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":32491439,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":81833984,"data_stored":81771096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5548,"internal_metadata":0},"log_size":464,"ondisk_log_size":464,"up":64,"acting":64,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738432,"num_pgs":72,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81524,"kb_used_data":46188,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94290316,"statfs":{"total":96636764160,"available":96553283584,"internally_reserved":0,"allocated":47296512,"data_stored":39787071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 25 04:26:49 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.46600000000000003}]},{"osd":1,"last update":"Sat Nov 25 04:26:45 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.436,"15min":0.44600000000000001},"min":{"1min":0.23300000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.61299999999999999,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.437,"15min":0.45100000000000001},"min":{"1min":0.249,"5min":0.184,"15min":0.184},"max":{"1min":0.63,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.50800000000000001}]}]},{"osd":1,"up_from":8,"seq":34359738432,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":95196,"kb_used_data":55700,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94276644,"statfs":{"total":96636764160,"available":96539283456,"internally_reserved":0,"allocated":57036800,"data_stored":50261554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 25 04:26:46 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.23999999999999999},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.255}]},{"osd":2,"last update":"Sat Nov 25 04:26:42 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.41999999999999998,"15min":0.41599999999999998},"min":{"1min":0.28299999999999997,"5min":0.182,"15min":0.182},"max":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.41899999999999998,"15min":0.41099999999999998},"min":{"1min":0.27000000000000002,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.35699999999999998}]}]},{"osd":0,"up_from":60,"seq":257698037783,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":91772,"kb_used_data":51764,"kb_used_omap":6,"kb_used_meta":39993,"kb_avail":94280068,"statfs":{"total":96636764160,"available":96542789632,"internally_reserved":0,"allocated":53006336,"data_stored":46141107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7143,"internal_metadata":40952857},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Nov 25 04:26:51 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.41999999999999998}]},{"osd":2,"last update":"Sat Nov 25 04:26:51 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.44900000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32530432,"data_stored":32496940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5548,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":29380608,"data_stored":29360982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":19922944,"data_stored":19913174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":19234816,"data_stored":12749181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":26415104,"data_stored":20005586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":26726400,"data_stored":19569279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-25T04:27:31.609 INFO:tasks.ceph:pgid 4.3a last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:27:31.609 INFO:tasks.ceph:pgid 4.32 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:27:31.610 INFO:tasks.ceph:pgid 4.26 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:27:31.610 INFO:tasks.ceph:pgid 4.20 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:27:31.610 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:27:31.610 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:27:31.611 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:27:31.611 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:27:31.611 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:27:31.611 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:27:31.612 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:27:31.612 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:27:31.612 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:27:31.612 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:27:31.613 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:27:31.613 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2023-11-25T04:22:45.004766+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=45, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:27:31.613 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:27:31.613 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-11-25T04:27:51.614 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-25T04:27:51.932 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:27:51.935 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2023-11-25T04:27:51.940 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":232,"stamp":"2023-11-25T04:27:51.578028+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":67806771,"num_objects":3481,"num_object_clones":0,"num_object_copies":6962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3481,"num_whiteouts":0,"num_read":171,"num_read_kb":452,"num_write":11061,"num_write_kb":82472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4201,"num_bytes_recovered":45240620,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14019,"ondisk_log_size":14019,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":274060,"kb_used_data":153652,"kb_used_omap":10,"kb_used_meta":120373,"kb_avail":282841460,"statfs":{"total":289910292480,"available":289629655040,"internally_reserved":0,"allocated":157339648,"data_stored":136189732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10333,"internal_metadata":123262883},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003475"},"pg_stats":[{"pgid":"4.3f","version":"75'212","reported_seq":291,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:48.487625+0000","last_change":"2023-11-25T04:26:48.487625+0000","last_active":"2023-11-25T04:26:48.487625+0000","last_peered":"2023-11-25T04:26:48.487625+0000","last_clean":"2023-11-25T04:26:48.487625+0000","last_became_active":"2023-11-25T04:22:47.039490+0000","last_became_peered":"2023-11-25T04:22:47.039490+0000","last_unstale":"2023-11-25T04:26:48.487625+0000","last_undegraded":"2023-11-25T04:26:48.487625+0000","last_fullsized":"2023-11-25T04:26:48.487625+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'212","last_scrub_stamp":"2023-11-25T04:26:48.487485+0000","last_deep_scrub":"75'212","last_deep_scrub_stamp":"2023-11-25T04:26:48.487485+0000","last_clean_scrub_stamp":"2023-11-25T04:26:48.487485+0000","objects_scrubbed":46,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:02:55.160351+0000","scrub_duration":0.031121083000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":577866,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":174,"num_write_kb":756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3e","version":"75'257","reported_seq":336,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:16.435276+0000","last_change":"2023-11-25T04:26:16.435276+0000","last_active":"2023-11-25T04:26:16.435276+0000","last_peered":"2023-11-25T04:26:16.435276+0000","last_clean":"2023-11-25T04:26:16.435276+0000","last_became_active":"2023-11-25T04:22:47.040511+0000","last_became_peered":"2023-11-25T04:22:47.040511+0000","last_unstale":"2023-11-25T04:26:16.435276+0000","last_undegraded":"2023-11-25T04:26:16.435276+0000","last_fullsized":"2023-11-25T04:26:16.435276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'257","last_scrub_stamp":"2023-11-25T04:26:16.435161+0000","last_deep_scrub":"75'257","last_deep_scrub_stamp":"2023-11-25T04:26:16.435161+0000","last_clean_scrub_stamp":"2023-11-25T04:26:16.435161+0000","objects_scrubbed":45,"log_size":257,"log_dups_size":0,"ondisk_log_size":257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:13:07.641904+0000","scrub_duration":0.036488224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":461579,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":236,"num_write_kb":841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3d","version":"75'209","reported_seq":286,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:12.330801+0000","last_change":"2023-11-25T04:26:12.330801+0000","last_active":"2023-11-25T04:26:12.330801+0000","last_peered":"2023-11-25T04:26:12.330801+0000","last_clean":"2023-11-25T04:26:12.330801+0000","last_became_active":"2023-11-25T04:25:52.923459+0000","last_became_peered":"2023-11-25T04:25:52.923459+0000","last_unstale":"2023-11-25T04:26:12.330801+0000","last_undegraded":"2023-11-25T04:26:12.330801+0000","last_fullsized":"2023-11-25T04:26:12.330801+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'209","last_scrub_stamp":"2023-11-25T04:26:12.330672+0000","last_deep_scrub":"75'209","last_deep_scrub_stamp":"2023-11-25T04:26:12.330672+0000","last_clean_scrub_stamp":"2023-11-25T04:26:12.330672+0000","objects_scrubbed":42,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:49:33.381569+0000","scrub_duration":0.027938688999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":145844,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":175,"num_write_kb":312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":138765,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"75'172","reported_seq":250,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:57.449017+0000","last_change":"2023-11-25T04:26:57.449017+0000","last_active":"2023-11-25T04:26:57.449017+0000","last_peered":"2023-11-25T04:26:57.449017+0000","last_clean":"2023-11-25T04:26:57.449017+0000","last_became_active":"2023-11-25T04:25:51.921633+0000","last_became_peered":"2023-11-25T04:25:51.921633+0000","last_unstale":"2023-11-25T04:26:57.449017+0000","last_undegraded":"2023-11-25T04:26:57.449017+0000","last_fullsized":"2023-11-25T04:26:57.449017+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'172","last_scrub_stamp":"2023-11-25T04:26:57.448886+0000","last_deep_scrub":"75'172","last_deep_scrub_stamp":"2023-11-25T04:26:57.448886+0000","last_clean_scrub_stamp":"2023-11-25T04:26:57.448886+0000","objects_scrubbed":48,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:58:16.336649+0000","scrub_duration":0.030012414000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61303,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119,"num_write_kb":161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":61303,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"75'236","reported_seq":316,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:03.442984+0000","last_change":"2023-11-25T04:27:03.442984+0000","last_active":"2023-11-25T04:27:03.442984+0000","last_peered":"2023-11-25T04:27:03.442984+0000","last_clean":"2023-11-25T04:27:03.442984+0000","last_became_active":"2023-11-25T04:22:47.039165+0000","last_became_peered":"2023-11-25T04:22:47.039165+0000","last_unstale":"2023-11-25T04:27:03.442984+0000","last_undegraded":"2023-11-25T04:27:03.442984+0000","last_fullsized":"2023-11-25T04:27:03.442984+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'236","last_scrub_stamp":"2023-11-25T04:27:03.442858+0000","last_deep_scrub":"75'236","last_deep_scrub_stamp":"2023-11-25T04:27:03.442858+0000","last_clean_scrub_stamp":"2023-11-25T04:27:03.442858+0000","objects_scrubbed":67,"log_size":236,"log_dups_size":0,"ondisk_log_size":236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:08:36.083041+0000","scrub_duration":0.043065032000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":118227,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":185,"num_write_kb":351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3a","version":"75'220","reported_seq":296,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:37.556263+0000","last_change":"2023-11-25T04:27:37.556287+0000","last_active":"2023-11-25T04:27:37.556263+0000","last_peered":"2023-11-25T04:27:37.556263+0000","last_clean":"2023-11-25T04:27:37.556263+0000","last_became_active":"2023-11-25T04:25:53.931982+0000","last_became_peered":"2023-11-25T04:25:53.931982+0000","last_unstale":"2023-11-25T04:27:37.556263+0000","last_undegraded":"2023-11-25T04:27:37.556263+0000","last_fullsized":"2023-11-25T04:27:37.556263+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'220","last_scrub_stamp":"2023-11-25T04:27:37.556170+0000","last_deep_scrub":"75'220","last_deep_scrub_stamp":"2023-11-25T04:27:37.556170+0000","last_clean_scrub_stamp":"2023-11-25T04:27:37.556170+0000","objects_scrubbed":51,"log_size":220,"log_dups_size":0,"ondisk_log_size":220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:16:43.691698+0000","scrub_duration":0.045038175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100304,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":161,"num_write_kb":424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":100304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"75'228","reported_seq":308,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:27.488226+0000","last_change":"2023-11-25T04:26:27.488269+0000","last_active":"2023-11-25T04:26:27.488226+0000","last_peered":"2023-11-25T04:26:27.488226+0000","last_clean":"2023-11-25T04:26:27.488226+0000","last_became_active":"2023-11-25T04:25:49.919354+0000","last_became_peered":"2023-11-25T04:25:49.919354+0000","last_unstale":"2023-11-25T04:26:27.488226+0000","last_undegraded":"2023-11-25T04:26:27.488226+0000","last_fullsized":"2023-11-25T04:26:27.488226+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'228","last_scrub_stamp":"2023-11-25T04:26:27.488145+0000","last_deep_scrub":"75'228","last_deep_scrub_stamp":"2023-11-25T04:26:27.488145+0000","last_clean_scrub_stamp":"2023-11-25T04:26:27.488145+0000","objects_scrubbed":60,"log_size":228,"log_dups_size":0,"ondisk_log_size":228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:22:24.257272+0000","scrub_duration":0.042766632999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":124558,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":167,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":111,"num_bytes_recovered":124533,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.38","version":"75'199","reported_seq":278,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:28.501531+0000","last_change":"2023-11-25T04:26:28.501582+0000","last_active":"2023-11-25T04:26:28.501531+0000","last_peered":"2023-11-25T04:26:28.501531+0000","last_clean":"2023-11-25T04:26:28.501531+0000","last_became_active":"2023-11-25T04:25:49.919236+0000","last_became_peered":"2023-11-25T04:25:49.919236+0000","last_unstale":"2023-11-25T04:26:28.501531+0000","last_undegraded":"2023-11-25T04:26:28.501531+0000","last_fullsized":"2023-11-25T04:26:28.501531+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'199","last_scrub_stamp":"2023-11-25T04:26:28.501441+0000","last_deep_scrub":"75'199","last_deep_scrub_stamp":"2023-11-25T04:26:28.501441+0000","last_clean_scrub_stamp":"2023-11-25T04:26:28.501441+0000","objects_scrubbed":47,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:21:13.937490+0000","scrub_duration":0.030866581000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":120147,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":120121,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.37","version":"75'220","reported_seq":300,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:28.772532+0000","last_change":"2023-11-25T04:26:28.772532+0000","last_active":"2023-11-25T04:26:28.772532+0000","last_peered":"2023-11-25T04:26:28.772532+0000","last_clean":"2023-11-25T04:26:28.772532+0000","last_became_active":"2023-11-25T04:22:47.037328+0000","last_became_peered":"2023-11-25T04:22:47.037328+0000","last_unstale":"2023-11-25T04:26:28.772532+0000","last_undegraded":"2023-11-25T04:26:28.772532+0000","last_fullsized":"2023-11-25T04:26:28.772532+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'220","last_scrub_stamp":"2023-11-25T04:26:28.772426+0000","last_deep_scrub":"75'220","last_deep_scrub_stamp":"2023-11-25T04:26:28.772426+0000","last_clean_scrub_stamp":"2023-11-25T04:26:28.772426+0000","objects_scrubbed":53,"log_size":220,"log_dups_size":0,"ondisk_log_size":220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:08:21.318788+0000","scrub_duration":0.036194551999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":174147,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":184,"num_write_kb":353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.36","version":"75'200","reported_seq":280,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:24.608840+0000","last_change":"2023-11-25T04:26:24.608840+0000","last_active":"2023-11-25T04:26:24.608840+0000","last_peered":"2023-11-25T04:26:24.608840+0000","last_clean":"2023-11-25T04:26:24.608840+0000","last_became_active":"2023-11-25T04:22:47.039892+0000","last_became_peered":"2023-11-25T04:22:47.039892+0000","last_unstale":"2023-11-25T04:26:24.608840+0000","last_undegraded":"2023-11-25T04:26:24.608840+0000","last_fullsized":"2023-11-25T04:26:24.608840+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'200","last_scrub_stamp":"2023-11-25T04:26:24.608737+0000","last_deep_scrub":"75'200","last_deep_scrub_stamp":"2023-11-25T04:26:24.608737+0000","last_clean_scrub_stamp":"2023-11-25T04:26:24.608737+0000","objects_scrubbed":51,"log_size":200,"log_dups_size":0,"ondisk_log_size":200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:49:43.116414+0000","scrub_duration":0.03530929,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":233330,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"75'211","reported_seq":289,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:25.542014+0000","last_change":"2023-11-25T04:26:25.542044+0000","last_active":"2023-11-25T04:26:25.542014+0000","last_peered":"2023-11-25T04:26:25.542014+0000","last_clean":"2023-11-25T04:26:25.542014+0000","last_became_active":"2023-11-25T04:25:51.921800+0000","last_became_peered":"2023-11-25T04:25:51.921800+0000","last_unstale":"2023-11-25T04:26:25.542014+0000","last_undegraded":"2023-11-25T04:26:25.542014+0000","last_fullsized":"2023-11-25T04:26:25.542014+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'211","last_scrub_stamp":"2023-11-25T04:26:25.541938+0000","last_deep_scrub":"75'211","last_deep_scrub_stamp":"2023-11-25T04:26:25.541938+0000","last_clean_scrub_stamp":"2023-11-25T04:26:25.541938+0000","objects_scrubbed":55,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:32:27.981332+0000","scrub_duration":0.043489338000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":302550,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":163,"num_write_kb":431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":300323,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.34","version":"75'186","reported_seq":462,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:12.416198+0000","last_change":"2023-11-25T04:27:12.416198+0000","last_active":"2023-11-25T04:27:12.416198+0000","last_peered":"2023-11-25T04:27:12.416198+0000","last_clean":"2023-11-25T04:27:12.416198+0000","last_became_active":"2023-11-25T04:25:50.922309+0000","last_became_peered":"2023-11-25T04:25:50.922309+0000","last_unstale":"2023-11-25T04:27:12.416198+0000","last_undegraded":"2023-11-25T04:27:12.416198+0000","last_fullsized":"2023-11-25T04:27:12.416198+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'186","last_scrub_stamp":"2023-11-25T04:27:12.416101+0000","last_deep_scrub":"75'186","last_deep_scrub_stamp":"2023-11-25T04:27:12.416101+0000","last_clean_scrub_stamp":"2023-11-25T04:27:12.416101+0000","objects_scrubbed":50,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:42:35.547259+0000","scrub_duration":0.040358303999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":93053,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":131,"num_write_kb":248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":89,"num_bytes_recovered":93053,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"75'183","reported_seq":262,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:19.516218+0000","last_change":"2023-11-25T04:26:19.516218+0000","last_active":"2023-11-25T04:26:19.516218+0000","last_peered":"2023-11-25T04:26:19.516218+0000","last_clean":"2023-11-25T04:26:19.516218+0000","last_became_active":"2023-11-25T04:22:47.040129+0000","last_became_peered":"2023-11-25T04:22:47.040129+0000","last_unstale":"2023-11-25T04:26:19.516218+0000","last_undegraded":"2023-11-25T04:26:19.516218+0000","last_fullsized":"2023-11-25T04:26:19.516218+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'183","last_scrub_stamp":"2023-11-25T04:26:19.516065+0000","last_deep_scrub":"75'183","last_deep_scrub_stamp":"2023-11-25T04:26:19.516065+0000","last_clean_scrub_stamp":"2023-11-25T04:26:19.516065+0000","objects_scrubbed":49,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:43:33.021008+0000","scrub_duration":0.038467327000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":149524,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":139,"num_write_kb":252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.32","version":"75'217","reported_seq":292,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:41.540077+0000","last_change":"2023-11-25T04:27:41.540124+0000","last_active":"2023-11-25T04:27:41.540077+0000","last_peered":"2023-11-25T04:27:41.540077+0000","last_clean":"2023-11-25T04:27:41.540077+0000","last_became_active":"2023-11-25T04:25:53.933694+0000","last_became_peered":"2023-11-25T04:25:53.933694+0000","last_unstale":"2023-11-25T04:27:41.540077+0000","last_undegraded":"2023-11-25T04:27:41.540077+0000","last_fullsized":"2023-11-25T04:27:41.540077+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'217","last_scrub_stamp":"2023-11-25T04:27:41.539982+0000","last_deep_scrub":"75'217","last_deep_scrub_stamp":"2023-11-25T04:27:41.539982+0000","last_clean_scrub_stamp":"2023-11-25T04:27:41.539982+0000","objects_scrubbed":43,"log_size":217,"log_dups_size":0,"ondisk_log_size":217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:13:57.919013+0000","scrub_duration":0.031854021000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":95797,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":181,"num_write_kb":547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":103,"num_bytes_recovered":95797,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.31","version":"75'212","reported_seq":292,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:29.518025+0000","last_change":"2023-11-25T04:26:29.518088+0000","last_active":"2023-11-25T04:26:29.518025+0000","last_peered":"2023-11-25T04:26:29.518025+0000","last_clean":"2023-11-25T04:26:29.518025+0000","last_became_active":"2023-11-25T04:25:49.921129+0000","last_became_peered":"2023-11-25T04:25:49.921129+0000","last_unstale":"2023-11-25T04:26:29.518025+0000","last_undegraded":"2023-11-25T04:26:29.518025+0000","last_fullsized":"2023-11-25T04:26:29.518025+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'212","last_scrub_stamp":"2023-11-25T04:26:29.517931+0000","last_deep_scrub":"75'212","last_deep_scrub_stamp":"2023-11-25T04:26:29.517931+0000","last_clean_scrub_stamp":"2023-11-25T04:26:29.517931+0000","objects_scrubbed":56,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:49:54.902849+0000","scrub_duration":0.039475015000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":163190,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":169,"num_write_kb":336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":154576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"75'229","reported_seq":305,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:20.470401+0000","last_change":"2023-11-25T04:26:20.470429+0000","last_active":"2023-11-25T04:26:20.470401+0000","last_peered":"2023-11-25T04:26:20.470401+0000","last_clean":"2023-11-25T04:26:20.470401+0000","last_became_active":"2023-11-25T04:25:53.933395+0000","last_became_peered":"2023-11-25T04:25:53.933395+0000","last_unstale":"2023-11-25T04:26:20.470401+0000","last_undegraded":"2023-11-25T04:26:20.470401+0000","last_fullsized":"2023-11-25T04:26:20.470401+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'229","last_scrub_stamp":"2023-11-25T04:26:20.470325+0000","last_deep_scrub":"75'229","last_deep_scrub_stamp":"2023-11-25T04:26:20.470325+0000","last_clean_scrub_stamp":"2023-11-25T04:26:20.470325+0000","objects_scrubbed":55,"log_size":229,"log_dups_size":0,"ondisk_log_size":229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:55:52.038708+0000","scrub_duration":0.036798694,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1423587,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":182,"num_write_kb":1626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":1414508,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2f","version":"75'203","reported_seq":283,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:18.718065+0000","last_change":"2023-11-25T04:26:18.718065+0000","last_active":"2023-11-25T04:26:18.718065+0000","last_peered":"2023-11-25T04:26:18.718065+0000","last_clean":"2023-11-25T04:26:18.718065+0000","last_became_active":"2023-11-25T04:22:47.039123+0000","last_became_peered":"2023-11-25T04:22:47.039123+0000","last_unstale":"2023-11-25T04:26:18.718065+0000","last_undegraded":"2023-11-25T04:26:18.718065+0000","last_fullsized":"2023-11-25T04:26:18.718065+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'203","last_scrub_stamp":"2023-11-25T04:26:18.717922+0000","last_deep_scrub":"75'203","last_deep_scrub_stamp":"2023-11-25T04:26:18.717922+0000","last_clean_scrub_stamp":"2023-11-25T04:26:18.717922+0000","objects_scrubbed":51,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:33:40.286516+0000","scrub_duration":0.039125398999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":122752,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":143,"num_write_kb":435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"75'199","reported_seq":279,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:58.496118+0000","last_change":"2023-11-25T04:26:58.496118+0000","last_active":"2023-11-25T04:26:58.496118+0000","last_peered":"2023-11-25T04:26:58.496118+0000","last_clean":"2023-11-25T04:26:58.496118+0000","last_became_active":"2023-11-25T04:22:47.036411+0000","last_became_peered":"2023-11-25T04:22:47.036411+0000","last_unstale":"2023-11-25T04:26:58.496118+0000","last_undegraded":"2023-11-25T04:26:58.496118+0000","last_fullsized":"2023-11-25T04:26:58.496118+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'199","last_scrub_stamp":"2023-11-25T04:26:58.496005+0000","last_deep_scrub":"75'199","last_deep_scrub_stamp":"2023-11-25T04:26:58.496005+0000","last_clean_scrub_stamp":"2023-11-25T04:26:58.496005+0000","objects_scrubbed":51,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:57:18.007638+0000","scrub_duration":0.041894438999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4770899,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":140,"num_write_kb":4870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2d","version":"75'194","reported_seq":274,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:47.814157+0000","last_change":"2023-11-25T04:26:47.814157+0000","last_active":"2023-11-25T04:26:47.814157+0000","last_peered":"2023-11-25T04:26:47.814157+0000","last_clean":"2023-11-25T04:26:47.814157+0000","last_became_active":"2023-11-25T04:22:47.038477+0000","last_became_peered":"2023-11-25T04:22:47.038477+0000","last_unstale":"2023-11-25T04:26:47.814157+0000","last_undegraded":"2023-11-25T04:26:47.814157+0000","last_fullsized":"2023-11-25T04:26:47.814157+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'194","last_scrub_stamp":"2023-11-25T04:26:47.814018+0000","last_deep_scrub":"75'194","last_deep_scrub_stamp":"2023-11-25T04:26:47.814018+0000","last_clean_scrub_stamp":"2023-11-25T04:26:47.814018+0000","objects_scrubbed":51,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:20:21.601431+0000","scrub_duration":0.038348491999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":115975,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"75'230","reported_seq":307,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:19.428426+0000","last_change":"2023-11-25T04:26:19.428460+0000","last_active":"2023-11-25T04:26:19.428426+0000","last_peered":"2023-11-25T04:26:19.428426+0000","last_clean":"2023-11-25T04:26:19.428426+0000","last_became_active":"2023-11-25T04:25:52.922367+0000","last_became_peered":"2023-11-25T04:25:52.922367+0000","last_unstale":"2023-11-25T04:26:19.428426+0000","last_undegraded":"2023-11-25T04:26:19.428426+0000","last_fullsized":"2023-11-25T04:26:19.428426+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'230","last_scrub_stamp":"2023-11-25T04:26:19.428350+0000","last_deep_scrub":"75'230","last_deep_scrub_stamp":"2023-11-25T04:26:19.428350+0000","last_clean_scrub_stamp":"2023-11-25T04:26:19.428350+0000","objects_scrubbed":63,"log_size":230,"log_dups_size":0,"ondisk_log_size":230,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:10:12.216682+0000","scrub_duration":0.042625061999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":747090,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":211,"num_write_kb":985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":732276,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"75'211","reported_seq":288,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:17.381634+0000","last_change":"2023-11-25T04:26:17.381662+0000","last_active":"2023-11-25T04:26:17.381634+0000","last_peered":"2023-11-25T04:26:17.381634+0000","last_clean":"2023-11-25T04:26:17.381634+0000","last_became_active":"2023-11-25T04:25:52.923378+0000","last_became_peered":"2023-11-25T04:25:52.923378+0000","last_unstale":"2023-11-25T04:26:17.381634+0000","last_undegraded":"2023-11-25T04:26:17.381634+0000","last_fullsized":"2023-11-25T04:26:17.381634+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'211","last_scrub_stamp":"2023-11-25T04:26:17.381560+0000","last_deep_scrub":"75'211","last_deep_scrub_stamp":"2023-11-25T04:26:17.381560+0000","last_clean_scrub_stamp":"2023-11-25T04:26:17.381560+0000","objects_scrubbed":55,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:26:19.664059+0000","scrub_duration":0.036605545000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69580,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":142,"num_write_kb":315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":98,"num_bytes_recovered":69580,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2a","version":"75'222","reported_seq":530,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:14.438320+0000","last_change":"2023-11-25T04:27:14.438320+0000","last_active":"2023-11-25T04:27:14.438320+0000","last_peered":"2023-11-25T04:27:14.438320+0000","last_clean":"2023-11-25T04:27:14.438320+0000","last_became_active":"2023-11-25T04:25:52.924844+0000","last_became_peered":"2023-11-25T04:25:52.924844+0000","last_unstale":"2023-11-25T04:27:14.438320+0000","last_undegraded":"2023-11-25T04:27:14.438320+0000","last_fullsized":"2023-11-25T04:27:14.438320+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'222","last_scrub_stamp":"2023-11-25T04:27:14.438222+0000","last_deep_scrub":"75'222","last_deep_scrub_stamp":"2023-11-25T04:27:14.438222+0000","last_clean_scrub_stamp":"2023-11-25T04:27:14.438222+0000","objects_scrubbed":51,"log_size":222,"log_dups_size":0,"ondisk_log_size":222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:18:44.242226+0000","scrub_duration":0.036903918000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":115997,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":162,"num_write_kb":261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":115997,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.29","version":"75'211","reported_seq":509,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:12.692632+0000","last_change":"2023-11-25T04:26:12.692632+0000","last_active":"2023-11-25T04:26:12.692632+0000","last_peered":"2023-11-25T04:26:12.692632+0000","last_clean":"2023-11-25T04:26:12.692632+0000","last_became_active":"2023-11-25T04:25:49.919886+0000","last_became_peered":"2023-11-25T04:25:49.919886+0000","last_unstale":"2023-11-25T04:26:12.692632+0000","last_undegraded":"2023-11-25T04:26:12.692632+0000","last_fullsized":"2023-11-25T04:26:12.692632+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'211","last_scrub_stamp":"2023-11-25T04:26:12.692467+0000","last_deep_scrub":"75'211","last_deep_scrub_stamp":"2023-11-25T04:26:12.692467+0000","last_clean_scrub_stamp":"2023-11-25T04:26:12.692467+0000","objects_scrubbed":59,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:13:03.899209+0000","scrub_duration":0.043132342999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":212878,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":212878,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"75'218","reported_seq":297,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:00.495508+0000","last_change":"2023-11-25T04:27:00.495508+0000","last_active":"2023-11-25T04:27:00.495508+0000","last_peered":"2023-11-25T04:27:00.495508+0000","last_clean":"2023-11-25T04:27:00.495508+0000","last_became_active":"2023-11-25T04:22:47.036135+0000","last_became_peered":"2023-11-25T04:22:47.036135+0000","last_unstale":"2023-11-25T04:27:00.495508+0000","last_undegraded":"2023-11-25T04:27:00.495508+0000","last_fullsized":"2023-11-25T04:27:00.495508+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'218","last_scrub_stamp":"2023-11-25T04:27:00.495361+0000","last_deep_scrub":"75'218","last_deep_scrub_stamp":"2023-11-25T04:27:00.495361+0000","last_clean_scrub_stamp":"2023-11-25T04:27:00.495361+0000","objects_scrubbed":49,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:17:48.678120+0000","scrub_duration":0.034866902999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":102957,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":172,"num_write_kb":610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"75'217","reported_seq":527,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:18.423856+0000","last_change":"2023-11-25T04:27:18.423894+0000","last_active":"2023-11-25T04:27:18.423856+0000","last_peered":"2023-11-25T04:27:18.423856+0000","last_clean":"2023-11-25T04:27:18.423856+0000","last_became_active":"2023-11-25T04:25:51.922364+0000","last_became_peered":"2023-11-25T04:25:51.922364+0000","last_unstale":"2023-11-25T04:27:18.423856+0000","last_undegraded":"2023-11-25T04:27:18.423856+0000","last_fullsized":"2023-11-25T04:27:18.423856+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'217","last_scrub_stamp":"2023-11-25T04:27:18.423755+0000","last_deep_scrub":"75'217","last_deep_scrub_stamp":"2023-11-25T04:27:18.423755+0000","last_clean_scrub_stamp":"2023-11-25T04:27:18.423755+0000","objects_scrubbed":50,"log_size":217,"log_dups_size":0,"ondisk_log_size":217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T16:12:12.360172+0000","scrub_duration":0.042117390999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":182683,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":171,"num_write_kb":791,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":182683,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"54'187","reported_seq":507,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:09.671589+0000","last_change":"2023-11-25T04:25:51.922529+0000","last_active":"2023-11-25T04:26:09.671589+0000","last_peered":"2023-11-25T04:26:09.671589+0000","last_clean":"2023-11-25T04:26:09.671589+0000","last_became_active":"2023-11-25T04:25:51.922064+0000","last_became_peered":"2023-11-25T04:25:51.922064+0000","last_unstale":"2023-11-25T04:26:09.671589+0000","last_undegraded":"2023-11-25T04:26:09.671589+0000","last_fullsized":"2023-11-25T04:26:09.671589+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":187,"log_dups_size":0,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":86311,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":171,"num_write_kb":274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":116,"num_bytes_recovered":86311,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.25","version":"75'201","reported_seq":278,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:10.399641+0000","last_change":"2023-11-25T04:26:10.399668+0000","last_active":"2023-11-25T04:26:10.399641+0000","last_peered":"2023-11-25T04:26:10.399641+0000","last_clean":"2023-11-25T04:26:10.399641+0000","last_became_active":"2023-11-25T04:25:52.924795+0000","last_became_peered":"2023-11-25T04:25:52.924795+0000","last_unstale":"2023-11-25T04:26:10.399641+0000","last_undegraded":"2023-11-25T04:26:10.399641+0000","last_fullsized":"2023-11-25T04:26:10.399641+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'201","last_scrub_stamp":"2023-11-25T04:26:10.399561+0000","last_deep_scrub":"75'201","last_deep_scrub_stamp":"2023-11-25T04:26:10.399561+0000","last_clean_scrub_stamp":"2023-11-25T04:26:10.399561+0000","objects_scrubbed":52,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:15:15.077501+0000","scrub_duration":0.038027564,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":128288,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":167,"num_write_kb":267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":120265,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.24","version":"75'195","reported_seq":482,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:43.546743+0000","last_change":"2023-11-25T04:26:43.546743+0000","last_active":"2023-11-25T04:26:43.546743+0000","last_peered":"2023-11-25T04:26:43.546743+0000","last_clean":"2023-11-25T04:26:43.546743+0000","last_became_active":"2023-11-25T04:25:50.922216+0000","last_became_peered":"2023-11-25T04:25:50.922216+0000","last_unstale":"2023-11-25T04:26:43.546743+0000","last_undegraded":"2023-11-25T04:26:43.546743+0000","last_fullsized":"2023-11-25T04:26:43.546743+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'195","last_scrub_stamp":"2023-11-25T04:26:43.546552+0000","last_deep_scrub":"75'195","last_deep_scrub_stamp":"2023-11-25T04:26:43.546552+0000","last_clean_scrub_stamp":"2023-11-25T04:26:43.546552+0000","objects_scrubbed":44,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:16:44.328689+0000","scrub_duration":0.034140957,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":107507,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":148,"num_write_kb":288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":107494,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.23","version":"75'222","reported_seq":302,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:50.770920+0000","last_change":"2023-11-25T04:26:50.770920+0000","last_active":"2023-11-25T04:26:50.770920+0000","last_peered":"2023-11-25T04:26:50.770920+0000","last_clean":"2023-11-25T04:26:50.770920+0000","last_became_active":"2023-11-25T04:22:47.036823+0000","last_became_peered":"2023-11-25T04:22:47.036823+0000","last_unstale":"2023-11-25T04:26:50.770920+0000","last_undegraded":"2023-11-25T04:26:50.770920+0000","last_fullsized":"2023-11-25T04:26:50.770920+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'222","last_scrub_stamp":"2023-11-25T04:26:50.770805+0000","last_deep_scrub":"75'222","last_deep_scrub_stamp":"2023-11-25T04:26:50.770805+0000","last_clean_scrub_stamp":"2023-11-25T04:26:50.770805+0000","objects_scrubbed":67,"log_size":222,"log_dups_size":0,"ondisk_log_size":222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:16:51.552942+0000","scrub_duration":0.048356029000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4304582,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":152,"num_write_kb":4331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.22","version":"75'253","reported_seq":593,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:27.393520+0000","last_change":"2023-11-25T04:27:27.393555+0000","last_active":"2023-11-25T04:27:27.393520+0000","last_peered":"2023-11-25T04:27:27.393520+0000","last_clean":"2023-11-25T04:27:27.393520+0000","last_became_active":"2023-11-25T04:25:54.939584+0000","last_became_peered":"2023-11-25T04:25:54.939584+0000","last_unstale":"2023-11-25T04:27:27.393520+0000","last_undegraded":"2023-11-25T04:27:27.393520+0000","last_fullsized":"2023-11-25T04:27:27.393520+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'253","last_scrub_stamp":"2023-11-25T04:27:27.393439+0000","last_deep_scrub":"75'253","last_deep_scrub_stamp":"2023-11-25T04:27:27.393439+0000","last_clean_scrub_stamp":"2023-11-25T04:27:27.393439+0000","objects_scrubbed":64,"log_size":253,"log_dups_size":0,"ondisk_log_size":253,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:03:46.694422+0000","scrub_duration":0.044005476000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":129145,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":185,"num_write_kb":432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":121,"num_bytes_recovered":129107,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"75'260","reported_seq":340,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:06.501506+0000","last_change":"2023-11-25T04:27:06.501536+0000","last_active":"2023-11-25T04:27:06.501506+0000","last_peered":"2023-11-25T04:27:06.501506+0000","last_clean":"2023-11-25T04:27:06.501506+0000","last_became_active":"2023-11-25T04:22:47.037206+0000","last_became_peered":"2023-11-25T04:22:47.037206+0000","last_unstale":"2023-11-25T04:27:06.501506+0000","last_undegraded":"2023-11-25T04:27:06.501506+0000","last_fullsized":"2023-11-25T04:27:06.501506+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'260","last_scrub_stamp":"2023-11-25T04:27:06.501413+0000","last_deep_scrub":"75'260","last_deep_scrub_stamp":"2023-11-25T04:27:06.501413+0000","last_clean_scrub_stamp":"2023-11-25T04:27:06.501413+0000","objects_scrubbed":59,"log_size":260,"log_dups_size":0,"ondisk_log_size":260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:18:02.436135+0000","scrub_duration":0.044736607999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":215311,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":194,"num_write_kb":381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"75'231","reported_seq":308,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:35.525083+0000","last_change":"2023-11-25T04:27:35.525083+0000","last_active":"2023-11-25T04:27:35.525083+0000","last_peered":"2023-11-25T04:27:35.525083+0000","last_clean":"2023-11-25T04:27:35.525083+0000","last_became_active":"2023-11-25T04:25:53.931937+0000","last_became_peered":"2023-11-25T04:25:53.931937+0000","last_unstale":"2023-11-25T04:27:35.525083+0000","last_undegraded":"2023-11-25T04:27:35.525083+0000","last_fullsized":"2023-11-25T04:27:35.525083+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'231","last_scrub_stamp":"2023-11-25T04:27:35.524975+0000","last_deep_scrub":"75'231","last_deep_scrub_stamp":"2023-11-25T04:27:35.524975+0000","last_clean_scrub_stamp":"2023-11-25T04:27:35.524975+0000","objects_scrubbed":51,"log_size":231,"log_dups_size":0,"ondisk_log_size":231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:30:27.829518+0000","scrub_duration":0.034850857999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104603,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":172,"num_write_kb":267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":104603,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"75'193","reported_seq":273,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:32.532758+0000","last_change":"2023-11-25T04:26:32.532758+0000","last_active":"2023-11-25T04:26:32.532758+0000","last_peered":"2023-11-25T04:26:32.532758+0000","last_clean":"2023-11-25T04:26:32.532758+0000","last_became_active":"2023-11-25T04:22:47.036107+0000","last_became_peered":"2023-11-25T04:22:47.036107+0000","last_unstale":"2023-11-25T04:26:32.532758+0000","last_undegraded":"2023-11-25T04:26:32.532758+0000","last_fullsized":"2023-11-25T04:26:32.532758+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'193","last_scrub_stamp":"2023-11-25T04:26:32.532624+0000","last_deep_scrub":"75'193","last_deep_scrub_stamp":"2023-11-25T04:26:32.532624+0000","last_clean_scrub_stamp":"2023-11-25T04:26:32.532624+0000","objects_scrubbed":56,"log_size":193,"log_dups_size":0,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:09:38.769552+0000","scrub_duration":0.038075093999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":183586,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":127,"num_write_kb":284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"75'117","reported_seq":203,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:31.496279+0000","last_change":"2023-11-25T04:26:31.496307+0000","last_active":"2023-11-25T04:26:31.496279+0000","last_peered":"2023-11-25T04:26:31.496279+0000","last_clean":"2023-11-25T04:26:31.496279+0000","last_became_active":"2023-11-25T04:25:49.920010+0000","last_became_peered":"2023-11-25T04:25:49.920010+0000","last_unstale":"2023-11-25T04:26:31.496279+0000","last_undegraded":"2023-11-25T04:26:31.496279+0000","last_fullsized":"2023-11-25T04:26:31.496279+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'117","last_scrub_stamp":"2023-11-25T04:26:31.496196+0000","last_deep_scrub":"75'117","last_deep_scrub_stamp":"2023-11-25T04:26:31.496196+0000","last_clean_scrub_stamp":"2023-11-25T04:26:31.496196+0000","objects_scrubbed":3,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:26:02.874431+0000","scrub_duration":0.0076030029999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2816216,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":113,"num_write_kb":2816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":2816182,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"75'189","reported_seq":268,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:40.589251+0000","last_change":"2023-11-25T04:26:40.589251+0000","last_active":"2023-11-25T04:26:40.589251+0000","last_peered":"2023-11-25T04:26:40.589251+0000","last_clean":"2023-11-25T04:26:40.589251+0000","last_became_active":"2023-11-25T04:22:47.037158+0000","last_became_peered":"2023-11-25T04:22:47.037158+0000","last_unstale":"2023-11-25T04:26:40.589251+0000","last_undegraded":"2023-11-25T04:26:40.589251+0000","last_fullsized":"2023-11-25T04:26:40.589251+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'189","last_scrub_stamp":"2023-11-25T04:26:40.589106+0000","last_deep_scrub":"75'189","last_deep_scrub_stamp":"2023-11-25T04:26:40.589106+0000","last_clean_scrub_stamp":"2023-11-25T04:26:40.589106+0000","objects_scrubbed":44,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:32:56.267980+0000","scrub_duration":0.032171351000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":205976,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.12","version":"49'2","reported_seq":92,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:30.784752+0000","last_change":"2023-11-25T04:26:30.784752+0000","last_active":"2023-11-25T04:26:30.784752+0000","last_peered":"2023-11-25T04:26:30.784752+0000","last_clean":"2023-11-25T04:26:30.784752+0000","last_became_active":"2023-11-25T04:25:47.000516+0000","last_became_peered":"2023-11-25T04:25:47.000516+0000","last_unstale":"2023-11-25T04:26:30.784752+0000","last_undegraded":"2023-11-25T04:26:30.784752+0000","last_fullsized":"2023-11-25T04:26:30.784752+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'2","last_scrub_stamp":"2023-11-25T04:26:30.784639+0000","last_deep_scrub":"49'2","last_deep_scrub_stamp":"2023-11-25T04:26:30.784639+0000","last_clean_scrub_stamp":"2023-11-25T04:26:30.784639+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:38:15.370559+0000","scrub_duration":0.001803528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"54'175","reported_seq":491,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:09.671448+0000","last_change":"2023-11-25T04:25:52.923694+0000","last_active":"2023-11-25T04:26:09.671448+0000","last_peered":"2023-11-25T04:26:09.671448+0000","last_clean":"2023-11-25T04:26:09.671448+0000","last_became_active":"2023-11-25T04:25:52.923045+0000","last_became_peered":"2023-11-25T04:25:52.923045+0000","last_unstale":"2023-11-25T04:26:09.671448+0000","last_undegraded":"2023-11-25T04:26:09.671448+0000","last_fullsized":"2023-11-25T04:26:09.671448+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":171823,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":185,"num_write_kb":340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":166387,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":90,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:18.488445+0000","last_change":"2023-11-25T04:26:18.488445+0000","last_active":"2023-11-25T04:26:18.488445+0000","last_peered":"2023-11-25T04:26:18.488445+0000","last_clean":"2023-11-25T04:26:18.488445+0000","last_became_active":"2023-11-25T04:25:46.993870+0000","last_became_peered":"2023-11-25T04:25:46.993870+0000","last_unstale":"2023-11-25T04:26:18.488445+0000","last_undegraded":"2023-11-25T04:26:18.488445+0000","last_fullsized":"2023-11-25T04:26:18.488445+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:26:18.488345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:26:18.488345+0000","last_clean_scrub_stamp":"2023-11-25T04:26:18.488345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:43:21.558596+0000","scrub_duration":0.002108399,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"75'208","reported_seq":503,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:45.248870+0000","last_change":"2023-11-25T04:27:45.248901+0000","last_active":"2023-11-25T04:27:45.248870+0000","last_peered":"2023-11-25T04:27:45.248870+0000","last_clean":"2023-11-25T04:27:45.248870+0000","last_became_active":"2023-11-25T04:25:50.922511+0000","last_became_peered":"2023-11-25T04:25:50.922511+0000","last_unstale":"2023-11-25T04:27:45.248870+0000","last_undegraded":"2023-11-25T04:27:45.248870+0000","last_fullsized":"2023-11-25T04:27:45.248870+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'208","last_scrub_stamp":"2023-11-25T04:27:45.248792+0000","last_deep_scrub":"75'208","last_deep_scrub_stamp":"2023-11-25T04:27:45.248792+0000","last_clean_scrub_stamp":"2023-11-25T04:27:45.248792+0000","objects_scrubbed":56,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:46:01.446310+0000","scrub_duration":0.040754302999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":579704,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":97,"num_bytes_recovered":570044,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.10","version":"75'12","reported_seq":52,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:06.483215+0000","last_change":"2023-11-25T04:27:06.483272+0000","last_active":"2023-11-25T04:27:06.483215+0000","last_peered":"2023-11-25T04:27:06.483215+0000","last_clean":"2023-11-25T04:27:06.483215+0000","last_became_active":"2023-11-25T04:25:46.998778+0000","last_became_peered":"2023-11-25T04:25:46.998778+0000","last_unstale":"2023-11-25T04:27:06.483215+0000","last_undegraded":"2023-11-25T04:27:06.483215+0000","last_fullsized":"2023-11-25T04:27:06.483215+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'12","last_scrub_stamp":"2023-11-25T04:27:06.483113+0000","last_deep_scrub":"75'12","last_deep_scrub_stamp":"2023-11-25T04:27:06.483113+0000","last_clean_scrub_stamp":"2023-11-25T04:27:06.483113+0000","objects_scrubbed":4,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:26:21.828117+0000","scrub_duration":0.0040538359999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"75'3","reported_seq":47,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:36.494324+0000","last_change":"2023-11-25T04:26:36.494358+0000","last_active":"2023-11-25T04:26:36.494324+0000","last_peered":"2023-11-25T04:26:36.494324+0000","last_clean":"2023-11-25T04:26:36.494324+0000","last_became_active":"2023-11-25T04:25:46.997108+0000","last_became_peered":"2023-11-25T04:25:46.997108+0000","last_unstale":"2023-11-25T04:26:36.494324+0000","last_undegraded":"2023-11-25T04:26:36.494324+0000","last_fullsized":"2023-11-25T04:26:36.494324+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'3","last_scrub_stamp":"2023-11-25T04:26:36.494205+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2023-11-25T04:26:36.494205+0000","last_clean_scrub_stamp":"2023-11-25T04:26:36.494205+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:47:03.918120+0000","scrub_duration":0.0026531879999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"75'208","reported_seq":503,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:07.384457+0000","last_change":"2023-11-25T04:27:07.384457+0000","last_active":"2023-11-25T04:27:07.384457+0000","last_peered":"2023-11-25T04:27:07.384457+0000","last_clean":"2023-11-25T04:27:07.384457+0000","last_became_active":"2023-11-25T04:25:50.921772+0000","last_became_peered":"2023-11-25T04:25:50.921772+0000","last_unstale":"2023-11-25T04:27:07.384457+0000","last_undegraded":"2023-11-25T04:27:07.384457+0000","last_fullsized":"2023-11-25T04:27:07.384457+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'208","last_scrub_stamp":"2023-11-25T04:27:07.384323+0000","last_deep_scrub":"75'208","last_deep_scrub_stamp":"2023-11-25T04:27:07.384323+0000","last_clean_scrub_stamp":"2023-11-25T04:27:07.384323+0000","objects_scrubbed":49,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:52:56.354420+0000","scrub_duration":0.034118876999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112103,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":164,"num_write_kb":240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":99,"num_bytes_recovered":103472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"75'3","reported_seq":93,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:36.492740+0000","last_change":"2023-11-25T04:26:36.492740+0000","last_active":"2023-11-25T04:26:36.492740+0000","last_peered":"2023-11-25T04:26:36.492740+0000","last_clean":"2023-11-25T04:26:36.492740+0000","last_became_active":"2023-11-25T04:25:46.994640+0000","last_became_peered":"2023-11-25T04:25:46.994640+0000","last_unstale":"2023-11-25T04:26:36.492740+0000","last_undegraded":"2023-11-25T04:26:36.492740+0000","last_fullsized":"2023-11-25T04:26:36.492740+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'3","last_scrub_stamp":"2023-11-25T04:26:36.492569+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2023-11-25T04:26:36.492569+0000","last_clean_scrub_stamp":"2023-11-25T04:26:36.492569+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:28:25.392135+0000","scrub_duration":0.0033623030000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"75'229","reported_seq":309,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:05.415841+0000","last_change":"2023-11-25T04:27:05.415841+0000","last_active":"2023-11-25T04:27:05.415841+0000","last_peered":"2023-11-25T04:27:05.415841+0000","last_clean":"2023-11-25T04:27:05.415841+0000","last_became_active":"2023-11-25T04:22:47.039296+0000","last_became_peered":"2023-11-25T04:22:47.039296+0000","last_unstale":"2023-11-25T04:27:05.415841+0000","last_undegraded":"2023-11-25T04:27:05.415841+0000","last_fullsized":"2023-11-25T04:27:05.415841+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'229","last_scrub_stamp":"2023-11-25T04:27:05.415739+0000","last_deep_scrub":"75'229","last_deep_scrub_stamp":"2023-11-25T04:27:05.415739+0000","last_clean_scrub_stamp":"2023-11-25T04:27:05.415739+0000","objects_scrubbed":56,"log_size":229,"log_dups_size":0,"ondisk_log_size":229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:46:38.706963+0000","scrub_duration":0.037100175999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154533,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":182,"num_write_kb":299,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"75'2","reported_seq":47,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:33.490361+0000","last_change":"2023-11-25T04:26:33.490414+0000","last_active":"2023-11-25T04:26:33.490361+0000","last_peered":"2023-11-25T04:26:33.490361+0000","last_clean":"2023-11-25T04:26:33.490361+0000","last_became_active":"2023-11-25T04:25:46.999604+0000","last_became_peered":"2023-11-25T04:25:46.999604+0000","last_unstale":"2023-11-25T04:26:33.490361+0000","last_undegraded":"2023-11-25T04:26:33.490361+0000","last_fullsized":"2023-11-25T04:26:33.490361+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'2","last_scrub_stamp":"2023-11-25T04:26:33.490271+0000","last_deep_scrub":"75'2","last_deep_scrub_stamp":"2023-11-25T04:26:33.490271+0000","last_clean_scrub_stamp":"2023-11-25T04:26:33.490271+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:29:08.502565+0000","scrub_duration":0.0030545450000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"54'147","reported_seq":212,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986571+0000","last_change":"2023-11-25T04:25:54.939274+0000","last_active":"2023-11-25T04:26:00.986571+0000","last_peered":"2023-11-25T04:26:00.986571+0000","last_clean":"2023-11-25T04:26:00.986571+0000","last_became_active":"2023-11-25T04:25:54.938236+0000","last_became_peered":"2023-11-25T04:25:54.938236+0000","last_unstale":"2023-11-25T04:26:00.986571+0000","last_undegraded":"2023-11-25T04:26:00.986571+0000","last_fullsized":"2023-11-25T04:26:00.986571+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":147,"log_dups_size":0,"ondisk_log_size":147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":87524,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":155,"num_write_kb":235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":87524,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":43,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:41.504866+0000","last_change":"2023-11-25T04:26:41.504892+0000","last_active":"2023-11-25T04:26:41.504866+0000","last_peered":"2023-11-25T04:26:41.504866+0000","last_clean":"2023-11-25T04:26:41.504866+0000","last_became_active":"2023-11-25T04:25:46.995712+0000","last_became_peered":"2023-11-25T04:25:46.995712+0000","last_unstale":"2023-11-25T04:26:41.504866+0000","last_undegraded":"2023-11-25T04:26:41.504866+0000","last_fullsized":"2023-11-25T04:26:41.504866+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:26:41.504775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:26:41.504775+0000","last_clean_scrub_stamp":"2023-11-25T04:26:41.504775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:35:58.006258+0000","scrub_duration":0.001777907,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"75'211","reported_seq":509,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:46.251693+0000","last_change":"2023-11-25T04:27:46.251738+0000","last_active":"2023-11-25T04:27:46.251693+0000","last_peered":"2023-11-25T04:27:46.251693+0000","last_clean":"2023-11-25T04:27:46.251693+0000","last_became_active":"2023-11-25T04:25:52.922549+0000","last_became_peered":"2023-11-25T04:25:52.922549+0000","last_unstale":"2023-11-25T04:27:46.251693+0000","last_undegraded":"2023-11-25T04:27:46.251693+0000","last_fullsized":"2023-11-25T04:27:46.251693+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'211","last_scrub_stamp":"2023-11-25T04:27:46.251589+0000","last_deep_scrub":"75'211","last_deep_scrub_stamp":"2023-11-25T04:27:46.251589+0000","last_clean_scrub_stamp":"2023-11-25T04:27:46.251589+0000","objects_scrubbed":56,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:21:12.066048+0000","scrub_duration":0.042300846000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":266919,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":264258,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"75'13","reported_seq":108,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:05.498699+0000","last_change":"2023-11-25T04:27:05.498699+0000","last_active":"2023-11-25T04:27:05.498699+0000","last_peered":"2023-11-25T04:27:05.498699+0000","last_clean":"2023-11-25T04:27:05.498699+0000","last_became_active":"2023-11-25T04:25:47.000687+0000","last_became_peered":"2023-11-25T04:25:47.000687+0000","last_unstale":"2023-11-25T04:27:05.498699+0000","last_undegraded":"2023-11-25T04:27:05.498699+0000","last_fullsized":"2023-11-25T04:27:05.498699+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'13","last_scrub_stamp":"2023-11-25T04:27:05.498602+0000","last_deep_scrub":"75'13","last_deep_scrub_stamp":"2023-11-25T04:27:05.498602+0000","last_clean_scrub_stamp":"2023-11-25T04:27:05.498602+0000","objects_scrubbed":2,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:44:21.302320+0000","scrub_duration":0.0090048890000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":4101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4194304,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"75'235","reported_seq":314,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:49.411583+0000","last_change":"2023-11-25T04:26:49.411614+0000","last_active":"2023-11-25T04:26:49.411583+0000","last_peered":"2023-11-25T04:26:49.411583+0000","last_clean":"2023-11-25T04:26:49.411583+0000","last_became_active":"2023-11-25T04:25:50.922409+0000","last_became_peered":"2023-11-25T04:25:50.922409+0000","last_unstale":"2023-11-25T04:26:49.411583+0000","last_undegraded":"2023-11-25T04:26:49.411583+0000","last_fullsized":"2023-11-25T04:26:49.411583+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'235","last_scrub_stamp":"2023-11-25T04:26:49.411506+0000","last_deep_scrub":"75'235","last_deep_scrub_stamp":"2023-11-25T04:26:49.411506+0000","last_clean_scrub_stamp":"2023-11-25T04:26:49.411506+0000","objects_scrubbed":62,"log_size":235,"log_dups_size":0,"ondisk_log_size":235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:38:13.278410+0000","scrub_duration":0.044042721,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":553893,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":169,"num_write_kb":770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":553893,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":43,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:05.477955+0000","last_change":"2023-11-25T04:27:05.477987+0000","last_active":"2023-11-25T04:27:05.477955+0000","last_peered":"2023-11-25T04:27:05.477955+0000","last_clean":"2023-11-25T04:27:05.477955+0000","last_became_active":"2023-11-25T04:25:46.997019+0000","last_became_peered":"2023-11-25T04:25:46.997019+0000","last_unstale":"2023-11-25T04:27:05.477955+0000","last_undegraded":"2023-11-25T04:27:05.477955+0000","last_fullsized":"2023-11-25T04:27:05.477955+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:27:05.477862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:27:05.477862+0000","last_clean_scrub_stamp":"2023-11-25T04:27:05.477862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:44:35.653542+0000","scrub_duration":0.0018186669999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"75'224","reported_seq":300,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:39.544596+0000","last_change":"2023-11-25T04:27:39.544646+0000","last_active":"2023-11-25T04:27:39.544596+0000","last_peered":"2023-11-25T04:27:39.544596+0000","last_clean":"2023-11-25T04:27:39.544596+0000","last_became_active":"2023-11-25T04:25:53.933471+0000","last_became_peered":"2023-11-25T04:25:53.933471+0000","last_unstale":"2023-11-25T04:27:39.544596+0000","last_undegraded":"2023-11-25T04:27:39.544596+0000","last_fullsized":"2023-11-25T04:27:39.544596+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'224","last_scrub_stamp":"2023-11-25T04:27:39.544483+0000","last_deep_scrub":"75'224","last_deep_scrub_stamp":"2023-11-25T04:27:39.544483+0000","last_clean_scrub_stamp":"2023-11-25T04:27:39.544483+0000","objects_scrubbed":61,"log_size":224,"log_dups_size":0,"ondisk_log_size":224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:29:28.945105+0000","scrub_duration":0.044510499000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":320867,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":190,"num_write_kb":577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":108,"num_bytes_recovered":311640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":90,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:26.554458+0000","last_change":"2023-11-25T04:26:26.554458+0000","last_active":"2023-11-25T04:26:26.554458+0000","last_peered":"2023-11-25T04:26:26.554458+0000","last_clean":"2023-11-25T04:26:26.554458+0000","last_became_active":"2023-11-25T04:25:46.994101+0000","last_became_peered":"2023-11-25T04:25:46.994101+0000","last_unstale":"2023-11-25T04:26:26.554458+0000","last_undegraded":"2023-11-25T04:26:26.554458+0000","last_fullsized":"2023-11-25T04:26:26.554458+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:26:26.554355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:26:26.554355+0000","last_clean_scrub_stamp":"2023-11-25T04:26:26.554355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:20:20.095629+0000","scrub_duration":0.0016349400000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"75'187","reported_seq":266,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:35.557378+0000","last_change":"2023-11-25T04:26:35.557414+0000","last_active":"2023-11-25T04:26:35.557378+0000","last_peered":"2023-11-25T04:26:35.557378+0000","last_clean":"2023-11-25T04:26:35.557378+0000","last_became_active":"2023-11-25T04:25:49.920971+0000","last_became_peered":"2023-11-25T04:25:49.920971+0000","last_unstale":"2023-11-25T04:26:35.557378+0000","last_undegraded":"2023-11-25T04:26:35.557378+0000","last_fullsized":"2023-11-25T04:26:35.557378+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'187","last_scrub_stamp":"2023-11-25T04:26:35.557305+0000","last_deep_scrub":"75'187","last_deep_scrub_stamp":"2023-11-25T04:26:35.557305+0000","last_clean_scrub_stamp":"2023-11-25T04:26:35.557305+0000","objects_scrubbed":46,"log_size":187,"log_dups_size":0,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:15:17.199719+0000","scrub_duration":0.039856336999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3502371,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":3718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":89,"num_bytes_recovered":3497669,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"49'1","reported_seq":80,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:52.518138+0000","last_change":"2023-11-25T04:26:52.518138+0000","last_active":"2023-11-25T04:26:52.518138+0000","last_peered":"2023-11-25T04:26:52.518138+0000","last_clean":"2023-11-25T04:26:52.518138+0000","last_became_active":"2023-11-25T04:22:46.034158+0000","last_became_peered":"2023-11-25T04:22:46.034158+0000","last_unstale":"2023-11-25T04:26:52.518138+0000","last_undegraded":"2023-11-25T04:26:52.518138+0000","last_fullsized":"2023-11-25T04:26:52.518138+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'1","last_scrub_stamp":"2023-11-25T04:26:52.518023+0000","last_deep_scrub":"49'1","last_deep_scrub_stamp":"2023-11-25T04:26:52.518023+0000","last_clean_scrub_stamp":"2023-11-25T04:26:52.518023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:44:08.321741+0000","scrub_duration":0.001721757,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"75'194","reported_seq":274,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:15.719633+0000","last_change":"2023-11-25T04:26:15.719633+0000","last_active":"2023-11-25T04:26:15.719633+0000","last_peered":"2023-11-25T04:26:15.719633+0000","last_clean":"2023-11-25T04:26:15.719633+0000","last_became_active":"2023-11-25T04:22:47.038081+0000","last_became_peered":"2023-11-25T04:22:47.038081+0000","last_unstale":"2023-11-25T04:26:15.719633+0000","last_undegraded":"2023-11-25T04:26:15.719633+0000","last_fullsized":"2023-11-25T04:26:15.719633+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'194","last_scrub_stamp":"2023-11-25T04:26:15.719490+0000","last_deep_scrub":"75'194","last_deep_scrub_stamp":"2023-11-25T04:26:15.719490+0000","last_clean_scrub_stamp":"2023-11-25T04:26:15.719490+0000","objects_scrubbed":53,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:43:29.224433+0000","scrub_duration":0.037765356,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":174088,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":146,"num_write_kb":385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"75'159","reported_seq":244,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:21.467705+0000","last_change":"2023-11-25T04:27:21.467705+0000","last_active":"2023-11-25T04:27:21.467705+0000","last_peered":"2023-11-25T04:27:21.467705+0000","last_clean":"2023-11-25T04:27:21.467705+0000","last_became_active":"2023-11-25T04:25:46.999721+0000","last_became_peered":"2023-11-25T04:25:46.999721+0000","last_unstale":"2023-11-25T04:27:21.467705+0000","last_undegraded":"2023-11-25T04:27:21.467705+0000","last_fullsized":"2023-11-25T04:27:21.467705+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'159","last_scrub_stamp":"2023-11-25T04:27:21.467591+0000","last_deep_scrub":"75'159","last_deep_scrub_stamp":"2023-11-25T04:27:21.467591+0000","last_clean_scrub_stamp":"2023-11-25T04:27:21.467591+0000","objects_scrubbed":2,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:55:47.253212+0000","scrub_duration":0.005680058,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":44,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:19.454533+0000","last_change":"2023-11-25T04:27:19.454533+0000","last_active":"2023-11-25T04:27:19.454533+0000","last_peered":"2023-11-25T04:27:19.454533+0000","last_clean":"2023-11-25T04:27:19.454533+0000","last_became_active":"2023-11-25T04:25:46.997693+0000","last_became_peered":"2023-11-25T04:25:46.997693+0000","last_unstale":"2023-11-25T04:27:19.454533+0000","last_undegraded":"2023-11-25T04:27:19.454533+0000","last_fullsized":"2023-11-25T04:27:19.454533+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:27:19.454370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:27:19.454370+0000","last_clean_scrub_stamp":"2023-11-25T04:27:19.454370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:43:29.432453+0000","scrub_duration":0.001737955,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"46'1","reported_seq":80,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:25.228633+0000","last_change":"2023-11-25T04:27:25.228654+0000","last_active":"2023-11-25T04:27:25.228633+0000","last_peered":"2023-11-25T04:27:25.228633+0000","last_clean":"2023-11-25T04:27:25.228633+0000","last_became_active":"2023-11-25T04:22:46.035916+0000","last_became_peered":"2023-11-25T04:22:46.035916+0000","last_unstale":"2023-11-25T04:27:25.228633+0000","last_undegraded":"2023-11-25T04:27:25.228633+0000","last_fullsized":"2023-11-25T04:27:25.228633+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"46'1","last_scrub_stamp":"2023-11-25T04:27:25.228531+0000","last_deep_scrub":"46'1","last_deep_scrub_stamp":"2023-11-25T04:27:25.228531+0000","last_clean_scrub_stamp":"2023-11-25T04:27:25.228531+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:47:55.490961+0000","scrub_duration":0.0016495870000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"75'204","reported_seq":283,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:28.483954+0000","last_change":"2023-11-25T04:27:28.483954+0000","last_active":"2023-11-25T04:27:28.483954+0000","last_peered":"2023-11-25T04:27:28.483954+0000","last_clean":"2023-11-25T04:27:28.483954+0000","last_became_active":"2023-11-25T04:25:51.922490+0000","last_became_peered":"2023-11-25T04:25:51.922490+0000","last_unstale":"2023-11-25T04:27:28.483954+0000","last_undegraded":"2023-11-25T04:27:28.483954+0000","last_fullsized":"2023-11-25T04:27:28.483954+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'204","last_scrub_stamp":"2023-11-25T04:27:28.483839+0000","last_deep_scrub":"75'204","last_deep_scrub_stamp":"2023-11-25T04:27:28.483839+0000","last_clean_scrub_stamp":"2023-11-25T04:27:28.483839+0000","objects_scrubbed":58,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:49:24.463794+0000","scrub_duration":0.041785592000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":502154,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":170,"num_write_kb":687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":99,"num_bytes_recovered":502101,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":44,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:30.509621+0000","last_change":"2023-11-25T04:26:30.509621+0000","last_active":"2023-11-25T04:26:30.509621+0000","last_peered":"2023-11-25T04:26:30.509621+0000","last_clean":"2023-11-25T04:26:30.509621+0000","last_became_active":"2023-11-25T04:25:47.000033+0000","last_became_peered":"2023-11-25T04:25:47.000033+0000","last_unstale":"2023-11-25T04:26:30.509621+0000","last_undegraded":"2023-11-25T04:26:30.509621+0000","last_fullsized":"2023-11-25T04:26:30.509621+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:26:30.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:26:30.509522+0000","last_clean_scrub_stamp":"2023-11-25T04:26:30.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:17:09.583541+0000","scrub_duration":0.001771145,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"75'38","reported_seq":167,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:44.819150+0000","last_change":"2023-11-25T04:26:44.819150+0000","last_active":"2023-11-25T04:26:44.819150+0000","last_peered":"2023-11-25T04:26:44.819150+0000","last_clean":"2023-11-25T04:26:44.819150+0000","last_became_active":"2023-11-25T04:25:46.995637+0000","last_became_peered":"2023-11-25T04:25:46.995637+0000","last_unstale":"2023-11-25T04:26:44.819150+0000","last_undegraded":"2023-11-25T04:26:44.819150+0000","last_fullsized":"2023-11-25T04:26:44.819150+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'38","last_scrub_stamp":"2023-11-25T04:26:44.819010+0000","last_deep_scrub":"75'38","last_deep_scrub_stamp":"2023-11-25T04:26:44.819010+0000","last_clean_scrub_stamp":"2023-11-25T04:26:44.819010+0000","objects_scrubbed":1,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:17:29.276433+0000","scrub_duration":0.003812251,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":319160,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":35,"num_read_kb":334,"num_write":35,"num_write_kb":334,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":319160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"75'210","reported_seq":290,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:38.568589+0000","last_change":"2023-11-25T04:26:38.568589+0000","last_active":"2023-11-25T04:26:38.568589+0000","last_peered":"2023-11-25T04:26:38.568589+0000","last_clean":"2023-11-25T04:26:38.568589+0000","last_became_active":"2023-11-25T04:22:47.039197+0000","last_became_peered":"2023-11-25T04:22:47.039197+0000","last_unstale":"2023-11-25T04:26:38.568589+0000","last_undegraded":"2023-11-25T04:26:38.568589+0000","last_fullsized":"2023-11-25T04:26:38.568589+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'210","last_scrub_stamp":"2023-11-25T04:26:38.568488+0000","last_deep_scrub":"75'210","last_deep_scrub_stamp":"2023-11-25T04:26:38.568488+0000","last_clean_scrub_stamp":"2023-11-25T04:26:38.568488+0000","objects_scrubbed":52,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:17:23.025912+0000","scrub_duration":0.034345986000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":169303,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":189,"num_write_kb":333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":82,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:12.445153+0000","last_change":"2023-11-25T04:27:12.445171+0000","last_active":"2023-11-25T04:27:12.445153+0000","last_peered":"2023-11-25T04:27:12.445153+0000","last_clean":"2023-11-25T04:27:12.445153+0000","last_became_active":"2023-11-25T04:22:42.000041+0000","last_became_peered":"2023-11-25T04:22:42.000041+0000","last_unstale":"2023-11-25T04:27:12.445153+0000","last_undegraded":"2023-11-25T04:27:12.445153+0000","last_fullsized":"2023-11-25T04:27:12.445153+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:27:12.445067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:27:12.445067+0000","last_clean_scrub_stamp":"2023-11-25T04:27:12.445067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:18:00.627826+0000","scrub_duration":0.001522886,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"75'72","reported_seq":166,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:15.436284+0000","last_change":"2023-11-25T04:26:15.436284+0000","last_active":"2023-11-25T04:26:15.436284+0000","last_peered":"2023-11-25T04:26:15.436284+0000","last_clean":"2023-11-25T04:26:15.436284+0000","last_became_active":"2023-11-25T04:25:47.001162+0000","last_became_peered":"2023-11-25T04:25:47.001162+0000","last_unstale":"2023-11-25T04:26:15.436284+0000","last_undegraded":"2023-11-25T04:26:15.436284+0000","last_fullsized":"2023-11-25T04:26:15.436284+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'72","last_scrub_stamp":"2023-11-25T04:26:15.436148+0000","last_deep_scrub":"75'72","last_deep_scrub_stamp":"2023-11-25T04:26:15.436148+0000","last_clean_scrub_stamp":"2023-11-25T04:26:15.436148+0000","objects_scrubbed":1,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:37:38.643270+0000","scrub_duration":0.01087811,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":69,"num_write_kb":4130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4194304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"75'220","reported_seq":300,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:14.718055+0000","last_change":"2023-11-25T04:26:14.718055+0000","last_active":"2023-11-25T04:26:14.718055+0000","last_peered":"2023-11-25T04:26:14.718055+0000","last_clean":"2023-11-25T04:26:14.718055+0000","last_became_active":"2023-11-25T04:22:47.038046+0000","last_became_peered":"2023-11-25T04:22:47.038046+0000","last_unstale":"2023-11-25T04:26:14.718055+0000","last_undegraded":"2023-11-25T04:26:14.718055+0000","last_fullsized":"2023-11-25T04:26:14.718055+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'220","last_scrub_stamp":"2023-11-25T04:26:14.717897+0000","last_deep_scrub":"75'220","last_deep_scrub_stamp":"2023-11-25T04:26:14.717897+0000","last_clean_scrub_stamp":"2023-11-25T04:26:14.717897+0000","objects_scrubbed":62,"log_size":220,"log_dups_size":0,"ondisk_log_size":220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:43:17.788148+0000","scrub_duration":0.044136959000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":115501,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":163,"num_write_kb":243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":44,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:04.449359+0000","last_change":"2023-11-25T04:27:04.449359+0000","last_active":"2023-11-25T04:27:04.449359+0000","last_peered":"2023-11-25T04:27:04.449359+0000","last_clean":"2023-11-25T04:27:04.449359+0000","last_became_active":"2023-11-25T04:25:46.997391+0000","last_became_peered":"2023-11-25T04:25:46.997391+0000","last_unstale":"2023-11-25T04:27:04.449359+0000","last_undegraded":"2023-11-25T04:27:04.449359+0000","last_fullsized":"2023-11-25T04:27:04.449359+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:27:04.449261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:27:04.449261+0000","last_clean_scrub_stamp":"2023-11-25T04:27:04.449261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:55:52.982507+0000","scrub_duration":0.00176385,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"50'1","reported_seq":44,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:22.429663+0000","last_change":"2023-11-25T04:27:22.429663+0000","last_active":"2023-11-25T04:27:22.429663+0000","last_peered":"2023-11-25T04:27:22.429663+0000","last_clean":"2023-11-25T04:27:22.429663+0000","last_became_active":"2023-11-25T04:25:46.998132+0000","last_became_peered":"2023-11-25T04:25:46.998132+0000","last_unstale":"2023-11-25T04:27:22.429663+0000","last_undegraded":"2023-11-25T04:27:22.429663+0000","last_fullsized":"2023-11-25T04:27:22.429663+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'1","last_scrub_stamp":"2023-11-25T04:27:22.429551+0000","last_deep_scrub":"50'1","last_deep_scrub_stamp":"2023-11-25T04:27:22.429551+0000","last_clean_scrub_stamp":"2023-11-25T04:27:22.429551+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:41:27.142323+0000","scrub_duration":0.0015038580000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"75'206","reported_seq":285,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:19.694459+0000","last_change":"2023-11-25T04:26:19.694459+0000","last_active":"2023-11-25T04:26:19.694459+0000","last_peered":"2023-11-25T04:26:19.694459+0000","last_clean":"2023-11-25T04:26:19.694459+0000","last_became_active":"2023-11-25T04:22:47.036924+0000","last_became_peered":"2023-11-25T04:22:47.036924+0000","last_unstale":"2023-11-25T04:26:19.694459+0000","last_undegraded":"2023-11-25T04:26:19.694459+0000","last_fullsized":"2023-11-25T04:26:19.694459+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'206","last_scrub_stamp":"2023-11-25T04:26:19.694341+0000","last_deep_scrub":"75'206","last_deep_scrub_stamp":"2023-11-25T04:26:19.694341+0000","last_clean_scrub_stamp":"2023-11-25T04:26:19.694341+0000","objects_scrubbed":49,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:49:38.202017+0000","scrub_duration":0.031428719000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":93321,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":148,"num_write_kb":360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":44,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:26.443241+0000","last_change":"2023-11-25T04:27:26.443241+0000","last_active":"2023-11-25T04:27:26.443241+0000","last_peered":"2023-11-25T04:27:26.443241+0000","last_clean":"2023-11-25T04:27:26.443241+0000","last_became_active":"2023-11-25T04:25:46.999403+0000","last_became_peered":"2023-11-25T04:25:46.999403+0000","last_unstale":"2023-11-25T04:27:26.443241+0000","last_undegraded":"2023-11-25T04:27:26.443241+0000","last_fullsized":"2023-11-25T04:27:26.443241+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:27:26.443086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:27:26.443086+0000","last_clean_scrub_stamp":"2023-11-25T04:27:26.443086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:44:08.788259+0000","scrub_duration":0.001551729,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":82,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:23.259065+0000","last_change":"2023-11-25T04:27:23.259088+0000","last_active":"2023-11-25T04:27:23.259065+0000","last_peered":"2023-11-25T04:27:23.259065+0000","last_clean":"2023-11-25T04:27:23.259065+0000","last_became_active":"2023-11-25T04:22:41.999915+0000","last_became_peered":"2023-11-25T04:22:41.999915+0000","last_unstale":"2023-11-25T04:27:23.259065+0000","last_undegraded":"2023-11-25T04:27:23.259065+0000","last_fullsized":"2023-11-25T04:27:23.259065+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:27:23.258941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:27:23.258941+0000","last_clean_scrub_stamp":"2023-11-25T04:27:23.258941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:46:56.550165+0000","scrub_duration":0.0020338119999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"75'203","reported_seq":495,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:21.405576+0000","last_change":"2023-11-25T04:27:21.405602+0000","last_active":"2023-11-25T04:27:21.405576+0000","last_peered":"2023-11-25T04:27:21.405576+0000","last_clean":"2023-11-25T04:27:21.405576+0000","last_became_active":"2023-11-25T04:25:53.932586+0000","last_became_peered":"2023-11-25T04:25:53.932586+0000","last_unstale":"2023-11-25T04:27:21.405576+0000","last_undegraded":"2023-11-25T04:27:21.405576+0000","last_fullsized":"2023-11-25T04:27:21.405576+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'203","last_scrub_stamp":"2023-11-25T04:27:21.405481+0000","last_deep_scrub":"75'203","last_deep_scrub_stamp":"2023-11-25T04:27:21.405481+0000","last_clean_scrub_stamp":"2023-11-25T04:27:21.405481+0000","objects_scrubbed":51,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:22:14.998841+0000","scrub_duration":0.038507947000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":329327,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":151,"num_write_kb":471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":97,"num_bytes_recovered":329278,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":79,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:28.272266+0000","last_change":"2023-11-25T04:27:28.272286+0000","last_active":"2023-11-25T04:27:28.272266+0000","last_peered":"2023-11-25T04:27:28.272266+0000","last_clean":"2023-11-25T04:27:28.272266+0000","last_became_active":"2023-11-25T04:22:46.035405+0000","last_became_peered":"2023-11-25T04:22:46.035405+0000","last_unstale":"2023-11-25T04:27:28.272266+0000","last_undegraded":"2023-11-25T04:27:28.272266+0000","last_fullsized":"2023-11-25T04:27:28.272266+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:27:28.272187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:27:28.272187+0000","last_clean_scrub_stamp":"2023-11-25T04:27:28.272187+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:42:51.403345+0000","scrub_duration":0.001573315,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":82,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:49.454240+0000","last_change":"2023-11-25T04:26:49.454240+0000","last_active":"2023-11-25T04:26:49.454240+0000","last_peered":"2023-11-25T04:26:49.454240+0000","last_clean":"2023-11-25T04:26:49.454240+0000","last_became_active":"2023-11-25T04:22:41.998798+0000","last_became_peered":"2023-11-25T04:22:41.998798+0000","last_unstale":"2023-11-25T04:26:49.454240+0000","last_undegraded":"2023-11-25T04:26:49.454240+0000","last_fullsized":"2023-11-25T04:26:49.454240+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:26:49.454102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:26:49.454102+0000","last_clean_scrub_stamp":"2023-11-25T04:26:49.454102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:49:39.814155+0000","scrub_duration":0.0015686599999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"75'221","reported_seq":530,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:13.479673+0000","last_change":"2023-11-25T04:26:13.479673+0000","last_active":"2023-11-25T04:26:13.479673+0000","last_peered":"2023-11-25T04:26:13.479673+0000","last_clean":"2023-11-25T04:26:13.479673+0000","last_became_active":"2023-11-25T04:25:51.921858+0000","last_became_peered":"2023-11-25T04:25:51.921858+0000","last_unstale":"2023-11-25T04:26:13.479673+0000","last_undegraded":"2023-11-25T04:26:13.479673+0000","last_fullsized":"2023-11-25T04:26:13.479673+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'221","last_scrub_stamp":"2023-11-25T04:26:13.479549+0000","last_deep_scrub":"75'221","last_deep_scrub_stamp":"2023-11-25T04:26:13.479549+0000","last_clean_scrub_stamp":"2023-11-25T04:26:13.479549+0000","objects_scrubbed":49,"log_size":221,"log_dups_size":0,"ondisk_log_size":221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:44:56.142187+0000","scrub_duration":0.035520731999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":129965,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":166,"num_write_kb":290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":129965,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"75'3","reported_seq":82,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:42.231561+0000","last_change":"2023-11-25T04:27:42.231598+0000","last_active":"2023-11-25T04:27:42.231561+0000","last_peered":"2023-11-25T04:27:42.231561+0000","last_clean":"2023-11-25T04:27:42.231561+0000","last_became_active":"2023-11-25T04:22:46.034229+0000","last_became_peered":"2023-11-25T04:22:46.034229+0000","last_unstale":"2023-11-25T04:27:42.231561+0000","last_undegraded":"2023-11-25T04:27:42.231561+0000","last_fullsized":"2023-11-25T04:27:42.231561+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'3","last_scrub_stamp":"2023-11-25T04:27:42.231482+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2023-11-25T04:27:42.231482+0000","last_clean_scrub_stamp":"2023-11-25T04:27:42.231482+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T16:12:36.167899+0000","scrub_duration":0.0031395300000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":82,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:10.500127+0000","last_change":"2023-11-25T04:27:10.500147+0000","last_active":"2023-11-25T04:27:10.500127+0000","last_peered":"2023-11-25T04:27:10.500127+0000","last_clean":"2023-11-25T04:27:10.500127+0000","last_became_active":"2023-11-25T04:22:41.999992+0000","last_became_peered":"2023-11-25T04:22:41.999992+0000","last_unstale":"2023-11-25T04:27:10.500127+0000","last_undegraded":"2023-11-25T04:27:10.500127+0000","last_fullsized":"2023-11-25T04:27:10.500127+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:27:10.500045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:27:10.500045+0000","last_clean_scrub_stamp":"2023-11-25T04:27:10.500045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:57:30.011678+0000","scrub_duration":0.0018675779999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"54'150","reported_seq":438,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:09.671351+0000","last_change":"2023-11-25T04:25:54.938479+0000","last_active":"2023-11-25T04:26:09.671351+0000","last_peered":"2023-11-25T04:26:09.671351+0000","last_clean":"2023-11-25T04:26:09.671351+0000","last_became_active":"2023-11-25T04:25:54.937838+0000","last_became_peered":"2023-11-25T04:25:54.937838+0000","last_unstale":"2023-11-25T04:26:09.671351+0000","last_undegraded":"2023-11-25T04:26:09.671351+0000","last_fullsized":"2023-11-25T04:26:09.671351+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":159585,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":154224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"75'15","reported_seq":110,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:43.221454+0000","last_change":"2023-11-25T04:27:43.221493+0000","last_active":"2023-11-25T04:27:43.221454+0000","last_peered":"2023-11-25T04:27:43.221454+0000","last_clean":"2023-11-25T04:27:43.221454+0000","last_became_active":"2023-11-25T04:25:46.997404+0000","last_became_peered":"2023-11-25T04:25:46.997404+0000","last_unstale":"2023-11-25T04:27:43.221454+0000","last_undegraded":"2023-11-25T04:27:43.221454+0000","last_fullsized":"2023-11-25T04:27:43.221454+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'15","last_scrub_stamp":"2023-11-25T04:27:43.221358+0000","last_deep_scrub":"75'15","last_deep_scrub_stamp":"2023-11-25T04:27:43.221358+0000","last_clean_scrub_stamp":"2023-11-25T04:27:43.221358+0000","objects_scrubbed":1,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:22:36.814719+0000","scrub_duration":0.004299148,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":43,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:00.412619+0000","last_change":"2023-11-25T04:27:00.412639+0000","last_active":"2023-11-25T04:27:00.412619+0000","last_peered":"2023-11-25T04:27:00.412619+0000","last_clean":"2023-11-25T04:27:00.412619+0000","last_became_active":"2023-11-25T04:25:46.994834+0000","last_became_peered":"2023-11-25T04:25:46.994834+0000","last_unstale":"2023-11-25T04:27:00.412619+0000","last_undegraded":"2023-11-25T04:27:00.412619+0000","last_fullsized":"2023-11-25T04:27:00.412619+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:27:00.412529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:27:00.412529+0000","last_clean_scrub_stamp":"2023-11-25T04:27:00.412529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:30:28.354258+0000","scrub_duration":0.0016774050000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"75'191","reported_seq":475,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:14.466619+0000","last_change":"2023-11-25T04:26:14.466619+0000","last_active":"2023-11-25T04:26:14.466619+0000","last_peered":"2023-11-25T04:26:14.466619+0000","last_clean":"2023-11-25T04:26:14.466619+0000","last_became_active":"2023-11-25T04:25:49.921465+0000","last_became_peered":"2023-11-25T04:25:49.921465+0000","last_unstale":"2023-11-25T04:26:14.466619+0000","last_undegraded":"2023-11-25T04:26:14.466619+0000","last_fullsized":"2023-11-25T04:26:14.466619+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'191","last_scrub_stamp":"2023-11-25T04:26:14.466515+0000","last_deep_scrub":"75'191","last_deep_scrub_stamp":"2023-11-25T04:26:14.466515+0000","last_clean_scrub_stamp":"2023-11-25T04:26:14.466515+0000","objects_scrubbed":56,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:04:34.752431+0000","scrub_duration":0.037056879000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":99503,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":148,"num_write_kb":250,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":93,"num_bytes_recovered":99503,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"75'218","reported_seq":296,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:59.486464+0000","last_change":"2023-11-25T04:26:59.486489+0000","last_active":"2023-11-25T04:26:59.486464+0000","last_peered":"2023-11-25T04:26:59.486464+0000","last_clean":"2023-11-25T04:26:59.486464+0000","last_became_active":"2023-11-25T04:25:51.922070+0000","last_became_peered":"2023-11-25T04:25:51.922070+0000","last_unstale":"2023-11-25T04:26:59.486464+0000","last_undegraded":"2023-11-25T04:26:59.486464+0000","last_fullsized":"2023-11-25T04:26:59.486464+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'218","last_scrub_stamp":"2023-11-25T04:26:59.486387+0000","last_deep_scrub":"75'218","last_deep_scrub_stamp":"2023-11-25T04:26:59.486387+0000","last_clean_scrub_stamp":"2023-11-25T04:26:59.486387+0000","objects_scrubbed":52,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:32:01.142142+0000","scrub_duration":0.040859969000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":185321,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":172,"num_write_kb":312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":164176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"75'54","reported_seq":66,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:58.474448+0000","last_change":"2023-11-25T04:26:58.474489+0000","last_active":"2023-11-25T04:26:58.474448+0000","last_peered":"2023-11-25T04:26:58.474448+0000","last_clean":"2023-11-25T04:26:58.474448+0000","last_became_active":"2023-11-25T04:25:46.994997+0000","last_became_peered":"2023-11-25T04:25:46.994997+0000","last_unstale":"2023-11-25T04:26:58.474448+0000","last_undegraded":"2023-11-25T04:26:58.474448+0000","last_fullsized":"2023-11-25T04:26:58.474448+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'54","last_scrub_stamp":"2023-11-25T04:26:58.474367+0000","last_deep_scrub":"75'54","last_deep_scrub_stamp":"2023-11-25T04:26:58.474367+0000","last_clean_scrub_stamp":"2023-11-25T04:26:58.474367+0000","objects_scrubbed":4,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:32:12.109149+0000","scrub_duration":0.0152154,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":4119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4190183,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"75'199","reported_seq":278,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:43.858609+0000","last_change":"2023-11-25T04:26:43.858609+0000","last_active":"2023-11-25T04:26:43.858609+0000","last_peered":"2023-11-25T04:26:43.858609+0000","last_clean":"2023-11-25T04:26:43.858609+0000","last_became_active":"2023-11-25T04:22:47.036821+0000","last_became_peered":"2023-11-25T04:22:47.036821+0000","last_unstale":"2023-11-25T04:26:43.858609+0000","last_undegraded":"2023-11-25T04:26:43.858609+0000","last_fullsized":"2023-11-25T04:26:43.858609+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'199","last_scrub_stamp":"2023-11-25T04:26:43.858473+0000","last_deep_scrub":"75'199","last_deep_scrub_stamp":"2023-11-25T04:26:43.858473+0000","last_clean_scrub_stamp":"2023-11-25T04:26:43.858473+0000","objects_scrubbed":46,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:28:32.758038+0000","scrub_duration":0.029966083000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":101117,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":161,"num_write_kb":242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"75'24","reported_seq":119,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:39.535658+0000","last_change":"2023-11-25T04:26:39.535658+0000","last_active":"2023-11-25T04:26:39.535658+0000","last_peered":"2023-11-25T04:26:39.535658+0000","last_clean":"2023-11-25T04:26:39.535658+0000","last_became_active":"2023-11-25T04:25:47.000478+0000","last_became_peered":"2023-11-25T04:25:47.000478+0000","last_unstale":"2023-11-25T04:26:39.535658+0000","last_undegraded":"2023-11-25T04:26:39.535658+0000","last_fullsized":"2023-11-25T04:26:39.535658+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'24","last_scrub_stamp":"2023-11-25T04:26:39.535504+0000","last_deep_scrub":"75'24","last_deep_scrub_stamp":"2023-11-25T04:26:39.535504+0000","last_clean_scrub_stamp":"2023-11-25T04:26:39.535504+0000","objects_scrubbed":2,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:50:41.041415+0000","scrub_duration":0.0040188020000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":90,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":43,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:54.349677+0000","last_change":"2023-11-25T04:26:54.349711+0000","last_active":"2023-11-25T04:26:54.349677+0000","last_peered":"2023-11-25T04:26:54.349677+0000","last_clean":"2023-11-25T04:26:54.349677+0000","last_became_active":"2023-11-25T04:25:46.994525+0000","last_became_peered":"2023-11-25T04:25:46.994525+0000","last_unstale":"2023-11-25T04:26:54.349677+0000","last_undegraded":"2023-11-25T04:26:54.349677+0000","last_fullsized":"2023-11-25T04:26:54.349677+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:26:54.349592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:26:54.349592+0000","last_clean_scrub_stamp":"2023-11-25T04:26:54.349592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:34:14.985604+0000","scrub_duration":0.0018546229999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"75'237","reported_seq":557,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:47.242584+0000","last_change":"2023-11-25T04:27:47.242637+0000","last_active":"2023-11-25T04:27:47.242584+0000","last_peered":"2023-11-25T04:27:47.242584+0000","last_clean":"2023-11-25T04:27:47.242584+0000","last_became_active":"2023-11-25T04:25:51.921938+0000","last_became_peered":"2023-11-25T04:25:51.921938+0000","last_unstale":"2023-11-25T04:27:47.242584+0000","last_undegraded":"2023-11-25T04:27:47.242584+0000","last_fullsized":"2023-11-25T04:27:47.242584+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'237","last_scrub_stamp":"2023-11-25T04:27:47.242471+0000","last_deep_scrub":"75'237","last_deep_scrub_stamp":"2023-11-25T04:27:47.242471+0000","last_clean_scrub_stamp":"2023-11-25T04:27:47.242471+0000","objects_scrubbed":60,"log_size":237,"log_dups_size":0,"ondisk_log_size":237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:04:06.543454+0000","scrub_duration":0.050113891000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":522494,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":168,"num_write_kb":749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":111,"num_bytes_recovered":522494,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"46'2","reported_seq":92,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:51.691533+0000","last_change":"2023-11-25T04:26:51.691533+0000","last_active":"2023-11-25T04:26:51.691533+0000","last_peered":"2023-11-25T04:26:51.691533+0000","last_clean":"2023-11-25T04:26:51.691533+0000","last_became_active":"2023-11-25T04:25:46.998508+0000","last_became_peered":"2023-11-25T04:25:46.998508+0000","last_unstale":"2023-11-25T04:26:51.691533+0000","last_undegraded":"2023-11-25T04:26:51.691533+0000","last_fullsized":"2023-11-25T04:26:51.691533+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"46'2","last_scrub_stamp":"2023-11-25T04:26:51.691391+0000","last_deep_scrub":"46'2","last_deep_scrub_stamp":"2023-11-25T04:26:51.691391+0000","last_clean_scrub_stamp":"2023-11-25T04:26:51.691391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:31:13.002647+0000","scrub_duration":0.0021417300000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.10","version":"75'209","reported_seq":289,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:02.586496+0000","last_change":"2023-11-25T04:27:02.586496+0000","last_active":"2023-11-25T04:27:02.586496+0000","last_peered":"2023-11-25T04:27:02.586496+0000","last_clean":"2023-11-25T04:27:02.586496+0000","last_became_active":"2023-11-25T04:22:47.037149+0000","last_became_peered":"2023-11-25T04:22:47.037149+0000","last_unstale":"2023-11-25T04:27:02.586496+0000","last_undegraded":"2023-11-25T04:27:02.586496+0000","last_fullsized":"2023-11-25T04:27:02.586496+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'209","last_scrub_stamp":"2023-11-25T04:27:02.586379+0000","last_deep_scrub":"75'209","last_deep_scrub_stamp":"2023-11-25T04:27:02.586379+0000","last_clean_scrub_stamp":"2023-11-25T04:27:02.586379+0000","objects_scrubbed":56,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:55:23.713881+0000","scrub_duration":0.037826797000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":218073,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1f","version":"75'214","reported_seq":518,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:16.412889+0000","last_change":"2023-11-25T04:27:16.412922+0000","last_active":"2023-11-25T04:27:16.412889+0000","last_peered":"2023-11-25T04:27:16.412889+0000","last_clean":"2023-11-25T04:27:16.412889+0000","last_became_active":"2023-11-25T04:25:49.921458+0000","last_became_peered":"2023-11-25T04:25:49.921458+0000","last_unstale":"2023-11-25T04:27:16.412889+0000","last_undegraded":"2023-11-25T04:27:16.412889+0000","last_fullsized":"2023-11-25T04:27:16.412889+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'214","last_scrub_stamp":"2023-11-25T04:27:16.412805+0000","last_deep_scrub":"75'214","last_deep_scrub_stamp":"2023-11-25T04:27:16.412805+0000","last_clean_scrub_stamp":"2023-11-25T04:27:16.412805+0000","objects_scrubbed":58,"log_size":214,"log_dups_size":0,"ondisk_log_size":214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:05:28.377468+0000","scrub_duration":0.044986529999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":107712,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":103,"num_bytes_recovered":107703,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"48'1","reported_seq":80,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:27.250447+0000","last_change":"2023-11-25T04:27:27.250447+0000","last_active":"2023-11-25T04:27:27.250447+0000","last_peered":"2023-11-25T04:27:27.250447+0000","last_clean":"2023-11-25T04:27:27.250447+0000","last_became_active":"2023-11-25T04:22:46.035495+0000","last_became_peered":"2023-11-25T04:22:46.035495+0000","last_unstale":"2023-11-25T04:27:27.250447+0000","last_undegraded":"2023-11-25T04:27:27.250447+0000","last_fullsized":"2023-11-25T04:27:27.250447+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"48'1","last_scrub_stamp":"2023-11-25T04:27:27.250306+0000","last_deep_scrub":"48'1","last_deep_scrub_stamp":"2023-11-25T04:27:27.250306+0000","last_clean_scrub_stamp":"2023-11-25T04:27:27.250306+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:52:37.111984+0000","scrub_duration":0.001682764,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"75'200","reported_seq":489,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:48.198585+0000","last_change":"2023-11-25T04:27:48.198635+0000","last_active":"2023-11-25T04:27:48.198585+0000","last_peered":"2023-11-25T04:27:48.198585+0000","last_clean":"2023-11-25T04:27:48.198585+0000","last_became_active":"2023-11-25T04:25:50.923375+0000","last_became_peered":"2023-11-25T04:25:50.923375+0000","last_unstale":"2023-11-25T04:27:48.198585+0000","last_undegraded":"2023-11-25T04:27:48.198585+0000","last_fullsized":"2023-11-25T04:27:48.198585+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'200","last_scrub_stamp":"2023-11-25T04:27:48.198510+0000","last_deep_scrub":"75'200","last_deep_scrub_stamp":"2023-11-25T04:27:48.198510+0000","last_clean_scrub_stamp":"2023-11-25T04:27:48.198510+0000","objects_scrubbed":48,"log_size":200,"log_dups_size":0,"ondisk_log_size":200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:12:22.989388+0000","scrub_duration":0.034965662000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":76435,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":76435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":90,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:23.683288+0000","last_change":"2023-11-25T04:26:23.683288+0000","last_active":"2023-11-25T04:26:23.683288+0000","last_peered":"2023-11-25T04:26:23.683288+0000","last_clean":"2023-11-25T04:26:23.683288+0000","last_became_active":"2023-11-25T04:25:46.994871+0000","last_became_peered":"2023-11-25T04:25:46.994871+0000","last_unstale":"2023-11-25T04:26:23.683288+0000","last_undegraded":"2023-11-25T04:26:23.683288+0000","last_fullsized":"2023-11-25T04:26:23.683288+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:26:23.683176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:26:23.683176+0000","last_clean_scrub_stamp":"2023-11-25T04:26:23.683176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:31:54.450606+0000","scrub_duration":0.00162587,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"75'207","reported_seq":492,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:23.401113+0000","last_change":"2023-11-25T04:27:23.401113+0000","last_active":"2023-11-25T04:27:23.401113+0000","last_peered":"2023-11-25T04:27:23.401113+0000","last_clean":"2023-11-25T04:27:23.401113+0000","last_became_active":"2023-11-25T04:25:52.925038+0000","last_became_peered":"2023-11-25T04:25:52.925038+0000","last_unstale":"2023-11-25T04:27:23.401113+0000","last_undegraded":"2023-11-25T04:27:23.401113+0000","last_fullsized":"2023-11-25T04:27:23.401113+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'207","last_scrub_stamp":"2023-11-25T04:27:23.400949+0000","last_deep_scrub":"75'207","last_deep_scrub_stamp":"2023-11-25T04:27:23.400949+0000","last_clean_scrub_stamp":"2023-11-25T04:27:23.400949+0000","objects_scrubbed":49,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:20:49.215409+0000","scrub_duration":0.040233620999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":209500,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":139,"num_write_kb":351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":209500,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1a","version":"75'5","reported_seq":84,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:51.508225+0000","last_change":"2023-11-25T04:26:51.508225+0000","last_active":"2023-11-25T04:26:51.508225+0000","last_peered":"2023-11-25T04:26:51.508225+0000","last_clean":"2023-11-25T04:26:51.508225+0000","last_became_active":"2023-11-25T04:22:46.033944+0000","last_became_peered":"2023-11-25T04:22:46.033944+0000","last_unstale":"2023-11-25T04:26:51.508225+0000","last_undegraded":"2023-11-25T04:26:51.508225+0000","last_fullsized":"2023-11-25T04:26:51.508225+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'5","last_scrub_stamp":"2023-11-25T04:26:51.508119+0000","last_deep_scrub":"75'5","last_deep_scrub_stamp":"2023-11-25T04:26:51.508119+0000","last_clean_scrub_stamp":"2023-11-25T04:26:51.508119+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:55:12.635621+0000","scrub_duration":0.0083650619999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"75'218","reported_seq":298,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:28.540082+0000","last_change":"2023-11-25T04:26:28.540082+0000","last_active":"2023-11-25T04:26:28.540082+0000","last_peered":"2023-11-25T04:26:28.540082+0000","last_clean":"2023-11-25T04:26:28.540082+0000","last_became_active":"2023-11-25T04:22:47.039218+0000","last_became_peered":"2023-11-25T04:22:47.039218+0000","last_unstale":"2023-11-25T04:26:28.540082+0000","last_undegraded":"2023-11-25T04:26:28.540082+0000","last_fullsized":"2023-11-25T04:26:28.540082+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'218","last_scrub_stamp":"2023-11-25T04:26:28.539970+0000","last_deep_scrub":"75'218","last_deep_scrub_stamp":"2023-11-25T04:26:28.539970+0000","last_clean_scrub_stamp":"2023-11-25T04:26:28.539970+0000","objects_scrubbed":55,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:46:27.913704+0000","scrub_duration":0.040559401000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":165883,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"75'10","reported_seq":105,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:20.499236+0000","last_change":"2023-11-25T04:26:20.499236+0000","last_active":"2023-11-25T04:26:20.499236+0000","last_peered":"2023-11-25T04:26:20.499236+0000","last_clean":"2023-11-25T04:26:20.499236+0000","last_became_active":"2023-11-25T04:25:46.999998+0000","last_became_peered":"2023-11-25T04:25:46.999998+0000","last_unstale":"2023-11-25T04:26:20.499236+0000","last_undegraded":"2023-11-25T04:26:20.499236+0000","last_fullsized":"2023-11-25T04:26:20.499236+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'10","last_scrub_stamp":"2023-11-25T04:26:20.499093+0000","last_deep_scrub":"75'10","last_deep_scrub_stamp":"2023-11-25T04:26:20.499093+0000","last_clean_scrub_stamp":"2023-11-25T04:26:20.499093+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:35:51.569332+0000","scrub_duration":0.0094161829999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":4101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4194304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"75'212","reported_seq":291,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:39.477807+0000","last_change":"2023-11-25T04:26:39.477840+0000","last_active":"2023-11-25T04:26:39.477807+0000","last_peered":"2023-11-25T04:26:39.477807+0000","last_clean":"2023-11-25T04:26:39.477807+0000","last_became_active":"2023-11-25T04:25:50.922164+0000","last_became_peered":"2023-11-25T04:25:50.922164+0000","last_unstale":"2023-11-25T04:26:39.477807+0000","last_undegraded":"2023-11-25T04:26:39.477807+0000","last_fullsized":"2023-11-25T04:26:39.477807+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'212","last_scrub_stamp":"2023-11-25T04:26:39.477723+0000","last_deep_scrub":"75'212","last_deep_scrub_stamp":"2023-11-25T04:26:39.477723+0000","last_clean_scrub_stamp":"2023-11-25T04:26:39.477723+0000","objects_scrubbed":63,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:34:17.609339+0000","scrub_duration":0.044092971000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":179506,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":155,"num_write_kb":317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":179506,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"75'3","reported_seq":47,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:25.424173+0000","last_change":"2023-11-25T04:27:25.424197+0000","last_active":"2023-11-25T04:27:25.424173+0000","last_peered":"2023-11-25T04:27:25.424173+0000","last_clean":"2023-11-25T04:27:25.424173+0000","last_became_active":"2023-11-25T04:25:46.995160+0000","last_became_peered":"2023-11-25T04:25:46.995160+0000","last_unstale":"2023-11-25T04:27:25.424173+0000","last_undegraded":"2023-11-25T04:27:25.424173+0000","last_fullsized":"2023-11-25T04:27:25.424173+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'3","last_scrub_stamp":"2023-11-25T04:27:25.424088+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2023-11-25T04:27:25.424088+0000","last_clean_scrub_stamp":"2023-11-25T04:27:25.424088+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:47:26.636974+0000","scrub_duration":0.0032298869999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1a","version":"75'213","reported_seq":288,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:46.569042+0000","last_change":"2023-11-25T04:27:46.569068+0000","last_active":"2023-11-25T04:27:46.569042+0000","last_peered":"2023-11-25T04:27:46.569042+0000","last_clean":"2023-11-25T04:27:46.569042+0000","last_became_active":"2023-11-25T04:25:54.937821+0000","last_became_peered":"2023-11-25T04:25:54.937821+0000","last_unstale":"2023-11-25T04:27:46.569042+0000","last_undegraded":"2023-11-25T04:27:46.569042+0000","last_fullsized":"2023-11-25T04:27:46.569042+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'213","last_scrub_stamp":"2023-11-25T04:27:46.568971+0000","last_deep_scrub":"75'213","last_deep_scrub_stamp":"2023-11-25T04:27:46.568971+0000","last_clean_scrub_stamp":"2023-11-25T04:27:46.568971+0000","objects_scrubbed":51,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:40:48.120829+0000","scrub_duration":0.035024894000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":134672,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":169,"num_write_kb":311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":126291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"75'62","reported_seq":159,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:30.554596+0000","last_change":"2023-11-25T04:26:30.554596+0000","last_active":"2023-11-25T04:26:30.554596+0000","last_peered":"2023-11-25T04:26:30.554596+0000","last_clean":"2023-11-25T04:26:30.554596+0000","last_became_active":"2023-11-25T04:25:47.000722+0000","last_became_peered":"2023-11-25T04:25:47.000722+0000","last_unstale":"2023-11-25T04:26:30.554596+0000","last_undegraded":"2023-11-25T04:26:30.554596+0000","last_fullsized":"2023-11-25T04:26:30.554596+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'62","last_scrub_stamp":"2023-11-25T04:26:30.554482+0000","last_deep_scrub":"75'62","last_deep_scrub_stamp":"2023-11-25T04:26:30.554482+0000","last_clean_scrub_stamp":"2023-11-25T04:26:30.554482+0000","objects_scrubbed":3,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:38:15.140403+0000","scrub_duration":0.015921703999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8388608,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"75'221","reported_seq":301,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:55.521345+0000","last_change":"2023-11-25T04:26:55.521345+0000","last_active":"2023-11-25T04:26:55.521345+0000","last_peered":"2023-11-25T04:26:55.521345+0000","last_clean":"2023-11-25T04:26:55.521345+0000","last_became_active":"2023-11-25T04:22:47.036406+0000","last_became_peered":"2023-11-25T04:22:47.036406+0000","last_unstale":"2023-11-25T04:26:55.521345+0000","last_undegraded":"2023-11-25T04:26:55.521345+0000","last_fullsized":"2023-11-25T04:26:55.521345+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'221","last_scrub_stamp":"2023-11-25T04:26:55.521198+0000","last_deep_scrub":"75'221","last_deep_scrub_stamp":"2023-11-25T04:26:55.521198+0000","last_clean_scrub_stamp":"2023-11-25T04:26:55.521198+0000","objects_scrubbed":58,"log_size":221,"log_dups_size":0,"ondisk_log_size":221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:17:51.455919+0000","scrub_duration":0.041323743000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":527891,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":163,"num_write_kb":687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1e","version":"75'3","reported_seq":82,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:35.296060+0000","last_change":"2023-11-25T04:27:35.296086+0000","last_active":"2023-11-25T04:27:35.296060+0000","last_peered":"2023-11-25T04:27:35.296060+0000","last_clean":"2023-11-25T04:27:35.296060+0000","last_became_active":"2023-11-25T04:22:46.035113+0000","last_became_peered":"2023-11-25T04:22:46.035113+0000","last_unstale":"2023-11-25T04:27:35.296060+0000","last_undegraded":"2023-11-25T04:27:35.296060+0000","last_fullsized":"2023-11-25T04:27:35.296060+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'3","last_scrub_stamp":"2023-11-25T04:27:35.295959+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2023-11-25T04:27:35.295959+0000","last_clean_scrub_stamp":"2023-11-25T04:27:35.295959+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:19:05.099964+0000","scrub_duration":0.0087757789999999992,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.18","version":"54'195","reported_seq":260,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:00.986052+0000","last_change":"2023-11-25T04:25:54.938573+0000","last_active":"2023-11-25T04:26:00.986052+0000","last_peered":"2023-11-25T04:26:00.986052+0000","last_clean":"2023-11-25T04:26:00.986052+0000","last_became_active":"2023-11-25T04:25:54.937444+0000","last_became_peered":"2023-11-25T04:25:54.937444+0000","last_unstale":"2023-11-25T04:26:00.986052+0000","last_undegraded":"2023-11-25T04:26:00.986052+0000","last_fullsized":"2023-11-25T04:26:00.986052+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:22:46.014230+0000","last_clean_scrub_stamp":"2023-11-25T04:22:46.014230+0000","objects_scrubbed":0,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":128641,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":185,"num_write_kb":408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":122,"num_bytes_recovered":128641,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"75'20","reported_seq":115,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:49.762359+0000","last_change":"2023-11-25T04:26:49.762359+0000","last_active":"2023-11-25T04:26:49.762359+0000","last_peered":"2023-11-25T04:26:49.762359+0000","last_clean":"2023-11-25T04:26:49.762359+0000","last_became_active":"2023-11-25T04:25:46.998870+0000","last_became_peered":"2023-11-25T04:25:46.998870+0000","last_unstale":"2023-11-25T04:26:49.762359+0000","last_undegraded":"2023-11-25T04:26:49.762359+0000","last_fullsized":"2023-11-25T04:26:49.762359+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'20","last_scrub_stamp":"2023-11-25T04:26:49.762253+0000","last_deep_scrub":"75'20","last_deep_scrub_stamp":"2023-11-25T04:26:49.762253+0000","last_clean_scrub_stamp":"2023-11-25T04:26:49.762253+0000","objects_scrubbed":3,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:38:21.361584+0000","scrub_duration":0.0095441299999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":4103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4194304,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":26330855,"num_objects":3447,"num_object_clones":0,"num_object_copies":6894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10423,"num_write_kb":38550,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4189,"num_bytes_recovered":12749181,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":72376320,"data_stored":52324046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13393,"ondisk_log_size":13393,"up":128,"acting":128,"num_store_stats":3},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":40885548,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":35,"num_read_kb":334,"num_write":435,"num_write_kb":40462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":32491439,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":81833984,"data_stored":81771096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5548,"internal_metadata":0},"log_size":467,"ondisk_log_size":467,"up":64,"acting":64,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738436,"num_pgs":72,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":99572,"kb_used_data":46188,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94272268,"statfs":{"total":96636764160,"available":96534802432,"internally_reserved":0,"allocated":47296512,"data_stored":39787071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 25 04:26:49 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.51900000000000002}]},{"osd":1,"last update":"Sat Nov 25 04:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.44400000000000001,"15min":0.44900000000000001},"min":{"1min":0.39400000000000002,"5min":0.216,"15min":0.216},"max":{"1min":0.65200000000000002,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.501,"5min":0.44600000000000001,"15min":0.45400000000000001},"min":{"1min":0.41399999999999998,"5min":0.184,"15min":0.184},"max":{"1min":0.59599999999999997,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.45900000000000002}]}]},{"osd":1,"up_from":8,"seq":34359738436,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":95196,"kb_used_data":55700,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94276644,"statfs":{"total":96636764160,"available":96539283456,"internally_reserved":0,"allocated":57036800,"data_stored":50261554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 25 04:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.436,"15min":0.438},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.57099999999999995,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.20999999999999999},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.45400000000000001,"15min":0.45600000000000002},"min":{"1min":0.255,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.61899999999999999,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.27200000000000002}]},{"osd":2,"last update":"Sat Nov 25 04:27:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.439,"15min":0.42199999999999999},"min":{"1min":0.379,"5min":0.182,"15min":0.182},"max":{"1min":0.60099999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.439,"15min":0.41799999999999998},"min":{"1min":0.35699999999999998,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.629,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.24299999999999999}]}]},{"osd":0,"up_from":60,"seq":257698037787,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79292,"kb_used_data":51764,"kb_used_omap":6,"kb_used_meta":27513,"kb_avail":94292548,"statfs":{"total":96636764160,"available":96555569152,"internally_reserved":0,"allocated":53006336,"data_stored":46141107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7143,"internal_metadata":28173337},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Nov 25 04:26:51 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.41899999999999998}]},{"osd":2,"last update":"Sat Nov 25 04:26:51 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.52100000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32530432,"data_stored":32496940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5548,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":29380608,"data_stored":29360982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":19922944,"data_stored":19913174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":19234816,"data_stored":12749181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":26415104,"data_stored":20005586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":26726400,"data_stored":19569279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-25T04:27:51.949 INFO:tasks.ceph:pgid 4.26 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:27:51.950 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:27:51.950 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:27:51.951 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:27:51.952 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2023-11-25T04:22:46.014230+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=22, tm_sec=46, tm_wday=5, tm_yday=329, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=25, tm_hour=4, tm_min=26, tm_sec=7, tm_wday=5, tm_yday=329, tm_isdst=0) 2023-11-25T04:27:51.952 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-11-25T04:28:11.953 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-25T04:28:12.268 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:28:12.270 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2023-11-25T04:28:12.279 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":242,"stamp":"2023-11-25T04:28:11.583532+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":67806771,"num_objects":3481,"num_object_clones":0,"num_object_copies":6962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3481,"num_whiteouts":0,"num_read":171,"num_read_kb":452,"num_write":11061,"num_write_kb":82472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4201,"num_bytes_recovered":45240620,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14319,"ondisk_log_size":14319,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":274060,"kb_used_data":153652,"kb_used_omap":10,"kb_used_meta":120373,"kb_avail":282841460,"statfs":{"total":289910292480,"available":289629655040,"internally_reserved":0,"allocated":157339648,"data_stored":136189732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10333,"internal_metadata":123262883},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003320"},"pg_stats":[{"pgid":"4.3f","version":"75'212","reported_seq":291,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:48.487625+0000","last_change":"2023-11-25T04:26:48.487625+0000","last_active":"2023-11-25T04:26:48.487625+0000","last_peered":"2023-11-25T04:26:48.487625+0000","last_clean":"2023-11-25T04:26:48.487625+0000","last_became_active":"2023-11-25T04:22:47.039490+0000","last_became_peered":"2023-11-25T04:22:47.039490+0000","last_unstale":"2023-11-25T04:26:48.487625+0000","last_undegraded":"2023-11-25T04:26:48.487625+0000","last_fullsized":"2023-11-25T04:26:48.487625+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'212","last_scrub_stamp":"2023-11-25T04:26:48.487485+0000","last_deep_scrub":"75'212","last_deep_scrub_stamp":"2023-11-25T04:26:48.487485+0000","last_clean_scrub_stamp":"2023-11-25T04:26:48.487485+0000","objects_scrubbed":46,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:02:55.160351+0000","scrub_duration":0.031121083000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":577866,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":174,"num_write_kb":756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3e","version":"75'257","reported_seq":336,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:16.435276+0000","last_change":"2023-11-25T04:26:16.435276+0000","last_active":"2023-11-25T04:26:16.435276+0000","last_peered":"2023-11-25T04:26:16.435276+0000","last_clean":"2023-11-25T04:26:16.435276+0000","last_became_active":"2023-11-25T04:22:47.040511+0000","last_became_peered":"2023-11-25T04:22:47.040511+0000","last_unstale":"2023-11-25T04:26:16.435276+0000","last_undegraded":"2023-11-25T04:26:16.435276+0000","last_fullsized":"2023-11-25T04:26:16.435276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'257","last_scrub_stamp":"2023-11-25T04:26:16.435161+0000","last_deep_scrub":"75'257","last_deep_scrub_stamp":"2023-11-25T04:26:16.435161+0000","last_clean_scrub_stamp":"2023-11-25T04:26:16.435161+0000","objects_scrubbed":45,"log_size":257,"log_dups_size":0,"ondisk_log_size":257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:13:07.641904+0000","scrub_duration":0.036488224,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":461579,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":236,"num_write_kb":841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3d","version":"75'209","reported_seq":286,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:12.330801+0000","last_change":"2023-11-25T04:26:12.330801+0000","last_active":"2023-11-25T04:26:12.330801+0000","last_peered":"2023-11-25T04:26:12.330801+0000","last_clean":"2023-11-25T04:26:12.330801+0000","last_became_active":"2023-11-25T04:25:52.923459+0000","last_became_peered":"2023-11-25T04:25:52.923459+0000","last_unstale":"2023-11-25T04:26:12.330801+0000","last_undegraded":"2023-11-25T04:26:12.330801+0000","last_fullsized":"2023-11-25T04:26:12.330801+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'209","last_scrub_stamp":"2023-11-25T04:26:12.330672+0000","last_deep_scrub":"75'209","last_deep_scrub_stamp":"2023-11-25T04:26:12.330672+0000","last_clean_scrub_stamp":"2023-11-25T04:26:12.330672+0000","objects_scrubbed":42,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:49:33.381569+0000","scrub_duration":0.027938688999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":145844,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":175,"num_write_kb":312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":138765,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"75'172","reported_seq":250,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:57.449017+0000","last_change":"2023-11-25T04:26:57.449017+0000","last_active":"2023-11-25T04:26:57.449017+0000","last_peered":"2023-11-25T04:26:57.449017+0000","last_clean":"2023-11-25T04:26:57.449017+0000","last_became_active":"2023-11-25T04:25:51.921633+0000","last_became_peered":"2023-11-25T04:25:51.921633+0000","last_unstale":"2023-11-25T04:26:57.449017+0000","last_undegraded":"2023-11-25T04:26:57.449017+0000","last_fullsized":"2023-11-25T04:26:57.449017+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'172","last_scrub_stamp":"2023-11-25T04:26:57.448886+0000","last_deep_scrub":"75'172","last_deep_scrub_stamp":"2023-11-25T04:26:57.448886+0000","last_clean_scrub_stamp":"2023-11-25T04:26:57.448886+0000","objects_scrubbed":48,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:58:16.336649+0000","scrub_duration":0.030012414000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61303,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119,"num_write_kb":161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":61303,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"75'236","reported_seq":316,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:03.442984+0000","last_change":"2023-11-25T04:27:03.442984+0000","last_active":"2023-11-25T04:27:03.442984+0000","last_peered":"2023-11-25T04:27:03.442984+0000","last_clean":"2023-11-25T04:27:03.442984+0000","last_became_active":"2023-11-25T04:22:47.039165+0000","last_became_peered":"2023-11-25T04:22:47.039165+0000","last_unstale":"2023-11-25T04:27:03.442984+0000","last_undegraded":"2023-11-25T04:27:03.442984+0000","last_fullsized":"2023-11-25T04:27:03.442984+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'236","last_scrub_stamp":"2023-11-25T04:27:03.442858+0000","last_deep_scrub":"75'236","last_deep_scrub_stamp":"2023-11-25T04:27:03.442858+0000","last_clean_scrub_stamp":"2023-11-25T04:27:03.442858+0000","objects_scrubbed":67,"log_size":236,"log_dups_size":0,"ondisk_log_size":236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:08:36.083041+0000","scrub_duration":0.043065032000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":118227,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":185,"num_write_kb":351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3a","version":"75'220","reported_seq":296,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:37.556263+0000","last_change":"2023-11-25T04:27:37.556287+0000","last_active":"2023-11-25T04:27:37.556263+0000","last_peered":"2023-11-25T04:27:37.556263+0000","last_clean":"2023-11-25T04:27:37.556263+0000","last_became_active":"2023-11-25T04:25:53.931982+0000","last_became_peered":"2023-11-25T04:25:53.931982+0000","last_unstale":"2023-11-25T04:27:37.556263+0000","last_undegraded":"2023-11-25T04:27:37.556263+0000","last_fullsized":"2023-11-25T04:27:37.556263+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'220","last_scrub_stamp":"2023-11-25T04:27:37.556170+0000","last_deep_scrub":"75'220","last_deep_scrub_stamp":"2023-11-25T04:27:37.556170+0000","last_clean_scrub_stamp":"2023-11-25T04:27:37.556170+0000","objects_scrubbed":51,"log_size":220,"log_dups_size":0,"ondisk_log_size":220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:16:43.691698+0000","scrub_duration":0.045038175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100304,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":161,"num_write_kb":424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":100304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"75'228","reported_seq":308,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:27.488226+0000","last_change":"2023-11-25T04:26:27.488269+0000","last_active":"2023-11-25T04:26:27.488226+0000","last_peered":"2023-11-25T04:26:27.488226+0000","last_clean":"2023-11-25T04:26:27.488226+0000","last_became_active":"2023-11-25T04:25:49.919354+0000","last_became_peered":"2023-11-25T04:25:49.919354+0000","last_unstale":"2023-11-25T04:26:27.488226+0000","last_undegraded":"2023-11-25T04:26:27.488226+0000","last_fullsized":"2023-11-25T04:26:27.488226+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'228","last_scrub_stamp":"2023-11-25T04:26:27.488145+0000","last_deep_scrub":"75'228","last_deep_scrub_stamp":"2023-11-25T04:26:27.488145+0000","last_clean_scrub_stamp":"2023-11-25T04:26:27.488145+0000","objects_scrubbed":60,"log_size":228,"log_dups_size":0,"ondisk_log_size":228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:22:24.257272+0000","scrub_duration":0.042766632999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":124558,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":167,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":111,"num_bytes_recovered":124533,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.38","version":"75'199","reported_seq":278,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:28.501531+0000","last_change":"2023-11-25T04:26:28.501582+0000","last_active":"2023-11-25T04:26:28.501531+0000","last_peered":"2023-11-25T04:26:28.501531+0000","last_clean":"2023-11-25T04:26:28.501531+0000","last_became_active":"2023-11-25T04:25:49.919236+0000","last_became_peered":"2023-11-25T04:25:49.919236+0000","last_unstale":"2023-11-25T04:26:28.501531+0000","last_undegraded":"2023-11-25T04:26:28.501531+0000","last_fullsized":"2023-11-25T04:26:28.501531+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'199","last_scrub_stamp":"2023-11-25T04:26:28.501441+0000","last_deep_scrub":"75'199","last_deep_scrub_stamp":"2023-11-25T04:26:28.501441+0000","last_clean_scrub_stamp":"2023-11-25T04:26:28.501441+0000","objects_scrubbed":47,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:21:13.937490+0000","scrub_duration":0.030866581000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":120147,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":120121,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.37","version":"75'220","reported_seq":300,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:28.772532+0000","last_change":"2023-11-25T04:26:28.772532+0000","last_active":"2023-11-25T04:26:28.772532+0000","last_peered":"2023-11-25T04:26:28.772532+0000","last_clean":"2023-11-25T04:26:28.772532+0000","last_became_active":"2023-11-25T04:22:47.037328+0000","last_became_peered":"2023-11-25T04:22:47.037328+0000","last_unstale":"2023-11-25T04:26:28.772532+0000","last_undegraded":"2023-11-25T04:26:28.772532+0000","last_fullsized":"2023-11-25T04:26:28.772532+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'220","last_scrub_stamp":"2023-11-25T04:26:28.772426+0000","last_deep_scrub":"75'220","last_deep_scrub_stamp":"2023-11-25T04:26:28.772426+0000","last_clean_scrub_stamp":"2023-11-25T04:26:28.772426+0000","objects_scrubbed":53,"log_size":220,"log_dups_size":0,"ondisk_log_size":220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:08:21.318788+0000","scrub_duration":0.036194551999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":174147,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":184,"num_write_kb":353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.36","version":"75'200","reported_seq":280,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:24.608840+0000","last_change":"2023-11-25T04:26:24.608840+0000","last_active":"2023-11-25T04:26:24.608840+0000","last_peered":"2023-11-25T04:26:24.608840+0000","last_clean":"2023-11-25T04:26:24.608840+0000","last_became_active":"2023-11-25T04:22:47.039892+0000","last_became_peered":"2023-11-25T04:22:47.039892+0000","last_unstale":"2023-11-25T04:26:24.608840+0000","last_undegraded":"2023-11-25T04:26:24.608840+0000","last_fullsized":"2023-11-25T04:26:24.608840+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'200","last_scrub_stamp":"2023-11-25T04:26:24.608737+0000","last_deep_scrub":"75'200","last_deep_scrub_stamp":"2023-11-25T04:26:24.608737+0000","last_clean_scrub_stamp":"2023-11-25T04:26:24.608737+0000","objects_scrubbed":51,"log_size":200,"log_dups_size":0,"ondisk_log_size":200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:49:43.116414+0000","scrub_duration":0.03530929,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":233330,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"75'211","reported_seq":290,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:25.542044+0000","last_change":"2023-11-25T04:26:25.542044+0000","last_active":"2023-11-25T04:26:25.542044+0000","last_peered":"2023-11-25T04:26:25.542044+0000","last_clean":"2023-11-25T04:26:25.542044+0000","last_became_active":"2023-11-25T04:25:51.921800+0000","last_became_peered":"2023-11-25T04:25:51.921800+0000","last_unstale":"2023-11-25T04:26:25.542044+0000","last_undegraded":"2023-11-25T04:26:25.542044+0000","last_fullsized":"2023-11-25T04:26:25.542044+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'211","last_scrub_stamp":"2023-11-25T04:26:25.541938+0000","last_deep_scrub":"75'211","last_deep_scrub_stamp":"2023-11-25T04:26:25.541938+0000","last_clean_scrub_stamp":"2023-11-25T04:26:25.541938+0000","objects_scrubbed":55,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:32:27.981332+0000","scrub_duration":0.043489338000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":302550,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":163,"num_write_kb":431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":300323,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.34","version":"75'186","reported_seq":462,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:12.416198+0000","last_change":"2023-11-25T04:27:12.416198+0000","last_active":"2023-11-25T04:27:12.416198+0000","last_peered":"2023-11-25T04:27:12.416198+0000","last_clean":"2023-11-25T04:27:12.416198+0000","last_became_active":"2023-11-25T04:25:50.922309+0000","last_became_peered":"2023-11-25T04:25:50.922309+0000","last_unstale":"2023-11-25T04:27:12.416198+0000","last_undegraded":"2023-11-25T04:27:12.416198+0000","last_fullsized":"2023-11-25T04:27:12.416198+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'186","last_scrub_stamp":"2023-11-25T04:27:12.416101+0000","last_deep_scrub":"75'186","last_deep_scrub_stamp":"2023-11-25T04:27:12.416101+0000","last_clean_scrub_stamp":"2023-11-25T04:27:12.416101+0000","objects_scrubbed":50,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:42:35.547259+0000","scrub_duration":0.040358303999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":93053,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":131,"num_write_kb":248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":89,"num_bytes_recovered":93053,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"75'183","reported_seq":262,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:19.516218+0000","last_change":"2023-11-25T04:26:19.516218+0000","last_active":"2023-11-25T04:26:19.516218+0000","last_peered":"2023-11-25T04:26:19.516218+0000","last_clean":"2023-11-25T04:26:19.516218+0000","last_became_active":"2023-11-25T04:22:47.040129+0000","last_became_peered":"2023-11-25T04:22:47.040129+0000","last_unstale":"2023-11-25T04:26:19.516218+0000","last_undegraded":"2023-11-25T04:26:19.516218+0000","last_fullsized":"2023-11-25T04:26:19.516218+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'183","last_scrub_stamp":"2023-11-25T04:26:19.516065+0000","last_deep_scrub":"75'183","last_deep_scrub_stamp":"2023-11-25T04:26:19.516065+0000","last_clean_scrub_stamp":"2023-11-25T04:26:19.516065+0000","objects_scrubbed":49,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:43:33.021008+0000","scrub_duration":0.038467327000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":149524,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":139,"num_write_kb":252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.32","version":"75'217","reported_seq":293,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:41.540124+0000","last_change":"2023-11-25T04:27:41.540124+0000","last_active":"2023-11-25T04:27:41.540124+0000","last_peered":"2023-11-25T04:27:41.540124+0000","last_clean":"2023-11-25T04:27:41.540124+0000","last_became_active":"2023-11-25T04:25:53.933694+0000","last_became_peered":"2023-11-25T04:25:53.933694+0000","last_unstale":"2023-11-25T04:27:41.540124+0000","last_undegraded":"2023-11-25T04:27:41.540124+0000","last_fullsized":"2023-11-25T04:27:41.540124+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'217","last_scrub_stamp":"2023-11-25T04:27:41.539982+0000","last_deep_scrub":"75'217","last_deep_scrub_stamp":"2023-11-25T04:27:41.539982+0000","last_clean_scrub_stamp":"2023-11-25T04:27:41.539982+0000","objects_scrubbed":43,"log_size":217,"log_dups_size":0,"ondisk_log_size":217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:13:57.919013+0000","scrub_duration":0.031854021000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":95797,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":181,"num_write_kb":547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":103,"num_bytes_recovered":95797,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.31","version":"75'212","reported_seq":293,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:29.518088+0000","last_change":"2023-11-25T04:26:29.518088+0000","last_active":"2023-11-25T04:26:29.518088+0000","last_peered":"2023-11-25T04:26:29.518088+0000","last_clean":"2023-11-25T04:26:29.518088+0000","last_became_active":"2023-11-25T04:25:49.921129+0000","last_became_peered":"2023-11-25T04:25:49.921129+0000","last_unstale":"2023-11-25T04:26:29.518088+0000","last_undegraded":"2023-11-25T04:26:29.518088+0000","last_fullsized":"2023-11-25T04:26:29.518088+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'212","last_scrub_stamp":"2023-11-25T04:26:29.517931+0000","last_deep_scrub":"75'212","last_deep_scrub_stamp":"2023-11-25T04:26:29.517931+0000","last_clean_scrub_stamp":"2023-11-25T04:26:29.517931+0000","objects_scrubbed":56,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:49:54.902849+0000","scrub_duration":0.039475015000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":163190,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":169,"num_write_kb":336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":154576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"75'229","reported_seq":306,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:20.470429+0000","last_change":"2023-11-25T04:26:20.470429+0000","last_active":"2023-11-25T04:26:20.470429+0000","last_peered":"2023-11-25T04:26:20.470429+0000","last_clean":"2023-11-25T04:26:20.470429+0000","last_became_active":"2023-11-25T04:25:53.933395+0000","last_became_peered":"2023-11-25T04:25:53.933395+0000","last_unstale":"2023-11-25T04:26:20.470429+0000","last_undegraded":"2023-11-25T04:26:20.470429+0000","last_fullsized":"2023-11-25T04:26:20.470429+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'229","last_scrub_stamp":"2023-11-25T04:26:20.470325+0000","last_deep_scrub":"75'229","last_deep_scrub_stamp":"2023-11-25T04:26:20.470325+0000","last_clean_scrub_stamp":"2023-11-25T04:26:20.470325+0000","objects_scrubbed":55,"log_size":229,"log_dups_size":0,"ondisk_log_size":229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:55:52.038708+0000","scrub_duration":0.036798694,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1423587,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":182,"num_write_kb":1626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":1414508,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2f","version":"75'203","reported_seq":283,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:18.718065+0000","last_change":"2023-11-25T04:26:18.718065+0000","last_active":"2023-11-25T04:26:18.718065+0000","last_peered":"2023-11-25T04:26:18.718065+0000","last_clean":"2023-11-25T04:26:18.718065+0000","last_became_active":"2023-11-25T04:22:47.039123+0000","last_became_peered":"2023-11-25T04:22:47.039123+0000","last_unstale":"2023-11-25T04:26:18.718065+0000","last_undegraded":"2023-11-25T04:26:18.718065+0000","last_fullsized":"2023-11-25T04:26:18.718065+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'203","last_scrub_stamp":"2023-11-25T04:26:18.717922+0000","last_deep_scrub":"75'203","last_deep_scrub_stamp":"2023-11-25T04:26:18.717922+0000","last_clean_scrub_stamp":"2023-11-25T04:26:18.717922+0000","objects_scrubbed":51,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:33:40.286516+0000","scrub_duration":0.039125398999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":122752,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":143,"num_write_kb":435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2e","version":"75'199","reported_seq":279,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:58.496118+0000","last_change":"2023-11-25T04:26:58.496118+0000","last_active":"2023-11-25T04:26:58.496118+0000","last_peered":"2023-11-25T04:26:58.496118+0000","last_clean":"2023-11-25T04:26:58.496118+0000","last_became_active":"2023-11-25T04:22:47.036411+0000","last_became_peered":"2023-11-25T04:22:47.036411+0000","last_unstale":"2023-11-25T04:26:58.496118+0000","last_undegraded":"2023-11-25T04:26:58.496118+0000","last_fullsized":"2023-11-25T04:26:58.496118+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'199","last_scrub_stamp":"2023-11-25T04:26:58.496005+0000","last_deep_scrub":"75'199","last_deep_scrub_stamp":"2023-11-25T04:26:58.496005+0000","last_clean_scrub_stamp":"2023-11-25T04:26:58.496005+0000","objects_scrubbed":51,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:57:18.007638+0000","scrub_duration":0.041894438999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4770899,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":140,"num_write_kb":4870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2d","version":"75'194","reported_seq":274,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:47.814157+0000","last_change":"2023-11-25T04:26:47.814157+0000","last_active":"2023-11-25T04:26:47.814157+0000","last_peered":"2023-11-25T04:26:47.814157+0000","last_clean":"2023-11-25T04:26:47.814157+0000","last_became_active":"2023-11-25T04:22:47.038477+0000","last_became_peered":"2023-11-25T04:22:47.038477+0000","last_unstale":"2023-11-25T04:26:47.814157+0000","last_undegraded":"2023-11-25T04:26:47.814157+0000","last_fullsized":"2023-11-25T04:26:47.814157+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'194","last_scrub_stamp":"2023-11-25T04:26:47.814018+0000","last_deep_scrub":"75'194","last_deep_scrub_stamp":"2023-11-25T04:26:47.814018+0000","last_clean_scrub_stamp":"2023-11-25T04:26:47.814018+0000","objects_scrubbed":51,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:20:21.601431+0000","scrub_duration":0.038348491999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":115975,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"75'230","reported_seq":308,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:19.428460+0000","last_change":"2023-11-25T04:26:19.428460+0000","last_active":"2023-11-25T04:26:19.428460+0000","last_peered":"2023-11-25T04:26:19.428460+0000","last_clean":"2023-11-25T04:26:19.428460+0000","last_became_active":"2023-11-25T04:25:52.922367+0000","last_became_peered":"2023-11-25T04:25:52.922367+0000","last_unstale":"2023-11-25T04:26:19.428460+0000","last_undegraded":"2023-11-25T04:26:19.428460+0000","last_fullsized":"2023-11-25T04:26:19.428460+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'230","last_scrub_stamp":"2023-11-25T04:26:19.428350+0000","last_deep_scrub":"75'230","last_deep_scrub_stamp":"2023-11-25T04:26:19.428350+0000","last_clean_scrub_stamp":"2023-11-25T04:26:19.428350+0000","objects_scrubbed":63,"log_size":230,"log_dups_size":0,"ondisk_log_size":230,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:10:12.216682+0000","scrub_duration":0.042625061999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":747090,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":211,"num_write_kb":985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":732276,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"75'211","reported_seq":289,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:17.381662+0000","last_change":"2023-11-25T04:26:17.381662+0000","last_active":"2023-11-25T04:26:17.381662+0000","last_peered":"2023-11-25T04:26:17.381662+0000","last_clean":"2023-11-25T04:26:17.381662+0000","last_became_active":"2023-11-25T04:25:52.923378+0000","last_became_peered":"2023-11-25T04:25:52.923378+0000","last_unstale":"2023-11-25T04:26:17.381662+0000","last_undegraded":"2023-11-25T04:26:17.381662+0000","last_fullsized":"2023-11-25T04:26:17.381662+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'211","last_scrub_stamp":"2023-11-25T04:26:17.381560+0000","last_deep_scrub":"75'211","last_deep_scrub_stamp":"2023-11-25T04:26:17.381560+0000","last_clean_scrub_stamp":"2023-11-25T04:26:17.381560+0000","objects_scrubbed":55,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:26:19.664059+0000","scrub_duration":0.036605545000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69580,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":142,"num_write_kb":315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":98,"num_bytes_recovered":69580,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2a","version":"75'222","reported_seq":530,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:14.438320+0000","last_change":"2023-11-25T04:27:14.438320+0000","last_active":"2023-11-25T04:27:14.438320+0000","last_peered":"2023-11-25T04:27:14.438320+0000","last_clean":"2023-11-25T04:27:14.438320+0000","last_became_active":"2023-11-25T04:25:52.924844+0000","last_became_peered":"2023-11-25T04:25:52.924844+0000","last_unstale":"2023-11-25T04:27:14.438320+0000","last_undegraded":"2023-11-25T04:27:14.438320+0000","last_fullsized":"2023-11-25T04:27:14.438320+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'222","last_scrub_stamp":"2023-11-25T04:27:14.438222+0000","last_deep_scrub":"75'222","last_deep_scrub_stamp":"2023-11-25T04:27:14.438222+0000","last_clean_scrub_stamp":"2023-11-25T04:27:14.438222+0000","objects_scrubbed":51,"log_size":222,"log_dups_size":0,"ondisk_log_size":222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:18:44.242226+0000","scrub_duration":0.036903918000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":115997,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":162,"num_write_kb":261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":115997,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.29","version":"75'211","reported_seq":509,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:12.692632+0000","last_change":"2023-11-25T04:26:12.692632+0000","last_active":"2023-11-25T04:26:12.692632+0000","last_peered":"2023-11-25T04:26:12.692632+0000","last_clean":"2023-11-25T04:26:12.692632+0000","last_became_active":"2023-11-25T04:25:49.919886+0000","last_became_peered":"2023-11-25T04:25:49.919886+0000","last_unstale":"2023-11-25T04:26:12.692632+0000","last_undegraded":"2023-11-25T04:26:12.692632+0000","last_fullsized":"2023-11-25T04:26:12.692632+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'211","last_scrub_stamp":"2023-11-25T04:26:12.692467+0000","last_deep_scrub":"75'211","last_deep_scrub_stamp":"2023-11-25T04:26:12.692467+0000","last_clean_scrub_stamp":"2023-11-25T04:26:12.692467+0000","objects_scrubbed":59,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:13:03.899209+0000","scrub_duration":0.043132342999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":212878,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":212878,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"75'218","reported_seq":297,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:00.495508+0000","last_change":"2023-11-25T04:27:00.495508+0000","last_active":"2023-11-25T04:27:00.495508+0000","last_peered":"2023-11-25T04:27:00.495508+0000","last_clean":"2023-11-25T04:27:00.495508+0000","last_became_active":"2023-11-25T04:22:47.036135+0000","last_became_peered":"2023-11-25T04:22:47.036135+0000","last_unstale":"2023-11-25T04:27:00.495508+0000","last_undegraded":"2023-11-25T04:27:00.495508+0000","last_fullsized":"2023-11-25T04:27:00.495508+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'218","last_scrub_stamp":"2023-11-25T04:27:00.495361+0000","last_deep_scrub":"75'218","last_deep_scrub_stamp":"2023-11-25T04:27:00.495361+0000","last_clean_scrub_stamp":"2023-11-25T04:27:00.495361+0000","objects_scrubbed":49,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:17:48.678120+0000","scrub_duration":0.034866902999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":102957,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":172,"num_write_kb":610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"75'217","reported_seq":527,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:18.423856+0000","last_change":"2023-11-25T04:27:18.423894+0000","last_active":"2023-11-25T04:27:18.423856+0000","last_peered":"2023-11-25T04:27:18.423856+0000","last_clean":"2023-11-25T04:27:18.423856+0000","last_became_active":"2023-11-25T04:25:51.922364+0000","last_became_peered":"2023-11-25T04:25:51.922364+0000","last_unstale":"2023-11-25T04:27:18.423856+0000","last_undegraded":"2023-11-25T04:27:18.423856+0000","last_fullsized":"2023-11-25T04:27:18.423856+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'217","last_scrub_stamp":"2023-11-25T04:27:18.423755+0000","last_deep_scrub":"75'217","last_deep_scrub_stamp":"2023-11-25T04:27:18.423755+0000","last_clean_scrub_stamp":"2023-11-25T04:27:18.423755+0000","objects_scrubbed":50,"log_size":217,"log_dups_size":0,"ondisk_log_size":217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T16:12:12.360172+0000","scrub_duration":0.042117390999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":182683,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":171,"num_write_kb":791,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":182683,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"75'248","reported_seq":578,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:50.182582+0000","last_change":"2023-11-25T04:27:50.182627+0000","last_active":"2023-11-25T04:27:50.182582+0000","last_peered":"2023-11-25T04:27:50.182582+0000","last_clean":"2023-11-25T04:27:50.182582+0000","last_became_active":"2023-11-25T04:25:51.922064+0000","last_became_peered":"2023-11-25T04:25:51.922064+0000","last_unstale":"2023-11-25T04:27:50.182582+0000","last_undegraded":"2023-11-25T04:27:50.182582+0000","last_fullsized":"2023-11-25T04:27:50.182582+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'248","last_scrub_stamp":"2023-11-25T04:27:50.182493+0000","last_deep_scrub":"75'248","last_deep_scrub_stamp":"2023-11-25T04:27:50.182493+0000","last_clean_scrub_stamp":"2023-11-25T04:27:50.182493+0000","objects_scrubbed":61,"log_size":248,"log_dups_size":0,"ondisk_log_size":248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:56:51.214164+0000","scrub_duration":0.043147048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":86311,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":171,"num_write_kb":274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":116,"num_bytes_recovered":86311,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.25","version":"75'201","reported_seq":279,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:10.399668+0000","last_change":"2023-11-25T04:26:10.399668+0000","last_active":"2023-11-25T04:26:10.399668+0000","last_peered":"2023-11-25T04:26:10.399668+0000","last_clean":"2023-11-25T04:26:10.399668+0000","last_became_active":"2023-11-25T04:25:52.924795+0000","last_became_peered":"2023-11-25T04:25:52.924795+0000","last_unstale":"2023-11-25T04:26:10.399668+0000","last_undegraded":"2023-11-25T04:26:10.399668+0000","last_fullsized":"2023-11-25T04:26:10.399668+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'201","last_scrub_stamp":"2023-11-25T04:26:10.399561+0000","last_deep_scrub":"75'201","last_deep_scrub_stamp":"2023-11-25T04:26:10.399561+0000","last_clean_scrub_stamp":"2023-11-25T04:26:10.399561+0000","objects_scrubbed":52,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:15:15.077501+0000","scrub_duration":0.038027564,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":128288,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":167,"num_write_kb":267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":120265,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.24","version":"75'195","reported_seq":482,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:43.546743+0000","last_change":"2023-11-25T04:26:43.546743+0000","last_active":"2023-11-25T04:26:43.546743+0000","last_peered":"2023-11-25T04:26:43.546743+0000","last_clean":"2023-11-25T04:26:43.546743+0000","last_became_active":"2023-11-25T04:25:50.922216+0000","last_became_peered":"2023-11-25T04:25:50.922216+0000","last_unstale":"2023-11-25T04:26:43.546743+0000","last_undegraded":"2023-11-25T04:26:43.546743+0000","last_fullsized":"2023-11-25T04:26:43.546743+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'195","last_scrub_stamp":"2023-11-25T04:26:43.546552+0000","last_deep_scrub":"75'195","last_deep_scrub_stamp":"2023-11-25T04:26:43.546552+0000","last_clean_scrub_stamp":"2023-11-25T04:26:43.546552+0000","objects_scrubbed":44,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:16:44.328689+0000","scrub_duration":0.034140957,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":107507,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":148,"num_write_kb":288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":107494,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.23","version":"75'222","reported_seq":302,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:50.770920+0000","last_change":"2023-11-25T04:26:50.770920+0000","last_active":"2023-11-25T04:26:50.770920+0000","last_peered":"2023-11-25T04:26:50.770920+0000","last_clean":"2023-11-25T04:26:50.770920+0000","last_became_active":"2023-11-25T04:22:47.036823+0000","last_became_peered":"2023-11-25T04:22:47.036823+0000","last_unstale":"2023-11-25T04:26:50.770920+0000","last_undegraded":"2023-11-25T04:26:50.770920+0000","last_fullsized":"2023-11-25T04:26:50.770920+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'222","last_scrub_stamp":"2023-11-25T04:26:50.770805+0000","last_deep_scrub":"75'222","last_deep_scrub_stamp":"2023-11-25T04:26:50.770805+0000","last_clean_scrub_stamp":"2023-11-25T04:26:50.770805+0000","objects_scrubbed":67,"log_size":222,"log_dups_size":0,"ondisk_log_size":222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:16:51.552942+0000","scrub_duration":0.048356029000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4304582,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":152,"num_write_kb":4331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.22","version":"75'253","reported_seq":593,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:27.393520+0000","last_change":"2023-11-25T04:27:27.393555+0000","last_active":"2023-11-25T04:27:27.393520+0000","last_peered":"2023-11-25T04:27:27.393520+0000","last_clean":"2023-11-25T04:27:27.393520+0000","last_became_active":"2023-11-25T04:25:54.939584+0000","last_became_peered":"2023-11-25T04:25:54.939584+0000","last_unstale":"2023-11-25T04:27:27.393520+0000","last_undegraded":"2023-11-25T04:27:27.393520+0000","last_fullsized":"2023-11-25T04:27:27.393520+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'253","last_scrub_stamp":"2023-11-25T04:27:27.393439+0000","last_deep_scrub":"75'253","last_deep_scrub_stamp":"2023-11-25T04:27:27.393439+0000","last_clean_scrub_stamp":"2023-11-25T04:27:27.393439+0000","objects_scrubbed":64,"log_size":253,"log_dups_size":0,"ondisk_log_size":253,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:03:46.694422+0000","scrub_duration":0.044005476000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":129145,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":185,"num_write_kb":432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":121,"num_bytes_recovered":129107,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"75'260","reported_seq":340,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:06.501506+0000","last_change":"2023-11-25T04:27:06.501536+0000","last_active":"2023-11-25T04:27:06.501506+0000","last_peered":"2023-11-25T04:27:06.501506+0000","last_clean":"2023-11-25T04:27:06.501506+0000","last_became_active":"2023-11-25T04:22:47.037206+0000","last_became_peered":"2023-11-25T04:22:47.037206+0000","last_unstale":"2023-11-25T04:27:06.501506+0000","last_undegraded":"2023-11-25T04:27:06.501506+0000","last_fullsized":"2023-11-25T04:27:06.501506+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'260","last_scrub_stamp":"2023-11-25T04:27:06.501413+0000","last_deep_scrub":"75'260","last_deep_scrub_stamp":"2023-11-25T04:27:06.501413+0000","last_clean_scrub_stamp":"2023-11-25T04:27:06.501413+0000","objects_scrubbed":59,"log_size":260,"log_dups_size":0,"ondisk_log_size":260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:18:02.436135+0000","scrub_duration":0.044736607999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":215311,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":194,"num_write_kb":381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"75'231","reported_seq":308,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:35.525083+0000","last_change":"2023-11-25T04:27:35.525083+0000","last_active":"2023-11-25T04:27:35.525083+0000","last_peered":"2023-11-25T04:27:35.525083+0000","last_clean":"2023-11-25T04:27:35.525083+0000","last_became_active":"2023-11-25T04:25:53.931937+0000","last_became_peered":"2023-11-25T04:25:53.931937+0000","last_unstale":"2023-11-25T04:27:35.525083+0000","last_undegraded":"2023-11-25T04:27:35.525083+0000","last_fullsized":"2023-11-25T04:27:35.525083+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'231","last_scrub_stamp":"2023-11-25T04:27:35.524975+0000","last_deep_scrub":"75'231","last_deep_scrub_stamp":"2023-11-25T04:27:35.524975+0000","last_clean_scrub_stamp":"2023-11-25T04:27:35.524975+0000","objects_scrubbed":51,"log_size":231,"log_dups_size":0,"ondisk_log_size":231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:30:27.829518+0000","scrub_duration":0.034850857999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104603,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":172,"num_write_kb":267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":104603,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"75'193","reported_seq":273,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:32.532758+0000","last_change":"2023-11-25T04:26:32.532758+0000","last_active":"2023-11-25T04:26:32.532758+0000","last_peered":"2023-11-25T04:26:32.532758+0000","last_clean":"2023-11-25T04:26:32.532758+0000","last_became_active":"2023-11-25T04:22:47.036107+0000","last_became_peered":"2023-11-25T04:22:47.036107+0000","last_unstale":"2023-11-25T04:26:32.532758+0000","last_undegraded":"2023-11-25T04:26:32.532758+0000","last_fullsized":"2023-11-25T04:26:32.532758+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'193","last_scrub_stamp":"2023-11-25T04:26:32.532624+0000","last_deep_scrub":"75'193","last_deep_scrub_stamp":"2023-11-25T04:26:32.532624+0000","last_clean_scrub_stamp":"2023-11-25T04:26:32.532624+0000","objects_scrubbed":56,"log_size":193,"log_dups_size":0,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:09:38.769552+0000","scrub_duration":0.038075093999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":183586,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":127,"num_write_kb":284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"75'117","reported_seq":204,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:31.496307+0000","last_change":"2023-11-25T04:26:31.496307+0000","last_active":"2023-11-25T04:26:31.496307+0000","last_peered":"2023-11-25T04:26:31.496307+0000","last_clean":"2023-11-25T04:26:31.496307+0000","last_became_active":"2023-11-25T04:25:49.920010+0000","last_became_peered":"2023-11-25T04:25:49.920010+0000","last_unstale":"2023-11-25T04:26:31.496307+0000","last_undegraded":"2023-11-25T04:26:31.496307+0000","last_fullsized":"2023-11-25T04:26:31.496307+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'117","last_scrub_stamp":"2023-11-25T04:26:31.496196+0000","last_deep_scrub":"75'117","last_deep_scrub_stamp":"2023-11-25T04:26:31.496196+0000","last_clean_scrub_stamp":"2023-11-25T04:26:31.496196+0000","objects_scrubbed":3,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:26:02.874431+0000","scrub_duration":0.0076030029999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2816216,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":113,"num_write_kb":2816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":2816182,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"75'189","reported_seq":268,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:40.589251+0000","last_change":"2023-11-25T04:26:40.589251+0000","last_active":"2023-11-25T04:26:40.589251+0000","last_peered":"2023-11-25T04:26:40.589251+0000","last_clean":"2023-11-25T04:26:40.589251+0000","last_became_active":"2023-11-25T04:22:47.037158+0000","last_became_peered":"2023-11-25T04:22:47.037158+0000","last_unstale":"2023-11-25T04:26:40.589251+0000","last_undegraded":"2023-11-25T04:26:40.589251+0000","last_fullsized":"2023-11-25T04:26:40.589251+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'189","last_scrub_stamp":"2023-11-25T04:26:40.589106+0000","last_deep_scrub":"75'189","last_deep_scrub_stamp":"2023-11-25T04:26:40.589106+0000","last_clean_scrub_stamp":"2023-11-25T04:26:40.589106+0000","objects_scrubbed":44,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:32:56.267980+0000","scrub_duration":0.032171351000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":205976,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.12","version":"49'2","reported_seq":92,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:30.784752+0000","last_change":"2023-11-25T04:26:30.784752+0000","last_active":"2023-11-25T04:26:30.784752+0000","last_peered":"2023-11-25T04:26:30.784752+0000","last_clean":"2023-11-25T04:26:30.784752+0000","last_became_active":"2023-11-25T04:25:47.000516+0000","last_became_peered":"2023-11-25T04:25:47.000516+0000","last_unstale":"2023-11-25T04:26:30.784752+0000","last_undegraded":"2023-11-25T04:26:30.784752+0000","last_fullsized":"2023-11-25T04:26:30.784752+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'2","last_scrub_stamp":"2023-11-25T04:26:30.784639+0000","last_deep_scrub":"49'2","last_deep_scrub_stamp":"2023-11-25T04:26:30.784639+0000","last_clean_scrub_stamp":"2023-11-25T04:26:30.784639+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:38:15.370559+0000","scrub_duration":0.001803528,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"75'238","reported_seq":564,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:28:00.215092+0000","last_change":"2023-11-25T04:28:00.215152+0000","last_active":"2023-11-25T04:28:00.215092+0000","last_peered":"2023-11-25T04:28:00.215092+0000","last_clean":"2023-11-25T04:28:00.215092+0000","last_became_active":"2023-11-25T04:25:52.923045+0000","last_became_peered":"2023-11-25T04:25:52.923045+0000","last_unstale":"2023-11-25T04:28:00.215092+0000","last_undegraded":"2023-11-25T04:28:00.215092+0000","last_fullsized":"2023-11-25T04:28:00.215092+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'238","last_scrub_stamp":"2023-11-25T04:28:00.215010+0000","last_deep_scrub":"75'238","last_deep_scrub_stamp":"2023-11-25T04:28:00.215010+0000","last_clean_scrub_stamp":"2023-11-25T04:28:00.215010+0000","objects_scrubbed":63,"log_size":238,"log_dups_size":0,"ondisk_log_size":238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:01:03.314863+0000","scrub_duration":0.049439622000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":171823,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":185,"num_write_kb":340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":166387,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":90,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:18.488445+0000","last_change":"2023-11-25T04:26:18.488445+0000","last_active":"2023-11-25T04:26:18.488445+0000","last_peered":"2023-11-25T04:26:18.488445+0000","last_clean":"2023-11-25T04:26:18.488445+0000","last_became_active":"2023-11-25T04:25:46.993870+0000","last_became_peered":"2023-11-25T04:25:46.993870+0000","last_unstale":"2023-11-25T04:26:18.488445+0000","last_undegraded":"2023-11-25T04:26:18.488445+0000","last_fullsized":"2023-11-25T04:26:18.488445+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:26:18.488345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:26:18.488345+0000","last_clean_scrub_stamp":"2023-11-25T04:26:18.488345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:43:21.558596+0000","scrub_duration":0.002108399,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"75'208","reported_seq":503,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:45.248870+0000","last_change":"2023-11-25T04:27:45.248901+0000","last_active":"2023-11-25T04:27:45.248870+0000","last_peered":"2023-11-25T04:27:45.248870+0000","last_clean":"2023-11-25T04:27:45.248870+0000","last_became_active":"2023-11-25T04:25:50.922511+0000","last_became_peered":"2023-11-25T04:25:50.922511+0000","last_unstale":"2023-11-25T04:27:45.248870+0000","last_undegraded":"2023-11-25T04:27:45.248870+0000","last_fullsized":"2023-11-25T04:27:45.248870+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'208","last_scrub_stamp":"2023-11-25T04:27:45.248792+0000","last_deep_scrub":"75'208","last_deep_scrub_stamp":"2023-11-25T04:27:45.248792+0000","last_clean_scrub_stamp":"2023-11-25T04:27:45.248792+0000","objects_scrubbed":56,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:46:01.446310+0000","scrub_duration":0.040754302999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":579704,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":97,"num_bytes_recovered":570044,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.10","version":"75'12","reported_seq":53,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:06.483272+0000","last_change":"2023-11-25T04:27:06.483272+0000","last_active":"2023-11-25T04:27:06.483272+0000","last_peered":"2023-11-25T04:27:06.483272+0000","last_clean":"2023-11-25T04:27:06.483272+0000","last_became_active":"2023-11-25T04:25:46.998778+0000","last_became_peered":"2023-11-25T04:25:46.998778+0000","last_unstale":"2023-11-25T04:27:06.483272+0000","last_undegraded":"2023-11-25T04:27:06.483272+0000","last_fullsized":"2023-11-25T04:27:06.483272+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'12","last_scrub_stamp":"2023-11-25T04:27:06.483113+0000","last_deep_scrub":"75'12","last_deep_scrub_stamp":"2023-11-25T04:27:06.483113+0000","last_clean_scrub_stamp":"2023-11-25T04:27:06.483113+0000","objects_scrubbed":4,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:26:21.828117+0000","scrub_duration":0.0040538359999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"75'3","reported_seq":48,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:36.494358+0000","last_change":"2023-11-25T04:26:36.494358+0000","last_active":"2023-11-25T04:26:36.494358+0000","last_peered":"2023-11-25T04:26:36.494358+0000","last_clean":"2023-11-25T04:26:36.494358+0000","last_became_active":"2023-11-25T04:25:46.997108+0000","last_became_peered":"2023-11-25T04:25:46.997108+0000","last_unstale":"2023-11-25T04:26:36.494358+0000","last_undegraded":"2023-11-25T04:26:36.494358+0000","last_fullsized":"2023-11-25T04:26:36.494358+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'3","last_scrub_stamp":"2023-11-25T04:26:36.494205+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2023-11-25T04:26:36.494205+0000","last_clean_scrub_stamp":"2023-11-25T04:26:36.494205+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:47:03.918120+0000","scrub_duration":0.0026531879999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"75'208","reported_seq":503,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:07.384457+0000","last_change":"2023-11-25T04:27:07.384457+0000","last_active":"2023-11-25T04:27:07.384457+0000","last_peered":"2023-11-25T04:27:07.384457+0000","last_clean":"2023-11-25T04:27:07.384457+0000","last_became_active":"2023-11-25T04:25:50.921772+0000","last_became_peered":"2023-11-25T04:25:50.921772+0000","last_unstale":"2023-11-25T04:27:07.384457+0000","last_undegraded":"2023-11-25T04:27:07.384457+0000","last_fullsized":"2023-11-25T04:27:07.384457+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'208","last_scrub_stamp":"2023-11-25T04:27:07.384323+0000","last_deep_scrub":"75'208","last_deep_scrub_stamp":"2023-11-25T04:27:07.384323+0000","last_clean_scrub_stamp":"2023-11-25T04:27:07.384323+0000","objects_scrubbed":49,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:52:56.354420+0000","scrub_duration":0.034118876999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112103,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":164,"num_write_kb":240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":99,"num_bytes_recovered":103472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"75'3","reported_seq":93,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:36.492740+0000","last_change":"2023-11-25T04:26:36.492740+0000","last_active":"2023-11-25T04:26:36.492740+0000","last_peered":"2023-11-25T04:26:36.492740+0000","last_clean":"2023-11-25T04:26:36.492740+0000","last_became_active":"2023-11-25T04:25:46.994640+0000","last_became_peered":"2023-11-25T04:25:46.994640+0000","last_unstale":"2023-11-25T04:26:36.492740+0000","last_undegraded":"2023-11-25T04:26:36.492740+0000","last_fullsized":"2023-11-25T04:26:36.492740+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'3","last_scrub_stamp":"2023-11-25T04:26:36.492569+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2023-11-25T04:26:36.492569+0000","last_clean_scrub_stamp":"2023-11-25T04:26:36.492569+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:28:25.392135+0000","scrub_duration":0.0033623030000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"75'229","reported_seq":309,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:05.415841+0000","last_change":"2023-11-25T04:27:05.415841+0000","last_active":"2023-11-25T04:27:05.415841+0000","last_peered":"2023-11-25T04:27:05.415841+0000","last_clean":"2023-11-25T04:27:05.415841+0000","last_became_active":"2023-11-25T04:22:47.039296+0000","last_became_peered":"2023-11-25T04:22:47.039296+0000","last_unstale":"2023-11-25T04:27:05.415841+0000","last_undegraded":"2023-11-25T04:27:05.415841+0000","last_fullsized":"2023-11-25T04:27:05.415841+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'229","last_scrub_stamp":"2023-11-25T04:27:05.415739+0000","last_deep_scrub":"75'229","last_deep_scrub_stamp":"2023-11-25T04:27:05.415739+0000","last_clean_scrub_stamp":"2023-11-25T04:27:05.415739+0000","objects_scrubbed":56,"log_size":229,"log_dups_size":0,"ondisk_log_size":229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:46:38.706963+0000","scrub_duration":0.037100175999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154533,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":182,"num_write_kb":299,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"75'2","reported_seq":48,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:33.490414+0000","last_change":"2023-11-25T04:26:33.490414+0000","last_active":"2023-11-25T04:26:33.490414+0000","last_peered":"2023-11-25T04:26:33.490414+0000","last_clean":"2023-11-25T04:26:33.490414+0000","last_became_active":"2023-11-25T04:25:46.999604+0000","last_became_peered":"2023-11-25T04:25:46.999604+0000","last_unstale":"2023-11-25T04:26:33.490414+0000","last_undegraded":"2023-11-25T04:26:33.490414+0000","last_fullsized":"2023-11-25T04:26:33.490414+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'2","last_scrub_stamp":"2023-11-25T04:26:33.490271+0000","last_deep_scrub":"75'2","last_deep_scrub_stamp":"2023-11-25T04:26:33.490271+0000","last_clean_scrub_stamp":"2023-11-25T04:26:33.490271+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:29:08.502565+0000","scrub_duration":0.0030545450000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"75'206","reported_seq":282,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:52.606438+0000","last_change":"2023-11-25T04:27:52.606438+0000","last_active":"2023-11-25T04:27:52.606438+0000","last_peered":"2023-11-25T04:27:52.606438+0000","last_clean":"2023-11-25T04:27:52.606438+0000","last_became_active":"2023-11-25T04:25:54.938236+0000","last_became_peered":"2023-11-25T04:25:54.938236+0000","last_unstale":"2023-11-25T04:27:52.606438+0000","last_undegraded":"2023-11-25T04:27:52.606438+0000","last_fullsized":"2023-11-25T04:27:52.606438+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'206","last_scrub_stamp":"2023-11-25T04:27:52.606329+0000","last_deep_scrub":"75'206","last_deep_scrub_stamp":"2023-11-25T04:27:52.606329+0000","last_clean_scrub_stamp":"2023-11-25T04:27:52.606329+0000","objects_scrubbed":59,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:43:13.103068+0000","scrub_duration":0.037089984999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":87524,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":155,"num_write_kb":235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":87524,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":44,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:41.504892+0000","last_change":"2023-11-25T04:26:41.504892+0000","last_active":"2023-11-25T04:26:41.504892+0000","last_peered":"2023-11-25T04:26:41.504892+0000","last_clean":"2023-11-25T04:26:41.504892+0000","last_became_active":"2023-11-25T04:25:46.995712+0000","last_became_peered":"2023-11-25T04:25:46.995712+0000","last_unstale":"2023-11-25T04:26:41.504892+0000","last_undegraded":"2023-11-25T04:26:41.504892+0000","last_fullsized":"2023-11-25T04:26:41.504892+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:26:41.504775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:26:41.504775+0000","last_clean_scrub_stamp":"2023-11-25T04:26:41.504775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:35:58.006258+0000","scrub_duration":0.001777907,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"75'211","reported_seq":509,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:46.251693+0000","last_change":"2023-11-25T04:27:46.251738+0000","last_active":"2023-11-25T04:27:46.251693+0000","last_peered":"2023-11-25T04:27:46.251693+0000","last_clean":"2023-11-25T04:27:46.251693+0000","last_became_active":"2023-11-25T04:25:52.922549+0000","last_became_peered":"2023-11-25T04:25:52.922549+0000","last_unstale":"2023-11-25T04:27:46.251693+0000","last_undegraded":"2023-11-25T04:27:46.251693+0000","last_fullsized":"2023-11-25T04:27:46.251693+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'211","last_scrub_stamp":"2023-11-25T04:27:46.251589+0000","last_deep_scrub":"75'211","last_deep_scrub_stamp":"2023-11-25T04:27:46.251589+0000","last_clean_scrub_stamp":"2023-11-25T04:27:46.251589+0000","objects_scrubbed":56,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:21:12.066048+0000","scrub_duration":0.042300846000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":266919,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":264258,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"75'13","reported_seq":108,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:05.498699+0000","last_change":"2023-11-25T04:27:05.498699+0000","last_active":"2023-11-25T04:27:05.498699+0000","last_peered":"2023-11-25T04:27:05.498699+0000","last_clean":"2023-11-25T04:27:05.498699+0000","last_became_active":"2023-11-25T04:25:47.000687+0000","last_became_peered":"2023-11-25T04:25:47.000687+0000","last_unstale":"2023-11-25T04:27:05.498699+0000","last_undegraded":"2023-11-25T04:27:05.498699+0000","last_fullsized":"2023-11-25T04:27:05.498699+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'13","last_scrub_stamp":"2023-11-25T04:27:05.498602+0000","last_deep_scrub":"75'13","last_deep_scrub_stamp":"2023-11-25T04:27:05.498602+0000","last_clean_scrub_stamp":"2023-11-25T04:27:05.498602+0000","objects_scrubbed":2,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:44:21.302320+0000","scrub_duration":0.0090048890000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":4101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4194304,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"75'235","reported_seq":315,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:49.411614+0000","last_change":"2023-11-25T04:26:49.411614+0000","last_active":"2023-11-25T04:26:49.411614+0000","last_peered":"2023-11-25T04:26:49.411614+0000","last_clean":"2023-11-25T04:26:49.411614+0000","last_became_active":"2023-11-25T04:25:50.922409+0000","last_became_peered":"2023-11-25T04:25:50.922409+0000","last_unstale":"2023-11-25T04:26:49.411614+0000","last_undegraded":"2023-11-25T04:26:49.411614+0000","last_fullsized":"2023-11-25T04:26:49.411614+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'235","last_scrub_stamp":"2023-11-25T04:26:49.411506+0000","last_deep_scrub":"75'235","last_deep_scrub_stamp":"2023-11-25T04:26:49.411506+0000","last_clean_scrub_stamp":"2023-11-25T04:26:49.411506+0000","objects_scrubbed":62,"log_size":235,"log_dups_size":0,"ondisk_log_size":235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:38:13.278410+0000","scrub_duration":0.044042721,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":553893,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":169,"num_write_kb":770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":553893,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":44,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:05.477987+0000","last_change":"2023-11-25T04:27:05.477987+0000","last_active":"2023-11-25T04:27:05.477987+0000","last_peered":"2023-11-25T04:27:05.477987+0000","last_clean":"2023-11-25T04:27:05.477987+0000","last_became_active":"2023-11-25T04:25:46.997019+0000","last_became_peered":"2023-11-25T04:25:46.997019+0000","last_unstale":"2023-11-25T04:27:05.477987+0000","last_undegraded":"2023-11-25T04:27:05.477987+0000","last_fullsized":"2023-11-25T04:27:05.477987+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:27:05.477862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:27:05.477862+0000","last_clean_scrub_stamp":"2023-11-25T04:27:05.477862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:44:35.653542+0000","scrub_duration":0.0018186669999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"75'224","reported_seq":301,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:39.544646+0000","last_change":"2023-11-25T04:27:39.544646+0000","last_active":"2023-11-25T04:27:39.544646+0000","last_peered":"2023-11-25T04:27:39.544646+0000","last_clean":"2023-11-25T04:27:39.544646+0000","last_became_active":"2023-11-25T04:25:53.933471+0000","last_became_peered":"2023-11-25T04:25:53.933471+0000","last_unstale":"2023-11-25T04:27:39.544646+0000","last_undegraded":"2023-11-25T04:27:39.544646+0000","last_fullsized":"2023-11-25T04:27:39.544646+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'224","last_scrub_stamp":"2023-11-25T04:27:39.544483+0000","last_deep_scrub":"75'224","last_deep_scrub_stamp":"2023-11-25T04:27:39.544483+0000","last_clean_scrub_stamp":"2023-11-25T04:27:39.544483+0000","objects_scrubbed":61,"log_size":224,"log_dups_size":0,"ondisk_log_size":224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:29:28.945105+0000","scrub_duration":0.044510499000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":320867,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":190,"num_write_kb":577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":108,"num_bytes_recovered":311640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":90,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:26.554458+0000","last_change":"2023-11-25T04:26:26.554458+0000","last_active":"2023-11-25T04:26:26.554458+0000","last_peered":"2023-11-25T04:26:26.554458+0000","last_clean":"2023-11-25T04:26:26.554458+0000","last_became_active":"2023-11-25T04:25:46.994101+0000","last_became_peered":"2023-11-25T04:25:46.994101+0000","last_unstale":"2023-11-25T04:26:26.554458+0000","last_undegraded":"2023-11-25T04:26:26.554458+0000","last_fullsized":"2023-11-25T04:26:26.554458+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:26:26.554355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:26:26.554355+0000","last_clean_scrub_stamp":"2023-11-25T04:26:26.554355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:20:20.095629+0000","scrub_duration":0.0016349400000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"75'187","reported_seq":267,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:35.557414+0000","last_change":"2023-11-25T04:26:35.557414+0000","last_active":"2023-11-25T04:26:35.557414+0000","last_peered":"2023-11-25T04:26:35.557414+0000","last_clean":"2023-11-25T04:26:35.557414+0000","last_became_active":"2023-11-25T04:25:49.920971+0000","last_became_peered":"2023-11-25T04:25:49.920971+0000","last_unstale":"2023-11-25T04:26:35.557414+0000","last_undegraded":"2023-11-25T04:26:35.557414+0000","last_fullsized":"2023-11-25T04:26:35.557414+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'187","last_scrub_stamp":"2023-11-25T04:26:35.557305+0000","last_deep_scrub":"75'187","last_deep_scrub_stamp":"2023-11-25T04:26:35.557305+0000","last_clean_scrub_stamp":"2023-11-25T04:26:35.557305+0000","objects_scrubbed":46,"log_size":187,"log_dups_size":0,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:15:17.199719+0000","scrub_duration":0.039856336999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3502371,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":3718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":89,"num_bytes_recovered":3497669,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"49'1","reported_seq":80,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:52.518138+0000","last_change":"2023-11-25T04:26:52.518138+0000","last_active":"2023-11-25T04:26:52.518138+0000","last_peered":"2023-11-25T04:26:52.518138+0000","last_clean":"2023-11-25T04:26:52.518138+0000","last_became_active":"2023-11-25T04:22:46.034158+0000","last_became_peered":"2023-11-25T04:22:46.034158+0000","last_unstale":"2023-11-25T04:26:52.518138+0000","last_undegraded":"2023-11-25T04:26:52.518138+0000","last_fullsized":"2023-11-25T04:26:52.518138+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'1","last_scrub_stamp":"2023-11-25T04:26:52.518023+0000","last_deep_scrub":"49'1","last_deep_scrub_stamp":"2023-11-25T04:26:52.518023+0000","last_clean_scrub_stamp":"2023-11-25T04:26:52.518023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:44:08.321741+0000","scrub_duration":0.001721757,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"75'194","reported_seq":274,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:15.719633+0000","last_change":"2023-11-25T04:26:15.719633+0000","last_active":"2023-11-25T04:26:15.719633+0000","last_peered":"2023-11-25T04:26:15.719633+0000","last_clean":"2023-11-25T04:26:15.719633+0000","last_became_active":"2023-11-25T04:22:47.038081+0000","last_became_peered":"2023-11-25T04:22:47.038081+0000","last_unstale":"2023-11-25T04:26:15.719633+0000","last_undegraded":"2023-11-25T04:26:15.719633+0000","last_fullsized":"2023-11-25T04:26:15.719633+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'194","last_scrub_stamp":"2023-11-25T04:26:15.719490+0000","last_deep_scrub":"75'194","last_deep_scrub_stamp":"2023-11-25T04:26:15.719490+0000","last_clean_scrub_stamp":"2023-11-25T04:26:15.719490+0000","objects_scrubbed":53,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:43:29.224433+0000","scrub_duration":0.037765356,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":174088,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":146,"num_write_kb":385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"75'159","reported_seq":244,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:21.467705+0000","last_change":"2023-11-25T04:27:21.467705+0000","last_active":"2023-11-25T04:27:21.467705+0000","last_peered":"2023-11-25T04:27:21.467705+0000","last_clean":"2023-11-25T04:27:21.467705+0000","last_became_active":"2023-11-25T04:25:46.999721+0000","last_became_peered":"2023-11-25T04:25:46.999721+0000","last_unstale":"2023-11-25T04:27:21.467705+0000","last_undegraded":"2023-11-25T04:27:21.467705+0000","last_fullsized":"2023-11-25T04:27:21.467705+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'159","last_scrub_stamp":"2023-11-25T04:27:21.467591+0000","last_deep_scrub":"75'159","last_deep_scrub_stamp":"2023-11-25T04:27:21.467591+0000","last_clean_scrub_stamp":"2023-11-25T04:27:21.467591+0000","objects_scrubbed":2,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:55:47.253212+0000","scrub_duration":0.005680058,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":44,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:19.454533+0000","last_change":"2023-11-25T04:27:19.454533+0000","last_active":"2023-11-25T04:27:19.454533+0000","last_peered":"2023-11-25T04:27:19.454533+0000","last_clean":"2023-11-25T04:27:19.454533+0000","last_became_active":"2023-11-25T04:25:46.997693+0000","last_became_peered":"2023-11-25T04:25:46.997693+0000","last_unstale":"2023-11-25T04:27:19.454533+0000","last_undegraded":"2023-11-25T04:27:19.454533+0000","last_fullsized":"2023-11-25T04:27:19.454533+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:27:19.454370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:27:19.454370+0000","last_clean_scrub_stamp":"2023-11-25T04:27:19.454370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:43:29.432453+0000","scrub_duration":0.001737955,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"46'1","reported_seq":80,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:25.228633+0000","last_change":"2023-11-25T04:27:25.228654+0000","last_active":"2023-11-25T04:27:25.228633+0000","last_peered":"2023-11-25T04:27:25.228633+0000","last_clean":"2023-11-25T04:27:25.228633+0000","last_became_active":"2023-11-25T04:22:46.035916+0000","last_became_peered":"2023-11-25T04:22:46.035916+0000","last_unstale":"2023-11-25T04:27:25.228633+0000","last_undegraded":"2023-11-25T04:27:25.228633+0000","last_fullsized":"2023-11-25T04:27:25.228633+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"46'1","last_scrub_stamp":"2023-11-25T04:27:25.228531+0000","last_deep_scrub":"46'1","last_deep_scrub_stamp":"2023-11-25T04:27:25.228531+0000","last_clean_scrub_stamp":"2023-11-25T04:27:25.228531+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:47:55.490961+0000","scrub_duration":0.0016495870000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"75'204","reported_seq":283,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:28.483954+0000","last_change":"2023-11-25T04:27:28.483954+0000","last_active":"2023-11-25T04:27:28.483954+0000","last_peered":"2023-11-25T04:27:28.483954+0000","last_clean":"2023-11-25T04:27:28.483954+0000","last_became_active":"2023-11-25T04:25:51.922490+0000","last_became_peered":"2023-11-25T04:25:51.922490+0000","last_unstale":"2023-11-25T04:27:28.483954+0000","last_undegraded":"2023-11-25T04:27:28.483954+0000","last_fullsized":"2023-11-25T04:27:28.483954+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'204","last_scrub_stamp":"2023-11-25T04:27:28.483839+0000","last_deep_scrub":"75'204","last_deep_scrub_stamp":"2023-11-25T04:27:28.483839+0000","last_clean_scrub_stamp":"2023-11-25T04:27:28.483839+0000","objects_scrubbed":58,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:49:24.463794+0000","scrub_duration":0.041785592000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":502154,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":170,"num_write_kb":687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":99,"num_bytes_recovered":502101,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":44,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:30.509621+0000","last_change":"2023-11-25T04:26:30.509621+0000","last_active":"2023-11-25T04:26:30.509621+0000","last_peered":"2023-11-25T04:26:30.509621+0000","last_clean":"2023-11-25T04:26:30.509621+0000","last_became_active":"2023-11-25T04:25:47.000033+0000","last_became_peered":"2023-11-25T04:25:47.000033+0000","last_unstale":"2023-11-25T04:26:30.509621+0000","last_undegraded":"2023-11-25T04:26:30.509621+0000","last_fullsized":"2023-11-25T04:26:30.509621+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:26:30.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:26:30.509522+0000","last_clean_scrub_stamp":"2023-11-25T04:26:30.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:17:09.583541+0000","scrub_duration":0.001771145,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"75'38","reported_seq":167,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:44.819150+0000","last_change":"2023-11-25T04:26:44.819150+0000","last_active":"2023-11-25T04:26:44.819150+0000","last_peered":"2023-11-25T04:26:44.819150+0000","last_clean":"2023-11-25T04:26:44.819150+0000","last_became_active":"2023-11-25T04:25:46.995637+0000","last_became_peered":"2023-11-25T04:25:46.995637+0000","last_unstale":"2023-11-25T04:26:44.819150+0000","last_undegraded":"2023-11-25T04:26:44.819150+0000","last_fullsized":"2023-11-25T04:26:44.819150+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'38","last_scrub_stamp":"2023-11-25T04:26:44.819010+0000","last_deep_scrub":"75'38","last_deep_scrub_stamp":"2023-11-25T04:26:44.819010+0000","last_clean_scrub_stamp":"2023-11-25T04:26:44.819010+0000","objects_scrubbed":1,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:17:29.276433+0000","scrub_duration":0.003812251,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":319160,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":35,"num_read_kb":334,"num_write":35,"num_write_kb":334,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":319160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"75'210","reported_seq":290,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:38.568589+0000","last_change":"2023-11-25T04:26:38.568589+0000","last_active":"2023-11-25T04:26:38.568589+0000","last_peered":"2023-11-25T04:26:38.568589+0000","last_clean":"2023-11-25T04:26:38.568589+0000","last_became_active":"2023-11-25T04:22:47.039197+0000","last_became_peered":"2023-11-25T04:22:47.039197+0000","last_unstale":"2023-11-25T04:26:38.568589+0000","last_undegraded":"2023-11-25T04:26:38.568589+0000","last_fullsized":"2023-11-25T04:26:38.568589+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'210","last_scrub_stamp":"2023-11-25T04:26:38.568488+0000","last_deep_scrub":"75'210","last_deep_scrub_stamp":"2023-11-25T04:26:38.568488+0000","last_clean_scrub_stamp":"2023-11-25T04:26:38.568488+0000","objects_scrubbed":52,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:17:23.025912+0000","scrub_duration":0.034345986000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":169303,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":189,"num_write_kb":333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":82,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:12.445153+0000","last_change":"2023-11-25T04:27:12.445171+0000","last_active":"2023-11-25T04:27:12.445153+0000","last_peered":"2023-11-25T04:27:12.445153+0000","last_clean":"2023-11-25T04:27:12.445153+0000","last_became_active":"2023-11-25T04:22:42.000041+0000","last_became_peered":"2023-11-25T04:22:42.000041+0000","last_unstale":"2023-11-25T04:27:12.445153+0000","last_undegraded":"2023-11-25T04:27:12.445153+0000","last_fullsized":"2023-11-25T04:27:12.445153+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:27:12.445067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:27:12.445067+0000","last_clean_scrub_stamp":"2023-11-25T04:27:12.445067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:18:00.627826+0000","scrub_duration":0.001522886,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"75'72","reported_seq":166,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:15.436284+0000","last_change":"2023-11-25T04:26:15.436284+0000","last_active":"2023-11-25T04:26:15.436284+0000","last_peered":"2023-11-25T04:26:15.436284+0000","last_clean":"2023-11-25T04:26:15.436284+0000","last_became_active":"2023-11-25T04:25:47.001162+0000","last_became_peered":"2023-11-25T04:25:47.001162+0000","last_unstale":"2023-11-25T04:26:15.436284+0000","last_undegraded":"2023-11-25T04:26:15.436284+0000","last_fullsized":"2023-11-25T04:26:15.436284+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'72","last_scrub_stamp":"2023-11-25T04:26:15.436148+0000","last_deep_scrub":"75'72","last_deep_scrub_stamp":"2023-11-25T04:26:15.436148+0000","last_clean_scrub_stamp":"2023-11-25T04:26:15.436148+0000","objects_scrubbed":1,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T07:37:38.643270+0000","scrub_duration":0.01087811,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":69,"num_write_kb":4130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4194304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"75'220","reported_seq":300,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:14.718055+0000","last_change":"2023-11-25T04:26:14.718055+0000","last_active":"2023-11-25T04:26:14.718055+0000","last_peered":"2023-11-25T04:26:14.718055+0000","last_clean":"2023-11-25T04:26:14.718055+0000","last_became_active":"2023-11-25T04:22:47.038046+0000","last_became_peered":"2023-11-25T04:22:47.038046+0000","last_unstale":"2023-11-25T04:26:14.718055+0000","last_undegraded":"2023-11-25T04:26:14.718055+0000","last_fullsized":"2023-11-25T04:26:14.718055+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'220","last_scrub_stamp":"2023-11-25T04:26:14.717897+0000","last_deep_scrub":"75'220","last_deep_scrub_stamp":"2023-11-25T04:26:14.717897+0000","last_clean_scrub_stamp":"2023-11-25T04:26:14.717897+0000","objects_scrubbed":62,"log_size":220,"log_dups_size":0,"ondisk_log_size":220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:43:17.788148+0000","scrub_duration":0.044136959000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":115501,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":163,"num_write_kb":243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":44,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:04.449359+0000","last_change":"2023-11-25T04:27:04.449359+0000","last_active":"2023-11-25T04:27:04.449359+0000","last_peered":"2023-11-25T04:27:04.449359+0000","last_clean":"2023-11-25T04:27:04.449359+0000","last_became_active":"2023-11-25T04:25:46.997391+0000","last_became_peered":"2023-11-25T04:25:46.997391+0000","last_unstale":"2023-11-25T04:27:04.449359+0000","last_undegraded":"2023-11-25T04:27:04.449359+0000","last_fullsized":"2023-11-25T04:27:04.449359+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:27:04.449261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:27:04.449261+0000","last_clean_scrub_stamp":"2023-11-25T04:27:04.449261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:55:52.982507+0000","scrub_duration":0.00176385,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"50'1","reported_seq":44,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:22.429663+0000","last_change":"2023-11-25T04:27:22.429663+0000","last_active":"2023-11-25T04:27:22.429663+0000","last_peered":"2023-11-25T04:27:22.429663+0000","last_clean":"2023-11-25T04:27:22.429663+0000","last_became_active":"2023-11-25T04:25:46.998132+0000","last_became_peered":"2023-11-25T04:25:46.998132+0000","last_unstale":"2023-11-25T04:27:22.429663+0000","last_undegraded":"2023-11-25T04:27:22.429663+0000","last_fullsized":"2023-11-25T04:27:22.429663+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'1","last_scrub_stamp":"2023-11-25T04:27:22.429551+0000","last_deep_scrub":"50'1","last_deep_scrub_stamp":"2023-11-25T04:27:22.429551+0000","last_clean_scrub_stamp":"2023-11-25T04:27:22.429551+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:41:27.142323+0000","scrub_duration":0.0015038580000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"75'206","reported_seq":285,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:19.694459+0000","last_change":"2023-11-25T04:26:19.694459+0000","last_active":"2023-11-25T04:26:19.694459+0000","last_peered":"2023-11-25T04:26:19.694459+0000","last_clean":"2023-11-25T04:26:19.694459+0000","last_became_active":"2023-11-25T04:22:47.036924+0000","last_became_peered":"2023-11-25T04:22:47.036924+0000","last_unstale":"2023-11-25T04:26:19.694459+0000","last_undegraded":"2023-11-25T04:26:19.694459+0000","last_fullsized":"2023-11-25T04:26:19.694459+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'206","last_scrub_stamp":"2023-11-25T04:26:19.694341+0000","last_deep_scrub":"75'206","last_deep_scrub_stamp":"2023-11-25T04:26:19.694341+0000","last_clean_scrub_stamp":"2023-11-25T04:26:19.694341+0000","objects_scrubbed":49,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:49:38.202017+0000","scrub_duration":0.031428719000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":93321,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":148,"num_write_kb":360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":44,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:26.443241+0000","last_change":"2023-11-25T04:27:26.443241+0000","last_active":"2023-11-25T04:27:26.443241+0000","last_peered":"2023-11-25T04:27:26.443241+0000","last_clean":"2023-11-25T04:27:26.443241+0000","last_became_active":"2023-11-25T04:25:46.999403+0000","last_became_peered":"2023-11-25T04:25:46.999403+0000","last_unstale":"2023-11-25T04:27:26.443241+0000","last_undegraded":"2023-11-25T04:27:26.443241+0000","last_fullsized":"2023-11-25T04:27:26.443241+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:27:26.443086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:27:26.443086+0000","last_clean_scrub_stamp":"2023-11-25T04:27:26.443086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:44:08.788259+0000","scrub_duration":0.001551729,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":82,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:23.259065+0000","last_change":"2023-11-25T04:27:23.259088+0000","last_active":"2023-11-25T04:27:23.259065+0000","last_peered":"2023-11-25T04:27:23.259065+0000","last_clean":"2023-11-25T04:27:23.259065+0000","last_became_active":"2023-11-25T04:22:41.999915+0000","last_became_peered":"2023-11-25T04:22:41.999915+0000","last_unstale":"2023-11-25T04:27:23.259065+0000","last_undegraded":"2023-11-25T04:27:23.259065+0000","last_fullsized":"2023-11-25T04:27:23.259065+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:27:23.258941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:27:23.258941+0000","last_clean_scrub_stamp":"2023-11-25T04:27:23.258941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:46:56.550165+0000","scrub_duration":0.0020338119999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"75'203","reported_seq":495,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:21.405576+0000","last_change":"2023-11-25T04:27:21.405602+0000","last_active":"2023-11-25T04:27:21.405576+0000","last_peered":"2023-11-25T04:27:21.405576+0000","last_clean":"2023-11-25T04:27:21.405576+0000","last_became_active":"2023-11-25T04:25:53.932586+0000","last_became_peered":"2023-11-25T04:25:53.932586+0000","last_unstale":"2023-11-25T04:27:21.405576+0000","last_undegraded":"2023-11-25T04:27:21.405576+0000","last_fullsized":"2023-11-25T04:27:21.405576+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'203","last_scrub_stamp":"2023-11-25T04:27:21.405481+0000","last_deep_scrub":"75'203","last_deep_scrub_stamp":"2023-11-25T04:27:21.405481+0000","last_clean_scrub_stamp":"2023-11-25T04:27:21.405481+0000","objects_scrubbed":51,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:22:14.998841+0000","scrub_duration":0.038507947000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":329327,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":151,"num_write_kb":471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":97,"num_bytes_recovered":329278,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":79,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:28.272266+0000","last_change":"2023-11-25T04:27:28.272286+0000","last_active":"2023-11-25T04:27:28.272266+0000","last_peered":"2023-11-25T04:27:28.272266+0000","last_clean":"2023-11-25T04:27:28.272266+0000","last_became_active":"2023-11-25T04:22:46.035405+0000","last_became_peered":"2023-11-25T04:22:46.035405+0000","last_unstale":"2023-11-25T04:27:28.272266+0000","last_undegraded":"2023-11-25T04:27:28.272266+0000","last_fullsized":"2023-11-25T04:27:28.272266+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:27:28.272187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:27:28.272187+0000","last_clean_scrub_stamp":"2023-11-25T04:27:28.272187+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:42:51.403345+0000","scrub_duration":0.001573315,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":82,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:49.454240+0000","last_change":"2023-11-25T04:26:49.454240+0000","last_active":"2023-11-25T04:26:49.454240+0000","last_peered":"2023-11-25T04:26:49.454240+0000","last_clean":"2023-11-25T04:26:49.454240+0000","last_became_active":"2023-11-25T04:22:41.998798+0000","last_became_peered":"2023-11-25T04:22:41.998798+0000","last_unstale":"2023-11-25T04:26:49.454240+0000","last_undegraded":"2023-11-25T04:26:49.454240+0000","last_fullsized":"2023-11-25T04:26:49.454240+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:26:49.454102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:26:49.454102+0000","last_clean_scrub_stamp":"2023-11-25T04:26:49.454102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:49:39.814155+0000","scrub_duration":0.0015686599999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"75'221","reported_seq":530,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:13.479673+0000","last_change":"2023-11-25T04:26:13.479673+0000","last_active":"2023-11-25T04:26:13.479673+0000","last_peered":"2023-11-25T04:26:13.479673+0000","last_clean":"2023-11-25T04:26:13.479673+0000","last_became_active":"2023-11-25T04:25:51.921858+0000","last_became_peered":"2023-11-25T04:25:51.921858+0000","last_unstale":"2023-11-25T04:26:13.479673+0000","last_undegraded":"2023-11-25T04:26:13.479673+0000","last_fullsized":"2023-11-25T04:26:13.479673+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'221","last_scrub_stamp":"2023-11-25T04:26:13.479549+0000","last_deep_scrub":"75'221","last_deep_scrub_stamp":"2023-11-25T04:26:13.479549+0000","last_clean_scrub_stamp":"2023-11-25T04:26:13.479549+0000","objects_scrubbed":49,"log_size":221,"log_dups_size":0,"ondisk_log_size":221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:44:56.142187+0000","scrub_duration":0.035520731999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":129965,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":166,"num_write_kb":290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":129965,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"75'3","reported_seq":82,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:42.231561+0000","last_change":"2023-11-25T04:27:42.231598+0000","last_active":"2023-11-25T04:27:42.231561+0000","last_peered":"2023-11-25T04:27:42.231561+0000","last_clean":"2023-11-25T04:27:42.231561+0000","last_became_active":"2023-11-25T04:22:46.034229+0000","last_became_peered":"2023-11-25T04:22:46.034229+0000","last_unstale":"2023-11-25T04:27:42.231561+0000","last_undegraded":"2023-11-25T04:27:42.231561+0000","last_fullsized":"2023-11-25T04:27:42.231561+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'3","last_scrub_stamp":"2023-11-25T04:27:42.231482+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2023-11-25T04:27:42.231482+0000","last_clean_scrub_stamp":"2023-11-25T04:27:42.231482+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T16:12:36.167899+0000","scrub_duration":0.0031395300000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":82,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:10.500127+0000","last_change":"2023-11-25T04:27:10.500147+0000","last_active":"2023-11-25T04:27:10.500127+0000","last_peered":"2023-11-25T04:27:10.500127+0000","last_clean":"2023-11-25T04:27:10.500127+0000","last_became_active":"2023-11-25T04:22:41.999992+0000","last_became_peered":"2023-11-25T04:22:41.999992+0000","last_unstale":"2023-11-25T04:27:10.500127+0000","last_undegraded":"2023-11-25T04:27:10.500127+0000","last_fullsized":"2023-11-25T04:27:10.500127+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:27:10.500045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:27:10.500045+0000","last_clean_scrub_stamp":"2023-11-25T04:27:10.500045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:57:30.011678+0000","scrub_duration":0.0018675779999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"75'208","reported_seq":506,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:54.123484+0000","last_change":"2023-11-25T04:27:54.123532+0000","last_active":"2023-11-25T04:27:54.123484+0000","last_peered":"2023-11-25T04:27:54.123484+0000","last_clean":"2023-11-25T04:27:54.123484+0000","last_became_active":"2023-11-25T04:25:54.937838+0000","last_became_peered":"2023-11-25T04:25:54.937838+0000","last_unstale":"2023-11-25T04:27:54.123484+0000","last_undegraded":"2023-11-25T04:27:54.123484+0000","last_fullsized":"2023-11-25T04:27:54.123484+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'208","last_scrub_stamp":"2023-11-25T04:27:54.123369+0000","last_deep_scrub":"75'208","last_deep_scrub_stamp":"2023-11-25T04:27:54.123369+0000","last_clean_scrub_stamp":"2023-11-25T04:27:54.123369+0000","objects_scrubbed":58,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:45:07.470986+0000","scrub_duration":0.040064571,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":159585,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":154224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"75'15","reported_seq":110,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:43.221454+0000","last_change":"2023-11-25T04:27:43.221493+0000","last_active":"2023-11-25T04:27:43.221454+0000","last_peered":"2023-11-25T04:27:43.221454+0000","last_clean":"2023-11-25T04:27:43.221454+0000","last_became_active":"2023-11-25T04:25:46.997404+0000","last_became_peered":"2023-11-25T04:25:46.997404+0000","last_unstale":"2023-11-25T04:27:43.221454+0000","last_undegraded":"2023-11-25T04:27:43.221454+0000","last_fullsized":"2023-11-25T04:27:43.221454+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'15","last_scrub_stamp":"2023-11-25T04:27:43.221358+0000","last_deep_scrub":"75'15","last_deep_scrub_stamp":"2023-11-25T04:27:43.221358+0000","last_clean_scrub_stamp":"2023-11-25T04:27:43.221358+0000","objects_scrubbed":1,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:22:36.814719+0000","scrub_duration":0.004299148,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":44,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:00.412639+0000","last_change":"2023-11-25T04:27:00.412639+0000","last_active":"2023-11-25T04:27:00.412639+0000","last_peered":"2023-11-25T04:27:00.412639+0000","last_clean":"2023-11-25T04:27:00.412639+0000","last_became_active":"2023-11-25T04:25:46.994834+0000","last_became_peered":"2023-11-25T04:25:46.994834+0000","last_unstale":"2023-11-25T04:27:00.412639+0000","last_undegraded":"2023-11-25T04:27:00.412639+0000","last_fullsized":"2023-11-25T04:27:00.412639+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:27:00.412529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:27:00.412529+0000","last_clean_scrub_stamp":"2023-11-25T04:27:00.412529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:30:28.354258+0000","scrub_duration":0.0016774050000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"75'191","reported_seq":475,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:14.466619+0000","last_change":"2023-11-25T04:26:14.466619+0000","last_active":"2023-11-25T04:26:14.466619+0000","last_peered":"2023-11-25T04:26:14.466619+0000","last_clean":"2023-11-25T04:26:14.466619+0000","last_became_active":"2023-11-25T04:25:49.921465+0000","last_became_peered":"2023-11-25T04:25:49.921465+0000","last_unstale":"2023-11-25T04:26:14.466619+0000","last_undegraded":"2023-11-25T04:26:14.466619+0000","last_fullsized":"2023-11-25T04:26:14.466619+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'191","last_scrub_stamp":"2023-11-25T04:26:14.466515+0000","last_deep_scrub":"75'191","last_deep_scrub_stamp":"2023-11-25T04:26:14.466515+0000","last_clean_scrub_stamp":"2023-11-25T04:26:14.466515+0000","objects_scrubbed":56,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:04:34.752431+0000","scrub_duration":0.037056879000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":99503,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":148,"num_write_kb":250,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":93,"num_bytes_recovered":99503,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"75'218","reported_seq":297,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:59.486489+0000","last_change":"2023-11-25T04:26:59.486489+0000","last_active":"2023-11-25T04:26:59.486489+0000","last_peered":"2023-11-25T04:26:59.486489+0000","last_clean":"2023-11-25T04:26:59.486489+0000","last_became_active":"2023-11-25T04:25:51.922070+0000","last_became_peered":"2023-11-25T04:25:51.922070+0000","last_unstale":"2023-11-25T04:26:59.486489+0000","last_undegraded":"2023-11-25T04:26:59.486489+0000","last_fullsized":"2023-11-25T04:26:59.486489+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'218","last_scrub_stamp":"2023-11-25T04:26:59.486387+0000","last_deep_scrub":"75'218","last_deep_scrub_stamp":"2023-11-25T04:26:59.486387+0000","last_clean_scrub_stamp":"2023-11-25T04:26:59.486387+0000","objects_scrubbed":52,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:32:01.142142+0000","scrub_duration":0.040859969000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":185321,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":172,"num_write_kb":312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":164176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"75'54","reported_seq":67,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:58.474489+0000","last_change":"2023-11-25T04:26:58.474489+0000","last_active":"2023-11-25T04:26:58.474489+0000","last_peered":"2023-11-25T04:26:58.474489+0000","last_clean":"2023-11-25T04:26:58.474489+0000","last_became_active":"2023-11-25T04:25:46.994997+0000","last_became_peered":"2023-11-25T04:25:46.994997+0000","last_unstale":"2023-11-25T04:26:58.474489+0000","last_undegraded":"2023-11-25T04:26:58.474489+0000","last_fullsized":"2023-11-25T04:26:58.474489+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'54","last_scrub_stamp":"2023-11-25T04:26:58.474367+0000","last_deep_scrub":"75'54","last_deep_scrub_stamp":"2023-11-25T04:26:58.474367+0000","last_clean_scrub_stamp":"2023-11-25T04:26:58.474367+0000","objects_scrubbed":4,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:32:12.109149+0000","scrub_duration":0.0152154,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":4119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4190183,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"75'199","reported_seq":278,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:43.858609+0000","last_change":"2023-11-25T04:26:43.858609+0000","last_active":"2023-11-25T04:26:43.858609+0000","last_peered":"2023-11-25T04:26:43.858609+0000","last_clean":"2023-11-25T04:26:43.858609+0000","last_became_active":"2023-11-25T04:22:47.036821+0000","last_became_peered":"2023-11-25T04:22:47.036821+0000","last_unstale":"2023-11-25T04:26:43.858609+0000","last_undegraded":"2023-11-25T04:26:43.858609+0000","last_fullsized":"2023-11-25T04:26:43.858609+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'199","last_scrub_stamp":"2023-11-25T04:26:43.858473+0000","last_deep_scrub":"75'199","last_deep_scrub_stamp":"2023-11-25T04:26:43.858473+0000","last_clean_scrub_stamp":"2023-11-25T04:26:43.858473+0000","objects_scrubbed":46,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:28:32.758038+0000","scrub_duration":0.029966083000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":101117,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":161,"num_write_kb":242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"75'24","reported_seq":119,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:39.535658+0000","last_change":"2023-11-25T04:26:39.535658+0000","last_active":"2023-11-25T04:26:39.535658+0000","last_peered":"2023-11-25T04:26:39.535658+0000","last_clean":"2023-11-25T04:26:39.535658+0000","last_became_active":"2023-11-25T04:25:47.000478+0000","last_became_peered":"2023-11-25T04:25:47.000478+0000","last_unstale":"2023-11-25T04:26:39.535658+0000","last_undegraded":"2023-11-25T04:26:39.535658+0000","last_fullsized":"2023-11-25T04:26:39.535658+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'24","last_scrub_stamp":"2023-11-25T04:26:39.535504+0000","last_deep_scrub":"75'24","last_deep_scrub_stamp":"2023-11-25T04:26:39.535504+0000","last_clean_scrub_stamp":"2023-11-25T04:26:39.535504+0000","objects_scrubbed":2,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:50:41.041415+0000","scrub_duration":0.0040188020000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":90,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":44,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:54.349711+0000","last_change":"2023-11-25T04:26:54.349711+0000","last_active":"2023-11-25T04:26:54.349711+0000","last_peered":"2023-11-25T04:26:54.349711+0000","last_clean":"2023-11-25T04:26:54.349711+0000","last_became_active":"2023-11-25T04:25:46.994525+0000","last_became_peered":"2023-11-25T04:25:46.994525+0000","last_unstale":"2023-11-25T04:26:54.349711+0000","last_undegraded":"2023-11-25T04:26:54.349711+0000","last_fullsized":"2023-11-25T04:26:54.349711+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:26:54.349592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:26:54.349592+0000","last_clean_scrub_stamp":"2023-11-25T04:26:54.349592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T11:34:14.985604+0000","scrub_duration":0.0018546229999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"75'237","reported_seq":557,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:47.242584+0000","last_change":"2023-11-25T04:27:47.242637+0000","last_active":"2023-11-25T04:27:47.242584+0000","last_peered":"2023-11-25T04:27:47.242584+0000","last_clean":"2023-11-25T04:27:47.242584+0000","last_became_active":"2023-11-25T04:25:51.921938+0000","last_became_peered":"2023-11-25T04:25:51.921938+0000","last_unstale":"2023-11-25T04:27:47.242584+0000","last_undegraded":"2023-11-25T04:27:47.242584+0000","last_fullsized":"2023-11-25T04:27:47.242584+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'237","last_scrub_stamp":"2023-11-25T04:27:47.242471+0000","last_deep_scrub":"75'237","last_deep_scrub_stamp":"2023-11-25T04:27:47.242471+0000","last_clean_scrub_stamp":"2023-11-25T04:27:47.242471+0000","objects_scrubbed":60,"log_size":237,"log_dups_size":0,"ondisk_log_size":237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T06:04:06.543454+0000","scrub_duration":0.050113891000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":522494,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":168,"num_write_kb":749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":111,"num_bytes_recovered":522494,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"46'2","reported_seq":92,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:51.691533+0000","last_change":"2023-11-25T04:26:51.691533+0000","last_active":"2023-11-25T04:26:51.691533+0000","last_peered":"2023-11-25T04:26:51.691533+0000","last_clean":"2023-11-25T04:26:51.691533+0000","last_became_active":"2023-11-25T04:25:46.998508+0000","last_became_peered":"2023-11-25T04:25:46.998508+0000","last_unstale":"2023-11-25T04:26:51.691533+0000","last_undegraded":"2023-11-25T04:26:51.691533+0000","last_fullsized":"2023-11-25T04:26:51.691533+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"46'2","last_scrub_stamp":"2023-11-25T04:26:51.691391+0000","last_deep_scrub":"46'2","last_deep_scrub_stamp":"2023-11-25T04:26:51.691391+0000","last_clean_scrub_stamp":"2023-11-25T04:26:51.691391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T04:31:13.002647+0000","scrub_duration":0.0021417300000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.10","version":"75'209","reported_seq":289,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:02.586496+0000","last_change":"2023-11-25T04:27:02.586496+0000","last_active":"2023-11-25T04:27:02.586496+0000","last_peered":"2023-11-25T04:27:02.586496+0000","last_clean":"2023-11-25T04:27:02.586496+0000","last_became_active":"2023-11-25T04:22:47.037149+0000","last_became_peered":"2023-11-25T04:22:47.037149+0000","last_unstale":"2023-11-25T04:27:02.586496+0000","last_undegraded":"2023-11-25T04:27:02.586496+0000","last_fullsized":"2023-11-25T04:27:02.586496+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'209","last_scrub_stamp":"2023-11-25T04:27:02.586379+0000","last_deep_scrub":"75'209","last_deep_scrub_stamp":"2023-11-25T04:27:02.586379+0000","last_clean_scrub_stamp":"2023-11-25T04:27:02.586379+0000","objects_scrubbed":56,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:55:23.713881+0000","scrub_duration":0.037826797000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":218073,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1f","version":"75'214","reported_seq":518,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:16.412889+0000","last_change":"2023-11-25T04:27:16.412922+0000","last_active":"2023-11-25T04:27:16.412889+0000","last_peered":"2023-11-25T04:27:16.412889+0000","last_clean":"2023-11-25T04:27:16.412889+0000","last_became_active":"2023-11-25T04:25:49.921458+0000","last_became_peered":"2023-11-25T04:25:49.921458+0000","last_unstale":"2023-11-25T04:27:16.412889+0000","last_undegraded":"2023-11-25T04:27:16.412889+0000","last_fullsized":"2023-11-25T04:27:16.412889+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'214","last_scrub_stamp":"2023-11-25T04:27:16.412805+0000","last_deep_scrub":"75'214","last_deep_scrub_stamp":"2023-11-25T04:27:16.412805+0000","last_clean_scrub_stamp":"2023-11-25T04:27:16.412805+0000","objects_scrubbed":58,"log_size":214,"log_dups_size":0,"ondisk_log_size":214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:05:28.377468+0000","scrub_duration":0.044986529999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":107712,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":103,"num_bytes_recovered":107703,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"48'1","reported_seq":80,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:27.250447+0000","last_change":"2023-11-25T04:27:27.250447+0000","last_active":"2023-11-25T04:27:27.250447+0000","last_peered":"2023-11-25T04:27:27.250447+0000","last_clean":"2023-11-25T04:27:27.250447+0000","last_became_active":"2023-11-25T04:22:46.035495+0000","last_became_peered":"2023-11-25T04:22:46.035495+0000","last_unstale":"2023-11-25T04:27:27.250447+0000","last_undegraded":"2023-11-25T04:27:27.250447+0000","last_fullsized":"2023-11-25T04:27:27.250447+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"48'1","last_scrub_stamp":"2023-11-25T04:27:27.250306+0000","last_deep_scrub":"48'1","last_deep_scrub_stamp":"2023-11-25T04:27:27.250306+0000","last_clean_scrub_stamp":"2023-11-25T04:27:27.250306+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:52:37.111984+0000","scrub_duration":0.001682764,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"75'200","reported_seq":489,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:48.198585+0000","last_change":"2023-11-25T04:27:48.198635+0000","last_active":"2023-11-25T04:27:48.198585+0000","last_peered":"2023-11-25T04:27:48.198585+0000","last_clean":"2023-11-25T04:27:48.198585+0000","last_became_active":"2023-11-25T04:25:50.923375+0000","last_became_peered":"2023-11-25T04:25:50.923375+0000","last_unstale":"2023-11-25T04:27:48.198585+0000","last_undegraded":"2023-11-25T04:27:48.198585+0000","last_fullsized":"2023-11-25T04:27:48.198585+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'200","last_scrub_stamp":"2023-11-25T04:27:48.198510+0000","last_deep_scrub":"75'200","last_deep_scrub_stamp":"2023-11-25T04:27:48.198510+0000","last_clean_scrub_stamp":"2023-11-25T04:27:48.198510+0000","objects_scrubbed":48,"log_size":200,"log_dups_size":0,"ondisk_log_size":200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:12:22.989388+0000","scrub_duration":0.034965662000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":76435,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":76435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":90,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:23.683288+0000","last_change":"2023-11-25T04:26:23.683288+0000","last_active":"2023-11-25T04:26:23.683288+0000","last_peered":"2023-11-25T04:26:23.683288+0000","last_clean":"2023-11-25T04:26:23.683288+0000","last_became_active":"2023-11-25T04:25:46.994871+0000","last_became_peered":"2023-11-25T04:25:46.994871+0000","last_unstale":"2023-11-25T04:26:23.683288+0000","last_undegraded":"2023-11-25T04:26:23.683288+0000","last_fullsized":"2023-11-25T04:26:23.683288+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-25T04:26:23.683176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-25T04:26:23.683176+0000","last_clean_scrub_stamp":"2023-11-25T04:26:23.683176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:31:54.450606+0000","scrub_duration":0.00162587,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"75'207","reported_seq":492,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:23.401113+0000","last_change":"2023-11-25T04:27:23.401113+0000","last_active":"2023-11-25T04:27:23.401113+0000","last_peered":"2023-11-25T04:27:23.401113+0000","last_clean":"2023-11-25T04:27:23.401113+0000","last_became_active":"2023-11-25T04:25:52.925038+0000","last_became_peered":"2023-11-25T04:25:52.925038+0000","last_unstale":"2023-11-25T04:27:23.401113+0000","last_undegraded":"2023-11-25T04:27:23.401113+0000","last_fullsized":"2023-11-25T04:27:23.401113+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'207","last_scrub_stamp":"2023-11-25T04:27:23.400949+0000","last_deep_scrub":"75'207","last_deep_scrub_stamp":"2023-11-25T04:27:23.400949+0000","last_clean_scrub_stamp":"2023-11-25T04:27:23.400949+0000","objects_scrubbed":49,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:20:49.215409+0000","scrub_duration":0.040233620999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":209500,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":139,"num_write_kb":351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":209500,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1a","version":"75'5","reported_seq":84,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:51.508225+0000","last_change":"2023-11-25T04:26:51.508225+0000","last_active":"2023-11-25T04:26:51.508225+0000","last_peered":"2023-11-25T04:26:51.508225+0000","last_clean":"2023-11-25T04:26:51.508225+0000","last_became_active":"2023-11-25T04:22:46.033944+0000","last_became_peered":"2023-11-25T04:22:46.033944+0000","last_unstale":"2023-11-25T04:26:51.508225+0000","last_undegraded":"2023-11-25T04:26:51.508225+0000","last_fullsized":"2023-11-25T04:26:51.508225+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'5","last_scrub_stamp":"2023-11-25T04:26:51.508119+0000","last_deep_scrub":"75'5","last_deep_scrub_stamp":"2023-11-25T04:26:51.508119+0000","last_clean_scrub_stamp":"2023-11-25T04:26:51.508119+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:55:12.635621+0000","scrub_duration":0.0083650619999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"75'218","reported_seq":298,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:28.540082+0000","last_change":"2023-11-25T04:26:28.540082+0000","last_active":"2023-11-25T04:26:28.540082+0000","last_peered":"2023-11-25T04:26:28.540082+0000","last_clean":"2023-11-25T04:26:28.540082+0000","last_became_active":"2023-11-25T04:22:47.039218+0000","last_became_peered":"2023-11-25T04:22:47.039218+0000","last_unstale":"2023-11-25T04:26:28.540082+0000","last_undegraded":"2023-11-25T04:26:28.540082+0000","last_fullsized":"2023-11-25T04:26:28.540082+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'218","last_scrub_stamp":"2023-11-25T04:26:28.539970+0000","last_deep_scrub":"75'218","last_deep_scrub_stamp":"2023-11-25T04:26:28.539970+0000","last_clean_scrub_stamp":"2023-11-25T04:26:28.539970+0000","objects_scrubbed":55,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T08:46:27.913704+0000","scrub_duration":0.040559401000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":165883,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"75'10","reported_seq":105,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:20.499236+0000","last_change":"2023-11-25T04:26:20.499236+0000","last_active":"2023-11-25T04:26:20.499236+0000","last_peered":"2023-11-25T04:26:20.499236+0000","last_clean":"2023-11-25T04:26:20.499236+0000","last_became_active":"2023-11-25T04:25:46.999998+0000","last_became_peered":"2023-11-25T04:25:46.999998+0000","last_unstale":"2023-11-25T04:26:20.499236+0000","last_undegraded":"2023-11-25T04:26:20.499236+0000","last_fullsized":"2023-11-25T04:26:20.499236+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'10","last_scrub_stamp":"2023-11-25T04:26:20.499093+0000","last_deep_scrub":"75'10","last_deep_scrub_stamp":"2023-11-25T04:26:20.499093+0000","last_clean_scrub_stamp":"2023-11-25T04:26:20.499093+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:35:51.569332+0000","scrub_duration":0.0094161829999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":4101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4194304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"75'212","reported_seq":291,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:39.477807+0000","last_change":"2023-11-25T04:26:39.477840+0000","last_active":"2023-11-25T04:26:39.477807+0000","last_peered":"2023-11-25T04:26:39.477807+0000","last_clean":"2023-11-25T04:26:39.477807+0000","last_became_active":"2023-11-25T04:25:50.922164+0000","last_became_peered":"2023-11-25T04:25:50.922164+0000","last_unstale":"2023-11-25T04:26:39.477807+0000","last_undegraded":"2023-11-25T04:26:39.477807+0000","last_fullsized":"2023-11-25T04:26:39.477807+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'212","last_scrub_stamp":"2023-11-25T04:26:39.477723+0000","last_deep_scrub":"75'212","last_deep_scrub_stamp":"2023-11-25T04:26:39.477723+0000","last_clean_scrub_stamp":"2023-11-25T04:26:39.477723+0000","objects_scrubbed":63,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:34:17.609339+0000","scrub_duration":0.044092971000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":179506,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":155,"num_write_kb":317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":179506,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"75'3","reported_seq":47,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:25.424173+0000","last_change":"2023-11-25T04:27:25.424197+0000","last_active":"2023-11-25T04:27:25.424173+0000","last_peered":"2023-11-25T04:27:25.424173+0000","last_clean":"2023-11-25T04:27:25.424173+0000","last_became_active":"2023-11-25T04:25:46.995160+0000","last_became_peered":"2023-11-25T04:25:46.995160+0000","last_unstale":"2023-11-25T04:27:25.424173+0000","last_undegraded":"2023-11-25T04:27:25.424173+0000","last_fullsized":"2023-11-25T04:27:25.424173+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'3","last_scrub_stamp":"2023-11-25T04:27:25.424088+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2023-11-25T04:27:25.424088+0000","last_clean_scrub_stamp":"2023-11-25T04:27:25.424088+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T05:47:26.636974+0000","scrub_duration":0.0032298869999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1a","version":"75'213","reported_seq":288,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:46.569042+0000","last_change":"2023-11-25T04:27:46.569068+0000","last_active":"2023-11-25T04:27:46.569042+0000","last_peered":"2023-11-25T04:27:46.569042+0000","last_clean":"2023-11-25T04:27:46.569042+0000","last_became_active":"2023-11-25T04:25:54.937821+0000","last_became_peered":"2023-11-25T04:25:54.937821+0000","last_unstale":"2023-11-25T04:27:46.569042+0000","last_undegraded":"2023-11-25T04:27:46.569042+0000","last_fullsized":"2023-11-25T04:27:46.569042+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'213","last_scrub_stamp":"2023-11-25T04:27:46.568971+0000","last_deep_scrub":"75'213","last_deep_scrub_stamp":"2023-11-25T04:27:46.568971+0000","last_clean_scrub_stamp":"2023-11-25T04:27:46.568971+0000","objects_scrubbed":51,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T15:40:48.120829+0000","scrub_duration":0.035024894000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":134672,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":169,"num_write_kb":311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":126291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"75'62","reported_seq":159,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:30.554596+0000","last_change":"2023-11-25T04:26:30.554596+0000","last_active":"2023-11-25T04:26:30.554596+0000","last_peered":"2023-11-25T04:26:30.554596+0000","last_clean":"2023-11-25T04:26:30.554596+0000","last_became_active":"2023-11-25T04:25:47.000722+0000","last_became_peered":"2023-11-25T04:25:47.000722+0000","last_unstale":"2023-11-25T04:26:30.554596+0000","last_undegraded":"2023-11-25T04:26:30.554596+0000","last_fullsized":"2023-11-25T04:26:30.554596+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'62","last_scrub_stamp":"2023-11-25T04:26:30.554482+0000","last_deep_scrub":"75'62","last_deep_scrub_stamp":"2023-11-25T04:26:30.554482+0000","last_clean_scrub_stamp":"2023-11-25T04:26:30.554482+0000","objects_scrubbed":3,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T09:38:15.140403+0000","scrub_duration":0.015921703999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8388608,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"75'221","reported_seq":301,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:55.521345+0000","last_change":"2023-11-25T04:26:55.521345+0000","last_active":"2023-11-25T04:26:55.521345+0000","last_peered":"2023-11-25T04:26:55.521345+0000","last_clean":"2023-11-25T04:26:55.521345+0000","last_became_active":"2023-11-25T04:22:47.036406+0000","last_became_peered":"2023-11-25T04:22:47.036406+0000","last_unstale":"2023-11-25T04:26:55.521345+0000","last_undegraded":"2023-11-25T04:26:55.521345+0000","last_fullsized":"2023-11-25T04:26:55.521345+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'221","last_scrub_stamp":"2023-11-25T04:26:55.521198+0000","last_deep_scrub":"75'221","last_deep_scrub_stamp":"2023-11-25T04:26:55.521198+0000","last_clean_scrub_stamp":"2023-11-25T04:26:55.521198+0000","objects_scrubbed":58,"log_size":221,"log_dups_size":0,"ondisk_log_size":221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T12:17:51.455919+0000","scrub_duration":0.041323743000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":527891,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":163,"num_write_kb":687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1e","version":"75'3","reported_seq":82,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:35.296060+0000","last_change":"2023-11-25T04:27:35.296086+0000","last_active":"2023-11-25T04:27:35.296060+0000","last_peered":"2023-11-25T04:27:35.296060+0000","last_clean":"2023-11-25T04:27:35.296060+0000","last_became_active":"2023-11-25T04:22:46.035113+0000","last_became_peered":"2023-11-25T04:22:46.035113+0000","last_unstale":"2023-11-25T04:27:35.296060+0000","last_undegraded":"2023-11-25T04:27:35.296060+0000","last_fullsized":"2023-11-25T04:27:35.296060+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'3","last_scrub_stamp":"2023-11-25T04:27:35.295959+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2023-11-25T04:27:35.295959+0000","last_clean_scrub_stamp":"2023-11-25T04:27:35.295959+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T10:19:05.099964+0000","scrub_duration":0.0087757789999999992,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.18","version":"75'254","reported_seq":330,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:27:51.632613+0000","last_change":"2023-11-25T04:27:51.632613+0000","last_active":"2023-11-25T04:27:51.632613+0000","last_peered":"2023-11-25T04:27:51.632613+0000","last_clean":"2023-11-25T04:27:51.632613+0000","last_became_active":"2023-11-25T04:25:54.937444+0000","last_became_peered":"2023-11-25T04:25:54.937444+0000","last_unstale":"2023-11-25T04:27:51.632613+0000","last_undegraded":"2023-11-25T04:27:51.632613+0000","last_fullsized":"2023-11-25T04:27:51.632613+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'254","last_scrub_stamp":"2023-11-25T04:27:51.632488+0000","last_deep_scrub":"75'254","last_deep_scrub_stamp":"2023-11-25T04:27:51.632488+0000","last_clean_scrub_stamp":"2023-11-25T04:27:51.632488+0000","objects_scrubbed":59,"log_size":254,"log_dups_size":0,"ondisk_log_size":254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T13:59:44.368075+0000","scrub_duration":0.051129967999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":128641,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":185,"num_write_kb":408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":122,"num_bytes_recovered":128641,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"75'20","reported_seq":115,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-25T04:26:49.762359+0000","last_change":"2023-11-25T04:26:49.762359+0000","last_active":"2023-11-25T04:26:49.762359+0000","last_peered":"2023-11-25T04:26:49.762359+0000","last_clean":"2023-11-25T04:26:49.762359+0000","last_became_active":"2023-11-25T04:25:46.998870+0000","last_became_peered":"2023-11-25T04:25:46.998870+0000","last_unstale":"2023-11-25T04:26:49.762359+0000","last_undegraded":"2023-11-25T04:26:49.762359+0000","last_fullsized":"2023-11-25T04:26:49.762359+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'20","last_scrub_stamp":"2023-11-25T04:26:49.762253+0000","last_deep_scrub":"75'20","last_deep_scrub_stamp":"2023-11-25T04:26:49.762253+0000","last_clean_scrub_stamp":"2023-11-25T04:26:49.762253+0000","objects_scrubbed":3,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-26T14:38:21.361584+0000","scrub_duration":0.0095441299999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":4103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4194304,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":26330855,"num_objects":3447,"num_object_clones":0,"num_object_copies":6894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10423,"num_write_kb":38550,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4189,"num_bytes_recovered":12749181,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":72376320,"data_stored":52324046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13693,"ondisk_log_size":13693,"up":128,"acting":128,"num_store_stats":3},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":40885548,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":35,"num_read_kb":334,"num_write":435,"num_write_kb":40462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":32491439,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":81833984,"data_stored":81771096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5548,"internal_metadata":0},"log_size":467,"ondisk_log_size":467,"up":64,"acting":64,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738440,"num_pgs":72,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":99572,"kb_used_data":46188,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94272268,"statfs":{"total":96636764160,"available":96534802432,"internally_reserved":0,"allocated":47296512,"data_stored":39787071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 25 04:27:50 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.45000000000000001,"15min":0.44800000000000001},"min":{"1min":0.33100000000000002,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.56200000000000006,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.45200000000000001,"15min":0.45000000000000001},"min":{"1min":0.314,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.59499999999999997,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.40100000000000002}]},{"osd":1,"last update":"Sat Nov 25 04:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.44400000000000001,"15min":0.44900000000000001},"min":{"1min":0.39400000000000002,"5min":0.216,"15min":0.216},"max":{"1min":0.65200000000000002,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.501,"5min":0.44600000000000001,"15min":0.45400000000000001},"min":{"1min":0.41399999999999998,"5min":0.184,"15min":0.184},"max":{"1min":0.59599999999999997,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.58799999999999997}]}]},{"osd":1,"up_from":8,"seq":34359738440,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":95196,"kb_used_data":55700,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94276644,"statfs":{"total":96636764160,"available":96539283456,"internally_reserved":0,"allocated":57036800,"data_stored":50261554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Nov 25 04:27:46 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.436,"15min":0.438},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.57099999999999995,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.45400000000000001,"15min":0.45600000000000002},"min":{"1min":0.255,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.61899999999999999,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.34499999999999997}]},{"osd":2,"last update":"Sat Nov 25 04:27:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.439,"15min":0.42199999999999999},"min":{"1min":0.379,"5min":0.182,"15min":0.182},"max":{"1min":0.60099999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.439,"15min":0.41799999999999998},"min":{"1min":0.35699999999999998,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.629,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.42999999999999999}]}]},{"osd":0,"up_from":60,"seq":257698037791,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79292,"kb_used_data":51764,"kb_used_omap":6,"kb_used_meta":27513,"kb_avail":94292548,"statfs":{"total":96636764160,"available":96555569152,"internally_reserved":0,"allocated":53006336,"data_stored":46141107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7143,"internal_metadata":28173337},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Nov 25 04:27:52 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.47099999999999997,"15min":0.47499999999999998},"min":{"1min":0.29399999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.60499999999999998,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.438},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.47999999999999998,"15min":0.48199999999999998},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.627,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.33200000000000002}]},{"osd":2,"last update":"Sat Nov 25 04:27:52 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.33600000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.65100000000000002,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.498,"15min":0.5},"min":{"1min":0.36599999999999999,"5min":0.214,"15min":0.214},"max":{"1min":0.64200000000000002,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.51700000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32530432,"data_stored":32496940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5548,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":29380608,"data_stored":29360982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":19922944,"data_stored":19913174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":19234816,"data_stored":12749181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":26415104,"data_stored":20005586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":26726400,"data_stored":19569279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-25T04:28:12.285 DEBUG:teuthology.orchestra.run.smithi067:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-11-25T04:28:12.648 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-25T04:28:12.648 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-25T04:28:12.980 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:28:12.981 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-25T04:22:46.384357+0000","modified":"2023-11-25T04:22:48.396173+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4233},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4233":{"gid":4233,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":3,"addr":"172.21.15.67:6803/2443032509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2443032509},{"type":"v1","addr":"172.21.15.67:6803","nonce":2443032509}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-25T04:28:12.981 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 5 2023-11-25T04:28:12.994 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-25T04:28:13.326 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:28:13.326 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-25T04:22:46.384357+0000","modified":"2023-11-25T04:22:48.396173+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4233},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4233":{"gid":4233,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":3,"addr":"172.21.15.67:6803/2443032509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2443032509},{"type":"v1","addr":"172.21.15.67:6803","nonce":2443032509}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-25T04:28:13.327 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 5 2023-11-25T04:28:13.340 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-25T04:28:13.635 INFO:teuthology.orchestra.run.smithi067.stdout: 2023-11-25T04:28:13.636 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":75,"fsid":"a677f511-1143-4e76-8da6-5ddfadaff077","created":"2023-11-25T04:22:25.140474+0000","modified":"2023-11-25T04:26:00.979788+0000","last_up_change":"2023-11-25T04:25:45.970689+0000","last_in_change":"2023-11-25T04:22:25.985372+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-25T04:22:35.548359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-25T04:22:40.218527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-25T04:22:44.844906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-25T04:22:45.365536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bc397d35-53ea-430c-a96a-63a4160e64c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":20,"up_from":60,"up_thru":68,"down_at":21,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1420426974},{"type":"v1","addr":"172.21.15.76:6801","nonce":1420426974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1420426974},{"type":"v1","addr":"172.21.15.76:6803","nonce":1420426974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1420426974},{"type":"v1","addr":"172.21.15.76:6807","nonce":1420426974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1420426974},{"type":"v1","addr":"172.21.15.76:6805","nonce":1420426974}]},"public_addr":"172.21.15.76:6801/1420426974","cluster_addr":"172.21.15.76:6803/1420426974","heartbeat_back_addr":"172.21.15.76:6807/1420426974","heartbeat_front_addr":"172.21.15.76:6805/1420426974","state":["exists","up"]},{"osd":1,"uuid":"d4f1b88a-bbd4-4e58-9519-569489ced7b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6801","nonce":3130474108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6803","nonce":3130474108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6807","nonce":3130474108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3130474108},{"type":"v1","addr":"172.21.15.92:6805","nonce":3130474108}]},"public_addr":"172.21.15.92:6801/3130474108","cluster_addr":"172.21.15.92:6803/3130474108","heartbeat_back_addr":"172.21.15.92:6807/3130474108","heartbeat_front_addr":"172.21.15.92:6805/3130474108","state":["exists","up"]},{"osd":2,"uuid":"55c82801-e090-4c10-91e5-5fa9dbfba36d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6801","nonce":4099978417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6803","nonce":4099978417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6807","nonce":4099978417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":4099978417},{"type":"v1","addr":"172.21.15.195:6805","nonce":4099978417}]},"public_addr":"172.21.15.195:6801/4099978417","cluster_addr":"172.21.15.195:6803/4099978417","heartbeat_back_addr":"172.21.15.195:6807/4099978417","heartbeat_front_addr":"172.21.15.195:6805/4099978417","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-25T04:22:31.039867+0000","dead_epoch":20},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-25T04:22:31.021863+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-25T04:22:31.009821+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-25T04:28:13.648 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-25T04:28:13.993 INFO:teuthology.orchestra.run.smithi067.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-25T04:28:14.010 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-25T04:28:14.029 INFO:tasks.ceph.mds.a.smithi067.stdout:starting mds.a at 2023-11-25T04:28:15.004 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-25T04:28:16.166 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_metadata' removed 2023-11-25T04:28:16.182 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-25T04:28:17.170 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_data' removed 2023-11-25T04:28:17.187 INFO:teuthology.misc:Shutting down mds daemons... 2023-11-25T04:28:17.187 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-25T04:28:17.187 INFO:teuthology.orchestra.run:waiting for 300 2023-11-25T04:28:17.197 INFO:tasks.ceph.mds.a:Stopped 2023-11-25T04:28:17.197 INFO:teuthology.misc:Shutting down osd daemons... 2023-11-25T04:28:17.198 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-11-25T04:28:17.198 INFO:teuthology.orchestra.run:waiting for 300 2023-11-25T04:28:17.229 INFO:tasks.ceph.osd.0:Stopped 2023-11-25T04:28:17.229 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-11-25T04:28:17.230 INFO:teuthology.orchestra.run:waiting for 300 2023-11-25T04:28:17.260 INFO:tasks.ceph.osd.1:Stopped 2023-11-25T04:28:17.260 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-11-25T04:28:17.260 INFO:teuthology.orchestra.run:waiting for 300 2023-11-25T04:28:17.300 INFO:tasks.ceph.osd.2:Stopped 2023-11-25T04:28:17.301 INFO:teuthology.misc:Shutting down mgr daemons... 2023-11-25T04:28:17.301 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-11-25T04:28:17.301 INFO:teuthology.orchestra.run:waiting for 300 2023-11-25T04:28:17.329 INFO:tasks.ceph.mgr.x:Stopped 2023-11-25T04:28:17.329 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-11-25T04:28:17.329 INFO:teuthology.orchestra.run:waiting for 300 2023-11-25T04:28:17.394 INFO:tasks.ceph.mgr.y:Stopped 2023-11-25T04:28:17.395 INFO:teuthology.misc:Shutting down mon daemons... 2023-11-25T04:28:17.395 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-11-25T04:28:17.395 INFO:teuthology.orchestra.run:waiting for 300 2023-11-25T04:28:17.447 INFO:tasks.ceph.mon.a:Stopped 2023-11-25T04:28:17.447 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-11-25T04:28:17.447 INFO:teuthology.orchestra.run:waiting for 300 2023-11-25T04:28:17.498 INFO:tasks.ceph.mon.b:Stopped 2023-11-25T04:28:17.499 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-11-25T04:28:17.499 INFO:teuthology.orchestra.run:waiting for 300 2023-11-25T04:28:17.550 INFO:tasks.ceph.mon.c:Stopped 2023-11-25T04:28:17.567 INFO:tasks.ceph:Checking cluster log for badness... 2023-11-25T04:28:17.567 DEBUG:teuthology.orchestra.run.smithi067:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(MDS_TRIM\)' | egrep -v '\(MDS_SLOW_REQUEST\)' | egrep -v MDS_SLOW_METADATA_IO | egrep -v 'Behind on trimming' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | head -n 1 2023-11-25T04:28:17.627 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi076.front.sepia.ceph.com 2023-11-25T04:28:17.627 DEBUG:teuthology.orchestra.run.smithi076:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-11-25T04:28:17.801 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi092.front.sepia.ceph.com 2023-11-25T04:28:17.801 DEBUG:teuthology.orchestra.run.smithi092:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-11-25T04:28:17.994 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi195.front.sepia.ceph.com 2023-11-25T04:28:17.994 DEBUG:teuthology.orchestra.run.smithi195:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-11-25T04:28:18.220 INFO:tasks.ceph:Archiving mon data... 2023-11-25T04:28:18.223 DEBUG:teuthology.misc:Transferring archived files from smithi067:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2023-11-24_21:20:59-powercycle-main-distro-default-smithi/7466144/data/mon.a.tgz 2023-11-25T04:28:18.224 DEBUG:teuthology.orchestra.run.smithi067:> mktemp 2023-11-25T04:28:18.242 INFO:teuthology.orchestra.run.smithi067.stdout:/tmp/tmp.SpFQSkJW2v 2023-11-25T04:28:18.242 DEBUG:teuthology.orchestra.run.smithi067:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.SpFQSkJW2v 2023-11-25T04:28:18.410 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 0666 /tmp/tmp.SpFQSkJW2v 2023-11-25T04:28:18.501 DEBUG:teuthology.orchestra.remote:smithi067:/tmp/tmp.SpFQSkJW2v is 386KB 2023-11-25T04:28:18.601 DEBUG:teuthology.orchestra.run.smithi067:> rm -fr /tmp/tmp.SpFQSkJW2v 2023-11-25T04:28:18.616 DEBUG:teuthology.misc:Transferring archived files from smithi067:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2023-11-24_21:20:59-powercycle-main-distro-default-smithi/7466144/data/mon.b.tgz 2023-11-25T04:28:18.616 DEBUG:teuthology.orchestra.run.smithi067:> mktemp 2023-11-25T04:28:18.670 INFO:teuthology.orchestra.run.smithi067.stdout:/tmp/tmp.lgrQiXF6UB 2023-11-25T04:28:18.671 DEBUG:teuthology.orchestra.run.smithi067:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.lgrQiXF6UB 2023-11-25T04:28:18.847 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 0666 /tmp/tmp.lgrQiXF6UB 2023-11-25T04:28:18.936 DEBUG:teuthology.orchestra.remote:smithi067:/tmp/tmp.lgrQiXF6UB is 531KB 2023-11-25T04:28:19.036 DEBUG:teuthology.orchestra.run.smithi067:> rm -fr /tmp/tmp.lgrQiXF6UB 2023-11-25T04:28:19.051 DEBUG:teuthology.misc:Transferring archived files from smithi067:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2023-11-24_21:20:59-powercycle-main-distro-default-smithi/7466144/data/mon.c.tgz 2023-11-25T04:28:19.051 DEBUG:teuthology.orchestra.run.smithi067:> mktemp 2023-11-25T04:28:19.106 INFO:teuthology.orchestra.run.smithi067.stdout:/tmp/tmp.knZyDTFKbT 2023-11-25T04:28:19.106 DEBUG:teuthology.orchestra.run.smithi067:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.knZyDTFKbT 2023-11-25T04:28:19.282 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 0666 /tmp/tmp.knZyDTFKbT 2023-11-25T04:28:19.372 DEBUG:teuthology.orchestra.remote:smithi067:/tmp/tmp.knZyDTFKbT is 531KB 2023-11-25T04:28:19.469 DEBUG:teuthology.orchestra.run.smithi067:> rm -fr /tmp/tmp.knZyDTFKbT 2023-11-25T04:28:19.485 INFO:tasks.ceph:Cleaning ceph cluster... 2023-11-25T04:28:19.485 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-25T04:28:19.527 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-25T04:28:19.529 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-25T04:28:19.532 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-25T04:28:19.727 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-11-25T04:28:19.727 DEBUG:teuthology.orchestra.run.smithi067:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-25T04:28:19.729 DEBUG:teuthology.orchestra.run.smithi076:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-25T04:28:19.732 DEBUG:teuthology.orchestra.run.smithi092:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-25T04:28:19.734 DEBUG:teuthology.orchestra.run.smithi195:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-25T04:28:19.760 INFO:teuthology.orchestra.run.smithi067.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-25T04:28:19.765 INFO:teuthology.orchestra.run.smithi092.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-25T04:28:19.767 INFO:teuthology.orchestra.run.smithi076.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-25T04:28:19.767 INFO:teuthology.orchestra.run.smithi195.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-25T04:28:19.772 INFO:tasks.ceph:Archiving crash dumps... 2023-11-25T04:28:19.774 DEBUG:teuthology.misc:Transferring archived files from smithi067:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-11-24_21:20:59-powercycle-main-distro-default-smithi/7466144/remote/smithi067/crash 2023-11-25T04:28:19.775 DEBUG:teuthology.orchestra.run.smithi067:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-25T04:28:19.836 DEBUG:teuthology.misc:Transferring archived files from smithi076:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-11-24_21:20:59-powercycle-main-distro-default-smithi/7466144/remote/smithi076/crash 2023-11-25T04:28:19.837 DEBUG:teuthology.orchestra.run.smithi076:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-25T04:28:19.881 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-11-24_21:20:59-powercycle-main-distro-default-smithi/7466144/remote/smithi092/crash 2023-11-25T04:28:19.882 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-25T04:28:19.922 DEBUG:teuthology.misc:Transferring archived files from smithi195:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-11-24_21:20:59-powercycle-main-distro-default-smithi/7466144/remote/smithi195/crash 2023-11-25T04:28:19.923 DEBUG:teuthology.orchestra.run.smithi195:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-25T04:28:19.976 INFO:tasks.ceph:Compressing logs... 2023-11-25T04:28:19.976 DEBUG:teuthology.orchestra.run.smithi067:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-11-25T04:28:19.979 DEBUG:teuthology.orchestra.run.smithi076:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-11-25T04:28:19.981 DEBUG:teuthology.orchestra.run.smithi092:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-11-25T04:28:19.984 DEBUG:teuthology.orchestra.run.smithi195:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-11-25T04:28:20.007 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83060.log 2023-11-25T04:28:20.007 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86681.log 2023-11-25T04:28:20.007 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84503.log 2023-11-25T04:28:20.015 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-11-25T04:28:20.018 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83095.log 2023-11-25T04:28:20.018 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83135.log 2023-11-25T04:28:20.018 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83055.log 2023-11-25T04:28:20.018 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.83095.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.83095.log.gz 2023-11-25T04:28:20.018 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.83135.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-11-25T04:28:20.018 INFO:teuthology.orchestra.run.smithi195.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.83135.log.gz 2023-11-25T04:28:20.019 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.83055.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.83055.log.gz 2023-11-25T04:28:20.028 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.782.log 2023-11-25T04:28:20.029 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-11-25T04:28:20.029 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.782.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.782.log.gz 2023-11-25T04:28:20.092 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.83060.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.83060.log.gz 2023-11-25T04:28:20.092 INFO:teuthology.orchestra.run.smithi067.stderr:gzip/var/log/ceph/ceph-client.admin.86681.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.96984.log 2023-11-25T04:28:20.092 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86681.log.gz 2023-11-25T04:28:20.093 INFO:teuthology.orchestra.run.smithi067.stderr:gzip/var/log/ceph/ceph-client.admin.84503.log: -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-11-25T04:28:20.093 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.84503.log.gz 2023-11-25T04:28:20.093 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.96984.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97913.log 2023-11-25T04:28:20.093 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96984.log.gz 2023-11-25T04:28:20.093 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97157.log 2023-11-25T04:28:20.093 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.97913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86807.log 2023-11-25T04:28:20.093 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97913.log.gz 2023-11-25T04:28:20.093 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.97157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88599.log 2023-11-25T04:28:20.094 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97157.log.gz 2023-11-25T04:28:20.094 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.86807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96589.log 2023-11-25T04:28:20.094 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86807.log.gz 2023-11-25T04:28:20.094 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.88599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84377.log 2023-11-25T04:28:20.094 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.88599.log.gz 2023-11-25T04:28:20.094 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.96589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86243.log 2023-11-25T04:28:20.094 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96589.log.gz 2023-11-25T04:28:20.095 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.84377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84269.log 2023-11-25T04:28:20.095 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.84377.log.gz 2023-11-25T04:28:20.095 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.86243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91305.log 2023-11-25T04:28:20.095 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86243.log.gz 2023-11-25T04:28:20.095 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.84269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83868.log 2023-11-25T04:28:20.095 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.84269.log.gz 2023-11-25T04:28:20.095 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.91305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98291.log 2023-11-25T04:28:20.095 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91305.log.gz 2023-11-25T04:28:20.096 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.83868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87687.log 2023-11-25T04:28:20.096 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.83868.log.gz 2023-11-25T04:28:20.096 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.98291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96675.log 2023-11-25T04:28:20.096 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98291.log.gz 2023-11-25T04:28:20.096 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.87687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86591.log 2023-11-25T04:28:20.096 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87687.log.gz 2023-11-25T04:28:20.096 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.96675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83663.log 2023-11-25T04:28:20.096 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96675.log.gz 2023-11-25T04:28:20.097 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.86591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82769.log 2023-11-25T04:28:20.097 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86591.log.gz 2023-11-25T04:28:20.097 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.83663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98709.log 2023-11-25T04:28:20.097 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.83663.log.gz 2023-11-25T04:28:20.097 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.82769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88917.log 2023-11-25T04:28:20.097 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.82769.log.gz 2023-11-25T04:28:20.097 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.98709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84461.log 2023-11-25T04:28:20.097 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98709.log.gz 2023-11-25T04:28:20.098 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.88917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83316.log 2023-11-25T04:28:20.098 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.88917.log.gz 2023-11-25T04:28:20.098 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.84461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84797.log 2023-11-25T04:28:20.098 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.84461.log.gz 2023-11-25T04:28:20.098 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.83316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83102.log 2023-11-25T04:28:20.098 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.83316.log.gz 2023-11-25T04:28:20.098 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.84797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86856.log 2023-11-25T04:28:20.098 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.84797.log.gz 2023-11-25T04:28:20.099 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.83102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88686.log 2023-11-25T04:28:20.099 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.83102.log.gz 2023-11-25T04:28:20.099 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.86856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98165.log 2023-11-25T04:28:20.099 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86856.log.gz 2023-11-25T04:28:20.099 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.88686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82497.log 2023-11-25T04:28:20.099 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.88686.log.gz 2023-11-25T04:28:20.099 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.98165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83228.log 2023-11-25T04:28:20.099 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98165.log.gz 2023-11-25T04:28:20.100 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.82497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91239.log 2023-11-25T04:28:20.100 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.82497.log.gz 2023-11-25T04:28:20.100 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.85907.log/var/log/ceph/ceph-client.admin.83228.log: 2023-11-25T04:28:20.100 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.83228.log.gz 2023-11-25T04:28:20.100 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.91239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84209.log 2023-11-25T04:28:20.100 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91239.log.gz 2023-11-25T04:28:20.100 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.0.85907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83705.log 2023-11-25T04:28:20.100 INFO:teuthology.orchestra.run.smithi067.stderr: 33.1% -- replaced with /var/log/ceph/ceph-client.0.85907.log.gz 2023-11-25T04:28:20.101 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.84209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88374.log 2023-11-25T04:28:20.101 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.84209.log.gz 2023-11-25T04:28:20.101 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.83705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96422.log 2023-11-25T04:28:20.101 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.83705.log.gz 2023-11-25T04:28:20.101 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.88374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85257.log 2023-11-25T04:28:20.101 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.88374.log.gz 2023-11-25T04:28:20.101 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.96422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86068.log 2023-11-25T04:28:20.101 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96422.log.gz 2023-11-25T04:28:20.102 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.85257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84839.log 2023-11-25T04:28:20.102 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.85257.log.gz 2023-11-25T04:28:20.102 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.86068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97199.log 2023-11-25T04:28:20.102 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86068.log.gz 2023-11-25T04:28:20.102 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.84839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85173.log 2023-11-25T04:28:20.102 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.84839.log.gz 2023-11-25T04:28:20.102 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.97199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87600.log 2023-11-25T04:28:20.102 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97199.log.gz 2023-11-25T04:28:20.102 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.85173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86283.log 2023-11-25T04:28:20.103 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.85173.log.gz 2023-11-25T04:28:20.103 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.87600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93066.log 2023-11-25T04:28:20.103 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87600.log.gz 2023-11-25T04:28:20.103 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.86283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97784.log 2023-11-25T04:28:20.103 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86283.log.gz 2023-11-25T04:28:20.103 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.93066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84229.log 2023-11-25T04:28:20.103 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93066.log.gz 2023-11-25T04:28:20.103 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.97784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85487.log 2023-11-25T04:28:20.104 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97784.log.gz 2023-11-25T04:28:20.104 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.84229.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.84229.log.gz 2023-11-25T04:28:20.104 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89004.log 2023-11-25T04:28:20.104 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.85487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86943.log 2023-11-25T04:28:20.104 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.85487.log.gz 2023-11-25T04:28:20.104 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.89004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98459.log 2023-11-25T04:28:20.104 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.89004.log.gz 2023-11-25T04:28:20.104 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.86943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86765.log 2023-11-25T04:28:20.105 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86943.log.gz 2023-11-25T04:28:20.105 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.98459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96721.log 2023-11-25T04:28:20.105 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98459.log.gz 2023-11-25T04:28:20.105 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.86765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86156.log 2023-11-25T04:28:20.105 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86765.log.gz 2023-11-25T04:28:20.105 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.96721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82713.log 2023-11-25T04:28:20.105 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96721.log.gz 2023-11-25T04:28:20.105 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.86156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95937.log 2023-11-25T04:28:20.106 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86156.log.gz 2023-11-25T04:28:20.106 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.82713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88245.log 2023-11-25T04:28:20.106 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.82713.log.gz 2023-11-25T04:28:20.106 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.95937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83018.log 2023-11-25T04:28:20.106 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95937.log.gz 2023-11-25T04:28:20.106 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.88245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85319.log 2023-11-25T04:28:20.106 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.88245.log.gz 2023-11-25T04:28:20.106 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.83018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97619.log 2023-11-25T04:28:20.107 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.83018.log.gz 2023-11-25T04:28:20.107 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.85319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87075.log 2023-11-25T04:28:20.107 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.85319.log.gz 2023-11-25T04:28:20.107 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.97619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85402.log 2023-11-25T04:28:20.107 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97619.log.gz 2023-11-25T04:28:20.107 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.87075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97493.log 2023-11-25T04:28:20.107 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87075.log.gz 2023-11-25T04:28:20.107 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.85402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87336.log 2023-11-25T04:28:20.108 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.85402.log.gz 2023-11-25T04:28:20.108 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.97493.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.81406.log 2023-11-25T04:28:20.108 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97493.log.gz 2023-11-25T04:28:20.108 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.87336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96131.log 2023-11-25T04:28:20.108 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87336.log.gz 2023-11-25T04:28:20.108 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph.tmp-client.admin.81406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98501.log 2023-11-25T04:28:20.108 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.81406.log.gz 2023-11-25T04:28:20.108 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.96131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97533.log 2023-11-25T04:28:20.108 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96131.log.gz 2023-11-25T04:28:20.109 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.98501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87294.log 2023-11-25T04:28:20.109 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98501.log.gz 2023-11-25T04:28:20.109 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.97533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97073.log 2023-11-25T04:28:20.109 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97533.log.gz 2023-11-25T04:28:20.109 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.87294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93494.log 2023-11-25T04:28:20.109 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87294.log.gz 2023-11-25T04:28:20.109 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.97073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98375.log 2023-11-25T04:28:20.109 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97073.log.gz 2023-11-25T04:28:20.110 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.93494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95938.log 2023-11-25T04:28:20.110 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93494.log.gz 2023-11-25T04:28:20.110 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.98375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97955.log 2023-11-25T04:28:20.110 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98375.log.gz 2023-11-25T04:28:20.110 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.95938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88739.log 2023-11-25T04:28:20.110 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95938.log.gz 2023-11-25T04:28:20.110 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.97955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88200.log 2023-11-25T04:28:20.110 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97955.log.gz 2023-11-25T04:28:20.111 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.88739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84122.log 2023-11-25T04:28:20.111 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.88739.log.gz 2023-11-25T04:28:20.111 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.88200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87555.log 2023-11-25T04:28:20.111 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.88200.log.gz 2023-11-25T04:28:20.111 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.84122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82811.log 2023-11-25T04:28:20.111 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.84122.log.gz 2023-11-25T04:28:20.111 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.87555.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-11-25T04:28:20.111 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87555.log.gz 2023-11-25T04:28:20.112 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.82811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83997.log 2023-11-25T04:28:20.112 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.82811.log.gz 2023-11-25T04:28:20.112 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85049.log 2023-11-25T04:28:20.112 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.83997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89045.log 2023-11-25T04:28:20.112 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.83997.log.gz 2023-11-25T04:28:20.112 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.85049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87862.log 2023-11-25T04:28:20.112 INFO:teuthology.orchestra.run.smithi067.stderr: 49.1% -- replaced with /var/log/ceph/ceph-client.admin.85049.log.gz 2023-11-25T04:28:20.112 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.89045.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.89045.log.gz 2023-11-25T04:28:20.113 INFO:teuthology.orchestra.run.smithi067.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.98877.log 2023-11-25T04:28:20.113 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.87862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88826.log 2023-11-25T04:28:20.113 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87862.log.gz 2023-11-25T04:28:20.113 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.98877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98980.log 2023-11-25T04:28:20.113 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98877.log.gz 2023-11-25T04:28:20.113 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.88826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90571.log 2023-11-25T04:28:20.113 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.88826.log.gz 2023-11-25T04:28:20.113 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.98980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94164.log 2023-11-25T04:28:20.113 INFO:teuthology.orchestra.run.smithi067.stderr: 95.1% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-11-25T04:28:20.114 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98980.log.gz 2023-11-25T04:28:20.114 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.90571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83871.log 2023-11-25T04:28:20.114 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90571.log.gz 2023-11-25T04:28:20.114 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.94164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88506.log 2023-11-25T04:28:20.114 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94164.log.gz 2023-11-25T04:28:20.114 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.83871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86963.log 2023-11-25T04:28:20.114 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.83871.log.gz 2023-11-25T04:28:20.114 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.88506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83495.log 2023-11-25T04:28:20.115 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.88506.log.gz 2023-11-25T04:28:20.115 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.86963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82637.log 2023-11-25T04:28:20.115 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86963.log.gz 2023-11-25T04:28:20.115 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.83495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89503.log 2023-11-25T04:28:20.115 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.83495.log.gz 2023-11-25T04:28:20.115 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.82637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83621.log 2023-11-25T04:28:20.115 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.82637.log.gz 2023-11-25T04:28:20.115 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.89503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98333.log 2023-11-25T04:28:20.116 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.89503.log.gz 2023-11-25T04:28:20.116 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.83621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97031.log 2023-11-25T04:28:20.116 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.83621.log.gz 2023-11-25T04:28:20.116 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.98333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87508.log 2023-11-25T04:28:20.116 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98333.log.gz 2023-11-25T04:28:20.116 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.97031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96402.log 2023-11-25T04:28:20.116 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97031.log.gz 2023-11-25T04:28:20.116 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.87508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87203.log 2023-11-25T04:28:20.116 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87508.log.gz 2023-11-25T04:28:20.117 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.96402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96634.log 2023-11-25T04:28:20.117 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96402.log.gz 2023-11-25T04:28:20.117 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.87203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83996.log 2023-11-25T04:28:20.117 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87203.log.gz 2023-11-25T04:28:20.117 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.96634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88287.log 2023-11-25T04:28:20.117 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96634.log.gz 2023-11-25T04:28:20.117 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.83996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87774.log 2023-11-25T04:28:20.117 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.83996.log.gz 2023-11-25T04:28:20.118 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.88287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82872.log 2023-11-25T04:28:20.118 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.88287.log.gz 2023-11-25T04:28:20.118 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.87774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88154.log 2023-11-25T04:28:20.118 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87774.log.gz 2023-11-25T04:28:20.118 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.82872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97785.log 2023-11-25T04:28:20.118 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.82872.log.gz 2023-11-25T04:28:20.118 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.88154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82265.log 2023-11-25T04:28:20.118 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.88154.log.gz 2023-11-25T04:28:20.119 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.97785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83408.log 2023-11-25T04:28:20.119 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97785.log.gz 2023-11-25T04:28:20.119 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.82265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98585.log 2023-11-25T04:28:20.119 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.82265.log.gz 2023-11-25T04:28:20.119 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.83408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86549.log 2023-11-25T04:28:20.119 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.83408.log.gz 2023-11-25T04:28:20.119 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.98585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83579.log 2023-11-25T04:28:20.119 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98585.log.gz 2023-11-25T04:28:20.120 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.86549.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.86549.log.gz -5 2023-11-25T04:28:20.120 INFO:teuthology.orchestra.run.smithi067.stderr: --verbose -- /var/log/ceph/ceph-client.admin.88459.log 2023-11-25T04:28:20.120 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.83579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88783.log 2023-11-25T04:28:20.120 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.83579.log.gz 2023-11-25T04:28:20.120 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.88459.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.88459.log.gz 2023-11-25T04:28:20.120 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84545.log 2023-11-25T04:28:20.120 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.88783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87030.log 2023-11-25T04:28:20.120 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.88783.log.gz 2023-11-25T04:28:20.121 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.84545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96089.log 2023-11-25T04:28:20.121 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.84545.log.gz 2023-11-25T04:28:20.121 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.87030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98081.log 2023-11-25T04:28:20.121 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87030.log.gz 2023-11-25T04:28:20.121 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.96089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96173.log 2023-11-25T04:28:20.121 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96089.log.gz 2023-11-25T04:28:20.122 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.98081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86503.log 2023-11-25T04:28:20.122 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98081.log.gz 2023-11-25T04:28:20.122 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.96173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97115.log 2023-11-25T04:28:20.122 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96173.log.gz 2023-11-25T04:28:20.122 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.86503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98835.log 2023-11-25T04:28:20.122 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86503.log.gz 2023-11-25T04:28:20.123 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.97115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85401.log 2023-11-25T04:28:20.123 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97115.log.gz 2023-11-25T04:28:20.123 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.98835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98627.log 2023-11-25T04:28:20.123 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98835.log.gz 2023-11-25T04:28:20.123 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.85401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86330.log 2023-11-25T04:28:20.123 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.85401.log.gz 2023-11-25T04:28:20.123 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.98627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85131.log 2023-11-25T04:28:20.123 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98627.log.gz 2023-11-25T04:28:20.124 INFO:teuthology.orchestra.run.smithi067.stderr:gzip/var/log/ceph/ceph-client.admin.86330.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.98123.log 2023-11-25T04:28:20.124 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86330.log.gz 2023-11-25T04:28:20.124 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.85131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88644.log 2023-11-25T04:28:20.124 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.85131.log.gz 2023-11-25T04:28:20.124 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.98123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98543.log 2023-11-25T04:28:20.124 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98123.log.gz 2023-11-25T04:28:20.124 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.88644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87468.log 2023-11-25T04:28:20.124 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.88644.log.gz 2023-11-25T04:28:20.125 INFO:teuthology.orchestra.run.smithi067.stderr:gzip/var/log/ceph/ceph-client.admin.98543.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.99022.log 2023-11-25T04:28:20.125 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98543.log.gz 2023-11-25T04:28:20.125 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.87468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86636.log 2023-11-25T04:28:20.125 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87468.log.gz 2023-11-25T04:28:20.125 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.99022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84755.log 2023-11-25T04:28:20.125 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99022.log.gz 2023-11-25T04:28:20.125 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.86636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84335.log 2023-11-25T04:28:20.125 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86636.log.gz 2023-11-25T04:28:20.125 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.84755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98793.log 2023-11-25T04:28:20.126 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.84755.log.gz 2023-11-25T04:28:20.126 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.84335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97532.log 2023-11-25T04:28:20.126 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.84335.log.gz 2023-11-25T04:28:20.126 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.98793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98669.log 2023-11-25T04:28:20.126 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98793.log.gz 2023-11-25T04:28:20.126 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.97532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84671.log 2023-11-25T04:28:20.126 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97532.log.gz 2023-11-25T04:28:20.126 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.98669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96360.log 2023-11-25T04:28:20.127 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98669.log.gz 2023-11-25T04:28:20.127 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.84671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87728.log 2023-11-25T04:28:20.127 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.84671.log.gz 2023-11-25T04:28:20.127 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.96360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96930.log 2023-11-25T04:28:20.127 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96360.log.gz 2023-11-25T04:28:20.127 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.87728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88332.log 2023-11-25T04:28:20.127 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87728.log.gz 2023-11-25T04:28:20.127 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.96930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83870.log 2023-11-25T04:28:20.129 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96930.log.gz 2023-11-25T04:28:20.129 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.88332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83186.log 2023-11-25T04:28:20.129 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.88332.log.gz 2023-11-25T04:28:20.129 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.83870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86897.log 2023-11-25T04:28:20.129 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.83870.log.gz 2023-11-25T04:28:20.130 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.83186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89388.log 2023-11-25T04:28:20.130 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.83186.log.gz 2023-11-25T04:28:20.130 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.86897.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86897.log.gz 2023-11-25T04:28:20.130 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96273.log 2023-11-25T04:28:20.130 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.89388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83789.log 2023-11-25T04:28:20.130 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.89388.log.gz 2023-11-25T04:28:20.130 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.96273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83747.log 2023-11-25T04:28:20.130 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96273.log.gz 2023-11-25T04:28:20.131 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.83789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82976.log 2023-11-25T04:28:20.131 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.83789.log.gz 2023-11-25T04:28:20.131 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.83747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95876.log 2023-11-25T04:28:20.131 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.83747.log.gz 2023-11-25T04:28:20.131 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.82976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86723.log 2023-11-25T04:28:20.131 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.82976.log.gz 2023-11-25T04:28:20.131 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.95876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98751.log 2023-11-25T04:28:20.132 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95876.log.gz 2023-11-25T04:28:20.132 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.86723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96527.log 2023-11-25T04:28:20.132 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86723.log.gz 2023-11-25T04:28:20.132 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.98751.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.96231.log 2023-11-25T04:28:20.132 INFO:teuthology.orchestra.run.smithi067.stderr: -- replaced with /var/log/ceph/ceph-client.admin.98751.log.gz 2023-11-25T04:28:20.132 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.96527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88068.log 2023-11-25T04:28:20.132 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96527.log.gz 2023-11-25T04:28:20.132 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.96231.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.96231.log.gz /var/log/ceph/ceph-client.admin.98417.log 2023-11-25T04:28:20.133 INFO:teuthology.orchestra.run.smithi067.stderr: 2023-11-25T04:28:20.133 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.88068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98207.log 2023-11-25T04:28:20.133 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.88068.log.gz 2023-11-25T04:28:20.133 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.98417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84629.log 2023-11-25T04:28:20.133 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98417.log.gz 2023-11-25T04:28:20.133 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.98207.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-11-25T04:28:20.133 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98207.log.gz 2023-11-25T04:28:20.133 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.84629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82934.log 2023-11-25T04:28:20.134 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.84629.log.gz 2023-11-25T04:28:20.134 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-11-25T04:28:20.134 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.82934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87117.log 2023-11-25T04:28:20.134 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.82934.log.gz 2023-11-25T04:28:20.134 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97745.log 2023-11-25T04:28:20.134 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.87117.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87117.log.gz 2023-11-25T04:28:20.134 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84083.log 2023-11-25T04:28:20.134 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83451.log 2023-11-25T04:28:20.135 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.97745.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97745.log.gz 2023-11-25T04:28:20.135 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.84083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83272.log 2023-11-25T04:28:20.135 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.84083.log.gz 2023-11-25T04:28:20.135 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.83451.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.83451.log.gz 2023-11-25T04:28:20.135 INFO:teuthology.orchestra.run.smithi067.stderr: /var/log/ceph/ceph-client.admin.89905.log 2023-11-25T04:28:20.135 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85007.log 2023-11-25T04:28:20.135 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.83272.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.83272.log.gz 2023-11-25T04:28:20.136 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.89905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86198.log 2023-11-25T04:28:20.136 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.89905.log.gz 2023-11-25T04:28:20.136 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.85007.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.85007.log.gz -- 2023-11-25T04:28:20.136 INFO:teuthology.orchestra.run.smithi067.stderr: /var/log/ceph/ceph-client.admin.97406.log 2023-11-25T04:28:20.136 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.86198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84881.log 2023-11-25T04:28:20.136 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86198.log.gz 2023-11-25T04:28:20.136 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.97406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86375.log 2023-11-25T04:28:20.137 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97406.log.gz 2023-11-25T04:28:20.137 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.84881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97871.log 2023-11-25T04:28:20.137 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.84881.log.gz 2023-11-25T04:28:20.137 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96888.log 2023-11-25T04:28:20.137 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.86375.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86375.log.gz 2023-11-25T04:28:20.137 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.97871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96313.log 2023-11-25T04:28:20.137 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97871.log.gz 2023-11-25T04:28:20.137 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.96888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87249.log 2023-11-25T04:28:20.138 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96888.log.gz 2023-11-25T04:28:20.138 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87947.log 2023-11-25T04:28:20.138 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.96313.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96313.log.gz 2023-11-25T04:28:20.138 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97405.log 2023-11-25T04:28:20.138 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.87947.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87947.log.gz 2023-11-25T04:28:20.138 INFO:teuthology.orchestra.run.smithi067.stderr:gzip 93.7% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-11-25T04:28:20.138 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.87249.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.90269.log 2023-11-25T04:28:20.139 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87249.log.gz 2023-11-25T04:28:20.139 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.97405.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97405.log.gz 2023-11-25T04:28:20.139 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96485.log 2023-11-25T04:28:20.139 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.90269.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90269.log.gzgzip 2023-11-25T04:28:20.139 INFO:teuthology.orchestra.run.smithi067.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.96046.log 2023-11-25T04:28:20.139 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.96485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84923.log 2023-11-25T04:28:20.139 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96485.log.gz 2023-11-25T04:28:20.139 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.96046.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96046.log.gz 2023-11-25T04:28:20.140 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97658.log 2023-11-25T04:28:20.140 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.84923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83366.log 2023-11-25T04:28:20.140 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.84923.log.gz 2023-11-25T04:28:20.140 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.97658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97283.log 2023-11-25T04:28:20.140 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97658.log.gz 2023-11-25T04:28:20.140 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.83366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83537.log 2023-11-25T04:28:20.140 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.83366.log.gz 2023-11-25T04:28:20.140 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.97283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98039.log 2023-11-25T04:28:20.141 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97283.log.gz 2023-11-25T04:28:20.141 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.83537.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-11-25T04:28:20.141 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.83537.log.gz 2023-11-25T04:28:20.141 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.98039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84713.log 2023-11-25T04:28:20.141 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98039.log.gz 2023-11-25T04:28:20.141 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87642.log 2023-11-25T04:28:20.141 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.84713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87794.log 2023-11-25T04:28:20.142 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.84713.log.gz 2023-11-25T04:28:20.142 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.87642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87381.log 2023-11-25T04:28:20.142 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87642.log.gz 2023-11-25T04:28:20.142 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84587.log 2023-11-25T04:28:20.142 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.87794.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87794.log.gz 2023-11-25T04:28:20.142 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.87381.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87381.log.gz 2023-11-25T04:28:20.142 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97407.log 2023-11-25T04:28:20.142 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87162.log 2023-11-25T04:28:20.143 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.84587.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.84587.log.gz 2023-11-25T04:28:20.143 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.97407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84965.log 2023-11-25T04:28:20.143 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97407.log.gz 2023-11-25T04:28:20.143 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84419.log 2023-11-25T04:28:20.143 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.87162.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87162.log.gz 2023-11-25T04:28:20.143 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.84965.log: 94.5% -- replaced with /var/log/ceph/ceph.log.gz 2023-11-25T04:28:20.143 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85215.log 2023-11-25T04:28:20.143 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.84965.log.gz 2023-11-25T04:28:20.144 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-11-25T04:28:20.144 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.84419.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.84419.log.gz 2023-11-25T04:28:20.144 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.85215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82474.log 2023-11-25T04:28:20.144 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.85215.log.gz 2023-11-25T04:28:20.144 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96766.log 2023-11-25T04:28:20.144 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.82474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93397.log 2023-11-25T04:28:20.144 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.82474.log.gz 2023-11-25T04:28:20.145 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.96766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98249.log 2023-11-25T04:28:20.145 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96766.log.gz 2023-11-25T04:28:20.145 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.93397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95958.log 2023-11-25T04:28:20.145 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93397.log.gz 2023-11-25T04:28:20.145 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86417.log 2023-11-25T04:28:20.145 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.98249.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98249.log.gz 2023-11-25T04:28:20.145 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.95958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97997.log 2023-11-25T04:28:20.145 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95958.log.gz 2023-11-25T04:28:20.146 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5/var/log/ceph/ceph-client.admin.86417.log: --verbose -- /var/log/ceph/ceph-client.admin.88959.log 2023-11-25T04:28:20.146 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86417.log.gz 2023-11-25T04:28:20.146 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.97997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97325.log 2023-11-25T04:28:20.146 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97997.log.gz 2023-11-25T04:28:20.146 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.88959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93605.log 2023-11-25T04:28:20.146 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.88959.log.gz 2023-11-25T04:28:20.146 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.97325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97241.log 2023-11-25T04:28:20.147 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97325.log.gz 2023-11-25T04:28:20.147 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.93605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88113.log 2023-11-25T04:28:20.147 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93605.log.gz 2023-11-25T04:28:20.147 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88419.log 2023-11-25T04:28:20.147 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.97241.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97241.log.gz 2023-11-25T04:28:20.147 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83957.log 2023-11-25T04:28:20.147 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.88113.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.88113.log.gz 2023-11-25T04:28:20.147 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.88419.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-11-25T04:28:20.148 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.88419.log.gz 2023-11-25T04:28:20.148 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.83957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84123.log 2023-11-25T04:28:20.148 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.83957.log.gz 2023-11-25T04:28:20.148 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88551.log 2023-11-25T04:28:20.148 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.84123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83144.log 2023-11-25T04:28:20.148 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.84123.log.gz 2023-11-25T04:28:20.148 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.88551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87423.log 2023-11-25T04:28:20.148 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.88551.log.gz 2023-11-25T04:28:20.149 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.83144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86462.log 2023-11-25T04:28:20.149 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.83144.log.gz 2023-11-25T04:28:20.149 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.87423.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-11-25T04:28:20.149 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87423.log.gz 2023-11-25T04:28:20.149 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.86462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98937.log 2023-11-25T04:28:20.149 INFO:teuthology.orchestra.run.smithi067.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86462.log.gz 2023-11-25T04:28:20.149 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97659.log 2023-11-25T04:28:20.149 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.98937.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98937.log.gz 2023-11-25T04:28:20.150 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/ceph-client.admin.97659.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97659.log.gz 2023-11-25T04:28:20.150 INFO:teuthology.orchestra.run.smithi067.stderr: 92.5% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-11-25T04:28:20.150 INFO:teuthology.orchestra.run.smithi067.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-11-25T04:28:20.192 INFO:teuthology.orchestra.run.smithi067.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-11-25T04:28:20.204 INFO:teuthology.orchestra.run.smithi067.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-11-25T04:28:20.279 INFO:teuthology.orchestra.run.smithi067.stderr: 90.4% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-11-25T04:28:20.281 INFO:teuthology.orchestra.run.smithi067.stderr: 2023-11-25T04:28:20.282 INFO:teuthology.orchestra.run.smithi067.stderr:real 0m0.291s 2023-11-25T04:28:20.282 INFO:teuthology.orchestra.run.smithi067.stderr:user 0m0.704s 2023-11-25T04:28:20.282 INFO:teuthology.orchestra.run.smithi067.stderr:sys 0m0.118s 2023-11-25T04:28:21.045 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-osd.0.log: 95.2% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-11-25T04:28:21.047 INFO:teuthology.orchestra.run.smithi076.stderr: 2023-11-25T04:28:21.048 INFO:teuthology.orchestra.run.smithi076.stderr:real 0m1.053s 2023-11-25T04:28:21.048 INFO:teuthology.orchestra.run.smithi076.stderr:user 0m0.974s 2023-11-25T04:28:21.048 INFO:teuthology.orchestra.run.smithi076.stderr:sys 0m0.047s 2023-11-25T04:28:22.288 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-osd.1.log: 94.5% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-11-25T04:28:22.289 INFO:teuthology.orchestra.run.smithi092.stderr: 2023-11-25T04:28:22.290 INFO:teuthology.orchestra.run.smithi092.stderr:real 0m2.292s 2023-11-25T04:28:22.290 INFO:teuthology.orchestra.run.smithi092.stderr:user 0m2.203s 2023-11-25T04:28:22.290 INFO:teuthology.orchestra.run.smithi092.stderr:sys 0m0.098s 2023-11-25T04:28:22.473 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-osd.2.log: 94.5% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-11-25T04:28:22.475 INFO:teuthology.orchestra.run.smithi195.stderr: 2023-11-25T04:28:22.475 INFO:teuthology.orchestra.run.smithi195.stderr:real 0m2.476s 2023-11-25T04:28:22.475 INFO:teuthology.orchestra.run.smithi195.stderr:user 0m2.385s 2023-11-25T04:28:22.475 INFO:teuthology.orchestra.run.smithi195.stderr:sys 0m0.102s 2023-11-25T04:28:22.476 INFO:tasks.ceph:Archiving logs... 2023-11-25T04:28:22.476 DEBUG:teuthology.misc:Transferring archived files from smithi067:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-11-24_21:20:59-powercycle-main-distro-default-smithi/7466144/remote/smithi067/log 2023-11-25T04:28:22.477 DEBUG:teuthology.orchestra.run.smithi067:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-25T04:28:22.816 DEBUG:teuthology.misc:Transferring archived files from smithi076:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-11-24_21:20:59-powercycle-main-distro-default-smithi/7466144/remote/smithi076/log 2023-11-25T04:28:22.816 DEBUG:teuthology.orchestra.run.smithi076:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-25T04:28:22.982 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-11-24_21:20:59-powercycle-main-distro-default-smithi/7466144/remote/smithi092/log 2023-11-25T04:28:22.983 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-25T04:28:23.334 DEBUG:teuthology.misc:Transferring archived files from smithi195:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-11-24_21:20:59-powercycle-main-distro-default-smithi/7466144/remote/smithi195/log 2023-11-25T04:28:23.335 DEBUG:teuthology.orchestra.run.smithi195:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-25T04:28:23.708 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-11-25T04:28:23.718 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-11-25T04:28:23.718 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-25T04:28:23.721 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-25T04:28:23.724 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-25T04:28:23.726 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-25T04:28:23.852 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-11-25T04:28:23.862 INFO:teuthology.task.clock:Checking final clock skew... 2023-11-25T04:28:23.862 DEBUG:teuthology.orchestra.run.smithi067:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-25T04:28:23.865 DEBUG:teuthology.orchestra.run.smithi076:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-25T04:28:23.867 DEBUG:teuthology.orchestra.run.smithi092:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-25T04:28:23.870 DEBUG:teuthology.orchestra.run.smithi195:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-25T04:28:23.880 INFO:teuthology.orchestra.run.smithi067.stderr:bash: line 1: ntpq: command not found 2023-11-25T04:28:23.881 INFO:teuthology.orchestra.run.smithi076.stderr:bash: line 1: ntpq: command not found 2023-11-25T04:28:23.884 INFO:teuthology.orchestra.run.smithi092.stderr:bash: line 1: ntpq: command not found 2023-11-25T04:28:23.885 INFO:teuthology.orchestra.run.smithi067.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-25T04:28:23.885 INFO:teuthology.orchestra.run.smithi067.stdout:=============================================================================== 2023-11-25T04:28:23.886 INFO:teuthology.orchestra.run.smithi067.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 22 +409us[ +412us] +/- 79ms 2023-11-25T04:28:23.886 INFO:teuthology.orchestra.run.smithi067.stdout:^* hv02.front.sepia.ceph.com 3 6 377 20 -761us[ -758us] +/- 69ms 2023-11-25T04:28:23.886 INFO:teuthology.orchestra.run.smithi067.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 19 +428us[ +428us] +/- 65ms 2023-11-25T04:28:23.886 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-11-25T04:28:23.887 INFO:teuthology.orchestra.run.smithi076.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-25T04:28:23.887 INFO:teuthology.orchestra.run.smithi076.stdout:=============================================================================== 2023-11-25T04:28:23.887 INFO:teuthology.orchestra.run.smithi076.stdout:^+ hv01.front.sepia.ceph.com 2 6 77 38 +407us[ +407us] +/- 79ms 2023-11-25T04:28:23.887 INFO:teuthology.orchestra.run.smithi076.stdout:^+ hv02.front.sepia.ceph.com 3 6 77 39 -790us[ -830us] +/- 69ms 2023-11-25T04:28:23.887 INFO:teuthology.orchestra.run.smithi076.stdout:^* hv03.front.sepia.ceph.com 3 6 77 38 +422us[ +383us] +/- 65ms 2023-11-25T04:28:23.888 INFO:teuthology.orchestra.run.smithi076.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-11-25T04:28:23.888 INFO:teuthology.orchestra.run.smithi195.stderr:bash: line 1: ntpq: command not found 2023-11-25T04:28:23.888 INFO:teuthology.orchestra.run.smithi092.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-25T04:28:23.889 INFO:teuthology.orchestra.run.smithi092.stdout:=============================================================================== 2023-11-25T04:28:23.889 INFO:teuthology.orchestra.run.smithi092.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 20 +425us[ +425us] +/- 79ms 2023-11-25T04:28:23.889 INFO:teuthology.orchestra.run.smithi092.stdout:^* hv02.front.sepia.ceph.com 3 6 377 21 -768us[ -765us] +/- 69ms 2023-11-25T04:28:23.889 INFO:teuthology.orchestra.run.smithi092.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 18 +448us[ +448us] +/- 65ms 2023-11-25T04:28:23.889 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-11-25T04:28:23.891 INFO:teuthology.orchestra.run.smithi195.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-25T04:28:23.891 INFO:teuthology.orchestra.run.smithi195.stdout:=============================================================================== 2023-11-25T04:28:23.891 INFO:teuthology.orchestra.run.smithi195.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 20 +442us[ +442us] +/- 79ms 2023-11-25T04:28:23.891 INFO:teuthology.orchestra.run.smithi195.stdout:^* hv02.front.sepia.ceph.com 3 6 377 20 -760us[ -737us] +/- 69ms 2023-11-25T04:28:23.891 INFO:teuthology.orchestra.run.smithi195.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 21 +445us[ +467us] +/- 65ms 2023-11-25T04:28:23.892 INFO:teuthology.orchestra.run.smithi195.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-11-25T04:28:23.892 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-11-25T04:28:23.902 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-11-25T04:28:23.903 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-11-25T04:28:23.942 DEBUG:teuthology.orchestra.run.smithi067:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-25T04:28:24.056 DEBUG:teuthology.orchestra.run.smithi076:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-25T04:28:24.216 DEBUG:teuthology.orchestra.run.smithi092:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-25T04:28:24.320 DEBUG:teuthology.orchestra.run.smithi195:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-25T04:28:24.422 DEBUG:teuthology.orchestra.run.smithi067:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-25T04:28:24.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-25T04:28:24.462 DEBUG:teuthology.orchestra.run.smithi076:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-25T04:28:24.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-25T04:28:24.497 DEBUG:teuthology.orchestra.run.smithi092:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-25T04:28:24.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-25T04:28:24.532 DEBUG:teuthology.orchestra.run.smithi195:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-25T04:28:24.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-25T04:28:24.566 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-11-25T04:28:24.579 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-11-25T04:28:24.614 INFO:teuthology.task.internal:Duration was 831.635767 seconds 2023-11-25T04:28:24.614 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-11-25T04:28:24.651 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-11-25T04:28:24.651 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-25T04:28:24.654 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-25T04:28:24.657 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-25T04:28:24.659 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-25T04:28:24.703 INFO:teuthology.orchestra.run.smithi195.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-25T04:28:24.704 INFO:teuthology.orchestra.run.smithi092.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-25T04:28:24.709 INFO:teuthology.orchestra.run.smithi067.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-25T04:28:24.723 INFO:teuthology.orchestra.run.smithi076.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-25T04:28:25.116 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-11-25T04:28:25.116 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi067.front.sepia.ceph.com 2023-11-25T04:28:25.116 DEBUG:teuthology.orchestra.run.smithi067:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-25T04:28:25.139 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi076.front.sepia.ceph.com 2023-11-25T04:28:25.139 DEBUG:teuthology.orchestra.run.smithi076:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-25T04:28:25.173 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi092.front.sepia.ceph.com 2023-11-25T04:28:25.173 DEBUG:teuthology.orchestra.run.smithi092:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-25T04:28:25.202 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi195.front.sepia.ceph.com 2023-11-25T04:28:25.202 DEBUG:teuthology.orchestra.run.smithi195:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-25T04:28:25.229 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-11-25T04:28:25.230 DEBUG:teuthology.orchestra.run.smithi067:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-25T04:28:25.233 DEBUG:teuthology.orchestra.run.smithi076:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-25T04:28:25.236 DEBUG:teuthology.orchestra.run.smithi092:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-25T04:28:25.245 DEBUG:teuthology.orchestra.run.smithi195:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-25T04:28:25.298 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-11-25T04:28:25.299 DEBUG:teuthology.orchestra.run.smithi067:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-25T04:28:25.307 DEBUG:teuthology.orchestra.run.smithi076:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-25T04:28:25.310 DEBUG:teuthology.orchestra.run.smithi092:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-25T04:28:25.314 DEBUG:teuthology.orchestra.run.smithi195:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-25T04:28:25.439 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-11-25T04:28:25.452 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-11-25T04:28:25.452 DEBUG:teuthology.orchestra.run.smithi067:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-25T04:28:25.484 DEBUG:teuthology.orchestra.run.smithi076:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-25T04:28:25.513 DEBUG:teuthology.orchestra.run.smithi092:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-25T04:28:25.540 DEBUG:teuthology.orchestra.run.smithi195:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-25T04:28:25.568 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-11-25T04:28:25.579 DEBUG:teuthology.orchestra.run.smithi067:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-25T04:28:25.581 DEBUG:teuthology.orchestra.run.smithi076:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-25T04:28:25.584 DEBUG:teuthology.orchestra.run.smithi092:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-25T04:28:25.586 DEBUG:teuthology.orchestra.run.smithi195:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-25T04:28:25.608 INFO:teuthology.orchestra.run.smithi067.stdout:kernel.core_pattern = core 2023-11-25T04:28:25.609 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern = core 2023-11-25T04:28:25.631 INFO:teuthology.orchestra.run.smithi076.stdout:kernel.core_pattern = core 2023-11-25T04:28:25.632 INFO:teuthology.orchestra.run.smithi195.stdout:kernel.core_pattern = core 2023-11-25T04:28:25.668 DEBUG:teuthology.orchestra.run.smithi067:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-25T04:28:25.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-25T04:28:25.683 DEBUG:teuthology.orchestra.run.smithi076:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-25T04:28:25.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-25T04:28:25.698 DEBUG:teuthology.orchestra.run.smithi092:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-25T04:28:25.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-25T04:28:25.712 DEBUG:teuthology.orchestra.run.smithi195:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-25T04:28:25.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-25T04:28:25.726 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-11-25T04:28:25.737 INFO:teuthology.task.internal:Transferring archived files... 2023-11-25T04:28:25.737 DEBUG:teuthology.misc:Transferring archived files from smithi067:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-11-24_21:20:59-powercycle-main-distro-default-smithi/7466144/remote/smithi067 2023-11-25T04:28:25.737 DEBUG:teuthology.orchestra.run.smithi067:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-25T04:28:25.783 DEBUG:teuthology.misc:Transferring archived files from smithi076:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-11-24_21:20:59-powercycle-main-distro-default-smithi/7466144/remote/smithi076 2023-11-25T04:28:25.783 DEBUG:teuthology.orchestra.run.smithi076:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-25T04:28:25.819 DEBUG:teuthology.misc:Transferring archived files from smithi092:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-11-24_21:20:59-powercycle-main-distro-default-smithi/7466144/remote/smithi092 2023-11-25T04:28:25.819 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-25T04:28:25.855 DEBUG:teuthology.misc:Transferring archived files from smithi195:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-11-24_21:20:59-powercycle-main-distro-default-smithi/7466144/remote/smithi195 2023-11-25T04:28:25.856 DEBUG:teuthology.orchestra.run.smithi195:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-25T04:28:25.893 INFO:teuthology.task.internal:Removing archive directory... 2023-11-25T04:28:25.894 DEBUG:teuthology.orchestra.run.smithi067:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-25T04:28:25.896 DEBUG:teuthology.orchestra.run.smithi076:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-25T04:28:25.899 DEBUG:teuthology.orchestra.run.smithi092:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-25T04:28:25.901 DEBUG:teuthology.orchestra.run.smithi195:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-25T04:28:25.943 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-11-25T04:28:25.953 INFO:teuthology.task.internal:Not uploading archives. 2023-11-25T04:28:25.953 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-11-25T04:28:26.016 INFO:teuthology.task.internal:Tidying up after the test... 2023-11-25T04:28:26.017 DEBUG:teuthology.orchestra.run.smithi067:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-25T04:28:26.019 DEBUG:teuthology.orchestra.run.smithi076:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-25T04:28:26.022 DEBUG:teuthology.orchestra.run.smithi092:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-25T04:28:26.024 DEBUG:teuthology.orchestra.run.smithi195:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-25T04:28:26.032 INFO:teuthology.orchestra.run.smithi067.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 25 04:28 /home/ubuntu/cephtest 2023-11-25T04:28:26.034 INFO:teuthology.orchestra.run.smithi076.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 25 04:28 /home/ubuntu/cephtest 2023-11-25T04:28:26.036 INFO:teuthology.orchestra.run.smithi092.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 25 04:28 /home/ubuntu/cephtest 2023-11-25T04:28:26.038 INFO:teuthology.orchestra.run.smithi195.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 25 04:28 /home/ubuntu/cephtest 2023-11-25T04:28:26.039 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-11-25T04:28:26.062 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-11-25T04:28:26.188 INFO:teuthology.nuke:Checking targets against current locks 2023-11-25T04:28:26.221 DEBUG:teuthology.nuke:shortname: smithi067 2023-11-25T04:28:26.222 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-25T04:28:26.251 DEBUG:teuthology.nuke:shortname: smithi076 2023-11-25T04:28:26.252 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-25T04:28:26.258 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi067.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-24_21:20:59-powercycle-main-distro-default-smithi/7466144', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-25 04:05:21.922153', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNyhVVF/XtgfyRRDww3bmZ7EPginiqC6uqiypg/ZyPafcAIeLt7VaE94CXqC6yGNd5D2c5UnO4VwxGQYoizloRo='} 2023-11-25T04:28:26.279 DEBUG:teuthology.nuke:shortname: smithi092 2023-11-25T04:28:26.279 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-25T04:28:26.285 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi076.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-24_21:20:59-powercycle-main-distro-default-smithi/7466144', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-25 04:05:21.918425', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCC/Z4U69AnhkPFrQh+/iMSSZP8/tiy64EQsC7bzJt6p7iGC8jRA87LBnDjvkLlrBj8b8RNN7AYBEuytF8IFbws='} 2023-11-25T04:28:26.306 DEBUG:teuthology.nuke:shortname: smithi195 2023-11-25T04:28:26.307 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-25T04:28:26.323 INFO:teuthology.orchestra.console.smithi067:Power off 2023-11-25T04:28:26.324 DEBUG:teuthology.orchestra.console.smithi067:pexpect command: ipmitool -H smithi067.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-25T04:28:26.337 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi092.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-24_21:20:59-powercycle-main-distro-default-smithi/7466144', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-25 04:05:21.920373', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIfYQcptwzwhxujcHjAVShwsFCWARoaYbV62s9iIILQuBy0lgscMzeefkNhuHZOia3Xr4VIL6eduVJA3y+mnv9A='} 2023-11-25T04:28:26.355 DEBUG:teuthology.orchestra.console.smithi067:power off output: Chassis Power Control: Down/Off 2023-11-25T04:28:26.356 DEBUG:teuthology.orchestra.console.smithi067:pexpect command: ipmitool -H smithi067.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-25T04:28:26.379 INFO:teuthology.orchestra.console.smithi076:Power off 2023-11-25T04:28:26.380 DEBUG:teuthology.orchestra.console.smithi076:pexpect command: ipmitool -H smithi076.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-25T04:28:26.395 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi195.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-24_21:20:59-powercycle-main-distro-default-smithi/7466144', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-25 04:05:21.923666', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLDl41IEbYPQ77UvNCnICkwMfSo5Jnv8VURPY8nB7lwu6U41Fbg5IMmek6b8dvVfg1GWiwOdYIFT+w6cMVtbB3Q='} 2023-11-25T04:28:26.407 DEBUG:teuthology.orchestra.console.smithi067:check power output: Chassis Power is on 2023-11-25T04:28:26.422 DEBUG:teuthology.orchestra.console.smithi076:power off output: Chassis Power Control: Down/Off 2023-11-25T04:28:26.422 DEBUG:teuthology.orchestra.console.smithi076:pexpect command: ipmitool -H smithi076.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-25T04:28:26.443 INFO:teuthology.orchestra.console.smithi092:Power off 2023-11-25T04:28:26.444 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-25T04:28:26.479 DEBUG:teuthology.orchestra.console.smithi076:check power output: Chassis Power is on 2023-11-25T04:28:26.480 DEBUG:teuthology.orchestra.console.smithi092:power off output: Chassis Power Control: Down/Off 2023-11-25T04:28:26.481 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-25T04:28:26.512 INFO:teuthology.orchestra.console.smithi195:Power off 2023-11-25T04:28:26.512 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-25T04:28:26.522 DEBUG:teuthology.orchestra.console.smithi092:check power output: Chassis Power is on 2023-11-25T04:28:26.545 DEBUG:teuthology.orchestra.console.smithi195:power off output: Chassis Power Control: Down/Off 2023-11-25T04:28:26.546 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-25T04:28:26.572 DEBUG:teuthology.orchestra.console.smithi195:check power output: Chassis Power is on 2023-11-25T04:28:30.409 DEBUG:teuthology.orchestra.console.smithi067:pexpect command: ipmitool -H smithi067.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-25T04:28:30.480 DEBUG:teuthology.orchestra.console.smithi076:pexpect command: ipmitool -H smithi076.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-25T04:28:30.520 DEBUG:teuthology.orchestra.console.smithi067:check power output: Chassis Power is on 2023-11-25T04:28:30.523 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-25T04:28:30.573 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-25T04:28:30.591 DEBUG:teuthology.orchestra.console.smithi076:check power output: Chassis Power is on 2023-11-25T04:28:30.633 DEBUG:teuthology.orchestra.console.smithi092:check power output: Chassis Power is on 2023-11-25T04:28:30.683 DEBUG:teuthology.orchestra.console.smithi195:check power output: Chassis Power is on 2023-11-25T04:28:34.522 DEBUG:teuthology.orchestra.console.smithi067:pexpect command: ipmitool -H smithi067.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-25T04:28:34.592 DEBUG:teuthology.orchestra.console.smithi076:pexpect command: ipmitool -H smithi076.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-25T04:28:34.635 DEBUG:teuthology.orchestra.console.smithi067:check power output: Chassis Power is off 2023-11-25T04:28:34.635 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-25T04:28:34.685 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-25T04:28:34.703 DEBUG:teuthology.orchestra.console.smithi076:check power output: Chassis Power is off 2023-11-25T04:28:34.736 INFO:teuthology.orchestra.console.smithi067:Power off completed 2023-11-25T04:28:34.747 DEBUG:teuthology.orchestra.console.smithi092:check power output: Chassis Power is on 2023-11-25T04:28:34.795 DEBUG:teuthology.orchestra.console.smithi195:check power output: Chassis Power is off 2023-11-25T04:28:34.805 INFO:teuthology.orchestra.console.smithi076:Power off completed 2023-11-25T04:28:34.896 INFO:teuthology.orchestra.console.smithi195:Power off completed 2023-11-25T04:28:38.750 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-25T04:28:38.861 DEBUG:teuthology.orchestra.console.smithi092:check power output: Chassis Power is off 2023-11-25T04:28:38.962 INFO:teuthology.orchestra.console.smithi092:Power off completed 2023-11-25T04:28:39.067 INFO:teuthology.run:Summary data: description: powercycle/osd/{clusters/3osd-1per-target ignorelist_health objectstore/bluestore-low-osd-mem-target powercycle/default supported-distros/centos_latest tasks/cfuse_workunit_suites_fsx thrashosds-health} duration: 831.6357674598694 failure_reason: 'Command failed (workunit test suites/fsx.sh) on smithi067 with status 2: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ec928368edcc5fc71884a56fd9fd821b148f4e30 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh''' flavor: default owner: scheduled_yuriw@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=8e4f33aab3754c9bb578e9a1b80e8d3e status: fail success: false 2023-11-25T04:28:39.068 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-25T04:28:39.144 INFO:teuthology.run:FAIL